Mining Kandinsky-2 on Arbius

This guide will walk you through the process of mining Kandinsky-2 on the Arbius network. Kadinsky-2 is the model Arbius is launching with as the model to bootstrap miners to the network and reward them. For this model, it is configured to run on Nvidia A100 GPU. In the future, we plan to support other chipsets, for now you will need to have access to a A100 GPU to mine Arbius.

Get A100 GPU

We recommend Lambda Labs cloud hosting for this model, this been tested to function properly. There are many service providers of A100 chips via GPU clouds and other places, in the likely case that you don't have a A100 chip yourself.

The recommended storage space required for the computer running the A100 chip is 60GB. The miner and the A100 GPU instance does not need to be the same computer, the actual miner software just needs the URL for the instance running the COG.

What is a Cog file?

Cog is an open-source tool that packages machine learning models in a standard, production-ready container. In Arbius we deploy cogs via docker, and the miner connects to the docker instance for model input and output.

Therefore to mine Arbius, we have to deploy a docker image of a registered model.

Docker

You will need to have docker installed on the computer with the A100 chip. Some GPU cloud services offer dockerized containers, so you will not need to install docker yourself, but will need to run the miner on your own computer or separate VPS.

If running your own computer with the A100 chip, after installing docker:

docker run -d -p 5000:5000 --gpus=all r8.im/kasumi-1/kandinsky-2@sha256:373fa540ae197fc89f0679ed835bc4524152956d4f3027580244e10b09d6d3a5

The URL above is the Kadinsky-2 model required to mine Arbius. There is only one model for now.

./vast create instance <instance-id> --image r8.im/kasumi-1/kandinsky-2@sha256:373fa540ae197fc89f0679ed835bc4524152956d4f3027580244e10b09d6d3a5 --env '-p 5000:5000 -p 22:22' --onstart-cmd 'python -m cog.server.http' --jupyter-dir /src --ssh --direct --disk 100

Setup Mining Software and Dependencies

You may run the mining software on a local machine or on a cloud instance. Especially if using a cloud instance, ensure that you properly secure the machine.

Install NVM

NVM is a version manager for Node.js, which is required to run the mining software.

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash

Install IPFS (InterPlanetary File System)

IPFS is a distributed file system that will be used to share generated content.

Follow this guide to ensure you have IPFS installed:

https://docs.ipfs.tech/install/command-line/

Make sure you are running IPFS as a daemon:

ipfs init
ipfs daemon

You will need to keep IPFS on whilst the miner is running.

Install Arbius Reference Miner

The Arbius reference miner is a simple mining software that will be used to mine Kandinsky-2 on the Arbius network. It is designed to be easy to use and debug. We recommend reading the code to understand how it works, and to contribute improvements upstream.

git clone https://github.com/semperai/arbius.git
cd arbius/miner
nvm use
# if you don't have yarn installed already
npm install -g yarn
yarn # install dependencies

Setup RPC Endpoint

You will need to provide an RPC endpoint for the Arbius network. You can run your own node, or use a public node. We recommend running your own node for security and performance reasons, however it is not necessary. For testing setup, you can use a public node.

https://docs.arbitrum.io/node-running/node-providers

Note: you will need to run your node on Arbitrum Nova network.

Generate Wallet for Miner

Arbius has a built in wallet generator, you can also use MetaMask or other tools to generate a new private key. We strongly recommend using a new private key for each miner.

cd arbius/contract
cp .env.example .env
nvm use
yarn
npx hardhat gen-wallet

Configure Mining Software

Within the arbius/miner directory, copy the example configuration file to a new file called MiningConfig.json.

cp  MiningConfig.example.json MiningConfig.json

Read-only Configuration

Edit MiningConfig.json to include the readonly flag. This will prevent the miner from submitting solutions to the network. This is useful for testing the miner on a live network without performing any sort of transactions. You may want to use this while testing various prob configurations.

{
  "readonly": true
}

Blockchain Configuration

Edit MiningConfig.json to include your miners private key and the RPC endpoint.

{
  "blockchain": {
    "private_key": "0x...",
    "rpc_url": "https://..."
  }
}

Automine Configuration

Arbius reference miner has an automine feature that will automatically generate new tasks. These tasks are available for anyone to solve, and you will be rewarded for solving them. You can enable automine by setting the automine flag to true in MiningConfig.json. If automine is disabled, your miner will only mine tasks that it notices on the network.

{
  "automine": {
    "enabled": true
  }
}

You will want to set the correct model and a unique prompt for tasks.

ML Configuration

Here is where you'll configure to connect to a running Cog container. Grab the prediction url from the api link of the Cog container you launched earlier. It will look something like: https://xxxxxxxxxxxxx.notebooksd.jarvislabs.net/predictions. Make sure you do not share this url with anyone, as it can be used to access your Cog container.

If your cog container is running on a separate server, you will need to make sure that the server allows incoming connections on the provided port.

{
  "ml": {
    "strategy": "cog",
    "cog": {
      "0x98617a8cd4a11db63100ad44bea4e5e296aecfd78b2ef06aee3e364c7307f212": {
        "url": "https://..."
      }
    }
  }
}

IPFS Configuration

Here is where you'll configure to connect to a running IPFS node. If you are using pinata, you will need to use the pinata API url (see example cofig). You will want to use http_client strategy, to connect to your own IPFS, or pinata to connect to.. pinata.

{
  "ipfs": {
    "strategy": "http_client",
    "http_client": {
      "url": "http://localhost:5001"
    }
  }
}

Probability Configuration

The prob configuration is used to set the probability of the miner to perform certain actions. Setting this to 0 will disable the action, and setting it to 1 will ensure the action is performed every time. To participate only in contestations, you can set every value to 0 except for contestation_submitted. To only validate others solutions, you can set every value to 0 except for solution_submitted (this will still cause you to contest invalid solutions). To only attempt to mine solutions and not validate others tasks, you can turn task to 1 or some smaller number if that is too much for your miner.

{
  "prob": {
    "task": 0.05,
    "contestation_vote_finish": 0.2,
    "contestation_submitted": 1.0,
    "solution_submitted": 0.01,
    "task_retracted": 1.0
  }
}

Load Up Miner Wallet

You will need to load your miner wallet with some funds to pay for gas. You can do this by sending some funds to the address associated with your private key.

Arbitrum Nova

You can bridge funds from Ethereum to Arbitrum Nova using the Arbitrum bridge. You will need to use the Arbitrum bridge to send funds to your miner wallet.

https://bridge.arbitrum.io/?destinationChain=arbitrum-nova&sourceChain=ethereum

Arbius Token

Arbius(AIUS) token needs to be staked to mine. For the first day, no AIUS token is required to begin mining. After the first day, you will need to stake AIUS token to mine. You can obtain AIUS token from Uniswap. The miner will automatically stake AIUS token when it is required.

Test Mining Software

Once your miner has sufficient funds and has been configured you can start it by running:

yarn start:dev MiningConfig.json

Your miner will perform a bootup sanity check by checking version as well as a test generation of the Kandinsky-2 model. If everything is good, it will start mining.

Troubleshooting

If you encounter issues, please investigate the logs to determine the cause of the issue. If you are unable to resolve the issue, please reach out to the Arbius community for support.

If the miner seems stuck or is not mining, you can try restarting it. If this does not resolve the issue, try copying the db.sqlite file to a new location (for backup) and restarting the miner. This will allow the miner to start fresh.

RPC Operation (advanced)

The miner provides a http api that allows the operator to control operation. This should never be exposed to the internet.

Queue Jobs

To queue a job use the api/jobs/queue route.

curl "localhost:8335/api/jobs/queue" \
-X POST \
-H 'Content-Type: application/json' \
--data '{"method":"automine", "priority": 5, "waituntil": 0, "concurrent": false, "data": {}}'

A higher priority will increase the priority the job is run. waituntil defines the unix timestamp at which to run the task. concurrent sets whether or not this job will run concurrent with other jobs (usually it should be set to true, otherwise errors may occur).

Methods

pinTaskInput

• taskid [string]
• input [object]

task

• taskid [string]
• txid [string]

validatorStake

• validatorMinimum [string]

automine

solve

• taskid [string]

claim

• taskid [string]

contestationVoteFinish

• taskid [string]

List jobs

curl "localhost:8335/api/jobs/list" \
-X POST \
-H 'Content-Type: application/json'

Delete job

curl "localhost:8335/api/jobs/delete" \
-X POST \
-H 'Content-Type: application/json'
--data '{"id":37}'