Running a Worker

NuCypher staking operations are divided into two roles “Staker” and “Worker” - This Guide is for Workers.

Overview

Role in the Network

The Worker is the bonded delegate of a Staker and an active network node. Each staking account or “Staker” is bonded to exactly one Worker. Workers must remain online to provide uninterrupted re-encryption services to network users on-demand and perform periodic automated transactions to signal continued commitment to availability.

Core Components

Worker nodes have three core components:

  • Ethereum software wallet (keystore)

  • Local or hosted ethereum provider

  • Worker node; Local or cloud server

Minimum System Requirements

  • Debian/Ubuntu (Recommended)

  • 20GB storage

  • 4GB RAM

  • x86 architecture

  • Static IP address

  • Exposed TCP port 9151

Workers can be run on cloud infrastructure – for example, Digital Ocean 4GB Basic Droplet satisfies requirements listed above.

1. Establish Ethereum Provider

Worker Ursula transactions can be broadcasted using either a local or remote ethereum node.

For general background information about choosing a node technology and operation, see https://web3py.readthedocs.io/en/stable/node.html.

Note

Additional requirements are needed to run a local Ethereum node on the same system.

2. Establish Worker Ethereum Account

By default, all transaction and message signing requests are forwarded to the configured ethereum provider. When using a remote ethereum provider (e.g. Infura, Alchemy, other hosted node), a local transaction signer must be configured in addition to the broadcasting node. For workers this can be a software wallet, or clef. For more detailed information see Transaction Signing.

Caution

Stay safe handling ETH and NU:

  • Workers do not need NU for any reason; do not keep NU on the worker’s account.

  • Do not store large amounts of ETH on the worker; keep only enough to pay for gas fees.

  • Store the ethereum account password in a password manager when using a keystore.

Because worker nodes perform periodic automated transactions to signal continued commitment to providing service, The worker’s ethereum account must remain unlocked while the node is running. While there are several types of accounts workers can use, a software based wallet is the easiest.

Note

To create a new ethereum software account using the geth CLI

geth account new

3. Run Worker

Important

Before proceeding it is important to know that the worker must spend ETH to unlock staked NU. Periodic automated commitments are required to signal continued availability. Currently, Worker nodes must perform one commitment transaction every 24 hours each costing ~200k gas.

Use the –max-gas-price option to set the maximum commitment gas price you are willing to spend. Workers will automatically retry and replace any previous commitment attempts. Too low of a gas price may result in missed commitments.

Run Worker with systemd (Alternate)

Instead of using docker, the nucypher worker can be run as a systemd service.

Note

Running a worker with systemd required a local installation of nucypher. See Installation Reference.

  1. Install nucypher into a virtual environment.

$(nucypher) pip install -U nucypher
  1. Configure the worker using the nucypher CLI.

$(nucypher) nucypher ursula init \
--provider <PROVIDER URI>        \
--network <NETWORK NAME>         \
--signer <SIGNER URI>            \
--max-gas-price <GWEI>

Replace the following values with your own:

  • <PROVIDER URI> - The URI of a local or hosted ethereum node

  • <NETWORK NAME> - The name of a nucypher network (mainnet, ibex, or lynx)

  • <SIGNER URI> - The URI to an ethereum keystore or signer: keystore:///root/.ethereum/keystore

  • <GWEI> - The maximum price of gas to spend on commitment transactions

  1. Use this template to create a file named ursula.service and place it in /etc/systemd/system/.

[Unit]
Description="Ursula, a NuCypher Worker."

[Service]
User=<YOUR USER>
Type=simple
Environment="NUCYPHER_WORKER_ETH_PASSWORD=<YOUR WORKER ADDRESS PASSWORD>"
Environment="NUCYPHER_KEYRING_PASSWORD=<YOUR PASSWORD>"
ExecStart=<VIRTUALENV PATH>/bin/nucypher ursula run

[Install]
WantedBy=multi-user.target

Replace the following values with your own:

  • <YOUR USER> - The host system’s username to run the process with (best practice is to use a dedicated user)

  • <YOUR WORKER ADDRESS PASSWORD> - Worker’s ETH account password

  • <YOUR PASSWORD> - Ursula’s keyring password

  • <VIRTUALENV PATH> - The absolute path to the python virtual environment containing the nucypher executable

  1. Enable Ursula System Service

$ sudo systemctl enable ursula
  1. Run Ursula System Service

To start Ursula services using systemd

$ sudo systemctl start ursula

Check Ursula service status

# Application Logs
$ tail -f ~/.local/share/nucypher/nucypher.log

# Systemd status
$ systemctl status ursula

# Systemd Logs
$ journalctl -f -t ursula

To restart your node service

$ sudo systemctl restart ursula

Run Worker Manually

  1. Configure the Worker

If you’d like to use another own method of running the worker process in the background, or are using one of the testnets, here is how to run Ursula using the CLI directly.

$(nucypher) nucypher ursula init \
--provider <PROVIDER URI>        \
--network <NETWORK NAME>         \
--signer <SIGNER URI>            \
--max-gas-price <GWEI>

Replace the following values with your own:

  • <PROVIDER URI> - The URI of a local or hosted ethereum node

  • <NETWORK NAME> - The name of a nucypher network (mainnet, ibex, or lynx)

  • <SIGNER URI> - The URI to an ethereum keystore or signer: keystore:///root/.ethereum/keystore

  • <GWEI> - The maximum price of gas to spend on commitment transactions

Note

All worker configuration values can be changed using the config command (Note that the worker must be restarted for new changes to take effect):

nucypher ursula config --<OPTION> <NEW VALUE>

# Update the max gas price setting
nucypher ursula config --max-gas-price <GWEI>

# Update the max gas price setting
nucypher ursula config --provider <PROVIDER URI>

# View the current configuration
nucypher ursula config
  1. Start the worker

# Run Worker
nucypher ursula run

Replace the following values with your own:

  • <PROVIDER URI> - The URI of a local or hosted ethereum node

  • <NETWORK NAME> - The name of a nucypher network (mainnet, ibex, or lynx)

  • <SIGNER URI> - The URI to an ethereum keystore or signer: keystore:///root/.ethereum/keystore

4. Qualify Worker

Workers must be fully qualified (funded and bonded) in order to fully start. Workers that are launched before qualification will pause until they are have a balance greater than 0 ETH, and are bonded to a staking account. Once both of these requirements are met, the worker will automatically resume startup.

Waiting for qualification:

...
Authenticating Ursula
Qualifying worker
ⓘ  Worker startup is paused. Waiting for bonding and funding ...
ⓘ  Worker startup is paused. Waiting for bonding and funding ...
ⓘ  Worker startup is paused. Waiting for bonding and funding ...

Resuming startup after funding and bonding:

...
ⓘ  Worker startup is paused. Waiting for bonding and funding ...
✓ Worker is bonded to 0x37f320567b6C4dF121302EaED8A9B7029Fe09Deb
✓ Worker is funded with 0.01 ETH
✓ External IP matches configuration
Starting services
✓ Database Pruning
✓ Work Tracking
✓ Rest Server https://1.2.3.4:9151
Working ~ Keep Ursula Online!

5. Monitor Worker

Ursula’s Logs

A reliable way to check the status of a worker node is to view the logs.

View logs for a docker-launched Ursula:

docker logs -f ursula

View logs for a CLI-launched or systemd Ursula:

# Application Logs
tail -f ~/.local/share/nucypher/nucypher.log

# Systemd Logs
journalctl -f -t ursula

Status Webpage

Once Ursula is running, you can view its public status page at https://<node_ip>:9151/status. It should eventually be listed on the Status Monitor Page (this can take a few minutes).

Prometheus Endpoint

Ursula can optionally provide a Prometheus metrics endpoint to be used for as a data source for real-time monitoring. For docker users, the Prometheus client library is installed by default.

For pip installations, The Prometheus client library is not included by default and must be explicitly installed:

(nucypher)$ pip install nucypher[ursula]

The metrics endpoint is disabled by default but can be enabled by providing the following parameters to the nucypher ursula run command:

  • --prometheus - a boolean flag to enable the prometheus endpoint

  • --metrics-port <PORT> - the HTTP port to run the prometheus endpoint on

The corresponding endpoint, http://<node_ip>:<METRICS PORT>/metrics, can be used as a Prometheus data source for monitoring including the creation of alert criteria.

By default, metrics will be collected every 90 seconds but this can be modified using the --metrics-interval option. Collection of metrics will increase the number of RPC requests made to your provider endpoint; increasing the frequency of metrics collection will further increase this number.

During the Technical Contributor Phase of our testnet, P2P Validator contributed a self-hosted node monitoring suite that uses a Grafana dashboard to visualize and monitor the metrics produced by the prometheus endpoint.

../_images/p2p_validator_dashboard.png