Managing The Validator

This page describes the regular tasks that you may need to do, as a node runner.

Handling Upgrades

Depending on the type of upgrade being undertaken by the wider network, one of the options below should be used to perform the upgrade. Automatic upgrades via the x/upgrade module and/or Cosmovisor are not yet supported.

The type of upgrade required by the network impacts which options you may select.

Option 1: Upgrade with chain-id change. Without coordination.

This type of upgrade is the most commonly used by Exocore. For example, the testnet was upgraded from exocoretestnet_233-5 to exocoretestnet_233-6 with this option.

  1. Stop the node.

  2. Remove the existing data

    rm -rf $HOMEDIR/config/genesis.json $HOMEDIR/data/*.db $HOMEDIR/data/snapshots $HOMEDIR/data/cs.wal

  3. Overwrite the validator's signature history, if any

    jq -n '{"height": "0", "round": 0, "step": 0}' > $HOMEDIR/data/priv_validator_state.json

  4. Download the newer binary, overwriting the previous one /usr/bin/exocored. Verify that the version of this binary is correct by running exocored version

  5. Store the new CHAIN_ID in the environment with CHAIN_ID=<new-chain-id>

  6. Set the new CHAIN_ID in the configuration with exocored --home $HOMEDIR config chain-id $CHAIN_ID and make other configuration changes, if any

  7. Download the genesis file

    wget -O $HOMEDIR/config/genesis.json https://raw.githubusercontent.com/ExocoreNetwork/testnets/main/genesis/$CHAIN_ID.json

  8. Start the node

Option 2: Upgrade with chain-id change. With coordination.

  1. Initialize a new node with the increased chain-id at a new location (for example exocoretestnet_233-3 becomes exocoretestnet_233-4) and configure it. Make sure to re-use the same validator private key ($HOMEDIR/config/priv_validator_key.json) or provide the mnemonic via --recover during the initialization.

  2. Modify the systemd service to use the provided --halt-height or --halt-time so that all v3 nodes stop at the same time.

  3. Once the network is halted, obtain the new genesis file and add it to the new $HOMEDIR Additionally, upgrade the binary by overwriting the existing one.

  4. Edit the systemd service to point to the new $HOMEDIR and remove the --halt parameters.

  5. Restart the service.

Option 3: No chain-id change. With coordination.

  1. Edit the systemd service with the specified --halt-height or --halt-time.

  2. Once the network is halted, remove the --halt parameters from the systemd file and upgrade the binary.

  3. Restart the service.

Upgrading from Testnet v5 to v6

This upgrade was performed using Option 1. The genesis file retained the operator registrations as well as the associations between the stakers and the operators. However, deposits, delegations and opt-ins to the Exocore-chain-as-AVS were dropped to avoid unnecessary slashing. To that end, validators must redo (almost) all of the steps listed under Becoming a Validator, skipping only the register-operator and associateOperatorWithEVMStaker transactions. In other words, tokens must be acquired, deposited, and delegated again, and, the opt-into-avs transaction must be sent again.

The base denomination changed from aexo to hua as part of this upgrade. This change affects the configuration of minimum-gas-prices as well as the CLI commands specifying the gas price with the option --gas-prices.

Additionally, the oracle price feeder needs a new configuration file $HOMEDIR/config/oracle_env_beaconchain.yaml, which is documented on its page.

Monitoring Uptime

Enable Prometheus metrics in the configuration by editing $HOMEDIR/config/app.toml

[telemetry]
enabled = true
prometheus-retention-time = 0

Set up a Prometheus instance to scrape the metrics exposed by your node.

scrape_configs:
  - job_name: 'exocored'
    static_configs:
      - targets: ['localhost:26660']

Visualize these with Grafana by using its dashboards. Metrics include:

  • cometbft_consensus_latest_block_height: The latest block height

  • cometbft_consensus_block_syncing: Whether the node is synced or still catching up.

  • cometbft_consensus_block_interval_seconds: Time between blocks, useful for monitoring block production delays.

In addition, system level information can be loaded with prometheus-node-exporter into Grafana to complement the metrics exposed by exocored.

Lastly, third-party monitoring services such as ping.pub may be used when they add support for Exocore.

Useful Commands

Stop the node

systemctl stop exocore

Start the node

systemctl start exocore

Restart the node

systemctl restart exocore

Find my exo1... (account) address

exocored --home $HOMEDIR keys show -a $ACCOUNT_KEY_NAME

Get current height

exocored query block | jq .block.header.height

Check sync status

exocored status | jq

Get current validator set

exocored query tendermint-validator-set

Get own validator address

exocored --home $HOMEDIR tendermint show-address

Convert address formats

exocored debug addr <VALUE>

Get bytes32 val pub key

exocored --home $HOMEDIR --output json keys consensus-pubkey-to-bytes | jq -r .bytes32

Get bytes32 val pub key (another method)

exocored debug pubkey $(exocored --home $HOMEDIR tendermint show-validator)

Get JSON val pub key

exocored --home $HOMEDIR tendermint show-validator

Check node logs

journalctl -u exocore -f

Get the number of peers

curl -s http://localhost:26657/net_info

Check transaction status

exocored query tx <hash>

Find my consensus address

exocored --home $HOMEDIR tendermint show-address

Find my P2P ID

exocored --home $HOMEDIR tendermint show-node-id

(assuming the node port hasn't changed; if it has, it may be supplied with --node)

PreviousFaucetsNextSecurity Best Practices

Last updated