Nimbus update: v1.2 CPU 🔥

If  v1.1 was the big I/O update, v1.2 is all about the CPU - together, these updates help secure Nimbus against future network growth, and provide us with a higher security margin and substantial profitability improvements.

To highlight just one data point, CPU usage has been cut by up to 40% over v1.1.0 ( thanks to batched attestation processing and various other optimisations).

This means we are finally comfortable recommending Nimbus for mainnet validating on a Raspberry Pi 4 (even taking into account the load of the beacon chain after the merge).

Highlights include:

• Beta support for the official Beacon Node REST API:
  https://ethereum.github.io/eth2.0-APIs/. Enable it by launching the client with the --rest:on command-line flag
• Batched attestation verification and other reforms -> massive reduction in overall CPU usage.
• Improved attestation aggregation logic -> denser aggregations which in turn improve the overall health of the network and improve block production.
• More efficient LibP2P connection handling code -> reduction in overall memory usage.
• A fix to a critical stability issue in attestation processing.
• Web3 provider UX bug fix: Nimbus no longer prompts you for a web3 provider URL if you’ve already specified a --web3-url command-line option.

See here for the full release notes.

Profitability improvements

Our new packing algorithm is the main driver of the profitability improvements mentioned above.

If your knowledge on validator committees and signature aggregation needs brushing up, Jacek has written a great thread adding context here:

To sum up, in v1.2 combining votes is delayed until the latest moment possible, allowing single-vote attestations to top up aggregates when the block is produced.

The end result is more densely aggregated attestations with less overlap. This makes it easier for us to discard aggregates that are pure subsets of other known aggregates in the pool (sets that contain each other cannot be further combined).

The second improvement is a simple scoring mechanism that prioritises aggregates that provide the most “new” votes compared to what we already know – either from previous blocks or from aggregates we’ve already decided to add to the block - in general, this is a computationally hard problem, so we make an approximation with a greedy algorithm.

Note: These improvements are best seen on large testnets like Prater where node quality is low and validator counts are double that of mainnet – this means twice as many committees per slot and more forks, in general!

Performance improvements

Batching is how we describe grouping verification of N messages by N validators.

The general idea involves grouping the incoming gossiped attestations into batches, and then verifying that all the signatures in the group are correct in one go. This substantially reduces the amount of computations required (and is the main source of performance improvements in v1.2).

To elaborate a little here, verifying BLS signatures takes about 1.5ms per signature: 50% of this time is spent on a Miller Loop, and the other 50% on final exponentiation.

But when we batch signatures together, we only need to do the final exponentiation once per batch. This means that, given enough signatures to verify, we can improve throughput by almost 2x.

Since signature verification is the main CPU resource hog, the end result is a drastic cut to CPU usage.

Database sync hotfix

Shortly after releasing v1.2.0, we realised we had introduced a database migration issue. Luckily, this issue – which essentially reset the state database causing Nimbus to start re-syncing the chain from scratch – only affected new users (those who had used v1.1.0 to sync with the network from genesis).

We quickly pushed a hotfix release (v1.2.1).

As mentioned in the discord announcement and release notes, if you’ve been affected, you have a couple of options available to you:

1. If you backed-up your database prior to upgrading to v1.2.0, you can restore the database from backup and execute the migration successfully after upgrading to this release.
2. If you don’t have a database backup, you can upgrade to v1.2.1 at your convenience; rest assured it won’t delete your sync progress.

Please accept our sincerest apologies for any inconvenience we may have caused. We are reviewing our release testing policies to ensure that we cover a greater number of possible upgrade paths going forward 🙏

Parting words

If you still haven’t had a chance to upgrade to a 1.2 release, please note that upgrading is highly recommended for a number of compelling reasons. Apart from significant performance and profitability improvements, 1.2 contains a critical stability fix.

N.B. If you have not updated to v1.2.0 yet, please update directly to v1.2.1.

SHA512 checksums (v1.2.1)

# Linux AMD64
6bbdad0d424aadfef9119277c00702b58e7c7f020a329a7baa7237144cc3fc49e252cdfcc146b5b8f384207be6253655b1daacb342fc255defec1d56c671a3df  nimbus_beacon_node
# Linux ARM64
ab759a3cf6eadaee97565d139e3546ef1a4716bede03d50aebe650b82624fc65020dc385d77e1afa78779537dd0173201121855fd64c61d74c8e5e29536b90ef  nimbus_beacon_node
# Linux ARM
aaa31248357a670c9ae3e4757c6f6b7f8b03cd5716d01ab519e17ab891d7b3fd08b1693f9639cec87634e8093106016730857256094515d0d648c3da50038279  nimbus_beacon_node
# Windows AMD64
7bf0bd09a297102f9724dad6912504e088be70908aeb1cb7f96ba7000dd34e780e59c40d188f4de890d09780e4ca2e5716d5684350e409e081c2020e1652bbcc  nimbus_beacon_node

SHA512 checksums (v1.2.0)

# Linux AMD64
7e999b115667cf5da94c6b251bae82118af0c5b13ebd1ca800431bbb9f75188fb945bd7ae0cbdc7457cd74f695431075682a8ba52f442f48bca33d74a16e6f6c  nimbus_beacon_node
# Linux ARM64
1a79c60a9872c09501185add421dc2924035ade05822da7e7e94f31fb687ddba7ce6354bf8a00073384687be90b9cde9ff8b298d4d20d07608deca49ecd39ccc  nimbus_beacon_node
# Linux ARM
9eb8700fd105a3bc3cd269b51290084e836542b86a883a7b0fbe011d80a8288a4181119027810b469c470933297af0f832f877386fa2cef27e37496c7b51a098  nimbus_beacon_node
# Windows AMD64
4266cae744a63c8a51b7b5eda6c5ecb0adb7af7e72bcad96588d3e6c0532f0bfbeea4a7e9d2fc4e826021b5cb5b5aa744c6dc91881bd9bbc70551dceb76b608b nimbus_beacon_node