Nimbus status update - March 6th
Some updates and musings since last time.
Status bounty program
We’re happy to announce that we’re restarting the Status bounty program for the development of Nimbus.
We had to close it during crypto winter last year, but now that winter is over, and we’ve delivered on our core applications, we think it makes sense to restart it and get the community involved again.
In the past, the bounty program has been a great way for us to introduce more developers to our projects and libraries. As well as a useful tool for finding more permanent hires.
Andri Lim, for example, was one of our most active bounty hunters last time around (he sits as the top committer in the Nimbus repository). Today he’s a full-time member of our team and the sole driving force behind our Ethereum 1.0 development.
For those of you unfamiliar with the concept of open-source bounties, here is a short description of how this program works:
- You pick a task for a varied list of available bounties (all involving the Nim programming language)
- You complete the work
- Once the work is complete, you get a predefined reward equal to roughly 30 USD per hour (paid in ETH, DAI or SNT)
- Everything you create will be given back to the community as open-source software (licensed under MIT and Apache 2.0)
For the initial list of tasks, see this post. We look forward to your contributions!
Client logs: a call for a common standard
Currently, most of the bugs we see manifest themselves as finalisation issues.
Usually because either:
- We don’t get an attestation (networking issue)
- The node is late (it’s not fast enough and doesn’t finalise, or perhaps there’s a bug in the spec and it doesn’t finalise well)
Logging is currently our best way to understand what’s going on. In other words, it’s the best tool we have to understand whether a specific finalisation issue has it’s root in the networking layer or the consensus layer.
As it stands, the labels we give to our logs don’t always highlight what’s important to us. This means it can sometimes take longer than it should to get to the root of an issue.
We’ve started thinking about how to rework these labels, but since there are multiple clients working on eth2, we think it would be useful at this stage to start converging towards some sort of common standard.
While the exact scope of this standard is a matter of discussion, it should be broad enough to allow the same log analysis tool to work on multi-client testnets.
This would allow clients to avoid duplication of effort and ensure that all client logs can be parsed automatically.
Although the need for a standard was briefly discussed at the last September’s interop meeting, no formal API has yet been proposed.
Since we’ll be experimenting with multi-client testnets very soon, we believe now is a good time to start thinking again about this again.
Low-power device constraints: a shift in focus
It’s clear that a significant part of the world skipped the desktop computer and went straight to mobile. For eth2 to serve these communities, a mobile-focused client is needed.
There are three main challenges involved with designing for resource-contrained devices (inlcuding mobiles, rasperry Pi’s, routers, and older phones):
- If the device is not fast enough to process a slot, then it will fall behind and may never manage to catch up to the chain, rendering it essentially useless. To ensure as many devices as possible are able to validate, we need to make finalisation as efficient as possible. This means we need to spend a lot of time testing on low-power devices.
- Since mobile phones are not always connected and have a limited battery life, syncing needs to be very efficient and fast. One of the issues we need to watch out for with eth2 is that once you reach the latest finalised block, you need to keep track of potential new heads that diverge from the canonical chain at earlier points in time. These potential back and forths are costly from both a CPU and a storage point of view (since we need to store temporary blocks to be able to go back). They’re also a potential attack vector.
- Since mobile phones go offline a lot, peer discovery becomes even more of a challenge. How do you know if a peer has disappeared? How do you find peers once you’re back online? Hard coding is not an option, because if you hard code a list of peers, you open yourself up to centralisation issues (a government could easily spy on this hardcoded list).
While we’re still very much focused on making sure we are processing the spec correctly, we’re starting to think more and more about how to solve these networking and discovery issues on low-power devices.
We’ll be digging deeper into the specific challenges involved with designing for low-power devices in a follow-up post.
Honest validator work update
Honest validator work revolves around making sure that the Ethereum 2.0 Phase 0 – Honest Validator document is properly implemented.
This document represents the expected behaviour of an honest validator during Phase 0.
At this stage we’ve implemented pretty much everything outlined in the doc except for a couple of things related to signatures and attestations.
Signatures
The signature part of things is actually done, but we haven’t integrated it into Nimbus yet because we want to make sure they pass all the BLS test vectors first.
As it stands, 3/30 EF test vectors are buggy, so we’re waiting for the next spec release to integrate (this should happen any day now).
Attestations
We’re pretty close to being done with the attestation side of thing. We’re doing some aggregating, as the spec says, just at different times (as with signatures, fixing this requires integrating BLS changes first).
One (non-critical) thing left to do is slashing detection. At the moment we don’t track what the client signed, so they can sign conflicting messages (two blocks in one slot).
At some point we’ll need to be able to track this, and ideally add a mechanism to prevent them from doing this accidentally.
For example, if a validator signs a message, crashes before saving it, and then comes back online, they may sign a conflicting message unknowingly.
However, as this doesn’t affect interop, right now this isn’t a priority.
A note on test vectors
The latest BLS draft saw the release of official test vectors. These official test vectors differ from the ethereum test vectors in that they are testing a core component (hash-to-point), whereas the ethereum test vectors are testing the BLS signature part, which rely on hash-to-curve. So they are complementary.
BLS spec changes
The IETF’s BLS spec is still evolving to be more robust. And we expect there to be a new draft soon (this week or next).
The code has already been merged.
Libp2p native continues to stabilise
We’re continuing to make progress with stabilising our native Nim libp2p implementation (we’re working on getting it up to par with the much older and more stable Go implementation).
This is important as it’s been causing all sorts of finalisation issues.
Note: we currently have two testnets: testnet0, which uses the Go libp2p implementation, and finalises correctly; and testnet1, which uses our Nim libp2p implementation, and doesn’t finalise correctly.
The main reason we need a native implementation is that, in the long-term, we want to use a libp2p implementation in multiple projects where we’ll need full control and flexibility.
March milestones
At a high-level, we want to be at the interop stage by the end of March, so that we can be (multi-client) testnet ready for April.
We’re aiming to interop with one client – at the networking and discovery level, but not necessarily at the consensus level (since consensus will only be possible if other clients update to cater for the upcoming BLS changes).
For more details on specific milestones, see here.
In closing, we’re happy to announce that Tina Lee has joined us as a technical writer :)
That’s it for this week, we’ll talk again soon!