Nimbus Development Update - 12.07.2019.

Here's what's new since our last update!

Spec Freeze

The Ethereum 2.0 specification was recently frozen to version 0.8. This means that the design documentation of Ethereum 2.0's phase 0 will not change, regardless of improvements we find, except if it's a showstopping bug. With this freeze in place on time, all the teams are scrambling to get in sync with it so that everyone can start working on making the clients talk to each other.

Nimbus is currently (almost) in sync with 0.8. - what's missing are a dozen minor things which should be mostly resolved by the time the next update is out. Once that's done, we'll hook into the official tests suite and make sure everything passes as specced.

Also as part of these 0.8 sync ops, we've achieved a 10-15x (yes!) speedup in processing crosslinks and figuring out differences between them, and a 2x speed boost in crosslink committee fetching by introducing more caching. Does that mean we're this bad at developing, or this good at optimizing? 🤔 Regardless, our simulation is flying, and out testnet is getting ready to perform at a much more serious level. A pending restart with more validators and shards and shorter slot times is around the corner.

Metrics dumping

As the project grows, it started to become ever harder to track what's going on without going in and dumping everything we can think of to the screen as we need it. With that in mind, we've got a basic metrics output repo set up at nim-metrics which is built to conform to what Prometheus, the monitoring toolkit, expects to get from tools.

To get metrics out of Nimbus, compile it with the -d:metrics --threads:on flags. this is already the case when following official instructions.

Eth1 + Beacon Node

The nim-web3 package is functioning at a viable enough level to register validator deposits from Ethereum 1 inputs. What we have right now is the ability to call Eth1 contract methods from nim, which lends itself nicely to our beacon node needs. It's now possible to generate deposits in something like a locally running Ganache and trigger the Ethereum 2.0 genesis event. However, we messed up. We missed an update in the spec about signaling deposit events and building the genesis event on the client side. By not considering every individual deposit as a viable genesis event, our genesis construction is currently out of date with the spec. 🙈 That'll be fixed by the time the next update rolls around!

The implementation of validators depositing into a running Ganache can be tested in this PR if you're interested.

There's a silver lining in all this - our web3 library is now mature enough to power a rudimentary wallet. Since it can issue and read transactions, it can also send, receive, and list assets. Stay tuned for a neat little proof of concept on that!

Libp2p testnet

Our playground testnet - Testnet1 - is now running on libp2p.

It doesn't work on Windows right now because of a pending update in our nim-libp2p wrapper. A fix will be deployed soon! That said, it'll work fine in a Linux VM, so if you're on Windows, a Vagrant box is always an option.

To try the libp2p testnet out, check out the Nimbus devel branch, then run make update deps. Then just run:

cd vendor/nim-beacon-chain
make clean-testnet1 testnet1 # clean cache and rebuild binary
./build/testnet1_node # this launches the testnet0-specific node you just built

Nimplay

Our domain specific language for Ethereum smart contracts targeting eWASM is shaping up nicely.

Here's an example of the KingOfTheHill contract in which the last submitter is registered as king of the hill. A simple proof of concept for the language:

contract("KingOfTheHill"):

    # Globals
    var
        king_name: bytes32
        king_addr: address
        king_value: uint128

    proc becomeKing*(name: bytes32) {.payable.} =
        if msg.value > self.king_value:
            self.king_name = name
            self.king_addr = msg.sender
            self.king_value = msg.value

    proc getKing*(): bytes32 = 
        return self.king_name

    proc getKingAddr*(): address = 
        return self.king_addr

    proc getKingValue*(): uint128 = 
        return self.king_value

The * means "public" just like in good old Nim.

As you can see, this syntax is as clean as Vyper, only cooler (because it targets eWASM) 🤘

Nimbus Connecting to Parity and Geth

Nimbus (Ethereum 1 version) now recognizes Parity and Geth as peers just fine. Here's how you can test this:

1. Run Geth or Parity in full/warp sync mode.
2. Run nimbus with the --staticnodes flag, providing the enode address of Geth or Parity after this argument.

You first need to build Nimbus for this to work. Follow the official instructions to do that.

The fact that we can now connect to these other nodes and remain connected will help Nimbus fetch missing blocks like a real node, no longer needing to "cheat" by fetching block by block from Geth directly as it did previously.

We also made Nimbus much more stable. It can now process up to block 800k and counting. We'll see how that works out in terms of database size and speed once we hit the dangerous blocks of Shanghai DOS and similar problems, but it's looking good so far!

Other improvements

• Nim Chronos (https://github.com/status-im/nim-chronos/pull/42/files) now has a simple HTTP client (no SSL support for now). This will eventually replace our curl dependency in the beacon node code which is there as a hack, but for now the new HTTP client serves as a means to avoid problems with NAT-UPNP (nodes being able to find each other) and for http-json-rpc, i.e. RPC calls to other nodes. We should also be able to roll out websocket support on top of this.
• Nim Stew is our hodgepodge of utilities and mini libraries that don't have a home of their own. This is our experimentation ground and our incubator for larger libs and modules - if something outgrows the stew, we'll put it into a dedicated repo with its own docs. We did this to somewhat shrink our dependency graph and import verbosity.
• Nimbus is mostly single-threaded right now. Because of that, we don't do a very good job of talking to Go programs like the libp2p go daemon - if CPU intensive tasks like verification of signatures and hash calculation take over a thread, that thread cannot handle network events and can cause timeouts with peers. We've been working on async channels to improve that. Basically, we plan to introduce a separate thread for block processing and the network I/O thread will communicate with it through an async channel (currently being developed as part of Nim Chronos)
• On the Whisper front, we made some modifications which bring us in relative sync with other Whisper-enabled tools like Geth and Parity.

via GIPHY