Conversation

We just wrapped up a *very* eventful #AllCoreDevs Agenda: github.com/ethereum/pm/is Stream: youtu.be/dByC5Bw8DvU Will recap here, but IMO if you only listen to a handful of calls per year, this one should be on your list 🎧
Quote Tweet
And that was it for today! Note that we'll have a CL call next Thursday, a Merge Community Call next Friday (github.com/ethereum/pm/is) and AllCoreDevs two weeks from today, June 10 at 14:00 UTC 👋🏻
Show this thread
Pre-merge, some CL clients had issues tracking deposits to the beacon chain. This was because they made specific assumptions around block times which did not hold under Ropsten's chaotic block times due to its variable hash rate as a testnet.
2
44
After The Merge on Ropsten, we saw a ~14% participation drop on the network. Danny has tweeted about this:
Quote Tweet
~14% of validators had downtime at the transition 1. 9% -- nimbus-team config issue, fixed with CLI change and redeploy 2. 1.8% -- known concurrency bug in nethermind, required a reboot for some nodes 3. 2.5-3% -- nimbus-besu web-sockets problem, swapped to use http 2/3
Show this thread
2
32
With the tweaks mentioned in the above tweet, participation rate went back up to ~99%, which is similar to mainnet. It's now a bit lower as teams are testing fixes and running other tests on live validators.
Image
1
50
We dove into the client issues in more details on the call. Again, strongly recommend watching the livestream to get all the nuance.
3
32
For the concurrency issue occurred because, at TTD, some nodes would receive new blocks both from the EL p2p network _and_ their CL client. This was made worse on Ropsten because of the high uncle block rate due to the bad mining conditions.
1
21
The Nimbus team shared a workaround with Besu which is working now, but it requires disabling JWT. The Besu team is currently working on a proper fix for the issue.
1
21
This specific bug was fixed, but the "deeper" cause is that Erigon does not support PoW mining. This means their code for block creation isn't s robust as other parts of the client and they are now focusing heavily on that part of the codebase.
2
31
Next up, we discussed an issue around CLs asking ELs for blocks before these have had time to create a block. There was a lot of talk about the intricacies of this interaction on the call ( even showed up!) so, again, recommend watching the livestream for the details!
1
20
In short, some CLs sometime ask ELs for a block before these have had a time to create one. Per the spec, the correct behaviour is for clients to return an empty block. This obviously isn't ideal because the EL can usually create _some_ block pretty quickly.
2
24
Because of this, Geth & Nethermind have implemented a workaround where, if this happens, they simply create a block "ASAP" and send that back. This isn't great, because it doesn't fix the root cause of CLs sending asking for the block too soon.
2
25
Again, most CLs have this fixed already, and others are focusing on it now. Once this is fixed, ELs will likely revert this functionality to instead follow the spec. In short, when they get asked for a block, they'll stop iterating on its construction, seal it, and send it back!
3
20
To unpack that a bit more: post-merge, validators control the gas limit on Ethereum, like miners do today. This parameter is set as part of the EL, when producing a block. Each time a block is chosen, you can raise/lower the gas limit by 1/1024th, or leave it as is.
2
21
If you assume that a staker is the one producing their block, then the staker passes their desired gas limit to their EL, and that's it! Things get more complex in a world where the staker may not be building the block themselves!
2
18
In practice, this is likely due to MEV. Validators who want to benefit from MEV fees need to run additional software to receive these blocks from searchers/relays.
2
19
(side note: the entire MEV design around the merge is out of scope, but had a great thread about it recently twitter.com/lightclients/s)
Quote Tweet
After the merge, there will be a much larger faction of "at home" block producers than we've seen under proof-of-work. In order for these independent stakers to extract MEV, we need to think differently about our approach to block production (thread).
Show this thread
2
20
So, for external block builders to be able to follow validators' gas limit preferences, it would be useful to have them be passed explicitly as part of Engine API calls. This is what the proposed change does :-)
2
18
We agreed on the call that this should _not_ come as a change to the current API, but rather as a separate version for it. There were also questions about whether other info should be passed to builders, such as the `extraData` field.
2
18
Everyone seems to agree this is useful, but there are still some details to iron out, which we'll keep doing over the coming weeks!
2
16
Similarly to Clique, the validator set on Sepolia will be permissionned. The Beacon Chain will go live in 10 days, and quickly run through Altair + Bellatrix with an artificially high TTD. When we are ready to merge Sepolia, we will perform a TTD override, like for Ropsten.
3
52
After this, I asked the various EL teams what they'd like to see before being confident in moving forward with testnets. Once more: there was plenty shared on the call, these are just rough highlights, please watch the livestream 📺!
2
24
Nethermind went first: for them, it was seeing hive/kurtosis tests all passing, block proposal issues fixed, and obviously having their current concurrency issue fully resolved. They said they'd want at least 1/2 of the testnets to have code which is basically final.
2
17
Erigon echo'ed passing tests, saying they still have a lot of them to fix themselves. Beyond that, they reiterated wanting to solidify their block building code, as well as resolving an issue related to returning the `latest` block in JSON RPC post-merge.
2
15
Show replies