protocols

eth2 insights: network performance


The fourth post in our eth2 Insights Series discusses Medalla’s arc of development, the metrics to gauge overall network health, and shares perspective on eth2 Mainnet

eth2 insights: network performance

By Elias Simos · Nov 23 2020

Welcome to our eth2 insights series, presented by Elias Simos–Protocol Specialist at Bison Trails. In this series, Elias reveals insights he uncovered in his joint research effort with Sid Shekhar (Blockchain Research lead at Coinbase), eth2 Medalla—A journey through the underbelly of eth2’s final call before takeoff.

In this fourth post, Elias reviews how Medalla developed from beginning to close, introduces metrics that will help participants gauge overall network health and highlights some key takeaways ahead of Mainnet launch!


Key highlights and findings

  • Overall network health improved gradually over time in Medalla, paving the way for a strong Mainnet launch.
  • If Medalla is anything to go by, Genesis will be where the eth2 network will be the most vulnerable. Clients teams and operators need to be ready well in advance.
  • In a similar spirit, it seems that pace/rhythm matters for the network–as an interruption in momentum (roughtime) hampered the Medalla network’s ability to finalize over the desired window for much longer than the interruption window.
  • Given that economic incentives will be activated in Mainnet, we will see some of the baseline network health indicators improve drastically compared to Medalla.
  • Given the overall complexity of the endeavor, the early stages of Phase 0 will be concentrated both on the “ETH at stake” level as well as the deeper layers of the network (infrastructure providers, client software etc). This spells a very attractive opportunity (from a rewards perspective) for the brave early adopters that choose to swim alongside the innovators because rewards are shared between fewer parties.

Intro

In this fourth and final post of eth2 insights stemming from the data that the Medalla Testnet produced, I will be looking into overall network health metrics. As we edge closer to Mainnet, with over 50% of the ETH required committed at the time of writing, this feels like the right send-off. My goal here is to equip you–the reader–with the right metrics and enough background information so that you are better able to contextualize activity on the network come Mainnet.

The analysis in this post spans between epochs [0, 14572] which map to slots [0, 466,335]—a time window that corresponds with approximately the beginning of Medalla (Aug 4th, 2020) to the first half of October 2020 (8th).

Over the course of this window over 78,846 unique validator indices became active on Medalla. The maximum number of activated validators observed stood at 72,554, and the maximum amount of staked Testnet ETH stood at 2.35M.


Network growth over time


Figure 1: Validators in the active set over time–and threshold for finality
Figure 1: Validators in the active set over time–and threshold for finality


27% of the validators that were active over the Testnet’s life cycle were activated on epoch 0. This means that the threshold for finality at the start of Medalla stood at ~13.3k valuable attestations.

From that point onwards it was a steady rise for the active validator count; roughly 10k validators were added to Medalla every 2000 epochs (~9 days). Interestingly, though while more validators were deployed, no new significant operator groups entered the Testnet at its later stages. In fact, from the Top-20 operators, most entered before epoch 5,000 and a large majority were active since Genesis.

A glaring exception in the steady growth pattern of activated validators is the window between epoch 2,000 and epoch 4,000—the period in which the roughtime incident took place.

The count of ETH deposits over time gives us a clearer view of the network interruption during the incident; no deposits took place approx. between epochs [2200, 3500], while we are observing a similar density of deposits at Testnet start and at Testnet “re-start.”


Figure 2: Testnet ETH deposit count over time
Figure 2: Testnet ETH deposit count over time



Defining normality

Now, the above already hints clearly towards the fact that not all network states are made equal! Given this, there is merit in thinking along the lines of “normal” and “abnormal” network conditions.

During the roughtime incident, and for a period of approx. 1,500 epochs (1 week), 75% of the slots available to block proposers were missed. Without enough blocks being proposed, the network could not achieve finality for a prolonged period of time.


Figure 3: Rate of slots proposed to slots missed
Figure 3: Rate of slots proposed to slots missed


In a perfect world, 100% of the block slots available would be proposed—helping the network to continue moving in perpetuity, and all activated attesters would vote and so on; alas, the world is not perfect—but it can still be “normal!”

When plotting out the distributions of missed and proposed block slots over the set of epochs we based this analysis on we found that they both follow approximately normal distributions, with fat tails left and right that capture the essence of “abnormal” network conditions, like the roughtime incident.


Figure 4: Distributions of block slots proposed and missed
Figure 4: Distributions of block slots proposed and missed
Figure 4: Distributions of block slots proposed and missed


When excluding the roughtime incident (the fat tail) from the set of observations—broadly epochs (2000, 4000)—we found that under “normal” conditions Medalla clocked at 3:1 proposed : missed slots! This means that, in every epoch, 24 slots were proposed and 7 were missed on average.

In and of itself, the fact that so many slots were missed under normal network conditions should raise some eyebrows. With a lot of slots being missed, the chain is slower to reach finality and the load of information included per slot increases—likely making it harder to survey past chain states for misconduct.

What could be different come Mainnet, however, is that since being negligent will be punished, a lot of the “absenteeism” we saw in Medalla is likely to course-correct. Indeed, as Pintail uncovered in their research, approximately 10% of validators in Medalla were entirely dormant or abandoned for the majority of the Testnet’s duration.


Figure 5: Distributions of validator states (source: pintail.xyz)
Figure 5: Distributions of validator states (source: pintail.xyz)


At this point, it’s also worth noting that under “normal” conditions the uncle rate for Medalla stood at roughly 1% of all block slots per epoch—a stark contrast to the ~5% on average that eth1.x produces.


Figure 6: Proportion of blocks orphaned
Figure 6: Proportion of blocks orphaned


The rate of orphaned blocks (correct blocks that do not make it on the canonical chain due to latency issues in the P2P layer of the network—i.e. the message doesn’t make it on time) remained stable over the course of the Testnet, with notable spikes around the beginning of the roughtime incident and at around epoch 13,700, when the network appears to have experienced another short spurt of high latency. This second spurt of high latency led to both 70% of the blocks being orphaned for the duration of an epoch and more than 60% of all block slots being missed a few epochs later.


Aggregate inclusion delay

The inclusion delay is a key moderating factor for the reward an attester stands to achieve by voting on blocks. For every additional block that the inclusion is delayed, the rewards reduce further. The reason why this is the case is that the network prefers that information relating to its state to be committed sooner rather than later, thus enhancing liveness and security.


Figure 7: Aggregate inclusion delay over time in Medalla
Figure 7: Aggregate inclusion delay over time in Medalla


In the aggregated view above, we looked at the whole set of attestations that were submitted on the Tesnet and their respective inclusion delays, calculating the average delays per epoch, per validator.

While pre-roughtime the aggregate inclusion delay ranged between 5 and 15 block slots, post-roughtime the aggregate inclusion delay smoothed out over time—from an average of 9 slots in epoch 4,000 to approximately 4 slots by epoch 14,700.

This view highlights that kickstarting a network is hard. The improvement we saw in Medalla over time took place as multiple client upgrades were released and then subsequently adopted by the network. Translating this into a learning for Mainnet, it means that client teams need to be ready with their releases and give operators enough headway pre-genesis to adopt them.


Finality distance

As a reminder finality is a state the chain achieves where a block (and its contents—e.g. transactions) can no longer be changed without a large percentage of stake getting destroyed. Eth2 guarantees finality, with a desired target of 2 epochs to reach it, at any given point in time.

Over the course of our research in eth2data.github.io we thought it would be interesting to come up with a visualization that surfaces a sense of timing in the chain’s efforts to reach finality. We did this by benchmarking for the target slot and counting backwards for when the attestations necessary to reach finality actually arrived.


Figure 8: Finality distance in epochs [0, 3000]
Figure 8: Finality distance in epochs [0, 3000]
Figure 9: Finality distance in epochs [0, 14500]
Figure 9: Finality distance in epochs [0, 14500]


From the above, it is clear that pre-roughtime Medalla was able to finalize within the target 2 epochs for over 80% of the time. However, as roughtime kicked in, the problem becomes crystal clear: between epochs [2500, 3200] Medalla could barely get 15% of the total valuable attestations necessary to achieve finality included!

When the network re-started, and for a period roughly 4500 epochs long, Medalla was finalizing within 3+ epochs consistently. It took until about epoch 11800 to start finalizing in the 2 epoch desired window again.

It looks as if pace/rhythm really matters for the network—as an interruption in momentum (roughtime) hampered the network’s ability to finalize over the desired window for much longer than the interruption window.


Conclusion

Medalla was a vibrant Testnet, with valuable learnings for participants and observers alike. Zooming out on the analysis we did in eth2data.github.io and the eth2 Insights Series, it seems as if Medalla truly served its purpose; we saw a clear improvement in overall network conditions as the Testnet matured, and also had the chance to take learnings from a doomsday-level fire-drill (roughtime).

Given that economic incentives will be activated in Mainnet, I also fully expect as Phase 0 takes hold we will see some of the baseline indicators (such as blocks proposed : blocks missed) improve drastically.

At the same time, high complexity, incomplete tooling, and frozen ETH will be deterring factors for players that have not been active in the Testnets. As such, Phase 0 adoption over time will probably be a case of “gradually and then suddenly,” with a chorus of familiar voices active from the start, but with the second and third cohorts trickling in gradually over time.

This will likely lead to a relatively concentrated Phase 0 in its early stages, both on the “ETH at stake” level as well as the deeper layers of the network (infrastructure providers, client software etc); but it also spells a very attractive opportunity (from a rewards perspective) for the brave early adopters that choose to swim alongside the innovators.

For all intents and purposes, we are still VERY early in what is slated to be the most exciting network launch in years.

Buckle up buckaroos—we’re in for a hell of a ride!


More eth2 insights

  • The first post in our eth2 Insights Series reveals important insights into the performance, and importance, of validator attestation aggregation in the eth2 network.
  • The second post in our eth2 Insights Series zooms into slashings in Medalla, examining their correlates and probable causes.
  • The third post in our eth2 Insights Series discusses the parameters governing validator effectiveness in eth2 and how validators were distributed along those in Medalla.
  • The fourth post in our eth2 Insights Series discusses Medalla’s arc of development, the metrics to gauge overall network health, and shares perspective on eth2 Mainnet.
  • Read our Q&A with Elias on the challenges faced during the eth2 Medalla Data Challenge and how data-informed research can help improve eth2 in the progression to a mainnet launch.

For individuals

For custodial staking, use Coinbase's eth2 retail staking solution powered by Bison Trails.

Learn about Coinbase Staking


Become an eth2 Pioneer

It’s not too late to be an eth2 Pioneer. Learn more about the eth2 Pioneer Program for enterprise. We want you to have early access to build on the Beacon Chain!

Contact us


About Bison Trails


Pioneering Blockchain Infrastructure®

Bison Trails is a blockchain infrastructure platform-as-a-service (PaaS) company based in New York City. We built a platform for anyone who wants to participate in 21 new chains effortlessly.

We also make it easy for anyone building Web 3.0 applications to connect to blockchain data from 33 protocols with Query & Transact (QT). Our goal is for the entire blockchain ecosystem to flourish by providing robust infrastructure for the pioneers of tomorrow.

In January, 2021, we announced Bison Trails joined Coinbase to accelerate our mission to provide easy-to-use blockchain infrastructure, now as a standalone product line. The Bison Trails platform will continue to support our customers. With Coinbase’s backing, we will enhance our infrastructure platform and make it even easier to participate in decentralized networks and build applications that connect to blockchain data.


bison cool

THIS DOCUMENT IS FOR INFORMATIONAL PURPOSES ONLY. PLEASE DO NOT CONSTRUE ANY SUCH INFORMATION OR OTHER MATERIAL CONTAINED IN THIS DOCUMENT AS LEGAL, TAX, INVESTMENT, FINANCIAL, OR OTHER ADVICE. THIS DOCUMENT AND THE INFORMATION CONTAINED HEREIN IS NOT A RECOMMENDATION OR ENDORSEMENT OF ANY DIGITAL ASSET, PROTOCOL, NETWORK OR PROJECT. HOWEVER, BISON TRAILS (INCLUDING ITS AFFILIATES AND/OR EMPLOYEES) MAY HAVE, OR MAY IN THE FUTURE HAVE, A SIGNIFICANT FINANCIAL INTEREST IN, AND MAY RECEIVE COMPENSATION FOR SERVICES RELATED TO, ONE OR MORE OF THE DIGITAL ASSETS, PROTOCOLS, NETWORKS, ENTITIES, PROJECTS AND/OR VENTURES DISCUSSED HEREIN.

THE RISK OF LOSS IN CRYPTOCURRENCY, INCLUDING STAKING, CAN BE SUBSTANTIAL AND NOTHING HEREIN IS INTENDED TO BE A GUARANTEE AGAINST THE POSSIBILITY OF LOSS. THIS DOCUMENT AND THE CONTENT CONTAINED HEREIN ARE BASED ON INFORMATION WHICH IS BELIEVED TO BE RELIABLE AND HAS BEEN OBTAINED FROM SOURCES BELIEVED TO BE RELIABLE BUT BISON TRAILS MAKES NO REPRESENTATION OR WARRANTY, EXPRESS OR IMPLIED, AS TO THE FAIRNESS, ACCURACY, ADEQUACY, REASONABLENESS OR COMPLETENESS OF SUCH INFORMATION.

ANY USE OF BISON TRAILS’ SERVICES MAY BE CONTINGENT ON COMPLETION OF BISON TRAILS’ ONBOARDING PROCESS, INCLUDING ENTRANCE INTO APPLICABLE LEGAL DOCUMENTATION AND WILL BE, AT ALL TIMES, SUBJECT TO AND GOVERNED BY BISON TRAILS’ POLICIES, INCLUDING WITHOUT LIMITATION, ITS TERMS OF SERVICE AND PRIVACY POLICY, AS MAY BE AMENDED FROM TIME TO TIME.

Latest news

help

Contact us

Get in touch

General
Sales
Media
Legal