protocols

eth2 Insights: Slashings

The second post in our eth2 Insights Series zooms into slashings in Medalla, examining their correlates and probable causes.

eth2 Insights: Slashings

By Elias Simos · Nov 9 2020
Last updated: Nov 10 2020

Welcome to our eth2 Insights Series, presented by Elias Simos–Protocol Specialist at Bison Trails. In this series, Elias reveals insights he uncovered in his joint research effort with Sid Shekhar (Blockchain Research lead at Coinbase), eth2 Medalla—A journey through the underbelly of eth2’s final call before takeoff.

In this second post, Elias looks at slashings in eth2 along with their correlates, probable causes, and the most common sources of slashing malaise in Medalla.


Intro

Slashing is a core protocol function and mechanism enabler in proof of stake networks. In very crude terms, if block rewards are the “carrot,” slashing is the “stick”—a negative incentive to keep network participants from misbehaving.

In Phase 0 of eth2, slashing is the fate of those validators that either: (i) double attest or double propose, or (ii) surround vote. To better understand slashing and the conditions that define its various forms, it’s important to remember how consensus is achieved in eth2. At a very high level, the body of validators broadcasts votes on the state of the chain (attestations) and subsequently these votes are aggregated and then passed on to the block producer to include in a block. But what do these votes include exactly?


Figure 1: The form of an aggregate attestation as it is included on-chain
Figure 1: The form of an aggregate attestation as it is included on-chain


The form of attester votes

When an attester votes, they must provide two types of votes: (i) a vote towards the head of the chain, and (ii) a vote towards finality. As such, an attester’s vote is described by the following core components/fields:

  • Target slot – this signifies the slot in which the attester has been allocated to attest in by the RANDAO, in a given epoch. It also signifies the vote towards the head of the chain, by LMD-GHOST.
  • Committee index – this is the group of validators that the attester has been assigned to attest with. Committees help with aggregating votes together, which end up on-chain as “grouped” attestations.
  • Source checkpoint [s] – a checkpoint is the first (proposed) block of an epoch. The source checkpoint is a vote towards what the attester recognizes as the last justified epoch boundary, by Casper FFG. A checkpoint is considered justified if ⅔ of validators in the active set have voted for it.
  • Target checkpoint [t] – this is a vote towards what should be the next justified checkpoint, by Casper FFG. The source and target votes help towards getting the chain state to finalize.

As soon as ⅔ of active validator votes are included on-chain for a said pair of source (s) and target (t) checkpoints related to an epoch, they become justified. The previous epoch’s pair become finalized—meaning that the state of the chain before that checkpoint boundary becomes unchangeable.


Slashing rules!

The Casper FFG paper outlines the two slashing conditions as follows:

  • Double vote – when two conflicting attester votes from the same validator that share the same target checkpoint (t1 = t2) get included on-chain. For proposers, a double vote translates to the proposer signing two different beacon blocks in the same slot.
  • Surround vote – an attester vote with source and target surrounding or surrounded by previously made votes, such that: s1 < s2 < t2 < t1

Slashing in Phase 0 not only triggers a balance reduction (which partly depends on how many other validators get slashed around the same time), but also an exit from the active set. This means that validators will join the exit queue and will not earn back the right to participate in consensus and earn rewards. It also means that their ETH will be frozen until Phase 1.5.

The entity responsible for calling out a slashable offence and enacting protocol level penalties is known as the “whistleblower.” The whistleblower collects the evidence that outlines the offence, and then commits the proof(s) to a block proposer for inclusion on-chain. While beyond Phase 0 the whistleblower stands to earn the majority of those rewards (⅞), for Phase 0 the block proposer will earn the full whistleblower reward.

Finally, slashable offences need not be discovered immediately after they actually take place. The protocol allows for a window of time between an offence being committed and the inclusion of a slashing proof, known as the “weak-subjectivity” period. This can span for a maximum of 54k epochs—a long time!

This window of time implies that the whistleblower needs to store an extensive record of chain history, and use that to survey for violations of protocol rules. Given the size of the chain’s state and the large share that attestations occupy, it follows that a leaner aggregated attestations state leads to a leaner overall history, which might then lead to better detection of rule violations!


The Medalla slashathon

Over the sample of 14,500 epochs we surveyed in eth2data.github.io, we observed approximately 5,100 attester slashings and 50 proposer slashings. Note that the overall number of slashings observed—taken from the beaconcha.in API—does not agree with the number reported by beaconscan.com which stands at ~1,900.

The misreporting is likely on beaconcha.in and it likely relates to beaconcha.in tracking “submitted slashing proofs” and not actual slashing events. Proofs for the same event can be submitted multiple times, but it is only the first that enforces the punishment.

Regardless, while the nominal values might be misleading, there is still value in surfacing the overall trends.

Attester slashings

The crushing majority of the attester violations in Medalla took place around epoch 2,200—at the beginning of the roughtime incident. To provide a little context, during roughtime Prysm client clocks went out of sync with network time. Because of this, validators incorrectly proposed blocks and submitted attestations for future slots.


Figure 2: Density plot of attester violations committed over time (s = blue, t = orange)
Figure 2: Density plot of attester violations committed over time (s = blue, t = orange)


While the offences were centered around the beginning of roughtime, only about 20% of the total slashing proofs were included around the same time. 94% of the whole set of slashing proofs was included by epoch 4,000. The remainder kept rolling in up until the very end of the sample of 14,500 epochs we looked at—arriving with a significant delay.


Figure 3: Slashing proofs included on-chain over time
Figure 3: Slashing proofs included on-chain over time


For clarity and ease of use, let’s call the delta between the offence taking place and the proof getting actually included on-chain, “detection delay.”

On average, the attester offence detection delay stood at 750 epochs. The top 10% of slashers, however, managed to get their proofs included in under 20 epochs! When trying to link those fast slashers to client choice, no deviation from the network-wide client distribution was found among the ⅓ that we were able to identify.


Figure 4: Distribution plot of detection delays
Figure 4: Distribution plot of detection delays


Now, things get really interesting when comparing the aggregate attestations surplus over time, with the average detection delay in slashable offences, measured on a per epoch basis.


Figure 5: Attestations surplus vs average detection delay over time
Figure 5: Attestations surplus vs average detection delay over time


The two trends appear to correlate! When the attestations surplus was at its highest early on in Medalla (up to epoch 2,200), slashings took on average 3000 epochs to get detected. As the Testnet matured things improved—with the average detection delay dropping to approximately 500 epochs. However, even then it appears that as the attestations surplus grew, the detection delay followed suit. This is a very important insight to keep in mind, as it means the poor aggregation performance in Medalla has a direct effect on the enactment of slashings!

Now, getting a little more granular with respect to the nature of attester violations, we observed that all enacted slashings link back to double votes! More specifically, 90% were down to double votes with the same source and target epochs but different target slots. Another 8% of the double vote slashings were attestations that disagreed both in the target slot and source epoch fields, while double vote slashings due to pure asynchrony in checkpoint values was a much less frequent phenomenon.


Table 1: Type of slashing observed, described by s, t and target slot
Table 1: Type of slashing observed, described by s, t and target slot


Note that 6.23% of the reported slashings do not appear to be slashable offences—likely due to a bug in the beaconcha.in API.

Moreover, none of the observations satisfy the surround vote condition. This doesn’t necessarily mean that no surround vote violations took place. However, surfacing these violations is orders of magnitude more complex than it is for simple double votes—and is an area of active research within the technical Ethereum community.

Finally, when investigating the client types of the proposers that included the slashing proofs, we found that a greater proportion of those were running Lighthouse than is representative of the overall client software distribution among validators in Medalla. There are two reasons why this might be the case.

The first is that as a lot of Prysm validators went down during roughtime, validators running Lighthouse represented a larger share of the population!


Figure 6: Distribution of proposers that included the slashing proofs, by client type
Figure 6: Distribution of proposers that included the slashing proofs, by client type


When looking at proofs submitted after epoch 4,000—when the network returned to “normality”—Lighthouse’s share appears even greater! Caveating for small sample size, this could indicate that the second reason for Lighthouse’s overperformance on the slashing front, is that the slashing software packed in the Lighthouse client might have an edge over its competition!

Proposer slashings

Proposer slashings are a little more straightforward than attester slashings and center around double proposals of blocks. In Medalla, these have been few and far between (47 events in total), with the majority concentrated around epoch 13,700. Our initial thought was that this was the case either due to a bug, or a client software upgrade that included a better “slasher” module.


Figure 7: Distribution of proposer slashing proofs included on-chain, over time
Figure 7: Distribution of proposer slashing proofs included on-chain, over time


After inspecting both the proposers that included the proofs and the proposers that got slashed, we think that the former is more likely as there was no evident pattern in the whistleblower/proposer client type.

We were able to map the slashed proposers to an overwhelming majority running the Nimbus client (~60%). When zooming in to the high density proposer slashing epoch range, that percentage grew to over 85%! Given that Nimbus was subject to syncing issues as attention of stakers turned from Medalla to Spading/Zinken around the same time, we think it’s likely that this was the cause of the proposer slashings here.


Figure 8: Distribution of slashed proposers by client_identifier
Figure 8: Distribution of slashed proposers by client_identifier


It’s also worth noting that the detection delay in proposer slashings was 1 slot for ~85% of the cases observed. This is a stark contrast to the average 750 epochs detection delay observed in attester slashings and a testament to the fact that detecting attester slashings is orders of magnitude more complex than detecting proposer slashings!



Figure 9: Distribution of detection delay in proposer slashings in Medalla
Figure 9: Distribution of detection delay in proposer slashings in Medalla



Conclusion

Overall, over the course of our research, as we were able to attribute the majority of slashings in Medalla to their most probable root causes, it becomes apparent that client syncing issues (specifically with Prysm during roughtime and Nimbus around epoch 13,700) have been the most common source of malaise for validators in the Testnet.

While we didn’t have enough time to survey the attestations log for slashable offences that weren’t enacted, in this post we showed that–indeed–it appears that poor aggregation performance has a direct effect on how quickly slashable offences are detected.

Given our thinking around the attestations surplus and how it can potentially degrade the performance of whistleblowers, it’s reasonable to expect that correlating attestations surplus over time with unpunished slashable offences might hold even more interesting insights!

And that’s perhaps a story for another time...


More eth2 insights

  • The first post in our eth2 Insights Series reveals important insights into the performance, and importance, of validator attestation aggregation in the eth2 network.
  • The second post in our eth2 Insights Series zooms into slashings in Medalla, examining their correlates and probable causes.
  • The third post in our eth2 Insights Series discusses the parameters governing validator effectiveness in eth2 and how validators were distributed along those in Medalla.
  • The fourth post in our eth2 Insights Series discusses Medalla’s arc of development, the metrics to gauge overall network health, and shares perspective on eth2 Mainnet.
  • Read our Q&A with Elias on the challenges faced during the eth2 Medalla Data Challenge and how data-informed research can help improve eth2 in the progression to a mainnet launch.

For Individuals

Are you an individual with a large amount of ETH? Please contact us to learn how to participate in eth2. ETH holders can also participate with LiquidStake powered by Bison Trails.

Contact Us


Become an eth2 Pioneer

It’s not too late to be an eth2 Pioneer. Learn more about the eth2 Pioneer Program for enterprise. We want you to have early access to build on the Beacon Chain!

Contact Us


About Bison Trails


Our mission is to provide superior infrastructure on multiple blockchains, to strengthen the entire ecosystem, and enable the pioneers of tomorrow.

Pioneering Blockchain Infrastructure®

Bison Trails is a blockchain infrastructure company based in New York City. We built a platform for anyone who wants to participate in 19 new chains effortlessly. We also make it easy for anyone building Web 3.0 applications to connect to blockchain data from 27 protocols with QT. Our goal is for the entire blockchain ecosystem to flourish by providing robust infrastructure for the pioneers of tomorrow.


bison cool

THIS DOCUMENT IS FOR INFORMATIONAL PURPOSES ONLY. PLEASE DO NOT CONSTRUE ANY SUCH INFORMATION OR OTHER MATERIAL CONTAINED IN THIS DOCUMENT AS LEGAL, TAX, INVESTMENT, FINANCIAL, OR OTHER ADVICE. THIS DOCUMENT AND THE INFORMATION CONTAINED HEREIN IS NOT A RECOMMENDATION OR ENDORSEMENT OF ANY DIGITAL ASSET, PROTOCOL, NETWORK OR PROJECT. HOWEVER, BISON TRAILS (INCLUDING ITS AFFILIATES AND/OR EMPLOYEES) MAY HAVE, OR MAY IN THE FUTURE HAVE, A SIGNIFICANT FINANCIAL INTEREST IN, AND MAY RECEIVE COMPENSATION FOR SERVICES RELATED TO, ONE OR MORE OF THE DIGITAL ASSETS, PROTOCOLS, NETWORKS, ENTITIES, PROJECTS AND/OR VENTURES DISCUSSED HEREIN.

THE RISK OF LOSS IN CRYPTOCURRENCY, INCLUDING STAKING, CAN BE SUBSTANTIAL AND NOTHING HEREIN IS INTENDED TO BE A GUARANTEE AGAINST THE POSSIBILITY OF LOSS. THIS DOCUMENT AND THE CONTENT CONTAINED HEREIN ARE BASED ON INFORMATION WHICH IS BELIEVED TO BE RELIABLE AND HAS BEEN OBTAINED FROM SOURCES BELIEVED TO BE RELIABLE BUT BISON TRAILS MAKES NO REPRESENTATION OR WARRANTY, EXPRESS OR IMPLIED, AS TO THE FAIRNESS, ACCURACY, ADEQUACY, REASONABLENESS OR COMPLETENESS OF SUCH INFORMATION.

ANY USE OF BISON TRAILS’ SERVICES MAY BE CONTINGENT ON COMPLETION OF BISON TRAILS’ ONBOARDING PROCESS, INCLUDING ENTRANCE INTO APPLICABLE LEGAL DOCUMENTATION AND WILL BE, AT ALL TIMES, SUBJECT TO AND GOVERNED BY BISON TRAILS’ POLICIES, INCLUDING WITHOUT LIMITATION, ITS TERMS OF SERVICE AND PRIVACY POLICY, AS MAY BE AMENDED FROM TIME TO TIME.

Latest News

help

Contact Us

Get in touch

General
Sales
Press
Legal