protocols

Who’s who in eth2: Raul Jordan from Prysmatic Labs


In the first edition of Bison Trails’ series highlighting eth2 client teams, Elias Simos interviews Raul on the team’s ethos, community building, and more.

Who’s who in eth2: Raul Jordan from Prysmatic Labs

By Elias Simos · Feb 27 2021

Welcome to Who’s who in eth2, presented by Elias Simos, Protocol Specialist at Bison Trails. In this series, Elias interviews key contributors at eth2’s four client teams—Prysmatic Labs, Sigma Prime, Nimbus, and Teku—to explore their involvement in eth2, visions for the future, and insider perspectives on what eth2 means for the world.

In this first post, Elias interviews Raul Jordan, Co-lead Ethereum Developer at Prysmatic Labs, on the team’s origin story, ethos, community building efforts, future outlook, and more.


Why don't we start with a little bit of background on yourself and your involvement with eth2?

Absolutely, and thank you once again for doing this. My name is Raul Jordan. I'm one of the maintainers of Prysmatic Labs. I'm originally from Honduras and first came to the US–where I currently live–to study CompSci at Harvard.


Raul Jordan, Prysmatic Labs

I was always fascinated with what you could do with a computer. The fact that you can use a keyboard and just type in magic words and all of a sudden create something that people on the other side of the world can use was pretty mind blowing to me as a student.

During my time as a student, I was introduced to Ethereum. I met a few people from the community (e.g. the teams at Augur and Aragon) and was immediately intrigued.

Towards the end of 2017, after the whole hype of the bull run, the conversations around the future of Ethereum were really ramping up. At the time Vitalik [Buterin] put out a lot of content on the topic and also published a grant for work related to Ethereum scaling.

Back then, I was still an “outsider” wanting to break into the ecosystem and scaling seemed like one of the most interesting problems to apply myself to. But at the time, it was really not obvious who was working on these problems. The only information available was something like one FAQ page and a bunch of scattered blog posts.

I'm somebody that likes to take problems into my own hands. I really think that if you work with the right people, you can really accomplish anything. So I just gave it a shot. I put out a call on Reddit that went something like, “Hey, I’m a developer looking into Ethereum. Anyone want to collaborate with me on prototyping this sharding thing Vitalik talks about?”


Figure 1: Raul’s first email to everybody who responded to the call-to-action Reddit thread


And then I actually got a bunch of responses. The first one was from Preston Van Loon (co-founder at Prysmatic Labs) saying he was interested. Next thing you know we are meeting up at his office in Google, where he was working full-time. We bounced ideas back and forth and settled on giving this a shot. And then suddenly we had around six or seven volunteers on the same boat.

Looking back, I think it was all about timing and being at the right place at the right time. We eventually managed to get a grant from the Ethereum Foundation to do this on a more incentivized basis. And that's how Prysmatic Labs came to be.

"I'm somebody that likes to take problems into my own hands. I really think that if you work with the right people, you can really accomplish anything."


Let’s dig in a little bit deeper in the background of the team—how it all came together and the journey of Prysmatic labs from inception to present day.

Absolutely! I think if you want to pull off an organic open source initiative with people from around the world, you need to have people that are really motivated by a shared goal.

I started this effort with Preston while he was at Google. And two other people that we had from day one that are still with us today are Terrence (Tsao) and Nishant (Das), both of whom are just stellar individuals. They were there from the beginning, always learning, curious, and wanting to improve.

Most of the early team initially were basically working two jobs. They would work as software engineers at their day jobs, and then come back at five and work until midnight on this volunteer project. We then started getting a few more grants from the foundation, and that's when things really took off.

I remember I was with Preston in an Ethereum meetup and then he told me “the only reason I would leave Google is if I had a 10x opportunity.” And I think that, in terms of the opportunity that shipping eth2 would provide, it was more than enough for Preston and the other guys to jump ship and join full-time.


And what about the values that are core to Prysmatic Labs?

We want to showcase to people that we are grassroots. We are builders that like to bring good software engineering practices to the mix and be very community driven, which we have been from day one.

That's something that I noticed personally was lacking in the eth1 development space. People complained a lot about things happening behind closed doors. So we really wanted this to be an open forum, and I think that drove the point home. I think by now we must have roughly 10k people on our discord server.


Figure 2: Subscribers on Prysmatic Labs’ Discord server between November 2020 and February 2021


"I think that our company culture is 'no nonsense.' We just want to build code that works and has been proven to work."


How does that openness, having the community involved from early on in the development journey, impact the end-product? Where do you think it really shines? Where do you think it also introduces bottlenecks in the workflow that you wouldn't expect to have if the process was more controlled?

I think the openness is incredible–thinking about what it brings to the robustness of protocols. If you only have one implementation of the spec, in my view, you're 99% more likely to have serious bugs, meaning that you don't properly implement the specification. And you will never know because there's no other implementation that could point you in the direction of production level disagreements (e.g. blocks, state roots). Having multiple implementations is a big strength.

On the flipside, I would say wrangling interoperability between teams that are distributed all over the place can be challenging. Things improved as time went by, but initially–in part because of fragmented communication–we had to rewrite our code from scratch almost two or three times because the specification changed under our feet.

And when there are disagreements, without a clear leader in the process, how do you resolve that conflict? You end up having this deadlock, and it feels like things are moving really slowly.

Overall, however, the relationship between all the teams has been really cordial. The Ethereum Foundation had a big role to play here, driving the research and project managing the development at a high level. None of us would be here if we didn't have help from the likes of Danny Ryan and protolambda from the Foundation. They are really the heroes of this.

At the end of the day, I think all the difficulties along the way have been for the better. We have a really stable mainnet so far.


As far as communication between the different teams that were implementing the specification, would you say it manifested as a hub and spokes flow, or more as a nexus where you had free cross-communication flow between the different implementers?

I would say it's probably more of a nexus approach. The EF is basically in charge of the research, but that doesn't mean that they dictate it. The field was open from the beginning for broader members of the core implementer community to come in with feedback and ideas. Actually, I believe the top contributor outside of the core research team is Terrence from our team.

During the latter months of development, maybe six or seven months leading up to mainnet launch, everything was driven pretty much by client implementers as the spec was frozen, meaning that there were no major changes on the Phase 0 spec. In that time, all the teams were basically just communicating with each other, doing multi-client testing, and such.

"If you only have one implementation of the spec, in my view, you're 99% more likely to have serious bugs, meaning that you don't properly implement the specification. And you will never know because there's no other implementation that could point you in the direction of production level disagreements (e.g. blocks, state roots). Having multiple implementations is a big strength."


Q: Going back to the 10x opportunity—I have a sense that based on the way you framed it, it's not only about 10x financial opportunity. What is it about eth2 that makes it a 10x opportunity in your mind?

What that means is that this really represents a breakthrough technology in computer science, and leaving a mark as one of the major implementations running the protocol is really what drives us to do what we do. The team has had incredible chemistry. We want to prove to people that we can ship this, that we can be a part of history.

Thinking further down the line I think we can go in so many directions after the roadmap is complete. And I would love to do it with the current team that we have today.


Let's now talk a little about the process of creating Prysm (the client). What were some of the key design decisions that you made early on— thinking about software itself, as well as things outside of that?

The way we approached the problem from very early on was very hands-on. We wanted to take problems into our own hands and really build something useful.

When picking a programming language, often it comes down to personal preference or your past experience. But for us, we didn't have long experience in Go development (Prysm is built in Go). But we wanted the software to be practical and made the intentional decision to build it in Go.

I think that our company culture is “no nonsense.” We just want to build code that works and has been proven to work.

So we picked Go as the programming language because I would say it's the language of the cloud for the last four or five years. A significant portion of cloud services and large scale applications that serve billions of users today are written in Go. And because of the large developer base that builds with Go there is a very long library of tools that we can draw from in the development process; that means we don’t need to go write an API or a json RPC from scratch.

As an example, in Prysm we use protocol buffers, which are a transport mechanism for API payloads. For that we use GRPC, which is an RPC library, for implementing these RPC services at scale.

These things serve, quite literally, billions of users on a day-to-day basis. They run all of Uber, they run all of Google, for example, and countless other companies. So we know these things work. We don't feel we need to reinvent the wheel.


What about non-technical decisions?

One of the biggest things that helped us get to where we are today was deciding to launch a single client public testnet for people to try out very early on. We didn't even have a peer-to-peer networking specification at the time we launched the first Prysm testnet. And we were like–screw it! We're going to build our own, because people want to try this out. They wanted to see what staking was like. They wanted to test out the limits of the software.

We also were very deliberate about building a lot of documentation, onboarding guides etc., so that people can engage with the software. Having users trying it out and giving us continuous feedback really helped to push the envelope. It wasn't until a year later that we had a multi-client testnet.


Figure 3: The Prysmatic Labs team

That's a really great segue to the community building part of things. You mentioned that having strong written communication really helped push things forward. What other approaches did you leverage to engage with the community that you think drove results?

Writing is for sure appreciated by the community. All of us on our team are always answering stakers’ questions. People jump in the Discord server and they're like, “What does this mean? can you guys explain it?” And, somebody always pops in and answers.

Over time, we built up a strong culture of DevOps. For example, when the Medalla incident happened, we followed our standard practice of doing post-mortems. Then we go through, we try to understand what happened and take out learnings, identify where we got lucky, and ask “how can this never happen again?” We built that culture of post-mortems and we made them public. In fact, the Medalla Medium post was actually just us basically copy pasting that post-mortem into a blog post. Thankfully, now the eth2 initiative has a dedicated response team–the Blue Team–where you have people not just from our team, but from other teams and the foundation that are able to respond to incidents if/when they happen.


I say client diversity is key for long-term success. You say...

Totally, I would say the same. I think it's really nice to be in a position where a lot of people are using your software, but it also comes with a lot more responsibility. So if something were to go wrong with Prysm, for example, we would really depend on all of the other clients to step up and keep the network going. Diversity helps the software become a lot more resilient.

Even if a client has a small share of the entire set of operators, it's still critical that that project is maintained and that it's supported by the parties that need to support it. If there's a serious issue in the network and those clients are still well-maintained they can help remediate these problems and people can migrate those clients.

The relationship we have with all the other teams has improved a lot over time. We appreciate each other and we appreciate the improvements that everyone is making to their software.

"The key to the success that Prysm has had so far is that we focused a lot on the average staker."


How do you think eth2 can achieve an appropriate level of client diversity?

The key to the success that Prysm has had so far is that we focused a lot on the average staker. I will tell you, the average staker right now most likely runs Windows, is somewhat technical and as such is aware of the risks and what it means to stake, but is not a guru on the command line—doesn't have a Linux set up in their living room, so to speak. So the first thing I’d say is to really focus on the user.

Another one is really good documentation, good communication, making sure you respond to feature requests. And eventually, making the software much easier to use through point-and-click interfaces.


Do you think there are any spec level changes in the cards that could incentivize a broader distribution of clients?

So that question is a difficult one. I think there's a lot of ways that you can try, but the problem is that you game those metrics. It's really difficult to create an incentive for beacon nodes that can not be gamed–that's something to keep in mind. Honestly, the best we can do really is to just focus on improving the bottom line.


Thinking about the future, what do you think the biggest challenges that lie ahead for eth2 are?

Now, we have a ticking time bomb on our hands, in the sense that eth2 is live. We still have to ship Phase 1, add sharding, etc. That, in and of itself, is a separate beast. And these things need to be done in parallel. So, how do we prioritize our time as a team to accomplish this?

So far, we’ve relied a lot on the wisdom of the Ethereum Foundation on the research side for what to do next. They've been incredibly insightful. And now we need to ship the next phases ASAP–ideally this upcoming year (2021).

That's the main challenge I see ahead. We’re on a treadmill that is not going to stop. We need to get going and to implement this stuff.


Further along down the line, after it’s all said and done, what would you see Prysmatic Labs becoming?

I want to see Prysmatic Labs as a highly regarded core implementation team. We are a team that thrives on experimentation, with a core focus on improving the Layer 1. And we want to keep going, even once that roadmap is done. I think that our team can succeed at any future challenge. It will be a dream come true for me if we can build something on top of eth2 in the future. So whether that is a really interesting smart contract based application or tackling scalability, I'll be really thrilled to do that.


Final question: what's your favorite detail/anecdote about another client?

I thought it was super cool when Nimbus posted a photo of themselves running a small connected network of Raspberry Pis or something, running perfectly on like some large scale testnet. And just how little resources the client consumes is really amazing. Like any sort of device can probably run Nimbus–that is really cool. I think those guys are really pushing the envelope of what's possible on lower-end devices.

And like I said, if there's a team/client that remains well-maintained, and is aligned with the specification and works on mainnet, we need it to be used by people, we need it to keep going and to be supported. Hopefully one day they can get it running on a smart fridge or something similar. I think those guys have the ability to do it.


Interview by Elias Simos


About Bison Trails


Pioneering Blockchain Infrastructure®

Bison Trails is a blockchain infrastructure platform-as-a-service (PaaS) company based in New York City. We built a platform for anyone who wants to participate in 21 new chains effortlessly.

We also make it easy for anyone building Web 3.0 applications to connect to blockchain data from 33 protocols with Query & Transact (QT). Our goal is for the entire blockchain ecosystem to flourish by providing robust infrastructure for the pioneers of tomorrow.

In January, 2021, we announced Bison Trails joined Coinbase to accelerate our mission to provide easy-to-use blockchain infrastructure, now as a standalone product line. The Bison Trails platform will continue to support our customers. With Coinbase’s backing, we will enhance our infrastructure platform and make it even easier to participate in decentralized networks and build applications that connect to blockchain data.


bison cool

THIS DOCUMENT IS FOR INFORMATIONAL PURPOSES ONLY. PLEASE DO NOT CONSTRUE ANY SUCH INFORMATION OR OTHER MATERIAL CONTAINED IN THIS DOCUMENT AS LEGAL, TAX, INVESTMENT, FINANCIAL, OR OTHER ADVICE. THIS DOCUMENT AND THE INFORMATION CONTAINED HEREIN IS NOT A RECOMMENDATION OR ENDORSEMENT OF ANY DIGITAL ASSET, PROTOCOL, NETWORK OR PROJECT. HOWEVER, BISON TRAILS (INCLUDING ITS AFFILIATES AND/OR EMPLOYEES) MAY HAVE, OR MAY IN THE FUTURE HAVE, A SIGNIFICANT FINANCIAL INTEREST IN, AND MAY RECEIVE COMPENSATION FOR SERVICES RELATED TO, ONE OR MORE OF THE DIGITAL ASSETS, PROTOCOLS, NETWORKS, ENTITIES, PROJECTS AND/OR VENTURES DISCUSSED HEREIN.

THE RISK OF LOSS IN CRYPTOCURRENCY, INCLUDING STAKING, CAN BE SUBSTANTIAL AND NOTHING HEREIN IS INTENDED TO BE A GUARANTEE AGAINST THE POSSIBILITY OF LOSS. THIS DOCUMENT AND THE CONTENT CONTAINED HEREIN ARE BASED ON INFORMATION WHICH IS BELIEVED TO BE RELIABLE AND HAS BEEN OBTAINED FROM SOURCES BELIEVED TO BE RELIABLE BUT BISON TRAILS MAKES NO REPRESENTATION OR WARRANTY, EXPRESS OR IMPLIED, AS TO THE FAIRNESS, ACCURACY, ADEQUACY, REASONABLENESS OR COMPLETENESS OF SUCH INFORMATION.

ANY USE OF BISON TRAILS’ SERVICES MAY BE CONTINGENT ON COMPLETION OF BISON TRAILS’ ONBOARDING PROCESS, INCLUDING ENTRANCE INTO APPLICABLE LEGAL DOCUMENTATION AND WILL BE, AT ALL TIMES, SUBJECT TO AND GOVERNED BY BISON TRAILS’ POLICIES, INCLUDING WITHOUT LIMITATION, ITS TERMS OF SERVICE AND PRIVACY POLICY, AS MAY BE AMENDED FROM TIME TO TIME.

Latest news

help

Contact us

Get in touch

General
Sales
Media
Legal