Decentralization is the key innovation enabled by blockchain technology and one of the most important characteristics of web3 protocols. As a result, web3 participants, policymakers, and regulators must develop a more uniform and nuanced understanding of what constitutes decentralization to enable more accurate assessments and comparisons of the decentralization of various web3 protocols. This will also better position web3 regulation and policy to understand and account for how decentralization can reduce risk and ultimately incentivize web3 builders to pursue decentralization in such a way that maximizes the public benefits that web3 promises.

To that end, we define three types of decentralization and suggest factors relevant for each type of decentralization in the context of (1) tokenized blockchain protocols (e.g., layer 1 and layer 2 blockchains like Bitcoin, Ethereum, Polygon, Solana, Optimism, Arbitrum, zksync, etc.) and (2) tokenized smart contract protocols deployed to blockchains (e.g., Uniswap, Aave, Compound, Curve, etc.). We also introduce two tables – one for (1) tokenized blockchain protocols and one for (2) tokenized smart contract protocols, viewable at their respective links – that should help enumerate the components of decentralization to provide a more concrete and standardized definition.

Any analysis of the decentralization of a blockchain protocol or smart contract protocol must consider the totality of the circumstances surrounding such a protocol. The factors we lay out here are an attempt to chart a course for such analysis.

Why decentralization?

Web3 has ushered in a new age of the internet – the age of “read, write, own.” The technology underpinning web3 enables “trustless computation,” which removes the need to rely on a centralized entity to navigate the web and databases. This makes it possible to develop more complex and sophisticated protocols that offer the functionality of the modern internet but that can also be owned by users. For example, a decentralized social media protocol could enable various applications to be built, all of which leverage the same protocol to distribute ownership and control over that protocol to a broad constituency of developers and users through token ownership.

Decentralization is the critical feature of web3 protocols that enables this paradigm shift. Decentralization will drive the creation of a democratized internet and will enable three important shifts: promoting competition, safeguarding freedoms, and rewarding stakeholders.

First, decentralization enables web3 systems to be credibly neutral (they cannot discriminate against any individual stakeholder or any group of stakeholders, which is critical to incentivize developers to build within ecosystems) and composable (to mix and match software components like Lego bricks). As a result, web3 systems function more like public infrastructure than proprietary technology platforms. In contrast to the gated software of Web2, web3 protocols provide decentralized internet infrastructure on which anybody can build and create an internet business. Crucially, in web3, this can be done without the permission of the original deployer of the protocol or the need to use a centrally controlled interface. For example, contrast, say, Twitter with a web3 protocol that provides an underlying data architecture designed for social media that is controlled by the public through token ownership rather than a corporation. In such a system, anyone could build their own client or application on top of the protocol and gain access to its network of users.

This is abstract, so consider this diagram of a web3 ecosystem, with a decentralized blockchain, a decentralized smart contract protocol governed by a DAO of token-holders, and several proprietary clients operated as separate businesses using traditional entity forms on top of the network and protocol. Each of the blockchain and smart contract protocols functions as decentralized internet infrastructure on which businesses can build, compete, and innovate.

Second, decentralization necessitates the broad distribution of control and participation in web3 protocols, ensuring that the evolution and usage of the network reflects the input of a wide variety of stakeholders and not just the companies that created them. Properly structured protocols that promote decentralization limit the power that can accrue to one or a small group of companies. As such, decentralization should limit corporate or individual power to gatekeep, and ensure that any changes to the protocol are aligned with its broad ecosystem of users that hold tokens and ultimately govern the network.

Third, decentralization enables the design of systems that prioritize stakeholder capitalism – systems that are designed to more equitably serve the interests of all participants rather than a certain subset of them. Token-incentivized stakeholder capitalism distributes ownership and control to a broader set of stakeholders rather than prioritizing equity holders over all other stakeholders, including customers and employees. As a result, web3 protocols and networks serve as a fertile design space for systems that more equitably serve the interests of all stakeholders. And such decentralized protocols provide more stable internet infrastructure on which a broader group of stakeholders can confidently build. 

Types of decentralization

There are three different but interrelated lenses through which to view decentralization: Technical, Economic, and Legal. All three are important but often have competing interests, creating a complex design challenge in maximizing overall decentralization and utility.

Technical decentralization (T)

Technical decentralization relates primarily to the security and structural mechanisms of web3 systems. Programmable blockchains and autonomous smart contract protocols can support technical decentralization by providing an autonomous, permissionless, trustless, and verifiable ecosystem in which value can be transferred. Products and services can be deployed and run without requiring trusted, centralized intermediaries to operate (or pull the rug out from under) them, opening a vast world of possibilities.

For blockchain protocols, technical decentralization is an exceedingly challenging problem, and one that requires a balancing of several competing forces. But for smart contract protocols, this type of decentralization can be achieved relatively quickly and easily, by making the smart contracts immutable (i.e., uncontrollable and unupgradable by anyone). (For more examples, see here and here.)

Economic decentralization (E)

The ability of blockchains and smart contract protocols to make use of their own native tokens unlocks the potential for these open source and decentralized systems to have their own decentralized economies (i.e., autonomous free-market economies) and for more people to participate in and benefit from these decentralized ecosystems.

Builders of web3 systems can facilitate the formation of decentralized economies through careful design decisions that lead to the exchange and accrual of value — whether information, economic value, voting power, or otherwise — from a broad array of sources. Decentralized ecosystems, if properly structured, can use tokens to incentivize participants to contribute value to the ecosystem and correspondingly distribute that value more equitably among system stakeholders according to their contributions. To achieve this, web3 systems need to vest meaningful power, control, and ownership with system stakeholders (via airdrops, other token distributions, decentralized governance, etc.). As a consequence, the value of the ecosystem as a whole accrues to a broader array of participants rather than one central entity and its shareholders.

The ongoing balancing of incentives among the stakeholders — developers, contributors, and consumers — can then drive further contributions of value to the overall system, to the benefit of all. In other words: all the benefits of modern network effects, but without the pitfalls of centralized control and captive economies.

Legal decentralization (L)

Legal decentralization depends on whether the decentralization of a system eliminates the risks that a specific regulation may be intended to address.

For instance, technically decentralized blockchains and smart contract protocols can eliminate the risks associated with trusted intermediaries. As a result, the technical decentralization of such systems may also mean that they are legally decentralized when it comes to regulations that target trusted intermediaries.

Systems that are technically and economically decentralized can eliminate additional risks, including those associated with a web3 system’s tokens and their underlying value. Such decentralization would negate the need to apply U.S. securities laws to transactions of tokens that might otherwise significantly restrict their broad distribution.

Based on SEC guidance, we can define this level of legal decentralization as that point at which a web3 system can eliminate both the potential for significant information asymmetries to arise and reliance on essential managerial efforts of others to drive the success or failure of that enterprise. Upon meeting such a threshold the system may be “sufficiently decentralized” so that the application of U.S. securities laws to such system’s tokens should be unnecessary. As a threshold matter, this necessitates that a given token does not provide its holder with any contractual rights with respect to the ongoing efforts, assets, income, or resources of the issuer or any of its affiliates. 

Factors of decentralization

In web3 systems that make use of native tokens, the three types of decentralization — technical, economic, legal — must be viewed holistically. Changes to one may affect the others. For example, decentralized economies help drive systems towards legal decentralization by prioritizing decentralized ownership among stakeholders, value accretion from decentralized sources, and value distribution to decentralized stakeholders. All of these decrease the risk of information asymmetries and the need to rely on managerial efforts of individuals.

Conversely, if the value of a digital asset of a web3 system is dependent on the ongoing managerial efforts of the original development team, then the decentralization of the system on all three levels may be jeopardized. For example, the management team’s departure could place enormous downward pressure on the price of the digital asset, which could make the system more susceptible to a 51% attack.

Given this interplay, we have broken down decentralization into the many factors that may influence it. Tables 1 and 2 present a comprehensive list of the most important factors that influence the decentralization of tokenized consensus blockchain protocols and tokenized smart contract protocols, respectively. These factors are broken down by both type (Technical, Economic, and Legal) and category (Computational, Development, Governance, Value Accrual, and Usage & Accessibility).

***

Decentralization is a process that is assessed not on an absolute basis but on a spectrum that includes the totality of the circumstances of any web3 system. The relative importance of the factors will change depending on the web3 system and the purpose of the assessor. Additionally, the preferred tradeoffs between different types of decentralization may differ between projects and people.

The two matrices we’ve prepared should prove, we hope, useful tools in providing a more concrete and standardized definition of decentralization. In turn, we hope this allows web3 participants to contribute to building more decentralized projects while empowering policymakers and regulators to design regulatory frameworks that recognize the power of decentralization to reduce and eliminate risks.

***

***

Miles Jennings is General Counsel and Head of Decentralization of a16z crypto, where he advises the firm and its portfolio companies on decentralization, DAOs, governance, NFTs, and state and federal securities laws.

Steve Wink is Co-Chair of Latham’s Fintech Industry Group and Global Digital Assets & Web3 Practice and is a Partner in the New York office of Latham & Watkins. He advises clients on a variety of matters involving the regulation of markets, as well as related compliance and enforcement matters, and is ranked in Band 1 by Chambers’ Professional Advisers FinTech, and is recognized by Chambers USA as one of the country’s leading financial services broker-dealer regulation lawyers.

Adam Zuckerman is a member of the Digital Assets & Web3 and Emerging Companies practices in the Bay Area offices of Latham & Watkins, where he advises crypto and Web3 clients across their lifecycle on token launches, DAO formation, product counseling, a variety of regulatory matters, general corporate matters, and financings.

***

Disclaimer: These materials were written in partnership between a16z crypto and Latham & Watkins LLP and are for informational purposes only.  The materials do not and should not be construed as legal advice for any particular facts or circumstances.  None of the materials provided hereby are intended to be treated as legal advice or to create an attorney-client relationship.  The materials might not reflect all current updates to the law or applicable interpretive guidance and the authors disclaim any obligation to update the materials.  We strongly urge you to contact a reputable attorney in your jurisdiction to address your specific legal needs.

Under applicable Rules of Professional Responsibility, portions of this communication may contain attorney advertising.  Prior results do not guarantee a similar outcome.  Results depend upon a variety of factors unique to each representation.  Please direct all inquiries regarding the conduct of Latham & Watkins attorneys under New York’s Disciplinary Rules to Latham & Watkins LLP, 885 1271 Avenue of the Americas, New York, NY 10020, Phone: +1.212.906.1200.

Additionally, the views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not necessarily the views of a16z or its affiliates. a16z is an investment adviser registered with the U.S. Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party information; a16z has not reviewed such material and does not endorse any advertising content contained therein.

This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities, digital assets, investment strategies or techniques are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments or investment strategies will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.

Additionally, this material is provided for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. Investing in pooled investment vehicles and/or digital assets includes many risks not fully discussed herein, including but not limited to, significant volatility, liquidity, technological, and regulatory risks. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. All materials used in this document, unless otherwise stated, are joint copyright works of a16z crypto and Latham & Watkins. Please see https://a16z.com/disclosures/ and https://www.lw.com/ for additional important information regarding regulatory disclosures.

All voting systems rely on integrity and transparency to function in any meaningful way. At face value, this makes blockchains an ideal platform to build these systems on – and indeed, many decentralized organizations have embraced permissionless voting to express collective intent, often in the context of wielding substantial treasuries or tuning critical protocol parameters. But there are drawbacks to on-chain voting, and privacy remains unexplored to the detriment of web3 voting systems – in the majority of on-chain voting protocols used today, ballots and vote tallies are fully public. Without privacy, voting results are susceptible to manipulation and misaligned voter incentives, potentially leading to undemocratic outcomes. 

That’s why we are releasing Cicada: a new, open source Solidity library that leverages time-lock puzzles and zero-knowledge proofs for private on-chain voting. Compared to existing systems, Cicada has novel privacy properties, minimizes trust assumptions, and is efficient enough to be used on Ethereum mainnet. 

In this post, we survey the landscape of voting privacy and provide a high-level description of how Cicada works (with formal proofs to come). We also encourage developers to check out the GitHub repository – Cicada can be adapted and extended in many ways to support different voting schemes and features, and we hope to collaborate with the community to explore these possibilities. 

A brief survey of private voting

In any voting system (on-chain or otherwise), there are many different layers of privacy to consider. The disclosure of individual ballots, the running tally, and voter identities can all influence voter incentives in different ways. Which privacy properties are necessary depends on the context of the vote. A few that frequently arise in cryptography and social science literature:

• Ballot privacy: The secret ballot, also called “the Australian ballot” was developed for real-world voting systems as a way to keep the preferences of individual voters private, and mitigate bribery and coercion (in an on-chain setting, we may need a stronger property than ballot privacy – see “receipt-freeness” below). Ballot privacy also mitigates social desirability bias – there is less pressure for someone to vote based on what others think of their choice.

• Running tally privacy: Many voting systems hide the running tally, or how many votes have been cast for each option, while voters are still casting ballots to avoid impacting turnout and voter incentives. We’ve seen this play out in the real world; for example, US Senators who vote later are more likely to align with their party than those who vote earlier. And on-chain: in token-weighted voting, whales can lull their opponents into a false sense of security by letting them hold the lead (some may not bother to vote, assuming they will win regardless) and then cast their ballot at the last minute to swing the outcome.

• Voter anonymity: Your vote is private in many real-world voting systems, but the fact that you voted is often public. This can be important as a safeguard against voter fraud, because publishing records of who voted allows people to check whether someone else cast a ballot in their name. On-chain, however, we can prevent voter fraud while preserving anonymity using cryptographic primitives – with Semaphore, for example, you can prove in zero knowledge that you are an eligible voter who hasn’t cast a ballot yet. 

• Receipt-freeness: Individual voters provide a “receipt” of their ballot to prove how they voted to third parties, which otherwise might lead to vote selling. A closely related but stronger property is coercion-resistance, which prevents someone from coercing a voter to vote a certain way. These properties are especially appealing in a decentralized setting, where voting power can be made liquid via smart contract marketplaces. Unfortunately, they are also very difficult to achieve – in fact, Juels et al. state that it is impossible in a permissionless setting without trusted hardware.

Cicada is focused on running tally privacy, but (as we discuss later) it can be composed with zero-knowledge group membership proofs to obtain voter anonymity and ballot privacy as well. 

Introducing Cicada: Tally privacy from homomorphic time-lock puzzles

To achieve running tally privacy, Cicada draws from cryptographic primitives that (to our knowledge) have never been used on-chain before. 

First, a time-lock puzzle (Rivest, Shamir, Wagner, 1996) is a cryptographic puzzle that encapsulates a secret which can only be revealed after some predetermined amount of time has elapsed – more specifically, the puzzle can be decrypted by repeatedly performing some non-parallelizable computation. Time-lock puzzles are useful in the context of voting for achieving running tally privacy: Users can submit their ballots as time-lock puzzles, so that they are secret during the vote but can be revealed afterwards. Unlike most other private voting constructions, this enables running tally privacy without relying on tallying authorities (like election staff counting paper or digital ballots), threshold encryption (where several trusted parties must cooperate to decrypt a message), or any other trusted parties: anybody can solve a time-lock puzzle to ensure results are revealed after the vote.

Second, a homomorphic time-lock puzzle (Malavolta Thyagarajan, 2019) has the additional property that some computation on the encrypted value is possible knowing the secret key, decrypting the puzzle, or using a backdoor. In particular, a linearly homomorphic time-lock puzzle allows us to combine puzzles together, producing a new puzzle that encapsulates the sum of the original puzzles’ secret values.

As the authors of the paper note, linearly homomorphic time-lock puzzles are a particularly suitable primitive for private voting: Ballots can be encoded as puzzles, and they can be homomorphically combined to obtain a puzzle encoding the final tally. This means that only one computation is needed to reveal the final tally, instead of solving a unique puzzle for every ballot.

A new construction: efficiency and tradeoffs

There are several more considerations to make for a voting scheme to be practical on-chain. First, an attacker may try to manipulate the vote by casting an incorrectly encoded ballot. For example, we might expect each ballot’s time-lock puzzle to encode a boolean value: “1” to support the voted-upon proposal, “0” to oppose it. An ardent supporter of the proposal may instead attempt to encode e.g. “100” to amplify their effective voting power.

We can prevent this sort of attack by having voters submit a zero-knowledge proof of ballot validity alongside the ballot itself. Zero-knowledge proofs can be computationally expensive though – to keep the costs of voter participation as low as possible, the proof should be (1) efficiently computable client-side and (2) efficiently verifiable on-chain.

To make proving as efficient as possible, we use a bespoke sigma protocol – a zero-knowledge proof designed for a specific algebraic relation, as opposed to a generalized proof system. This enables extremely fast prover times: generating a ballot validity proof in Python takes 14ms on an off-the-shelf laptop. 

Though the verifier for this sigma protocol is conceptually simple, it requires a few large modular exponentiations. Malavolta and Thyagarajan’s linearly-homomorphic scheme uses Paillier encryption, so these exponentiations would be performed modulo N^2 for some RSA modulus N. For a reasonably sized N, the exponentiations are prohibitively expensive (millions of gas) on most EVM chains. To reduce this cost, Cicada instead uses exponential ElGamal – exponential ElGamal still provides additive homomorphism, but works over a much smaller modulus (N instead of N^2).

One downside of using ElGamal is that the last step of decrypting the tally requires brute-forcing a discrete log (note that this is done off-chain and efficiently verified on-chain). As such, it is only suitable if the expected final tally is reasonably small (e.g. less than 2^32, or about 4.3 million votes). In the original Paillier-based scheme, the tally can be efficiently decrypted regardless of its size.

Selecting the RSA modulus N also involves a tradeoff. Our implementation uses a 1024-bit modulus for gas efficiency. While this is well above the largest RSA modulus ever publicly factored (which was 829 bits) it is below the normally recommended size of 2048 bits for use with RSA encryption or signatures. However, we don’t need long-term security in our application: once an election is finished there is no risk if N is factored in the future. The tally and ballots are assumed to become public after the time-lock expires, so it is reasonable to use a relatively small modulus. (This can also be easily updated in the future if factoring algorithms improve.)

Anonymity and voter eligibility

Cicada, as described above, provides running tally privacy – the time-lock puzzle property keeps the tally private for the duration of the vote. However, each individual ballot is also a time-lock puzzle, encrypted under the same public parameters. This means that just as the tally can be decrypted (by performing the requisite computation), so can each individual ballot. In other words, Cicada only guarantees ballot privacy for the duration of the vote – if a curious observer wishes to decrypt a particular voter’s ballot, they can do so. Decrypting any individual ballot is as expensive as decrypting the final tally, so naively it requires O(n) work to fully decrypt a vote with n voters. But all of these ballots can be decrypted in parallel (assuming enough machines), taking the same amount of wall-clock time as it takes to decrypt the final tally.

For some votes this may not be desirable. While we are satisfied with temporary running tally privacy, we may want indefinite ballot privacy. To accomplish this, we can combine Cicada with an anonymous voter eligibility protocol, instantiated by zero-knowledge group membership proofs. This way, even if a ballot is decrypted, all it would reveal is that someone voted that way – which we would already know from the tally.

In our repository we include an example contract that uses Semaphore for voter anonymity. Note, however, that the Cicada contract itself makes no assumptions on how voter eligibility is determined or enforced. In particular, you could replace Semaphore with e.g. Semacaulk or ZK state proofs (as proposed here and here). 

Tallying authorities

One of our priorities in designing Cicada was to avoid the need for tallying authorities: Many private voting constructions require a semi-trusted tallying authority (or committee of authorities, coordinating via secure multi-party computation) who receives and aggregates the ballots. In a blockchain context, this means that these schemes cannot be conducted by a smart contract alone and that some human intervention and trust is needed.

In most constructions, tallying authorities are not trusted for integrity (they cannot manipulate the vote count), but they are trusted for liveness – if they go offline, the final result cannot be computed, indefinitely stalling the outcome of the vote. In some constructions they are also trusted to maintain privacy – that is, they learn how each individual votes but are expected to publish the vote result without revealing this information.

Though tallying authorities are a reasonable (and necessary) assumption in many real-world scenarios, they are not ideal in a blockchains context, where our objective is to minimize trust and ensure censorship resistance. 

***
Cicada explores one of many directions in the field of on-chain voting privacy, and complements much of the ongoing research being done by other teams. As mentioned above, Cicada goes hand-in-hand with anonymous group-membership technologies like Semaphore, ZK storage proofs, and rate-limiting nullifiers. Cicada could also integrate the optimistic proof checker proposed by Nouns Vortex team to reduce the gas burden on voters. 

There are also opportunities to adapt Cicada to support different voting schemes (e.g. token-weighted voting, quadratic voting) – more complex schemes may be too computationally expensive for Ethereum mainnet, but they could be practical on L2s. With that in mind, we welcome your contributions, forks, and suggestions on where to take Cicada next. 

Acknowledgements: Cicada was developed jointly with Joseph Bonneau. Thanks to Andrew Hall for supplying context around the historical context of voting privacy. Thanks also to Robert Hackett for feedback on this post. Special thanks to Stephanie Zinn for editing. 

***

Views expressed in “posts” (including articles, podcasts, videos, and social media) are those of the individuals quoted therein and are not necessarily the views of AH Capital Management, L.L.C. (“a16z”) or its respective affiliates. a16z is an investment adviser registered with the U.S. Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party information; a16z has not reviewed such material and does not endorse any advertising content contained therein.

This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities, digital assets, investment strategies or techniques are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments or investment strategies will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.

Additionally, this material is provided for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. Investing in pooled investment vehicles and/or digital assets includes many risks not fully discussed herein, including but not limited to, significant volatility, liquidity, technological, and regulatory risks. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures/ for additional important information.

The White House recently proposed a new mining tax. Their 30 percent tax will be imposed not on the mining of minerals but on the mining of cryptocurrencies. But the so-called DAME Tax seems unlikely to accomplish any of the goals that the administration has laid out to justify it. The intention is to reduce any negative impact from cryptocurrency mining on local electricity prices and global pollution. However, while a tax can reduce crypto-mining in the US, it is far from obvious, from an economics perspective, that the other goals will follow. 

Mining taxes can be popular amongst economists. Minerals are a pure gift from the land that no one who has been around for the last 100,000 years was responsible for. While it takes some effort to extract and then ship minerals, some great luck is involved in having the license to mine a particular tract of land. The economic term for the profits that flow from this is called rent. Even if the government imposes a resource rent tax, the land will likely be mined anyway. All that changes is who gets the rent – the miner or the government – which makes the tax less popular amongst miners.

It seems perhaps unfair to compare mineral mining to crypto mining, but the crypto community chose the name for a reason. In Proof of Work blockchains, what legitimizes nodes – which propose and confirm blocks of transactions at any given time – is that they have to demonstrate that they are there to behave well and not obstruct or attack the underlying network. Satoshi Nakamoto went to some considerable effort to outline what might constitute “work” for the Bitcoin blockchain and to design it in such a way that it could scale.

Nakamoto’s notion of work was imposed on everybody who wanted to be a node to help run the network – an entry fee if you will. To get the chance to confirm a block of new transactions, nodes would have to compete to solve a simple computational puzzle. The puzzle was meaningless – no one cared about the answer – but it was integrated neatly into the blockchain itself. To win this computational contest, miners had to find the answer before anyone else. While they couldn’t guarantee that outcome, the more resources – that is, compute – they applied to the problem, the more likely it was that they would win. 

And what did miners get for their trouble? Transaction fees paid by users, for starters. But, more significantly, they also received new bitcoins. Just how much they earned changed over time, but when bitcoins started to be worth some real money, the prize for ten minutes of computational effort became significant. They searched and dug (by offering resources and guessing puzzle answers), and then “extracted” (by receiving tokens if they solved it): mining. 

Like mineral mining, the total amount of resources (compute) devoted to Bitcoin mining was dictated by the value of the outcome. The more bitcoins were worth, the more intense the computational contest. If blocks were minted too quickly each one would process fewer transactions per reward. Thus, to keep mining to an average of 10 minutes per block confirmation, the difficulty of the computational puzzle would adjust. The more compute added to the contest, the more difficult it became, and vice versa.

Given that the contest to mine bitcoins is open to anyone with a computer worldwide, how did miners make money? After all, the outcome of the broader game was driven by free entry. If there were profits to be had, it would pay someone somewhere to devote a processor and their electricity bill to the contest. They wouldn’t win often, but on average, their winnings would cover their costs. Indeed, Nakamoto outlined in his whitepaper a more democratic process. But economically, expected profits would be low, and  not at the more predictable level of diamond mines. 

Consequently, the mining business became big, and miners formed pools to provide a more certain stream of income. The miners also became more sophisticated, evolving from a person with a computer in their basement to big data centers with thousands of specialized ASIC processors devoted to crypto mining. The electricity bill from those data centers grew. Electricity companies (and chip makers) weren’t complaining, though – any more than shovel makers did during the Gold Rush.

One result is that, by some estimates, crypto mining was consuming as much electricity as a small country. And for what, skeptics (some might say cynics) asked? To play a computational game? For cryptocurrency that seemed to some like monopoly money or worse, casino chips? What did the rest of society get from this other than higher electricity bills and increased pollution locally and potentially globally? For the past decade or more, the crypto community – at least those focused on Proof of Work –  hasn’t really had a good answer to that question. 

Yet, despite this, if you were to ask an economist, they would be hard-pressed to condemn crypto mining-based electricity consumption relative to any other electricity consumption. Yes, crypto mining might look like a waste of resources – and if there is one thing economists don’t like, it is wasted resources. Many critiqued Bitcoin, for instance, for using the electricity generated by a reasonably sized country like Sweden. But you know who else is using the electricity generated by a reasonably sized country like Sweden? Sweden. And economists don’t particularly seem to mind Sweden. The point being that people are actually paying for the crypto-mining electricity and seemingly of their own free will. Who are we to judge?

Apparently, many governments are happy to judge. Some, like China, have prohibited crypto mining altogether (albeit for reasons beyond environmental concerns). The Biden proposal, called the DAME (Digital Asset Mining Energy) excise tax, stops short of prohibition but would mark up US crypto miners’ electricity bills by a flat 30 percent. The goals are, ostensibly, to lower electricity prices and, although this seems contradictory, to reduce both local and carbon pollution. 

The tax  is not expected to be a big revenue generator – just a few billion over the next decade – because mining electricity bills aren’t actually that large; and also, crypto mining is globally competitive. Raise costs by that much and, unlike minerals, crypto miners can relocate to anywhere with an internet or satellite connection. 

Therein lies the issue. If the goal were to reduce what was regarded as sinful waste (in the same way one might tax tobacco to reduce health problems), it is unlikely to happen at a global level. Crypto mining exists in the U.S. because it is cheaper to mine there than anywhere else on the planet. If the tax causes some of those mines to shut down and others to open elsewhere, there would be more, not less, waste produced. 

But it is even worse than that. It is far from obvious how this change will actually reduce global pollution. Reducing local pollution in the U.S. may be possible, but that pollution will follow the miners somewhere else, so this is what we would call a “beggar-my-neighbour” outcome. As the name suggests, it’s kind of selfish. Moreover, the U.S. government’s massive climate policy that passed last year involved billions of dollars of investment in renewables and innovations to make energy production less climate-damaging. (Not to mention that many Proof-of-Work cryptocurrency miners have been locating their efforts in areas where there is latent capacity, or mixing in more renewable energy.) 

The DAME tax will cause some of the users of that cleaner energy to go elsewhere. Frankly, it is highly unlikely they will end up in a cleaner place.

Indeed, this seems counter to the moves by some in the proof-of-work crypto industry to try and promote more clean energy. While I am personally skeptical about some of the ways advocates claim this might be done, if mining demand, as a large potential user of electricity in a region, can underwrite new investment in renewable generation, then this is a way that approach to crypto may spur cleaner energy in the long run. Such schemes are being proposed – and the DAME tax may threaten them. If you take a renewable project’s biggest customer and say that 30 percent of the bill must go to the government, much of the cost will likely come off the bottom line of the renewable electricity provider. That is not an investment we would want to deter. 

The point here is that the DAME tax targets crypto mining for reasons that would otherwise apply to many electricity users – including those not involved in mining cryptocurrency. Given that mining is globally competitive, it is unlikely to move the needle on the environment and could, in fact, subvert it. A better approach would be to tax miners who rely on non-renewable electricity generation. But that sounds like what it is: a carbon tax. And some in the U.S. government are loath to impose such a measure, even though it would undoubtedly help the environment.

***

Joshua Gans is a Professor of Strategic Management at the Rotman School of Management, University of Toronto and Chief Economist of its Creative Destruction Lab. His forthcoming book is The Economics of Blockchain Consensus (to be published by Palgrave/Macmillan in July). Joshua Gans is not associated with a16z, nor has he referred to any work or persons at a16z. All views are his. 

***

Views expressed in “posts” (including articles, podcasts, videos, and social media) are those of the individuals quoted therein and are not necessarily the views of AH Capital Management, L.L.C. (“a16z”) or its respective affiliates. a16z is an investment adviser registered with the U.S. Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party information; a16z has not reviewed such material and does not endorse any advertising content contained therein.

This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities, digital assets, investment strategies or techniques are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments or investment strategies will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.

Additionally, this material is provided for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. Investing in pooled investment vehicles and/or digital assets includes many risks not fully discussed herein, including but not limited to, significant volatility, liquidity, technological, and regulatory risks. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures/ for additional important information.

Robert: Hi everyone, Robert Hackett back here again with another episode for web3 with a16z. I recently chatted live with some of our researchers on the topic of Data Availability Sampling and Danksharding – which is relevant to blockchain scaling, as well as paving the way for more advanced blockchain networks & user applications. While much of the discussion is especially relevant to Ethereum, some of the concepts we cover are also applicable to advances in computing and networking generally. 

This discussion involves some specific math, which we cover (and briefly explain) in the episode.

For quick context as you listen, “polynomial commitments”, which you’ll hear more about, are a tool that helps reduce the amount of data needed to verify complex computations. And “interpolation”, another term you’ll be hearing, is a way to reconstruct data from a limited set of data points.

Be sure to check out the paper referenced in this episode for a deeper explanation at a16zcrypto.com/das – that’s DAS for data availability sampling. As always, none of the following is investment, business, legal or tax advice. See a16z.com/disclosures for more important information, including a link to a list of our investments.

Robert: Okay, so today we’re going to be talking about data availability sampling and danksharding. Now, if you’re unfamiliar with those things, don’t be scared because I’m here with a couple of experts who are gonna break things down for you. We’ve got Dan Boneh, Stanford professor, eminent cryptographer, and a16z crypto research advisor, and Lera Nikolaenko, research partner at a16z crypto.

Lera: Great to be here. Thanks, Robert. 

Dan: Thanks, Robert. Great to be here. 

Robert: Thanks, Lera. Thanks, Dan. So, as mentioned, Dan and Lera recently wrote up a great post and have a proposal related to ways to improve protodanksharding, which is this upgrade that is planned for later this year in Ethereum. There is a lot of rich detail in that post, so we’ll dig in there.

But before we get into all the details, I thought maybe we could zoom out and back up and start with a bigger picture here. So let’s start with Dan. Help us understand the subject of today’s talk, data availability sampling. And maybe you could do it without breaking anyone’s minds here.

Maybe we could keep it as you know, succinct and accessible to a broader audience as you might be able to. 

Dan: Sure, so maybe we can zoom out a little bit before we talk about data availability sampling. Maybe let’s talk about the general goal here. So this is part of the efforts to scale Ethereum.

And so one of the ways to scale Ethereum is using rollups. And rollups actually need to push a lot of data on chain. So a rollup allows you to take, say, a hundred or a thousand transactions and process them all as a single transaction on the Ethereum Layer 1. And then all the data associated with those transactions basically gets pushed on chain.

And as a result, there’s a need to actually store quite a lot of data on chain. And the question is how to do that. And that’s exactly where danksharding comes into play. So it’s a really beautiful, beautiful idea. It came from the Ethereum Foundation, particularly from Dankrad Feist. It’s a really elegant construction and basically Lera and I wanted to give an overview of how this construction works and potentially look at some options in which it may be, can be improved a little bit.

Lera: Yeah. Rollups essentially allow execution to scale for Ethereum, but the question is, How do you make the data scale? And danksharding/data availability sampling basically adds this missing piece to Ethereum that will allow it to achieve full scaling. This post is kind of quite technical. Some aspects we don’t explore, like networking, which is also interesting to research and to write about, but we are mainly focusing on the cryptographic aspects of danksharding.

Dan: And I would actually even add that there’s really beautiful open questions that are still left to think about for researchers if you are interested in this area. There are beautiful questions around coding and polynomial commitments. There are really quite interesting questions. If they could be resolved, we would end up with even more efficient systems.

Robert: That’s excellent context. I definitely want to ask you about those open areas for potential inquiry in a little bit. But before we do that, let’s talk more about this proposal that would free up space on the Ethereum blockchain. So the blockchain is basically this giant record of transactions happening from the time that this system launched way back when to the Genesis block. How are developers thinking about freeing up space, making it get greater throughput, scalability, cheaper transactions, all of these things that sound great and would make the system much more usable. How do you actually get there? What is required to get these efficiency gains? 

Lera: I think the main challenge is that you cannot just ask validators to store more data. It won’t get you too far. If you want to scale the blockchain to increase the block size by several orders of magnitude, you have to split your block and disperse it to validators so that each validator only stores some fragment of the block. There’s where the ideas from error correcting and erasure coding comes in that allows you to do that.

So basically increasing the block size without putting too much burden on the validators. I would say that’s the main technical difficulty. That’s what danksharding is solving.

Robert: You mentioned erasure coding, and it sounds like that’s a key piece of this technology that enables it to work. Maybe you could provide some more detail there. What is erasure coding? How does it work, and how does it apply in this context? 

Lera: Sure, absolutely. So you basically take a block with users’ data and you expand it. You erasure code it, turn a smaller block into a larger block, and this larger block can tolerate some omissions from it. So you can lose some portion of the block.

In the case of danksharding, you can lose 25% of the block and still be able to reconstruct these missing pieces from what you have. So when you disperse this expanded block to the validators – and the validators go down because some of them are byzantine or faulty – you can still reconstruct from those validators and that’s okay if they go down and they lose their pieces, the rest of the validators who are still online and still honest can recover those missing pieces. And that’s why the properties of erasure coding are useful here, just to substitute for byzantine or faulty validators. 

Dan: And maybe we can even explain it a bit more with an analogy. It’s like, you know, if you’re watching a Netflix movie and say, you know, the Netflix servers are sending packets to your computer and you are watching the movie, well imagine 10% of the packets actually don’t make it through and your computer only gets to see 90% of the packets.

So, normally you would start to see all sorts of lossy video and degradation in performance. With erasure coding, what happens is if the movie is coded using an erasure code, even if only 90% of the packets get through the laptop has enough information to actually reconstruct the entire movie.

What’s interesting is erasure coding is used everywhere, like communication networks wouldn’t be able to work without erasure coding. And maybe again, just another example, when you have a deep space probe, it’s sending messages back to earth. There’s a lot of noise and a lot of them get either dropped or they get garbled, and yet on Earth we’re able to recover the signal and get those crisp images from Mars.

That actually also is done using a slightly stronger technique called error correcting codes, where we’re not only losing packets, but also we are recovering from packets being distorted, being changed in value. What’s interesting in the context of the blockchain is that all the data is being signed. So we actually don’t care so much about data corruption because the signature layer will detect data corruption.

So really all we care about, the only thing that a malicious node can do – someone who’s trying to prevent the data from being reconstructed – the only thing that that node can do is, in some sense, remove pieces that make up the data. So we don’t care so much about recovering from corruption of the data because that’s taken care of by the signature.

But we do worry quite a lot about pieces of the data simply missing, and as a result, maybe whoever’s trying to reconstruct the data is not able to do it. And so that’s exactly where erasure coding comes in, where we know that the only thing that can happen is that a certain piece of the data is missing. It can’t be garbled. So if we got it, we got the right one, and that’s because of the signature. But if it’s missing, we have to somehow recover. And that’s exactly as Lera was saying, that’s the idea of erasure coding. 

The way you do that is basically you take your original data – in the case of Ethereum, you would take a block and you would expand it a little bit.

Actually, you expand it by like a factor of four to get more data in the block, so that the data is now redundant in the block, and now you break it into little pieces. Now you can ask every validator, “Oh, you know, you don’t have to store the entire block – you only have to store this small piece of the block.”

And if enough validators do their job and store those little pieces – and when we need to reconstruct the block, they send those pieces back to us – if enough pieces get back, then we are able to reconstruct the entire block. In danksharding in particular – again, it’s a beautiful, beautiful proposal – the recovery rate is 75%. So if 75% of the validators respond and we’re able to recover 75% of the pieces, then we’re able to reconstruct the entire block. That’s kind of the core mechanism that’s used in danksharding.

Robert: That’s really useful. So it sounds like erasure coding is this kind of technique that enables you to apply some redundancy and backups so that you don’t lose all of the data so that you can still sort of assemble it, get access to it, see that it exists.

Dan, you mentioned that the reason for doing this data availability sampling is to prevent bad actors from doing certain things, like getting rid of some of the data. What exactly are we defending against here?

Dan: Yeah, that’s a great place to go next. So in fact, what happens is, with these proposals for solving the data problem on Ethereum, there’s gonna be a new transaction type that’s going to be introduced.

This is called a “blob-carrying transaction”, and what it would allow folks to do is basically embed blobs. Each blob is 128 kilobytes. So you embed blobs into blocks. So normally blocks are made up of transactions to do all sorts of things to the Ethereum state. So now, in addition to just the regular transactions that we know and love, there’s also going to be a blob-carrying transaction or a few blob-carrying transactions per block and as I said, each one is gonna be 128 kilobytes.

Now the long-term plan, and actually maybe Lera can talk about the transition to how we get there, but the long-term plan is that there could be quite a few data-carrying blobs in every block, which would actually make the block quite large, right? I mean, each blob is 128 kilobytes.

If you put a bunch of those together, you could end up, actually, you will end up with blocks that are 30 megabytes – and it’s just not reasonable to ask validators to store these huge blocks. Today, the blocks are only a hundred kilobytes or so, so these would be much, much larger blocks. And so the idea is to basically break up these large blocks into little pieces.

Every validator, every node will actually store only this little piece. And now the problem is, what happens if they say that they stored the piece, but in fact they didn’t? Right? What do we do then? And that’s exactly where data availability sampling comes in, which is a very efficient way to test at block creation time that, in fact, everybody received their pieces, everybody currently has their pieces, and currently the block can be reconstructed even though it was broken into many, many small pieces. And these pieces are stored, distributed across the network.

Robert: And just before Lera takes over, I just want to make sure that I understand something here. So you said blocks today are about 100 kilobytes. And the idea is that, after all these upgrades, they’re going to be about 30 megabytes. And part of the reason for that expansion of the block size is to accommodate this new type of data, this blob data, that has a different purpose than what you usually shove into blocks, which is just pure transaction data.

And this blob data is really related to helping these off-chain Layer 2 networks store some data in a more ephemeral manner. Did I get that right?

Dan: Yeah, I’m glad you reiterated that cause that’s important. So these blobs basically are gonna be used by these rollups, right? So the rollups have to store the rollup data.

And so instead today what they do is they store it as what’s called “call data”, which is somewhat expensive and not what call data was meant for. So instead, they’re gonna store this data as blobs in blocks. And what’s interesting is these blobs are actually not going to be available to the execution layer of Ethereum.

They’re just gonna be stored as blobs in blocks. The execution layer will only see hashes of these big blobs. They won’t be able to access individual bytes or elements in the blobs, which is the new mechanism. So today, the way this is stored, is, as we said, in call data, and call data, is all available to the execution layer of Ethereum.

So because of this simplification, storing blob data is going to be a lot cheaper than [storing] call data. So in principle, the goal of this is to reduce the costs of Layer 2 systems, right? Because today they have to pay quite a lot to store all their data as call data. In the future, once danksharding is deployed, or even once protodanksharding is deployed, the costs will be a lot lower.

So L2 systems will become much, much cheaper and easier to use.

Robert: That’s great. I love that we’re having a sophisticated conversation about cryptography and very technical software, and yet we keep using the word “blob”. It reminds me of the ‘80s sci-fi movie Attack of the Blob. But Lera, yeah, maybe you could chime in now and talk about this trend and how we get from where we are now to that vision in the future of expanded block sizes.

Lera: Yeah, for sure. Before I dive into that, just to add two comments to what Dan said, I want to mention that it’s important the blobs are going to expire and the validators don’t give any guarantee that they’re gonna be storing these blobs forever. Right now the expiry is set roughly to 30 to 60 days, that’s what the Ethereum Foundation is thinking about.

So after this period, during which you will have an opportunity to download all of these blobs and store it locally. You know, the network will drop them, but the commitments to these blobs will persist. And if you need, so if you have the blobs themselves, you can always resupply them to the execution layer with call data.

So as long as you store the blobs, you can prove that those are the correct blobs that you have by resupplying them, because the chain continues storing the hashes of those blobs, the commitments. I also want to mention another important thing is that the fee market for those blobs is gonna be different.

So there’s going to be another fee market. They’re gonna be priced a little differently. So Ethereum is gonna have these two pipes. One pipe, if it gets congested, you are paying larger fees, say for execution, and if a data pipe gets congested, you’re paying larger fees for storing the data. So we don’t yet know how expensive the storage is gonna be. The intuition is that it must be less expensive than call data today. But again, we have to experiment to see exactly how much cheaper it’s gonna be. And protodanksharding is actually, it’s a step towards full danksharding but it’s like an experiment that we are gonna carry out to see how validators are gonna handle this additional load and how expensive the fees are gonna be for storing those data blobs.

So on the way to danksharding, we’re gonna do this experiment with protodanksharding. In protodanksharding basically, you don’t apply any erasure coding or error correcting. All you do is you add this special transaction type that carries data blobs. And those blobs are gonna have an expiry and that’s it.

So the block size is gonna be increased by a little bit in Ethereum. So right now, as Dan was saying, it’s around 100 kilobytes. With protodanksharding it’s gonna be around 500 kilobytes or so. So it’s not that big of an increase, but we’re still gonna test like all of the hypotheses that hopefully will check out and Ethereum will continue moving to full danksharding.

Robert: So Lera, you said a number of things in there. I wanna make sure that all those points came across. You mentioned that the blob data is gonna have a different fee market. So is the right way to think about this, like you have different highway systems, or perhaps you have a lane on a highway where like you have an E-ZPass or something and maybe it’s cheaper for someone in this HOV, E-ZPass-style lane? You get lower fees to put this kind of data on the blockchain versus someone who’s just a regular commuter having to pay a toll. I know I’m kind of mixing some metaphors here, but I’m wondering if that’s a physical analogy to describe these differences in fee markets for different types of data. 

Lera: Yeah, I would say that it’s hard to imagine this new data’s fees being more expensive than what we currently have, so the hypothesis is it’s always gonna be cheaper to put your data in those blobs if you don’t care about accessing this data from the execution layer. So it’s as if opening another lane on your highway, that will just increase traffic, but for certain types of transactions, you’ll go into this lane and for those transactions that are kind of data heavy and for transactions that are execution heavy, you’ll continue going through the main lanes, if that makes sense.

Robert: Yes, it does. And it sounds like one of the reasons why it can be cheaper is because it has this expiration date, as you mentioned. I think you said that the current idea is that’s gonna last 30 to 60 days for this blob data, at which point it would simply vanish and there would just be a trace remaining – a sort of commitment as you described.

Lera: Yes, exactly. 

Dan: Maybe to add to that, basically what happens is when you submit transactions, there’s a fee market, as Lera was saying, in that if there are lots of transactions being submitted all at once, say there’s a big NFT mint, and everybody wants to issue transactions, then of course the price per transaction immediately goes up.

Well, blob data is gonna be a parallel fee market, so presumably there are fewer people submitting blobs than there are people submitting transactions. So hopefully there will be less congestion over blobs. But in principle, it could happen. Hopefully not, but it could happen that all of a sudden, for some reason, there’s huge congestion over blobs and then the fee market for blobs will actually go up.

But in principle, again, because there’s less demand for submitting blobs than there is for submitting transactions, the hope is that the cost for blobs will be lower than the cost for transactions. 

Robert: Okay. That’s really helpful to understand as well. So we’ve laid out a sort of timeline for these updates. Perhaps, people listening in, maybe you’re familiar with “The Merge”, which happened last year in the fall, this big Ethereum upgrade that basically eliminated the environmental impact of Ethereum in terms of its energy consumption. And now we’re entering this new period of upgrades, which Vitalik [Buterin], co-creator of Ethereum, has termed “The Surge”, meaning that all of a sudden, these updates are going to enable the blockchain to scale a lot more powerfully than it’s been able to before. So part of this update, one of the first ones, is protodanksharding. It’s happening later this year. 

What happens in between that upgrade and full on danksharding? What are the differences between the two and when do we get that fully formed vision of danksharding in the future?

Lera: That’s a great question. I think there are still some research problems to figure out along the way, especially around networking because when those validators are storing only fragments of the blob, they need to help each other reconstruct those fragments. If some validator falls asleep for a while, when it wakes up, it wants other validators to help it reconstruct its missing fragments.

So it’s quite an involved networking protocol that is still in the making for danksharding and there are other exciting research problems that potentially can improve the scheme, but I think so far it looks like quite a clear path to get from protodanksharding to danksharding. And the question is just maybe how to make it better, how to improve different aspects of it, make it more efficient.

Dan: Maybe it’s worthwhile adding that in the protodanksharding approach there are at most four blobs per block, which is why each blob is 128 kilobytes. Four times 128 gives us half a megabyte, which is why that’s the limit on block sizes in protodanksharding, and that’s actually imminent. That’s supposed to happen later this year.

And then, yeah, going all the way to danksharding still takes some work. In fact, there was a very big event recently with generating parameters jointly for danksharding. And so there’s a lot of work to do in preparation of getting to full danksharding.

Lera: Yeah, that’s quite exciting. I think they’re still accepting contributions to participate in the trusted setup ceremony. Yeah, so it’s still ongoing. It’s a large community effort that’s really fun to watch and people come up with lots of creative ideas to contribute. So check this out, definitely.

Robert: That’s super cool. I actually participated in some of the trusted setup ceremonies for some of the early privacy coins. Is this trusted setup ceremony as ostentatious as some of those, where you had people burning laptops and exploding hard drives and things like that?

Lera: From what I’ve seen, it’s just people coming up with different creative ways to generate entropy. Some use their pets, dogs and cats. Some are creating some sophisticated marble run machines and such. There was even one contribution done from a satellite, the highest altitude contribution there.

Dan: Actually we have to mention that. It was pretty cool actually. This company Cryptosat actually has satellites in orbit that sample noise up in space and then contribute and participate in the trusted setup protocol.

So that was pretty cool to see. 

Robert: Wow, that is awesome. Did not know there was crypto in space just yet. Dan, you said that protodanksharding is gonna have four blobs per block. What is danksharding gonna enable? How many blobs are we talking here? 

Dan: Yeah, so by the way, protodanksharding is up to four blobs per block.

They’re actually targeting two, but up to four. And then danksharding, as we said, they’re targeting at most 30 megabyte blocks. So just divide 30 megabytes by 128 kilobytes. And that tells you it’s on the order of a hundred, I guess it’s about a hundred blobs per block.

Lera: Yeah, I think the current plan is 128 blobs target and maybe up to 256 blobs per one block.

Robert: Great. 

Dan: And that’s exactly when the blocks become quite large. And then it becomes a little difficult for the validators to keep the entire blocks themselves. And that’s where we have to start breaking them up into pieces. And each validator will store one piece. 

Robert: Got it. I appreciate the basic division. Maybe some of the more technical math that you get into in your post would be a little bit harder to convey here, but that makes sense. Maybe we could talk a little bit about some of the proposals that you made in your recent post. So you did this research and you found that with some adjustments, you could potentially get even more efficiency out of EIP-4844, which is the more technical name for protodanksharding.

Lera: Yeah, the bulk of the work was just to understand the danksharding proposal in full, and then we observed some different ways to look into the mathematics – the cryptographic components of it – that hopefully unravel new toolkits that we can use there. So the rough idea, not going too much in depth, is you fit a polynomial through your block.

It’s a bivariate polynomial because your block is rectangular. And then you evaluate this polynomial at more points, kind of expanding the block. And the idea is that if you’ve used these bivariate polynomials – instead of as danksharding was doing it, as a list of multivariate polynomials – you can then apply techniques for bivariate evaluation, bivariate interpolation, and possibly even try to apply bivariate error-correcting codes to that. But it’s very much an open door for more research and exploration. So in this post we try to explain where more research can be done to improve the scheme in that direction.

Dan: Yeah. Well maybe I can add to that. I mean, danksharding is such a beautiful idea. Really. The Ethereum Foundation deserves a huge amount of credit for it. And, you know, Dankrad [Feist] in particular. It’s really quite an elegant construction.

Initially, we were just trying to understand the details and it took some time to recover exactly all the details of how everything works. And we figured maybe it’ll help the world to have another writeup that explains how the mechanism works. Initially, I guess the original danksharding, the way it’s described, is all using univariate polynomial commitments, where we take each row, we view it as a polynomial. Erasure coding is all done using polynomials.

Maybe I can even teach a little bit of erasure coding here in one sentence. Suppose you have two points in a plane and you want to do erasure coding on them. What you can do is you can just pass a line through those two points, and now you can just publish, instead of just those two points, you can publish those two points plus maybe two additional points on the line.

So now you have four points total. And you know that if two out of those four points make it to the receiver, the receiver can use the two points that it received to recover the line and then recover the original two points. That’s the whole idea of erasure coding. So of course, instead of lines, we use higher degree polynomials to achieve different thresholds, but that’s the idea.

Basically, we have two points. We pass a line. We get more points on the line. If only two points of the line make it to the receiver, the receiver can reconstruct the line and recover the original points. And so danksharding actually does this really, really quite elegantly by looking at the block as a matrix, as a rectangular set of data.

And then it basically extends, using these line ideas, both horizontally and vertically. And that actually gives the coded block – where then, pieces of that rectangle are sent to the different validators. What’s interesting is now that you have a rectangle, you can think of it as a two-dimensional object. That very naturally leads to thinking about this as a bivariate polynomial, exactly as Lera was saying.

What danksharding does is it gives a very interesting way to commit to these bivariate polynomials. So it builds a way to commit to bivariate polynomials by using commitments to univariate polynomials. And so it turns out that the reconstruction mechanism that was done in danksharding is also based on construction along lines and columns – also construction as done using univariate polynomials. And then as we were working through this, there was this realization that, hey, everything here really is about rectangles and bivariate polynomials. Maybe there’s a way in which the reconstruction can also be done by using interpolation of bivariate polynomials. 

Lera: So in fact, you take your block and you expand it to X by the factor of two in both directions. So you have 4X more points as a result. But those 4X points only encode a small quadrant.

So in principle, you only need one quadrant in order to interpolate and recover all the rest of this encoded block. So 25% of the points should be enough. But as danksharding works by doing univariate interpolations, it needs 75% of the block to do column and row-wise reconstruction.

If you do bivariate interpolation directly, you should be good with just 25% instead of 75%. So that will improve the number of elements you need in order to reconstruct, to recover the block. It will also improve communication and data availability sampling, but it’s all due to improved reconstruction.

And now the question, it becomes kind of a mathematical question of how do you do efficient bivariate interpolation? 

There is an obvious need for a better algorithm there. And we were researching this a little bit and it appears so far to be a little bit underexplored. So maybe there were no applications for bivariate interpolation before. Maybe it’s just a hard problem, or we don’t know. But that’s definitely an interesting direction to basically try and improve bilinear interpolation algorithms.

Dan: So what I love about this is just like Lera was saying, for the more algorithms folks in the audience, is that there’s been a ton of work on doing univariate interpolation. If I give you points on a univariate polynomial, like points on a line, and I ask you to reconstruct that polynomial, there are very good algorithms for univariate polynomial interpolation.

And it turns out the bivariate interpolation problem, somehow it seems like it received less attention. And what’s really cool here is all of a sudden, the blockchain, Ethereum, danksharding is creating an application for this really natural algorithmic problem of bivariate polynomial interpolation. 

We really need it here. If we had a good algorithm for bivariate polynomial interpolation, we could make danksharding better because reconstruction, as Lera was saying, would go down from 75% to 25%. So to me, this is really quite beautiful in that the blockchain, Ethereum, danksharding is creating this new area of research, or at least prioritizing this new area of research showing, we really need better algorithms, new algorithms, efficient algorithms to do bivariate polynomial interpolation.

So hopefully that will encourage and spur a lot more research on these types of algorithms and presumably they will be very useful for this problem. 

Robert: So I love that we dug into the methodology here and didn’t shy away from the mathematics. I know some of it might sound a little complicated – bivariate polynomials and univariate polynomials. But I especially appreciate how you described these in terms of geometry and shapes, cause I think everybody here can really picture a line going through some points or how a rectangle functions. So I think that really helps ground the kind of work that you’re doing.

I want to double click on this statistic that you mentioned where the current proposal as it exists would require 75% of samples in order to be reconstructed, whereas what you’re proposing would trim that down to 25%. So that’s a giant difference. 75% to 25%. But it also sounds like just for a casual observer, if you’re only having 25%, the fact that it’s just less than 50%, it sounds like, is that really enough to make assurances that this data is available and was made available?

When you go down to 25% it sounds like, I don’t know, you might be cutting some corners. So how do you assure people that in fact, just having 25% of data samples is actually enough and that things can work at that level? 

Lera: Yeah, that gets us to the topic of data availability sampling, and what it achieves, I guess, because this reconstruction threshold – 75% or 25% – basically determines how many samples you need to get high assurance that the data is there.

The way you do the sample is that you ask the network of validators to give you back one element, a random element of this encoded block. And if you get it back successfully – and you can verify also once the validators give you back the proof of the validity of this piece, and you can verify it against the commitments that the chain persistently stores – so when you get back the successful sample that makes you sure that the data is available with probability that is one minus either one quarter or three quarters, depending on how your reconstruction algorithm works, whether it requires 25% of the data or 75% of the data. 

So every time you do a random sample and it comes back successfully, basically your false positive [rate] – the probability that you think the data is available when it’s not – drops down exponentially. And the rate at which it drops down depends on how much data you are required in the reconstruction. So if your reconstruction requests 25% of the data, you do less samples and your assurance – the false positive rate – goes down quicker than if you only have a reconstruction algorithm that requires 75% of the data. 

So depending on how efficient your reconstruction is, you might need fewer samples in order to have the same assurance that the data is available. So that’s why you not only improve the reconstruction here, but you also improve the number of samples you need to do for your data availability sampling.

And data availability sampling is interesting because it’s probabilistic. So the more samples you do, the higher your assurance is that the data is available, right? And you can always amplify this probability by doing more samples to make it clear. 

Dan: I think Lera, what you just explained is really, really important.

That’s the heart of danksharding and data availability sampling. So I’ll say it one more time, just so that the audience will hear it twice, cause that’s really the heart of it. So maybe think of the block. We said the block is gonna be encoded as this rectangle, right? So somehow we go from a block to a rectangle.

The specifics of how that is done is using this erasure coding. But let’s pretend we went from a block of data to a rectangle. So now imagine this rectangle is literally a rectangle of dots. Every dot corresponds to one piece of the data that’s gonna be distributed to one validator. So now to do data availability sampling, somebody wants to verify that enough of the dots are actually available.

We know that if more than 75% of the dots are available, then the block can be reconstructed using the erasure coding method. Or maybe, if what we’re saying would be used, then only 25% of the dots are sufficient to reconstruct the original rectangle. But how do you know that 75% of the dots are available?

So that’s exactly the data availability sampling mechanism. What you do is you can kind of imagine you are throwing darts at this rectangle, right? So every time you throw a dart, you hit a random point in the rectangle and then the validator that holds that point has to prove, “Yes, I really have that point.”

Now you wanna verify that 75% of the dots are available. So imagine you have this rectangle. Maybe only 75% of the dots are there. Some of the dots disappeared for some reason. You wanna verify that 75% of the dots are available, cause if 75% are available, you can reconstruct the entire rectangle. And so what are you gonna do to verify that 75% are there?

You’re gonna throw a bunch of darts at the rectangle. And for every time the dart hits a dot, the validator that it hits has to prove that the dot really is there. And so if you throw a hundred darts and all hundred darts come back saying, “Yes, the data really is there”, that gives you a pretty good idea that more than 75% of the data is available.

Because you know, if less than 75% is available and you throw four darts, you expect one dart to hit a missing dot. If you throw a hundred darts, and all of them come back saying the data’s available, you have pretty good assurance that more than 75% of the dots are there. And so that’s the idea of data availability sampling.

You just try lots and lots and lots of random points, like a hundred of them. If all of them are there, then you have pretty good assurance that more than 75% are available and you can reconstruct the data. And you see, if you want to get 75% assurance, you need to throw a hundred darts. If you need only 25% assurance, you would need to throw fewer darts than that.

So that would basically reduce the amount of data that’s needed to satisfy the sampling mechanism. Now, maybe it’s worthwhile saying that once the data availability sampling check succeeds – so all the hundred darts come back saying, “Yes, the dots really are there” – then the validator says, “Ah, the data’s available”.

And then [the validator] goes ahead and signs the block as saying, “This block passed data availability sampling”. Yeah, and that’s used later on in consensus. So that’s what the test is. Basically, data availability sampling is a very efficient way to test that enough data is available to reconstruct the block without actually reconstructing the block completely.

So I think it’s good to hear it twice and maybe even a third and a fourth time. But that’s kind of the core idea that makes this all work. 

Robert: I love that, and I especially love the physical analogies that you’re using with a dart board and throwing darts. I think that really brings it home for people. So we’re nearing the top of the hour here.

I wanna get ready to close out. But before we do that, just maybe I’ll throw it to you both in one sentence. What is the upshot of all of this? Like why does it matter to get these efficiency gains? 

Lera: Well, I would say the ultimate goal, of course, is to scale the blockchain and these new techniques would allow it to do that, for Ethereum to achieve full scaling.

That’s a really interesting approach, I would say, because in the beginning, Ethereum was thinking about doing sharding, full sharding, and arriving at quite complicated designs. But having rollups helps scale the execution layer of Ethereum, and this leaves it up to Ethereum to scale its data availability layer. Basically, increase the space while rollups increase the execution capacity and those pieces together will give us cheaper and faster blockchains.

Dan: Yeah, Lera said it perfectly. I mean, really the scaling story for Ethereum is rollups and the way to get rollups to be more efficient and cheaper to use is by solving the data problem. And danksharding is a very elegant and efficient way to do that. So the upshot is a scalable version of Ethereum where rollups are much cheaper to use than they are today.

Robert: That’s great. And if you get cheaper transactions and the ability to make more transactions, I think that opens up Ethereum to do all sorts of new applications that weren’t possible before with high gas costs and fees. So this has been great. Thank you all for joining us. I hope that you all learned a little bit about data availability sampling and danksharding.

If you’d like to learn more, as mentioned, you can check out Dan and Lera’s post. It’s a really, really great piece, so I highly recommend reading it and checking out all the resources contained in there.

Thank you all for joining us. I’m looking forward to this weekend. I’m gonna go to my local bar and teach everybody about erasure coding at the dartboard. So thank you all and take care.

Lera: Sounds great. Thank you.

Dan: Thank you. This has been fun. Bye bye.

Thank you for listening to web3 with a16z. You can find show notes with links to resources, books, or papers discussed; transcripts, and more at a16zcrypto.com. This episode was technically edited by our audio editor Justin Golden. Credit also to Moonshot Design for the art. And all thanks to support from a16z crypto.

To follow more of our work and get updates, resources from us, and from others, be sure to subscribe to our web3 weekly newsletter — you can find it on our website at a16zcrypto.com.

 Thank you for listening, and for subscribing. Let’s go!

In a “vampire attack,” a platform tries to directly incentivize its competitor’s customers to switch. This isn’t a new idea — airlines have had “status match” for ages — but it’s much easier in the world of crypto, where transactions are stored publicly on blockchains. With public transaction histories, a new entrant can just simply read its competitors’ transaction records and provide direct rewards or other incentives to top customers who switch.

What does this mean for consumers? The short version is that vampire attacks can result in virtuous platform competition that drives down prices. In short, “blood-sucking platform competition” can be good for consumers.

To arrive at this conclusion, in a new paper, we built a simple model in which platforms have some customers who are “captive,” and others who are mobile across platforms — and price-sensitive. In the real world, customers might be captive because of, for instance, search costs or lack of information about alternatives.

When platforms are only able to offer a single price to both types of consumers, the price curve declines in the share of the market that’s mobile, denoted by λ in the figure below. This makes sense, since the more of those consumers there are, the more firms want to compete for them.

Next, we consider what happens when firms introduce loyalty programs as a way of convincing mobile customers to stay put. Consumers who are part of a platform’s loyalty program receive a discount at that platform but have to pay the standard price at a competitor’s.

It turns out that loyalty programs are actually bad for all consumers — even the ones who get the discount. Why? First, firms now charge their captive consumers more, since they can protect their mobile consumers with a loyalty program. The resulting captive consumer price is the dashed green line in the figure below.

But if the platforms are monopolizing their captive customers, that means they have to drop their price a lot — and take a big loss — if they want to attract their competitors’ loyal customers… which reduces the likelihood that a platform will try to steal its competitor’s loyal customers. As a result, firms can charge even their loyal customers more; this is the solid green line in the figure below.

But when each firm can see who its competitors’ loyal customers are, vampire attacks are possible — that is, each firm can offer a competitor’s loyal customers the same price it’s giving its own loyal customers. This in turn leads to much more competition over those customers, driving the price they face down far below even the price before loyalty programs were introduced. This is the solid red line in the figure below.

In short: Vampire attacks restore competition for loyal customers.

This theory broadly accords with what we’ve seen among NFT trading platforms, with vampire attacks leading to intense competition for customer loyalty. A caveat, however, is that it’s not clear how sustainable the current vampire attack models are, especially since the legality and regulatory statuses of the strategies that many of them rely on are uncertain, and the relative success of such attacks may be different in scenarios where regulatory clarity and consistent enforcement exists.

Moreover, as Shai Bernstein and one of us (Kominers) discussed in a recent Harvard Business Review article, the competition induced by vampire attacks can only benefit consumers if it doesn’t incentivize firms to engage in malbehavior (such as with centralized crypto finance platforms taking on excessive risk in order to offer higher rates of return). 

That said, overall, our analysis suggests that vampire attacks may contribute to increased — and hopefully virtuous — platform competition. This has the potential to reshape the digital platform landscape in a way that’s better for everyone.

*   *   *

P.S. Any ideas for alternate names for the “vampire attack” concept? The name is evocative, but doesn’t on its face sound like something you’d necessarily want to help enable. 

P.P.S. There is in fact an economics literature on that other type of vampire attack.

This article is an adaptation of a recent Twitter thread summarizing our new economic theory paper, “A Simple Theory of Vampire Attacks.”

John William Hatfield is the Arthur Andersen & Co. Alumni Centennial Professor in Finance at the McCombs School of Business at the University of Texas at Austin. 

Scott Duke Kominers is a Professor of Business Administration at Harvard Business School, a Faculty Affiliate of the Harvard Department of Economics, and a Research Partner a16z crypto. He also advises a number of companies on marketplace and incentive design; for further disclosures, see his website.

Special thanks to our editor, Tim Sullivan, as well as Shai Bernstein, Josh Bobrowsky, Christian Catalini, Sonal Chokshi, Lauren Cohen, Chris Dixon, Piotr Dworczak, Jad Esber, Joshua Gans, Zachary Gray, Hanna Halaburda, Mason Hall, Miles Jennings, Michele Korver, Sriram Krishnan, Eddy Lazzrin, Collin McCune, Kim Milosevich, Vivek Ravishanker, Natalia Rigol, April Roth, Ben Roth, Tim Roughgarden, Bryan Routledge, Scott Walker, Carra Wu, Ali Yahya, the Lab for Economic Design, and presentation audiences at the American Economic Association meetings, Harvard Business School, and a16z crypto for helpful comments.

—

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not necessarily the views of a16z or its affiliates. a16z is an investment adviser registered with the U.S. Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party information; a16z has not reviewed such material and does not endorse any advertising content contained therein.

This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities, digital assets, investment strategies or techniques are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments or investment strategies will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.

Additionally, this material is provided for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. Investing in pooled investment vehicles and/or digital assets includes many risks not fully discussed herein, including but not limited to, significant volatility, liquidity, technological, and regulatory risks. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.

Robert: Hi everyone, Robert Hackett back here again with another episode for web3 with a16z. I recently chatted live with some of our researchers on the topic of Data Availability Sampling and Danksharding – which is relevant to blockchain scaling, as well as paving the way for more advanced blockchain networks & user applications. While much of the discussion is especially relevant to Ethereum, some of the concepts we cover are also applicable to advances in computing and networking generally. 

This discussion involves some specific math, which we cover (and briefly explain) in the episode.

For quick context as you listen, “polynomial commitments”, which you’ll hear more about, are a tool that helps reduce the amount of data needed to verify complex computations. And “interpolation”, another term you’ll be hearing, is a way to reconstruct data from a limited set of data points.

Be sure to check out the paper referenced in this episode for a deeper explanation at a16zcrypto.com/das – that’s DAS for data availability sampling. As always, none of the following is investment, business, legal or tax advice. See a16z.com/disclosures for more important information, including a link to a list of our investments.

Robert: Okay, so today we’re going to be talking about data availability sampling and danksharding. Now, if you’re unfamiliar with those things, don’t be scared because I’m here with a couple of experts who are gonna break things down for you. We’ve got Dan Boneh, Stanford professor, eminent cryptographer, and a16z crypto research advisor, and Lera Nikolaenko, research partner at a16z crypto.

Lera: Great to be here. Thanks, Robert. 

Dan: Thanks, Robert. Great to be here. 

Robert: Thanks, Lera. Thanks, Dan. So, as mentioned, Dan and Lera recently wrote up a great post and have a proposal related to ways to improve protodanksharding, which is this upgrade that is planned for later this year in Ethereum. There is a lot of rich detail in that post, so we’ll dig in there.

But before we get into all the details, I thought maybe we could zoom out and back up and start with a bigger picture here. So let’s start with Dan. Help us understand the subject of today’s talk, data availability sampling. And maybe you could do it without breaking anyone’s minds here.

Maybe we could keep it as you know, succinct and accessible to a broader audience as you might be able to. 

Dan: Sure, so maybe we can zoom out a little bit before we talk about data availability sampling. Maybe let’s talk about the general goal here. So this is part of the efforts to scale Ethereum.

And so one of the ways to scale Ethereum is using rollups. And rollups actually need to push a lot of data on chain. So a rollup allows you to take, say, a hundred or a thousand transactions and process them all as a single transaction on the Ethereum Layer 1. And then all the data associated with those transactions basically gets pushed on chain.

And as a result, there’s a need to actually store quite a lot of data on chain. And the question is how to do that. And that’s exactly where danksharding comes into play. So it’s a really beautiful, beautiful idea. It came from the Ethereum Foundation, particularly from Dankrad Feist. It’s a really elegant construction and basically Lera and I wanted to give an overview of how this construction works and potentially look at some options in which it may be, can be improved a little bit.

Lera: Yeah. Rollups essentially allow execution to scale for Ethereum, but the question is, How do you make the data scale? And danksharding/data availability sampling basically adds this missing piece to Ethereum that will allow it to achieve full scaling. This post is kind of quite technical. Some aspects we don’t explore, like networking, which is also interesting to research and to write about, but we are mainly focusing on the cryptographic aspects of danksharding.

Dan: And I would actually even add that there’s really beautiful open questions that are still left to think about for researchers if you are interested in this area. There are beautiful questions around coding and polynomial commitments. There are really quite interesting questions. If they could be resolved, we would end up with even more efficient systems.

Robert: That’s excellent context. I definitely want to ask you about those open areas for potential inquiry in a little bit. But before we do that, let’s talk more about this proposal that would free up space on the Ethereum blockchain. So the blockchain is basically this giant record of transactions happening from the time that this system launched way back when to the Genesis block. How are developers thinking about freeing up space, making it get greater throughput, scalability, cheaper transactions, all of these things that sound great and would make the system much more usable. How do you actually get there? What is required to get these efficiency gains? 

Lera: I think the main challenge is that you cannot just ask validators to store more data. It won’t get you too far. If you want to scale the blockchain to increase the block size by several orders of magnitude, you have to split your block and disperse it to validators so that each validator only stores some fragment of the block. There’s where the ideas from error correcting and erasure coding comes in that allows you to do that.

So basically increasing the block size without putting too much burden on the validators. I would say that’s the main technical difficulty. That’s what danksharding is solving.

Robert: You mentioned erasure coding, and it sounds like that’s a key piece of this technology that enables it to work. Maybe you could provide some more detail there. What is erasure coding? How does it work, and how does it apply in this context? 

Lera: Sure, absolutely. So you basically take a block with users’ data and you expand it. You erasure code it, turn a smaller block into a larger block, and this larger block can tolerate some omissions from it. So you can lose some portion of the block.

In the case of danksharding, you can lose 25% of the block and still be able to reconstruct these missing pieces from what you have. So when you disperse this expanded block to the validators – and the validators go down because some of them are byzantine or faulty – you can still reconstruct from those validators and that’s okay if they go down and they lose their pieces, the rest of the validators who are still online and still honest can recover those missing pieces. And that’s why the properties of erasure coding are useful here, just to substitute for byzantine or faulty validators. 

Dan: And maybe we can even explain it a bit more with an analogy. It’s like, you know, if you’re watching a Netflix movie and say, you know, the Netflix servers are sending packets to your computer and you are watching the movie, well imagine 10% of the packets actually don’t make it through and your computer only gets to see 90% of the packets.

So, normally you would start to see all sorts of lossy video and degradation in performance. With erasure coding, what happens is if the movie is coded using an erasure code, even if only 90% of the packets get through the laptop has enough information to actually reconstruct the entire movie.

What’s interesting is erasure coding is used everywhere, like communication networks wouldn’t be able to work without erasure coding. And maybe again, just another example, when you have a deep space probe, it’s sending messages back to earth. There’s a lot of noise and a lot of them get either dropped or they get garbled, and yet on Earth we’re able to recover the signal and get those crisp images from Mars.

That actually also is done using a slightly stronger technique called error correcting codes, where we’re not only losing packets, but also we are recovering from packets being distorted, being changed in value. What’s interesting in the context of the blockchain is that all the data is being signed. So we actually don’t care so much about data corruption because the signature layer will detect data corruption.

So really all we care about, the only thing that a malicious node can do – someone who’s trying to prevent the data from being reconstructed – the only thing that that node can do is, in some sense, remove pieces that make up the data. So we don’t care so much about recovering from corruption of the data because that’s taken care of by the signature.

But we do worry quite a lot about pieces of the data simply missing, and as a result, maybe whoever’s trying to reconstruct the data is not able to do it. And so that’s exactly where erasure coding comes in, where we know that the only thing that can happen is that a certain piece of the data is missing. It can’t be garbled. So if we got it, we got the right one, and that’s because of the signature. But if it’s missing, we have to somehow recover. And that’s exactly as Lera was saying, that’s the idea of erasure coding. 

The way you do that is basically you take your original data – in the case of Ethereum, you would take a block and you would expand it a little bit.

Actually, you expand it by like a factor of four to get more data in the block, so that the data is now redundant in the block, and now you break it into little pieces. Now you can ask every validator, “Oh, you know, you don’t have to store the entire block – you only have to store this small piece of the block.”

And if enough validators do their job and store those little pieces – and when we need to reconstruct the block, they send those pieces back to us – if enough pieces get back, then we are able to reconstruct the entire block. In danksharding in particular – again, it’s a beautiful, beautiful proposal – the recovery rate is 75%. So if 75% of the validators respond and we’re able to recover 75% of the pieces, then we’re able to reconstruct the entire block. That’s kind of the core mechanism that’s used in danksharding.

Robert: That’s really useful. So it sounds like erasure coding is this kind of technique that enables you to apply some redundancy and backups so that you don’t lose all of the data so that you can still sort of assemble it, get access to it, see that it exists.

Dan, you mentioned that the reason for doing this data availability sampling is to prevent bad actors from doing certain things, like getting rid of some of the data. What exactly are we defending against here?

Dan: Yeah, that’s a great place to go next. So in fact, what happens is, with these proposals for solving the data problem on Ethereum, there’s gonna be a new transaction type that’s going to be introduced.

This is called a “blob-carrying transaction”, and what it would allow folks to do is basically embed blobs. Each blob is 128 kilobytes. So you embed blobs into blocks. So normally blocks are made up of transactions to do all sorts of things to the Ethereum state. So now, in addition to just the regular transactions that we know and love, there’s also going to be a blob-carrying transaction or a few blob-carrying transactions per block and as I said, each one is gonna be 128 kilobytes.

Now the long-term plan, and actually maybe Lera can talk about the transition to how we get there, but the long-term plan is that there could be quite a few data-carrying blobs in every block, which would actually make the block quite large, right? I mean, each blob is 128 kilobytes.

If you put a bunch of those together, you could end up, actually, you will end up with blocks that are 30 megabytes – and it’s just not reasonable to ask validators to store these huge blocks. Today, the blocks are only a hundred kilobytes or so, so these would be much, much larger blocks. And so the idea is to basically break up these large blocks into little pieces.

Every validator, every node will actually store only this little piece. And now the problem is, what happens if they say that they stored the piece, but in fact they didn’t? Right? What do we do then? And that’s exactly where data availability sampling comes in, which is a very efficient way to test at block creation time that, in fact, everybody received their pieces, everybody currently has their pieces, and currently the block can be reconstructed even though it was broken into many, many small pieces. And these pieces are stored, distributed across the network.

Robert: And just before Lera takes over, I just want to make sure that I understand something here. So you said blocks today are about 100 kilobytes. And the idea is that, after all these upgrades, they’re going to be about 30 megabytes. And part of the reason for that expansion of the block size is to accommodate this new type of data, this blob data, that has a different purpose than what you usually shove into blocks, which is just pure transaction data.

And this blob data is really related to helping these off-chain Layer 2 networks store some data in a more ephemeral manner. Did I get that right?

Dan: Yeah, I’m glad you reiterated that cause that’s important. So these blobs basically are gonna be used by these rollups, right? So the rollups have to store the rollup data.

And so instead today what they do is they store it as what’s called “call data”, which is somewhat expensive and not what call data was meant for. So instead, they’re gonna store this data as blobs in blocks. And what’s interesting is these blobs are actually not going to be available to the execution layer of Ethereum.

They’re just gonna be stored as blobs in blocks. The execution layer will only see hashes of these big blobs. They won’t be able to access individual bytes or elements in the blobs, which is the new mechanism. So today, the way this is stored, is, as we said, in call data, and call data, is all available to the execution layer of Ethereum.

So because of this simplification, storing blob data is going to be a lot cheaper than [storing] call data. So in principle, the goal of this is to reduce the costs of Layer 2 systems, right? Because today they have to pay quite a lot to store all their data as call data. In the future, once danksharding is deployed, or even once protodanksharding is deployed, the costs will be a lot lower.

So L2 systems will become much, much cheaper and easier to use.

Robert: That’s great. I love that we’re having a sophisticated conversation about cryptography and very technical software, and yet we keep using the word “blob”. It reminds me of the ‘80s sci-fi movie Attack of the Blob. But Lera, yeah, maybe you could chime in now and talk about this trend and how we get from where we are now to that vision in the future of expanded block sizes.

Lera: Yeah, for sure. Before I dive into that, just to add two comments to what Dan said, I want to mention that it’s important the blobs are going to expire and the validators don’t give any guarantee that they’re gonna be storing these blobs forever. Right now the expiry is set roughly to 30 to 60 days, that’s what the Ethereum Foundation is thinking about.

So after this period, during which you will have an opportunity to download all of these blobs and store it locally. You know, the network will drop them, but the commitments to these blobs will persist. And if you need, so if you have the blobs themselves, you can always resupply them to the execution layer with call data.

So as long as you store the blobs, you can prove that those are the correct blobs that you have by resupplying them, because the chain continues storing the hashes of those blobs, the commitments. I also want to mention another important thing is that the fee market for those blobs is gonna be different.

So there’s going to be another fee market. They’re gonna be priced a little differently. So Ethereum is gonna have these two pipes. One pipe, if it gets congested, you are paying larger fees, say for execution, and if a data pipe gets congested, you’re paying larger fees for storing the data. So we don’t yet know how expensive the storage is gonna be. The intuition is that it must be less expensive than call data today. But again, we have to experiment to see exactly how much cheaper it’s gonna be. And protodanksharding is actually, it’s a step towards full danksharding but it’s like an experiment that we are gonna carry out to see how validators are gonna handle this additional load and how expensive the fees are gonna be for storing those data blobs.

So on the way to danksharding, we’re gonna do this experiment with protodanksharding. In protodanksharding basically, you don’t apply any erasure coding or error correcting. All you do is you add this special transaction type that carries data blobs. And those blobs are gonna have an expiry and that’s it.

So the block size is gonna be increased by a little bit in Ethereum. So right now, as Dan was saying, it’s around 100 kilobytes. With protodanksharding it’s gonna be around 500 kilobytes or so. So it’s not that big of an increase, but we’re still gonna test like all of the hypotheses that hopefully will check out and Ethereum will continue moving to full danksharding.

Robert: So Lera, you said a number of things in there. I wanna make sure that all those points came across. You mentioned that the blob data is gonna have a different fee market. So is the right way to think about this, like you have different highway systems, or perhaps you have a lane on a highway where like you have an E-ZPass or something and maybe it’s cheaper for someone in this HOV, E-ZPass-style lane? You get lower fees to put this kind of data on the blockchain versus someone who’s just a regular commuter having to pay a toll. I know I’m kind of mixing some metaphors here, but I’m wondering if that’s a physical analogy to describe these differences in fee markets for different types of data. 

Lera: Yeah, I would say that it’s hard to imagine this new data’s fees being more expensive than what we currently have, so the hypothesis is it’s always gonna be cheaper to put your data in those blobs if you don’t care about accessing this data from the execution layer. So it’s as if opening another lane on your highway, that will just increase traffic, but for certain types of transactions, you’ll go into this lane and for those transactions that are kind of data heavy and for transactions that are execution heavy, you’ll continue going through the main lanes, if that makes sense.

Robert: Yes, it does. And it sounds like one of the reasons why it can be cheaper is because it has this expiration date, as you mentioned. I think you said that the current idea is that’s gonna last 30 to 60 days for this blob data, at which point it would simply vanish and there would just be a trace remaining – a sort of commitment as you described.

Lera: Yes, exactly. 

Dan: Maybe to add to that, basically what happens is when you submit transactions, there’s a fee market, as Lera was saying, in that if there are lots of transactions being submitted all at once, say there’s a big NFT mint, and everybody wants to issue transactions, then of course the price per transaction immediately goes up.

Well, blob data is gonna be a parallel fee market, so presumably there are fewer people submitting blobs than there are people submitting transactions. So hopefully there will be less congestion over blobs. But in principle, it could happen. Hopefully not, but it could happen that all of a sudden, for some reason, there’s huge congestion over blobs and then the fee market for blobs will actually go up.

But in principle, again, because there’s less demand for submitting blobs than there is for submitting transactions, the hope is that the cost for blobs will be lower than the cost for transactions. 

Robert: Okay. That’s really helpful to understand as well. So we’ve laid out a sort of timeline for these updates. Perhaps, people listening in, maybe you’re familiar with “The Merge”, which happened last year in the fall, this big Ethereum upgrade that basically eliminated the environmental impact of Ethereum in terms of its energy consumption. And now we’re entering this new period of upgrades, which Vitalik [Buterin], co-creator of Ethereum, has termed “The Surge”, meaning that all of a sudden, these updates are going to enable the blockchain to scale a lot more powerfully than it’s been able to before. So part of this update, one of the first ones, is protodanksharding. It’s happening later this year. 

What happens in between that upgrade and full on danksharding? What are the differences between the two and when do we get that fully formed vision of danksharding in the future?

Lera: That’s a great question. I think there are still some research problems to figure out along the way, especially around networking because when those validators are storing only fragments of the blob, they need to help each other reconstruct those fragments. If some validator falls asleep for a while, when it wakes up, it wants other validators to help it reconstruct its missing fragments.

So it’s quite an involved networking protocol that is still in the making for danksharding and there are other exciting research problems that potentially can improve the scheme, but I think so far it looks like quite a clear path to get from protodanksharding to danksharding. And the question is just maybe how to make it better, how to improve different aspects of it, make it more efficient.

Dan: Maybe it’s worthwhile adding that in the protodanksharding approach there are at most four blobs per block, which is why each blob is 128 kilobytes. Four times 128 gives us half a megabyte, which is why that’s the limit on block sizes in protodanksharding, and that’s actually imminent. That’s supposed to happen later this year.

And then, yeah, going all the way to danksharding still takes some work. In fact, there was a very big event recently with generating parameters jointly for danksharding. And so there’s a lot of work to do in preparation of getting to full danksharding.

Lera: Yeah, that’s quite exciting. I think they’re still accepting contributions to participate in the trusted setup ceremony. Yeah, so it’s still ongoing. It’s a large community effort that’s really fun to watch and people come up with lots of creative ideas to contribute. So check this out, definitely.

Robert: That’s super cool. I actually participated in some of the trusted setup ceremonies for some of the early privacy coins. Is this trusted setup ceremony as ostentatious as some of those, where you had people burning laptops and exploding hard drives and things like that?

Lera: From what I’ve seen, it’s just people coming up with different creative ways to generate entropy. Some use their pets, dogs and cats. Some are creating some sophisticated marble run machines and such. There was even one contribution done from a satellite, the highest altitude contribution there.

Dan: Actually we have to mention that. It was pretty cool actually. This company Cryptosat actually has satellites in orbit that sample noise up in space and then contribute and participate in the trusted setup protocol.

So that was pretty cool to see. 

Robert: Wow, that is awesome. Did not know there was crypto in space just yet. Dan, you said that protodanksharding is gonna have four blobs per block. What is danksharding gonna enable? How many blobs are we talking here? 

Dan: Yeah, so by the way, protodanksharding is up to four blobs per block.

They’re actually targeting two, but up to four. And then danksharding, as we said, they’re targeting at most 30 megabyte blocks. So just divide 30 megabytes by 128 kilobytes. And that tells you it’s on the order of a hundred, I guess it’s about a hundred blobs per block.

Lera: Yeah, I think the current plan is 128 blobs target and maybe up to 256 blobs per one block.

Robert: Great. 

Dan: And that’s exactly when the blocks become quite large. And then it becomes a little difficult for the validators to keep the entire blocks themselves. And that’s where we have to start breaking them up into pieces. And each validator will store one piece. 

Robert: Got it. I appreciate the basic division. Maybe some of the more technical math that you get into in your post would be a little bit harder to convey here, but that makes sense. Maybe we could talk a little bit about some of the proposals that you made in your recent post. So you did this research and you found that with some adjustments, you could potentially get even more efficiency out of EIP-4844, which is the more technical name for protodanksharding.

Lera: Yeah, the bulk of the work was just to understand the danksharding proposal in full, and then we observed some different ways to look into the mathematics – the cryptographic components of it – that hopefully unravel new toolkits that we can use there. So the rough idea, not going too much in depth, is you fit a polynomial through your block.

It’s a bivariate polynomial because your block is rectangular. And then you evaluate this polynomial at more points, kind of expanding the block. And the idea is that if you’ve used these bivariate polynomials – instead of as danksharding was doing it, as a list of multivariate polynomials – you can then apply techniques for bivariate evaluation, bivariate interpolation, and possibly even try to apply bivariate error-correcting codes to that. But it’s very much an open door for more research and exploration. So in this post we try to explain where more research can be done to improve the scheme in that direction.

Dan: Yeah. Well maybe I can add to that. I mean, danksharding is such a beautiful idea. Really. The Ethereum Foundation deserves a huge amount of credit for it. And, you know, Dankrad [Feist] in particular. It’s really quite an elegant construction.

Initially, we were just trying to understand the details and it took some time to recover exactly all the details of how everything works. And we figured maybe it’ll help the world to have another writeup that explains how the mechanism works. Initially, I guess the original danksharding, the way it’s described, is all using univariate polynomial commitments, where we take each row, we view it as a polynomial. Erasure coding is all done using polynomials.

Maybe I can even teach a little bit of erasure coding here in one sentence. Suppose you have two points in a plane and you want to do erasure coding on them. What you can do is you can just pass a line through those two points, and now you can just publish, instead of just those two points, you can publish those two points plus maybe two additional points on the line.

So now you have four points total. And you know that if two out of those four points make it to the receiver, the receiver can use the two points that it received to recover the line and then recover the original two points. That’s the whole idea of erasure coding. So of course, instead of lines, we use higher degree polynomials to achieve different thresholds, but that’s the idea.

Basically, we have two points. We pass a line. We get more points on the line. If only two points of the line make it to the receiver, the receiver can reconstruct the line and recover the original points. And so danksharding actually does this really, really quite elegantly by looking at the block as a matrix, as a rectangular set of data.

And then it basically extends, using these line ideas, both horizontally and vertically. And that actually gives the coded block – where then, pieces of that rectangle are sent to the different validators. What’s interesting is now that you have a rectangle, you can think of it as a two-dimensional object. That very naturally leads to thinking about this as a bivariate polynomial, exactly as Lera was saying.

What danksharding does is it gives a very interesting way to commit to these bivariate polynomials. So it builds a way to commit to bivariate polynomials by using commitments to univariate polynomials. And so it turns out that the reconstruction mechanism that was done in danksharding is also based on construction along lines and columns – also construction as done using univariate polynomials. And then as we were working through this, there was this realization that, hey, everything here really is about rectangles and bivariate polynomials. Maybe there’s a way in which the reconstruction can also be done by using interpolation of bivariate polynomials. 

Lera: So in fact, you take your block and you expand it to X by the factor of two in both directions. So you have 4X more points as a result. But those 4X points only encode a small quadrant.

So in principle, you only need one quadrant in order to interpolate and recover all the rest of this encoded block. So 25% of the points should be enough. But as danksharding works by doing univariate interpolations, it needs 75% of the block to do column and row-wise reconstruction.

If you do bivariate interpolation directly, you should be good with just 25% instead of 75%. So that will improve the number of elements you need in order to reconstruct, to recover the block. It will also improve communication and data availability sampling, but it’s all due to improved reconstruction.

And now the question, it becomes kind of a mathematical question of how do you do efficient bivariate interpolation? 

There is an obvious need for a better algorithm there. And we were researching this a little bit and it appears so far to be a little bit underexplored. So maybe there were no applications for bivariate interpolation before. Maybe it’s just a hard problem, or we don’t know. But that’s definitely an interesting direction to basically try and improve bilinear interpolation algorithms.

Dan: So what I love about this is just like Lera was saying, for the more algorithms folks in the audience, is that there’s been a ton of work on doing univariate interpolation. If I give you points on a univariate polynomial, like points on a line, and I ask you to reconstruct that polynomial, there are very good algorithms for univariate polynomial interpolation.

And it turns out the bivariate interpolation problem, somehow it seems like it received less attention. And what’s really cool here is all of a sudden, the blockchain, Ethereum, danksharding is creating an application for this really natural algorithmic problem of bivariate polynomial interpolation. 

We really need it here. If we had a good algorithm for bivariate polynomial interpolation, we could make danksharding better because reconstruction, as Lera was saying, would go down from 75% to 25%. So to me, this is really quite beautiful in that the blockchain, Ethereum, danksharding is creating this new area of research, or at least prioritizing this new area of research showing, we really need better algorithms, new algorithms, efficient algorithms to do bivariate polynomial interpolation.

So hopefully that will encourage and spur a lot more research on these types of algorithms and presumably they will be very useful for this problem. 

Robert: So I love that we dug into the methodology here and didn’t shy away from the mathematics. I know some of it might sound a little complicated – bivariate polynomials and univariate polynomials. But I especially appreciate how you described these in terms of geometry and shapes, cause I think everybody here can really picture a line going through some points or how a rectangle functions. So I think that really helps ground the kind of work that you’re doing.

I want to double click on this statistic that you mentioned where the current proposal as it exists would require 75% of samples in order to be reconstructed, whereas what you’re proposing would trim that down to 25%. So that’s a giant difference. 75% to 25%. But it also sounds like just for a casual observer, if you’re only having 25%, the fact that it’s just less than 50%, it sounds like, is that really enough to make assurances that this data is available and was made available?

When you go down to 25% it sounds like, I don’t know, you might be cutting some corners. So how do you assure people that in fact, just having 25% of data samples is actually enough and that things can work at that level? 

Lera: Yeah, that gets us to the topic of data availability sampling, and what it achieves, I guess, because this reconstruction threshold – 75% or 25% – basically determines how many samples you need to get high assurance that the data is there.

The way you do the sample is that you ask the network of validators to give you back one element, a random element of this encoded block. And if you get it back successfully – and you can verify also once the validators give you back the proof of the validity of this piece, and you can verify it against the commitments that the chain persistently stores – so when you get back the successful sample that makes you sure that the data is available with probability that is one minus either one quarter or three quarters, depending on how your reconstruction algorithm works, whether it requires 25% of the data or 75% of the data. 

So every time you do a random sample and it comes back successfully, basically your false positive [rate] – the probability that you think the data is available when it’s not – drops down exponentially. And the rate at which it drops down depends on how much data you are required in the reconstruction. So if your reconstruction requests 25% of the data, you do less samples and your assurance – the false positive rate – goes down quicker than if you only have a reconstruction algorithm that requires 75% of the data. 

So depending on how efficient your reconstruction is, you might need fewer samples in order to have the same assurance that the data is available. So that’s why you not only improve the reconstruction here, but you also improve the number of samples you need to do for your data availability sampling.

And data availability sampling is interesting because it’s probabilistic. So the more samples you do, the higher your assurance is that the data is available, right? And you can always amplify this probability by doing more samples to make it clear. 

Dan: I think Lera, what you just explained is really, really important.

That’s the heart of danksharding and data availability sampling. So I’ll say it one more time, just so that the audience will hear it twice, cause that’s really the heart of it. So maybe think of the block. We said the block is gonna be encoded as this rectangle, right? So somehow we go from a block to a rectangle.

The specifics of how that is done is using this erasure coding. But let’s pretend we went from a block of data to a rectangle. So now imagine this rectangle is literally a rectangle of dots. Every dot corresponds to one piece of the data that’s gonna be distributed to one validator. So now to do data availability sampling, somebody wants to verify that enough of the dots are actually available.

We know that if more than 75% of the dots are available, then the block can be reconstructed using the erasure coding method. Or maybe, if what we’re saying would be used, then only 25% of the dots are sufficient to reconstruct the original rectangle. But how do you know that 75% of the dots are available?

So that’s exactly the data availability sampling mechanism. What you do is you can kind of imagine you are throwing darts at this rectangle, right? So every time you throw a dart, you hit a random point in the rectangle and then the validator that holds that point has to prove, “Yes, I really have that point.”

Now you wanna verify that 75% of the dots are available. So imagine you have this rectangle. Maybe only 75% of the dots are there. Some of the dots disappeared for some reason. You wanna verify that 75% of the dots are available, cause if 75% are available, you can reconstruct the entire rectangle. And so what are you gonna do to verify that 75% are there?

You’re gonna throw a bunch of darts at the rectangle. And for every time the dart hits a dot, the validator that it hits has to prove that the dot really is there. And so if you throw a hundred darts and all hundred darts come back saying, “Yes, the data really is there”, that gives you a pretty good idea that more than 75% of the data is available.

Because you know, if less than 75% is available and you throw four darts, you expect one dart to hit a missing dot. If you throw a hundred darts, and all of them come back saying the data’s available, you have pretty good assurance that more than 75% of the dots are there. And so that’s the idea of data availability sampling.

You just try lots and lots and lots of random points, like a hundred of them. If all of them are there, then you have pretty good assurance that more than 75% are available and you can reconstruct the data. And you see, if you want to get 75% assurance, you need to throw a hundred darts. If you need only 25% assurance, you would need to throw fewer darts than that.

So that would basically reduce the amount of data that’s needed to satisfy the sampling mechanism. Now, maybe it’s worthwhile saying that once the data availability sampling check succeeds – so all the hundred darts come back saying, “Yes, the dots really are there” – then the validator says, “Ah, the data’s available”.

And then [the validator] goes ahead and signs the block as saying, “This block passed data availability sampling”. Yeah, and that’s used later on in consensus. So that’s what the test is. Basically, data availability sampling is a very efficient way to test that enough data is available to reconstruct the block without actually reconstructing the block completely.

So I think it’s good to hear it twice and maybe even a third and a fourth time. But that’s kind of the core idea that makes this all work. 

Robert: I love that, and I especially love the physical analogies that you’re using with a dart board and throwing darts. I think that really brings it home for people. So we’re nearing the top of the hour here.

I wanna get ready to close out. But before we do that, just maybe I’ll throw it to you both in one sentence. What is the upshot of all of this? Like why does it matter to get these efficiency gains? 

Lera: Well, I would say the ultimate goal, of course, is to scale the blockchain and these new techniques would allow it to do that, for Ethereum to achieve full scaling.

That’s a really interesting approach, I would say, because in the beginning, Ethereum was thinking about doing sharding, full sharding, and arriving at quite complicated designs. But having rollups helps scale the execution layer of Ethereum, and this leaves it up to Ethereum to scale its data availability layer. Basically, increase the space while rollups increase the execution capacity and those pieces together will give us cheaper and faster blockchains.

Dan: Yeah, Lera said it perfectly. I mean, really the scaling story for Ethereum is rollups and the way to get rollups to be more efficient and cheaper to use is by solving the data problem. And danksharding is a very elegant and efficient way to do that. So the upshot is a scalable version of Ethereum where rollups are much cheaper to use than they are today.

Robert: That’s great. And if you get cheaper transactions and the ability to make more transactions, I think that opens up Ethereum to do all sorts of new applications that weren’t possible before with high gas costs and fees. So this has been great. Thank you all for joining us. I hope that you all learned a little bit about data availability sampling and danksharding.

If you’d like to learn more, as mentioned, you can check out Dan and Lera’s post. It’s a really, really great piece, so I highly recommend reading it and checking out all the resources contained in there.

Thank you all for joining us. I’m looking forward to this weekend. I’m gonna go to my local bar and teach everybody about erasure coding at the dartboard. So thank you all and take care.

Lera: Sounds great. Thank you.

Dan: Thank you. This has been fun. Bye bye.

Thank you for listening to web3 with a16z. You can find show notes with links to resources, books, or papers discussed; transcripts, and more at a16zcrypto.com. This episode was technically edited by our audio editor Justin Golden. Credit also to Moonshot Design for the art. And all thanks to support from a16z crypto.

To follow more of our work and get updates, resources from us, and from others, be sure to subscribe to our web3 weekly newsletter — you can find it on our website at a16zcrypto.com.

 Thank you for listening, and for subscribing. Let’s go!

Web3 and how we got here

Protocol design: Why and how

Cryptography for blockchains: Avoiding common mistakes

Workshop: Web3 pricing and business models

Leadership talk with Uniswap COO Mary-Catherine Lader

Editor”s note: The a16z crypto Regulatory Update is a series that highlights the latest regulatory and policy happenings relevant to builders in web3 and crypto, as tracked and curated by the a16z crypto regulatory team. The roundups are based on recent news, the latest updates, new guidance, ongoing legislation, and frameworks released by regulatory agencies/bodies, industry consortia and professional associations, banks, governments, and other entities as they impact the crypto industry (or applications) around the world. We also occasionally include select other resources such as talks, posts, or other commentary – from us or from others – with the updates.

🧠 tl;dr

1. The SEC charged Bittrex and its co-founder and former CEO William Shihara with operating an unregistered national securities exchange, broker, and clearing agency, while also alleging that the following tokens are securities: OMG, DASH, ALGO, TKN, NGC, and IHT.
2. SEC Chair Gary Gensler testified before the U.S. House of Representatives Committee on Financial Services.
3. The European Parliament passed the Markets in Crypto Assets (MiCA) regulation, making the European Union the first major market to regulate crypto asset transfers.

🏜️ Arizona

• Arizona Governor Katie Hobbs vetoed legislation that would have prevented municipalities from taxing residential cryptocurrency mining operations.

🏞️ Arkansas

• The Arkansas House of Representatives and Senate approved a bill to regulate bitcoin mining in Arkansas. The legislation will go into effect if the governor signs it.

🐻 California

• The California Department of Financial Protection and Innovation issued desist and refrain orders against five entities that solicited funds from investors by falsely claiming to offer high yield investment programs that use artificial intelligence to trade crypto assets.

🌽 Commodity Futures Trading Commission

• The CFTC filed a civil enforcement action against a New York resident, alleging that he fraudulently solicited retail investments in a digital asset trading fund and misappropriated at least $1 million in assets. The Department of Justice filed a parallel criminal action.
• The CFTC charged 14 entities, including crypto companies, for fraudulently claiming to be CFTC-registered futures commission merchants and retail foreign exchange dealers.

🦅 Congress

• SEC Chair Gary Gensler testified before the U.S. House of Representatives Committee on Financial Services. He faced questions about ether”s status as a security or commodity, the SEC”s fast-paced rulemaking agenda, and many other issues.
• The U.S. House of Representatives Committee on Financial Services published a draft stablecoin bill.
• Republican House Representatives sent a letter to SEC Chair Gary Gensler, calling his push to have crypto firms “come in and register” a “willful misrepresentation of the SEC”s non-existent registration process” and blaming the SEC for the lack of the crypto registrants.
• Senator Elizabeth Warren (D-Mass.) and House Representative Alexandria Ocasio-Cortez (D-N.Y.) sent letters to BlockFi, Circle, and twelve non-crypto firms, asking about their companies” decisions to “bank with and maintain large, uninsured deposits at Silicon Valley Bank.”
• Discussing Operation Choke Point 2.0, House Representative Byron Donalds (R-Fla.) said that “nothing is a coincidence” with government agencies, and that agencies are “finding various ways to squeeze an outcome that they want.”
• House Representatives Patrick McHenry (R-N.C.) and Bill Huizenga (R-Mich.) wrote to SEC Chair Gary Gensler reiterating their request for documents relating to the SEC”s charges against SBF and noting the SEC”s failure to comply with previous requests.
• House Representative Patrick McHenry (R-N.C.) said that Europe is “ahead of the game” in terms of web3 regulation in light of the passage of the Markets in Crypto Assets Regulation, and the United States is “behind.”
• House Representative Warren Davidson (R-Ohio) released plans to introduce legislation that would remove SEC Chair Gary Gensler and replace his role with an Executive Director that reports to a board.

⚖️ Department of Justice

• An individual involved in unlawfully obtaining approximately 50,000 bitcoins from the Silk Road dark web internet marketplace was sentenced to one year and one day in prison for committing wire fraud.
• The DOJ charged two U.S. citizens and a South African national with conspiring to manipulate the market for HYDRO, a crypto asset issued by the Hydrogen Technology Corporation.
• An individual who led a scheme to defraud banks and a cryptocurrency exchange of more than $4 million pled guilty to one count of conspiracy to commit wire fraud.

💵 Department of the Treasury

• The Internal Revenue Service is sending four agents who specialize in investigating cybercrime to Australia, Singapore, Columbia, and Germany starting this summer as part of a pilot program “to help combat the use of cryptocurrency, decentralized finance and mixing services in international financial and tax crimes.”
• The Office of Foreign Assets Control sanctioned three individuals for providing support to North Korea through illicit financing and malicious cyber activity. Two of the sanctioned individuals assisted North Korea”s Lazarus Group in converting stolen crypto into fiat currency. The Department of Justice charged one of the sanctioned individuals as well.

🏦 Federal Reserve

• Federal Reserve Board Governor Michelle W. Bowman said that there could be “some promise for wholesale CBDCs” (central bank digital currencies), but that it would be “difficult to imagine a world” where the benefits of a direct access CBDC would outweigh the unintended consequences.
• Federal Reserve Board Governor Christopher J. Waller said that there is “considerable promise” associated with tokenization and the use of smart contracts.

🏔️ Montana

• Montana”s House of Representatives passed legislation that protects cryptocurrency mining operations from certain actions, such as prohibitions on at-home mining and discriminatory utility rates for miners. The legislation will go into effect if the governor signs it.

🗽 New York

• The New York Department of Financial Services adopted a final regulation establishing how companies holding a Bitlicense will be assessed for costs of their supervision and examination.

📈 Securities and Exchange Commission

• The SEC charged Bittrex and its co-founder and former CEO William Shihara with operating an unregistered national securities exchange, broker, and clearing agency, while also alleging that the following tokens are securities: OMG, DASH, ALGO, TKN, NGC, and IHT.
• Chair Gary Gensler released an investor-education video relating to digital assets.
• The SEC listed new job postings for its crypto enforcement unit in New York, San Francisco, and Washington D.C.

🌍 International

🇦🇷 Argentina

• Argentina”s National Securities Commission authorized the launch of a bitcoin index-based futures contract.

🇧🇲 Bermuda

• Coinbase received a regulatory license to operate in Bermuda.

🇸🇻 El Salvador

• Bitfinex announced that it has received El Salvador”s first license for digital asset service providers.

🇪🇺 European Union

• The European Parliament passed the Markets in Crypto Assets regulation, making the European Union the first major market to regulate crypto asset transfers.
• European Central Bank Board member Fabio Panetta said that a digital euro will aim to “complement” cash and mimic its best features, but that it may not afford the same level of privacy.
• European Banking Authority Chairman José Manuel Campa wrote that “the EBA will be paying special attention to diversification” of stablecoin reserves.

🇫🇷 France

• A French Central Bank report acknowledged that regulations must take into account the specific features of decentralized finance (DeFi) and not “replicate the systems that currently govern traditional finance.”
• The French Ministry of Economics and Finance published a public consultation regarding metaverse strategies, noting that its purpose is to “propose an alternative to the virtual online worlds today put forward by international giants.”

🇭🇰 Hong Kong

• Hong Kong”s Financial Secretary Paul Chan said that now is the “right time” to push web3 adoption forward, despite volatility in crypto markets.
• A Hong Kong court declared for the first time that cryptocurrencies are property and capable of being held in trust.

🇮🇳 India

• Indian Finance Minister Nirmala Sitharaman said that effective regulation of crypto assets would require every nation”s consent and global consensus.

💶 International Monetary Fund

• An IMF report said that the “collapse of multiple entities in the crypto asset ecosystem has again made the call more urgent for comprehensive and consistent regulation and adequate supervision.”

🇮🇪 Ireland

• Kraken received regulatory approval from the Central Bank of Ireland to operate as a virtual asset service provider in the country.

🇮🇱 Israel

• The Bank of Israel published a report outlining potential scenarios that would influence its decision on whether to issue a digital shekel. One such scenario involves a significant increase in stablecoin usage.

🇯🇵 Japan

• Japan”s ruling Liberal Democratic Party”s web3 project team published a white paper that sets forth recommendations for boosting the country”s crypto industry.

🇲🇪 Montenegro

• Ripple announced an agreement with the Central Bank of Montenegro to develop a pilot program for a CBDC.
• Montenegrin prosecutors submitted an indictment proposal against Kwon Do-Hyung (“Do Kwon”), co-founder of Terraform Labs, the company behind cryptocurrencies TerraUSD and Luna, for forging personal documents.

🇳🇱 Netherlands

• A Dutch court released, with electronic monitoring conditions, Alexey Pertsev, a Russian developer who worked on code for the Tornado Cash protocol, pending trial.

🇰🇷 South Korea

• South Korean prosecutors indicted Terra co-founder Daniel Shin and nine others in connection with the collapse of the TerraUSD and Luna cryptocurrencies.

🇦🇪 United Arab Emirates

• The UAE”s Securities and Commodities Authority announced that it will start accepting licensing applications from companies that wish to provide virtual asset services in the country.

🇬🇧 United Kingdom

• Bank of England Governor Andrew Bailey called for stablecoins to be regulated like commercial bank money, and he said that the U.K. will likely need a CBDC to “anchor the value of all forms of money, including new digital ones.”
• Bank of England Deputy Governor Jon Cunliffe said that the potential tokenization of money has “major implications” for the Bank of England, and that it should consider whether to put limits on stablecoins used for payments, among other things.
• The Bank of England successfully tested the use of distributed ledger technology to run interbank transactions through the Bank of International Settlements.
• Financial Conduct Authority (FCA) Executive Director Sarah Pritchard said that the FCA wants to work with crypto firms to shape regulation.

🇿🇲 Zambia

• Zambia”s Science and Technology Minister Felix Mutati told Reuters that Zambia plans on finishing tests that simulate real-world cryptocurrency usage by June.

🇿🇼 Zimbabwe

• Reserve Bank of Zimbabwe Governor Dr. John Magudya told the Sunday Mail that Zimbabwe plans to introduce gold-backed digital tokens.

***

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the current or enduring accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party advertisements; a16z has not reviewed such advertisements and does not endorse any advertising content contained therein.

This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.

Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.

Deciding how to price a new product or service is one of the key challenges that founders need to tackle early in the product development lifecycle. In a relatively new market, like crypto, it can be difficult to know even where to start. 

The strategy you choose will depend on your circumstances. Fully decentralized projects which have their own tokens – like many decentralized apps and web3 protocols – tend to have their own unique logic and design considerations. But for founders building other crypto-related businesses – like infrastructure providers, services-based startups, and such – some universal principles provide a template for thinking through product pricing.

Getting one”s approach to pricing right can have a dramatic impact on a company or project”s early customer wins, go-to-market strategy, and long term success. The key is ensuring that a product”s capabilities and features align to a philosophy that allows for optimal pricing. Conversely, once a product is in market, early customer expectations” become set. After that, changing prices can be hard.

One of the coauthors learned this lesson the hard way while he was CEO of Ning, an early social company. In 2005, Ning began as an ad-supported platform that allowed anyone to create their own vertical social network. Several years into the company, it became clear, however, that a subscription-based business model was a better approach for Ning”s target market. While the transition ultimately succeeded, the path to get there was not easy.

Determining the right pricing for a new product in a new market doesn”t have to feel so overwhelming. Below, we pose six questions every founder should ask that can make the process more manageable… and dare we say satisfying?

1. What are all of the different ways I could price, and which approach makes the most sense for my market?

The first step is to identify all of the different possible approaches to pricing. By weighing each option, you can better find the approach – or combination of approaches – most suited to your circumstances. This exercise can unlock new insights or hybrid pricing approaches that better align with the needs of customers or the state of the market.

Here are some common pricing models along with brief descriptions of how they apply, plus a few traditional and crypto-specific examples for each. (It”s worth reiterating that these pricing frameworks largely apply to web3-adjacent projects that don’t have a token and aren’t fully decentralized.)

Now that you know the most common pricing methods, you can begin to apply this framework in practice.

Consider some examples from the crypto industry. While not every customer action is on-chain, you could use on-chain data to better forecast the differences between each pricing approach. For an insurance product, you could price based on the net asset value in a customer”s wallet. For a crypto security service, you could price based on an NFT collection”s floor price multiplied by the number of NFT holders.

You may find compelling ways to mix and match pricing methods. While examples above might be listed in a single category, they often cover multiple pricing model types. Alchemy uses both usage- and feature-based approaches, for example, and The Block uses both usage- and subscription-based ones. Choose the combination that works best for your purposes.

2. Are there opportunities for me to price differentiate within my product line?

Most technology products appeal to a range of customers. The gamut spans from casual first-timers with basic needs to self-sufficient pros seeking more advanced features to enterprise customers who often want the highest level of service and capability that a company can offer. This range of needs and desires creates an opportunity to tailor product offerings to different customer segments through a Good, Better, Best framework.

Here are some examples of the framework in action:

The first offering – Good – provides a solid set of capabilities at an attractive price point. This typically least expensive option gets the job done for a large swath of the customer base. Better builds on Good and usually adds a few additional core capabilities that cater to customers who see the offering as a key tool in their daily workflow. Best appeals to customers who require a specialized feature-set to accomplish their goals or who simply want to know that they are purchasing the most sophisticated offering available. 

Within this framework, founders will often want to experiment and iterate on the product capabilities that go into each bucket. 

Pro tip: When selling to enterprises, ensuring that the Best offering includes an appropriate level of support, security, and onboarding and deployment assistance is key. Without this level of product robustness and focus on customer success, implementations will often fail to gain traction. In these cases, you may lose a customer or reference, which can make it significantly harder to sign the next major customer.

Think about the customer you”re trying to target. Who is this person? What does the person need or desire? One helpful exercise is to map the various stakeholders within a prospective customer”s organization to understand all the places where value accrues. For example, a marketing automation tool might directly benefit the marketing team, but it might also free up analytic resources on adjacent teams, like a data science group, that can now be used on other projects. Focus not just on benefits for a single stakeholder but to address differentiated selling points for all types of stakeholders.

3. Am I charging the right price?

While founders often worry about overpricing their product, a more common mistake is to underprice. 

For a new technology product to gain market traction, it has become conventional wisdom to talk about the new product needing to improve that which preceded it by 10X. In today”s tech market – which has increasingly sophisticated customers, a plethora of competing products and services, and entrenched providers who enjoy large moats and lock-in – a 10X improvement is likely the low bar of what new entrants need to offer to break through. An improvement of this magnitude is basically table stakes.

When a team builds a new product that really does deliver at this level and beyond, the company has the opportunity to capture meaningful economic value in exchange for the productivity improvement, growth acceleration, or cost savings they are delivering to their customers. Discovering where the efficient frontier of this value exchange lies is a key factor in successfully defining pricing for a new product. 

One effective and simple way to conduct this exploration is to separate prospective customers into two cohorts – Cohort A and Cohort B – and to share dramatically different price points with each of them. Imagine that for Cohort A, a startup offers their Good offering for X dollars per month, Better for Y dollars per month, and Best for Z dollars per month. Simultaneously, the startup pitches Cohort B the same offering at 5X for Good, 5Y for Better, and 5Z for Best. In this example, Cohort B”s pricing is literally five times higher than that of Cohort A. What the startup should observe, in this case, is the variation in win rate between the two cohorts. If four out of five prospects in Cohort B still sign up for the service, this suggests that the pricing shown to Cohort A undervalues the offering. 

You might be surprised by how much people are willing to dish out on your product or service!

Another consideration is the denomination of your pricing. In other words, you need to decide what combination of fiat, stablecoins, and/or tokens you accept. While it may be more customer-friendly to accept a wide variety of payment methods, this may also increase your exposure to market fluctuations. Consider that, if your expenses are primarily in fiat, it might make sense only to accept fiat and stablecoins. You must be able to meet your expenses.

4. How do other products or services in adjacent markets price?

When thinking through how to price a new product or service, a good place to start is to look at the approach taken by peers in adjacent markets. Surveying comparable products can provide a helpful starting point for your own pricing strategy. 

For founders building in web3-related categories such as infrastructure, gaming, developer tooling, and so on, look at how analogous web2 products price. A number of web3 startups are innovating in the security and scam prevention space, for example. By looking at the pricing pages of the many companies that offer similar services for web2 consumers and enterprises – like Aura, Bitwarden and CrowdStrike – founders can quickly identify a number of potential pricing models that are already well understood in the market.

5. Do my unit economics work?

Losing money on every customer but making it up in volume is a strategy that doesn”t work for very long, particularly in an economic environment where there has been a broad pullback in startup funding, like the one in which we find ourselves currently. This is a recipe for bankruptcy.

Founders should understand the unit economics of their company from early on in the startup”s life. They should also develop a strong point of view about how these unit economics will change, evolve, and improve over time. With a little preparation, founders can significantly increase their chances for success.

In the early dotcom era, founders pursued a number of ambitious ideas such as grocery and one hour delivery of convenience items. Many of these startups went bust – but then, years later, businesses with strikingly similar models, like Instacart and Doordash, succeeded. A combination of at least four factors made the unit economics of these newer businesses work whereas prior attempts failed. These differences included:

• A 10x+ increase in the number of internet users
• The advent of the ubiquitous, always-connected smartphone
• The development of sophisticated routing and logistics software
• The willingness of suppliers, like grocery stores and restaurants, to integrate with this software (based on a realization that doing so could drive incremental business)

Having the foresight, tenacity, and ability to change the trajectory of one”s unit economics can lead to a long term sustained advantage that others will struggle to emulate. The superior margins that Apple enjoys in its iPhone business are, for example, a function of at least three factors: 1) enormous volume which enables a superior cost structure for the business, 2) a supply chain that the company has built, refined, and heavily negotiated over more than a decade, and 3) a premium brand. While gross margins don”t determine the overall fate of a startup, they can have a long lasting impact on the degree to which companies can invest in R&D, marketing, and other critical functions that determine long term growth rate and success.

6. If I imagined that my market was going to grow 10X-100X and I captured 100% market share, would I end up with an interesting business?

As a founder/CEO, it”s normal to spend countless hours and sleepless nights obsessing over the strategy and details of your company in an effort to find product-market fit, overcome numerous obstacles, and achieve long term success. In the back (and frequently the front) of every founder’s brain is the ever-present fear of failure. 

However, in our experience, there are outcomes much more painful than failure. What”s even worse is to spend years building a product and company and make just about every decision within one”s control correctly only to discover that the market opportunity isn”t large enough to justify the incredible investment of time, talent, and capital that the founder and team poured into the pursuit. Beware Pyrrhic victory. 

A simple way to test whether this failure mode is likely (and hopefully to avoid it!) is to do a simple top-down spreadsheet exercise early in the history of the company. Imagine for a moment that the market you are pursuing grows 10X to 100X and your company captures 100% share of this future market.

Next make an assumption about the average revenue per customer and multiply future market size * average revenue per customer * market share (in this case 100%). How does the math look? Does the imaginable market opportunity look large enough that one can build an interesting company? If so, excellent! If the market doesn”t look large enough to be interesting, perhaps there are changes to be made in market sizing or average revenue assumptions. As Disney CEO, Bob Iger, so aptly put it, the best business advice he ever received was to avoid getting into the business of manufacturing “trombone oil” – a limited market, to be sure. While it might be possible to become the best trombone oil manufacturer in the world, the world only consumes a few quarts of it per year.

Very few founders take the time to do this math – but this simple exercise can prevent heartache, pain, and missed opportunity down the road.

***

Jason Rosenthal is an Operating Partner at a16z crypto, where he works closely with the CEOs and founders in our portfolio to provide leadership guidance, help them plan for and react to the aspects of running companies in rapidly-changing markets, and generally help them become the best version of their professional selves. Jason has spent more than 25 years as an internet entrepreneur and executive, including more than 10 years as a startup CEO, and his career has been dedicated to the development of transformational new platforms. 

Maggie Hsu is a partner at Andreessen Horowitz leading go-to-market for the crypto portfolio. She previously led go-to-market for Amazon Managed Blockchain at Amazon Web Services, and before that led business development for AirSwap, a decentralized exchange. Maggie has held executive positions at Zappos.com and Hilton Worldwide. She was also a consultant at McKinsey and Company. Maggie is also a cofounder of Gold House, a nonprofit collective of pioneering Asian leaders. 

***

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not necessarily the views of a16z or its affiliates. a16z is an investment adviser registered with the U.S. Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party information; a16z has not reviewed such material and does not endorse any advertising content contained therein.

This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities, digital assets, investment strategies or techniques are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments or investment strategies will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.

Additionally, this material is provided for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. Investing in pooled investment vehicles and/or digital assets includes many risks not fully discussed herein, including but not limited to, significant volatility, liquidity, technological, and regulatory risks. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures/ for additional important information.

Danksharding is an approach to scaling the amount of data on chain for a future version of Ethereum. The goal of this upgrade is to ensure that the data on chain was made available to archiving parties when it was first posted. It does so using a technique called Data Availability Sampling, or simply DAS.

In this post we survey how data availability in danksharding works and propose some modifications to the underlying technique. In particular, we explore a small modification that may improve data recovery: the current proposal requires 75% of the shares to recover a block, whereas the modification may reduce this bound to 25%.

Protodanksharding

Danksharding is scheduled to follow protodanksharding (EIP-4844). Protodanksharding will allow clients to write more data to the blockchain by introducing a new transaction type called a “blob-carrying transaction.” Initially, this new transaction type will carry with it a list of up to four data blobs, where each blob is at most 128 KB (4096 field elements of 32 bytes each), adding up to 512 KB additional data per block (targeting 256 KB on average), compared to current Ethereum’s block size, which is around 100 KB on average. 

These data blobs will be handled differently: (i) they will only be stored for a limited time, for example, 30-60 days, and (ii) the data will not be directly accessible to smart contracts despite being part of the transaction data. Instead, only a short commitment to the blob data, called DATAHASH, will be available to smart contracts. The additional burden on the validators seems tolerable: validators currently store less than 100 GB to maintain the blockchain’s state. After protodanksharding they will have to store an additional 50-100 GB.

Danksharding will follow. It will increase the data available to clients by 60x over protodanksharding by increasing the maximum number of blobs per block. Blocks will grow from 0.5 MB per block to 30 MB. But because validators cannot be made to store 60x more data, the data will be dispersed among them such that each validator will store only a small subset of data. Yet they can come to agreement on whether they collectively store all of the data through a data-availability sampling (DAS) protocol.

The blob-data will be priced via a mechanism similar to EIP-1559, and will target about 1 data-gas per byte. Calldata, which is the current cheapest alternative, is priced at 16 gas per byte. But since there will be two different fee markets, these costs are not directly comparable. Roll-up clients will benefit from those upgrades, because currently more than 90% of their client’s fees go to paying Ethereum data fees.

Other projects, such as Celestia and EigenLayer, are also adopting DAS techniques to increase the available data space. These designs are much simpler than fully sharding the Ethereum network.

The goal of data availability sampling

We describe the scheme assuming a proposer-builder separation (PBS) design: 

• Clients submit their blob-carrying transactions to block builders.
• A block builder forms a block B by selecting N client data-blobs. Data-blob number i comes with a short commitment C_i signed by the client that sent it. Let C = (C₁, . . . , C_N) be the list of all N signed commitments in the block B.
• Block builders submit their proposed blocks to the current block proposer (one of the validators). The block proposer selects one of the blocks and posts it to the network as is.

The challenge is to ensure that the block B can be reconstructed at a later time. To do so, the builder replicates the block across a large network of V validators. One could ask each and every validator to store the entire block, but that is considered too expensive. Instead, the block builder

1. encodes the block B into a larger block E using erasure coding;
2. breaks the block E into V overlapping fragments P₁, . . . , P_V;
3. sends to validator number i the pair (P_i, C).

Every validator checks that the fragment P_i that it received is consistent with the list C of signed commitments. The block builder provides proofs for the validators to facilitate these checks.

With this setup, a Data Availability Sampling scheme has two protocols: 

1. A sampling protocol runs between a sampling verifier and the validator set. The sampling verifier takes the list C as input and requests randomly selected elements of the block E from the validator set. The sampling verifier outputs success if it received all the requested elements and all are consistent with C. 
2. A reconstruction protocol runs between a reconstruction agent and the validator set. The reconstruction agent takes C as input and requests elements of block E from the validator set. Once it collects more than 75% of the elements, and all are valid, the reconstruction agent computes and outputs the block B. 

(We discuss an approach below that may reduce the number of elements needed for reconstruction.)

The requirement is that if the sampling verifier outputs success, then the reconstruction agent will output the block B, provided it receives over three quarters of the elements as input. Reconstruction should succeed as long as enough elements are provided, even if the provided elements are selected adversarially.

To recap, the following parties participate in danksharding: 

• Client: sends data blobs (which are either transactions or bundles) to a builder.
• Builder: makes a block and sends fragments of this block to validators.
• Block proposer (one of the validators): posts the block to the network.
• Sampling verifier (any of the validators): runs the sampling protocol and signs the block header if the protocol outputs success. 
• Reconstruction agent: reconstructs a previously posted block when needed by interacting with the entire validator set. Reconstruction succeeds if the validators respond with more than three-quarters of valid elements.

Erasure coding and polynomial commitments 

There are two building blocks to the scheme that we explain next: erasure coding and polynomial commitments.

Building block #1: Erasure-coding

Erasure coding dates back to the 1960s, motivated by the need to transmit information over a lossy channel. It is used in danksharding to protect against validators that lose their data fragments. The technique expands the data from N elements to M elements (M > N) such that the original data can be reconstructed from any intact N out of M elements of the expanded data. Imagine taking N elements (original data), encoding them into M = 2N elements, and giving one encoded element to each of the 2N validators. If a majority of validators are honest, they can always jointly reconstruct the original data. This technique protects against crash faults of any half of the validators. It can be extended to protect against byzantine behavior of a half of the validators using polynomial commitments discussed in the next section.

Here is how expansion works in more detail. To expand the data from N field elements d₁, d₂, . . . , d_N ∈ 𝔽_p to M encoded elements e₁, e₂, . . . , e_M ∈ 𝔽_p, and we interpolate the unique degree N – 1 polynomial p(x) that satisfies p(i) = d_i for i = 1, . . . , N. The polynomial is then evaluated at M points: e_i = (i,p(i)) for i = 1,. . . , M. The original polynomial, p(x), can be reconstructed from any N out of the M encoded elements using Lagrange interpolation. This expansion method is called Reed-Solomon encoding or low-degree polynomial expansion.

Building block #2: Polynomial commitments

Polynomial commitment is a building lock of a DAS scheme. It allows the scheme to do two things:

1. commit to a univariate polynomial p in 𝔽_p[X] of bounded degree D using a short commitment string, C.
2. open the committed polynomial at a given point x ∈ 𝔽_p.

More precisely, given x,y ∈ 𝔽_p, the committer can create a short proof π that the committed polynomial p satisfies p(x) = y and that its degree is at most D. The proof π is verified with respect to the commitment string C and the values x, y. This π is called an evaluation proof.

Security guarantees that the commitment is binding to the polynomial and that an adversary cannot create a false proof.

A number of practical polynomial commitment schemes can be used here. Danksharding uses the KZG commitment scheme that requires a trusted setup ceremony to generate public parameters (called an SRS) but has constant-size commitments and constant-size evaluation proofs. KZG commitments have a number of properties that make them especially well suited for danksharding:

• the commitment is homomorphic: if C₁ is a commitment to p₁ and C₂ is a commitment to p₂, then C₁ + C₂ is a commitment to p₁ + p₂;
• the commitment is unique: if two people independently compute a commitment to a polynomial p, they obtain the same commitment;
• evaluation proofs are homomorphic: for a given x, if π₁ is a proof that p₁(x) = y₁ and π₂ is a proof that p₂(x) = y₂, then π₁ + π₂ is a proof that (p₁ + p₂)(x) = y₁ + y₂.

We can now explain how a client commits to its data-blob. First, the client interprets the data-blob as a vector of m field elements d₁, . . . , d_m ∈ 𝔽_p, for m ≤ 4096. Next, it interpolates a univariate polynomial p ∈ 𝔽_p[X] of degree at most m – 1 such that p(i) = d_i for i = 1, . . . , m. (Technically, danksharding uses 1, ω, ω², . . . , ω^m-1 ∈ 𝔽_p as the evaluation points, where ω ∈ 𝔽_p is an mth root of unity, and a reverse-bit order; this is done for efficiency, but we will not consider it here for simplicity.) Finally, the client constructs the KZG polynomial commitment to the polynomial p. It signs the commitment and sends the commitment-signature pair to the builder. This process requires public parameters (an SRS) containing 4096 group elements. 

Data Dispersal

Next, we explain how the block builder encodes a block and splits it into fragments to send to the validators. Fix some 256-bit prime p. The block builder does the following:

input: A block B in danksharding can contain up to 256 data blobs (64 times more blobs than in protodanksharding), where each data blob is a vector of 4096 elements in 𝔽_p. Thus, we can represent a block as a 256 × 4096 matrix of elements in 𝔽_p. Each row in this matrix corresponds to one data-blob from a client. Recall that each client sends to the builder one row of B along with a signed KZG polynomial commitment for that row. The builder collects 256 signed polynomial commitments C₀, . . . , C₂₅₅, one commitment per row.

step 1: The builder interpolates a bivariate polynomial d(X,Y) such that d(i,j) = B[i,j] for i = 0, . . . , 255 and j = 0, . . . , 4095. This bivariate polynomial has degree at most 255 in X and degree at most 4095 in Y.

step 2: The builder uses the erasure coding method described above to expand the block by a factor of two in each direction. That is, it forms a 512 × 8192 matrix E of field elements by setting E[i,j] ← d(i,j) for i = 0, . . . , 511 and j = 0, . . . , 8191. This is illustrated in the following figure.

step 3: The builder verifies that each signed C_i is a KZG commitment to the univariate polynomial d_i(Y) := d(i,Y), for all i = 0, . . . , 255. Observe that the polynomial d_i(Y) is an interpolation of row i of B, and therefore must be the same as the polynomial committed to by client i. The builder rejects all data-blobs for which C_i is malformed.

Now the builder uses C = (C₀, . . . , C₂₅₅) as a commitment to the block B, or more precisely, to the bivariate polynomial d(X,Y). 

Let’s show that C = (C₀, . . . , C₂₅₅) is indeed a polynomial commitment to the polynomial d(X,Y). For some given x,y,z ∈ 𝔽_p, let us construct an evaluation proof that convinces the verifier that d(x,y) = z with respect to the commitment C. Since the degree of X in d(X,Y) is at most 255, there are constants λ₀, . . . , λ₂₅₅ ∈ 𝔽_p that depend only on x such that 

d(x,Y) = λ₀ · d(0,Y) + . . . + λ₂₅₅ · d(255,Y).

Then, by the homomorphic property of KZG commitments it follows that 

C_x := λ₀ · C₀ + . . . + λ₂₅₅ · C₂₅₅ 

is a KZG commitment to the univariate polynomial d_x(Y) := d(x,Y). Hence, the verifier can construct C_x from C on its own. Let π be the KZG evaluation proof for the polynomial d_x(Y) that convinces the verifier that d_x(y) = z with respect to the commitment C_x. This π is the required evaluation proof for d(X,Y) at the point (x,y).

This argument shows that C is a polynomial commitment to d(X,Y). It is worth noting that while each client signs a row of B independently of the other clients, the collection of all client signatures functions as a signature on the polynomial commitment to d(X,Y).

step 4: The minimum unit of communication in this DAS scheme is a sample, which is a 16-elements row vector. The matrix E with 512 × 8192 elements is treated as a square matrix of 512 × 512 samples. Let V be the number of validators. Then the block builder breaks the matrix E into V overlapping fragments P₁, . . . , P_V, where P_i comprises exactly two rows and two columns of samples of E, chosen at random from the 512 rows and 512 columns of samples. Thus, P_i contains 2 × 512 × 16 + 2 × 8192 = 9216 field elements in 𝔽_p. This is much smaller than the full block B, which is about a million field elements. The minimum number of validators in Ethereum is 128 (and currently around ~500,000), so enough validators exist to ensure that the whole block is sufficiently well covered. 

step 5: The block builder sends the triple (P_i, C, π_i) to validator number i, where π_i is a list of evaluation proofs for all the elements in P_i: one proof for each cell d(x,y) in the two rows and two columns of samples in P_i. The proofs in π_i enable the validator to verify that every field element in P_i is consistent with the commitment C.

In danksharding the number of validators can greatly exceed the number of columns or rows in the block. Therefore, some columns and rows might be stored by multiple validators. As such, danksharding uses both replication and Reed-Solomon erasure coding to ensure that the data can be reconstructed. 

Full block’s reconstruction

When a reconstruction agent needs to reconstruct the entire block, it has C, and asks the validator set to send it their fragments. In response, an honest validator i sends (P_i, π_i). The reconstruction agent checks the opening proofs in π_i, and if valid, it accepts the values in P_i. Hence, byzantine validators cannot send false data. However, Byzantine validators may refuse to send their data and not respond to the reconstruction agent. 

When some data is missing, danksharding needs at least 75% of the matrix E to reconstruct the block. Since the validators only store complete rows and columns, the worst-case scenario is the following one where 75% – ε of the block’s elements are present, yet it is impossible to reconstruct the missing elements.

To see why the data cannot be reconstructed in this case, observe that there is a non-zero bivariate polynomial δ(X,Y) that evaluates to zero on the entire “present” region and has the required degree bounds on X and Y. Therefore, during reconstruction it is not possible to tell if the missing white area comes from the correct polynomial d(X,Y) or the polynomial d(X,Y) + δ(X,Y); both polynomials agree on the present data. As a result, when the missing data follows this pattern (up to a permutation of rows and columns) the missing data cannot be reconstructed. Note that if a validator drops some of its data, thereby becoming “byzantine,” it drops it all, both rows and both columns.

So less than 75% of the block is not enough in the worst case. However when 75% or more of the block is present the reconstruction is guaranteed to succeed by a simple greedy algorithm.

The reconstruction algorithm: Reconstruction works by iteratively finding an incomplete row (or a column) with at least 50% of available elements in it and using univariate interpolation to reconstruct the missing elements of the row (or column). To see why this procedure eventually reconstructs the whole block, let”s assume the block is still missing some elements, yet we can find no row or column that can be reconstructed. This means that each row and column either has 50% of unavailable elements, the columns through those elements also have to have >50% unavailable elements, which immediately implies that >25% of the block is unavailable, which contradicts the original assumption.

To render one block unreconstructable, the adversary needs to attack at least 15/16 of the validator set. The attackers in this case would pick a quadrant to erase, and attack those validators that have at least one of their two rows or one of their two columns in that quadrant. 

Local reconstruction of validator’s data

Suppose that an honest validator i crashes and loses its data. When it comes back online it wants to reconstruct its two rows and two columns P_i as well as the opening proofs π_i. Danksharding enables a validator to do that without reconstructing the entire block. The validator can download its data from another validator that stores the same exact set of points, or by obtaining 50% of the elements of its row (or column) from other validators, and reconstructing the rest of the row through univariate interpolation. This process does not have to go through full reconstruction.

For example, imagine a validator stores at least 50% of column number 42, but less than 100% and it needs to reconstruct the missing elements. That is, the validator holds pairs (E_(i,42), π_i) ∈ (𝔽, 𝔾) for all i ∈ S, for a set S where 256 ≤ |S| < 512. To reconstruct the missing pairs, the validator does the following:

1. Using Lagrange interpolation over the field 𝔽 constructs a polynomial p(x) of degree 255 s.t. p(i) = E_(i,42) for all i ∈ S.
2. Evaluates the polynomial to obtain the missing elements: E_(i,42) := p(i) for all i ∈ {0..511}S. Note that the points are guaranteed to lie on a 255-degree polynomial because the validator has checked them against the commitments C.
3. Obtain the missing proofs via a multi-exponentiation by doing polynomial interpolation “in the exponent”: for all i in {0..511}S compute π_i := ∑_j∈S π_j · Λ_j,S(i), where Λ_j,S(i) is the Lagrange coefficient Λ_j,S(i) := ∏_k∈S,k≠j(i – k)/(j – k).

Step 3 uses the fact that KZG polynomial commitments have homomorphic proofs. To the best of our knowledge, KZG is the only polynomial commitment scheme with this property.

Sampling for data availability in danksharding

To determine whether enough of the expanded block E is available for reconstructing the original block B, the sampling client queries for random samples of E. In response to each query, it gets a sample (16-elements row vector) and an evaluation proof to check the sample against the commitment C. If all its queries are successfully answered, the client accepts the data as available. If the client makes a total of Q queries, then it falsely accepts unavailable data with probability (3/4)^Q which drops exponentially quickly with the number of queries Q. The rate 3/4 corresponds to the reconstruction threshold of 75% from the previous section. 

In the Ethereum network, sampling will be done by three types of parties: (i) full-nodes who can’t afford to store the blob-data in full, (ii) light clients, and (iii) the validators themselves. The validators have to certify that the data is available before casting any votes on a block and its predecessors. Each epoch has a fixed set of validators, which are split into 32 committees randomly. Each committee is assigned a 12 seconds slot of the epoch (there are 32 slots per epoch). One block is expected to be confirmed per slot. Each validator receives the fragments (2 rows and 2 columns of samples) of each of the blocks, even for blocks when the validator is not part of the attesting committee. In addition, each validator samples the current block and all the previous blocks to ensure that all the blocks are both valid and available before selecting a block for attestation according to a fork-choice rule.

The protocol guarantees that the data will be available for an epoch of 32 slots, that is, for 6 minutes. Additional approaches such as proofs of retrievability or incentive mechanisms can help ensure the data is made available for longer periods of time, e.g., 30-60 days when the blobs should be available before expiring.

A proposal for 25% reconstruction

In this section we explain how to make reconstruction succeed if only 25% of the data is available (compared to 75% as in current danksharding explained above).

When the fraction of available points is below 75%, then the greedy column-wise and row-wise univariate interpolation can fail. Instead, we propose to do reconstruction directly through bivariate polynomial interpolation. In this case, full reconstruction is possible even if only 25% of the points of E are available. However, this requires a small modification to the way that fragments are assigned to validators. Instead of giving each validator two complete rows and two complete columns of samples, we propose that each validator is assigned to store 

• one complete row (512 samples), one complete column (512 samples), and 
• a collection of 1024 samples randomly (or pseudorandomly) spread around the matrix E.

The overall storage requirement per validator is unchanged, but now reconstruction is possible even if fewer than 75% of the samples are available. Moreover, multiple validators store the same set of points, so that if any validator needs to reconstruct its fragment, it can request it from any other validator that stores the exact same fragment. If no such validator is available, it can do local or full reconstruction to obtain its fragment.

This hybrid proposal allows cheap reconstruction in the happy-case when the number of byzantine validators is small, so that more than 75% of the samples of E are available. In the unhappy case, when the number of byzantine validators becomes too high, the data can be reconstructed using full-bivariate interpolation from only 25% of the samples thanks to the random dispersion of samples throughout the matrix E.

Bivariate interpolation can naively be done by solving a linear system of equations on the coefficients of the polynomial. This simple approach requires constructing an interpolation matrix with 2²⁰ × 2²⁰ field elements (32TB). This is quite large but not infeasible. However, there are better methods that can be used. (See for example the survey from P.J.Olver-2016.) While the cost of bivariate interpolation is non-trivial, it is only needed when the fraction of recovered points is below 75%, and can be treated as a safety measure. To enable this safety measure, danksharding needs to be slightly modified to assign fragments to validators as outlined above. 

Danksharding with IPA commitments

The main drawback of the previous construction is the need for a trusted setup for the KZG polynomial commitment scheme. Otherwise the scheme is blazingly fast. However, the trusted setup typically needs a lot of work and coordination. (Our work on on-chain KZG setups can help simplify the ceremony for future projects). Some other polynomial commitment schemes do not need a trusted setup (e.g., Bulletproofs), although they do not have homomorphic proofs required for the efficiency of validators when reconstructing the data they need to store (as pointed out by Vitalik). 

The construction can be altered, however, to avoid the need for homomorphic proofs and still have light-weight validators. The high-level idea is to make block builders compute commitments to the columns of the block-matrix E. With this approach, the validators won’t need to reconstruct the column-proofs; they will simply reconstruct their columns in full themselves and recompute the proofs against the column commitment from scratch. Through consensus, the validators would make sure they agree on the same set of columns-commitments.

More precisely, the steps 1-4 are the same as in danksharding explained above. Step 5, however, is different.

step 5: The block builder computes polynomial-commitments to the columns of B: denote them by T = (T₀, T₁, . . . , T₄₀₉₅), each T_j is a commitment to d(X, j) (concretely it could be Pedersen vector commitment to the vector of coefficients of d(X, j)). Next, it creates a proof that C and T commit to the same matrix as follows. The block’s builder chooses a (pseudo)random point (x̂, ŷ), uses the homomorphism of the commitments to interpolate and compute the column commitment T_ŷ to a univariate polynomial d(X, ŷ), and a row commitment C_x̂ to a univariate polynomial d(x̂, Y) and create two proofs: π_x̂ – the polynomial evaluation proof of d(x̂, ŷ) against C_x̂, and π_ŷ – the polynomial evaluation proof of d(x̂, ŷ) against T_ŷ. The point (x̂, ŷ) is universal for all of the validators, and can be obtained for example from a random beacon output or generated pseudo-randomly with a Fiat-Shamir transform. The block’s builder then sends (P_i, C, T, π_x̂, π_ŷ, d(x̂, ŷ)) to validator number i, where P_i is the two rows and two columns of E. The builder computes no proofs in this construction.

The validator verifies the proofs π_x̂, π_ŷ to catch a malicious block”s builder who generates the column commitments T incorrectly. The validator then recommits to two rows and columns in P_i and verifies that those match the corresponding elements in C and T. If the row, denote x, in P_i is within range of the original matrix, B: x ∈ {0..255}, the validator simply verifies that the commitment matches the corresponding element C_x from C; however if the row is in the expanded portion: x ∈ 256..511, the validator first interpolates through the commitments in C to obtain C_x:

C_x := λ₀ · C₀ + . . . + λ₂₅₅ · C₂₅₅

Note that interpolation is possible because the IPA commitments (not their proofs though) are homomorphic. The validator verifies the columns of P_i in a similar way against T, interpolating if needed.

In this construction, the block’s builder does not have to compute the proofs for the validators. The validators can compute all the proofs themselves. It’s an interesting research problem, however, to figure out an efficient algorithm to compute a large number of proofs at once (similar to how it’s done for KZG, or using Vitalik’s ideas for IPA).

The main bottleneck of this approach, as we see it, is the loss in efficiency for sampling clients, as the proofs for IPA-based schemes are logarithmic in size (in contrast to constant-size for KZG). Moreover, the sampling clients might need to download column commitments in addition to the row commitments, incurring an overhead in communication. However, we believe this to be a promising direction for further exploration.

Danksharding with Merkle commitments

Another plausible approach Vitalik explored recently to avoid the trusted setup is to use Merkle commitments instead of polynomial commitments. Merkle commitment is not a polynomial commitment scheme, so it’s more challenging to make it work with erasure-coding. The block’s builder will erasure code the data and commit to the expanded data with Merkle-tree roots. The main challenge is to detect incorrectly expanded data without downloading the full data. Fraud proofs can be used to get around this issue, but that relies on the existence of a client that would download the full data, check that it was erasure coded correctly and correctly committed to, and would raise an alarm if it is not by providing a fraud proof.

Alternatively, a FRI proof can be used to check that the leaves of the Merkle tree are close to a Reed-Solomon codeword (i.e., check that the data underlying the Merkle commitment was erasure-coded correctly). The sampling clients would check the FRI proof and sample a sufficient fraction of the leaves by requesting their Merkle proofs to ensure the data is available and can be reconstructed.

***

Data availability sampling, and danksharding as one of its concrete instantiations, would bring the cost of data-storage down, leading to more scalable and cheaper blockchains. The coding aspects of DAS is a potentially rich area for research with many possible directions to explore. We suggested one of the possible avenues: improving the reconstruction protocol to use fewer samples (25% instead of 75%). Another exciting direction is to explore alternative commitment schemes that do not require a trusted setup.

***

Read more

Protodanksharding

DAS for Ethereum and danksharding

Research on DAS

KZG commitments

***

Thanks to Ansgar Dietrichs for providing many insightful comments that informed the post.

***

Valeria Nikolaenko is a Research Partner at a16z crypto. Her research focuses on cryptography and blockchain security. She has also worked on topics such as long-range attacks in PoS consensus protocols, signature schemes, post-quantum security, and multi-party computation. She holds a PhD in Cryptography from Stanford University under advisorship of Professor Dan Boneh, and worked on the Diem blockchain as part of the core research team.

Dan Boneh leads the applied cryptography group at Stanford University and co-directs the computer security lab. Dan”s research focuses on applications of cryptography to computer security, and includes cryptosystems with novel properties, web security, security for mobile devices, and cryptanalysis. He is the author of over one hundred publications and is a recipient of the 2014 ACM prize, the 2013 Godel prize, and the 2011 Ishii award for industry education innovation.

***

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not necessarily the views of a16z or its affiliates. a16z is an investment adviser registered with the U.S. Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party information; a16z has not reviewed such material and does not endorse any advertising content contained therein.

This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities, digital assets, investment strategies or techniques are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments or investment strategies will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.

Additionally, this material is provided for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. Investing in pooled investment vehicles and/or digital assets includes many risks not fully discussed herein, including but not limited to, significant volatility, liquidity, technological, and regulatory risks. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.

Crypto markets can be volatile – but crypto innovation follows an underlying order. Builders brought in when prices were high have stuck around, resulting in a steady flow of new ideas, code, and projects. A new generation of web3 startups is working on the next wave of advances, and many are actively hiring. 

Meanwhile, the tech talent landscape has shifted dramatically in recent months. Layoffs across all sectors, but especially in larger tech companies, have left hundreds of thousands of workers looking for new challenges and opportunities. And, as a result, web3 startups with cash on hand and an optimistic outlook are seeing a very different talent pool than they were a year ago. But how can companies set themselves up to make smart, well-timed hires in periods of volatility? No matter the season, hiring the right people at the right time is critical to growing a resilient team.

In this post, we go over a few principles and best practices for navigating this new talent landscape as a web3 startup. As former leaders at high-growth web2 and web3 organizations, we have seen a wide variety of scale, talent needs, and market fluctuations. So here are our thoughts on how teams can make the most of their headcount (and budget) as they transform the proverbial hiring abyss into a functional and efficient hiring funnel.

Do the work up front

Hiring fast takes foresight. Without some thorough planning, it can take longer to fill a role that a team already (maybe even desperately) needs. A few principles for getting started: 

• Be realistic about hiring needs. A full-time hire isn’t necessarily a cure-all, especially when teams aren’t sprinting through a bull run. Teams may want to consider starting with agencies, freelancers, or other contingent workers in order to be able to scale up and down as budget and workload ebbs and flows. 
• Work backwards from business needs to define the role. If it isn’t clear whether to hire a senior individual contributor and a senior director, then take a moment to unpack the company’s needs. Some questions to ask: What will the person in this role do in their first week? What will they do in six months, or a year? And will they need to build a team? Or build out the nuts and bolts of their discipline?
• Define responsibilities and map them to skill sets. Can one person do it all, or does the team need to make multiple hires? For example, a token economist (or mechanism designer) may have the analytical and economic acumen to create a token program but you may want to consider hiring a software engineer to deploy and maintain these models in production.
• Avoid over-hiring by staying focused on immediate needs, now and in the near future. Members of smaller teams often end up doing more than what’s outlined in their job descriptions – trends, technologies, and market conditions can move fast in web3. Focus can help organizations stay nimble and goal-oriented, while identifying obvious skill gaps and opportunities for growth.
• Consult an expert on unfamiliar skills. Small companies must make a lot of “first” hires – particularly in newer, more niche web3 roles that didn’t exist a few years ago. Hiring managers seeking out skill sets that they don’t have (whether that’s writing Solidity or managing NFT communities) might opt to find an external advisor.
• Reconsider “web3 native” as a required skill. Many hiring managers are asking candidates to come in with web3 experience, a qualification that can limit teams to a select cohort of candidates (a small enough pool that challenges many companies, despite recent layoffs). Worst case, teams may be seeking people with a combination of skills that don’t actually exist. Instead, consider which roles are appropriate for deeply experienced web2 professionals, or enthusiastic new talent looking for challenging, career-defining experiences.

These are just a few best practices, but there are, of course, many more to keep in mind before you post that job description. It’s also okay if pieces of this process aren’t quite working — hiring managers should always debrief and make adjustments as they go.

Quality, not quantity

A year ago, many companies (in web3 and beyond) were breathlessly filling seats to keep pace with market pressures. Now the same companies are reducing headcount, or slowing their hiring. Teams looking to fill key roles might need to make some hard choices, and prioritize their hiring plans accordingly. When there are fewer roles to fill, hiring the right person is even more important.

At the same time, the talent pool has deepened, and excellent candidates that weren’t available last year may be interested in exploring new roles, opportunities, and risks. But despite the influx of talent, a good principle to keep in mind isn’t just finding the best of the best – but also finding people who are in it for the long run. If someone is willing to join a project in rockier times, they will stick around in good times, too.

Connecting over closing

As recruiters, we sometimes like to think of ourselves as sales people (and we can be!) – so it’s easy to slip into the habit of treating the time between receiving and signing an offer like a hard sell. This mindset falls short when it focuses more on rattling off selling points of a company, and less on identifying a person’s unique needs, wants, and expectations. (Also: talent is a long game. You want to keep people, not just companies, centered because over time as your relationships deepen with the talent pool they will change jobs multiple times.)

Closing actually starts from that first phone call, and lasts throughout the interview process by learning about a candidate. This is especially important in more volatile times as a focus on quality of connection will help close candidates faster (read: less time spent hiring) and result in less churn and attrition after employees join (read: less time spent backfilling). When big market shifts happen, you may even have to “re-close” a candidate, giving them a call to help them tease apart perception of media headlines from reality. 

Closing should focus on two areas in particular:

• Professional fulfillment: What makes a candidate passionate? And what problems are they trying to solve? With so many smart, idea-driven candidates in web3, there’s plenty to talk about. 
• Work-life balance: Every company likes to pitch stellar work-life balance to potential hires; but in reality, startups can come with longer hours, greater uncertainty, and a slew of tasks that fall outside of a candidate’s job description. How do candidates rank work-life balance among other benefits? And what can your company realistically deliver?

And also include a few key considerations that are even more important in more volatile times:

• Risk appetite: Is the candidate bought into the space and opportunity? This will matter more than usual to ensure there aren’t too many distractions in the new hire’s journey. Take the time to connect them to the role: Why them? Why here? Why now?
• Experience with ups and downs: Has the candidate experienced a volatile market before? A candidate with less experience may need to hear how a company deals with market cycles: How do you do more with less, and focus on the work vs. external noise?
• Base pay vs. equity: Make sure candidates understand the nuances to their compensation packages. Candidates may rightfully scrutinize packages that include tokens or equity, which can shift with the market. How can you provide information and mental models that help the candidate apply these to different outcomes? 

There are many ways to build trust throughout the interview process, and make the best possible impression on potential hires (get to know them, stay transparent about the hiring process, regularly communicate updates and expectations, and more). These insights can also paint a picture of why this role at this company at this time is the best fit. Not every offer will work out, but the right candidates will.

Focus on the future state

Speaking generally, the earlier a company is in its journey, the more work there is to do to help candidates “see the future,” especially if a candidate is more easily swayed by brand recognition. What is the company looking to accomplish? How does this hire fit into that vision? In an industry where change and innovation move quickly, it’s key that leaders keep their candidates (and employees) focused on the problem they are solving.

A prospective hire should be invested in the work a team is looking to do; they should also understand what success looks like, and what path the company is taking to be successful. Building in web3, for example, is dynamic and ever-changing — this can lead to a longer time focused on solving hard problems and being able to pivot quickly as the market evolves. 

Often (and especially in web3) this may require taking a step back and explaining a broader world view or ecosystem. Set the stage, and then dive into what matters: What needs to happen for there to be a successful exit here? Maybe a candidate would trade liquid compensation for more long-term incentives like equity. What does this growth story look like? 

To truly paint this picture, meet candidates where they are. Do they truly grasp the mission of web3 and understand where this company fits in? Consider trying a mental model for mapping the company’s growth to their personal growth (how can their role grow with the company, long term).  

***

Although every company, candidate, and hiring journey is unique, focusing on building a consistent process and hiring the right candidates will prevent churn in a time when teams need stability and precision the most. Being diligent when hiring is slow can be like building a muscle: The more teams practice, the stronger they’ll be when hiring speeds back up.

***

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the current or enduring accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party advertisements; a16z has not reviewed such advertisements and does not endorse any advertising content contained therein.

This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.

Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.