Don’t Trust When You Can Verify: A Primer on Zero-Knowledge Proofs
Zero-knowledge proofs (ZKPs) have emerged as a pivotal cryptographic innovation representing a paradigm shift replacing the need to trust with the ability to verify. This comprehensive exploration will shed light on how ZKPs are reshaping privacy and security paradigms across various sectors. By the end of this article, policymakers will have gained a nuanced understanding of ZKPs' potential to improve security while maintaining privacy across a wide range of use cases and why they are indispensable in today’s digital ecosystem.
Sonja O'Brien -Created in Canva
Introduction
From finance to communications to commerce, the most essential things in our lives are increasingly digital, and at a pace that only continues to accelerate. This has fostered unprecedented economic growth and technological innovations, however our most sensitive and private information has necessarily come along for the ride. This puts individuals, private organizations, and governments increasingly at risk of becoming victims of data breaches and hacks.
Solutions to these problems often call for increased know your customer practices (KYC) and reporting. KYC is a regulatory and security process where businesses verify the identity of their clients to prevent identity theft, financial fraud, money laundering, and terrorist financing. While there are benefits to this approach, it comes with the risk of sensitive data being distributed to even more parties, thus increasing the potential surface area for attack, and further reducing individual privacy. In a world where trust in institutions is historically low, this can be a hard pill for individuals to swallow. But what if there was a solution that allowed individuals, private organizations, and governments alike to verify things like identity and financial data with said data always remaining encrypted and obfuscated? What if you didn’t have to trust because instead, you could verify?
Zero-knowledge proofs (ZKPs) have emerged as a pivotal cryptographic innovation representing a paradigm shift replacing the need to trust with the ability to verify. This comprehensive exploration will shed light on how ZKPs are reshaping privacy and security paradigms across various sectors. By the end of this article, policymakers will have gained a nuanced understanding of ZKPs' potential to improve security while maintaining privacy across a wide range of use cases and why they are indispensable in today’s digital ecosystem.
An Idea Ahead of Its Time
First described in a 1985 MIT paper, zero-knowledge proofs (ZKPs) have been an idea simmering on the back burner of cryptographic research for decades. Although conceptually sound, the technology required for large-scale and cost-effective deployment was not yet available. However, with recent advancements in computational power and the advent of blockchain technology, ZKPs are now coming to the forefront.
The feasibility of ZKP applications now being able to be deployed at scale can be attributed primarily to two key developments:
Advancements in Computational Power: Historically, ZKPs were computationally intensive, requiring significant processing power which limited their practicality. The exponential growth in computational capabilities, in particular, more efficient processors and the widespread availability of cloud computing, has made it feasible to perform complex ZKP computations quickly and cost-effectively.
Synergies with Blockchain Technology: The cryptographic nature of ZKPs aligns seamlessly with blockchain technologies. Blockchain's inherent characteristics of decentralization, immutability, and transparency provide a conducive environment for ZKP applications. Several blockchain platforms have integrated ZKPs as a core component of their technology stack, fostering ready-built platforms for the development and deployment of ZKP-based applications.
These technological advancements have transformed ZKPs from a theoretical concept into a practical tool, unlocking their potential to enhance privacy and security in an increasingly digital world.
How Do Zero-Knowledge Proofs Work?
In a zero-knowledge proof, there are two parties: a prover and a verifier. In this transaction, the prover can assure a verifier of the truthfulness of a particular assertion regarding a data point without revealing any extra details about the corresponding data.
ZKPs can be categorized into two types:
Interactive, where the prover must engage in the verification process anew for each distinct verifier.
Non-interactive, where the prover creates a universally verifiable proof.
The three key characteristics that encapsulate a ZKP are:
Completeness: For any true statement, a credible prover can successfully persuade an honest verifier of their knowledge about the accurate input.
Soundness: In the case of a false statement, no deceitful prover can independently convince an honest verifier of their knowledge about the accurate input.
Zero-Knowledge: When the statement is true, the verifier gains no additional information from the prover beyond the fact that the statement is indeed true.
The math that makes this possible is beyond the scope of this piece, however, a simple example can help conceptualize how a ZKP works.
Imagine you have a "Where's Waldo?" book, and you claim to know exactly where Waldo is on a particular page. Your friend, however, is skeptical and wants proof of your claim. But here's the challenge: you need to prove you know where Waldo is without pointing him out directly, thus keeping his location a secret.
Where's Waldo? byBarbara Friedman
Here's how you can do it: "Where's Waldo?"
The Challenge: You open the "Where's Waldo?" book to the page in question and show it to your friend.
The Proof Technique: To prove your knowledge without revealing Waldo's location, you take a large piece of cardboard with a small hole cut in it.
The Verification: You place the cardboard over the page so that the hole reveals Waldo, but nothing else on the page is visible. Your friend can now see Waldo through the hole, confirming you know his location.
Maintaining Secrecy: Despite verifying your claim, your friend still doesn't know where on the page Waldo is. The rest of the page is covered, keeping the information hidden.
In this scenario:
Completeness: If you truly know where Waldo is, you can easily align the hole in the cardboard to reveal him, convincing your friend of your knowledge.
Soundness: If you don’t actually know where Waldo is, you won’t be able to align the hole correctly, and your friend won’t be convinced of your claim.
Zero-Knowledge: Despite proving you know Waldo’s location, you haven’t revealed any additional information about where on the page Waldo is.
These proofs enable one party to prove the validity of a statement to another without revealing any underlying information. In an era where data breaches are rampant, ZKPs offer hope for secure and private online interactions.
A Myriad of Real-World and Digital-World Applications
ZKPs have a wide array of applications, each demonstrating their versatility and importance in enhancing privacy and security in various digital domains:
Decentralized Identity and Authentication:
ZKPs can serve as a foundational technology for identity management systems by offering a dual advantage: they enable robust identity verification while safeguarding personal information. This is particularly relevant in systems built on blockchains, which are inherently global and interoperable, facilitating secure international information sharing.
Offline Applications: ZKPs can be used in everyday situations like confirming age for alcohol purchase or even verifying citizenship and non-sanctioned status at international borders. In these cases, ZKPs provide proof of specific personal attributes without revealing any additional personal information.
Online Applications: The decentralized nature of blockchain networks limits traditional KYC practices, but ZKPs offer a way for central entities to securely verify user identities.
Since it is one of the most impactful applications of ZKPs, the following section takes a deep dive into decentralized identity.
Voting Systems:
ZKPs introduce a new paradigm in voting systems, particularly in conjunction with distributed ledger technologies. Together, they enable a voting process that is both anonymous and verifiable, enhancing the integrity and transparency of elections.
Identity Verification and Eligibility Checks: Leveraging the decentralized identity solutions discussed earlier, ZKPs can be used to verify a voter's identity and their eligibility to vote. This approach ensures that only qualified individuals participate in the voting process and that no one can vote more than once. Such identity verification acts as a gatekeeper to access existing voting systems, upholding the principle that every vote counts and should be counted correctly.
Anonymous Verifiable Voting: Beyond verifying voter identity, ZKPs can preserve the anonymity of the voting process. Voters can confidently validate their eligibility and cast their ballots without having to reveal their identities or the specifics of their vote. This application of ZKPs allows each vote to remain confidential while ensuring the entire voting process is transparent and auditable.
Auditable Yet Private Voting System: The integration of ZKPs in voting systems is transformative, enhancing both the verification mechanism and the voting process itself. By incorporating ZKPs, the system not only links each vote to an eligible voter but also maintains voter anonymity. This dual capability creates a voting environment that is secure, transparent, and respectful of voter privacy. The system can be implemented either as an identity check before accessing existing voting platforms or as an integral part of a new blockchain-based voting system, embedding both identity verification and vote casting within a unified, secure framework.
AI and Machine Learning:
- Privacy Preservation in Data Usage:
A key challenge in AI and machine learning is accessing large datasets for training models without compromising sensitive information contained within the data. ZKPs enable the verification of data authenticity and integrity for these models while keeping the underlying data elements private.
The capability to verify information while maintain privacy is particularly crucial in industries like healthcare or finance, where data confidentiality is paramount. ZKPs allow for the utilization of detailed datasets for model training without exposing individual data points, thus upholding privacy standards and regulatory compliance.
- Blockchain Integration for Model Accountability:
Combining ZKPs with blockchain technology introduces a new level of transparency and accountability in AI and machine learning. Blockchain provides an immutable record of transactions which enables the tracking of data changes, model updates, and decision-making processes when applied to AI models.
This integration allows for a transparent audit trail of how AI models evolve over time and how data influences these models. It addresses concerns around model integrity and reliability, ensuring that changes or updates to AI systems are transparent and accountable.
- Enhancing Trust in AI Systems:
By utilizing ZKPs, AI and machine learning systems can demonstrate that they function as intended without revealing their internal workings or the proprietary data they process. This builds trust among users and stakeholders, assuring them that the AI systems are not only effective but also respect privacy and data security.
In scenarios where decisions made by AI systems need to be justified or explained, such as in credit scoring or medical diagnostics, the combination of ZKPs and blockchain can provide necessary assurances about the data and processes used without compromising sensitive information.
- Facilitating Collaborative AI Development:
ZKPs enable multiple parties to contribute data to AI models securely and privately. This collaborative approach, bolstered by the trust mechanisms of ZKPs and blockchain, can accelerate AI innovation by allowing diverse data sources to be used collectively, broadening the scope and applicability of AI solutions.
Privacy on Public Blockchains:
The immutable and public nature of data stored on a blockchain is a defining and often beneficial feature, this feature can become an issue in transactions that require privacy. For more on the challenges and opportunities presented by the transparent nature of public blockchains, check out our previous Blockchain Brief on blockchain privacy.
Private Transactions: ZKPs are used in blockchains for privacy-preserving transactions that keep details like monetary amounts and participant identities private. ZKPs can keep this information private while verifying that users are not participating in illicit activity or transacting with sanctioned entities.
Private Exchange and Settlement of Digital Assets: ZKPs ensure the privacy of transactions and participants during the exchange of digital tokens, such as equity tokens. This is crucial for both centralized and decentralized exchanges, allowing for efficient and private asset settlement. As more real-world assets (RWAs) like real estate and high-valuable collectibles are tokenized and more transactions involving RWAs are conducted on-chain, this efficiency and privacy is paramount.
Scalable Ethereum Layer 2 Rollups:
ZKPs, through methods like zero-knowledge rollups (ZK-rollups), are highly secure and scalable Layer 2 solutions on Ethereum that facilitate faster, more efficient transactions. If you need a refresher on the difference between a Layer 1 and Layer 2 blockchain before we dive deeper into the current state of ZK rollups, be sure to check out our previous Blockchain Brief where we break it down.
Oracle-Enhanced Verifiable Computations:
In decentralized oracle networks, ZKPs can be leveraged for verifiable computations related to off-chain data points. As explained in a previous Blockchain Brief, blockchain oracles provide smart contracts with access to external data and computation results. Using ZKPs in this context ensures the integrity and confidentiality of the data processed by the oracles. Relevant off-chain data points include:Financial Data: Verifying stock prices or foreign exchange rates for use in smart contracts without revealing sensitive market data.
Supply Chain Information: Confirming the authenticity of supply chain events like product manufacturing or shipping status without disclosing proprietary details.
Real-World Events: Maintaining data privacy while validating outcomes of real-world events, like sports results and election outcomes, that impact contract executions Weather Data: Securely integrating weather information for agriculture-related smart contracts without full data exposure.
Internet of Things (IoT):
Securing IoT device communications and reducing data transmission through ZKPs enhances both privacy and efficiency in interconnected devices.
Supply Chains:
ZKPs can authenticate compliance and origin in supply chain management without revealing sensitive data, which is crucial for businesses that wish to keep supplier information confidential.
These diverse applications across sectors underscores the significance of ZKPs in addressing privacy and security challenges in the digital age by offering robust solutions for secure, private, and efficient digital interactions.
A Deeper Dive into Decentralized Digital Identity and Securing Personal Identifiable Information
Zero-knowledge proofs present a unique solution to address the urgent need for safe and secure methods of digital identity verification and storage of personal identifiable information (PII). ZPKs minimize the exposure of PII by enabling the verification of essential information while keeping other personal details hidden, thereby fortifying systems against data breaches and preserving individual privacy. This is crucial in scenarios ranging from social media interactions to financial transactions.
Consider the routine action of showing an ID to purchase alcohol – you inadvertently expose more information than necessary.
The cashier only needs to verify two things: that the id is a valid, government issued id and that you are 21 or older. Yet you are revealing your full name, home address, birthdate and age, sex, height, weight, eye color, and driver’s license number. These pieces of information are individually harmless, but when they are used together, there is potential for abuse, not to mention a violation of an individual’s right to privacy.
This scenario is amplified online, where the sharing of social security numbers, financial details, and personal activities often leads to privacy invasion and increases the risk of cyberattacks. Consider the following three examples representing massive misuse and exposure of sensitive user data, and how Zero-Knowledge Proofs (ZKPs) could have potentially mitigated these incidents:
1. Cambridge Analytica Scandal:
Real-World Incident: A third-party app, This Is Your Digital Life, allowed unauthorized collection of personal data from millions of Facebook users. Cambridge Analytica used that data for political advertising, including in the 2016 U.S. presidential election campaign. This breach raised significant concerns about privacy and the influence of social media on politics.
ZKP Mitigation: ZKPs could have been used to verify the specific user attributes (e.g., age, location, political interests) necessary for academic research or targeted advertising without revealing or transferring the underlying data. That way, Facebook could confirm specific demographics of its user base to third parties without exposing individual user data.
2. T-Mobile Data Breaches:
Real-World Incident: Caused by a misconfigured API, T-Mobile experienced a data breach that exposed the personal data of 37 million customers. The stolen data was used by cybercriminals for sophisticated phishing, vishing, and smishing attacks.
ZKP Mitigation: ZKPs could have been employed in T-Mobile’s data handling processes to validate user identity and credentials without exposing user data. For instance, a customer service application could verify that a person is a T-Mobile customer and check their plan details without the API accessing the full range of personal information stored in the database.
3. Equifax Data Breach of 2017:
Real-World Incident: This breach, one of the largest in U.S. history, compromised the personal information of about 147.9 million Americans, as well as millions of British and Canadian citizens. Hackers accessed names, Social Security numbers, birth dates, addresses, and in some cases, driver's license numbers. The breach was deemed "entirely preventable" by the House Committee on Oversight.
ZKP Mitigation: If ZKPs had been used, Equifax could have allowed third-party services to confirm individuals' creditworthiness or identity without accessing detailed personal data like Social Security numbers or birth dates. For example, a lender could verify if a person's credit score falls within a specific range without Equifax needing to share the actual credit report.
In each of these scenarios, ZKPs would have offered a way to validate and prove necessary information without exposing data, thus significantly reducing the risk of data breaches and misuse of personal information. Implementing ZKPs requires rethinking how systems interact with personal data and moving towards a model where verification is possible without data exposure. The adoption of ZKPs benefits individuals whose personal data remains protected and private as well as businesses, which can avoid the substantial financial and reputational damage often associated with such data breaches. By enhancing data security and integrity, ZKPs provide two key benefits: they help in building trust among consumers and simultaneously safeguard businesses against the risks and liabilities of data mismanagement.
The Current State of ZK-Rollups
As noted earlier, the two developments that have allowed ZKPs to finally transition from academic concept to real world use have been increased power and availability of compute and blockchains. While smart contract platforms like Solana and Avalanche have seen significant growth in daily active users, ecosystem development, and capital inflows in 2023, Ethereum and its Layer 2 blockchains remain the dominant smart contract ecosystem. As of 1/17/24, Ethereum and its L2 chains have a combined market cap ($322B) 1.6X higher than every other smart contract platform combined and 72% ($78B) of total value locked (TVL) in decentralized finance (DeFi).
As discussed in our previous article, breaking down the difference between L1 and L2 Ethereum, L2 blockchains are pivotal in enabling Ethereum to scale. As a quick refresher, there are two types of rollups: Optimistic rollups and ZK rollups. While ZK rollups are broadly considered the superior technology, the increased complexity of ZKP development, higher compute requirements, and layer 1 technological limitations have given optimistic rollups a strong first mover advantage. The top three optimistic rollups, Arbitrum, Optimism, and Base, currently hold 81% of the total L2 market share.
The tide has begun to change, as several notable ZK-rollups went live in 2023. As of 2024, the development and implementation of ZK-rollups on Ethereum have advanced significantly, showing promising growth and innovation in various projects.
You can find a list of notable Optimistic and ZK-rollups in the previous Blockchain Brief, Understanding Ethereum's Layer 1 and Layer 2.
While optimistic rollups collectively represent the majority of L2 activity and TVL, ZK-rollups are quickly picking up steam and are expected to overtake their optimistic cousins. Notably, over the last 30 days, zkSync Era has processed more transactions than Arbitrum, Optimism, Base, and Ethereum Mainnet. Both Arbitrum and Optimism have both stated they expect to transition their respective chains to ZK-rollups eventually.
The key takeaway here is that the myriad of real-world use cases enabled by ZKPs requires a compatible blockchain. That compatibility now exists with the advent of ZK-rollups. This, in conjunction with the upcoming Dencun upgrade to Ethereum set to launch in the coming months (currently live on testnet as of 1/17/24), which will dramatically decrease transaction costs for L2s while bringing enhanced security, will not only allow all of these use cases to deploy but also allow them to do so cost-effectively at scale.
Why Policymakers Should Pay Attention
We live in an increasingly digital world. As individuals, private organizations, and governments move more of their interactions online, ever more of our most sensitive data is put at risk of data breaches and hacks. At a time when faith in public institutions is at an all time low and global ransomware attacks are predicted to exceed $265B by 2031, the need for technologies like ZKPs to restore faith in the integrity of elections, bolster online security, and protect an individual's right to privacy is abundantly clear.
There are many applications of ZKPs that can help balance privacy, security, and regulatory compliance. This is especially relevant in the context of global and interoperable blockchain technology, where ZKPs can bolster blockchains ability to enhance the resilience and accessibility of critical infrastructure and international cooperation.
Challenges in the US Regulatory Environment
The US’s ambiguous regulatory environment is currently driving blockchain and ZKP-based technology development overseas. A lack of clear guidelines is causing a brain drain in this sector, with developers and companies opting for more predictable and supportive environments in countries like Japan, Singapore, and parts of the EU. This regulatory uncertainty is particularly impactful in the area of decentralized finance (DeFi), where traditional frameworks struggle to adapt to the novel nature of blockchain technologies.
ZKPs: A Potential Solution to DeFi KYC
Within this challenging regulatory landscape, the proposed digital assets tax reporting regulations in the bipartisan infrastructure bill have defined decentralized entities as brokers. This definition is incompatible with the decentralized nature of these entities' protocols, as many lack a central entity capable of conducting traditional Know Your Customer (KYC) practices. The significant compliance and cost burden for startups could drive talent offshore, potentially cutting off US users from innovative products. Furthermore, the decentralized structure of these platforms allows users bypass front-end data collection and access the backend directly.
Zero-Knowledge Proofs (ZKPs) emerge as a feasible solution in this context. They enable the verification of essential information and reporting through smart contracts, all without the need to store sensitive data. By using ZKPs, compliance with regulations can be achieved while maintaining a balance between privacy, security, and innovation. This technology not only minimizes the risk of spreading sensitive data across multiple parties but also alleviates the compliance burden for emerging companies. In turn, this fosters a healthier environment for technological advancement in the US, particularly in the burgeoning field of DeFi.
Conclusion: A Call to Action for Policymakers
As we navigate the complexities of the digital age, Zero-Knowledge Proofs emerge as a powerful tool in preserving privacy while ensuring security. Policymakers face the challenge of fostering an environment that supports the growth and application of ZKPs while protecting consumer rights and privacy. Regulations should be forward-thinking, adaptable, and balanced to encourage innovation in this space. The harmonious blend of privacy and security that ZKPs provide is not just a technological advancement but a step towards a more secure and equitable digital world.
In conclusion, the potential of ZKPs is vast and multifaceted. From securing online transactions to safeguarding AI and machine learning models, they offer a solution that strikes a balance between privacy and security. As machine learning continues to evolve and our digital footprints expand, the role of ZKPs in protecting our digital identities is critical. It is imperative for policymakers to understand these dynamics and create a regulatory environment that nurtures innovation, protects consumers, and maintains the integrity of the financial system. By embracing the possibilities of Zero-Knowledge Proofs, we can pave the way for a more secure, private, and equitable digital future.
