The Invisible Hand of Web 3: How Profit-Driven Node Operators Ensure the Quality of Ethereum Swarm’s Decentralized Cloud Service
Although imperfect, a free-market economy is still the best economic system. Its success is based on a straightforward rule: be selfish! If this simple condition is met, meaning everyone is selfish and strives to maximize their profit, there is no need for any central regulation, as the ‘invisible hand’ of the market can optimally distribute scarce resources.
This principle also drives the Ethereum Swarm decentralized storage solution. In the case of Swarm, the system’s currency, the BZZ token, plays a crucial role. BZZ is not only the fuel of the system, but the ‘BZZ economy’ also determines the network’s topology and the distribution of data.
Just as the core element of a free-market economy is the rational consumer and producer, who always strives to maximize their benefit, the foundation of Ethereum Swarm is the rational node, which aims to maximize the ‘mined’ BZZ tokens.
In the following sections, I will examine in detail how Swarm’s incentive system works and how this system optimizes the topology and the distribution of data.
Bandwidth incentive
Unlike many other systems, with Swarm, we have to pay the serving nodes for the bandwidth used. At first, this philosophy might seem strange, as we are used to not paying for incoming data traffic on the web. However, anyone who has ever run something on a cloud service knows that bandwidth has a price, and if we handle large amounts of data traffic, this price can be a significant part of the costs. Users don’t see this because providers hide it. For example, when we use Facebook, Meta pays for the data traffic directed toward us, but the cost is covered by the advertisements that appear between posts. The service is free because you are the product!
Swarm is honest! There are no ads, and no one monetizes our data, but in return, we have to pay for the bandwidth.
This approach opens up entirely new business opportunities for content providers. If someone wants to launch a service that requires high bandwidth, such as a streaming service, a significant initial investment is needed to set up and maintain the infrastructure. However, with Swarm, the infrastructure is already in place, the system automatically scales, and the bandwidth costs are covered by the users. Therefore, the content provider only needs to pay for storage costs, which is a minimal investment. Additionally, the network grows along with the user base.
While in a traditional streaming service, accommodating an increased number of users can be a challenge, in the case of Swarm, every new user can also become a provider node, further increasing the total bandwidth of the network.
Every Swarm node keeps track of how much data it has sent and received from each peer it is connected to. This solution is similar to the Lightning Network’s payment channels.
As long as the difference between the peers in terms of network consumption remains within a certain limit, we have a truly equal give-and-take situation and no compensation is needed. If the balance exceeds a threshold, the node must pay for the data traffic. In Swarm, this does not happen through a blockchain transaction. Instead, the node issues a cheque. A cheque is a digitally signed commitment that works similarly to real cheques. Instead of the bank, Swarm cheques can be cashed on the blockchain through the checkbook contract. Every node has such a contract, where the cheques issued by the node can be redeemed by its peers. Since cheques can be redeemed in bulk, this helps to minimize the gas cost of transactions.
Swarm uses forwarding Kademlia. This means that when a node needs some content, it requests it from a connected peer. If that node doesn’t have the content, it also requests it from a connected peer, and this chain continues until the source of the content is reached. Due to the topology of the system, these chains are typically not long. If a node does not have the requested content, it must pay to acquire it, meaning it can only keep a portion of the amount paid by the requesting node. Therefore, if a node wants to optimize its profit, it is beneficial to cache popular content, so that when someone requests that content, the node can keep the full amount paid for it. Swarm’s forwarding Kademlia does not only mean that requests are relayed through successively closer peers to where the requested chunk is found, but also that the chunk is then relayed back the same route to the originator. This step allows the nodes en route to cache content prompted by retrieval, effectively implementing the automatic spreading of popular content.
This simple mechanism makes Swarm an ideal Content Delivery Network, as popular content will be more widely available, retrieved with lower latency, and potentially even persist despite not being insured through storage incentives. This solution relieves the source and ensures the optimal use of bandwidth. While the task of storage for each chunk is delegated to nodes in the neighborhood of the chunk, caching will relieve the nodes in this neighborhood from having to serve each and every request. This does not only mean that Swarm is resistant to DDoS attacks but also that it is actually anti-fragile, i.e., increases resilience under stress.
The bandwidth incentive ensures the optimal distribution of data in the long term, making Swarm an ideal Content Delivery Network.
Storage incentive
If we want to store data in Swarm, we need to attach a postage stamp to each data chunk. The postage stamp works similarly to a real postage stamp. If we want to send a letter through the postal service, we need to put a stamp on it. This is how we pay for the postal service. If the letter doesn’t have a stamp, the postal service won’t deliver it, as the cost of delivery hasn’t been paid. Similarly, in Swarm, if a chunk doesn’t have a postage stamp, the nodes have no incentive to store it.
Swarm nodes are organized into neighborhoods (I have an entire article about the neighborhood system), and all content is stored in its corresponding neighborhood. Storage nodes participate in a lottery, where the lottery ticket is the postage stamp and the associated chunk of content. A smart contract periodically selects a neighborhood. If the nodes in the winning neighborhood can prove (by Merkle proofs) that they are storing a sufficient amount of relevant content, they receive the reward. If a node is not storing the right content, it will be unable to generate the Merkle proof and will lose out on the reward.
The postage stamp is a digitally signed proof of pre-payment created by the user who wanted the data stored (uploader). To create a valid postage stamp, one needs a postage batch. The postage batch can be purchased from a smart contract. When a Swarm node wins the lottery and wants to claim its reward, this is only possible if the postage stamp belongs to a valid postage batch. Postage batches have an expiration date, so when that time passes, the postage stamp also becomes invalid. In such cases, the postage batch must either be renewed, or the content must be re-uploaded with a new postage stamp.
In summary, the mechanism works as follows:
- Users purchase postage batches. The funds collected this way flow into a common pool. The content of this pool is distributed through lottery draws. The postage batch has an expiration date, starting from the moment of purchase. A postage batch is actually a hot wallet from which the storage rent is deducted. The batch expires whenever its balance runs out.
- The user uses the postage batch to generate postage stamps, which are assigned to the data they want to upload. The amount of data that a particular postage batch can cover is determined by the user at the time of purchase (ex.: a batch for stamping 4G of data).
- The user uploads the data to the network, which the nodes forward to the appropriate neighborhood, where the nodes store the data.
- At certain intervals, the redistribution contract selects a neighborhood where the nodes can claim a portion of the pool’s contents as a reward, but only if they can prove with a valid postage stamp and Merkle proof that they are indeed storing the data.
This system ensures the fair distribution of postage batch revenues among the nodes in the long term.
Since the reward is distributed equally among the nodes in the neighborhood, if a neighborhood is empty or contains few nodes, the reward is larger there, encouraging other nodes to relocate to that neighborhood to maximize their profit. If there are too many nodes in a neighborhood, the reward per node decreases, making it more beneficial to try in other neighborhoods. The equilibrium is reached at around 4 nodes, which provides sufficient redundancy for secure data storage. (Further redundancy can be achieved using erasure coding, if necessary.)
The storage incentive ensures the ideal topology of the network in the long term, which, while providing sufficient redundancy, covers the address space with storage capacity in the most optimal way.
Ethereum Swarm is a great example of how a well-designed incentive system can enable the functioning of a decentralized protocol through crypto-economics. Even in such systems, the ‘invisible hand’ emerges, capable of shaping and operating the system optimally. And all it requires is one simple rule: be selfish!
If you are interested in learning more about how Ethereum Swarm’s incentive system works, you can read about it in detail in the Book of Swarm.
I would like to thank Viktor Trón, the architect of Ethereum Swarm, who greatly helped me in the creation of this article.