Decentralized Storage

Decentralized storage is a method of keeping digital files on a network of many independent computers, run by different operators around the world, instead of on servers owned by a single provider. Files are usually split into pieces, encrypted and copied across several machines, and the network uses cryptographic proofs to check that operators are really holding what they are paid to store. Developers use it to host website content, application data, backups and the images and metadata behind NFTs.

Decentralized storage is an alternative to conventional cloud computing services, which are controlled by a handful of large firms. It sits alongside blockchains rather than inside them: storing large files directly on a blockchain is extremely expensive, so most projects keep only a short reference, such as a content hash, on-chain and the file itself on a storage network. Well-known systems include the InterPlanetary File System (IPFS), Filecoin, Arweave, Storj, Sia and Swarm.

Imagine storing your family photos not in one warehouse, but by giving sealed, numbered fragments to dozens of neighbours across town. No single neighbour can see the full pictures, and if a few move away, the others still hold enough fragments to rebuild everything. Every so often you knock on doors to check that people still have the pieces, and you pay those who do.

That is the basic idea. Instead of trusting one company to keep files safe and online, the work is shared among many independent providers. Software handles the splitting, the checking and the payments automatically, so the user simply uploads a file and later downloads it again, without needing to know who holds which piece.

Most networks combine several techniques:

• Content addressing. Files are identified by a hash of their contents rather than by their location. If the content changes, the address changes, so users can be sure they receive exactly the file they asked for. This is the core of IPFS.
• Splitting and redundancy. Files are broken into chunks and either copied several times or encoded with erasure coding, which lets the original be rebuilt from only some of the pieces.
• Encryption. Many networks encrypt data on the user's device before upload, so operators cannot read it.
• Storage proofs. Operators must regularly prove they still hold the data. Filecoin uses proof of replication and proof of spacetime, while Arweave uses succinct proofs of random access as part of mining.
• Incentives. Operators, often called storage miners or providers, are paid in a network token and may lose collateral if they fail checks.

IPFS, developed by Protocol Labs, is a peer-to-peer protocol for finding and sharing content-addressed files, but on its own it does not pay anyone to keep data. Files stay available only while someone chooses to host, or "pin", them. Filecoin, also from Protocol Labs, adds a marketplace on top, in which users pay providers to store data for agreed periods, recorded on the Filecoin blockchain.

Arweave takes a different approach, offering storage intended to be permanent for a single up-front fee, part of which goes into an endowment meant to pay for storage far into the future. Its data structure is called a blockweave. Storj, whose earlier product was named Tardigrade, offers storage compatible with common cloud tools, with files spread across independent storage nodes and paid for in conventional currency or its token. Swarm, linked to the Ethereum community, aims to provide storage and messaging for decentralized applications.

Decentralized storage is used to host the media and metadata of NFTs, so that a token does not point to a server that might later go offline. It supports archives of public records and research data, front ends for decentralized applications that are harder to take down, and backup services. Supporters stress its censorship resistance, since no single company can delete content, and the potential for lower prices from a competitive market of providers.

Decentralized storage faces several practical hurdles. Retrieval can be slower and less predictable than from a major cloud provider, and tools are less mature. Prices paid in volatile tokens can be hard to plan around, and network subsidies have in some cases made usage look larger than real paying demand. Permanent storage raises legal problems: illegal or harmful content cannot easily be removed, and data-protection laws such as the right to erasure sit awkwardly with immutable networks. Some studies found that NFT files supposedly kept on decentralized networks were in practice served by a few gateways or had lost their hosts. Storage remains only as durable as the economic incentives behind it, so the long-term guarantees of each network depend on its token and its operators continuing to be viable.

• IPFS Documentation
• Filecoin
• Arweave