Amazon S3

Amazon S3 (Simple Storage Service) is a cloud storage service run by Amazon Web Services (AWS) that stores data as "objects" inside containers called buckets. Launched in the United States in March 2006, it was one of the first widely used public cloud services. Organisations of every size use it to keep documents, images, backups, logs and large datasets, paying for the amount stored and the data moved rather than buying their own storage hardware.

S3 is a core piece of internet infrastructure. Banks, payment companies, crypto exchanges and data providers rely on it for archives, statements and analytics, and many websites serve images and files from it. Its interface became a de facto standard: many other providers, including some decentralized storage networks, offer "S3-compatible" access so that software written for S3 works with them too.

Think of S3 as a vast self-storage warehouse that you rent by the shelf. You put a box in, give it a label, and can fetch it back from anywhere with an internet connection. You never see the building, and you never worry about the shelves wearing out; the company keeps several copies of each box in different rooms so one fire does not destroy it.

You decide who has keys to each box and whether anyone can look inside. Most problems with S3 come not from the warehouse losing things, but from customers leaving the door open by setting the wrong permissions.

Each object consists of the data, a unique key (its name) and metadata. Objects sit in buckets, which belong to an AWS account and a chosen geographic region. Programs store and retrieve objects through a web API. S3 is designed for 99.999999999% ("eleven nines") durability by storing data redundantly across multiple facilities in a region. Different storage classes trade cost against access speed:

• Standard. For frequently used data, with fast access.
• Infrequent access classes. Cheaper storage with retrieval fees, for data used now and then.
• Intelligent-Tiering. Automatically moves objects between tiers based on use.
• Glacier classes. Archive storage for records kept for years, with retrieval times from milliseconds to hours depending on the tier.

S3 offers encryption, access policies, logging and versioning. Since January 2023, new objects are encrypted by default, and since April 2023 new buckets block public access by default, changes made after years of data leaks caused by misconfigured buckets. Access is controlled through identity policies and bucket policies, often designed around role-based access control. The Object Lock feature stores data in a write-once, read-many form that cannot be deleted or changed during a retention period, which financial firms use to meet record-keeping rules such as US broker-dealer requirements and internal data retention schedules.

Financial institutions use S3 for statement archives, customer documents gathered during onboarding, transaction logs, risk models and data lakes for analytics. Crypto businesses use it to store blockchain data snapshots, which can help a new full node start faster, as well as market data and website assets. Many NFT collections have stored their images and metadata on centralised servers such as S3, which means the token on the blockchain points to a file that a company could change or delete. This is one reason some projects prefer decentralized storage such as the InterPlanetary File System, Filecoin or Arweave.

S3 failures can have wide effects. On 28 February 2017, an error during maintenance in AWS's US-East-1 region took part of S3 offline for several hours, disrupting many websites and apps that depended on it. Security incidents have more often involved customers than AWS itself: researchers have repeatedly found sensitive data, including bank and identity records, in buckets left open to the public. The 2019 Capital One breach, which exposed data on about 100 million people in the United States, involved an attacker exploiting a misconfigured firewall to reach data stored in S3.

For regulated firms, relying on S3 creates third-party risk and concentration risk, since many institutions depend on the same few cloud providers. Supervisors in the EU, UK and elsewhere now treat major cloud providers as potential critical service providers and expect firms to plan for outages and exit. Costs can rise unexpectedly through data transfer and request fees. Good practice includes encryption, least-privilege access, logging, backups in another region or provider, and regular checks for public exposure. For crypto projects, storing key data on S3 also undercuts claims of decentralization, because a single company can remove or alter it.

• Amazon S3