Augmented Reality (AR)

Augmented reality (AR) is technology that adds digital content, such as images, text, 3D objects or sound, to a live view of the real world. Users experience it through smartphone cameras, smart glasses or headsets that track their surroundings and place virtual elements so they appear anchored in physical space. Unlike virtual reality, which replaces the user's view entirely, AR blends the digital with what is really there.

AR is part of a broader group of technologies sometimes called extended reality, which also includes virtual and mixed reality. It is used in retail, gaming, manufacturing, medicine and education, and it features in visions of the metaverse and in some NFT projects. In finance, AR has been tested for shopping and payments, data visualisation and customer service.

Point your phone at your living room and see a new sofa appear exactly where it would sit, at the right size, before you buy it. Or hold your phone up in a city street and see arrows on the pavement showing the way. That is augmented reality: the real world stays in view, with helpful digital extras drawn on top.

The trick is that the device has to understand what it is looking at. Its camera and sensors work out where the floor, walls and objects are, so the digital sofa stays put as you walk around it.

Early experiments go back to the late 1960s, when the computer scientist Ivan Sutherland built a head-mounted display that overlaid simple graphics on the user's view. The term "augmented reality" is usually credited to Boeing researcher Thomas Caudell around 1990, describing a system to guide workers assembling aircraft wiring. AR reached a mass audience in 2016 with the mobile game Pokémon Go, which placed virtual creatures in real locations. Apple and Google released AR developer platforms for smartphones in 2017 and 2018, and headsets such as Microsoft's HoloLens and Apple's Vision Pro, released in 2024, brought more advanced mixed-reality features.

• Sensing. Cameras, depth sensors and motion sensors capture the environment and the device's position.
• Tracking and mapping. Software builds a map of surfaces and tracks movement so digital objects stay in place.
• Rendering. Graphics processors draw virtual content in real time, matched to lighting and perspective.
• Markers and recognition. Some apps use images or QR codes as anchors; others recognise faces, objects or places.
• Display. Content appears on a phone screen or on transparent lenses in glasses and headsets.

Retailers use AR to let shoppers try on glasses, make-up or clothes virtually and to preview furniture at home, linking directly to e-commerce checkouts. Banks have trialled AR features in mobile banking apps, such as pointing a phone at a house to see its estimated value and mortgage options, or at a shopfront to see offers. Some payment experiments let users pay by looking at a product or a code through smart glasses, combining AR with contactless payments. Trading firms and analysts have tested AR and mixed reality to visualise market data in three dimensions, though such uses remain niche.

Crypto projects have used AR to display NFTs in real spaces, for example placing digital art on a wall through a phone camera, or attaching virtual objects to physical locations that users can find and collect. AR filters and wearables for avatars have been sold as NFTs. Some projects describe AR as a gateway to a blended physical and digital economy in which ownership of virtual items is recorded on blockchains. Many of these projects launched during the 2021–2022 NFT boom and have since become inactive, and the practical demand for AR-based crypto products remains uncertain.

AR devices continuously capture video and sensor data about users and their surroundings, including bystanders, raising data privacy questions and, where faces or eyes are scanned, concerns about biometric data. Overlays could be manipulated to show false information, such as fake prices or payment details, creating new payment fraud risks. Headsets remain expensive, heavy and limited in battery life, and many consumers have been slow to adopt them. For financial services, the main barriers are security, regulation of how payments are authorised, and whether AR adds enough convenience over existing apps.