Overview
AR Mural is an augmented reality experience layered over a physical mural. When viewed through a mobile browser, the static artwork activates: painted characters animate, interactive elements respond to touch, and hidden details invisible to the naked eye become visible through the camera. No app install is required.
The project explores the intersection of physical and digital art, asking what a mural can be when the wall is only part of the canvas.
Technical Approach
The experience is built on WebXR with image tracking, delivered through a mobile browser. The mural itself serves as the tracking target: Meta Spark image tracking detects the mural’s visual features and anchors the digital layer precisely over the physical artwork regardless of viewing angle or distance.
- WebXR image tracking for markerless AR anchoring to the mural surface
- Animated 2D characters using CSS animation and canvas-based sprite sheets
- Touch-interactive elements triggering reveal animations on tap
- Progressive enhancement: the mural works as static art for viewers without a device
- Zero install friction: the experience loads in a standard mobile browser via QR code
- Sub-3 second load time with compressed asset pipeline for mobile networks
Design Integration
The hardest design problem was ensuring the digital layer served the physical artwork rather than competing with it. The AR additions were designed as extensions of the mural’s visual language, not overlays. Characters that appear painted in the mural continue their actions in the digital layer. Hidden elements use the mural’s colour palette so they feel like they were always part of the piece.
The interaction design is intentionally minimal. A single tap on a painted figure triggers its animation. There are no UI controls, no menus, and no instructions beyond the QR code invitation. Discovery is part of the experience.
Challenges
Image tracking in a browser without a dedicated AR SDK is significantly more constrained than native AR applications. The tracking stability depended heavily on the quality and variety of visual features in the mural. Early versions of the mural design had large flat colour regions that the tracker could not lock onto. The final design incorporated deliberate high-frequency detail in the tracking zones while keeping the artistic aesthetic intact.
What I Learned
The most valuable lesson from this project was about the relationship between physical and digital constraints. The mural could not be changed once painted. The AR layer had to adapt to it. Building the digital experience after the physical piece was finished forced a design discipline that purely digital work rarely requires: you cannot undo it, so you must get it right the first time.
