Augmented Reality (AR) Experiences of Interactive Media Art

Augmented Reality presents an extremely specific medium for artwork, and work that considers the formal qualities of this medium can make incredibly meaningful comments about the boundaries (or lack thereof) between the real and the digital. An artwork could have limited interactivity in and of itself, such as an image or animated 3D object, but with the help of spatialized AR it becomes an immersive experience. Spatialized AR is the illusion of a digital object occupying real world space, with the interaction facilitated through the user’s movement in space around the artwork.

Because mobile technology is typically the platform on which AR is happening, the same constraints apply as for mobile experiences of interactive media art.

• AR provides many benefits for hybrid exhibitions that happen both inside and outside a gallery, especially if the project involves considerations of space and place.
• Combining AR with geofencing creates extremely unique contexts for making work that responds to a particular location. Geofencing creates a boundary for virtual content that corresponds with real-world coordinates, meaning that the artwork could only be accessed in an intended location.
• AR can also work in tandem with physical triggers, in the form of images or printouts, that can accompany the artwork. These can be combined with an organization’s existing advertisement or promotional strategies by placing the triggers on flyers or gallery takeaways, or outside on billboards and posters.

• The limitations and challenges of developing broader mobile experiences, particularly optimization and publishing, apply to AR for mobile experiences. In addition, it is currently more challenging to create a mobile browser AR experience than a custom application (more on WebXR below).
• A decision in implementing AR is whether it will be trigger-based or not. Trigger-based AR is activated by a specific defined image (or object) and can (optionally) follow that image’s position in the real world. AR graphics that somehow integrate with their trigger create even more cohesive experiences. Non-triggered AR, however, has the advantage of not requiring the user to have the trigger for it to be activated.
• For developing AR experiences, social media platforms have user-friendly tools for creating AR “filters” on their applications. Snapchat has Lens Studio, while Meta/Facebook/Instagram has SparkAR—either is especially useful for creating AR that interacts with the user’s face or body, due to the selfie-centric nature of those platforms. Publishing on social media has major drawbacks in terms of control, as the end result is owned by the platform, not the artist, and the development environments themselves feature major limitations.
• The most robust tool for creating AR is the game engine Unity3D, which features plugins to handle the various frameworks for AR across Android and iOS devices, so the design need only happen once.
• Advances in WebXR are making browser-based AR experiences increasingly possible, which poses exciting possibilities for future website based AR projects. At time of writing, WebXR plugins for Unity can make trigger-based AR experiences happen. WebXR publishing circumvents the traditional publishing platforms reliant on Apple, Meta, or Google, by running XR directly in the browser. Publishing XR directly in the browser means that audiences don’t need to download a custom application, that the project would automatically be supported across Apple and Android devices, and that the publishing challenges that arise when working with commercial app stores can be avoided. At the time of writing, this workflow is still relatively new but we encourage its future exploration.