In the art world, it’s hard to say whether folks are legitimately thrilled, frightened, or simply perturbed by the overwhelming rise of AR/VR and AI technologies.
Certainly, these sentiments have been explored ad nauseam by both artists and critics over the past decade, and perhaps even more prominently in the past few years. Moreover, the inclination to make art that speaks to ills and tribulations of AI in our society is fairly common. (Take American Artist and Lynn Hershman Leeson’s insightful works about the draconian practices of predictive policing technologies as a recent example.)
As for art institutions, perhaps not surprisingly, they have taken a more optimistic and, of course, pragmatic approach to these burgeoning technologies. Leading the way to creating a happy and symbiotic relationship between art and tech—at least hypothetically—is Microsoft.
Just last month, in conjunction with the release of the HoloLens 2, the company’s latest mixed-reality headset device, Microsoft, in partnership with the Museum of History and Industry (MOHAI) in Seattle, opened the exhibition “Mont-Saint-Michel: Digital Perspectives on the Model,” an immersive, mixed-reality experience of the titular UNESCO World Heritage Site. It was a chance, perhaps, to get a glimpse of what a tech-augmented future might hold in the galleries.
Detail of the digital rendering for the model of Mont-Saint-Michel. © Microsoft – Iconem – Musée des Plans-Reliefs
A New Window on History
The exhibition, powered by Microsoft AI and mixed-reality technologies, takes as its starting point a 17th-century maquette, or three-dimensional map of Mont-Saint-Michel. The 3D military map, made by hand by Benedictine monks and gifted to Louis XIV, is on loan from the Musée des Plans-Reliefs in Paris. It is the first time it has ever left its home country. The 1/144-scale model sits in the middle of a small room of the museum, inside a large custom-built vitrine.
The model is impressive on its own, but to experience the piece through the HoloLens 2 device is worth the visit, not only because of its tech innovations and cool visual effects, but for the effective ways it relays information to the viewer. As a way to learn about history, it surely beats sifting through long and dry wall texts.
A previous version of the exhibition debuted at the Musée des Plans-Reliefs. Apart from the upgrade in the device—the HoloLens 2 is less clunky and more comfortable than its predecessor, and has more interactive capabilities (which this show does not utilize lest it overwhelm new users)—MOHAI’s is not all that different. A docent gets you situated and set up with the HoloLens 2, just as if you’re getting ready for a rollercoaster ride. Then, after a short series of exercises to get you oriented with the device, the program starts.
View of three-dimensional map of Mont-Saint-Michel, ca. 1691. © RMN – Grand Palais (Musée des Plans-Reliefs) / Adrien Didierjean
Lasting roughly 15 minutes, which is not long but not short either, the device overlays holographic images on top of the 3D model, with audio detailing the history of the object and the site itself. At one point, the device overlays the entire gallery space with a holographic copy of the abbey of the church, digitally transporting you into the architectural space. The experience overall is enjoyable and informative.
Augmenting the Narrative?
This is not the first time Microsoft has partnered with art institutions and artists using its AI tech. There was, for instance, the artist Mel Chin’s Wake and Unmoored (2018), a two-part mixed-reality experience (using HoloLens 1) that envisioned a future where New York City was submerged underwater due to climate change. And there is also Microsoft’s continued partnership with MIT and the Metropolitan Museum of Art to create new ways for audiences to experience the Met’s collection through AI.
Rendering of the rebuilt USS Nightingale in Mel Chin’s Unmoore. Image courtesy of Mel Chin.
In Chin’s work, the subject matter was imaginative yet bleak, which is not, perhaps, the best demo material if what you are looking for is happy stories about the technology. Meanwhile, the Met’s initiatives mined personal data to engage viewers with its collection. Given paranoia about tech and privacy, it therefore did not provoke a lot of confidence in the positive impact of AI for its many users.
Perhaps to change the narrative around AI, Microsoft started the AI for Good Program, a “$125 million, five-year commitment to use artificial intelligence to tackle some of society’s biggest challenges.” Microsoft has for some time been using AI to preserve endangered languages around the world, and this past summer, it announced the fourth pillar of this initiative: AI for Cultural Preservation.
Although the Met’s collaboration with Microsoft is part of this initiative, the Mont-Saint-Michel project feels more prescient, using historical and architectural data to digitize endangered historical sites, while also creating interactive experiences for audiences to see firsthand the breadth of these developments.
Just the Beginning
But where does the technology go from here? What’s next after this relatively modest application? More importantly, how can these technologies make lasting impressions across different communities and cultures? And what does that mean specifically for museums and cultural institutions today?
This visualization of Mont-Saint-Michel is interesting as a hint of how AR might enter into traditional museum outreach and education initiatives in the future. But it is only a hint—and the leading edge will be using the technology not just for education, but also “for good.”
Recently, the Knight Foundation awarded five art institutions $750,000 to share in the creation of “experimental projects that explore new ways to engage audiences through immersive experiences.” Knight partnered with Microsoft to allow these institutions to access its AI technology and receive coaching from AR/VR experts. The five organizations are the American Museum of Natural History, New York; the Museum of the Moving Image, Queens, New York; the Museum of Art and Design at Miami Dade College; the Japanese American Museum in San Jose; and the Colored Girls Museum, Philadelphia.
Representatives from organizations selected for the Knight Foundation technology initiative meet and learn from Microsoft teams in Redmond, WA. Photo: Nicole Ryan, courtesy Microsoft.
Interestingly enough, the majority of these projects are noticeably socially and community driven, and pretty exciting—at least in theory. For example, the Japan American Museum will create an AR experience that reveals the complex and multi-ethnic history of San Jose’s Japantown, from WWII-era Japanese Internment to the present. The museum plans to work closely with community members on projects that will take place both inside and outside the museum, bringing AR experience to the streets of San Jose.
Other examples include MOAD’s proposed project with collective Forensic Architecture that will create a meta-exhibition investigating the intersection of art, social justice, and immersive technologies. Meanwhile, the Museum of the Moving Image will partner with the Brooklyn-based AR/VR company Scatter to create a volumetric immersive film about the lives of community members in Queens.
As of now, based on the sample of what’s on show in Seattle, it’s hard to say what these projects might look like in the future, or if they will be successful in their respective missions. Still, the ideas in and of themselves seem quite promising. Ultimately, there’s no way of knowing the impact that they might have on a given community, or how these immersive technologies may have positive effects on the way we consume culture on a institutional level. Perhaps it can convince the most reliable skeptics that, at least, it doesn’t have to be all bad.