Cracking the Code of Pompeii With Artificial Intelligence

Cade DuPont, Tiffany Montgomery, David Fredrick

Villa of the Mysteries, Pompeii, with capstone team (from left): Cade DuPont, Shaun Bobbitt, Bethany Terry, Rafael Villaroel.

A capstone project by four seniors in computer science at the U of A -- Shaun Bobbitt, Cade DuPont, Bethany Terry and Rafael Villaroel -- is opening a new window on the ancient Roman city of Pompeii. Buried by volcanic ash during the eruption of Mt. Vesuvius in 79 CE, the unique preservation of its theaters, temples, streets, shops and houses has given Pompeii a central role in reconstructing daily life in the early Roman Empire, more than 2,000 years ago.

Working with associate professors David Fredrick of the Department of World Languages, Literatures and Cultures and Alex Nelson of the Department of Computer Science and Computer Engineering, the student team is applying AI techniques, including convolutional neural networks (CNNs) and large language models (LLMs) to the site's archaeological data, allowing researchers to see this data in ways that simply were not possible before.

One of the world's best known archaeological sites, Pompeii attracts 2.5 million visitors per year, and study of the site has a long history. Excavations began in the 18th century and continue to the present day. Much, however, remains to be discovered about Pompeii, partly because our understanding  of the city remains grounded in Roman literary sources, rather than archaeological data.

"What we think we know from upper-class texts about how Romans lived, thought and felt," Fredrick says, "is fundamentally challenged by the material evidence from Pompeii."

The problem, he notes, is coming to terms with Pompeii as "a city of data." A lot of data. The number of images and captions for wall paintings and mosaics in the 10-volume Italian encyclopedia Pompei: pitture e mosaici (PPM) runs to more than 15,000, a corpus that could not be comprehensively and intelligently searched until the work of students at the U of A.

"We simply do not know the spatial distribution of art in Pompeii, what styles, motifs and themes are found where," Fredrick says, "because it is impossible for this to be tracked by a human being either turning the pages of PPM or walking through the city, even with a thousand journeys. And not knowing what is actually where, we could not ask why."

Enter PPM Explorer, or PPMx. In its first form, PPMx was created as the dissertation project of Cindy Roullet, working with Fredrick, then the director of the Tesseract Center for Immersive Environments and Game Design; Rhodora Vennarucci, now at Denison University; and her dissertation director in computer science, professor John Gauch. Roullet received her doctorate in 2020, but with COVID, the Tesseract Center closed and PPMx was put on hold.

In 2022, however, Nelson began aligning senior capstone projects in computer science with Fredrick and his work with game design and the Unity3d game engine. This coincided with the birth of the World Languages and Digital Humanities Studio (located in J.B. Hunt Center 207) and the founding of the Digital Humanities Collective (JBHT 253-55), an interdisciplinary humanities space supported by the departments of English, History and World Languages. These pieces came together in spring 2024, when the capstone team was matched up by Nelson with Fredrick to restart PPMx.

Roullet played an essential role. Joining meetings by Zoom from southern France, where she is an independent game developer, Roullet guided the team through her use of AI in building the first version of PPMx. Roullet's approach used visual AI (CNNs) to track similarities and patterns within the wall painting images in PPM, while using language AI (Natural Language Processing - NLP) to track similarities and patterns within the Italian captions. The combination of the two allows specific themes (e.g. how many Minervas, and where?) as well as compositional similarities to be plotted across the map of Pompeii. "In their houses, Pompeians inhabited a world of myth, but also a world of visual and spatial rhetoric, and Roullet's approach opened the door to exploring them together," Fredrick says.

Coming to grips with Roullet's approach, the team realized that the surge in AI development since 2020 could mean dramatic improvements in speed, capacity and accuracy. As the team notes, "This would not have been possible without Cindy's foundational contributions to the project. While we are focused on integrating newly discovered artificial intelligence concepts, her approach of comparing feature vectors to obtain meaningful results from user queries remains at the core of the application's functionality."

Building on Roullet's work, the team turned to cutting-edge Microsoft Azure tools, including Cosmos (NoSQL database), Azure AI and Azure Machine Learning to produce an updated version of PPMx, with significantly greater speed and accuracy in searching PPM's images and captions. The team found that, "Services provided by Microsoft Azure have not only made PPMx more efficient for a better user experience, but have also accelerated our development timeline and opened doors for further feature integration."

The team also adapted an open-source GIS (Geographical Information System) map of Pompeii to allow users to see their search results in the correct region, city-block, doorway and room. Crucially, while the first version of PPMx was only accessible by sharing a remote desktop, the new version of PPMx is hosted on the web, which will allow global access by scholars and the public. Roullet is enthusiastic about how the Senior Capstone Team has taken her dissertation work to another level.

"The work the Capstone Team has done with PPMx is remarkable," Roullet says. "Seeing the project come back to life with even greater potential, thanks to their integration of new technologies, has been incredible. Their efforts have enhanced PPMx's ability to reveal connections and patterns within PPM's rich art historical data, and I'm super excited to see where this renewed potential will take the project in the future!"

The PPMx project demonstrates how emergent AI tools, when paired with motivated students at the U of A, can drive research breakthroughs in the digital humanities. As Fredrick observes, "With PPMx, researchers on Pompeii and Roman art will be able to assess for the first time the location of myths, themes and decorative elements holistically within Pompeii, to pursue how these themes work visually and spatially with each other, and how they shaped patterns of movement."

Fredrick and Nelson are working to secure additional funding for the Azure tools, to allow wider testing of the application by a select group of experts in Roman art and archaeology, some of whom tested Roullet's version back in 2020.

"They were already really impressed with Cindy's (Roullet's) PPMx," Fredrick says, "and I expect that they'll be stunned and delighted by this new version, which will fundamentally change how they can address research questions for Pompeii."

Using the game engine Unity, the Virtual Pompeii Projects is moving forward with the development of an immersive, interactive visualization environment for PPMx, to provide a more embodied experience of how the wall paintings "work" in their original spatial context.

For more information on PPMx and the Virtual Pompeii Project, including how to participate as a student, contact project director, David Fredrick, at dfredric@uark.edu. For information on Game Design and Digital Humanities curriculum, contact Fredrick or Curtis Maughan, director of the World Languages and Digital Humanities Studio, at cmaughan@uark.edu.