Virtual Reality for Edutainment
The Making of a Virtual Reality Experience of a World Heritage Site under a Conservation Order
The Hal Saflieni Hypogeum is not completely off limits to visitors. But the site is so delicate, so unique, its subterranean environment so critical for the conservation of the red ochre murals that visitors are limited to a very small section of the 5,000 year old complex, constrained by a gangway to keep feet off the ground. The hypogeum is in perpetual semi-darkness, except for a choreographed spot lighting of areas based upon the guide’s explanation.
The brief for this research project was to bring the Hal Saflieni Hypogeum back to life for all those who have cultural heritage at heart, providing a virtual model that may be explored in its totality and to its minute detail. This work is the result of research carried out by students and academic staff of the Department of Computing at Saint Martin’s Institute of Higher Education.
Understanding humans to adapt the technology
Humans are continuously alerted by sensory stimuli, namely Vision, Audio, Smell, Touch and Taste, each of which trigger diverse reactions from individuals. Humans also have the inner sense of balance, and will react as the body is placed in unnatural angles.
Computer scientists, having recognised the potential of the burgeoning power of hardware and software that makes use of the proliferation of smart devices, have set their eyes on the potential of harnessing this computing power to elevate the human computer interaction to new heights. Hence dawned the mixed reality era.
Mixed Reality
Mixed reality is the use of the substantial power of computing to generate artificial worlds that are communicated to humans through the manipulation of human senses. Massive monitors are used to display stunning computer generated images, supplanting props and artefacts as an addition to a real environment known as augmented reality. In other cases, wholly artificial environments are created through computer generated imagery experienced via an immersive headset to attain virtual reality.
Saint Martin’s Institute of Higher Education has been pursuing this field of study for the past decade, even before mixed reality moved into mainstream. With the introduction of its BSc (Hons) Creative Computing in 2006, the institute sowed the first seeds towards what was to become the main growth area in the computing industry. In the early years, few understood what the concepts behind creative computing really meant, but the institute continued to research the area and spawned specialisations such as virtual and augmented reality, user experience and further development of serious games and for entertainment.
The use of mixed reality in cultural heritage
Whilst it was evident that the Institute’s students were mostly interested in the computer games for entertainment industry, the faculty at SMI decided to take a more serious attitude to their research and focused on finding new technology that may portray the very rich variety of cultural treasures in Malta, aiming to educate, entertain and help in the conservation efforts of cultural assets whilst allowing their extended studies and monetization. After all, teaching young scientists requires that academics have experience through research and publication of findings in their arsenal, and not simply passing on what is readily available in text books. This particular area of study is quite unique worldwide.
Preparing the visual
The starting point of the Hal Saflieni Hypogeum virtual reality experience was the 3D documentation of the site financed through the Norwegian Financial Mechanism grant and concluded in 2012. This documentation is carried out through laser scanning from an identifiable central position in the labyrnth of rock hewn chambers. The result is a point cloud that was imported into Unity® Game Engine to develop a real environmental model of the hypogeum.
Working on the Audio
The experience necessitated the introduction of sound that was representative of the environment; sounds that were contextualised within the complex, with echoes and reverberations that mimic the acoustics of the place and therefore sounds that assisted in understanding the aural context of the site to immerse the virtual visitor.
The Audio Profile recording locations
Given the low frequencies of resonance reported in the literature, the sounds emitted for recording were male basso voice chants and a 16–20kHz sweep. Tests were carried out until there was reverberation. The resulting resonance was… beyond description.
Making the short 360° VR film ‘The Awakening’
Every film-maker aims to excite people in unimaginably original ways; triggering emotions through as many of the senses as is possible. Film has been technically relegated, for many decades, to the use of two main senses, those of vision and hearing. But here one must stop and think. Film has come a long way over the past century. The use of technology has enabled not just the facilitation of the use of sound and visuals in much more elaborate ways (through 3-dimensional specials effects (FX) and elaborate sound mixing), but has also brought forth particular elements which were unheard of prior to the last couple of decades. IMAX providing a very large image tends to provide some immersion, as also 3D films that ask viewers to wear disposable glasses and have the effect of objects and characters moving towards the viewer out of the screen. Entertainment parks, such as Disney’s Epcot have introduced 360° theatres, warning viewers to hold onto railing because the surrounding images will trick the brain that the theatre is moving.
The culmination of the 360° Hal Saflieni Hypogeum VR project — ‘The Awakening’, was meant to bring about an experience which could be viewed in the comfort of one’s home, similar to any other film experience, yet with the highlight of virtual reality (VR), without the shackles of heavy and expensive computing equipment for the user.
The director, using the Oculus Rift® headgear was able to explore with the cameraman, and develop the scenes that were to be selected for the short film. Thereon a camera track and dolly system was introduced in the VR model, thus ascertaining that the camera always followed a pre-set path around the hypogeum without deviating from what the creative director wished to communicate. This allowed for fine grained control over the speed, direction, position, and roll of the camera.
Using a simple device such as the Cardboard in conjunction to a SmartPhone downloadable VR APP one can enjoy the immersive experience of flying through the chambers of the Hal Saflieni Hypogeum.
The major deviation of a 360° film from a normal film is that the viewer is free to look around, soaking in the scenery provided by the camera. Whilst the creative director provides the focus, it is the viewer that chooses what is important in any particular scene.
This article is a summary of a number of published papers and book chapters behind this research project. If you wish more academic detail click here
