The Making of a Virtual Reality Experience of a World Heritage Site
Virtual Reality (VR) is the use of computing to offer an immersive sensation to a human being of environments that may be difficult or even outright impossible to visit.
VR is still at its infancy, but the hardware is advancing and prices are becoming relatively cheap. The impediment at this moment is the lack of content that attracts the market, and thus content developers are looking into providing material that would be of interest to the masses.
The Hal Saflieni Hypogeum is not completely off limits to visitors. But the site is so delicate, so unique, that its conservation takes precedence above all else. Thus, visitors are limited to specific sections of the complex, and constrained by a gangway to keep their feet off the ground. The hypogeum is in perpetual semi-darkness, except for a choreographed spot lighting of areas based upon the guide’s explanation.
Through this project, the Hal Saflieni Hypogeum is being brought to all those who have cultural heritage at heart, providing a virtual model that may be explored in its totality and to its minute detail. This work is researched and developed by students and academic staff of the Department of Computing at Saint Martin’s Institute of Higher Education.
Understanding humans to adapt the technology
Humans are continuously alerted by sensory stimuli, namely Vision, Audio, Smell, Touch and Taste, each of which trigger diverse reactions from individuals. Humans also have the inner sense of balance, and will react as the body is placed in unnatural angles.
Computer scientists, having recognised the potential of the burgeoning power of hardware and software that makes use of the proliferation of smart devices, have set their eyes on the potential of harnessing this computing power to elevate the human computer interaction to new heights. Hence dawned the mixed reality era.
Mixed Reality
Mixed reality is the use of the substantial power of computing to generate artificial worlds that are communicated to humans through the manipulation of human senses. Massive monitors are used to display stunning computer generated images, supplanting props and artefacts as an addition to a real environment known as augmented reality. In other cases, wholly artificial environments are created through computer generated imagery experienced via an immersive headset to attain virtual reality.
Saint Martin’s Institute of Higher Education has been pursuing this field of study for the past decade, even before mixed reality moved into mainstream. With the introduction of its BSc (Hons) Creative Computing in 2006, the institute sowed the first seeds towards what was to become the main growth area in the computing industry. In the early years, few understood what the concepts behind creative computing really meant, but the institute continued to research the area and spawned specialisations such as virtual and augmented reality, user experience and further development of serious games and for entertainment.
The use of mixed reality in cultural heritage
Whilst it was evident that the Institute’s students were mostly interested in the computer games for entertainment industry, the faculty at SMI decided to take a more serious attitude to their research and focused on finding new technology that may portray the very rich variety of cultural treasures in Malta, aiming to educate, entertain and help in the conservation efforts of cultural assets whilst allowing their extended studies and monetization. After all, teaching young scientists requires that academics have experience through research and publication of findings in their arsenal, and not simply passing on what is readily available in text books. This particular area of study is quite unique worldwide.
Preparing the visual
The starting point of the Hal Saflieni Hypogeum virtual reality experience was the 3D documentation of the site financed through the Norwegian Financial Mechanism grant and concluded in 2012. This documentation is carried out through laser scanning from an identifiable central position in the labyrnth of rock hewn chambers. The result is a point cloud that was imported into Unity® Game Engine to develop a real environmental model of the hypogeum.
Working on the Audio
The experience necessitated the introduction of sound that was representative of the environment; sounds that were contextualised within the complex, with echoes and reverberations that mimic the acoustics of the place and therefore sounds that assisted in understanding the aural context of the site to immerse the virtual visitor.
Given the low frequencies of resonance reported in the literature, the sounds emitted for recording were male basso voice chants and a 16–20kHz sweep. Tests were carried out until there was reverberation. The resulting resonance was… beyond description.
Motion
Walking in VR is very tricky. The headset will bar any possibility to see your actual surroundings, instead projecting a totally diverse environment. It is impossible to allow VR users to walk around safely in a fixed space, especially since the Oculus Rift® is hardwired to the machine that is processing the experience. The solution is to mimic flight by drone inside the Hal Saflieni Hypogeum, solving the many issues of lack of accessibility to many chambers within the complex, since there are no barriers to enter into spaces that are normally inaccessible to human visitors.
The rig will move through actuators according to the flight of the user whilst enjoying the experience, thus adding motion to the sight and audio of the experience. Designed by SMI.
Interested to join the class of 2020? Click here
Making the short 360° VR film ‘The Awakening’
Every film-maker aims to excite people in unimaginably original ways; triggering emotions through as many of the senses as is possible. Film has been technically relegated, for many decades, to the use of two main senses, those of vision and hearing. But here one must stop and think. Film has come a long way over the past century. The use of technology has enabled not just the facilitation of the use of sound and visuals in much more elaborate ways (through 3-dimensional specials effects (FX) and elaborate sound mixing), but has also brought forth particular elements which were unheard of prior to the last couple of decades. IMAX providing a very large image tends to provide some immersion, as also 3D films that ask viewers to wear disposable glasses and have the effect of objects and characters moving towards the viewer out of the screen. Entertainment parks, such as Disney’s Epcot have introduced 360° theatres, warning viewers to hold onto railing because the surrounding images will trick the brain that the theatre is moving.
The culmination of the 360° Hal Saflieni Hypogeum VR project — ‘The Awakening’, was meant to bring about an experience which could be viewed in the comfort of one’s home, similar to any other film experience, yet with the highlight of virtual reality (VR), without the shackles of heavy and expensive computing equipment for the user.
The director, using the Oculus Rift® headgear was able to explore with the cameraman, and develop the scenes that were to be selected for the short film. Thereon a camera track and dolly system was introduced in the VR model, thus ascertaining that the camera always followed a pre-set path around the hypogeum without deviating from what the creative director wished to communicate. This allowed for fine grained control over the speed, direction, position, and roll of the camera.
The major deviation of a 360° film from a normal film is that the viewer is free to look around, soaking in the scenery provided by the camera. Whilst the creative director provides the focus, it is the viewer that chooses what is important in any particular scene.
This article is a summary of a number of published papers and book chapters behind this research project. If you wish more academic detail click here
And this is made possible through — Computer Science
All this — and more — is made possible through the understanding of the technology available and the ability to use the technology available, and even to dream up and develop new devices to stretch even further the imagination.
Reading for a Computing degree with a specialisation in Games Design and Development or User Experience (both of these specialisations include the field of Virtual Reality) will mean learning how to design and develop systems that best stimulate human senses, based upon the characteristics of the experience itself.
A game that is designed to be played through a console in a living room must tap the graphics capabilities of the screen and the power of the console to generate fast moving high quality images for the player to react to, as well as the power of broadband internet for games such as MMOs that are played over the cloud.
Games developed for multi-player interaction can take new forms. A massively popular interactive game developed by Saint Martin’s Institute was The Queen Experience (referring to the band not Queen Elizabeth II) that was a focal exhibit for the MRO 2019 and Science in the City of the same year. The concept behind the interactive game, that was a final year project for undergraduate students, was to grade groups of up to four people based on their ability to mimic the movements of each Queen band member, with all human movements captured and compared to the postures and stage postures inspired by the original band members.
Reading for a degree in the area will offer students proficiency in programming skills, as well as an understanding on how the human user reacts to natural stimuli in order to generate perceptually equal stimuli through computing power (among many other areas, of course).
Get to know more about Saint Martin’s Institute of Higher Education portfolio of MQF level 5 Diploma and MQF level 6 BSc (Hons) degrees conferred by the University of London that will lead you to a career in Games Design & Development, Mixed Media & User Experience.
Entry Requirements
The requirements to embark on studies in Computing at Saint Martin’s Institute of Higher Education are;
Minimum 16 years or older;
A minimum of four ordinary sec levels at grade 1 to 5, including mathematics and english language.
Interested to join the class of 2020? Click here
Whilst reading for your programmes, you will receive stipends and maintenance grants, and under the GetQualified scheme you will receive 70% of your tuition fees as tax rebates when you are in employment. You may also qualify for an interest free, payment free whilst studying loan, under the Studies Plus Scheme. Visit our website for further information click here
Contact the Institute on infodesk@stmartins.edu or give us a call during normal office hours on +356 2123 5451
