The prototype, HoloCity is an AR game prototype by Cybercom, that was created for Malmö Museum “A room for Games” initiative through Media Evolution and Cross Motion project. HoloCity is a virtual city, a very unlucky one at that. The city is suffering from several environmental issues such as fires and earthquakes. A professor in the town has built a very capable robot, but unfortunately not a very smart one. The robot has the functions needed but it does not know how to use them. It is up to the users to teach the robot how to use its functions. This is being done by basic programming where the focus is on getting into the proper mindset rather than specific programming functions. The robot is physical with virtual pieces added onto it.
Using Augmented Reality and the facilities provided by Malmö Museum, the project offers visitors a unique VR experience that would be impossible at home. The project focuses on mixing the physical, museum itself with the virtual. The gaming area is on the table, so the user looks at the holograms slightly from above. This way there is less strain on the eyes, and there is a clear boundary to provide better understanding what is part of the game and what is the surrounding physical environment.
The prototype was created by the team from Cybercom led by Alexander Kostakis. First idea was to create a tower defence game where one could program their own robot. Initial plan was to use HoloLens and programming similar to Scratch. For three months the team tried Real- Time Image Processing to track and project holograms on top of a physical, moving robot. The solutions the developers came up were to connect the HoloLens and the physical robot with a power cable; increase the size of the tracking images. Conclusion was: ”Why take the latest technology and then put awkward limitations on it to fit our solution?”
“An innovative adventure, or an adventure of frustration? This stands true in a comical way, especially if you ask one of our developers, Jacob. Jacob was responsible for finding a working solution for the tracking, a.k.a. making sure that our virtual holograms were placed upon and following our physical robot. It did not sound too bad at first, but proved to be a major hindrance.”
Final idea did not include tower defence, instead it was a game where user solves different tasks that pop up at different places. The programming has taken another form. The result is a physical robot with virtual pieces added onto it. Google Tango had been chosen and it proved to be an accessible platform. The idea was to detect and track precise shades of red colour using OpenCV and in combination with the power of the Google Tango API. “We are comparing 2.500 EUR to standard phone prices – nothing to sneeze at especially considering our target group being mainly children. Making them able to use their phones instead of a HoloLens increases the accessibility immensely and makes it so that several people can use the solution at once.”
HoloCity was demoed in the Experience Lab during Aarhus Digital conference.
Left: Adam Krauser working with the HoloLens. Right: Jacob trying to get the tracking right, here using Vuforia.