In about the middle of 2016, just after I'd finished with university and started my new position at NEC, I started on an ambitious project that aimed to create a virtual world in which to visualise interesting data. The concept was a brainchild of my mentor at the time, Tim Rastall, and together we set about learning Geographic Information Systems (GIS), how to create interaction systems in VR using the HTC Vive, and how to work with large collections of data.
The project came about as a result of the close relationship between NEC Smart Cities and Wellington City Council, and it seemed natural that, on our quest to create our virtual world, we start with our own coolest little capital. As a result of a previously completed photogrammetry sweep of Wellington, as well as high definition heightmaps and LiDAR data, we were fairly quickly able to put together a sizeable piece of Wellington City. Terrain meshes were created using the heightmaps, and were textured using aerial photography. City meshes were broken up into buildings, which in turn were placed, geo-positioned, and given colliders to make them interactive. The ocean was just a large plane that intersected the heightmap - and despite the attention it gets in the media, remains so to this day. Roads were really tricky; since WCC (or any council, for that matter) had ever tried creating a detailed 3D representation of their city, roads only ever needed to be represented as polylines with no width or other useful visual information. In the end, we used the council's road shape data, which happens to be the inversion of kerbs in Wellington. Using procedural texturing and kerb shape generation, we had some pretty good looking roads going. Once we had a recognisable city, we turned our focus to the interaction.
Movement in VR was also starting to become standardised in 2016, despite a lack of friendly ways to implement it. Teleporting seemed like a solid way of getting around, and triggers seemed to be as close as we could get to physically grabbing something using the controllers. We also needed a way to allow the user to adjust their size, which could give them a truly unique perspective when exploring data in the city, and eventually decided on a "pinch-and-zoom" approach; the user could hold the grips (side buttons on an HTC Vive controller) and move the controllers together or away from each other, making the player's point of view grow smaller and bigger, respectively. Our audience was easily able to make the connection between pinch-and-zoom, which is often used on mobile devices to zoom in and out, and changing their size - regardless of how much experience they'd had with virtual reality.
In fact, one of my favourite parts of working on this project was how many people we had through who had never used virtual reality before. The transition from sceptical looks into cries of "wow!" and "it's so real!" was something we experienced often, and in almost all situations we found our audience walked away with a smile on their face and a million fantastic applications for what they just experiences on their minds.
I was unfortunately phased out of the project over some months at the end of 2016 in favour of something slightly more grounded in this reality (Kitestream Spectacle), but I've had the privilege of watching the project grow further over the past few years. I have no doubt we'll see crazy ideas like this take the world by storm in the next few years, so I'm looking forward to that.