About a month ago I spotted on twitter a link to a website AreoBrowser. The site showed off the open digital terrain models (DTMs) generated by the Mars Reconnaissance Orbiter High Resolution Imaging Science Experiment (HiRISE) and open to download from the Univeristy of Arizona HiRISE website. AreoBrowser looked like a really useful tool to browse the HiRISE catalog – but wouldn’t it be great to see these models in the HoloLens? After a quick conversation with the creater of AreoBrowser, Matt Brearley, I came up with a plan for how to get all 557 models downloaded, process the meshes to a usable number of triangles, and finally converted to a format that could be served up wirelessly to a HoloLens application. These should really be in the format of Unity AssetBundles as these are compressed files that can be loaded quickly at runtime on the HoloLens.
The first step, downloading the data, should be straightforward enough. Only the mesh data needs to be downloaded, the texture jpegs could stay online and can be referenced from within the HoloLens app. A wget command should do the trick.
Luckily the HiRISE team are up on visualizations and they’ve developed an add-on to Blender to allow for the import of the .IMG DTM files. Once imported in to Blender it was clear that these meshes were going to be way too big for the HoloLens’s limited processing power. Not to worry, a quick decimation and a good quality texture should make things look not too bad. After a brief foray into Blender python coding, all the .IMG files were decimated to a similar level and converted to .blend files which can be read by Unity. I probably should have used .OBJ … but here we are.
The .blend files are fine, but they’re uncompressed and take a little while to load into Unity at runtime (after the app has already started), and besides it would require using some third-party code. Preloading 500+ models is really not feasible (we’re still at GBs of data), the app would take forever to start. Bad idea. Unfortunately, bulk file conversion and compression isn’t really what Unity is made for. I wasn’t really ready to go through each model individually, add a unique filename, and compress each file in to a Unity AssetBundle. Thankfully, there are enough programmable steps that can be hacked together to import, save and compress the models which will make things more manageable.
Great, all the models are processed and can be uploaded to a public server (we’re upgrading to Microsoft Azure storage) ready to be loaded up on the HoloLens, and in particular, our public GeoXplorer application. Up until now I’ve really only had a few models to serve up, and they’ve been fairly well-spaced around the Earth. Now we’re dealing with a few-hundred models from a whole other planet. It maybe time to rethink the user interface.
I had thought about doing something similar to AreoBrowser and have a search field, but typing on the HoloLens is nigh-on impossible and certainly not ideal for repeat interactions. Besides, being a spatial visualization, it would be a lot nicer to have the icons represented in a spatial coordinate rather than in a list. It’s a bonus that the HiRISE team also provides the latitude and longitude of all the DTMs on their website. This made it straightforward enough to add small icons to locations on a textured sphere that represents Mars in GeoXplorer. But it doesn’t make selecting an individual icon sphere very easy. What may work is selecting multiple icons within an area around the HoloLens cursor. This would limit it to around half a dozen models which can then be displayed as a list.
OK, there’s still a list, but we can maybe improve on this!
A video outline these interactions is embedded below:
There are a few issues, notably that the meshes are still a little large (triangle count-wise) and they could be made a little smoother as well. Also when they’re imported into Unity, the origin point is not the center of the model but at the corner. This is best for the texture but not for when it comes to manipulation, it feels like the model is attached to something in the corner and that’s not very intuitive.
GeoXplorer will be updated in the coming weeks.