3D holographic models are all well and good, but where the technology may provide an edge are with models that are impossible to physically build (3D printing has come a long way, but it has yet to print out objects that can hover!). This became more apparent to me when asked if I could see if we could visualize a protein molecule using the HoloLens.
Now, being a geophysicist by trade, I had no idea where to begin and then what to load up that would be useful! I’m still a little unsure about what will be most beneficial to researchers and educators, but I think the implementation of holographic models in this instance could really help.
There is software that can represent proteins and other biological structures in 3D out there on the market, and I was pointed towards PyMol as being a good place to start. Ok, so I could get a molecule visualized in 3D, could it be rendered on the HoloLens or would it be too much information?
First up, I needed to convert the file format to something Unity could understand. This wasn’t trivial as although PyMol can export COLLADA .dae format, when reading these into Unity the colors (which can be important) are not retained. Shifting to Plan B, I loaded a .wrl exported from PyMol into Blender. This seemed to work, all the colors are retained and when I opened up the .blend file in Unity all the colors are present and correct. I loaded up an application on the HoloLens and took a look-see:
Nice, DNA strings and all! Now that was fine but some things needed tweeking. Firstly, the spheres are probably not the best to visualize the organic chains and some of the elements are overlain by others, so I’ll change them to PyMol’s ‘stick’ representation. Secondly, and slightly more importantly, the model was incredibly juddery, the loss of high frame rate is a symptom of trying to render too much information at once and leads to a lousy experience. Best to try and decimate the mesh a little. Lastly its a static hologram, we can do better than that!
Back in Blender I can see if there’s duplicate vertices that can be deleted. This is often the case when models are being generated by some scientific software that isn’t really optimized for this sort of thing. Yes, there was a whole bunch of duplicates so that will help a lot with reducing the file size and the rendering. While I was in Blender I also sorted out the shader so that the surfaces appeared smooth as opposed to faceted in the video above.
Back in Unity I loaded up the improved model (much faster!) and designed a new manipulation toolbox to rotate, move, and resize the hologram so I can place it on my desk. Here’s the final version:
It’s no longer jerky, those smooth shaders and much nicer to look at, and the ‘stick’s look good too! Now the next step will be hopefully adding a whole load more molecules, perhaps being able to visualize relative sizes and how they interact. As it turns out when I export this prefab as an AssetBundle, the file size is less than 1MB… should make for a fast load time from an online server!