Academic research, Sept. 2023 – May 2024
Developed as an ongoing joint project between Lamont-Doherty Earth Observatory and the Columbia Computer Graphics and User Interfaces Lab to develop a tool to help glaciologists visualize and study ice layers in virtual reality. The project is called VISER (Visualizing Ice Sheets in Extended Reality).
My contribution to VISER was the implementation of radar horizon tracing, a critical element of the program which allows the user to select ice layers and see them traced in real time in the application. I used a combination of computer vision and computer graphics techniques to automatically trace a line delineating two ice layers from a single seed point provided by the user. Before I implemented this feature, automated 3D ice layer picking did not exist, and glaciologists had to manually pick layers in 2D only with programs such as MATLAB.
I also rewrote the entire file importing pipeline to allow dynamic loading of assets at runtime. Before my modifications, the application would not compile and run as a stand-alone app due to the usage of Filmbox files (.fbx) to store the radargram meshes and textures. These radargram files are extremely large and detailed, and this level of detail is necessary to do scientific work. Due to the limitations of compute power of VR headsets (especially the Meta Quest 2 and 3 that we were using), loading all these giant files at initialization proved more than the devices could handle, meaning that they needed to be loaded on demand, in small batches. However, Unity does not support runtime loading of FBX files, which meant that our application could not run standalone without a computer running Unity and a data cord. This made distribution and testing of the application extremely difficult if not impossible.
My contribution and solution to this problem was to change to using GLTF files instead of FBX. I rewrote the entire import pipeline in C# to handle the extraction of meshes and textures at runtime,
Made in Unity with custom C# scripts.
Works with Meta Quest 2 and Meta Quest 3 headsets.
Read more about VISER here.
See the Github repo here.
Watch demo on YouTube here.