Cutting edge virtual art gallery with state of the art 3d and 2d animations made extremely efficient for lightning fast page loads, SEO, and performance on low end and mobile devices.
A high performance 3d gallery allows the user to view sculptures with immaculate detail, thanks to an innovative photogrammetry process and advanced compression and reprojection.
The goal: Adequately display the complex, geometric works of Miran Elbakyan on the web, with good performance.
Step 1 - Photogrammetry
First, hundreds of ultra high resolution photos are taken of the sculptures, providing gigabytes of visual information. These photos are then processed to enhance sharpness and contrast for more detailed textures and better edge detection, and are fed through a custom photogrammetry pipeline based on Apple's Object Capture API.
Step 2 - Compression
Photogrammetry results in massive meshes with a lot of redundant info -- thousands of triangles to represent flat surfaces. These inefficiencies present an issue for load times as well as performance on older devices. To address this, a custom compression script creates a mesh that fits the original within some tolerance, but uses a minimum number of triangles. This new mesh is then simplified by combining triangles whose removal minimally changes the geometry. We can see this process in the dramatic reduction in triangles representing the flat platform on which the character stands, while maintaining a lot of triangles in the complex geometry of the feet, clothing, hands, and hat of the character.
Step 3 - Texture Reprojection
Once the model is remeshed, the image textures which applied to the original models are no longer mapped to the geometry,
and the original texture files often include information about parts of the scene which are cropped from the original model. The texture is re-built by ray casting from the original textured model onto the simplified one, taking the color of the nearest part of the old geometry for every part of the new geometry. This texture is then fit to a dense and efficient texture map.
Step 4 - Web Deployment, SSR, Progressive & Lazy Loading
In deploying this project, one of the main goals was to have an zero javascript initial page load, and then to gradually load in complex and large JS components invisibly. We can see this on the homepage where the 3d gallery is behind some scrolling, which gives the user an instant interactive page load, and gives the 3d content ample time to load as the user scrolls to it. On future page views, the 3d models and rendering engine are cached and ready to go immediately. Most users will never see a loading bar despite the complex nature of the page.
Although the website is built in React, the site is served statically via GatsbyJS, resulting in substantially faster performance, and a functional website even without JavaScript.
3d pages are rendered progressively, with several levels of detail and render modes selected based on system performance, and loaded in sequentially to allow interactivity as soon as possible.
Through all these techniques, we can deliver a complellng 3d experience on desktop, mobile, and even legacy devices.
The 3d CMS
Using Sanity CMS's amazing custom integrations, we handle the use of 3d models a lot like images, with first tier preview, editing, and optimization in platform.