Gaussian Splatting
A new rendering primitive for 3D scans (14.08.2024)
Förderjahr 2023 / Projekt Call #18 / ProjektID: 6863 / Projekt: Potree-Next

In mid 2023, Kerbl & Kopanas et al. introduced a new 3D reconstruction algorithm that creates 3D scenes from photos, representing them via gaussian splats [1]. Since then, Gaussian Splats have led to a huge hype in the visual computing community since their visual quality is superior to triangle-based photogrammetry approaches, but they are also fairly fast to render and easily editable, unlike NERF-based approaches.

Although Gaussian Splatting was not part of the original proposal - they were published roughly at the time of the proposal's submission - their importance to 3D-scans of real-world environments turned them into a high-priority task afterwards. In recent months we spent time learning about splatting algorithms and building our own desktop renderers. In the following months, we will then gradually port them to Potree-Next.

The following figures shows gaussian splat models from our desktop renderer (model is from Kerbl & Kopanas et al.), and to the right the 2D-projected gaussians it consists of. Each gaussian can be thought of as an ellipsoid comprising a color, opacity, position, scale and rotation value. During rendering, the 3D ellipsoid is approximated by a screen-facing 2D ellipsoid (as shown), which is not a perfect representation, but one that is very fast to render. Of note are the larger splats in the right picture that seem to be missing in the left one. This is because the reconstruction algorithm tries to optimize splats so that they represent the original images as good as possible, and such large splats often imitate subtle illumination behaviours, including dust/mist/atmospheric scattering, or they can simply be undesirable artifacts like floaters. They still affect the image, but due to their low opacity in a more subtle way. 

Initially, we will integrate gaussians plats without levels-of-detail, i.e., the size of the models is somewhat restricted since loading larger models will take time. However, there are compression and progressive-streaming approaches that still work well for models with up to around 10 million splats, as seen by several dedicated splat viewers that were published in 2024. 

vase 1vase 2

 

garden

[1] https://github.com/graphdeco-inria/gaussian-splatting

CAPTCHA
Diese Frage dient der Überprüfung, ob Sie ein menschlicher Besucher sind und um automatisierten SPAM zu verhindern.
    Datenschutzinformation
    Der datenschutzrechtliche Verantwortliche (Internet Privatstiftung Austria - Internet Foundation Austria, Österreich) würde gerne mit folgenden Diensten Ihre personenbezogenen Daten verarbeiten. Zur Personalisierung können Technologien wie Cookies, LocalStorage usw. verwendet werden. Dies ist für die Nutzung der Website nicht notwendig, ermöglicht aber eine noch engere Interaktion mit Ihnen. Falls gewünscht, können Sie Ihre Einwilligung jederzeit via unserer Datenschutzerklärung anpassen oder widerrufen.