{{ message }}
Add Gaussian Splatting support (.ply / .splat / .spz)#892
Open
bobicloudvision wants to merge 5 commits into
Open
Add Gaussian Splatting support (.ply / .splat / .spz)#892bobicloudvision wants to merge 5 commits into
bobicloudvision wants to merge 5 commits into
Conversation
Adds full-pipeline support for Gaussian splatting assets across import, editor preview, save/load, export and runtime: - extensions: register .ply/.splat/.spz as model assets (SPLAT loader is already provided by babylonjs-loaders) - guards: add isGaussianSplattingMesh and recognize GaussianSplattingMesh as an abstract mesh so it shows in the graph, is selectable and gets the mesh inspector (transforms) - import: skip the glTF x100 centimeters scaling for splats (metric space) - preview/assets-browser: accept splat files on drop and in the browser - save: include GaussianSplattingMesh; the splat data is embedded inline by Babylon's native serialize(), so drop the unused serialized material and quad geometry to avoid writing a useless .babylonbinarymeshdata file - load: restore uniqueId/parent for splat meshes, skipping the regular material/geometry reconciliation - export: skip geometry externalization/delay-loading for splat meshes - tools runtime: import the GaussianSplatting side-effect module so the .babylon loader's Mesh.Parse can reconstruct splat meshes inline
…lized GaussianSplattingMesh frees its splat buffer right after uploading it to the GPU when keepInRam is false (the default). serialize() only writes splatsData when the buffer is still present, so saving/exporting a splat produced an empty mesh that failed to render on reload (glDrawElementsInstanced: vertex buffer not big enough). Pass `keepInRam: true` to the SPLAT loader on import so the data is retained and round-trips through save/load/export. The flag is serialized with the mesh, so reloaded and exported splats keep their data too. Ignored by non-splat loaders.
… passes - import: flip Y back on imported Gaussian splatting meshes so they appear upright. The SPLAT loader bakes a `scaling.y *= -1` to convert from the common Y-down splat convention, but plain .splat/.ply/.spz files carry no up-axis metadata and end up upside down in the editor's left-handed scene. - graph / configure / shadow-generators: never add Gaussian splatting meshes to shadow-map render lists (and strip them from lists persisted by older projects). Their thin-instance splat layout can't be drawn by the depth pass and triggered "glDrawElementsInstanced: vertex buffer not big enough". - mesh inspector: hide collision/physics/shadows/material sections for splats and skip the selection-outline pass (same depth-pass limitation). - layout: bump layout version to refresh persisted panel layout.
The SPLAT loader already orients the splat correctly (it bakes scaling.y = -1 to convert from the Y-down splat convention); the previous extra `scaling.y *= -1` flipped it upside down. Remove that flip. Also apply the editor's centimeters convention to splats: glTF gets a x100 scale via its __root__, but splat meshes have no root and were left at meter scale, appearing ~100x too small next to other content. Scale the splat mesh x100 on import.
…ave/export A GaussianSplattingMesh creates a per-camera proxy mesh at render time (name "<splat>_cameraMesh_<id>") flagged with reservedDataStore.hidden and doNotSerialize. These leaked into the editor scene graph and could be written as standalone meshes on save. - isNodeVisibleInGraph: treat reservedDataStore.hidden nodes as not visible, which excludes them from both the graph panel and the save filter. - export: keep reservedDataStore.hidden meshes flagged doNotSerialize so they are not emitted as standalone meshes.
Contributor
Contributor
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds end-to-end support for Gaussian Splatting assets (
.ply,.splat,.spz) across the whole editor pipeline: import, viewport rendering, scene graph, inspector, save/load, and export to the runtime. Previously the editor had no concept of splats — they couldn't be imported, rendered, or persisted.The data round-trip leans on Babylon's native
GaussianSplattingMeshand thebabylonjs-loadersSPLAT loader (engine 9.12.1):serialize()embeds the splat data inline andMesh.Parse()dispatches back to the Gaussian parser. The bulk of the work here is wiring those into the editor's import/graph/save/export flows, plus a set of splat-specific correctness fixes uncovered during testing.Changes Made
Import
.ply/.splat/.spzas model assets; they can be dropped into the viewport or imported from the Assets Browser.keepInRam: true, otherwise the splat buffer is freed right after GPU upload andserialize()would persist an empty mesh.__root__) directly to the splat mesh, since splats have no__root__. The SPLAT loader's own orientation (scaling.y = -1) is preserved.Scene graph & inspector
GaussianSplattingMeshis recognized as an abstract mesh, so it shows in the graph, is selectable, and gets the mesh inspector for transform editing.<splat>_cameraMesh_<id>, flaggedreservedDataStore.hidden) are excluded from the graph.Rendering correctness
glDrawElementsInstanced: vertex buffer not big enough. Splats persisted in a shadow render list by older projects are stripped on load.Save & Load
.babylonbinarymeshdatais written.Export & Runtime
tools/runtime imports the GaussianSplatting side-effect module so the.babylonloader can reconstruct splat meshes at runtime.Benefits
.ply/.splat/.spzcapture straight into a scene and see it rendered, positioned, and scaled consistently with the rest of their content.