Skip to content

🤔 Understanding Gaussian Splats

Gaussian Splatting (or 3D Gaussian Splats) is the core technology behind StorySplat's stunning real-time 3D scenes. Understanding the basics can help you make the most of the platform.

What are Gaussian Splats?

Imagine representing a 3D object or scene not with flat triangles (like traditional 3D models) but with millions of tiny, overlapping, semi-transparent "fuzzy" dots in space. Each dot is a "Gaussian Splat."

Instead of sharp edges, these splats blend together smoothly, creating highly detailed and photorealistic representations of real-world objects and environments.

Key Components of a Splat

Each individual Gaussian Splat point holds several pieces of information:

  • Position (XYZ): Its precise location in 3D space.
  • Covariance (Shape/Rotation): Describes how the "fuzziness" of the splat is stretched and oriented. This determines its 3D shape (like a tiny ellipsoid) rather than just being a flat circle.
  • Color (RGB): The base color of the splat.
  • Opacity (Alpha): How transparent or opaque the splat is, allowing for soft edges and blending.
  • Spherical Harmonics (SH - Optional): Complex data describing how the splat's color changes based on the viewing direction and incoming light. This is crucial for realistic lighting and reflections but isn't present in all source formats (like .splat or .ply from some scanners). .spz files typically preserve this data.

Why Use Gaussian Splats? (Advantages)

Compared to traditional polygon meshes, Gaussian Splats offer several benefits, especially for web-based experiences like StorySplat:

  1. 🚀 Faster Rendering & Loading: Splats can often be rendered more efficiently than complex meshes, leading to smoother real-time performance and quicker initial load times.
  2. 📸 Photorealistic Detail: Excellent at capturing fine details, complex textures, and subtle lighting variations directly from photographs or scans, often surpassing the realism achievable with traditional texturing on meshes.
  3. ☁️ Smaller File Sizes (Often): Optimized splat formats like .spz can be significantly smaller than equivalent high-detail meshes with large textures, making them ideal for web delivery.
  4. 🌫️ Softness & Transparency: Naturally handle fuzzy details, semi-transparent objects (like smoke or glass), and soft shadows better than sharp-edged polygons.

Common File Formats

You'll primarily encounter these formats when working with Gaussian Splats in StorySplat:

  • .ply: A versatile format that can store Gaussian Splat data, including SH if the generating software included it. Often large due to being uncompressed text or binary.
  • .splat: An uncompressed format specifically for splats, optimized for some web viewers but typically lacks SH data.
  • .spz: A compressed format based on spz-js, designed for efficient web delivery and preserves SH data. (Recommended for StorySplat when SH data is available).

See File Format Compatibility for more details on how StorySplat handles these formats.

How are They Created?

Gaussian Splats are typically generated from real-world data using techniques like:

  • Photogrammetry: Processing a series of photographs taken around an object or scene (e.g., using tools like Luma AI, Polycam, RealityCapture).
  • 3D Scanning: Using specialized hardware like LiDAR scanners.
  • Direct Conversion: Converting existing point clouds or meshes into splats.

Limitations

  • Editing: Editing individual splats (beyond basic transformation, deletion, or color tinting) is complex and often requires specialized external tools or retraining the model. StorySplat focuses on arranging and presenting splat scenes, not deep geometry editing.
  • Animation: Gaussian Splatting is primarily designed for static scenes. Animating splats themselves is an active area of research but not widely supported.
  • Tooling: The ecosystem of tools for creating and editing splats is still evolving compared to mature mesh-based workflows.

StorySplat's Role: Turning Splats into Stories

StorySplat leverages the visual richness and performance of Gaussian Splats to create immersive, interactive narratives. Instead of just viewing a static 3D model, StorySplat allows you to:

  • Guide the Journey: Use Waypoints to define a camera path, leading your audience through the scene and controlling their perspective to highlight key elements or reveal information sequentially.
  • Add Context and Detail: Attach text descriptions, images, audio narration, or even embed external content to specific points using Hotspots. This transforms a simple 3D view into an informative or emotionally resonant experience.
  • Enhance Atmosphere: Incorporate spatial audio that changes based on the viewer's position relative to waypoints, adding another layer of immersion. Add environmental effects with Particles and Skyboxes.
  • Create Dynamic Environments: Utilize SplatSwap to transition between different splat models along the camera path, showing changes over time or different variations of a scene.
  • Define Interaction: Set up Collision boundaries for realistic walkthroughs in Walk Mode.
  • Share Engaging Experiences: Easily Share and Export your finished story as a direct link, embeddable iframe, or self-hosted HTML package, making your interactive 3D narrative accessible anywhere on the web.

Essentially, StorySplat takes the raw visual data of a Gaussian Splat and provides the tools to overlay narrative structure, context, and interactivity, turning a static scene into a dynamic story.

Previous: Troubleshooting | Next: Waypoints