4DGS Playback

4DGS (4D Gaussian Splatting) enables volumetric video playback by rendering sequences of Gaussian Splat frames. This creates the effect of a 3D video that can be viewed from any angle.

Overview

4DGS works by:

1. Converting video frames to individual Gaussian Splat files (PLY or SOG)
2. Loading and playing back these frames in sequence
3. Using double-buffering for smooth transitions between frames

Basic Usage

NPM Package

import { createViewer } from 'storysplat-viewer';

const container = document.getElementById('viewer-container');

const viewer = createViewer(container, {
  name: 'My 4DGS Video',
  frameSequence: {
    frameUrls: [
      'https://cdn.example.com/frames/frame_000.ply',
      'https://cdn.example.com/frames/frame_001.ply',
      'https://cdn.example.com/frames/frame_002.ply',
      // ... more frames
    ],
    fps: 24,
    loop: true,
    preloadCount: 5,
    autoplay: false,
  },
  defaultCameraMode: 'orbit',
});

// Start playback
viewer.play();

Configuration Options

Option	Type	Default	Description
frameUrls	string[]	Required	Array of URLs to PLY/SOG frame files
fps	number	24	Playback frames per second
loop	boolean	true	Loop playback when reaching the end
preloadCount	number	5	Number of frames to preload ahead
autoplay	boolean	false	Start playing immediately when loaded

Playback Controls

Play / Pause / Stop

// Start playback
viewer.play();

// Pause playback
viewer.pause();

// Stop and reset to first frame
viewer.stop();

// Check if playing
if (viewer.isPlaying()) {
  console.log('Currently playing');
}

Frame Navigation

// Jump to specific frame (0-indexed)
viewer.setFrame(10);

// Get current frame number
const currentFrame = viewer.getCurrentFrame();

// Get total number of frames
const totalFrames = viewer.getTotalFrames();

// Get/set progress (0.0 to 1.0)
const progress = viewer.getFrameProgress();
viewer.setFrameProgress(0.5); // Jump to middle

Playback Speed

// Change FPS during playback
viewer.setFps(30);

// Get current FPS
const fps = viewer.getFps();

Events

Listen for 4DGS-specific events:

// Frame changed
viewer.on('frameChange', (frame: number, total: number) => {
  console.log(`Frame ${frame + 1} of ${total}`);
  updateProgressBar(frame / total);
});

// Playback completed (non-looping only)
viewer.on('frameComplete', () => {
  console.log('Playback finished');
});

// Playback started
viewer.on('playbackStart', () => {
  console.log('Started playing');
});

// Playback stopped
viewer.on('playbackStop', () => {
  console.log('Stopped playing');
});

Creating 4DGS Content

Using StorySplat Tools CLI

# Convert video to 4DGS frame sequence
npx @storysplat/splat-tools video2splat \
  --input video.mp4 \
  --output ./frames \
  --fps 24 \
  --format sog

Using the Desktop App

1. Open StorySplat Tools
2. Go to Video → 4DGS tab
3. Select your video file
4. Choose output directory and settings
5. Click Convert to 4DGS

Programmatic Conversion

import { video2splat } from '@storysplat/splat-tools';

const result = await video2splat({
  input: 'path/to/video.mp4',
  outputDir: './frames',
  fps: 24,
  format: 'sog',
  gpu: 'auto',
});

console.log(`Created ${result.totalFrames} frames`);
console.log('Frame URLs:', result.frames);

Hosting 4DGS Content

Frame Hosting Requirements

4DGS frames need to be hosted on a server that supports:

• CORS headers for cross-origin requests
• Byte-range requests (for large files)
• Reasonable bandwidth for streaming

Recommended Hosting Options

Service	Best For	Notes
Cloudflare R2	Production	Free egress, great performance
AWS S3 + CloudFront	Enterprise	Scalable, reliable
Vercel Blob	Small projects	Easy setup
Self-hosted	Full control	Configure CORS manually

Manifest File (Optional)

Generate a manifest for easier frame management:

import { createManifest } from '@storysplat/splat-tools';

const manifest = createManifest({
  frames: result.frames,
  fps: 24,
  loop: true,
  baseUrl: 'https://cdn.example.com/my-4dgs/',
});

// Save as manifest.json
fs.writeFileSync('manifest.json', JSON.stringify(manifest, null, 2));

Manifest format:

{
  "version": "1.0",
  "fps": 24,
  "loop": true,
  "frameCount": 120,
  "frames": [
    "frame_000.sog",
    "frame_001.sog",
    "..."
  ]
}

Load from manifest:

// Fetch manifest
const response = await fetch('https://cdn.example.com/my-4dgs/manifest.json');
const manifest = await response.json();

// Build frame URLs
const baseUrl = 'https://cdn.example.com/my-4dgs/';
const frameUrls = manifest.frames.map(f => baseUrl + f);

// Create viewer
const viewer = createViewer(container, {
  frameSequence: {
    frameUrls,
    fps: manifest.fps,
    loop: manifest.loop,
  },
});

Performance Optimization

Preloading Strategy

The preloadCount option controls how many frames are loaded ahead:

frameSequence: {
  frameUrls: [...],
  preloadCount: 10, // Load 10 frames ahead
}

• Low bandwidth: Use preloadCount: 3
• High bandwidth: Use preloadCount: 10-15
• Local files: Use preloadCount: 20+

Format Selection

Format	File Size	Load Time	Quality
SOG	Smallest	Fastest	Good
Compressed PLY	Medium	Medium	Better
PLY	Largest	Slowest	Best

For 4DGS, SOG format is recommended for web delivery due to smaller file sizes.

Memory Management

The viewer automatically manages memory by:

• Unloading frames that are far behind current position
• Maintaining a sliding window of loaded frames
• Using double-buffering to prevent frame drops

Troubleshooting

Frames not loading

1. Check CORS headers on your hosting
2. Verify frame URLs are accessible
3. Check browser console for errors

Choppy playback

1. Reduce fps setting
2. Increase preloadCount
3. Use SOG format for smaller files
4. Check network bandwidth

Memory issues

1. Reduce preloadCount
2. Use smaller frame dimensions
3. Use SOG format

Complete Example

<!DOCTYPE html>
<html>
<head>
  <title>4DGS Viewer</title>
  <style>
    #viewer { width: 100%; height: 600px; }
    .controls { margin-top: 10px; }
  </style>
</head>
<body>
  <div id="viewer"></div>
  <div class="controls">
    <button id="playBtn">Play</button>
    <button id="pauseBtn">Pause</button>
    <input type="range" id="progress" min="0" max="100" value="0">
    <span id="frameInfo">0 / 0</span>
  </div>

  <script type="module">
    import { createViewer } from 'https://unpkg.com/storysplat-viewer@latest/dist/index.esm.js';

    const viewer = createViewer(document.getElementById('viewer'), {
      frameSequence: {
        frameUrls: [
          // Your frame URLs here
        ],
        fps: 24,
        loop: true,
      },
    });

    // Controls
    document.getElementById('playBtn').onclick = () => viewer.play();
    document.getElementById('pauseBtn').onclick = () => viewer.pause();

    document.getElementById('progress').oninput = (e) => {
      viewer.setFrameProgress(e.target.value / 100);
    };

    // Update UI
    viewer.on('frameChange', (frame, total) => {
      document.getElementById('frameInfo').textContent = `${frame + 1} / ${total}`;
      document.getElementById('progress').value = (frame / (total - 1)) * 100;
    });
  </script>
</body>
</html>