README.md

metadata

title: Slapstack Studio
emoji: 🥞🪐
colorFrom: indigo
colorTo: blue
sdk: docker
pinned: false
license: mit

Slapstack Studio — Bet 9 (splat eraser · backdrop · band gains)

Bet 8 plus the editing layer: this is the "adaptive density control by hand" step — delete splats, clean images, control the look. Client-only delta: app.py, oracle.py, and the verified BP core are byte-identical to Bet 8, so deploying is replacing studio_ui.html, studio_core.js, and the built studio/studio.html on the existing Space (or pushing this repo as-is).

New in Bet 9 — all in the EDIT & LOOK card

Erase tool. Switch tool from pose to erase; three brush sizes (S/M/L); drag over the canvas and atoms under the brush are deleted. Works in soup and bound states. Mechanically an erase filters every per-atom parallel array (obs, gt, evidence, assignment, marginals) by the brush mask — atom order carries no meaning anywhere in the system (additive render, per-atom marginals), so Undo is simply re-appending the removed records, and the field restores bit-exactly (verified). Undo history survives until the next Bind/Scramble (BP re-inference makes the records stale, so the stack clears — honestly, rather than pretending).

Burn into template. By default erases are scene-level: Scramble rebuilds from the layer templates and the splats come back. Tick burn into template and each erased atom is also removed from its source layer's template (tracked by reference through the scene build), so the cleanup is permanent — the hand-operated version of 3DGS adaptive density pruning. Undo restores burned atoms too.

Backdrop. A color picker and an image upload. The backdrop is pure presentation state, touched by nothing in the inference loop — which is exactly why it persists across Bind/Scramble/drag instead of reverting to gray. The gray you see by default is not a background at all: it is sigmoid(0), the neutral of the additive field where no atom paints. The backdrop composites under the field using the rendered-deviation alpha (alphaFromPre), so cancelling atom pairs stay transparent over your image.

Gains. The dials from the original splatstack direct-fit editor, back: coarse (<4 c/img), mid (4–10), fine (>10) band gains plus a global opacity gain. Render-only, like before — occlusion evidence and BP are untouched. Exactness for free: the Gabor render is linear in atom color, so Σ_b gain_b·render(band_b) ≡ render(gain-scaled atoms), verified to 2.4e-7. Band gains apply in field mode; opacity applies in both modes.

Ledger for this drop. tests_edit.js: 13/13 GO (band edges, gain linearity, backdrop identities incl. the image path, brush geometry + veto, erase→undo bit-exact round trip, burn/unburn template membership). tests_studio.js regression: unchanged from shipped Bet 8 (11 GO and the one known inherited coverage-overlap FAIL, untouched by this delta). All four script blocks of the built studio/studio.html (106 KB) syntax-verified. Untouched and still honestly unverified: the GPU SDS path.

---

Slapstack Studio (Bet 8 Playroom)

Live at huggingface: https://huggingface.co/spaces/Aluode/SlapStackStudio

Video: https://studio.youtube.com/video/a-emh7csWQk/edit

An interactive, layered visual sandbox running multi-object Belief Propagation (BP) over Gabor packet representations. The system supports full 2D depth occlusion matting, spatial grab/drag/scale/rotate mechanics, semantic prompt interventions, and server-side neural layer distillation via Score Distillation Sampling (SDS).

Architecture & Division of Labor

• The Server (Python): Handles visual generation and heavy pixel-to-atom fitting. It uses an SDS pipeline with Stable Diffusion 2.1 or deterministic MSE optimization to turn pictures/prompts into compact, normalized Gabor-atom templates.
• The Client (Browser JS): Runs a highly optimized, loopy Belief Propagation factor graph entirely client-side. The frontend manages pose clustering, tracks identities through deep occlusion, calculates real-time alpha masks, and translates user gestures into exact group-algebra conditioning updates.

---

File Structure

slapstack-bet8/
├── app.py                # HuggingFace Space / Gradio server entry point
├── oracle.py             # Image-to-atom fit & text-to-atom SDS generation paths
├── bet5_gabor_sds.py     # Underlying Gabor Packet model & differentiable renderer
├── requirements.txt      # Python dependencies for GPU/CPU runtimes
│
├── studio/               # Static folder mounted by FastAPI
│   ├── studio.html       # Single-file production client (fully built)
│   └── builtins.json     # Pre-trained compact templates (tractor, star, boat)
│
├── studio_ui.html        # Raw source markup for the client layout
├── studio_core.js        # Real-time alpha-matting & painter compositor logic
├── build.js              # Assembly script (combines UI + core logic into studio.html)
└── tests_studio.js       # Headless test suite validating depth occlusion math

Getting Started

Prerequisites

Make sure your environment has Python 3.10+ installed. For neural text-to-layer distillation, a CUDA-capable GPU is highly recommended.

Installation

1. Clone or download the repository directory.
2. Install the necessary dependencies directly via pip:

pip install -r requirements.txt

Running the App Locally

Launch the Gradio server by executing:

python app.py

Open your browser and navigate to http://127.0.0.1:7860. The unified workspace will load instantly without requiring any building or file compilation.

Feature Overview & Control Guide

Interactive Canvas

• Scramble: Randomizes object poses to throw all atoms into an unassigned "soup."
• Bind: Runs loopy BP live in the browser. You can watch the atoms settle, crystallize into boundaries, and snap back to their matching identity profiles.
• Mouse Controls (Grabbable Posterior):
  • Click + Drag any recognized object to clamp its position. The environment dynamically re-settles around it.
  • Mouse Wheel rotates the targeted object about its calculated center.
  • Shift + Mouse Wheel scales the targeted object up or down.

Layer Manipulation & Occlusion Semantics

• Switch the scene blend setting from Field (additive summation) to Painter to observe full alpha-compositing across individual depths.
• Moving an object entirely behind another activates Depth Occlusion. Covered atoms cleanly release their assignments and revert back toward a state of uniform uncertainty (their marginal prior), while the shape's pose boundary coasts seamlessly behind the obstruction. Moving the object away re-establishes evidence and triggers immediate re-binding.

Spawning Custom Layers

• Fit (CPU): Upload any image containing a subject on a neutral, solid gray background. The optimizer will spend its entire atom budget crafting a tight, isolated layer template.
• Distill (GPU): Enter a descriptive text prompt to distill an atom map from Stable Diffusion.
  • Note for Local Deployment: The system is pre-configured to look for sd2-community/stable-diffusion-2-1-base to efficiently reuse local caches located under ~/.cache/huggingface/hub/.

Prompt Interventions

Enter clean, natural commands in the prompt box (e.g., "move the boat left, send the star behind the tractor") to execute hard algebraic transformations. The graph instantly recalculates the new joint posterior distribution around your action.