# SCION-26 · PixiJS — Archive Stage Rendering Engine · PAE-02 (cohort 7) ## Thesis A threshold scene as a **GPU stage of hundreds of small transparent archive fragments**: the dog, the person, the house, the porch and the sky are all sprites cut from public-domain footage and packed in one texture atlas, so the engine draws the whole frame in about thirty batched draw calls. What holds hundreds of pieces together as *one animal* is not drawing but rendering: a 3-D skeleton places the fragments, depth sorting decides who covers whom, and render-texture passes with a consolidation shader turn the cloud of pieces into a single cut silhouette with an outline. Bland would be: a sprite sheet of a dog, or confetti in a dog-shaped cloud. ## The engine (what was actually used) - **PixiJS 7.4.2 (MIT)**, installed with `npm install pixi.js@7.4.2` into `build/node_modules` (deleted after the build; `make.sh` reinstalls it). WebGL 1 through ANGLE/SwiftShader in Playwright's headless Chromium. I did not write my own WebGL engine. - **Texture atlas** (`build/atlas/atlas.png`, 2048×2048, 966 frames; `atlas.json` records for every frame the source file, frame index, crop box and mask rule). Built by `build/atlas.py`. - **Scene graph** (`build/engine.js`): six depth planes as containers: sky · yard · house (with the interior) · floor · actors · front. Fragment bodies are containers with `sortableChildren`, z-index from the projected depth of each fragment. - **Transform matrices**: a fixed pitched orthographic stage camera, `screen = (330 + 200·x, 520 − 200·(y·cos θ + z·sin θ))`, sin θ = 0.36. Fragment sprites carry per-frame position, rotation (along the projected bone) and anisotropic scale (stretched along the bone, by its foreshortening). The door leaf is a 2-triangle `SimpleMesh` whose corners are the projected leaf as it swings in. - **Alpha masks**: the interior room is a container masked by the door opening (stencil mask). Every coat fragment is cut inside a SAM mask polygon and carries a torn alpha edge. - **Render textures** per frame: dog body (512×384), dog head (320×280), person (512×560), light (640×360, half resolution), whole stage (1280×720). - **Filter stages**: (1) *consolidation* shader on the dog, head and person textures: alpha threshold (smoothstep 0.28–0.50), un-premultiplied colour, darkened inner rim, dilated dark outline; (2) BlurFilter on the light pass (the door light polygon, with shadow fragments projected from the lamp onto the floor, drawn black into it), added onto the boards; (3) final grain / flicker / vignette shader on the stage texture. - **Measured**: draw calls per frame (WebGL `drawElements`/`drawArrays` wrapped and counted): see `build/cache/measure.json`; 31 in every scene frame of the last build. JS update + submit time per frame is in the same file (a few ms; the GPU time on SwiftShader is hidden behind it). The whole 912-frame film takes about 11 minutes to render here (≈0.7 s per frame including the GPU read-back for measurement and JPEG encode). - **Culling**: not needed at this size (every fragment is on screen); not implemented. ## The dog: what it is made of and how it moves - **Made of** 652 fragments (body 529, head 123): coat patches cut inside the SAM masks of the archive dog (*Amateur Film New Jersey C*), pale fur from a lion and dark stripe patches from a tiger (*Andy's Animal Alphabet*) for ears and nose; eye dots are synthesized discs. Each fragment is grey (normalized, contrast pulled toward its mean) and coloured by a sprite tint: coat, darker saddle, pale belly and legs, dark ears and muzzle tip, then lit (dusk sky from above, a cool key, and the warm door light with falloff and a pool term once the door opens; fragments facing away from the camera are darker, which gives volume). - **Skeleton (a guide, never seen)**: capsules in 3-D: two torso segments, neck, skull, muzzle, two ears, a 4-segment tail, four legs (fore: shoulder–elbow–paw, elbow back; hind: hip–stifle–hock–paw, stifle forward), solved by two-bone IK. Fragments live on the capsule surfaces (u along the bone, θ around it) plus dark "core" discs along the axis so no holes open. They are billboards, so an end-on body is still a volume: a turn never collapses into a flat squash. - **Feet**: a gait phase from distance travelled *and* turning; each foot is planted at the spot it will occupy at mid-stance and swings between plants, so stance feet do not slide on the boards while walking, backing or turning on the spot. Paw sounds are placed at the rendered touch-downs (49 contacts). - **Acting**, in the dog's own timing: backs away crouched, tail tucked, head low (1.0–2.3 s); lifts its head to watch; the head starts the turn away (3.38 s), the body follows through the camera side (3.44–3.98 s) and the dog lies down with its back to the door; at 7.40 s the **head leads**: it swings round over the shoulder to the door (muzzle toward the door by 7.58 s) while the body is still down; the body rises and turns back 7.86–8.62 s; ears up, it watches; from 14.7 s it comes with the head below the line of its back for the whole approach (measured: 100 % of approach frames), its path curving in toward the wall; at the sill it lies down diagonally across it in depth (yaw 138°), creeping its body 0.24 m forward so the head and forelegs lie in the lit opening and the hindquarters and the tail (curled round on the floor) stay outside the right jamb; at 26.4 s it **looks in**: the head turns 70° into the room (muzzle 0.15 m deeper than the skull), the back of the head and the ears toward the viewer, with no lift (neck capped at 8°); then chin down on its paws, and stays. - The head is its own layer (own render texture and outline), drawn behind the body when it looks into the room, in front otherwise. That is what makes the head read against the body. ## The world The house façade is made of clapboards cut from fence rails (*Billie, the Buffalo Baby*) and ladder poles (*Robinson Crusoe C*); porch boards, sill, rail, trim and the crate the person sits on are the same strips. The door leaf is the door from *Straight Talk*; the interior (behind the opening mask) has a lantern (*Anthracite*), a picture (*Golden Gate City*, windmill), Quick, Draw! dog doodles pinned to the wall, and a lamp glow. Dusk: computed gradient, cloud cut-outs (*Top Of The Cloud*), star cut-outs (*Gift of Green*), chalk moon, trees and fence (Quick, Draw!). The person is 428 fragments of cloth from the coats in *Straight Talk* (hair from a cat, *Gateways to the Mind*), profile, no face; the offered hand is the hand from *The Door to Heaven*, cut by its SAM mask and mirrored. The door light is a polygon computed by projecting a ceiling lamp through the current opening onto the porch, with the dog's and the person's fragments projected from the lamp as shadows. ## Sound (synthesized, `build/sound.py`) A formant voice (glottal pulses through three resonators): "Come here" / "Come on" at 1.0 and 3.4 s (calls), "It's all right" at 7.45 and 10.45 s and "Come in" at 26.45 s (soft, breathy, low-passed). Measured layer levels: calls −12.5 dB RMS, soft voice −27.2 dB (**14.7 dB under the calls**, 18.5 dB above the room tone). Checked again on film.mp4 (ffmpeg volumedetect, 0.8 s windows): calls −16.4 / −14.8 dB mean, soft −29.8 / −30.7 / −30.6 dB, room tone alone −51.3 dB. Wind and crickets, door latch and creak at 6.2 s, a warm low chord from inside after the door opens, the person's steps and the crate, paws on boards at the rendered foot contacts, a thump as the dog lies down and a sigh as it settles. The voice is intelligible only roughly: it is a formant synthesizer, not a recording. ## Invariants (measured from the render, `trace.json`) `python3 tools/paeleo/check_handmade.py paeleo/handmade/26-pixijs-stage` → ALL OK. Onsets, each measured from the render (GPU read-back of the dog texture for position; projected skull/muzzle from the scene graph that placed the sprites): | choice | score | measured | how | |---|---|---|---| | retreat | 1.000 | 1.000 (+0.000) | centroid moves away > 0.4 px/frame | | watch | 2.333 | 2.333 (+0.000) | head lifts > 2 px from its backing-away carriage | | refuse | 3.417 | 3.500 (+0.083) | head swings > 12° off the door | | watch (turns to look) | 7.458 | 7.500 (+0.042) | projected muzzle points back to the door (body still down) | | approach | 14.708 | 14.708 (+0.000) | centroid moves toward the door > 0.4 px/frame | | lie down | 23.792 | 23.792 (-0.000) | body starts down at the sill | | looks in | 26.4 | 26.458 (+0.058) | muzzle turns > 0.02 m deeper than when the head lies along the body | I3 uses `x_screen` (static camera). The body creeps 0.24 m toward the door in the last 2 s of the approach (inside the lie-down) so that its front lies in the doorway; the score's x is kept in `x_score`. ## What failed and what I changed See `ledger.jsonl`. In short: a dark dog on a dark yard (lit the coat, lightened the yard); zebra boards (atlas contrast pulled down); a single outline merged head and body (head moved to its own render texture); bead-chain legs (axis cores); tints over 1.0 turned green (hue-preserving clamp); the look-in first read as a lift toward the light (neck capped, muzzle pitch no longer rises); the head-led look first stayed at the front of the body (neck now carries 72 % of the head yaw); a lying tail flipped up (it curled to the far side; now to the camera side, continuous); the film was 23 MB (two-pass 2.1 Mbit/s). Two of my first onset detectors were wrong (velocity stop for retreat→watch fired late; muzzle depth > 0 fired before the look) and were replaced by the criteria above, which match what a viewer sees. ## What is still weak (my view; not a verdict) - The coat is mottled: at full speed it reads as a dog, but up close it is a patchwork of archive grey tinted brown. - Legs in fast phases (getting up at 7.6–8.0 s) are still beaded and the stance is gangly. - The mid-turn frames (3.7, 8.2 s) read as a dog seen end-on, but small (≈110 px); the stage camera never comes closer. - The person is a dark silhouette with little gesture; the call is mostly carried by the sound and the reaching arm. - The voice is a formant synthesizer: the words are hard to make out. - The porch steps (x = 4.9) are just off the right edge of the frame. ## What I did not do No mobile operator page, device benchmarks or pixel-difference comparisons (the SCION deliverable contract asks for them; out of scope for this film). No culling. No independent critic (not mine to run). No camera move. No real recorded voice. GPU time on real hardware not measured (SwiftShader only). ## Repair round 1 (after the critic's DOES NOT PASS, reviews/round1.md) - **Turning to look (7.46).** The body now stays fully down until 7.74 s (lie held at 1), and the head swings round over the *far* shoulder (never face-on to the camera) to a left profile: measured from the encoded build, the projected muzzle points screen-left from 7.542 s, in every frame 7.542–7.792 s (7 consecutive frames) while the chest is within 10 % of its lowest height (0.166–0.182 m). The body rises 7.74–8.10 s and only then turns back, 8.0–8.75 s, through its far side, with the neck held level (no front-facing head over a side-on body; no bear-like twist). Frames checked from film.mp4. - **Head low.** Neck held at −46° and ears laid back for the whole approach: the top of the head (ear tips, skull top) is below the top of the back in 218 / 218 frames of 14.7–23.8 s (projected, from the scene graph). - **Lying.** The fold now runs 23.70–24.30 s (lie 0.5 at 24.00 s; forelegs visibly folding in the 24.00 and 24.04 s frames of film.mp4), and the dog lies diagonally *across* the sill in depth (yaw 138°): head and forelegs on the sill in the opening, hindquarters and tail out on the porch. - **Paws.** Paw contacts raised to about −17 dB peak: in film.mp4 the walk window 15–23 s peaks at −16.1 dB against −26.4 dB in the quiet window 11.8–14.4 s (room tone + room chord, no voice): +10.3 dB. - **Shadow.** Every dog fragment is now projected from the lamp (was every third), with smaller blobs and a lighter blur, so the shadow in the light pool carries the dog's back, head and legs. - assets.json now names build/atlas/atlas.json as the per-fragment provenance record.