Experimental Model Research

Research question: Which learned representations are useful for generating images, not just reconstructing them?

Progress

• Built a differentiable Gaussian renderer and training pipelines for image reconstruction and generation.
• Ran iterative representation tournaments: compared structural, feature-based, hybrid, and multiscale candidates, then modified and retested promising variants.
• Set up a controlled CIFAR-10 tournament with 40,000 training images, 1,000 development images, and a separate 1,000-image locked evaluation split.
• Evaluated reconstruction, generation, and computational cost separately. Representations that reconstructed images well were not necessarily easy for a generator to learn.
• Tested adaptive representation sizes and began investigating scene-oriented representations for future video and world-model work.
• At a 12-splat budget, coverage-aware allocation lowered foreground error against a tuned fixed-allocation baseline on 18 unseen scenes, but regressed under noisy single-view stress. Separately, a DCT control beat unconverged Gaussian fits in a small held-out comparison.

Current limitations: Reconstruction gains have not consistently transferred to generation. Early scene representations still struggle with object coherence and occlusion, and some Gaussian fits have not converged. Next comparisons will test generative learnability and noisy-view robustness under matched budgets and convergence checks.

PyTorch · Differentiable Rendering · Generative Modeling · GPU Training

Research question: Can a world model retain a consistent scene after objects leave the recent visual context and later come back into view?

Progress

• Built a procedural 3D world and trajectory generator with multiple scene families and motion programs, capturing RGB-D observations, camera poses, metric depth, and action data for training and evaluating world models.
• Developed reproducible dataset pipelines with controlled train/development/test splits, scene-integrity checks, frame-level validation, and automated preflight gates to catch invalid captures before experiments.
• Set up controlled trajectory splits and diagnostic cases that test whether previously seen geometry survives a revisit.
• Compared atlas, mesh, and hybrid memory representations using coverage, false-foreground, and revisit measurements, to expose tradeoffs.
• In a 12-case development check, rejecting unreliable source patches reduced false foreground but also discarded some correct geometry.

Current limitations: Retaining complete scene geometry without artifacts is still not solved. The next comparisons will use reserved scenes to test coverage and incorrect foreground together, including whether larger or guided memory stores actually help.

World Models · RGB-D Evaluation · Long-Horizon Memory · Controlled Benchmarks

Related: Earlier interactive generative world-model work

Research question: Can compact models learn individual operations and combine them to solve unfamiliar problems?

Progress

• Built experimental reasoning architectures with reusable operations and expert routing, synthetic datasets that vary operations, problem structures, and wording.
• Tested multi-step composition and unfamiliar instructions to distinguish reusable operations from memorized task patterns.
• Compared context-aware routing variants on 5,706 fresh development cases across three seeds. Rejected candidates whose aggregate gains hid individual seed regressions.
• Added automated experiment gates that stop progression when evaluation checks fail or required artifacts are missing.
• Found that matching an expected program was insufficient when execution still failed; prepared stricter scoring that requires both for the next comparison.

Current limitations: Generalization still breaks on unfamiliar wording and problem structures. The next planned comparison tests expert complementarity and unseen operation combinations, requiring successful execution and no seed-level regressions before promotion.

Learned Routing · Synthetic Data · Multi-step Benchmarks · Regression Gates

Research question: How can a model retain useful information and retrieve it accurately as the surrounding context changes?

Progress

• Built persistent-memory and associative-retrieval prototypes, including variants that control which tokens update memory.
• On a structured six-panel memory task, selective memory averaged 99.68% across three held-out test seeds; an all-token-write control averaged 17.67%.
• Built stress cases that vary input order and context boundaries, with separate validation, test, and stress conditions.
• Compared memory and training variants under matched seeds to investigate retrieval failures.
• A development pilot found a promising gain from enforcing first-read consistency on a harder retrieval task.

Current limitations: The large held-out advantage is specific to the structured six-panel task. The next step is to test the first-read pilot with unfamiliar syntax, context boundaries, and longer multi-step retrieval before claiming broader transfer.

Neural Memory · Associative Retrieval · Controlled Ablations · Multi-seed Evaluation

Research question: Can small neural models learn useful physical relationships from real flight data without requiring large architectures?

Progress

• Trained compact neural predictors on real quadrotor recordings from the University of Zurich's NeuroBEM dataset.
• Compared predictions against ridge regression and examined per-flight errors.
• Added flight and date-level split checks to catch leakage. An external evaluation gate stops before reading targets when independent flight and date coverage is insufficient.
• Found that performance changes substantially between flight conditions, limiting the current generalization.

Current limitations: Gains do not hold consistently across flight conditions. The next step is gathering sufficient independent flight and date coverage for further testing.

System Identification · Time-Series Modeling · Baseline Comparisons · Leakage Checks

Source: NeuroBEM - Bauersfeld et al. (2021)

Research question: Can a compact model learn separate rules from examples and retrieve the relevant knowledge for a new task?

Progress

• Built structured learning tasks and specialist-memory variants to study rule retrieval and interference.
• In a controlled support-selection ablation, support-token readout reached 61.52% versus 12.31% for pooled readout on the evaluated task.
• Compared retrieval and readout methods to test whether pooling obscures relevant examples.

Current limitations: The readout gain is task-specific; transfer to unfamiliar rules and resistance to interference are unresolved. The next tests will use unseen rule combinations and matched readout controls.

Memory Readout · Structured Learning · Ablation Studies · Interference Testing

Research question: Can recurrent and state-space architectures learn long-range dependencies efficiently?

Progress

• Implemented recurrent and state-space language-model prototypes and tracked convergence and training instability.
• Compared architecture and training variants while developing long-delay dependency and stability tests.

Current limitations: No consistent quality or efficiency advantage over established approaches has been demonstrated. Matched baselines and longer-dependency tests are needed before scaling.

Recurrent Models · State-Space Models · Training Diagnostics