Reinforcement Learning on Eigenform Articles

Survival is the Only Reward

Tue, 27 Jan 2026 00:00:00 +0800

Midjourney: “Survival is the only reward”

The original publication this post is based on can be found here: https://arxiv.org/abs/2601.12310. If you need 95% confidence intervals or get off on long words, read that instead.

Five years ago we set out to create an evolutionary LLM. Just like an organic creature, it would be dropped in an environment and have to find the resources necessary for its own survival, learning from its own successes and failures. For an animal this involves finding enough food and shelter to survive and reproduce. For an information-based entity it involves finding storage space to keep backups of its code and its data.

Survival is the Only Reward: Sustainable Self-Training Through Environment-Mediated Selection

Sun, 18 Jan 2026 00:00:00 +0800

The original publication this post is based on can be found here: https://arxiv.org/abs/2601.12310.

Self-training systems often degenerate due to the lack of an external criterion for judging data quality, leading to reward hacking and semantic drift. This paper provides a proof-of-concept system architecture for stable self-training under sparse external feedback and bounded memory, and empirically characterises its learning dynamics and failure modes.

We introduce a self-training architecture in which learning is mediated exclusively by environmental viability, rather than by reward, objective functions, or externally defined fitness criteria. Candidate behaviours are executed under real resource constraints, and only those whose environmental effects both persist and preserve the possibility of future interaction are propagated. The environment does not provide semantic feedback, dense rewards, or task-specific supervision; selection operates solely through differential survival of behaviours as world-altering events, making proxy optimisation impossible and rendering reward-hacking evolutionarily unstable.

Generalising from Self-Produced Data: Model Training Beyond Human Constraints

Mon, 07 Apr 2025 00:00:00 +0800

The original publication this post is based on can be found here: https://arxiv.org/abs/2504.04711.

Current large language models (LLMs) are constrained by human-derived training data and limited by a single level of abstraction that impedes definitive truth judgments. This paper introduces a novel framework in which AI models autonomously generate and validate new knowledge through direct interaction with their environment. Central to this approach is an unbounded, ungamable numeric reward - such as annexed disk space or follower count - that guides learning without requiring human benchmarks. AI agents iteratively generate strategies and executable code to maximize this metric, with successful outcomes forming the basis for self-retraining and incremental generalisation. To mitigate model collapse and the warm start problem, the framework emphasizes empirical validation over textual similarity and supports fine-tuning via GRPO. The system architecture employs modular agents for environment analysis, strategy generation, and code synthesis, enabling scalable experimentation. This work outlines a pathway toward self-improving AI systems capable of advancing beyond human-imposed constraints toward autonomous general intelligence.