2026 – present · Agents · AutoML In progress

Competition Science Agent (CSA)

An autonomous research agent for data-science competitions. You give it a competition link. It researches prior work, proposes and ranks ideas, implements and checks the best one against the real metric, and submits only if a judge decides the result is worth it.

Why competitions

ML competitions offer something rare in AI evaluation: a held-out test set, a fixed metric and thousands of human baselines. Together they form an objective, automatic evaluator that can't be talked into a wrong answer, the same kind of evaluator that makes systems like AlphaEvolve work. CSA separates a fast, unlimited local evaluator, which drives iteration, from the rate-limited leaderboard, which serves as the final check that can't be gamed.

We surveyed nine agents and benchmarks (MLE-bench, AutoKaggle, Agent K, MLEvolve and others) and found the same gaps again and again: train/test contamination, no learning from real competition feedback, single-modality setups, and no live system that finds and enters ongoing competitions.

How it works

CSA architecture: knowledge base, LLM panel, evaluator pool and hypothesis database around a four-phase controller loop — CSA architecture: a knowledge base, an ensemble LLM panel, an evaluator pool and a hypothesis/result database, run by a four-phase controller loop.

Contextualize: fetch the competition from Codabench or Kaggle, turn it into a structured task card, and search arXiv, Semantic Scholar, GitHub and Hugging Face for prior work to build a knowledge graph.
Refine and rank: a mixed panel of local and hosted LLMs proposes approaches grounded in that graph. An evaluator pool scores each one on an academic rubric (NTESCRU) and on novelty.
Implement and validate: turn the top hypothesis into a runnable solution, run it in a sandbox with a repair retry, and compute the real competition metric.
Judge and submit: combine rubric, novelty and validation scores into a submit or iterate verdict. Submission respects quotas and is a dry run by default.

Every stage has a deterministic offline fallback (--no-llm), so the whole pipeline runs without a model for testing and demos. A FastAPI web interface streams each stage live. After a real Codabench submission, it attaches the leaderboard score to the hypothesis that produced it.

AutoCompete

The engine ships as AutoCompete, a competition development kit. You give it a description and train/test data, and it returns the best solution it finds. A single protocol (ASDP) lets any search engine (OpenEvolve, an LLM agent, AutoML) run on any platform (a local grader, Codabench, Kaggle).