Hillclimbing

Agentic model searches — agents draft, backtest, and select models; you promote and deploy the winner.

rebase hillclimb runs agentic model searches with hillclimb: coding agents draft, debug, improve, and ensemble emflow Predictor classes. Every candidate is backtested leakage-safe on the problem's validation split, and the shipped model is selected on a hidden holdout the agents never see.

The artifact that flows through the whole loop is a single Python file exposing get_model() -> emflow.Predictor — the same class that won the backtest is what you commit to your workspace repo and deploy.

Requires the hillclimb extra:

uv pip install "rebase-toolkit[hillclimb]"

Start a search

Hosted on the platform by default (a long-running Cloud Run job), or on your own machine with --local:

rebase hillclimb start emflow://gefcom2014:solar --budget 2h
rebase hillclimb start emflow://gefcom2014:solar --budget 2h --local

Any verifiable target works: emflow registry problems (emflow://<name>) or plain problem folders. Useful options:

Option	Description
`--budget`	Wall-clock search budget, e.g. `2h`, `30m`.
`--name`	Search name shown in run listings.
`--model`	Agent model, e.g. `sonnet`.
`--backend dummy`	Run the search loop without agent calls (smoke tests).
`--local`	Run on this machine instead of the platform.
`--project`	Project the platform run is filed under (default `hillclimb`).

Watch and control

A hosted search syncs its state (candidate tree, scores, budget) to the workspace artifacts bucket every ~30 seconds:

rebase hillclimb list
rebase hillclimb status <run-id>   # candidates, best score, budget left
rebase hillclimb stop <run-id>     # graceful: parks after the current operator

rebase run get <run-id> shows the underlying platform run and its events.

How a search works

A baseline candidate (the problem's reference model, when it ships one) is evaluated first — agents must beat a real scored floor.
Agents draft diverse solutions, debug failures, and improve the best candidate; near the end of the budget an ensemble operator combines the top candidates.
Every candidate is fit and scored by a generic evaluator on the validation split — sandboxed, offline, and credential-free, so generated code can never read holdout data or secrets.
The orchestrator separately scores each ok candidate on the holdout split and selects the shipped model by a rank-blend of both signals — an overfit validation score gets vetoed by its holdout rank.
A finished search runs one official emflow verification on the selected model, recording the number of candidates tried (n_trials) for selection honesty.

Promote and deploy

Fetch the selected model into your workspace repo as versioned source, then deploy it like any other model:

rebase hillclimb promote <run-id>            # writes models/<problem_id>.py
# review, commit, open a PR (protected environments deploy through gitops)
rebase model deploy models/gefcom2014_solar.py

Billing and state

Hosted searches bill agent calls to the workspace's configured Claude credentials — a CLAUDE_CODE_OAUTH_TOKEN for subscription billing, or an API key. --local searches use your local Claude login.

Search state lives under gs://<artifacts-bucket>/hillclimb/<sync-id>/; set REBASE_HILLCLIMB_BUCKET for the CLI to read it. Server-side requirements (job image, secrets, artifacts bucket) are documented in the platform operations guide (toolkit/HILLCLIMB.md).