Sequential Decision Problems
The problem framing behind Rebase models - states, actions, exogenous information, and objectives.
Most energy problems are not one-shot predictions — they are sequential decision problems (SDPs): decisions made repeatedly over time, under uncertainty, where each decision changes the situation the next one faces. Forecasting for the day-ahead market, dispatching a battery, bidding a wind portfolio: all the same shape.
emflow, the modeling framework the toolkit builds on, adopts this framing explicitly — based on Gymnasium and Warren Powell's universal sequential decision framework.
The Loop
Every SDP is the same loop, repeated until the end of the horizon:
state S_t → action a_t = policy(S_t) → exogenous information W_t+1 arrives
→ new state S_t+1 → cost / contribution C(S_t, a_t, W_t+1)| Element | Meaning | Energy example |
|---|---|---|
| State | Everything knowable when the decision is made | Observed load history, issued weather forecasts, battery state of charge |
| Action | The decision the model takes | A 24-hour forecast, a charge/discharge schedule, a bid curve |
| Exogenous information | What the world reveals after the decision | Actual load, realized prices, the weather that occurred |
| Transition | How state evolves given the action and new information | Settlement, state-of-charge update, new market session |
| Objective | The cost or contribution accumulated over the horizon | Pinball loss, imbalance cost, trading revenue |
The goal is a policy — a rule mapping states to actions — that maximizes expected contribution (or minimizes cost) over the whole horizon, not on any single decision.
Two details make this framing unforgiving and realistic. The action is chosen before the exogenous information arrives — a model can only use what is knowable at decision time. And the score settles later, once the outcome is revealed — which is why point-in-time data discipline (see Backtesting) is the foundation of trustworthy evaluation.
Forecasting Is an SDP
Forecasting fits the loop as a special case where the action is the forecast: at each forecast origin the state is everything knowable at that moment, the action is the predicted trajectory, the exogenous information is what actually happened, and the objective scores the two against each other. This is exactly how emflow's ForecastEnv evaluates Predictors — and why the same machinery extends to trading (TradingEnv) and dispatch problems, where Optimizers and Agents act in the same loop.
Model First, Then Solve
emflow's method is problem-centric — understand the problem fully before reaching for a solution:
- Define the considered energy system;
- Define state, action, and exogenous variables;
- Create the environment and the transition function;
- Define the objective (cost or contribution);
- Create the model (predictor, optimizer, agent, and/or simulator) to operate in the environment;
- Run the sequential decision loop and evaluate performance.
Steps 1–4 define the problem; steps 5–6 create and evaluate the solution. On Rebase, steps 1–4 are packaged as verifiable targets, step 5 is your model code (or an autonomous search writing it), and step 6 is backtesting.