How the reinforcement-learning study was built

Three public sources, all from the Binance public data archive and its metrics endpoint, covering USD-M perpetual futures. Nothing paid and nothing proprietary.

Two archive quirks have to be handled or the panel silently misaligns. The kline files change their header convention partway through the history, and an early era stamps timestamps in microseconds rather than milliseconds. Both are normalised on read.

Open-interest coverage is a hard ceiling rather than a fetch limitation. The exchange publishes metrics for the second-largest contract only from December 2021, and earlier dates return not-found for most contracts. A backward extension across all 354 contracts ran for 148 minutes and added zero rows. That is why the feature comparison is restricted to the later windows.

354 contracts, of which 56 were delisted before the end of the sample and are kept in the history at the dates they actually traded. Selection is point-in-time: on each month boundary, contracts are ranked by mean daily dollar volume over the strictly preceding 30 days and the top 40 are eligible.

The eligibility filters are applied before ranking, so they cannot depend on an outcome. Quote currency must be USDT, dated contracts are excluded, and non-crypto underlyings are excluded.

One decision per hour. The action is 20 numbers in the interval minus one to plus one, one per slot, read as the target fraction of the account to hold in that contract. Positive is long, negative is short, and shorts are allowed because these are perpetual futures.

Four rules are then applied in order. Anything between minus and plus two hundredths is set to exactly zero, so holding nothing is an easy choice rather than a knife edge. Each contract is capped at a quarter of the account. If the absolute weights sum above one, everything is scaled down proportionally, so there is no borrowing. Slots holding a contract that is not trading that hour are forced to zero.

The observation is, per contract, its feature columns plus three numbers about the agent’s own position: the weight it currently holds, whether it is up or down on that position, and how long it has held it. Twelve account-level numbers follow: gross and net exposure, drawdown from the high-water mark, log equity, how many contracts are live and how many are held, progress through the episode, realised profit, average trade size, cumulative cost and cumulative funding, and the gross cap.

Positions are forcibly cut back at market if gross exposure drifts above one and a half times the cap, and an episode ends if equity falls below a quarter of its starting value.

Costs are charged inside the reward rather than subtracted afterwards, which matters because the two per-trade reward designs would otherwise be blind to them, and a cost-blind trade reward provably teaches churning.

The impact term is scaled by the contract’s realised per-step volatility rather than by participation alone. Without the volatility term the expression is dimensionally wrong: an early version billed a ten-thousand-dollar trade in the largest contract 2.3% of slippage.

Participation is measured against a fixed account size rather than against equity fractions, or the cap never binds. Trades below one millionth of the account are zeroed, which stops single-precision round-trip noise from being recorded as roughly a thousand phantom trades per run.

Twenty rolling windows. Each takes 18 months to train, 3 to validate and 3 to test, with a 7-day purge and a 1-day embargo at every boundary. The purge length is the longest feature lookback, so no training row can see a bar that overlaps a test target.

The scaler is fitted on training rows only. The checkpoint is chosen on validation equity across four training chunks, never on test. Test is touched once per configuration.

Screening used windows 1, 3, 5, 8, 11, 14, 17 and 19. Out-of-sample testing spans 17 November 2021 to 17 August 2026.

Six designs, identical in everything else. Two speak only when a position closes; four speak every hour.

Seven, run on the identical environment and cost model. Holding nothing, buying and holding the universe, buying and holding one contract, equal weight, a momentum rule, a reversal rule, and a random agent.

The random agent is the one that matters. It is matched to the measured turnover of the policy it is compared against, by holding its random book for however many steps reproduces that trading intensity. Comparing a policy to a buy-and-hold line instead would flatter it, because the policy pays far more in costs. A ladder of random agents at several holding periods is also run, so each policy can be judged against the rung nearest its own turnover.

Holding nothing is retained even though it never trades. Doing nothing is a legitimate strategy here, and disqualifying it for having no trades would remove the baseline that detects an agent learning to sit out.

The registered statistic is the 25th percentile of pooled out-of-sample risk-adjusted return across random starting points, not the median. Ranking on the median would let a result carried by three lucky starts out of five be reported as an edge, which is precisely what happened once in this study and was caught by this rule.

The trial count for any multiplicity correction is the number of distinct configurations, never the number of runs. Seeds of one configuration are not independent trials. Pooled out-of-sample series are scored directly; per-window figures are never averaged into a headline, and the number of windows is never multiplied into a strategy count.

Probability of backtest overfitting is computed by combinatorially symmetric cross-validation over the configuration family. Family-wise error is controlled with Holm and with Benjamini-Hochberg. The deflated Sharpe uses the distinct-configuration count as its trial count.

These run before any result is read, and a failure stops the study rather than producing a footnote.

The accounting gate is the one that earned its place. It caught a placeholder price being used to settle a rotation into a slot whose contract had changed, which reported a 31-fold return where the hand-computed truth was 1.46-fold.

219 runs, zero failures. The reward comparison is balanced at five random starting points per design; one design was later extended to forty for the cost experiment, and that extension is reported separately rather than mixed into a five-start table.

Three deviations from the plan are on the record. Two of the twenty windows reserved as holdout were used during screening, which burned them and reduced the reserve to two untouched windows. The feature comparison was restarted on a later window set once it became clear that open-interest coverage would otherwise have made its arms sit on different samples. And a secondary cadence grid was dropped because it measured out at roughly 26 hours against a four-hour budget, so the cadence question is answered by the reference agents alone, which is weaker evidence than a policy grid would have been.

One published number was corrected after the fact. An early version of the cost decomposition was built by summing trade-ledger columns across both windows and starting points, which double counts and adds per-window capital fractions that each restart at one. The ledger is first-in-first-out and realised-only, so its sum legitimately differs from total return by the mark-to-market on inventory still open at window end. Decompositions come from the simulator’s own accumulators instead.