Can reinforcement learning trade crypto profitably?

Give an agent 1-minute data on a survivorship-free universe of Binance USD-M perpetual contracts. Let it choose which contracts to hold, long or short, at what size, and when to change its mind. Charge it what trading actually costs. Does it make money?

The question is worth asking carefully because the published record on it is thin. Results are usually single training runs, costs are often absent, and the reference point is usually a buy-and-hold line rather than a random agent trading at the same intensity. Each of those choices flatters the method.

354 contracts, 56 of them delisted before the sample ends and kept in the history at the dates they actually traded. The universe is rebuilt point-in-time, ranking by dollar volume over the strictly preceding 30 days, with eligibility filters applied before ranking so they cannot depend on an outcome.

Decisions are hourly across 20 slots, long and short, capped at 25% of the account per contract and 100% gross. There is a deadband around zero, so holding nothing is an easy choice rather than a knife edge.

Rolling walk-forward over 20 windows: 18 months to train, 3 to validate, 3 to test, with a 7-day purge and a 1-day embargo at every boundary. Out-of-sample testing spans November 2021 to August 2026. The scaler is fitted on training rows only, the checkpoint is chosen on validation, and test is touched once per configuration.

Costs sit inside the reward rather than being subtracted afterwards: 5 basis points taker fee and 2 basis points of base slippage per fill, square-root market impact scaled by realised volatility, participation capped at 5% of the interval’s traded volume, and funding at realised rates on its true settlement bar.

The scoring rule was written down in advance: the 25th percentile of out-of-sample risk-adjusted return across random starting points, measured against a random agent trading at the same turnover. That choice of statistic turns out to matter more than anything else in the study.

No. All 219 runs lost money, and five of the six reward designs lose to a random agent trading at their own intensity. The best risk-adjusted return recorded anywhere in the study belongs to a random agent that buys and holds for roughly 42 days.

Probability of backtest overfitting across the configuration family is 1.00, which means the in-sample ranking of these designs carried no out-of-sample information in any split, and nothing survives correction for multiple comparisons. That would be an unremarkable null. Three results underneath it are not.

Splitting each design’s profit into what its positions earned before costs and what the trading cost separates two very different failures.

Three of the six designs found a positive directional edge out of sample across eight quarters. The ordering inverts. The design that loses most on net is the one that was most right about direction, by a factor of three over the next best.

Both designs at the top of the net table have a negative return before costs, so they rank well by being cheap rather than by being right. That is the mechanism behind a probability of backtest overfitting of 1.00: the ranking measures restraint, and restraint does not generalise as skill.

The reward controls trade size rather than trade frequency. Closes per hour sit between 9.66 and 10.23 across all six designs, a spread of 6%, while average trade size varies 19-fold. The designs are equally active and differ only in timidity.

One design choice was worth more than every other variable tested. The policy processes each of the 20 contracts through the same small encoder and shares one decision head across them, so it cannot treat one contract differently from another except through their data. The obvious alternative is a plain network that sees all 1,192 inputs as one undifferentiated vector.

A 19-point gap, with no overlap between the seed ranges. For scale, reward design spans about 2 points and feature choice does not separate at all. The plain network did not learn a worse policy, it never learned to hold a position: it turns over 85% of the book every hour and is wiped out in all eight quarters.

Given a flat vector it cannot discover that the input is really 20 contracts by 59 columns plus 12 account-level numbers, so its 20 outputs share no weights and nothing connects a contract’s data to that contract’s own position. It carries 16 times more parameters than the shared-encoder policy and still loses by 19 points, so this is about telling the network the structure of the problem rather than about model size.

This sets the floor rather than raising the ceiling. The shared-encoder policy still loses 89% of capital and still fails against a random agent at its own turnover. On a cost-free simulator the two would have looked far closer, and much of the published work in this area charges no costs.

One configuration, one dataset, ten random starting points: a range of 10 points of risk-adjusted return. Turnover across 40 starting points spans 50-fold on the same configuration. That is large enough to swamp every effect anyone would want to measure here.

A single training run, which is how most of the published work in this area reports, carries a 40% false-positive rate on this configuration. Two in five such studies would report a profitable median for something that loses 2.7 points over ten starts. That is a sufficient explanation for the field’s replication record without assuming anyone behaved dishonestly.

Costs are charged inside the reward, so the agent is paid net. One objection is that a single merged number cannot tell the agent whether it lost money on direction or on fees, leaving it unable to learn that trading is what harms it. The fee is between 1.6% and 10.2% of a typical hourly price move, so that attribution has to be recovered from under noise 10 to 60 times larger than the signal.

Showing the agent what its last decision cost changed nothing measurable. Charging it a penalty in direct proportion to what it had just traded, at 25 times the real cost rate, cut turnover 45% and costs by 8.7 points of capital while holding the return before costs flat, improving net by 5.6 points.

Four measures move together there, which is the combination that matters, because the penalty removed trading that was not paying for itself rather than trading that carried the edge. It still does not clear the pre-registered threshold, and the reason is the finding: the penalty lowers the lower half of the turnover distribution and leaves the upper quartile untouched. Some starting points still trade 12% of the book per hour while being charged twenty-five times what trading costs.

A cost signal, however loud, stops the marginal churner and not the committed one. The experiment that follows is therefore structural rather than economic: cap how much the portfolio may change per hour so churning is impossible instead of expensive, then ask whether the edge before costs survives at a turnover the cost structure can carry.

Three are worth naming because each would have produced a confident wrong answer rather than a crash.

A placeholder price was being used to settle a rotation into a slot whose contract had changed. The equal-weight reference agent reported a 31-fold return where hand-computed arithmetic gives 1.46-fold. Every run predating the fix was discarded.

A feature comparison was run where a narrower feature set survived the row-completeness rule on more rows than a wider one, so the arms sat on different samples and the comparison silently measured coverage rather than information. All arms are now required to share one row-count signature before any claim is made.

An early cost decomposition was built by summing trade-ledger columns across both windows and starting points, which double counts and adds per-window capital fractions that each restart at one. The ledger is first-in-first-out and realised-only, so its sum legitimately differs from total return by the mark-to-market on inventory still open at window end. Decompositions now come from the simulator’s own accumulators.

At five random starting points, a feature comparison produced +42.5% over four out-of-sample quarters on verified-identical rows, with a plausible mechanism, and it was written up. Extending the same configuration to ten starting points moved its median from +0.56 to −2.67 and its total from +42.5% to −85.9%, because the first five runs were the upper half of the distribution.

With all seven feature sets at ten starting points the apparent ranking then inverted, and no pair of feature sets is distinguishable after correcting for the 21 comparisons. The honest reading is that the feature ranking is noise at every sample size tried.

The pre-registered statistic refused the +42.5% before the extension explained why. That is the strongest argument in the study for writing the acceptance bar down before looking, and it is worth more than the finding would have been.

This is hourly decisions on one venue under one modelled cost stack, so it says nothing about faster horizons, other exchanges, market making or what a real desk would have paid. The 22.3% figure before costs is the least trustworthy number here, because the design that produced it trades 32% of the book per hour, which is far enough into the impact model that its own estimate is the most model-dependent of the six.

The feature comparison covers four late windows only, because open-interest history before December 2021 does not exist for most contracts. The random reference agent is matched on turnover rather than on concentration, although the policies hold a median 15 of 20 slots and touch 35 distinct contracts per window, so the diversification difference is small.

Two of the twenty walk-forward windows were used for screening in breach of the original plan, which burned them and left two untouched. A secondary cadence grid was dropped for time, so the cadence question rests on the reference agents alone. Nothing here is a strategy, and no model was run on live data.