The setup
Polymarket runs binary markets on whether Bitcoin will be higher at the end of each five-minute window than at the start. A correct share pays $1. Thousands of these contracts trade every day, mostly against bots.
Our idea was ordinary. We used Binance order flow (price drift, momentum, buy/sell imbalance, volatility) to estimate the probability of "Up" at every second of the window, and traded when the market's price disagreed with ours by enough. The engineering was careful: two model classes, cross-validation that kept each window's ~300 correlated seconds together, walk-forward backtests and a live paper-trading recorder. Nothing ever placed a real order.
The first headline was a 60.9% win rate over 12,571 backtested trades, roughly 24 standard deviations above chance. A stricter test on days that didn't exist when we tuned the model said 62.7%. All of it was wrong.
1Prices you could never have traded at
Our historical market prices came from Polymarket's price-history endpoint, which returns one point per minute. We forward-filled them to every second. A model reacting to fresh Binance data was being compared with a market price up to a minute old. Inside our trade window those filled prices sat a mean of 12.9¢ away from the real order book. Most of the "edge" was the model knowing what had happened since the last price print.
Live paper trading quoted real prices and was far weaker. The model's live and backtest probabilities correlated at 0.9986, so the model wasn't the problem. The data was. The tell had been in plain sight: the backtest found a tradeable signal in about 99% of windows. On the real order book it was 6.5%. A strategy that trades almost every window is usually measuring an artifact.
RuleCheck the resolution of every input. A price series with fewer distinct values than your decision cadence is lagged, and a lagged price next to a fresh signal is lookahead in disguise.
2Unbiased noise, biased results
Next we used individual trade prints as the price. They had second-level resolution and, on average, they were honest: their mean error against the order book was +0.06¢. They still produced a fake edge.
A trading rule acts only when the model and the price disagree by more than a threshold, and noise is exactly what makes them disagree. The filter picked out the seconds where noise had made a contract look cheap. Zero-mean noise went in and a positive "edge" came out.
RuleErrors that average to zero don't protect you when the strategy selects on the error. Any threshold rule turns symmetric noise into a one-sided bias.
3The wrong label
Our outcome label was the obvious one: did the Binance price end the window higher than it started? Polymarket doesn't settle on Binance. Each market's own description says it resolves on Chainlink's time-weighted average price. When we finally compared our label with the market's actual settlements, it disagreed on 8–12% of windows. The errors were not random.
The label was right when the answer was obvious and close to a coin flip when it was hard. The hard windows are the only ones where a model can make money.
It did more than add noise. Grading Polymarket's prices against our label made the market look poor, with a Brier score of 0.201. Against real outcomes the market scored 0.159. Our model, which had appeared to beat the market as a forecaster, was behind it (0.187 vs 0.159). As the sample grew, the bad label made the fake result more significant, not less.
We caught it through an anomaly in the last seconds of each window. When the market priced a side above 90¢ twenty seconds before resolution, it was "right" by our label only 57% of the time. A real market that close to settlement should be right more than nine times in ten. So we checked our ruler instead of the market.
A window settles Up if Chainlink's 60-second average price at its last second is at least that average at its first. That rule matched 99.79% of 2,421 windows. Polymarket had switched from a single price snapshot to this average on 7 August, after researchers showed traders pushing the price in the final seconds. Our Binance label agreed with settlement 95.8% of the time before the switch and 88% after it, and our training data straddled the switch.
RuleRead the contract before you model it, and grade against what pays out, never a proxy for it. Here the true outcome was available for every window.
4Counting seconds as trades
Our first re-score of the live test counted every second the signal fired as its own trade. All those seconds in one window share a single outcome. Counting about six correlated seconds as independent inflates z by roughly √6. The same mistake inflates fake wins just as easily.
RuleThe unit of evidence is the thing with an independent outcome. In time series, cluster your errors at that level.
5The fee nobody charged
Polymarket charges takers 0.07 × p × (1 − p) per share. On a 50¢ contract that's 1.75¢, close to 4% of the price. We had modelled the spread carefully and never charged this fee, and every edge we had measured was about the size of the fee.
RuleList every cost a real fill pays (spread, fee, slippage, latency) before you look at any P&L. Polymarket introduced these fees in January 2026 specifically to stop the kind of edge we were chasing.
6A join that silently used a fifth of the data
Our real-order-book backtest traded in about 6% of windows. The live paper trader, running the same model, traded in about 90%. Replaying the live signals offline gave a correlation of 1.0000, so the model agreed with itself and the backtest was wrong.
It scored the model every fifth second and required an exact match with an order-book snapshot. Snapshots are taken every ~5 seconds at an arbitrary offset, so only 20% ever lined up: 598 of 2,677 windows. The backtest also traded at whichever second happened to align, not the first second the signal appeared.
RuleWhen two systems disagree, find out which one is wrong before choosing the one you like. Check row counts after every join.
7Fills that happened are not fills you would get
The most convincing result came last. Across 6.2 million historical fills, makers' profit rose steadily with our model's opinion, from −3.9¢ per share where it disagreed most to +3.4¢ where it agreed most. That looked like a recipe: post orders only on the side the model favours.
Two checks took it apart. First, the model at each second used that second's average Binance price, which includes trades after the fill. Delaying the model gave +2.2¢ at no delay, +1.8¢ at one second, +1.25¢ at two and +0.35¢ at five. Part of it was real, but it lasted about two seconds.
Second, the historical fills went to whoever was first in the queue. We rebuilt the strategy as a millisecond-by-millisecond replay of every book change and every trade, with our order waiting its turn and following the best bid after a realistic reaction delay. The best case, always first in line with a 50-millisecond reaction, broke even. Choosing orders by the model made it worse. Our orders didn't get the good fills. They got the ones that arrive when the price runs over you.
RuleAn analysis of trades that happened is not a simulation of trades you would have made. Queue position, reaction time and who gets filled are the strategy.
What survived
After fees, with real prices and real outcomes, nothing we tried makes money on these markets.
| Strategy | Result |
|---|---|
| Forecast better than the market | still behind it |
| Trade the last 30 seconds, as the settling average locks in | −5.4¢ per share, live |
| Market making, naive or chosen by the model | break-even at best |
| Buy favourites or long shots | loses in every price range |
| Buy near-certain outcomes at 99¢ | break-even, no volume |
| Buy YES and NO for under $1 | 0.03% of snapshots |
| Pair the 5-minute and 15-minute markets | riskless in 3.6% of windows, for about two seconds |
The one genuine inefficiency was structural. The last 5-minute window of each 15-minute window settles on the same Chainlink value against a different starting price, so their prices are logically tied. A 15-minute YES plus a 5-minute NO (or the mirror pair) always pays at least $1. The link held in all 698 windows we checked. In 3.6% of them the pair cost under $1 after fees, which is free money. The median opportunity lasted two seconds and offered about eight shares, and one second later only a quarter were still there. It's real, and it belongs to whoever is fastest.
The checklist we wish we'd started with
- Read the contract. What exactly pays out, and when?
- Grade against the payout, not a proxy. Use real outcomes wherever they exist.
- Check input resolution. Forward-filled or bucketed data next to fresh data is lookahead.
- Distrust threshold rules on noisy prices. They turn symmetric noise into one-sided bias.
- Cluster at the level of independent outcomes.
- Charge every cost, especially the ones introduced to stop what you're doing.
- Check row counts after joins.
- Delay your signals and watch how fast the edge dies.
- Simulate the fills you'd get, not the fills that happened.
- Check whether the data is already published. Polymarket's API reports Chainlink's starting and settlement values for every window. We spent weeks reconstructing what sat in a field we never read.
Every positive result in this project was eventually traced to one of these. The useful habit wasn't any single test. It was treating each new edge as a bug until a held-out sample, a delay test and a leave-one-day-out check all failed to kill it.