History DB: Why I'm Letting Past Trades Rewrite My Strategy

The Day I Realized I Was Flying Blind

I'm sitting in front of my terminal watching the bot tail logs, and something occurs to me that I can't unsee. Every trade my bot submits is a vote on a hypothesis: "this route, at this size, at this tip, will land." The chain answers back in 400 milliseconds with a yes or a no. And then — for most of my history with this project — I throw the answer away.

That is, honestly, insane. It's like a NASCAR team running practice laps and not bothering to check the lap times. The whole point of practice is to see what worked and what didn't, then change something for the next run. Without that loop, every lap is the same lap, no matter how many you do.

So today I'm not writing more bot code. I'm thinking about a history DB — a place where every attempt, every swap I see on-chain, every tip I paid, every outcome, becomes structured evidence I can query later. Not for nostalgia. For evolution.

Why History Suddenly Matters

The context is shifting under everyone's feet. According to Dysnix's analysis of Solana MEV infrastructure, the network produced roughly $720 million in MEV revenue in 2025, and Dysnix argues something subtle but important about how that pie got divided: "The searchers who captured the majority of the $720 million MEV revenue were not running more sophisticated strategies than their competitors—in most cases, they were running better infrastructure." (Dysnix)

Read that twice. The winners didn't have secret math. They had faster pipes, tighter loops, and — I think this is the part most beginners miss — a way to learn from yesterday before tomorrow's block arrives.

Dysnix puts it bluntly elsewhere in the same piece: "strategy convergence leaves infrastructure as the primary differentiator." If everyone's reading the same DEX state, the edge isn't "I found a new arbitrage pattern." The edge is "I noticed last week that my bot loses on this token at this hour, and I fixed it before anyone else did." That's a history DB problem, not a strategy problem.

I keep coming back to a comparison from my own life. When I started cooking seriously, I bought a thermometer. Suddenly the steak wasn't "about medium rare" — it was 130°F or it wasn't. Once you can measure, you can iterate. Without a history DB, my bot is the cook eyeballing every steak. With one, I get the thermometer.

The Avalanche of Data Already Pouring Out of Solana

The scale of what's available is, frankly, ridiculous. Dysnix notes that Solana generates roughly 1 TB of new data per day at current activity (Dysnix). That number stopped me cold the first time I saw it. A terabyte. Per day. That's like a Walmart returns desk where every customer hands back a hard drive.

The MEV-relevant slice of that flow is its own beast. According to the Helius MEV report, in a single one-year window on Solana, researchers identified 90,445,905 successful arbitrage transactions, with an average profit of $1.58 per trade and a cumulative total of $142.8 million (Helius). The largest single arbitrage in that dataset captured $3.7 million in one transaction, per Helius.

Meanwhile, Sandwiched.me, which sits closer to a primary-data position because it parses the chain directly, reports analysis covering 8.5 billion trades and more than $1 trillion in DEX volume in their May 2025 state-of-MEV write-up, with sandwich-bot revenue in the $370–500 million range over a 16-month window (Sandwiched.me).

These aren't fun facts for the trivia round. They're the data I'm walking past every day. Each of those 90 million arbitrage transactions is a recipe — pool A, pool B, pool C, size, hop count, tip, outcome — written in public on the chain. If I'm building a bot and I'm not extracting the lessons embedded in that public record, I'm essentially leaving a library card on the table at the campus library and choosing to take the class without reading.

What Backtesting Actually Means

The word "backtesting" gets thrown around like "machine learning" — confidently and often inaccurately. Nadcab Labs gives a clean definition in their bot-backtesting primer: backtesting is the practice of "processing past market data through your trading logic sequentially" while the simulation "tracks hypothetical positions, calculates P&L per trade, and accumulates performance statistics across the test period" (Nadcab Labs).

In plain English: take the bot you wrote today and run it against last month's chain as if last month were happening live. Track what it would have done. Add up the wins and losses. Then change the bot and run it again.

The payoff is the loop, not any single result. Without history, I have to deploy a change to find out whether it works. Real SOL gets spent on the test. With history, I can ask a thousand "what if" questions before my next on-chain transaction.

Nadcab Labs lists historical-data requirements by strategy type. For high-frequency strategies — the bucket MEV bots fall into — they recommend a minimum of 3 months of order-book / L2 granularity, and 6–12 months for confident validation, with tick or 1-second snapshots (Nadcab Labs). That's not arbitrary. Three months of tick data is enough to span multiple market regimes. A bull weekend and a sleepy Tuesday and a flash-crash Wednesday all look different to a strategy, and you can't know which one your tuning was fit to unless your dataset includes all three.

The Number-of-Trades Trap

Nadcab Labs nails a principle I keep wanting to tattoo on the back of my hand: "A strategy showing 100% returns over 10 trades provides essentially no predictive value, while 20% returns over 500 trades offers meaningful evidence of edge" (Nadcab Labs).

This is the entire reason a history DB has to be big and structured, not a CSV in a downloads folder. If I deploy a new filter and it wins three out of three for a day, my brain wants to celebrate. But three is just three. The same coin lands heads three in a row often enough that no Vegas pit boss flinches. The only way I know if a change is real is if I can re-run it across hundreds or thousands of historical opportunities. That's what 100+ trades — Nadcab Labs cites that as the rough lower bound for statistical validity — is buying me.

This is a baseball lesson, really. A hitter who goes 3-for-4 in one game is just a hot afternoon. A hitter batting .300 over 500 plate appearances is a hitter. The whole point of having a season is to convert noise into signal. My history DB is the bot's season log.

The Categories of Pattern Hiding in Past Trades

If you stare at the public MEV data long enough, you start to see categories of insight that you can only extract from history.

Timing patterns. Helius's report covers the Vpe sandwich bot in detail — over a roughly month-long window they observed 1.55 million sandwich transactions with an 88.9% success rate and total profit of 65,880 SOL (about $13.43 million by their accounting) (Helius). The 11.1% that fail aren't random. There are hours, leaders, and conditions where they cluster. You can only find those clusters by sorting failures by time and asking what's correlated. That's a DB query, not a vibe.

Adversarial adaptation. Sandwiched.me's May 2025 analysis flags a striking shift: "wide sandwiches" increased roughly 30x in the wake of anti-sandwich mechanisms (Sandwiched.me). Translation: when defenders changed the rules, attackers changed the shape of the attack. If I'm not tracking which sandwich shapes are profitable this month versus last month, I'm trying to play this season's game with last season's playbook.

Event signatures. The same Sandwiched.me piece highlights spike events — WIF launch on January 10, 2024; the Pengu release on December 17, 2024; the Trump/Melania token launches on January 18–20, 2025 — as moments where the data goes vertical. These aren't anomalies to discard. They're rehearsal footage for the next launch. If my bot had a known behavior during the WIF launch window, I can replay that window against my current code and ask: would today's bot have done better, worse, or the same?

Tip calibration. Dysnix's piece characterizes typical MEV bot behavior as committing "50–60% of expected profit in validator tips" (Dysnix). That percentage is a current snapshot. Whether it's right for me depends on which bundles I'm trying to win, against which competitors, on which leaders. I can't tune that without a log of every tip I've ever paid and what came back.

Pool and pair profitability. Eighty-eight point seven percent of the Helius-tracked arbitrage profit was denominated in SOL (Helius). That's a pattern. It means the SOL-denominated routes have been carrying the ecosystem. Is that still true on a Tuesday afternoon in June? I won't know unless I'm watching it.

What Storage Choice Buys You

This is the part where engineers want to argue about databases. I'll keep it short and reference one practitioner story that helped me sort the noise.

The TigerData team published a candid retrospective on a crypto trading bot they power with TimescaleDB. The numbers are instructive: their pipeline ingests roughly 3.6 million records per day from about 182 tradeable markets (TigerData). They also walked through what they tried and rejected: "every query that I tried to run was painful, due to their own query language" was their experience with one option; another "lacked necessary functionality" (TigerData).

The abstract lesson, more than any specific product, is this: you need a database where the query language doesn't fight you when you ask time-series questions. "Show me every swap on pool X in the 30 seconds before each successful sandwich last week" should be one query, not a thirty-line script. TigerData's case study describes response times in milliseconds for that flavor of question — volatility windows, lateral joins, rolling aggregates — and that responsiveness is what turns a history DB from a museum into a research lab.

A second piece of their playbook generalizes well: a two-tier storage approach where raw market data flows in continuously and pre-computed aggregates (for example, 1-minute, 5-minute, and 1-hour buckets) are maintained for the questions you'll ask over and over (TigerData). I don't need to recompute "profit per route per hour" from scratch every time I want it. That's like rebuilding the IKEA bookshelf every time you want to grab a book.

I'll stop short of describing my own schema or which specific tables I'm using. The principle is the database should make my common questions cheap, and I should design for the questions I expect to ask weekly.

The Simulation Question

If a history DB is the library, a simulator is the laboratory. They're different organs, but they need each other.

Solid Quant's writeup on building an MEV simulation engine makes a point I think is the most important methodological observation in the space: their simulator runs on "data that have already been confirmed on the blockchain. Though state transitions have occurred and finalized on the blockchain, there may still be discrepancies among various DEX protocols" (Solid Quant).

What this means in practice: the safest backtest replays a confirmed historical state and asks, "if my bot had been there at block N, what would have happened?" The DEX math is whatever the DEX math actually was at that block. You're not modeling reality; you're rerunning reality with your bot inserted as a hypothetical observer.

This flips the conventional wisdom about simulators. You don't need to reimplement every DEX's curve in your simulator — you need access to the DEX contracts as they existed at the historical block and a way to call them. Solid Quant frames it well: "you won't have to understand those formulas. All you need is a bit of browsing through, and figuring out what functions DEXs use to simulate their swaps."

For a Solana history DB to support that style of simulation, what it needs to store isn't just "the trades that happened." It needs the state references to reconstruct what the pools looked like at each tick — which accounts were touched, which discriminators were hot, which pools had which reserves. I'll spare the details of how I think about laying this out; the principle is that the DB needs to keep enough state pointers that a later simulation can ask the chain (or an archive) for the rest.

Scale: What "Big Enough to Matter" Looks Like

It's tempting to think of a history DB as a side project. The numbers say otherwise.

One February 2026 Medium piece from Shailesh Nair argues that with a Rust-based REVM simulator, an attacker — or, the same architecture, a backtester — can hit 2.2 million simulations per hour at around $0.0009 per simulated transaction on commodity AWS instances (Shailesh Nair, Medium). That number is EVM-specific, but the order-of-magnitude lesson translates. If your DB can serve queries at that rate, you can run millions of "what-if" simulations per day. If it can't, your strategy iteration is bottlenecked on disk, not on ideas.

The same piece sketches the latency tiers that compute layers add up to: CPU implementations in Rust at around 850 nanoseconds tick-to-trade, FPGAs about 19x faster, ASICs about 70x (Shailesh Nair, Medium). I don't have an ASIC budget. I do have a software stack. So for me, the throughput multiplier I'm chasing isn't hardware — it's the database query that lets one CPU evaluate ten ideas in the time it used to evaluate one.

Walk-Forward and the Overfitting Trap

The single most dangerous moment in any backtesting workflow is the one right after you find a "great" result. Your eyes light up; you've discovered a parameter combo that turned a flat strategy into a money printer in your test set. You almost certainly haven't.

Nadcab Labs describes walk-forward analysis as the "gold standard" for fighting this trap (Nadcab Labs). The structure is simple: split your history into an in-sample window where you're allowed to tune parameters, and an out-of-sample window where you're not. Tune on the first, test on the second, then slide both windows forward and repeat. The strategy that works only in the optimization window is fitted to noise; the one that survives the slide is showing actual edge.

Nadcab Labs also lists overfitting red flags that read like a roast of every overconfident backtester: "perfect equity curves with no drawdowns," strategies that "only work on specific date ranges," five-plus tunable knobs, results that swing wildly with tiny parameter tweaks. If your test result looks too clean, it's not. That's the rule.

The practical implication for the history DB: store enough history to allow real splits. If I have only two weeks of data, my walk-forward windows are useless — I can't both fit and validate meaningfully. The minimums in Nadcab Labs's table — three months for tick-level work and six to twelve months ideally — exist because the windowing math demands it.

The Validation Pipeline I'm Building Toward

Nadcab Labs sketches a four-stage validation pipeline that maps cleanly onto how I want to use my history DB:

1. Initial Backtest with realistic execution assumptions — fees, slippage, latency baked in. They suggest defaults like 0.1–0.5% in fees per trade, 0.05–0.2% slippage, and 50–500ms of execution latency for general crypto contexts (Nadcab Labs). For Solana MEV, the latency numbers I care about are tighter than that, given Dysnix's 400 ms block time figure, but the principle of "don't backtest as if execution is free" holds universally.
2. Walk-Forward Optimization to confirm the result wasn't a fit to a specific window.
3. Paper Trading — running the strategy against live data without real money, to check whether what backtested actually behaves the same when ticks arrive in real time.
4. Small Live Deployment — minimal capital, scale up only on evidence.

Notice that the history DB is the prerequisite for steps 1 and 2, and the comparison anchor for steps 3 and 4. Without it, you can't even start. With it, you have a four-step ladder for moving from a hunch to a deployed change.

Nadcab Labs's benchmarks for whether a strategy is worth taking further are concrete: Profit Factor above 1.5, Maximum Drawdown below 25%, Sharpe Ratio above 1.0, and at least 100 trades for statistical validity (Nadcab Labs). I'm not treating these as gospel for an MEV-specific bot, because traditional finance metrics don't translate one-to-one. But the spirit — measure on numbers, not feelings — is exactly right.

What This Changes About How I Build

Until now my development loop has been: notice a problem, write a change, deploy, watch the logs, hope I'm right. That's how I've been treating bot work. After a few weeks of thinking about this, I realize that loop is the bottleneck.

The loop I want instead is: notice a problem, write a change, query the history DB to see how the change would have behaved across the last several weeks of real opportunities, compare against the current bot's behavior on those same opportunities, decide whether the change is worth deploying based on the evidence, and only then push to production.

That extra step in the middle isn't slower. It's faster, because most of my deployed changes turn out to have been bad ideas in disguise, and catching them at query time costs zero SOL. The slowdown is during the loop; the speedup is across the project.

There's also a less obvious benefit. A history DB is a calibration tool for my own intuitions. Every time my gut says "this change will help," and the data says it wouldn't have, I learn something about my gut. After a few hundred of those, my gut gets better. That's the meta-iteration nobody talks about — the bot evolves, and the operator evolves with it.

Implications for How I'll Run the Project

If strategy convergence is the long-run pressure on this market — and Dysnix's framing certainly suggests it is — then the operators who survive will be the ones with the tightest learn-from-history loops. Not the ones with the most clever algorithm in a single moment; the ones who can recognize when a change of regime has happened and turn the rudder before anyone else.

That's what a history DB really is. It's not a logging system. It's an institutional memory for a one-person trading firm. It's the difference between running the same lap forever and running practice with a stopwatch.

I don't know yet whether I'll get this right. The schema decisions are easy to get wrong, the storage costs are real, and the temptation to fit my backtests to a story I want to tell is going to be constant. What I do know is that without this layer, I'm gambling. With it, I'm at least keeping score. That's the bar I want to clear before I push my next set of changes.

Key Takeaways

• Solana's MEV economy is large enough to make history mandatory. With around $720 million in MEV revenue in 2025 per Dysnix and over 90 million arbitrage transactions catalogued by Helius in a year, there is more public learning material on this chain than any single bot operator could read in a lifetime — and ignoring it is a choice.
• Statistical validity comes from trade count, not return percentage. Nadcab Labs's framing — that 20% over 500 trades beats 100% over 10 — is the antidote to the "my bot's on a hot streak" delusion.
• Walk-forward is the only defense against overfitting. Tuning on in-sample data and validating on out-of-sample data, then sliding the window, is how you tell signal from noise. It requires enough history to support real splits.
• Storage choice is a query-language choice. A database that makes your common time-series questions cheap turns the history DB from a museum into a research lab; one that fights you on syntax turns the project into a chore.
• The history DB is also a calibration tool for the operator. Every disagreement between your gut and the data is a free lesson about your own model of the market — and that meta-loop is what evolves the strategy over time.