Logging Is the Lifeline When Your Trading Bot Breaks

The Moment the Market Walks Away

The bot ran for hours and a route that should have printed money came back as a failed transaction. There's no replay button. The leader slot is gone, the pool state has moved on, and whatever the runtime saw at that exact tick — the priced-out path, the simulated profit, the bundle response — is gone unless I wrote it down. The bot doesn't remember. The chain doesn't remember the parts that happened off-chain. Only the logs remember. So this article is about getting serious about logging, the unglamorous infrastructure that decides whether tomorrow's bug fix is a thirty-minute job or a three-day archaeology dig.

This matters more for a Solana arbitrage bot than for almost any other kind of software I've written. A web app that throws an exception can be reproduced — hit the endpoint again, watch it fail again, attach a debugger. A trading bot operates against a market that has already moved. The error you're chasing happened in a state of the world that no longer exists. If the logs don't capture that state in enough detail, the bug is unobservable. That's not "hard to debug." That's actually impossible to debug, which is a different category of pain.

Why print() Stopped Being Enough Around Day Three

For the first few days of the project, I did what every developer does. I sprinkled print() statements wherever I wanted to see something. It works fine for a script you run for two minutes and then close. It does not work for a process that runs for hours, talks to multiple RPC endpoints, listens to a streaming feed, builds candidate routes, simulates them, submits them, and produces hundreds of lines of output per minute.

The problems show up fast. There's no severity — every line looks equally important even when ninety percent of them are boring heartbeats. There's no timestamp unless I add one manually, and inevitably I forget on the most important line. There's no way to filter, no way to silence the noisy modules without commenting code, no way to redirect to a file without shell tricks that break when the process daemonizes. And the moment something interesting happens at 2:14 a.m. while I'm asleep, the scrollback buffer eats it.

The Python standard library's logging module solves all of this, and it has been the recommended approach since long before I started. Per the official Python documentation, the library exposes five canonical levels with numeric thresholds — DEBUG (10), INFO (20), WARNING (30), ERROR (40), CRITICAL (50) — and the default level is WARNING, meaning anything below is silently dropped unless you configure otherwise. That last detail is the one that bites new users: you call logger.info("started") and nothing appears, because you never set the level. The fix is one line, but the principle behind it is the whole point of using a logging library — the call site declares the severity, and the configuration decides what to show.

Levels Are a Contract, Not a Vibe

The biggest mistake I see (and made myself) is treating log levels as a feeling. "This seems important, so it's INFO. This seems really important, so it's ERROR." That's not a contract. That's a vibe, and vibes don't survive contact with a 500-megabyte log file.

The industry has converged on a fairly tight definition for each level, and once you internalize it, the noise problem largely solves itself. Pulling together guidance from the official Python documentation, Real Python's logging guide, and standard observability practice:

• DEBUG is for the developer at the keyboard. Internal state, intermediate calculations, parameters at function entry. Disabled in production by default. If you turn it on in production, you turn it off as soon as the diagnosis is done.
• INFO is for high-level events that matter to operating the system. Bot started. New leader slot detected. Route candidate above threshold found. Bundle submitted. Not every iteration of the inner loop — that way lies a gigabyte an hour.
• WARNING is for something unexpected that the system handled. A retry succeeded after one failure. An RPC node returned stale data and we failed over. A bundle was dropped but the bot is still running.
• ERROR is for a failed operation that needs human attention. The transaction reverted on-chain with an unexpected error code. The wallet's lamport balance dipped below the configured floor. A submit returned an unrecoverable status.
• CRITICAL is reserved for things that mean the bot cannot safely continue. The signing key file is unreadable. The configured RPC list is exhausted. State that affects funds is corrupt.

Some frameworks document a six-level hierarchy that adds TRACE below DEBUG and FATAL above CRITICAL, but the same principle holds: more verbose at the bottom, more urgent at the top, and your production default lives near INFO. There's a useful piece of advice that floats around the structured-logging literature: only log an ERROR if the system is misbehaving or losing data. If the bot tried a route, didn't find profit above the threshold, and moved on, that is not an error. That is the bot doing its job. Calling it an error is the operational equivalent of crying wolf, and after the third false alarm, nobody's reading the alerts.

The payoff from being disciplined here is that the level itself becomes a filter. "Show me every ERROR from the last twelve hours" is a one-line query that produces a triage queue. "Show me everything" is the same thing as "show me nothing," because no human is reading a million lines.

Structured Logging, or Why Your Logs Should Look Like JSON

The second shift, after levels, was moving from plain-text log lines to structured logs. The idea is simple and now widely accepted: instead of writing

2026-05-17 10:32:11 [ERROR] Failed to submit bundle: timeout after 800ms on endpoint X

you write

{
  "timestamp": "2026-05-17T10:32:11Z",
  "level": "ERROR",
  "event": "bundle_submit_failed",
  "reason": "timeout",
  "timeout_ms": 800,
  "endpoint": "primary",
  "slot": 285441923,
  "route_id": "r-abc123"
}

Some tools define this cleanly: "Structured logging is the practice of writing application logs in a consistent, machine-readable format instead of plain text." Same information, but every field is queryable. I can ask "how many bundle submits timed out in the last hour, grouped by endpoint" without writing a regex. I can correlate a route_id across the planning, simulation, and submission stages. I can compute the median timeout duration. None of that is possible against free-text logs without writing a parser that breaks every time someone changes a message.

There are two libraries in Python that come up over and over for this — one that binds context to a logger so common fields (session ID, run ID, slot) appear automatically on every line, and one that simply formats standard logging output as JSON. Either is fine. The point is the format, not the brand. As a leading logging guide puts it: in production, you want raw JSON because your aggregation pipeline can parse it without brittle pattern matching, and pretty-printing in production just adds overhead and makes parsing harder. Save the pretty colors for the dev console.

The OpenTelemetry log data model, which is becoming the de facto industry standard, formalizes this with a small set of fields — severityNumber, body, attributes, traceId, spanId, and timestamps for both when the event occurred and when it was observed. You don't have to adopt OTel to benefit from the schema discipline. Just pick a shape and stick to it.

The Fields That Actually Earn Their Keep

What goes in the structured record? Everyone publishing on this topic has a slightly different list, but the overlap is large. After combining the guidance, the fields that consistently pay for themselves on every log line are:

• timestamp in ISO 8601, with a UTC offset and milliseconds. "It happened around 10" is not a timestamp. "2026-05-17T10:32:11.482Z" is.
• level, both as a string and as the numeric severity if you're using OTel-style schemas.
• service or component name. Once the bot has more than one process, you need to know which one wrote the line.
• environment — dev, staging, prod. Cheap to add, expensive to miss when you've accidentally pointed a dev process at prod state.
• version — a commit hash or release tag. "Worked yesterday" is a hypothesis, not a fact, and the version field is how you check it.
• correlation ID — a unique ID that ties together every log line emitted while handling one logical unit of work. For a bot, the obvious unit is a single arbitrage attempt, from candidate detection through submission outcome.
• domain identifiers — slot number, route ID, transaction signature, wallet, mint addresses. These are the keys you'll actually query on later.
• a stable event name for machine-parseable filtering — bundle_submit_failed beats a free-text description because the latter drifts every time someone improves the wording.

One field deserves its own paragraph: the correlation ID. In a synchronous web app it usually shows up as a request ID propagated via an HTTP header. In an async or event-driven system — which a Solana bot is — it has to be threaded through manually. Every time a piece of work spawns child work, the child carries the parent's ID, or the relationship is lost. The discipline is a pain to set up. It pays for itself the first time you trace a single failed trade across detection, planning, simulation, and submission and see the entire timeline in five filtered log lines.

What to Log for a Solana Trading Bot, Specifically

The generic structured-logging guides cover the principles. The trading-bot literature gets more concrete, and a trading bot debugging guide is one of the few that addresses the domain directly. What earns a log line for this kind of bot:

For every strategy decision — the inputs (current pool reserves, observed price, configured thresholds), the output (a candidate route or a skip), and the reason the decision came out the way it did. Without the why, you can't tell later whether the bot made the right call or got lucky.

For every order or bundle attempt — the route, the expected profit, the simulated outcome, the priority fee or tip strategy, the slot it was aimed at, the response from the submission endpoint, and the eventual on-chain status. "Submitted at 10:32" is barely useful. "Submitted for slot N at tip ratio X, expected profit Y SOL, simulated success, returned status Z, included on-chain in slot N+1 with actual delta D" is a complete record you can replay against forever.

For every external interaction — RPC endpoint, latency, error code if any, retry count. The infrastructure layer is where a non-trivial share of weird bugs hide: a node that silently returns stale account data is invisible from inside the bot's business logic but obvious in the latency-and-error log.

For system events — startup with a summary of the loaded config (with secrets redacted, of course), shutdown with a reason, state transitions like "paused due to circuit breaker," balance changes.

The one piece of guidance I picked up the hard way: do not log every successful no-op. The bot evaluates thousands of candidate paths per minute, almost all of which fail to clear the threshold. Logging each one at INFO is a way to bury the few that matter. The classic structured-logging guidance is right here too: log meaningful events — failed transactions, permission changes, deployment events, request failures, external service issues, state transitions. Don't log every routine success or every health-check ping.

The Performance Gotcha Nobody Warns You About

There is one performance trap in Python's logging module that I walked straight into. It looks innocent, and the first time someone shows it to you it feels like a nitpick. It is not a nitpick.

# Looks fine. Is not fine.
logger.debug(f"evaluating route with {len(candidates)} candidates and reserves {reserves}")

The f-string is evaluated eagerly, the moment Python hits that line, regardless of whether DEBUG is enabled. If reserves is a large data structure with a custom __repr__ that walks a graph, you just paid the cost of that walk to produce a string that gets thrown away because the level is set to INFO in production. Do this in the inner loop and your bot is suddenly twice as slow for no observable reason.

The correct pattern uses the logger's own lazy interpolation:

logger.debug("evaluating route with %d candidates and reserves %s", len(candidates), reserves)

Now the formatting only happens if a handler is actually going to process the record. The same idea applies to anything expensive on the call site:

if logger.isEnabledFor(logging.DEBUG):
    logger.debug("state dump: %s", expensive_serialization())

The Python docs are explicit about this. So is the practical advice in most logging guides. It's still the single most common performance regression I've introduced in my own code. I no longer trust myself to remember; I rely on a lint rule.

Things You Absolutely Never Put in a Log Line

For a crypto bot the security list is shorter than the general industry one but more consequential. Never, under any circumstance, log:

• A private key, in any form. Not the bytes, not a base58 string, not a path that points to it with the contents inlined "just for debugging."
• A seed phrase or mnemonic.
• An API token for an external service.
• The full contents of an environment file or secrets manager response.

The industry guidance unanimously agrees on this — every responsible logging guide says it, and the reason it gets repeated is that smart people violate it under deadline pressure all the time. The mitigation pattern is also standard: emit opaque identifiers (a wallet's public key, a key ID, a token's last four characters) instead of the secrets themselves. If you find yourself thinking "I need to log the private key to debug a signing issue," stop. The thing to log is the public key, the message hash, and the signature. Those are sufficient to diagnose every signing bug I have ever seen.

Rotation, Retention, and the Disk That Eats Itself

A bot logging at INFO and emitting structured records will produce a non-trivial amount of data over a week — easily hundreds of megabytes for a busy strategy, gigabytes if DEBUG is left on. Python's standard library includes two handlers — RotatingFileHandler rotates by file size, TimedRotatingFileHandler rotates by time interval. Pick one. Configure it before the disk fills, not after.

For retention, some sources suggest a tiered model: hot tier 7–30 days for active troubleshooting, warm tier 30–90 days for post-incident review, cold/archive 1–7 years for compliance and audit trails. A solo developer doesn't need a multi-tier setup, but the principle is sound. Recent logs live where queries are fast. Older logs live where storage is cheap. The very old logs either live in cold storage or get deleted on a schedule, because nobody is querying a log from two years ago and that disk is needed elsewhere.

Logs as Messages to Your Future Self

The mental model that finally made logging click for me was this: the log line is a message to the version of me who has to debug this at 3 a.m. six weeks from now, with no memory of what I was thinking when I wrote it. That person doesn't have the context that was in my head. They will not remember which module emitted which message, which event names map to which code paths, or what the difference is between route_evaluated and route_considered.

So the log line has to be self-contained. The classic guidance — context must not depend on previous messages appearing first — is the right principle. The future debugger may be looking at a filtered slice. They may be looking at a single line, ripped from its context by a query. If that one line doesn't say what happened, where, when, who, and why, it has failed at its job. Logs should answer the W questions: what, where, when, who, and why.

The related discipline is to log decisions, not just outcomes. "Skipped route X" is half a log line. "Skipped route X because expected profit Y was below threshold Z" is the whole one. The first version tells you what the bot did. The second tells you why, which is what you actually need to know when the bug is "the bot is skipping routes it shouldn't be skipping."

What This Looks Like in Practice, Now

Where this leaves the project, today: every meaningful event in the bot emits a structured JSON record at an appropriate level, with timestamps in UTC, a correlation ID that follows each arbitrage attempt end-to-end, a stable event name, the domain identifiers that matter for that event, and the rationale where a decision was made. Production runs at INFO with WARNING and above triggering desktop notifications. DEBUG is available behind a runtime flag for when something genuinely needs deep inspection. Files rotate by size with a fixed retention window. Secrets are categorically excluded; the linter complains if certain field names appear in a log call.

It is not glamorous. It is not the part of the project that demonstrates technical sophistication. It is the part that determines whether, the next time the market walks away from a failed trade at an inconvenient hour, I have a fighting chance of figuring out what happened — or whether I'm staring at half-formed print() lines and guessing.

What This Means Going Forward

On-chain trading has a property that makes logging less optional than in most software: the operating environment cannot be re-created. Web apps get fresh test environments, batch jobs get reruns, even production incidents in a typical SaaS can often be reproduced from a captured request. Bots that compete on milliseconds against a market that is moving forward whether you're ready or not don't get any of that. The state of the chain at the moment your bot made its decision is gone the next slot. Whatever your process captured in that moment is the only evidence that will ever exist.

That reframes the cost-benefit calculation around logging entirely. Every minute spent up front making logs structured, queryable, leveled, and complete pays back the first time something weird happens at an inconvenient hour. Skipping that minute pays a much larger debt at exactly the wrong time.

Key Takeaways

• Logs are the only artifact that survives a missed trade, because the on-chain state at the moment of the bot's decision can't be reproduced. Treat the logging layer as part of the trading system, not as developer scaffolding.
• Log levels are a contract: DEBUG for developers, INFO for operations, WARNING for handled anomalies, ERROR for human-attention failures, CRITICAL for cannot-continue states. Pick the level by definition, not by feeling.
• Structured logging (JSON) is the default, not the exotic option. Every line should have a timestamp, level, service, environment, version, correlation ID, stable event name, and the domain identifiers (slot, route ID, signature) that you'll actually query on later.
• F-strings inside log calls are a performance trap in Python. Use the logger's lazy %s-style interpolation so disabled levels cost nothing.
• Never log secrets, ever — private keys, seed phrases, API tokens. Log opaque identifiers and signatures instead. Set up rotation and a retention policy before the disk decides for you.