Lorixen.
EN

Structured logging: JSON logs you can actually search

Short answer: Structured logging means writing each log entry as a set of named fields, usually one JSON object per line, instead of a free-text sentence. That lets you search and filter by customer, order, request or error type in seconds when something goes wrong. Agree a small set of standard fields, add a request ID to every entry, log to standard output, keep secrets and personal details out, and use log levels consistently.

Why plain-text logs let you down

Most apps start with logs like this:

Payment failed for order 4821 after 3 attempts

It reads fine for a person, but try answering practical questions with it. How many payments failed in the last hour? Were they all for the same card provider? Which customer did order 4821 belong to? You end up writing fragile search patterns, and the moment someone rewords the message, your searches quietly stop matching.

The structured version of the same event:

{"time":"2026-11-08T09:14:03Z","level":"error","event":"payment_failed","order_id":"4821","attempt":3,"provider":"card","request_id":"b7f2c1"}

Now every field can be filtered, counted and grouped by any log tool, without guessing at the wording.

Pick a small, consistent set of fields

The biggest win comes from agreeing field names once and sticking to them across every service. A practical core set:

Field What it holds
time When it happened, in UTC, in ISO 8601 format
level debug, info, warn or error
event A short, stable name, such as order_created
message Optional human-readable text
service Which app or component wrote it
request_id Ties all entries from one request together
user_id / account_id Internal IDs, not names or emails
error Error type and message, when there is one

The OpenTelemetry project's logs data model is a useful reference when choosing names. It defines a log record with fields such as a timestamp, severity text and number, a body, attributes and trace context (trace and span IDs). Even if you don't use OpenTelemetry yet, borrowing its shape makes a later move easier.

Keep event names stable. Treat them like an API: dashboards and alerts will depend on them.

Add a request ID everywhere

When one customer reports "checkout broke", you need every log line from that one request: the web server, the app, the payment call and the background job it triggered. Generate a request ID at the edge, or accept one from your load balancer, attach it to every log entry during that request and pass it on to other services in a header.

Most logging libraries support context or child loggers, so you set the ID once per request rather than adding it by hand to every line. If you later add tracing, put the trace ID in the logs too, so you can jump from a log line to the full trace.

Log to standard output and let the platform collect it

The Twelve-Factor App guidance on logs recommends that an app never concerns itself with routing or storing its log output. Instead, each process writes its event stream, unbuffered, to standard output. Your hosting platform, container runtime or log shipper then collects it and sends it to wherever you search logs.

This keeps the app simple and avoids log files filling the disk on a server nobody watches. In development, you can switch the output to a pretty, human-friendly format while keeping JSON in production.

Use levels the same way everywhere

Levels only help if everyone means the same thing by them:

  • error: something failed and someone may need to act, such as a payment provider being unreachable.
  • warn: something unexpected that the app handled, such as a retry that succeeded.
  • info: normal business events worth keeping, such as order created or user signed in.
  • debug: detail for diagnosing problems, normally off in production.

If every caught exception is logged as an error, real problems drown. If nothing is, alerts never fire.

Keep secrets and personal data out

Structured logs make it easy to log whole objects, which is exactly how passwords, tokens and card details end up in log storage. Protect against it:

  • Log IDs, not full records. Log customer_id, not the customer's name, email and address.
  • Add a redaction step in your logger for field names like password, token, authorization and card.
  • Never log request headers or bodies wholesale.
  • Set a retention period in your log tool, rather than keeping everything forever.

Treat logs as data that needs the same care as your database.

Make logs useful for alerts

Once events have stable names, alerting becomes straightforward. For example:

  • Alert when payment_failed events exceed a threshold in five minutes.
  • Alert on any error from the checkout service.
  • Build a daily count of signup_completed to spot a broken sign-up form.

These work because you're counting a field value, not hunting for a phrase.

A worked example

Picture a hypothetical Galway tour-booking app whose logs were sentences written by three different developers. A customer reported a double booking, and finding the relevant lines took most of an afternoon. The team switched to JSON logs with a shared field list, added a request ID through middleware and passed it to their booking worker. The next similar report took minutes: filter by request_id, read the sequence, and see that a retry had created the second booking.

FAQ

Do I need a paid log service?

Not necessarily. Many hosting platforms include basic log search. A dedicated tool helps when you have several services or need longer retention. Check current pricing and limits yourself.

Isn't JSON harder to read during development?

It can be. Most logging libraries offer a pretty-print mode for local use. Keep JSON in production, where machines read logs first.

Should I convert all old log lines at once?

No. Start with the paths that matter most, such as checkout, sign-in and background jobs, and convert the rest as you touch the code.

How long should we keep logs?

As long as you genuinely use them for debugging and security reviews. Set a deliberate retention period instead of keeping everything indefinitely.

If your app's logs are hard to search when something breaks, tell us how you log today and we'll suggest a field set.

Sources

WhatsAppBook a call