Technical notes · Observability
Market Risk Platform: observability
The interesting part of my Market Risk Platform project is the work around the calculation: knowing whether the service can do useful work, and being able to tell when it stops.
I built it as an open-source reference implementation. A shared Python risk library feeds a FastAPI service and a CLI. Prometheus metrics and a provisioned Grafana dashboard make the behavior visible. It uses a bundled sample book; this is not a client trading system or evidence of a production rollout.
Three endpoints, three questions
/health answers whether the process is serving requests. /ready runs the sample portfolio and FX option workloads, checks the results and returns 503 if those checks fail. /metrics exposes request telemetry and sample calculation results.
That separation matters. A running web process can still be unable to produce a usable result. The readiness check gives a deployment gate something more useful than an open port. Its scope is deliberately small: sample workload readiness does not prove that every possible input or external dependency is healthy.
Keep labels bounded
The request middleware records method, status and route template. Unknown URLs share an unmatched label. I do not put the raw request URL into the metric labels.
That keeps arbitrary paths from creating a new time series for every request. The latency histogram also uses route templates, so a dashboard can compare request rates, errors and latency without grouping a growing list of unrelated URLs.
Give freshness a precise meaning
The metrics refresh recomputes the bundled sample risk numbers. It updates a last-success timestamp only after the calculations succeed. If a calculation raises an error, a failure counter increments and that timestamp stays unchanged.
There is a limit here worth being explicit about: this timestamp measures successful sample calculation, not the age of upstream market data. A real ingestion pipeline would need a separate observation timestamp. Using one number for both would make the dashboard reassuring for the wrong reason.
Make the dashboard answer an operating question
The Grafana dashboard brings request rate, p95 latency, error ratio and calculation freshness together with the risk outputs. The aim is to connect service behavior to the work the service exists to do.
This project demonstrates how I approach observability: define what each signal means, keep the instrumentation bounded, and make the failure path visible. The next step for a real deployment would be to choose service objectives and alert thresholds against its actual workload.