Skip to content

Observability

ccproxy includes a comprehensive observability system to provide insights into the proxy's performance, usage, and health. The system is built on a hybrid architecture that combines real-time Prometheus metrics, structured logging, and an optional DuckDB-based data store for historical analytics.

Features

  • Prometheus Metrics: Exposes a /metrics endpoint for real-time operational monitoring.
  • Access Logs: Captures detailed logs for every request, including token counts, costs, and timing.
  • Log Storage: Persists access logs to a local DuckDB database for historical querying and analysis.
  • Query API: Provides endpoints to query and analyze stored access logs.
  • Real-time Dashboard: A web-based dashboard to visualize metrics and logs in real-time.
  • Pushgateway Support: Can push metrics to a Prometheus Pushgateway for environments where scraping is not feasible.

Configuration

Observability features are configured under the observability section in your configuration file or through environment variables with the OBSERVABILITY__ prefix.

Setting Environment Variable Default Description
metrics_endpoint_enabled OBSERVABILITY__METRICS_ENDPOINT_ENABLED False Enable the /metrics endpoint for Prometheus scraping.
logs_endpoints_enabled OBSERVABILITY__LOGS_ENDPOINTS_ENABLED False Enable the /logs/* endpoints for querying and analytics.
dashboard_enabled OBSERVABILITY__DASHBOARD_ENABLED False Enable the /dashboard endpoint.
logs_collection_enabled OBSERVABILITY__LOGS_COLLECTION_ENABLED False Enable storing access logs to the backend.
log_storage_backend OBSERVABILITY__LOG_STORAGE_BACKEND duckdb The storage backend for logs (duckdb or none).
duckdb_path OBSERVABILITY__DUCKDB_PATH ~/.local/share/ccproxy/metrics.duckdb The path to the DuckDB database file.
pushgateway_url OBSERVABILITY__PUSHGATEWAY_URL None The URL for the Prometheus Pushgateway.

Enabling Features

To enable all observability features, you can set the following in your .env file:

OBSERVABILITY__METRICS_ENDPOINT_ENABLED=true
OBSERVABILITY__LOGS_ENDPOINTS_ENABLED=true
OBSERVABILITY__DASHBOARD_ENABLED=true
OBSERVABILITY__LOGS_COLLECTION_ENABLED=true

Prometheus Metrics

When enabled, the /metrics endpoint exposes a wide range of metrics in Prometheus format.

Key Metrics

  • ccproxy_requests_total: Total number of requests (labels: method, endpoint, model, status, service_type).
  • ccproxy_response_duration_seconds: Histogram of response times (labels: model, endpoint, service_type).
  • ccproxy_tokens_total: Total number of tokens processed (labels: type, model, service_type).
  • ccproxy_cost_usd_total: Total estimated cost in USD (labels: model, cost_type, service_type).
  • ccproxy_errors_total: Total number of errors (labels: error_type, endpoint, model, service_type).
  • ccproxy_active_requests: Gauge of currently active requests.

Access Logs & Storage

When logs_collection_enabled is true, the proxy captures detailed information for each request and stores it in a DuckDB database. This allows for historical analysis of usage patterns, costs, and performance.

Log Schema

The access_logs table stores the following columns:

  • request_id
  • timestamp
  • method, endpoint, path, query
  • client_ip, user_agent
  • service_type, model, streaming
  • status_code, duration_ms, duration_seconds
  • tokens_input, tokens_output, cache_read_tokens, cache_write_tokens
  • cost_usd, cost_sdk_usd

Logs API Endpoints

When logs_endpoints_enabled is true, the following endpoints become available under the /logs prefix:

  • GET /logs/status: Get the status of the observability system.
  • GET /logs/query: Query access logs with filters.
  • GET /logs/analytics: Get aggregated analytics from the logs.
  • GET /logs/stream: Stream logs in real-time via Server-Sent Events (SSE).
  • GET /logs/entries: Get raw log entries from the database.
  • POST /logs/reset: Clear all stored log data.

Dashboard

When dashboard_enabled is true, a real-time web dashboard is available at the /dashboard endpoint. The dashboard provides a live view of requests, token usage, costs, and errors.