OpenAIUsageDebuggingPlugin

Logs OpenAI API usage metrics to a CSV file for debugging and analysis purposes.

Configuration example

{
  "$schema": "https://raw.githubusercontent.com/dotnet/dev-proxy/main/schemas/v2.0.0/rc.schema.json",
  "plugins": [
    {
      "name": "OpenAIUsageDebuggingPlugin",
      "enabled": true,
      "pluginPath": "~appFolder/plugins/DevProxy.Plugins.dll"
    }
  ]
}

Configuration properties

None

Command line options

None

Remarks

The OpenAIUsageDebuggingPlugin captures detailed usage metrics from OpenAI-compatible API requests and responses and writes them to a CSV file. This information is useful for debugging, tracking token consumption, monitoring rate limits, and analyzing API usage patterns over time.

Output file

The plugin creates a CSV file named devproxy_llmusage_<timestamp>.csv in the current directory when Dev Proxy starts. The timestamp format is yyyyMMddHHmmss.

CSV file structure

The CSV file contains the following columns:

Sample output

time,status,retry-after,policy,prompt tokens,completion tokens,cached tokens,total tokens,remaining tokens,remaining requests
2025-10-28T10:15:30.123Z,200,,,150,75,,225,9850,49
2025-10-28T10:15:35.456Z,200,,,200,100,50,300,9550,48
2025-10-28T10:15:40.789Z,429,60,rate-limit-policy-1,,,,,,0

Supported scenarios

The plugin logs metrics for:

• Successful requests (2xx status codes): Captures token usage metrics including prompt tokens, completion tokens, cached tokens, and remaining rate limits
• Error responses (4xx status codes): Captures rate limiting information including retry-after headers and policy IDs

Streaming responses

The plugin correctly handles streaming responses (using text/event-stream content type) by extracting the final chunk containing usage information.

Use cases

This plugin is useful for:

• Debugging token consumption: Understanding how many tokens your prompts and completions consume
• Monitoring rate limits: Tracking remaining tokens and requests to avoid hitting rate limits
• Cost analysis: Analyzing token usage patterns to estimate costs
• Performance optimization: Identifying requests with high token counts
• Prompt caching analysis: Tracking cached token usage to optimize prompt caching strategies

Comparison with OpenAITelemetryPlugin

While the OpenAITelemetryPlugin sends telemetry data to OpenTelemetry-compatible dashboards for real-time monitoring and visualization, the OpenAIUsageDebuggingPlugin focuses on creating detailed CSV logs for offline analysis and debugging. The two plugins complement each other:

• Use OpenAITelemetryPlugin for real-time monitoring, cost tracking, and integration with observability platforms
• Use OpenAIUsageDebuggingPlugin for detailed debugging, CSV-based analysis, and tracking rate limit information

You can enable both plugins simultaneously to get both real-time telemetry and detailed CSV logs.

Next step