Metrics
Prometheus-compatible metrics are made available on the default port, on the /metrics endpoint.
Below is a list of the metrics that are exposed:
| Metric Name | Type |
| -------------------------------------------- | --------- |
| lorax_request_count | Counter |
| lorax_request_success | Counter |
| lorax_request_failure | Counter |
| lorax_request_duration | Histogram |
| lorax_request_queue_duration | Histogram |
| lorax_request_validation_duration | Histogram |
| lorax_request_inference_duration | Histogram |
| lorax_request_mean_time_per_token_duration | Histogram |
| lorax_request_generated_tokens | Histogram |
| lorax_request_input_length | Histogram |
For all histograms, there are metrics that are autogenerated which are the metric name + _sum and _count, which are the sum of all values for that histogram, and the count of all instances of that histogram respectively.