compass.services.usage.LLMRateTracker#

class LLMRateTracker(label='_llm_usage_rates')[source]#

Bases: UserDict

Track run-wide and per-model LLM usage rates on calls

Parameters:: label (str, optional) – Top-level label to use when persisting rate statistics. By default, "_llm_usage_rates".

Methods

`add_to`(other)	Add the current rate statistics to another dictionary
`end_request_attempt`(model)	Record the end of an LLM request attempt
`record_request`(model, timestamp)	Record a submitted LLM request
`record_tokens`(model, tokens, timestamp)	Record actual tokens from a completed LLM request
`snapshot`()	dict: Run-wide and per-model rate summaries
`start_request_attempt`(model)	Record the start of an LLM request attempt

record_request(model, timestamp)[source]#: Record a submitted LLM request

record_tokens(model, tokens, timestamp)[source]#: Record actual tokens from a completed LLM request

start_request_attempt(model)[source]#: Record the start of an LLM request attempt

end_request_attempt(model)[source]#: Record the end of an LLM request attempt

snapshot()[source]#: dict: Run-wide and per-model rate summaries

add_to(other)[source]#: Add the current rate statistics to another dictionary