compass.services.usage.UsageTracker#

class UsageTracker(label, response_parser)[source]#

Bases: UserDict

Rate or API usage tracker

Parameters:

  • label (str) – Top-level label to use when adding this usage information to another dictionary.

  • response_parser (callable()) – A callable that takes the current usage info (in dictionary format) and an LLm response as inputs, updates the usage dictionary with usage info based on the response, and returns the updated dictionary. See, for example, compass.services.openai.usage_from_response().

Methods

add_to(other)

Add the contents of this usage information to another dict

update_from_model([model, response, sub_label])

Update usage from a model response

Attributes

UNKNOWN_MODEL_LABEL

Label used in the usage dictionary for unknown models

totals

Aggregated usage totals across all sub-labels
UNKNOWN_MODEL_LABEL = 'unknown_model'#

Label used in the usage dictionary for unknown models

add_to(other)[source]#

Add the contents of this usage information to another dict

The contents of this dictionary are stored under the label key that this object was initialized with.

Parameters:

other (dict) – A dictionary to add the contents of this one to.

property totals#

Aggregated usage totals across all sub-labels

Type:

dict

update_from_model(model=None, response=None, sub_label='default')[source]#

Update usage from a model response

Parameters:

  • model (str, optional) – Name of model that usage is being recorded for. If None or empty string, the usage will be placed under the UsageTracker.UNKNOWN_MODEL_LABEL label.

  • response (object, optional) – Model call response, which either contains usage information or can be used to infer/compute usage. If None, no update is made. By default, None.

  • sub_label (str, optional) – Optional label to categorize usage under. This can be used to track usage related to certain categories. By default, "default".