compass.llm.calling.SchemaOutputLLMCaller#

class SchemaOutputLLMCaller(llm_service, usage_tracker=None, **kwargs)[source]#

Bases: BaseLLMCaller

Class to support structured (JSON) LLM calling functionality

This class differs from JSONFromTextLLMCaller in that it is designed to work with LLM services that allow you to specify the expected output schema as part of the API call (e.g. OpenAI function calling). This allows for more direct retrieval of structured data from the LLM, without needing to parse JSON from text-based responses. The expected response format should be provided as a parameter to the call method, and should be formatted according to the specifications of the underlying LLM service for structured output.

See also

LLMCaller

Simple LLM caller, with no memory and no parsing utilities.

ChatLLMCaller

Chat-like LLM calling functionality.

JSONFromTextLLMCaller

LLM calling functionality that extracts structured data (JSON) from the text-based response.

Parameters:

  • llm_service (Service) – LLM service used for queries.

  • usage_tracker (UsageTracker, optional) – Optional tracker instance to monitor token usage during LLM calls. By default, None.

  • **kwargs –

    Keyword arguments to be passed to the underlying service processing function (i.e. llm_service.call(**kwargs)). Should not contain the following keys:

    • usage_sub_label

    • messages

    These arguments are provided by this caller object.

Methods

call(sys_msg, content, response_format[, ...])

Call LLM for structured data retrieval
async call(sys_msg, content, response_format, usage_sub_label=LLMUsageCategory.DEFAULT)[source]#

Call LLM for structured data retrieval

Parameters:

  • sys_msg (str) – The LLM system message. If this text does not contain the instruction text “Return your answer as a dictionary in JSON format”, it will be added.

  • content (str) – LLM call content (typically some text to extract info from).

  • usage_sub_label (str, optional) – Label to store token usage under. By default, "default".

  • response_format (dict) –

    Dictionary specifying the expected response format. This will be passed to the underlying LLM service (e.g. OpenAI) and should be formatted according to that service’s specifications for structured output. For example, for OpenAI GPT models, this should be a dictionary with the following keys:

    • type: Should be set to “json_schema” to indicate that the expected output is structured JSON.

    • json_schema: A dictionary specifying the expected JSON schema of the output. This should include the following keys:

      • name: A string name for this response format (e.g. “extracted_features”).

      • strict: A boolean indicating whether the LLM should strictly adhere to the provided schema (i.e. not include any keys not specified in the schema). If True, the LLM will be instructed to only include keys specified in the schema field. If False, the LLM may include additional keys not specified in the schema field.

      • schema: A dictionary specifying the expected JSON schema of the output. This should be formatted according to JSON Schema specifications, and should define the expected structure of the output JSON object. For example, it may specify that the output should be an object with certain required properties, and the expected data types of those properties.

Returns:

dict – Dictionary containing the LLM-extracted features. Dictionary may be empty if there was an error during the LLM call.