LLM

(Fn [(Ref String a)] ProviderConfig)

                        (anthropic api-key)
                    

creates a provider config for the Anthropic API.

(Fn [(Ref ProviderConfig a), (Ref LLMRequest b)] (Result LLMResponse LLMError))

                        (chat config req)
                    

sends a chat request to the configured provider. Returns (Result LLMResponse LLMError). On HTTP errors (4xx/5xx), returns a structured LLMError.Api with status code, error type, and message parsed from the provider's error response. On transport failures, returns LLMError.Transport.

(Fn [(Ref ProviderConfig a), (Ref String b), (Array Message), Int, Double, (Ref (Array ToolDef) c), (Fn [(Ref ToolCall d)] String e), Int] (Result LLMResponse LLMError))

                        (chat-loop config model messages max-tokens temperature tools handler max-iters)
                    

runs an agentic tool-use loop. Calls chat with the given tools, invokes handler for each tool call in the response, appends tool results to the conversation, and repeats until the model stops calling tools or max-iters iterations are reached.

The handler receives a (Ref ToolCall) and returns the tool result as a String.

Returns the final (Result LLMResponse LLMError). If the iteration limit is reached, the last response is returned (it may still contain tool calls).

(let [config (LLM.ollama "http://localhost:11434")
      tools [(ToolDef.init @"get_weather" @"Get weather" schema)]
      msgs [(Message.user "Weather in Paris?")]
      handler (fn [tc]
                (if (= (ToolCall.name tc) "get_weather")
                  @"22C, sunny"
                  @"unknown tool"))]
  (match (LLM.chat-loop &config "llama3" msgs 256 0.7 &tools handler 10)
    (Result.Success r) (println* (LLMResponse.content &r))
    (Result.Error e) (IO.errorln &(LLMError.str &e))))

(Fn [(Ref String a), (Array Message), Int, Double] LLMRequest)

                        (chat-request model messages max-tokens temperature)
                    

creates an LLM request without tools.

(Fn [(Ref String a), (Array Message), Int, Double] LLMRequest)

                        (chat-request-json model messages max-tokens temperature)
                    

creates an LLM request that asks for a JSON response.

Note: Anthropic has no native JSON mode, so this falls back to a system prompt instruction (best-effort, not guaranteed). All other providers use their native JSON mode.

(Fn [(Ref String a), (Array Message), Int, Double, JSON] LLMRequest)

                        (chat-request-with-schema model messages max-tokens temperature schema)
                    

creates an LLM request constrained to a JSON schema. The schema is a JSON value (use the JSON constructors).

(Fn [(Ref String a), (Array Message), Int, Double, (Array ToolDef)] LLMRequest)

                        (chat-request-with-tools model messages max-tokens temperature tools)
                    

creates an LLM request with tool definitions.

(Fn [(Ref ProviderConfig a), (Ref LLMRequest b)] (Result LlmStream LLMError))

                        (chat-stream config req)
                    

sends a streaming chat request and returns an LlmStream. Poll the stream for tokens. Returns (Result LlmStream LLMError). Checks the HTTP status code before returning the stream.

(Fn [(Ref ProviderConfig a), (Ref EmbeddingRequest b)] (Result EmbeddingResponse LLMError))

                        (embed config req)
                    

sends an embedding request to the configured provider. Returns (Result EmbeddingResponse LLMError).

Anthropic does not offer an embeddings API; calling this with an Anthropic config returns a Transport error.

(let [config (LLM.openai "sk-...")
      req (LLM.embedding-request "text-embedding-3-small"
            [@"hello" @"world"])]
  (match (LLM.embed &config &req)
    (Result.Success r)
      (println* (Array.length (EmbeddingResponse.embeddings &r)))
    (Result.Error e) (IO.errorln &(LLMError.str &e))))

(Fn [(Ref String a), (Array String)] EmbeddingRequest)

                        (embedding-request model input)
                    

creates an embedding request.

• model: the provider-specific embedding model identifier
• input: an array of text strings to embed

(Fn [(Ref String a)] ProviderConfig)

                        (gemini api-key)
                    

creates a provider config for the Google Gemini API.

(Fn [(Ref String a)] ProviderConfig)

                        (ollama base-url)
                    

creates a provider config for Ollama. Takes the base URL (e.g. "http://localhost:11434").

(Fn [(Ref String a)] ProviderConfig)

                        (openai api-key)
                    

creates a provider config for the OpenAI API.

(Fn [(Ref String a), (Ref String b)] ProviderConfig)

                        (vllm base-url api-key)
                    

creates a provider config for a vLLM server. Uses the OpenAI wire format. Takes the base URL (e.g. "http://dgx:8000") and an API key (pass empty string for none).