LLM

(Fn [(Ref String a)] ProviderConfig)

                        (anthropic api-key)
                    

creates a provider config for the Anthropic API.

(Fn [(Ref ProviderConfig a), (Ref LLMRequest b)] (Result LLMResponse LLMError))

                        (chat config req)
                    

sends a chat request to the configured provider. Returns (Result LLMResponse LLMError). On HTTP errors (4xx/5xx), returns a structured LLMError.Api with status code, error type, and message parsed from the provider's error response. On transport failures, returns LLMError.Transport.

Makes exactly one attempt. Use chat-with-retry to retry rate limits and server errors.

(Fn [(Ref ProviderConfig a), (Ref String b), (Array Message), Int, Double, (Ref (Array ToolDef) c), (Fn [(Ref ToolCall d)] String e), Int] (Result LLMResponse LLMError))

                        (chat-loop config model messages max-tokens temperature tools handler max-iters)
                    

runs an agentic tool-use loop. Calls chat with the given tools, invokes handler for each tool call in the response, appends tool results to the conversation, and repeats until the model stops calling tools or max-iters iterations are reached.

The handler receives a (Ref ToolCall) and returns the tool result as a String.

Returns the final (Result LLMResponse LLMError). If the iteration limit is reached, the last response is returned (it may still contain tool calls).

Each request makes exactly one attempt. Use chat-loop-with-retry to retry rate limits and server errors.

(let [config (LLM.ollama "http://localhost:11434")
      tools [(ToolDef.init @"get_weather" @"Get weather" schema)]
      msgs [(Message.user "Weather in Paris?")]
      handler (fn [tc]
                (if (= (ToolCall.name tc) "get_weather")
                  @"22C, sunny"
                  @"unknown tool"))]
  (match (LLM.chat-loop &config "llama3" msgs 256 0.7 &tools handler 10)
    (Result.Success r) (println* (LLMResponse.content &r))
    (Result.Error e) (IO.errorln &(LLMError.str &e))))

(Fn [(Ref ProviderConfig a), (Ref String b), (Array Message), Int, Double, (Ref (Array ToolDef) c), (Fn [(Ref ToolCall d)] String e), Int, (Ref RetryPolicy f)] (Result LLMResponse LLMError))

                        (chat-loop-with-retry config model messages max-tokens temperature tools handler max-iters policy)
                    

runs an agentic tool-use loop in which every request retries according to policy. The policy applies per request, not to the loop as a whole. See chat-loop for the arguments and the return value.

(Fn [(Ref String a), (Array Message), Int, Double] LLMRequest)

                        (chat-request model messages max-tokens temperature)
                    

creates an LLM request without tools.

(Fn [(Ref String a), (Array Message), Int, Double] LLMRequest)

                        (chat-request-json model messages max-tokens temperature)
                    

creates an LLM request that asks for a JSON response.

Note: Anthropic has no native JSON mode, so this falls back to a system prompt instruction (best-effort, not guaranteed). All other providers use their native JSON mode.

(Fn [(Ref String a), (Array Message), Int, Double, JSON] LLMRequest)

                        (chat-request-with-schema model messages max-tokens temperature schema)
                    

creates an LLM request constrained to a JSON schema. The schema is a JSON value (use the JSON constructors).

(Fn [(Ref String a), (Array Message), Int, Double, (Array ToolDef)] LLMRequest)

                        (chat-request-with-tools model messages max-tokens temperature tools)
                    

creates an LLM request with tool definitions.

(Fn [(Ref ProviderConfig a), (Ref LLMRequest b)] (Result LlmStream LLMError))

                        (chat-stream config req)
                    

sends a streaming chat request and returns an LlmStream. Poll the stream for tokens. Returns (Result LlmStream LLMError). Checks the HTTP status code before returning the stream.

Makes exactly one attempt. Use chat-stream-with-retry to retry rate limits and server errors.

(Fn [(Ref ProviderConfig a), (Ref LLMRequest b), (Ref RetryPolicy c)] (Result LlmStream LLMError))

                        (chat-stream-with-retry config req policy)
                    

sends a streaming chat request, retrying failures according to policy. Only the initial response counts: once a stream is handed back, a failure mid-stream is the caller's to handle. See chat-stream for the return value.

(Fn [(Ref ProviderConfig a), (Ref LLMRequest b), (Ref RetryPolicy c)] (Result LLMResponse LLMError))

                        (chat-with-retry config req policy)
                    

sends a chat request, retrying failures according to policy. See chat for the return value and RetryPolicy for the knobs.

(LLM.chat-with-retry &config &req &(RetryPolicy.default))

(Fn [(Ref ProviderConfig a), (Ref EmbeddingRequest b)] (Result EmbeddingResponse LLMError))

                        (embed config req)
                    

sends an embedding request to the configured provider. Returns (Result EmbeddingResponse LLMError).

Anthropic does not offer an embeddings API; calling this with an Anthropic config returns a Transport error.

Makes exactly one attempt. Use embed-with-retry to retry rate limits and server errors.

(let [config (LLM.openai "sk-...")
      req (LLM.embedding-request "text-embedding-3-small"
            [@"hello" @"world"])]
  (match (LLM.embed &config &req)
    (Result.Success r)
      (println* (Array.length (EmbeddingResponse.embeddings &r)))
    (Result.Error e) (IO.errorln &(LLMError.str &e))))

(Fn [(Ref ProviderConfig a), (Ref EmbeddingRequest b), (Ref RetryPolicy c)] (Result EmbeddingResponse LLMError))

                        (embed-with-retry config req policy)
                    

sends an embedding request, retrying failures according to policy. See embed for the return value.

(Fn [(Ref String a), (Array String)] EmbeddingRequest)

                        (embedding-request model input)
                    

creates an embedding request.

• model: the provider-specific embedding model identifier
• input: an array of text strings to embed

(Fn [(Ref String a)] ProviderConfig)

                        (gemini api-key)
                    

creates a provider config for the Google Gemini API.

(Fn [(Ref String a)] ProviderConfig)

                        (ollama base-url)
                    

creates a provider config for Ollama. Takes the base URL (e.g. "http://localhost:11434").

(Fn [(Ref String a)] ProviderConfig)

                        (openai api-key)
                    

creates a provider config for the OpenAI API.

(Fn [(Ref String a), (Ref String b)] ProviderConfig)

                        (vllm base-url api-key)
                    

creates a provider config for a vLLM server. Uses the OpenAI wire format. Takes the base URL (e.g. "http://dgx:8000") and an API key (pass empty string for none).