llm

LlmStream

is a pull-based stream of tokens and tool calls from an LLM response. Wraps a ResponseStream and parses provider-specific streaming formats. Use poll for text-only streaming or poll-event for streaming with tool call support.

(match (LLM.chat-stream &config &req)
  (Result.Success stream)
    (do
      (while-do true
        (match (LlmStream.poll-event &stream)
          (Maybe.Nothing) (break)
          (Maybe.Just evt)
            (match-ref &evt
              (StreamEvent.Text tok) (IO.print tok)
              (StreamEvent.ToolCallEvent tc)
                (IO.println &(String.concat
                  &[@"tool: " @(ToolCall.name tc)])))))
      (LlmStream.close stream))
  (Result.Error e) (IO.errorln &e))

close

defn

(Fn [LlmStream] ())

                        (close s)

closes the underlying connection.

copy

instantiate

(Fn [(Ref LlmStream a)] LlmStream)

copies a LlmStream.

delete

instantiate

(Fn [LlmStream] ())

deletes a LlmStream. Should usually not be called manually.

init

instantiate

(Fn [ResponseStream, String, String, String, String, String, (Array ToolCall), Bool] LlmStream)

creates a LlmStream.

linebuf

instantiate

(Fn [(Ref LlmStream a)] (Ref String a))

gets the linebuf property of a LlmStream.

pending-tcs

instantiate

(Fn [(Ref LlmStream a)] (Ref (Array ToolCall) a))

gets the pending-tcs property of a LlmStream.

poll

defn

(Fn [(Ref LlmStream a)] (Maybe String))

                        (poll s)

returns the next token from the LLM stream, or Nothing when complete. For text-only streaming. Use poll-event for streaming with tool call support.

poll-event

defn

(Fn [(Ref LlmStream a)] (Maybe StreamEvent))

                        (poll-event s)

returns the next StreamEvent from the LLM stream, or Nothing when complete. Unlike poll, this handles tool calls from streaming responses — incremental tool call data is accumulated internally and emitted as complete StreamEvent.ToolCallEvent values. Parallel tool calls (multiple tool calls in a single response) are fully supported.

Gemini does not provide tool call IDs, so ToolCall values from Gemini streams will have an empty string as their ID.

(while-do true
  (match (LlmStream.poll-event &stream)
    (Maybe.Nothing) (break)
    (Maybe.Just evt)
      (match-ref &evt
        (StreamEvent.Text tok) (IO.print tok)
        (StreamEvent.ToolCallEvent tc)
          (println* "tool: " (ToolCall.name tc)))))

prn

instantiate

(Fn [(Ref LlmStream a)] String)

converts a LlmStream to a string.

provider

instantiate

(Fn [(Ref LlmStream a)] (Ref String a))

gets the provider property of a LlmStream.

rs

instantiate

(Fn [(Ref LlmStream a)] (Ref ResponseStream a))

gets the rs property of a LlmStream.

set-linebuf

instantiate

(Fn [LlmStream, String] LlmStream)

sets the linebuf property of a LlmStream.

set-linebuf!

instantiate

(Fn [(Ref LlmStream a), String] ())

sets the linebuf property of a LlmStream in place.

set-pending-tcs

instantiate

(Fn [LlmStream, (Array ToolCall)] LlmStream)

sets the pending-tcs property of a LlmStream.

set-pending-tcs!

instantiate

(Fn [(Ref LlmStream a), (Array ToolCall)] ())

sets the pending-tcs property of a LlmStream in place.

set-provider

instantiate

(Fn [LlmStream, String] LlmStream)

sets the provider property of a LlmStream.

set-provider!

instantiate

(Fn [(Ref LlmStream a), String] ())

sets the provider property of a LlmStream in place.

set-rs

instantiate

(Fn [LlmStream, ResponseStream] LlmStream)

sets the rs property of a LlmStream.

set-rs!

instantiate

(Fn [(Ref LlmStream a), ResponseStream] ())

sets the rs property of a LlmStream in place.

set-stream-done

instantiate

(Fn [LlmStream, Bool] LlmStream)

sets the stream-done property of a LlmStream.

set-stream-done!

instantiate

(Fn [(Ref LlmStream a), Bool] ())

sets the stream-done property of a LlmStream in place.

set-tc-args

instantiate

(Fn [LlmStream, String] LlmStream)

sets the tc-args property of a LlmStream.

set-tc-args!

instantiate

(Fn [(Ref LlmStream a), String] ())

sets the tc-args property of a LlmStream in place.

set-tc-id

instantiate

(Fn [LlmStream, String] LlmStream)

sets the tc-id property of a LlmStream.

set-tc-id!

instantiate

(Fn [(Ref LlmStream a), String] ())

sets the tc-id property of a LlmStream in place.

set-tc-name

instantiate

(Fn [LlmStream, String] LlmStream)

sets the tc-name property of a LlmStream.

set-tc-name!

instantiate

(Fn [(Ref LlmStream a), String] ())

sets the tc-name property of a LlmStream in place.

str

instantiate

(Fn [(Ref LlmStream a)] String)

converts a LlmStream to a string.

stream-done

instantiate

(Fn [(Ref LlmStream a)] (Ref Bool a))

gets the stream-done property of a LlmStream.

tc-args

instantiate

(Fn [(Ref LlmStream a)] (Ref String a))

gets the tc-args property of a LlmStream.

tc-id

instantiate

(Fn [(Ref LlmStream a)] (Ref String a))

gets the tc-id property of a LlmStream.

tc-name

instantiate

(Fn [(Ref LlmStream a)] (Ref String a))

gets the tc-name property of a LlmStream.

update-linebuf

instantiate

(Fn [LlmStream, (Ref (Fn [String] String a) b)] LlmStream)

updates the linebuf property of a LlmStream using a function f.

update-pending-tcs

instantiate

(Fn [LlmStream, (Ref (Fn [(Array ToolCall)] (Array ToolCall) a) b)] LlmStream)

updates the pending-tcs property of a LlmStream using a function f.

update-provider

instantiate

(Fn [LlmStream, (Ref (Fn [String] String a) b)] LlmStream)

updates the provider property of a LlmStream using a function f.

update-rs

instantiate

(Fn [LlmStream, (Ref (Fn [ResponseStream] ResponseStream a) b)] LlmStream)

updates the rs property of a LlmStream using a function f.

update-stream-done

instantiate

(Fn [LlmStream, (Ref (Fn [Bool] Bool a) b)] LlmStream)

updates the stream-done property of a LlmStream using a function f.

update-tc-args

instantiate

(Fn [LlmStream, (Ref (Fn [String] String a) b)] LlmStream)

updates the tc-args property of a LlmStream using a function f.

update-tc-id

instantiate

(Fn [LlmStream, (Ref (Fn [String] String a) b)] LlmStream)

updates the tc-id property of a LlmStream using a function f.

update-tc-name

instantiate

(Fn [LlmStream, (Ref (Fn [String] String a) b)] LlmStream)

updates the tc-name property of a LlmStream using a function f.