llm

LlmStream

is a pull-based stream of tokens from an LLM response. Wraps a ResponseStream and parses provider-specific streaming formats. Implements the poll interface.

(match (LLM.chat-stream &config &req)
  (Result.Success stream)
    (do
      (while-do true
        (match (LlmStream.poll &stream)
          (Maybe.Nothing) (break)
          (Maybe.Just tok) (IO.print &tok)))
      (IO.println "")
      (LlmStream.close stream))
  (Result.Error e) (IO.errorln &e))

close

defn

(Fn [LlmStream] ())

                        (close s)

closes the underlying connection.

copy

instantiate

(Fn [(Ref LlmStream a)] LlmStream)

copies a LlmStream.

delete

instantiate

(Fn [LlmStream] ())

deletes a LlmStream. Should usually not be called manually.

init

instantiate

(Fn [ResponseStream, String, String] LlmStream)

creates a LlmStream.

linebuf

instantiate

(Fn [(Ref LlmStream a)] (Ref String a))

gets the linebuf property of a LlmStream.

poll

defn

(Fn [(Ref LlmStream a)] (Maybe String))

                        (poll s)

returns the next token from the LLM stream, or Nothing when complete.

prn

instantiate

(Fn [(Ref LlmStream a)] String)

converts a LlmStream to a string.

provider

instantiate

(Fn [(Ref LlmStream a)] (Ref String a))

gets the provider property of a LlmStream.

rs

instantiate

(Fn [(Ref LlmStream a)] (Ref ResponseStream a))

gets the rs property of a LlmStream.

set-linebuf

instantiate

(Fn [LlmStream, String] LlmStream)

sets the linebuf property of a LlmStream.

set-linebuf!

instantiate

(Fn [(Ref LlmStream a), String] ())

sets the linebuf property of a LlmStream in place.

set-provider

instantiate

(Fn [LlmStream, String] LlmStream)

sets the provider property of a LlmStream.

set-provider!

instantiate

(Fn [(Ref LlmStream a), String] ())

sets the provider property of a LlmStream in place.

set-rs

instantiate

(Fn [LlmStream, ResponseStream] LlmStream)

sets the rs property of a LlmStream.

set-rs!

instantiate

(Fn [(Ref LlmStream a), ResponseStream] ())

sets the rs property of a LlmStream in place.

str

instantiate

(Fn [(Ref LlmStream a)] String)

converts a LlmStream to a string.

update-linebuf

instantiate

(Fn [LlmStream, (Ref (Fn [String] String a) b)] LlmStream)

updates the linebuf property of a LlmStream using a function f.

update-provider

instantiate

(Fn [LlmStream, (Ref (Fn [String] String a) b)] LlmStream)

updates the provider property of a LlmStream using a function f.

update-rs

instantiate

(Fn [LlmStream, (Ref (Fn [ResponseStream] ResponseStream a) b)] LlmStream)

updates the rs property of a LlmStream using a function f.