Interface LlamaCppCallOptions

Hierarchy

Properties

callbacks? configurable? maxConcurrency? maxTokens? metadata? onToken? recursionLimit? runName? signal? stop? tags? timeout?

Properties

Optional callbacks

callbacks?: Callbacks

Callbacks for this call and any sub-calls (eg. a Chain calling an LLM). Tags are passed to all callbacks, metadata is passed to handle*Start callbacks.

Optional configurable

configurable?: Record<string, any>

Runtime values for attributes previously made configurable on this Runnable, or sub-Runnables.

Optional maxConcurrency

maxConcurrency?: number

Maximum number of parallel calls to make.

Optional maxTokens

maxTokens?: number

The maximum number of tokens the response should contain.

Optional metadata

metadata?: Record<string, unknown>

Metadata for this call and any sub-calls (eg. a Chain calling an LLM). Keys should be strings, values should be JSON-serializable.

Optional onToken

onToken?: ((tokens) => void)

Type declaration

    • (tokens): void

    • A function called when matching the provided token array

      Parameters

      • tokens: number[]

      Returns void

Optional recursionLimit

recursionLimit?: number

Maximum number of times a call can recurse. If not provided, defaults to 25.

Optional runName

runName?: string

Name for the tracer run for this call. Defaults to the name of the class.

Optional signal

signal?: AbortSignal

Abort signal for this call. If provided, the call will be aborted when the signal is aborted.

See

https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal

Optional stop

stop?: string[]

Stop tokens to use for this call. If not provided, the default stop tokens for the model will be used.

Optional tags

tags?: string[]

Tags for this call and any sub-calls (eg. a Chain calling an LLM). You can use these to filter calls.

Optional timeout

timeout?: number

Timeout for this call in milliseconds.

Settings

Member Visibility

Theme

On This Page

Generated using TypeDoc