Skip to content

Latest commit

 

History

History
147 lines (77 loc) · 3.88 KB

File metadata and controls

147 lines (77 loc) · 3.88 KB

Function: useLLMChatSession()

useLLMChatSession(config, options?): object

Defined in: hooks/useLLMChatSession.ts:26

React hook to load and run an LLM chat session.

This hook manages downloading (if remote URLs are provided) and loading the model assets and tokenizer files, tracking download progress and load errors, and releasing native memory when the component unmounts or the configuration changes.

For imperative usage, see createLLMChatSession.

Parameters

config

LLMModel

The LLM model configuration. See LLMModel.

options?

LLMChatSessionOptions & ResourceOptions

Chat session options and load/caching options. See LLMChatSessionOptions & ResourceOptions.

Returns

object

The same object as LLMChatSession (without dispose), combined with loading state and download progress.

downloadProgress

downloadProgress: number

error

error: Error | undefined

getHistory

getHistory: () => readonly ChatMessage[] | undefined = session.getHistory

Type Declaration

() => readonly ChatMessage[]

Returns a snapshot of the current message history in the session.

Returns

readonly ChatMessage[]

undefined

getKVCacheState

getKVCacheState: () => LLMKVCacheState | undefined = session.getKVCacheState

Type Declaration

() => LLMKVCacheState

Returns current KV cache occupancy and total capacity statistics for the session runner.

Returns

LLMKVCacheState

undefined

isReady

isReady: boolean = !!session

resource

resource: LLMModel | undefined

sendMessage

sendMessage: (message, onToken?, genConfig?) => Promise<LLMChatTurnResult> | undefined = session.sendMessage

Type Declaration

(message, onToken?, genConfig?) => Promise<LLMChatTurnResult>

Sends a user message or chat turn to the model and generates a response.

Parameters

message

ChatMessageContent

Message string or interleaved media payload array.

onToken?

(token) => void

Callback fired on the RN thread for each decoded token.

genConfig?

LLMGenerationConfig

Generation options overriding session defaults.

Returns

Promise<LLMChatTurnResult>

A promise resolving to the generated messages and turn stats.

Throws

With code INVALID_ARGUMENT if the message is malformed, or INVALID_STATE if the chat template is non-monotonic.

undefined

stop

stop: () => void | undefined = session.stop

Type Declaration

() => void

Interrupts and stops any active token generation call.

Returns

void

undefined

See

LLMChatSession