How do I use KV Cache to save sessions?

[Togaroda]

I noticed you added session management or something similar, but I didn't see any explanation of how to use it with LlamaBridge or the Android library.

[ferranpons]

Hi @Togaroda,

You’re right, session management is already implemented, but the documentation hasn’t been updated yet. Here’s a quick guide on how to use it:

LlamaBridge.initGenerateModel(modelPath)

// Starts a fresh generation/session
val first = LlamaBridge.generate("Explain Kotlin coroutines in simple terms")

// Continues using the existing KV cache
val second = LlamaBridge.generateContinue("Now give me a short example")

// Persist the current session to disk
LlamaBridge.sessionSave(sessionPath)

// Later, after loading the same model again
LlamaBridge.initGenerateModel(modelPath)
LlamaBridge.sessionLoad(sessionPath)

// Continue from the restored session
val third = LlamaBridge.generateContinue("Summarize everything in one paragraph")

// Reset the current session if needed
LlamaBridge.sessionReset()

Some notes:
• generate(...) starts a new session.
• generateContinue(...) reuses the current KV cache.
• sessionSave(...) / sessionLoad(...) allow persistence.
• sessionReset() clears the current session without unloading the model.
• WASM currently doesn’t support session persistence.

I’ll update the documentation soon to make this clearer. Thanks for pointing it out!