I have a problem: KV Cache isn't working with streaming. I've tried several methods, but they all fail; either it doesn't work at all, or the model loses its memory.
Environment: Android
I wanted to store the KV cache in the files and retrieve it for use with streaming, but that didn't work either.
I have a problem: KV Cache isn't working with streaming. I've tried several methods, but they all fail; either it doesn't work at all, or the model loses its memory.
Environment: Android
I wanted to store the KV cache in the files and retrieve it for use with streaming, but that didn't work either.