This paper introduces a quantization method that couples contiguous channels and quantizes them in this coupled form (using Fisher Information). It seems rather intuitive to extend this method into KVQuant to enhance the quantization of key activations. Are there any plans by this team to do such work?
This paper introduces a quantization method that couples contiguous channels and quantizes them in this coupled form (using Fisher Information). It seems rather intuitive to extend this method into KVQuant to enhance the quantization of key activations. Are there any plans by this team to do such work?