nf4

An innovative library for efficient LLM inference via low-bit quantization

Nodes to run Hunyuan Image 3 locally with BF16 and NF4 quantized options in Comfyui

This repository wraps the flux fill model as ComfyUI nodes. Compared to the flux fill dev model, these nodes can use the flux fill model to perform inpainting and outpainting work under lower VRM conditions

An optimized, high-performance NF4 (NormalFloat 4-bit) dequantization Triton GPU kernel achieving up to 1.41x speedup over bitsandbytes.

This project implements a classic Retrieval-Augmented Generation (RAG) system using HuggingFace models with quantization techniques. The system processes PDF documents, extracts their content, and enables interactive question-answering through a Streamlit web application.

Pure Gleam tensor library with quantization (INT8, NF4, AWQ), Flash Attention, and 2:4 Sparsity - 7.5x memory multiplication

Candle NF4 QLoRA layers on peft-rs; GGUF export is custom (not llama.cpp Q4_0).

This project presents a medical question–answering language model built by fine-tuning Google Gemma-2-2B-IT using LoRA (Low-Rank Adaptation) 🧠⚕️. The primary objective is to adapt a general-purpose large language model to the healthcare domain in a parameter-efficient, reproducible, and resource-aware manner.

Single-launch grouped W4A16 GEMM over fused NF4 MoE expert stacks (bitsandbytes #1949 conventions) — with pre-registered, OTS-stamped blind benchmarks, including the failures

Fine-tuned TinyLlama-1.1B on GSM8K using QLoRA (NF4 4-bit) for step-by-step math word problem solving, with a FastAPI inference endpoint.

QLoRA-compatible NF4 quantization for fused-3D MoE expert tensors — train through a frozen 743B MoE on one node (fills the still-open bitsandbytes #1849 gap)

Ideogram 4 本地 CUDA 图像生成器：内置 NF4 权重、官方 Magic Prompt 提示词优化、Windows/Linux 便携包 / Local open-weight text-to-image generator

A hands-on, runnable tour of LLM quantization: compress GPT-2 to 4-bit while tracking memory and quality, ending with QLoRA's NF4 and double quantization.

AI model compression benchmarks — NF4 beats INT8 in every metric