Ensembled Pipeline Inference for Human Preference Classification
Notebook in this folder
llm_classification_inference.ipynb
Given a prompt and two model responses, predict which one a human preferred. The difficulty is positional bias: a model that reads response A first will tend to favour it, and that preference has nothing to do with the responses themselves.
The countermeasure is to run the same data twice, once in each order, and average the results. Each pass uses a different backbone, so the ensemble also averages over architecture.
| Stage | What it does |
|---|---|
| Data structuring | Builds two views of the test set, one standard and one with Response A and Response B interchanged |
| Gemma-2 pass | Runs Gemma-2-9B with manual layer allocation across two GPUs and a variable-length collator |
| Llama-3 pass | Repeats the partitioning for Llama architecture dimensions, run against the swapped view |
| Ensemble | Averages the two passes, cancelling the order effect that either pass would carry alone |
Important
Inference runs with the internet disabled. The model weights and the
human_pref helper module are attached as Kaggle datasets rather than
installed, and the notebook expects them to be present in the environment.
Stack · torch transformers xformers pandas numpy