Skip to content

README.md

Latest commit

 

History

History
125 lines (97 loc) · 4.2 KB

README.md

File metadata and controls

125 lines (97 loc) · 4.2 KB
Raw

M²PO: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation

Official code for the ACL 2026 paper "M²PO: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation". Built on top of the ALMA framework.

Installation

Tested with Python 3.9 (3.9.19) and CUDA 12.8.

pip install -r requirements.txt
# optional, only for vLLM decoding during evaluation:
pip install vllm==0.11.0    # on Python 3.9, then run once: python tools/patch_vllm.py

Data

⚠️ The exact training data used in the paper is company-internal and cannot be released. As a public drop-in alternative, we use the calibration preference data from moore3930/calibrating-llm-mt (16 candidates/source scored by CometKiwi). Results with this substitute may differ slightly from the paper. Notable differences: it provides 5 of the 6 reported directions (no ja-en) and has 16 candidates/source (the paper uses list_size=8; this repo is configured for list_size=16 to match the substitute).

Training data — download the alternative and convert it to a single training JSONL ({"src","lp","tgt_list","score_list"} per line):

bash runs/prepare_train_data.sh          # -> data/mpo_train.jsonl

Test sets — WMT'23 / FLORES-200 / WMT24++, fetched from HuggingFace and written in the format eval.sh reads:

bash runs/download_testsets.sh           # -> data/testset/{wmt23,flores,wmt24}/

Training

Set your base model and paths, then run the driver (train → merge LoRA → evaluate):

BASE_MODEL=/path/to/GemmaX2-28-9B-v0.1 \
DATA_PATH=./data/mpo_train.jsonl \
TESTSET_DIR=./data/testset \
GPU_IDS=0,1 \
bash runs/mmpo_lora_9b.sh

Key hyperparameters (in runs/mmpo_lora_9b.sh):

Arg Value Meaning
--list_size 16 candidates per source (batch size must be a multiple of it)
--loss_type simpo preference loss form
--beta 0.1 SimPO temperature
--lambda_rank 0.5 weight of the calibration loss L_calib
--initial/max_model_weight 0.1 / 0.9 fusion-weight α_t warmup bounds
--lora_rank 32 LoRA rank (LoRA α = 2×rank = 64)

Evaluation

eval.sh decodes a test set with vLLM (using the same prompt as training) and scores with COMET. Run it standalone with a merged model:

TESTSET_DIR=./data/testset \
bash runs/eval.sh wmt23 /path/to/merged_model my_run 0,1     # tags: wmt23 / flores / wmt24

The COMET/BLEU scorers are environment-specific; point them via the variables at the top of runs/eval.sh.

Notes

  • vLLM on Python 3.9. Recent vLLM needs Python ≥3.10; on 3.9 run python tools/patch_vllm.py once to make it importable. For Gemma-family models set VLLM_FLASH_ATTN_VERSION=2 (softcapping) — tools/run_vllm_alma.py already does this by default.

Citation

@inproceedings{wang-etal-2026-m2po,
    title = "{M}$^2${PO}: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation",
    author = "Wang, Hao  and
      Xu, Linlong  and
      Liu, Heng  and
      Liu, Yangyang  and
      Zhao, Xiaohu  and
      Zeng, Bo  and
      Shao, Liangying  and
      Dong, Yichen  and
      Wu, Xinwei  and
      Zhou, Jiang  and
      Dong, Tianyu  and
      Zeng, Xiangxiang  and
      Wang, Longyue  and
      Luo, Weihua",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.acl-long.469/",
    doi = "10.18653/v1/2026.acl-long.469",
    pages = "10315--10336"
}

Acknowledgements

This codebase is built on ALMA. Released under the MIT License.