How to solve lm_head.weight | UNEXPECTED at UBC-NLP/AraT5-base?

[emadyounan]

import pandas as pd
import torch
from transformers import AutoModel, AutoTokenizer, AutoModelForSeq2SeqLM, Seq2SeqTrainingArguments, Seq2SeqTrainer
from datasets import Dataset
import os
from huggingface_hub import login

login(token="Token")


MODEL_NAME = "UBC-NLP/AraT5-base" 
CSV_FILE = "arabic_ocr_errors_3000.csv" 
OUTPUT_DIR = "./ocr_corrector_model"
BATCH_SIZE = 8
EPOCHS = 5  
MAX_LENGTH = 128


df = pd.read_csv(CSV_FILE)
dataset = Dataset.from_pandas(df[['input', 'target']])


dataset = dataset.train_test_split(test_size=0.1)
train_dataset = dataset['train']
eval_dataset = dataset['test']


tokenizer = AutoModel.from_pretrained(MODEL_NAME, dtype="auto")

I got warning here

[transformers] T5Model LOAD REPORT from: UBC-NLP/AraT5-base
Key            | Status     |  | 
---------------+------------+--+-
lm_head.weight | UNEXPECTED |  | 

Notes:
- UNEXPECTED:   can be ignored when loading from different task/architecture; not ok if you expect identical arch.

model = AutoModelForSeq2SeqLM.from_pretrained(MODEL_NAME)

def preprocess_function(examples):
    """تحويل النص إلى مدخلات رقمية"""
    # إضافة بادئة "تصحيح: " لتعريف النموذج بالمهمة
    inputs = ["target: " + text for text in examples["input"]]
    model_inputs = tokenizer(inputs, max_length=MAX_LENGTH, truncation=True, padding="max_length", return_tensors=None)

    # تجهيز النصوص الصحيحة كـ Labels
    labels = tokenizer(examples["target"], max_length=MAX_LENGTH, truncation=True, padding="max_length", return_tensors=None)

    # استبدل padding token بـ -100 علشان يتجاهله النموذج أثناء التدريب
    labels_ids = [
        [(l if l != tokenizer.pad_token_id else -100) for l in label]
        for label in labels["input_ids"]
    ]
    
    model_inputs["labels"] = labels_ids
    return model_inputs

I got an error here.

model_inputs = tokenizer(inputs, max_length=MAX_LENGTH, truncation=True, padding="max_length", return_tensors=None)

Exception has occurred: AttributeError
'list' object has no attribute 'size'
File "C:\VS_Code\arat5_base\train_ocr_corrector.py", line 41, in preprocess_function
model_inputs = tokenizer(inputs, max_length=MAX_LENGTH, truncation=True, padding="max_length", return_tensors=None)
File "C:\VS_Code\arat5_base\train_ocr_corrector.py", line 56, in
train_dataset = train_dataset.map(preprocess_function, batched=True)
AttributeError: 'list' object has no attribute 'size'

[emadyounan]

I'm using Windows 11
Python 3.13