getting error while fine tuning gemma 3 #2376
-
|
i tried to fine tune gemma 3 model using unsloth but i am getting the below error. |
Beta Was this translation helpful? Give feedback.
Answered by
danielhanchen
Oct 11, 2025
Replies: 4 comments 2 replies
-
|
hi, can you show your code here ? |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
hi, ) |
Beta Was this translation helpful? Give feedback.
-
|
Hi @uscneps , I am getting the same error with gemma3 12b it and gemma3 4b it model model, tokenizer = FastModel.from_pretrained(
model_name = "unsloth/gemma-3-12b-it",
max_seq_length = 2048, # Choose any for long context!
load_in_4bit = True, # 4 bit quantization to reduce memory
load_in_8bit = False,
full_finetuning = False, # [NEW!] We have full finetuning now!
)
model = FastModel.get_peft_model(
model,
finetune_vision_layers = False, # Turn off for just text!
finetune_language_layers = True, # Should leave on!
finetune_attention_modules = True, # Attention good for GRPO
finetune_mlp_modules = True, # SHould leave on always!
r = 8, # Larger = higher accuracy, but might overfit
lora_alpha = 8, # Recommended alpha == r at least
lora_dropout = 0,
bias = "none",
random_state = 3407,
)
# <data preparation and stuff copied from official unsloth gemma3 collab notebook>
from trl import SFTTrainer, SFTConfig
from transformers import TrainingArguments, DataCollatorForSeq2Seq, EarlyStoppingCallback
from unsloth import is_bfloat16_supported
trainer = SFTTrainer(
model = model,
data_collator=DataCollatorForSeq2Seq(tokenizer=tokenizer),
# tokenizer = tokenizer,
train_dataset = train_dataset,
eval_dataset = val_dataset,
callbacks=[EarlyStoppingCallback(early_stopping_patience=3)],
args=SFTConfig(
dataset_text_field = "text",
dataset_num_proc = 2,
max_seq_length = 2048,
packing = False,
learning_rate = 2e-4,
output_dir = "gemma-4b-it-checkpoints",
eval_strategy="epoch",
per_device_train_batch_size = 8,
per_device_eval_batch_size = 8,
gradient_accumulation_steps=4,
torch_empty_cache_steps = 16,
weight_decay = 0.01,
num_train_epochs=6,
lr_scheduler_type = "cosine",
warmup_steps = 50,
logging_steps = 1,
logging_nan_inf_filter = False,
save_strategy="epoch",
save_total_limit=3,
seed = 42,
bf16 = False,
fp16= False,
run_name="gemma3-4b-it-trial-01",
load_best_model_at_end=True,
metric_for_best_model="loss",
optim = "adamw_8bit",
report_to = "wandb",
),
)
trainer_stats = unsloth_train(trainer)Error: Detailed Error Stack |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
I got the same error when I tried to fine-tune "sarvamai/sarvam-translate", which is a gemma-3-based model error |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Oh my apologies I did not notice we have already solved this issue since late June 2025 and we did not notify any of you - so sorry! Gemma-3 works as expected, but you need to update Unsloth or rerun the Gemma-3 notebook for eg our Gemma 3 270M Chess example or Gemma 3 4B finetuning example To update Unsloth, please do: pip install --upgrade --force-reinstall --no-cache-dir --no-deps unsloth unsloth_zooTo enable full finetuning on Gemma-3 do: model, tokenizer = FastModel.from_pretrained(
model_name = "unsloth/gemma-3-270m-it",
max_seq_length = max_seq_length, # Choose any for long context!
load_in_4bit = False, # 4 bit quantization to reduce memory
load_in_8bit = False, # [NEW!] A bit more accurate, uses 2x memory
full_finetuning = True, # [NEW!] We have full finetuning now!
)To enable model, tokenizer = FastModel.from_pretrained(
model_name = "unsloth/gemma-3-270m-it",
max_seq_length = max_seq_length, # Choose any for long context!
# full_finetuning = True, # [NEW!] We have full finetuning now!
torch_dtype = torch.float32,
)If you experience OOMs with Gemma-3 270M full finetuning, reminder to change per_device_train_batch_size = 8,
gradient_accumulation_steps = 1, # Use GA to mimic batch size!to per_device_train_batch_size = 1,
gradient_accumulation_steps = 8, # Use GA to mimic batch size!and due to our universal Gradient Accumulation bug fix, both the above are equivalent, with the 2nd @peteparker123 @N-E-W-T-O-N @uscne @Preet-Sojitra So sorry probably tagging you all is way too late. |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Thank you, no worries and thanks for your time! |
Beta Was this translation helpful? Give feedback.
Answer selected by
danielhanchen
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Oh my apologies I did not notice we have already solved this issue since late June 2025 and we did not notify any of you - so sorry!
Gemma-3 works as expected, but you need to update Unsloth or rerun the Gemma-3 notebook for eg our Gemma 3 270M Chess example or Gemma 3 4B finetuning example
To update Unsloth, please do:
To enable full finetuning on Gemma-3 do: