Fine-Tuning an LLM for Log Anomaly Detection Using Sequence Probability

[joaorr13]

I’m exploring the use of LLMs for anomaly detection in logs. My idea is to fine-tune an LLM (e.g., LLaMA 3 8B) using only non-anomalous logs. After training, I’d like the model to estimate the likelihood of a log sequence and classify it as anomalous or not based on a probability threshold. First, do you think this could be an interesting approach for this task? Second, are you aware of similar work—maybe in a different domain—where an LLM was fine-tuned to detect anomalies or outliers, or to estimate the probability of a sequence? I’m very new to fine-tuning and would love pointers to examples or resources to see how it’s done.

[173787247]

Re: Fine-Tuning an LLM for Log Anomaly Detection Using Sequence Probability

Lock the variables before assuming a logic bug:

1. Exact version (not “latest”) and OS
2. Minimal repro: clean profile/session, extra plugins off
3. First error block (missing from the post — add it if you can)

No replies yet — a minimal repro (version + one command + first error) will get you a maintainer faster than a long essay.
Repo: unslothai/unsloth. With those three, this is usually install vs config vs upstream. Searching the title keywords in Issues avoids duplicating a known thread.