LLM Bug Batch

[mat888]

I'm using this single document as a shared reference and disclosure of AI usage.

I put ChatGPT 5.6 Luna (weakest) to the task of finding bugs in Odin source code, then 5.6 Sol (strongest) to clean, deduplicate, and make corrections. All issues from this batch will reference this discussion topic.

Issues numbered MXXX regard only bugs found in the src/ folder.

The initial prompt was:

We're going to do a deep review of the Odin languge, specifically the language source code (not the packages).
Any found issues should include a method of demonstrating the issue, how it might be encountered in the wild. The best would be a small odin file and paired compiler command which reproduces the bug.
Never run any found issue which is a security issue by yourself. Keep those in a high-priority folder. You can make a agent-work folder at this outermost level to do all your work in.
The source code is in the /src folder.
Any questions?

If this type of bug reporting is inundative or non-productive this may also serve as a place to flag those concerns; I understand this is an issue for other projects. I've attempted to deduplicate based on existing issues, PRs, and fixes, which explains some gaps in the numbering.

For reference I'm on a $20/month plan and probably used about 35% of a weekly usage limit. This type of output is becoming very affordable, and I won't claim to have "prompting expertise -" I'm sure more experienced maintainers could easily drive such a process better than I have.

[Yawning]

For what it's worth, I have no problem with AI assisted bug discovery, especially when:

I've attempted to deduplicate based on existing issues, PRs, and fixes, which explains some gaps in the numbering.

Which is a massive difference from what other projects are complaining about which is getting flooded with low-effort spam. A bit of redundancy is unavoidable with efforts going in parallel, but everything your efforts produced so far have been legitimate issues, that are actionable.

[ayeshaaa0134]

i would say don't post all the issues at once. Start with the 5-10 you're most sure are real bugs. On each one, say if you checked it yourself (it compiles and shows the bug) or if it's only what the AI found and no one tested it. This helps maintainers know which ones to trust first. You can post the rest later, in small groups, instead of all at once.

[antisaling]

ask your bot to use GH cli to search for existing similar bugs too!

[mocompute]

I'm not a core team member so ignore as you wish, but I wanted to also suggest the following: you might want to maintain a branch with automated known failure tests, and carefully maintain the tests. This can help the agents avoid duplication, and it will create an infrastructure where related issues can be quickly resolved as the compiler team releases bug fixes.

[mat888]

As an update, @kalsprite, who has greater competency than me around Odin, has pursued a similar undertaking. Rather than potentially do a lot of redundant work, I handed him my results so he could do a monolithic sweep himself. That's why I have stopped posting new issues regarding this.