Running as a vision provider on Apple silicon

[anthonyalayo]

Hey there,

Thanks a lot for this project. I read through the docs and also tried your Devin integration, but I was unable to get a clear answer. I would like to try having mineru as a vision provider to a coding agent. But as I go through the utils here, I see a couple gaps that are unclear:

1. There are multiple CPU programmatic steps in that the mineru stack takes care of while interacting with its models. Is there an openai compatible configuration that preserves this? Or do the model serving environments not include these essential components?
2. Is there support for the vllm-metal plugin when running this on macos? Otherwise, does the software have a way to utilize the MPS on Apple silicon?

Thanks, I am very interested to give this a shot,
Anthony