Picture tokens

[arthurwolf]

Hello.

The new -vision model lets you pass images (which end up as tokens).
To properly calculate token use for my stuff, I'd need to be able to pass my full conversation/payload, which includes images.
Does this tokenizer support that (reading the docs I don't see signs it does).
Would you have recommendations of how to implement that on my side?
And if I do implement it would that be a welcome PR?

Cheers.

[niieani]

No support at this moment, but this would definitely be a welcome PR! Thanks Arthur.

[henrymcl]

Hello.

The new -vision model lets you pass images (which end up as tokens). To properly calculate token use for my stuff, I'd need to be able to pass my full conversation/payload, which includes images. Does this tokenizer support that (reading the docs I don't see signs it does). Would you have recommendations of how to implement that on my side? And if I do implement it would that be a welcome PR?

Cheers.

How will you be supplying your images? In base64 or external URL?

[arthurwolf]

Either is fine, I use base64 right now.