RFC enable multimodal and tool usage at once for OAI endpoints ?

[flozi00]

https://github.com/huggingface/chat-ui/blob/8ed1691ecff94e07d10dfb2874d3936d293f4842/src/lib/server/endpoints/openai/endpointOai.ts#L191C53-L191C65

Just played around with combining both of this
What do you think about making tool calling only if no image is in conversation ?
Otherwise we need to insert models twice, once for multi modal and once for tool usage.

A quick solution could be just checking if image_url is part in one of the messages and if it is skip the tools check

Just struggled around because the upload file button was there but didnt worked to do something with the uploaded image until checking the code.

@nsarrazin wdyt ?