[
https://issues.apache.org/jira/browse/CAMEL-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Federico Mariani updated CAMEL-23739:
-------------------------------------
Issue Type: Bug (was: Improvement)
> camel-openai: support WrappedFile, byte[], and InputStream bodies for vision
> models
> -----------------------------------------------------------------------------------
>
> Key: CAMEL-23739
> URL: https://issues.apache.org/jira/browse/CAMEL-23739
> Project: Camel
> Issue Type: Bug
> Affects Versions: 4.20.0
> Reporter: Federico Mariani
> Priority: Major
>
> The camel-openai component only accepts _java.io.File_ as the message body
> for vision/image input. This prevents direct integration with file-based
> consumers (ftp, sftp, aws2-s3, azure-storage-blob, etc.) that produce
> _WrappedFile, byte[], or InputStream_ bodies.
> *Current behavior*
> _OpenAIProducer.buildUserMessage()_ checks body instanceof File (line 270).
> Any other body type falls through to buildTextMessage(), which converts the
> binary content to a garbage string.
> This means a simple route like:
> {code:java}
> from("ftp:images?binary=true&noop=true")
> .to("openai:chat-completion?userMessage=Describe this image")
> {code}
> fails because the FTP consumer produces a _GenericFile<FTPFile>_ (which
> implements WrappedFile), not a java.io.File.
> Additionally, _buildFileMessage()_ uses Files.probeContentType(path) for MIME
> detection, which returns null on macOS for temp files and certain
> directories, causing IllegalArgumentException: Only text and image files are
> supported even when the body is a valid File.
> *Expected behavior*
> The component should accept any body type that Camel consumers produce and
> handle image detection transparently:
> 1. Use Camel's type converter (in.getBody(File.class)) instead of instanceof
> check — this handles GenericFile → File conversion via GenericFileConverter
> 2. Support byte[] and InputStream bodies directly — read bytes,
> base64-encode, build ChatCompletionContentPartImage
> 3. Fall back to extension-based MIME detection when Files.probeContentType()
> returns null (e.g., URLConnection.guessContentTypeFromName() or an extension
> lookup)
> 4. Detect MIME type from component-specific headers (CamelAwsS3ContentType,
> CamelMinioContentType, Content-Type, CamelFileName extension) when the body
> has no file path
> *Reference implementation*
> _camel-langchain4j-agent_ already solves this problem in
> LangChain4jAgentConverter. It handles WrappedFile, byte[], and InputStream
> with extension-based MIME detection and cloud storage header support. The
> same approach could be applied to camel-openai.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
