Federico Mariani created CAMEL-23739:
----------------------------------------
Summary: camel-openai: support WrappedFile, byte[], and
InputStream bodies for vision models
Key: CAMEL-23739
URL: https://issues.apache.org/jira/browse/CAMEL-23739
Project: Camel
Issue Type: Improvement
Affects Versions: 4.20.0
Reporter: Federico Mariani
The camel-openai component only accepts _java.io.File_ as the message body for
vision/image input. This prevents direct integration with file-based consumers
(ftp, sftp, aws2-s3, azure-storage-blob, etc.) that produce _WrappedFile,
byte[], or InputStream_ bodies.
*Current behavior*
_OpenAIProducer.buildUserMessage()_ checks body instanceof File (line 270). Any
other body type falls through to buildTextMessage(), which converts the binary
content to a garbage string.
This means a simple route like:
{code:java}
from("ftp:images?binary=true&noop=true")
.to("openai:chat-completion?userMessage=Describe this image")
{code}
fails because the FTP consumer produces a _GenericFile<FTPFile>_ (which
implements WrappedFile), not a java.io.File.
Additionally, _buildFileMessage()_ uses Files.probeContentType(path) for MIME
detection, which returns null on macOS for temp files and certain directories,
causing IllegalArgumentException: Only text and image files are supported even
when the body is a valid File.
*Expected behavior*
The component should accept any body type that Camel consumers produce and
handle image detection transparently:
1. Use Camel's type converter (in.getBody(File.class)) instead of instanceof
check — this handles GenericFile → File conversion via GenericFileConverter
2. Support byte[] and InputStream bodies directly — read bytes, base64-encode,
build ChatCompletionContentPartImage
3. Fall back to extension-based MIME detection when Files.probeContentType()
returns null (e.g., URLConnection.guessContentTypeFromName() or an extension
lookup)
4. Detect MIME type from component-specific headers (CamelAwsS3ContentType,
CamelMinioContentType, Content-Type, CamelFileName extension) when the body has
no file path
*Reference implementation*
_camel-langchain4j-agent_ already solves this problem in
LangChain4jAgentConverter. It handles WrappedFile, byte[], and InputStream with
extension-based MIME detection and cloud storage header support. The same
approach could be applied to camel-openai.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
