[
https://issues.apache.org/jira/browse/IO-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800524#comment-17800524
]
Marcono1234 commented on IO-781:
--------------------------------
Thanks [~elharo]! There is one small issue with these changes; have created
https://github.com/apache/commons-io/pull/537 for that.
As mentioned in the description of this issue here I am wondering if it would
be better to have one common implementation for {{CharSequenceInputStream}} and
{{ReaderInputStream}} because {{CharsetEncoder}} usage is pretty error-prone
and the {{CharSequenceInputStream}} implementation is most likely incomplete at
the moment.
I am not sure if the approach I originally proposed in
https://github.com/apache/commons-io/pull/293 (and the comments there) would be
a reasonable way to solve this.
> CharSequenceInputStream.available() returns too large numbers in some cases
> ---------------------------------------------------------------------------
>
> Key: IO-781
> URL: https://issues.apache.org/jira/browse/IO-781
> Project: Commons IO
> Issue Type: Bug
> Components: Streams/Writers
> Affects Versions: 2.11.0
> Reporter: Marcono1234
> Priority: Major
> Fix For: 2.16.0
>
>
> h3. Description
> The {{available()}} method of
> {{org.apache.commons.io.input.CharSequenceInputStream}} erroneously returns
> values larger than the actual number of available bytes in some cases.
> The underlying issue is that {{CharSequenceInputStream}} makes incorrect
> assumptions about the relation between chars and bytes. The
> {{CodingErrorAction.REPLACE}} can convert 2 chars (1 supplementary code
> point) to one byte (the replacement char {{?}}). Additionally in case
> {{CharSequenceInputStream}} is ever extended to support specifying a
> {{CharsetEncoder}}, the {{CodingErrorAction.IGNORE}} would probably cause
> similar issues. There might also be some uncommon charsets which can encode 2
> chars to 1 byte; though I am not aware of such charset yet.
> This was originally mentioned in pull request
> [#293|https://github.com/apache/commons-io/pull/293]. That PR also proposed
> to replace the underlying {{CharSequenceInputStream}} implementation with
> {{ReaderInputStream}} because in general using {{CharsetEncoder}} is
> error-prone so it might be good to avoid having two classes implementing
> logic on top of it. (Potentially {{CharSequenceInputStream}} is missing a
> call to {{CharsetEncoder.flush}}, see also IO-714)
> h3. Example
> In the example below {{available()}} erroneously returns 2 even though only 1
> byte can be read.
> {code}
> Charset charset = Charset.forName("Big5");
> CharSequenceInputStream in = new CharSequenceInputStream("\uD800\uDC00",
> charset);
> // BUG: available() returns 2 but only 1 byte is read afterwards
> System.out.println("Available: " + in.available());
> // Note: readAllBytes() is a method added in Java 9
> System.out.println("Actually read: " + in.readAllBytes().length);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
