[jira] [Commented] (TIKA-4305) Tika producing empty output for UCS encoded txt files; parses UTF-7 files as UTF-8

[Manish S N (Jira)]

    [ 
https://issues.apache.org/jira/browse/TIKA-4305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17880485#comment-17880485
 ] 

Manish S N commented on TIKA-4305:
----------------------------------

 

I get correct output for UTF-16 and UTF-32 but not UCS-4 and UCS-2

this is the code i used to test tika
{code:java}
package org.manish;

import java.io.FileInputStream;
import org.apache.tika.Tika;

public class Main {
    public static void main(String[] args){
        Tika tika = new Tika();
        try{
            System.out.println("the parsed content is : ");
            System.out.println(tika.parseToString(new 
FileInputStream("multilingual_test_new_UCS-2.txt")));
            System.out.println("done");
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
} {code}
and this is the maven [^pom.xml] i use with the code

> Tika producing empty output for UCS encoded txt files; parses UTF-7 files as 
> UTF-8
> ----------------------------------------------------------------------------------
>
>                 Key: TIKA-4305
>                 URL: https://issues.apache.org/jira/browse/TIKA-4305
>             Project: Tika
>          Issue Type: Bug
>          Components: tika-app, tika-core
>    Affects Versions: 2.9.2
>         Environment: Ubuntu 22.04 LTS
>            Reporter: Manish S N
>            Priority: Minor
>         Attachments: multilingual_test_new_UCS-2.txt, 
> multilingual_test_new_UCS-4.txt, multilingual_test_new_UTF-7.txt, 
> multilingual_test_new_UTF-8.txt, pom.xml, tika_UTF-7_output.txt
>
>
> Tika producing empty string as output for UCS-2 and UCS-4 encoded txt files.
> No logs or errors just an empty string.
> Other formats are okay except UTF-7 files are parsed as UTF-8 which wreaks 
> havoc with non ascii characters.
> how do i know that?: I opened an the UTF-7 file as UTF-8 encoded using open 
> dialog of gedit and found the outputs similar
>  
> I am attaching all four encoded files along with tika's output from parsing 
> the UTF-7 for reference



--
This message was sent by Atlassian Jira
(v8.20.10#820010)