[GitHub] [doris] jollygrass opened a new issue, #11438: [Bug] After the output file compression is enabled in hive, the data read from hive external table is garbled

[GitBox]

jollygrass opened a new issue, #11438:
URL: https://github.com/apache/doris/issues/11438

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   1.1.0
   
   ### What's Wrong?
   
   After the output file compression is enabled in hive, the data read from 
hive external table is garbled 。
   
-----------------------------------------------------------------------------------------------------------------
   当hive中开启hive.exec.compress.output后，doris中通过外表方式查询hive表数据，获取的数据乱码。
   
   ### What You Expected?
   
   Query data correctly from hive external table.
   
   ### How to Reproduce?
   
   ----------------------------in hive----------------------------
   set hive.exec.compress.output=true;
   set 
mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.SnappyCodec;
   
   drop table if exists dim_test;
   
   create table dim_test (
        key_1 string,
        key_2 string
   );
   
   insert overwrite table dim_test values 
('123456','测试'),('abc','123456'),('测试','abc');
   
   select * from dim_test;
   
   key_1 |key_2 |
   ------|------|
   123456|测试    |
   abc   |123456|
   测试    |abc   |
   
   ----------------------------in doris----------------------------
   drop table if exists dim_test;
   
   CREATE TABLE `dim_test` (
        key_1 varchar(100),
        key_2 varchar(100)
   )
   ENGINE=HIVE
   PROPERTIES (
   'hive.metastore.uris' = 'thrift://192.168.1.201:9083',
   'database' = 'bigdata_dev',
   'table' = 'dim_test'
   );
   
   SELECT * from dim_test;
   
   key_1           |key_2|
   ----------------|-----|
      $   $$H123456|测试   |
   abc             |1  , |
   测试              |abc  |
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org