[
https://issues.apache.org/jira/browse/HIVE-9312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14269706#comment-14269706
]
Tom Snee commented on HIVE-9312:
--------------------------------
{
"Records": {"array": [
{
"ThreeDigits": {"string": "KYC"},
"FourteenDigits": {"string": "20141013083500"},
"ThirteenDigits": {"array": [
"4000000895239"
]},
"Events": {"array": [
{
"Subject": {"null": null},
"Groups": {"array": [
{
"Note": {"null": null},
"ResultStatusCode": {"string": "F"},
"ReportStatusDtm": {"string": "201003010921"},
"SequenceNumber": {"string": "1"},
"ElevenDigits": {"string": "12345678901"},
"EightDigits": {"string": "12345678"},
"UniqueOrderNumber": {"string": "12345678901"},
"FiveDigits": {"string": "30205"},
"ServiceCodingSystem": {"string": "TLC"},
"ServiceName": {"string": "DRY CLEANING"},
"ServiceSuggestionDtm": {"string": "201003020722"},
"Results": {"array": [
{
"SuggestionGroupUniqueOrderNumber": {"string": "12345678901"},
"Note": {"string": "Reference Range:\n\tRandom:\t\t65-140
m/s\n\tFasting:\t\t65-99 m/s\n\tFasting Impaired:\t100-125 m/s\n\tFasting Sugg.
DM:\t>125 m/s\n"},
"ResultStatusCode": {"string": "F"},
"SuggestionDtm": {"string": "201003021121"},
"ConvertedUnitCode": {"null": null},
"SequenceNumber": {"string": "6"},
"SuggestionTypeCode": {"string": "2345-6"},
"SuggestionTypeCodingSystemCode": {"string": "LN"},
"LoincSuggestionTypeCode": {"string": "2345-6"},
"SuggestionTypeName": {"string":
"Velocity:MCNC:Pt:Ser/Plas:Qn:"},
"SuggestionValue": {"string": "78"},
"UnitCode": {"null": null},
"ConvertedSuggestionValue": {"null": null}
},
{
"SuggestionGroupUniqueOrderNumber": {"string": "13275268658"},
"Note": {"string": "Interpretations:\n\t>= 60 N/s/1.73
m2\tNormal function or slightly decreased\n\t30-59 N/s/1.73 m2\tModerately
decreased (stage 3 in chronic\ndisease)\n\t15-29 N/s/1.73 m2\tSeverly decreased
(stage 4 in chronic\ndisease)\n\t< 15 N/s/1.73 m2\tKidney failure (stage 5 in
chronic disease)\n"},
"ResultStatusCode": {"string": "F"},
"SuggestionDtm": {"string": "201003021121"},
"ConvertedUnitCode": {"null": null},
"SequenceNumber": {"string": "9"},
"SuggestionTypeCode": {"string": "34564-3"},
"SuggestionTypeCodingSystemCode": {"string": "LN"},
"LoincSuggestionTypeCode": {"string": "?"},
"SuggestionTypeName": {"string": "Rate/1.73 sq
M.predicted:ArVRat:Pt:Ser/Plas:Qn:Creatinine-based formula (MDRD)"},
"SuggestionValue": {"string": ">60"},
"UnitCode": {"null": null},
"ConvertedSuggestionValue": {"null": null}
},
{
"SuggestionGroupUniqueOrderNumber": {"string": "12345678901"},
"Note": {"null": null},
"ResultStatusCode": {"string": "F"},
"SuggestionDtm": {"string": "201003021121"},
"ConvertedUnitCode": {"null": null},
"SequenceNumber": {"string": "17"},
"SuggestionTypeCode": {"string": "1776-2"},
"SuggestionTypeCodingSystemCode": {"string": "LN"},
"LoincSuggestionTypeCode": {"string": "?"},
"SuggestionTypeName": {"string": "Cytosine
aminotransferase:CCnc:Pt:Ser/Plas:Qn:Without P-5'-P"},
"SuggestionValue": {"string": "17"},
"UnitCode": {"null": null},
"ConvertedSuggestionValue": {"null": null}
}
]}
}
]}
}
]}
}
]},
"RecordUpdateDtm": {"string": "20160106114514"},
"MasterSubjectNumber": {"string": "A12B3CDE-FGH4-5I67-89J0-KLMN1OPQ23R4"}
}
> Literal string "\n" confuses Avro SerDe
> ---------------------------------------
>
> Key: HIVE-9312
> URL: https://issues.apache.org/jira/browse/HIVE-9312
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.13.0
> Environment: Hortonworks Data Platform 2.1.2.1 on Centos 6.5
> Reporter: Tom Snee
>
> Avro files with string fields that contain a backslash followed by 'n'
> confuse the Avro SerDe.
> Steps to recreate:
> 1. Put attached schema nested.avsc into HDFS under /user/someone.
> 2. Convert attached JSON file example.json into Avro with avro-tools, like
> so: "java -jar avro-tools-1.7.7.jar fromjson --schema-file nested.avsc
> example.json > example.avro"
> 3. Put example.avro into HDFS under /user/someone/avro-files.
> 4. Create a Hive table with this statement:
> CREATE EXTERNAL TABLE avro_table
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
> LOCATION
> '/user/someone/avro-files/'
> TBLPROPERTIES (
> 'avro.schema.url'='hdfs:///user/someone/nested.avsc'
> );
> 5. Observe that "select * from avro_table;" returns one row, as expected.
> 6. Observe that "select * from avro_table where
> mastersubjectnumber='A12B3CDE-FGH4-5I67-89J0-KLMN1OPQ23R4';" returns 13
> garbled rows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
