[
https://issues.apache.org/jira/browse/HIVE-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Subroto Sanyal updated HIVE-4986:
---------------------------------
Component/s: Serializers/Deserializers
> Problem in creating configration in
> org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils
> -------------------------------------------------------------------------------------
>
> Key: HIVE-4986
> URL: https://issues.apache.org/jira/browse/HIVE-4986
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Reporter: Subroto Sanyal
>
> While using AvroSerDe but, I am facing the problem:
> {noformat}
> java.lang.IllegalArgumentException: Wrong FS:
> hdfs://xxx:8020/abc/bbc/test.avsc, expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> at
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.getSchemaFromHDFS(AvroSerdeUtils.java:105)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.determineSchemaOrThrowException(AvroSerdeUtils.java:70)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.determineSchemaOrReturnErrorSchema(AvroSerdeUtils.java:87)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:59)
> at test.subroto.HiveUtil.getDeserializer(HiveUtil.java:102)
> {noformat}
> While debugging the problem I saw a piece of code in:
> {code:title=org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.java|borderStyle=solid}
> public static Schema More ...determineSchemaOrThrowException(Properties
> properties)
> throws IOException, AvroSerdeException {
> String schemaString = properties.getProperty(SCHEMA_LITERAL);
> if(schemaString != null && !schemaString.equals(SCHEMA_NONE))
> return Schema.parse(schemaString);
>
> // Try pulling directly from URL
> schemaString = properties.getProperty(SCHEMA_URL);
> if(schemaString == null || schemaString.equals(SCHEMA_NONE))
> throw new AvroSerdeException(EXCEPTION_MESSAGE);
>
> try {
> if(schemaString.toLowerCase().startsWith("hdfs://"))
> return getSchemaFromHDFS(schemaString, new Configuration());
> } catch(IOException ioe) {
> throw new AvroSerdeException("Unable to read schema from HDFS: " +
> schemaString, ioe);
> }
>
> return Schema.parse(new URL(schemaString).openStream());
> }
> {code}
> Here the configuration passed from the Deserilaizer is completely ignored and
> a new Configuration object is created.
> In my opinion the AvroSerDe should honor the Configuration being passed to
> Deserializer interface ; rather than creating a new one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
