Re: Hive Avro: Directly use of embedded Avro Scheme

2020-10-31 Thread David
Hey Dennis, Specifying the schema url is simply a convenience tool so you can have a single schema defined instead of having a SQL schema (CREATE TABLE) and a separate Avro schema file which reduces maintenance overhead and prevents a situation where the two could potentially fall out of sync. Th

AW: Hive Avro: Directly use of embedded Avro Scheme

2020-10-31 Thread Dennis Suhari
Understood. So to hold the schema stable you should have an external reference to an avrc url (eg registry) which can evolve. And checking new Avro against registry is made easy because avrc is embedded. And if changed you can easily create a new version. Is this the idea ? Br, Dennis _

Re: Hive Avro: Directly use of embedded Avro Scheme

2020-10-31 Thread David
What would your expectation be? That Hive reads the first file it finds and uses that schema in the table definition? What if the table is empty and a user attempts an INSERT? What should be the behavior? The real power of Avro is not so much that the schema can exist (optionally) in the file i

Hive Avro: Directly use of embedded Avro Scheme

2020-10-31 Thread Dennis Suhari
Hello Support, currently I have created the following AVRO Hive table which works fine. CREATE EXTERNAL TABLE blahblah.blublub STORED AS AVRO LOCATION "/***/in" TBLPROPERTIES ('avro.schema.url‘=‚/.../schema/blublub.avsc') As you can see I need to use the schema 'avro.schema.url' property whic