Hi, Just out of curiosity, does your MapReduce job run correctly if you manually just replace the "." with a "_" in your schema
Yes it worked. Regards, SBM On Tue, 30 Jul, 2019, 21:22 Ryan Skraba, <r...@skraba.com> wrote: > OK, I learned something new -- I have never seen the > setValidation(false) before! It looks like it was added for Avro 1.5 > (https://issues.apache.org/jira/browse/AVRO-838) only to be compatible > with files generated by Avro 1.4 implementations that (wrongly) > permitted invalid names. > > It looks like it isn't possible with the existing AvroJob > implementation in that case, and the static methods that use > Schema.parser() are widely baked into the Avro MapReduce classes, so I > can't see an easy workaround. > > You might raise a JIRA or discuss on the dev@ mailing list for a new > feature, but I suspect that the best route would be to try and move > your schemas to names that meet the specification! > > Just out of curiosity, does your MapReduce job run correctly if you > manually just replace the "." with a "_" in your schema? > > All my best, Ryan > > > On Tue, Jul 30, 2019 at 4:21 PM SB M <manjusree1...@gmail.com> wrote: > > > > Hi , > > > > What I mean when using a Schema.parse(), we can set the validation of > names to false, using setValidation method. > > > > But using avrojob there is no option to set this validation to false. > > > > I want this validation option to make the parser validation to false.so > that I can use it my code. > > > > > > What am trying to achieve is I have a avro schema with sub structure > with name has operators separated by dot(.), which is not a valid one when > schema gets parsing. It will throw error. > > > > But there is an option to set this name validation to false while > parsing by using new schema.Parser().setValidation(false); > > > > But with the AvroJob no option to set validation. I needed this feature. > > > > Thanks, > > Sree > > > > > > On Tue, 30 Jul, 2019, 14:11 Ryan Skraba, <r...@skraba.com> wrote: > >> > >> Hello! I'm not sure I understand your question. Some names are > >> *required* with a specific format in the Avro specification > >> (http://avro.apache.org/docs/1.8.2/spec.html#names) > >> > >> What are you looking to accomplish? I can think of two scenarios that > >> we've seen in the past: (1) anonymous records where the name has no > >> interest, and (2) mapping a structure that supports arbitrary UTF-8 > >> names (like a database table) to a record with the same field names. > >> Neither of those are supported in the Avro specification. > >> > >> For the first case (where we don't care about the record name), we > >> just autogenerated a "safe" but unused record name. > >> > >> For the second case, we used a custom annotation on the field > >> (something like "display.name") to contain the original value and > >> generated a "safe" field name. > >> > >> In both cases, being safe means that it meets the Avro spec > >> ([A-Za-z_][A-Za-z0-9_]*) and avoids collisions with other generated > >> names. > >> > >> I hope this helps! Ryan > >> > >> On Fri, Jul 26, 2019 at 1:52 PM SB M <manjusree1...@gmail.com> wrote: > >> > > >> > Hi All, > >> > > >> > Problem: I need a option to set name validation for schema parsing, > when setting with avrojob and avromultipleinputs. > >> > > >> > Is there any way to set schema name validation to false currently, > when am ho through source code am not able to find any options like that. > >> > > >> > Please give a some solution. > >> > > >> > Regards, > >> > Sree. > >> > >
