Re: Schema store for Parquet

Thanks Magnus, I’ll explore Atlas and see what I can find. On Wed, Mar 4, 2020 at 11:10 AM Magnus Nilsson wrote: > Apache Atlas is the apache data catalog. Maybe want to look into that. It > depends on what your use case is. > > On Wed, Mar 4, 2020 at 8:01 PM Ruijing Li wrote: > >> Thanks Luca

Re: Spark 2.4.5 - Structured Streaming - Failed Jobs expire from the UI

unsubscribe Disclaimer The information in this email and any attachments may contain proprietary and confidential information that is intended for the addressee(s) only. If you are not the intended recipient, you are he

Re: Spark 2.4.5 - Structured Streaming - Failed Jobs expire from the UI

Hi, Can I please get some love on this? Thanks, Puneet -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Still incompatible schemas

Yeah, thanks Zahid for the reply; but that's not it. I found two schemas that differ. So I have the sucker(s) now...but how to handle them? In this case there are two columns, one is a Double and the other is Decimal(19,5) which in Parquet seems to be represented at FIXED_LENGTH_BYTE_ARRAY price

Re: Still incompatible schemas

*This issue of has been discussed resolved on this page * *https://issues.apache.org/jira/browse/SPARK-17557 * *It is suggested by one person that by simply reading the parquet file in a different way as illustrated the error may go away. It

Re: Hostname :BUG

Hey floyd , I just realised something: You need to practice using the adduser command to create users or in your case useradd because that's less painless for you to create a user. Instead of working in root. Trust me it is good for you. Then you will realise this bit of code new SparkConf() is

Still incompatible schemas

Hi folks, Thanks for the help thus far. I'm trying to track down the source of this error: java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary w hen doing a message.show() Basically I'm reading in a single Parquet file (to try to narrow th