Hi,
We have a PR to support fixed length byte array in parquet file.
https://github.com/apache/spark/pull/1737
Can someone help verifying it?
Thanks.
2014-07-15 19:23 GMT+08:00 Pei-Lun Lee :
> Sorry, should be SPARK-2489
>
>
> 2014-07-15 19:22 GMT+08:00 Pei-Lun Lee :
>
> Filed SPARK-2446
>>
>
Sorry, should be SPARK-2489
2014-07-15 19:22 GMT+08:00 Pei-Lun Lee :
> Filed SPARK-2446
>
>
>
> 2014-07-15 16:17 GMT+08:00 Michael Armbrust :
>
> Oh, maybe not. Please file another JIRA.
>>
>>
>> On Tue, Jul 15, 2014 at 12:34 AM, Pei-Lun Lee wrote:
>>
>>> Hi Michael,
>>>
>>> Good to know it is
Filed SPARK-2446
2014-07-15 16:17 GMT+08:00 Michael Armbrust :
> Oh, maybe not. Please file another JIRA.
>
>
> On Tue, Jul 15, 2014 at 12:34 AM, Pei-Lun Lee wrote:
>
>> Hi Michael,
>>
>> Good to know it is being handled. I tried master branch (9fe693b5) and
>> got another error:
>>
>> scala>
Oh, maybe not. Please file another JIRA.
On Tue, Jul 15, 2014 at 12:34 AM, Pei-Lun Lee wrote:
> Hi Michael,
>
> Good to know it is being handled. I tried master branch (9fe693b5) and got
> another error:
>
> scala> sqlContext.parquetFile("/tmp/foo")
> java.lang.RuntimeException: Unsupported pa
Hi Michael,
Good to know it is being handled. I tried master branch (9fe693b5) and got
another error:
scala> sqlContext.parquetFile("/tmp/foo")
java.lang.RuntimeException: Unsupported parquet datatype optional
fixed_len_byte_array(4) b
at scala.sys.package$.error(package.scala:27)
at
org.apache.s
This is not supported yet, but there is a PR open to fix it:
https://issues.apache.org/jira/browse/SPARK-2446
On Mon, Jul 14, 2014 at 4:17 AM, Pei-Lun Lee wrote:
> Hi,
>
> I am using spark-sql 1.0.1 to load parquet files generated from method
> described in:
>
> https://gist.github.com/massie/7
