i tried with avro input, something like /data/txn_*/* and it works for me
On Wed, Mar 9, 2016 at 12:12 PM, Ted Yu wrote:
> Koert:
> I meant org.apache.hadoop.mapred.FileInputFormat doesn't support multi
> level wildcard.
>
> Cheers
>
> On Wed, Mar 9, 2016 at 8:22 AM, Koert Kuipers wrote:
>
>> i
Koert:
I meant org.apache.hadoop.mapred.FileInputFormat doesn't support multi
level wildcard.
Cheers
On Wed, Mar 9, 2016 at 8:22 AM, Koert Kuipers wrote:
> i use multi level wildcard with hadoop fs -ls, which is the exact same
> glob function call
>
> On Wed, Mar 9, 2016 at 9:24 AM, Ted Yu wro
i use multi level wildcard with hadoop fs -ls, which is the exact same glob
function call
On Wed, Mar 9, 2016 at 9:24 AM, Ted Yu wrote:
> Hadoop glob pattern doesn't support multi level wildcard.
>
> Thanks
>
> On Mar 9, 2016, at 6:15 AM, Koert Kuipers wrote:
>
> if its based on HadoopFsRelatio
Hi,
Unless I've misunderstood what you want to achieve, you could use:
sqlContext.read.json(sc.textFile("/mnt/views-p/base/2016/01/*/*-xyz.json"))
Regards,
Christophe.
On 09/03/16 15:24, Ted Yu wrote:
Hadoop glob pattern doesn't support multi level wildcard.
Thanks
On Mar 9, 2016, at 6:15 AM,
Hadoop glob pattern doesn't support multi level wildcard.
Thanks
> On Mar 9, 2016, at 6:15 AM, Koert Kuipers wrote:
>
> if its based on HadoopFsRelation shouldn't it support it? HadoopFsRelation
> handles globs
>
>> On Wed, Mar 9, 2016 at 8:56 AM, Ted Yu wrote:
>> This is currently not supp
if its based on HadoopFsRelation shouldn't it support it? HadoopFsRelation
handles globs
On Wed, Mar 9, 2016 at 8:56 AM, Ted Yu wrote:
> This is currently not supported.
>
> On Mar 9, 2016, at 4:38 AM, Jakub Liska wrote:
>
> Hey,
>
> is something like this possible?
>
> sqlContext.read.json("/m
This is currently not supported.
> On Mar 9, 2016, at 4:38 AM, Jakub Liska wrote:
>
> Hey,
>
> is something like this possible?
>
> sqlContext.read.json("/mnt/views-p/base/2016/01/*/*-xyz.json")
>
> I switched to DataFrames because my source files changed from TSV to JSON
> but now I'm not
