Re: problem writing to s3

2015-04-23 Thread Akhil Das
n using s3 paths and send the log information. >>> >>> thanks >>> Daniel >>> >>> thanks >>> Daniel >>> >>> On Wed, Apr 22, 2015 at 1:45 AM, Akhil Das >>> wrote: >>> >>>> Can you look in you

Re: problem writing to s3

2015-04-23 Thread Daniel Mahler
ening in there? Are >>> you able to write the same to your HDFS? >>> >>> Thanks >>> Best Regards >>> >>> On Wed, Apr 22, 2015 at 4:45 AM, Daniel Mahler >>> wrote: >>> >>>> I am having a strange problem wri

Re: problem writing to s3

2015-04-22 Thread Akhil Das
in there? Are >> you able to write the same to your HDFS? >> >> Thanks >> Best Regards >> >> On Wed, Apr 22, 2015 at 4:45 AM, Daniel Mahler wrote: >> >>> I am having a strange problem writing to s3 that I have distilled to >>> this minimal e

Re: problem writing to s3

2015-04-22 Thread Daniel Mahler
our worker logs and see whats happening in there? Are you > able to write the same to your HDFS? > > Thanks > Best Regards > > On Wed, Apr 22, 2015 at 4:45 AM, Daniel Mahler wrote: > >> I am having a strange problem writing to s3 that I have distilled to this >> mi

Re: problem writing to s3

2015-04-21 Thread Akhil Das
Can you look in your worker logs and see whats happening in there? Are you able to write the same to your HDFS? Thanks Best Regards On Wed, Apr 22, 2015 at 4:45 AM, Daniel Mahler wrote: > I am having a strange problem writing to s3 that I have distilled to this > minimal example: &

problem writing to s3

2015-04-21 Thread Daniel Mahler
I am having a strange problem writing to s3 that I have distilled to this minimal example: def jsonRaw = s"${outprefix}-json-raw" def jsonClean = s"${outprefix}-json-clean" val txt = sc.textFile(inpath)//.coalesce(shards, false) txt.count val res = txt.saveAsTextFil