Re: Writing to S3 and retrieving folder names

2015-03-05 Thread Mike Trienis
Please ignore my question, you can simply specify the root directory and it looks like redshift takes care of the rest. copy mobile from 's3://BUCKET_NAME/' credentials json 's3://BUCKET_NAME/jsonpaths.json' On Thu, Mar 5, 2015 at 3:33 PM, Mike Trienis wrote: > Hi All, > > I am receiving d

Writing to S3 and retrieving folder names

2015-03-05 Thread Mike Trienis
Hi All, I am receiving data from AWS Kinesis using Spark Streaming and am writing the data collected in the dstream to s3 using output function: dstreamData.saveAsTextFiles("s3n://XXX:XXX@/") After the run the application for several seconds, I end up with a sequence of directories in S3 tha