Hi Mike, This new APE for HDFS offers the same capabilities as APE 4, only with a different external filesystem as the target. I have incorporated the same update into the APE.
Thank you, Savyasach Reddy From: Mike Carey <dtab...@gmail.com> Date: Friday, 8 November 2024 at 11:21 AM To: dev@asterixdb.apache.org <dev@asterixdb.apache.org> Subject: Re: [DISCUSS] Support writing query result to HDFS QUESTION - What is the real/full proposed grammar for this as it relates to the containing AsterixDB SQL++ grammar? I'm asking because this APE (APE 15) seems to only allow (at least as described) copying a full collection to or from HDFS, where as the S3 case (APE 4) is much richer - it allows copying a query result to HDFS and also provides a way to structure the results (e.g., into a date-organized hierarchy). I was expecting this new APE for HDFS to support the same capabilities, including a SQL++ query as the source and result structuring, just for a different target external filesystem. What's the plan? Thanks, Mike On 11/7/24 3:16 PM, Murtadha Hubail wrote: > +1 to implementing this enhancement. > > -Murtadha > ________________________________ > From: Savyasach Reddy Enukonda<esavyared...@gmail.com> > Sent: Friday, November 8, 2024 12:08:51 AM > To:dev@asterixdb.apache.org <dev@asterixdb.apache.org> > Subject: [DISCUSS] Support writing query result to HDFS > > Hi all, > > I’m reaching out to start a discussion to support writing query results to > HDFS. > AsterixDB’s current external datasets implementation is restricted to reading > from unsecured HDFS. We propose enabling the reading and copying of results > to secured-HDFS clusters, similar to the copy and read operations from S3. > APE:https://cwiki.apache.org/confluence/x/I42MEw > > Best, > Savyasach Reddy >
