It is a good question. I'm not familiar with Apache commons VFS (which I
assume you are talking about, versus the BSD/Unix VFS layer). There no
doubt will be semantic differences between Hadoop FS interface and VFS. It
would be an interesting exercise to implement a connector that bridges the
gap,
+1 (binding)
Thanks to the Ozone folks for their efforts at maintaining good separation
with HDFS and common. I took a lot of heat for the unpopular opinion that
they should be separate, so I am glad the process has worked out well for
both codebases. It looks like my concerns were addressed and
Hi Wei-Chiu,
Can you share the calendar link again for this meeting?
Thanks,
Aaron
On Wed, Sep 4, 2019 at 9:31 AM Matt Foley wrote:
> Sorry I won’t be able to come today; a work meeting interferes.
> —Matt
>
> On Sep 4, 2019, at 9:10 AM, Wei-Chiu Chuang wrote:
>
> It's a short week so I didn'
Thank you Wei-Chiu for organizing this and sending out notes!
On Wed, Aug 21, 2019 at 1:10 PM Wei-Chiu Chuang wrote:
> We had a great turnout today, thanks to Konstantin for leading the
> discussion of the NameNode Fine-Grained Locking proposal.
>
> There were at least 16 participants joined the
Congrats Gabor! Have really enjoyed working with you, looking forward to
more good stuff
On Thu, Jul 4, 2019 at 4:31 AM Gabor Bota
wrote:
> Thank you!
>
> On Thu, Jul 4, 2019 at 8:50 AM Szilard Nemeth >
> wrote:
>
> > Congrats, Gabor!
> >
> > On Tue, Jul 2, 2019, 01:36 Sean Mackrory wrote:
> >
> Da
> > >
> > > On Mon, Jun 17, 2019 at 5:14 PM Ajay Kumar > > .invalid>
> > > wrote:
> > >
> > > > Congrats Aaron!!
> > > >
> > > > On Mon, Jun 17, 2019 at 4:00 PM Daniel Templeton
&g
+1 to unprotect feature branch (in general) for rebasing against trunk.
On Tue, May 14, 2019 at 7:53 PM Dinesh Chitlangia
wrote:
> +1(non-binding) for branch.
>
> -Dinesh
>
>
>
>
> On Tue, May 14, 2019 at 10:04 PM Brahma Reddy Battula
> wrote:
>
> > Yes Arpit,it’s not for trunk..
> >
> >
> > On
[
https://issues.apache.org/jira/browse/HADOOP-16251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Fabbri resolved HADOOP-16251.
---
Resolution: Fixed
Fix Version/s: 3.2.1
> ABFS: add FSMainOperationsBaseT
Aaron Fabbri created HADOOP-16291:
-
Summary: HDFS Permissions Guide appears incorrect about
getFileStatus()/getFileInfo()
Key: HADOOP-16291
URL: https://issues.apache.org/jira/browse/HADOOP-16291
our JIRA notification settings?
> 2. is it going to the right email address?
>
> On Wed, May 1, 2019 at 5:30 AM Aaron Fabbri wrote:
>
>> Hello,
>>
>> I haven't been getting emailed when someone tags or cc:s me in a JIRA. Is
>> there a way to change this?
>>
>> Thanks!
>> Aaron
>>
>
Hello,
I haven't been getting emailed when someone tags or cc:s me in a JIRA. Is
there a way to change this?
Thanks!
Aaron
+1 assuming we get the technical issues sorted. I'll give your patch a go
today.
As Steve mentioned on the JIRA, unsupported JVM is called out as an
exception in the compatibility guidelines:
"The JVM requirements will not change across point releases within the same
minor release except if the J
+1. I think formatted patches and PRs will be an improvement. I've used
the git --committer thing a couple of times here without issue.
Another unrelated improvement with github is handling of large changes. I
really think large patches should be split up into logical subcommitts and
PR's support
(list moved to bcc:)
Thank you for all the responses to my email. There is more interest than I
expected!
I've created a small survey to collect details from people interested in
being mentored. Please fill it out if you are interested.
https://goo.gl/forms/oRfOUe1nYhrZNX3u2
I also want to set
the google summer of code program is setting for the season, ASF supports
> this if you want to volunteer to help out there -it'd be great for the
> project and whoever joins up
>
> > On 14 Jan 2019, at 18:39, Aaron Fabbri wrote:
> >
> > Hi,
> >
> > I'
Hi,
I'd like to offer to mentor a developer who is interested in getting into
Apache Hadoop development.
If you or someone you know is interested, please email me (unicast). This
would probably be a three month thing where we meet every week or two.
I'm also curious if Apache and/or Hadoop commu
Thanks Sunil and everyone who has worked on this release.
+1 from me.
- Verified checksums for tar file.
- Built from tar.gz.
- Ran through S3A and S3Guard integration tests (in AWS us-west 2).
This includes a yarn minicluster test but is mostly focused on s3a/s3guard.
Cheers,
Aaron
On Thu, J
achieve the faster release
> cadence
> >>>>>> also.
> >>>>>>
> >>>>>> Thoughts?
> >>>>>>
> >>>>>> - Sunil
> >>>>>>
> >>>>>>
> >>>>>
Aaron Fabbri created HADOOP-15780:
-
Summary: S3Guard: document how to deal with non-S3Guard processes
writing data to S3Guarded buckets
Key: HADOOP-15780
URL: https://issues.apache.org/jira/browse/HADOOP-15780
Aaron Fabbri created HADOOP-15779:
-
Summary: S3guard: add inconsistency detection metrics
Key: HADOOP-15779
URL: https://issues.apache.org/jira/browse/HADOOP-15779
Project: Hadoop Common
+1 (Binding). Haven't thoroughly reviewed all the new code but successfully
ran all the integration and unit tests today. Based on the progress this
looks ready to merge. Good work folks.
On Wed, Sep 19, 2018 at 5:19 PM John Zhuge wrote:
> +1 (Binding) Nice effort all around.
>
> On Tue, Sep 1
Aaron Fabbri created HADOOP-15754:
-
Summary: s3guard: testDynamoTableTagging should clear existing
config
Key: HADOOP-15754
URL: https://issues.apache.org/jira/browse/HADOOP-15754
Project: Hadoop
Aaron Fabbri created HADOOP-15621:
-
Summary: s3guard: implement time-based (TTL) expiry for DynamoDB
Metadata Store
Key: HADOOP-15621
URL: https://issues.apache.org/jira/browse/HADOOP-15621
Project
On Tue, Jul 17, 2018 at 7:21 PM Steve Loughran
wrote:
>
>
> On 16 Jul 2018, at 23:45, Sunil G sun...@apache.org>> wrote:
>
> I would also would like to take this opportunity to come up with a detailed
> plan.
>
> - Feature freeze date : all features should be merged by August 10, 2018.
>
>
>
>
Aaron Fabbri created HADOOP-15525:
-
Summary: s3a: clarify / improve support for mixed ACL buckets
Key: HADOOP-15525
URL: https://issues.apache.org/jira/browse/HADOOP-15525
Project: Hadoop Common
[
https://issues.apache.org/jira/browse/HADOOP-11697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Fabbri resolved HADOOP-11697.
---
Resolution: Duplicate
Fix Version/s: 3.0.0
> Use larger value
Aaron Fabbri created HADOOP-15444:
-
Summary: ITestS3GuardToolDynamo should only run with -Ddynamo
Key: HADOOP-15444
URL: https://issues.apache.org/jira/browse/HADOOP-15444
Project: Hadoop Common
Aaron Fabbri created HADOOP-15420:
-
Summary: s3guard ITestS3GuardToolLocal failures in diff tests
Key: HADOOP-15420
URL: https://issues.apache.org/jira/browse/HADOOP-15420
Project: Hadoop Common
This appears to be happening again, e.g.
https://builds.apache.org/job/Hadoop-trunk-Commit/14050/
On Mon, Mar 26, 2018 at 3:31 PM, Xiaoyu Yao wrote:
> Could this caused by the docker base image changes? As shown in the error
> message, we are still expecting protobuf v2.5.0 but the one in the
Aaron Fabbri created HADOOP-15400:
-
Summary: Improve S3Guard documentation on Authoritative Mode
implemenation
Key: HADOOP-15400
URL: https://issues.apache.org/jira/browse/HADOOP-15400
Project
[
https://issues.apache.org/jira/browse/HADOOP-15207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Fabbri resolved HADOOP-15207.
---
Resolution: Invalid
Can someone please ban this user? Similar SEO spam has been pasted on
Yep, I'll review it now.
On Wed, Jan 24, 2018 at 9:19 AM, Steve Loughran
wrote:
> Can I get a review of HADOOP-14918
>
> All it does is rm the local dynamo test for S3Guard because the relevant
> JARs don't get updated by AWS any more, and it is out of sync with the
> latest AWS SDKs. If you (ex
Aaron Fabbri created HADOOP-15040:
-
Summary: AWS SDK NPE bug spams logs w/ Yarn Log Aggregation
Key: HADOOP-15040
URL: https://issues.apache.org/jira/browse/HADOOP-15040
Project: Hadoop Common
Aaron Fabbri created HADOOP-14936:
-
Summary: S3Guard: remove "experimental" from documentation
Key: HADOOP-14936
URL: https://issues.apache.org/jira/browse/HADOOP-14936
Project: Had
[
https://issues.apache.org/jira/browse/HADOOP-14746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Fabbri resolved HADOOP-14746.
---
Resolution: Duplicate
Fix Version/s: 3.0.0-beta1
Resolving this.. it was covered by
+1
Built from source. Ran S3A integration tests in us-west-2 with S3Guard
(both Local and Dynamo metadatastore).
Everything worked fine except I hit one integration test failure. It is a
minor test issue IMO and I've filed HADOOP-14927
Failed tests:
ITestS3GuardToolDynamoDB>AbstractS3GuardToo
Aaron Fabbri created HADOOP-14927:
-
Summary: ITestS3GuardTool failures in testDestroyNoBucket()
Key: HADOOP-14927
URL: https://issues.apache.org/jira/browse/HADOOP-14927
Project: Hadoop Common
> >>
> >> On Thu, Aug 24, 2017 at 5:01 AM, Steve Loughran >
> >> wrote:
> >>
> >>>
> >>> On 23 Aug 2017, at 19:21, Aaron Fabbri mailto:fa
> >>> b...@cloudera.com>> wrote:
> >>>
> >>>
&
nding)
>
> I'm happy with it; it's a great piece of work by (in no particular order):
> Chris Nauroth, Aaron Fabbri, Sean McRory & Mingliang Liu. plus a few bits
> in the corners where I got to break things while they were all asleep. Also
> deserving a mention: Thomas
hat way I can avoid fielding hard
> questions
>
>
Hah!
-AF
>
> On Fri, Aug 18, 2017 at 4:12 AM, Steve Loughran mailto:ste...@hortonworks.com>> wrote:
>
> I can do a demo of this next week if people are interested
>
> > On 17 Aug 2017, at 23:07, Aaron Fabbri mailto:fa
. Please shout if I've missed anything
important here or in the VOTE process.
Cheers,
Aaron Fabbri
On Wed, Aug 16, 2017 at 1:39 PM, Andrew Wang
wrote:
> Hi Steve,
>
> What's the target release vehicle, and the timeline for merging this? The
> target date for beta1 is mid-September, so any large code movements make me
> nervous.
>
I think this is ready to get in before beta1. Most of upstream
Hi,
I'd like to create a troubleshooting page on the Hadoop Confluence wiki for
HADOOP-14467.
Can someone please grant me access? Account under fab...@apache.org.
Aaron Fabbri created HADOOP-14633:
-
Summary: S3Guard: optimize create codepath
Key: HADOOP-14633
URL: https://issues.apache.org/jira/browse/HADOOP-14633
Project: Hadoop Common
Issue Type
Thanks for the hard work on this! +1 (non-binding)
- Built from source tarball on OS X w/ Java 1.8.0_45.
- Deployed mini/pseudo cluster.
- Ran grep and wordcount examples.
- Poked around ResourceManager and JobHistory UIs.
- Ran all s3a integration tests in US West 2.
On Thu, Jul 6, 2017 at 10:
Aaron Fabbri created HADOOP-14548:
-
Summary: S3Guard: issues running parallel tests w/ S3N
Key: HADOOP-14548
URL: https://issues.apache.org/jira/browse/HADOOP-14548
Project: Hadoop Common
Aaron Fabbri created HADOOP-14527:
-
Summary: ITestS3GuardListConsistency is too slow
Key: HADOOP-14527
URL: https://issues.apache.org/jira/browse/HADOOP-14527
Project: Hadoop Common
Issue
[
https://issues.apache.org/jira/browse/HADOOP-14509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Fabbri resolved HADOOP-14509.
---
Resolution: Duplicate
Resolving as duplicate, thank you [~mackrorysd
Aaron Fabbri created HADOOP-14509:
-
Summary: InconsistentAmazonS3Client adds extra paths to
listStatus() after delete.
Key: HADOOP-14509
URL: https://issues.apache.org/jira/browse/HADOOP-14509
Aaron Fabbri created HADOOP-14506:
-
Summary: Add create() contract test that verifies ancestor dir
creation
Key: HADOOP-14506
URL: https://issues.apache.org/jira/browse/HADOOP-14506
Project: Hadoop
Aaron Fabbri created HADOOP-14505:
-
Summary: simplify mkdirs() after S3Guard delete tracking change
Key: HADOOP-14505
URL: https://issues.apache.org/jira/browse/HADOOP-14505
Project: Hadoop Common
Aaron Fabbri created HADOOP-14504:
-
Summary: ProvidedFileStatusIterator#next() may throw
IndexOutOfBoundsException
Key: HADOOP-14504
URL: https://issues.apache.org/jira/browse/HADOOP-14504
Project
Aaron Fabbri created HADOOP-14468:
-
Summary: S3Guard: make short-circuit getFileStatus() configurable
Key: HADOOP-14468
URL: https://issues.apache.org/jira/browse/HADOOP-14468
Project: Hadoop Common
Aaron Fabbri created HADOOP-14467:
-
Summary: S3Guard: Improve FNFE message when opening a stream
Key: HADOOP-14467
URL: https://issues.apache.org/jira/browse/HADOOP-14467
Project: Hadoop Common
Aaron Fabbri created HADOOP-14428:
-
Summary: s3a: mkdir appears to be broken
Key: HADOOP-14428
URL: https://issues.apache.org/jira/browse/HADOOP-14428
Project: Hadoop Common
Issue Type: Bug
[
https://issues.apache.org/jira/browse/HADOOP-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Fabbri resolved HADOOP-13981.
---
Resolution: Fixed
Documentation was added as part of S3Guard CLI JIRAs, resolving
Aaron Fabbri created HADOOP-14391:
-
Summary: s3a: auto-detect region for bucket and use right endpoint
Key: HADOOP-14391
URL: https://issues.apache.org/jira/browse/HADOOP-14391
Project: Hadoop Common
Aaron Fabbri created HADOOP-14323:
-
Summary: ITestS3GuardListConsistency failure w/ Local,
authoritative metadata store
Key: HADOOP-14323
URL: https://issues.apache.org/jira/browse/HADOOP-14323
Aaron Fabbri created HADOOP-14283:
-
Summary: S3A may hang due to bug in AWS SDK 1.11.86
Key: HADOOP-14283
URL: https://issues.apache.org/jira/browse/HADOOP-14283
Project: Hadoop Common
Issue
Aaron Fabbri created HADOOP-14144:
-
Summary: s3guard: CLI import does not yield an empty diff.
Key: HADOOP-14144
URL: https://issues.apache.org/jira/browse/HADOOP-14144
Project: Hadoop Common
Aaron Fabbri created HADOOP-14096:
-
Summary: s3guard: regression in dirListingUnion
Key: HADOOP-14096
URL: https://issues.apache.org/jira/browse/HADOOP-14096
Project: Hadoop Common
Issue
Aaron Fabbri created HADOOP-14087:
-
Summary: S3A typo in pom.xml test exclusions
Key: HADOOP-14087
URL: https://issues.apache.org/jira/browse/HADOOP-14087
Project: Hadoop Common
Issue Type
I will take a look (though my review will be non-binding).
On Mon, Feb 6, 2017 at 8:28 AM, Steve Loughran
wrote:
> can I get a review/feedback on, S3A block output streams don't delete
> temporary files in multipart uploads"
> https://issues.apache.org/jira/browse/HADOOP-14028
>
> ...this is pre
Aaron Fabbri created HADOOP-14051:
-
Summary: S3Guard: link docs from index, fix typos
Key: HADOOP-14051
URL: https://issues.apache.org/jira/browse/HADOOP-14051
Project: Hadoop Common
Issue
On Thu, Feb 2, 2017 at 2:56 PM, Steve Loughran
wrote:
>
> > On 2 Feb 2017, at 08:52, Aaron Fabbri wrote:
> >
> > Hello,
> >
> > I'd like to propose merging the HADOOP-13345 feature branch to trunk.
> >
> > I just wrote up a status summary on
Hello,
I'd like to propose merging the HADOOP-13345 feature branch to trunk.
I just wrote up a status summary on HADOOP-13998 "initial s3guard preview"
that goes into more detail.
Cheers,
Aaron
Aaron Fabbri created HADOOP-14036:
-
Summary: S3Guard: intermittent duplicate item keys failure
Key: HADOOP-14036
URL: https://issues.apache.org/jira/browse/HADOOP-14036
Project: Hadoop Common
Aaron Fabbri created HADOOP-14013:
-
Summary: S3Guard: fix multi-bucket integration tests
Key: HADOOP-14013
URL: https://issues.apache.org/jira/browse/HADOOP-14013
Project: Hadoop Common
Aaron Fabbri created HADOOP-13995:
-
Summary: s3guard cli: make tests easier to run and address failure
Key: HADOOP-13995
URL: https://issues.apache.org/jira/browse/HADOOP-13995
Project: Hadoop Common
Aaron Fabbri created HADOOP-13981:
-
Summary: S3Guard CLI: Add documentation
Key: HADOOP-13981
URL: https://issues.apache.org/jira/browse/HADOOP-13981
Project: Hadoop Common
Issue Type: Sub
Aaron Fabbri created HADOOP-13980:
-
Summary: S3Guard CLI: Add fsck check command
Key: HADOOP-13980
URL: https://issues.apache.org/jira/browse/HADOOP-13980
Project: Hadoop Common
Issue Type
Aaron Fabbri created HADOOP-13914:
-
Summary: s3guard: improve S3AFileStatus#isEmptyDirectory handling
Key: HADOOP-13914
URL: https://issues.apache.org/jira/browse/HADOOP-13914
Project: Hadoop Common
Aaron Fabbri created HADOOP-13886:
-
Summary: s3guard:
ITestS3AFileOperationCost.testFakeDirectoryDeletion failure
Key: HADOOP-13886
URL: https://issues.apache.org/jira/browse/HADOOP-13886
Project
Aaron Fabbri created HADOOP-13877:
-
Summary: S3Guard: fix TestDynamoDBMetadataStore when
fs.s3a.s3guard.ddb.table is set
Key: HADOOP-13877
URL: https://issues.apache.org/jira/browse/HADOOP-13877
Aaron Fabbri created HADOOP-13876:
-
Summary: S3Guard: better support for multi-bucket access including
read-only
Key: HADOOP-13876
URL: https://issues.apache.org/jira/browse/HADOOP-13876
Project
Aaron Fabbri created HADOOP-13793:
-
Summary: s3guard: add inconsistency injection, integration tests
Key: HADOOP-13793
URL: https://issues.apache.org/jira/browse/HADOOP-13793
Project: Hadoop Common
Aaron Fabbri created HADOOP-13761:
-
Summary: S3Guard: implement retries
Key: HADOOP-13761
URL: https://issues.apache.org/jira/browse/HADOOP-13761
Project: Hadoop Common
Issue Type: Sub-task
Aaron Fabbri created HADOOP-13760:
-
Summary: S3Guard: add delete tracking
Key: HADOOP-13760
URL: https://issues.apache.org/jira/browse/HADOOP-13760
Project: Hadoop Common
Issue Type: Sub
Aaron Fabbri created HADOOP-13651:
-
Summary: S3Guard: S3AFileSystem Integration with MetadataStore
Key: HADOOP-13651
URL: https://issues.apache.org/jira/browse/HADOOP-13651
Project: Hadoop Common
Aaron Fabbri created HADOOP-13649:
-
Summary: s3guard: implement time-based (TTL) expiry for
LocalMetadataStore
Key: HADOOP-13649
URL: https://issues.apache.org/jira/browse/HADOOP-13649
Project
Aaron Fabbri created HADOOP-13631:
-
Summary: S3Guard: implement move() for LocalMetadataStore, add
unit tests
Key: HADOOP-13631
URL: https://issues.apache.org/jira/browse/HADOOP-13631
Project: Hadoop
+1, non-binding.
I built everything on OS X and ran the s3a contract tests successfully:
mvn test -Dtest=org.apache.hadoop.fs.contract.s3a.\*
...
Results :
Tests run: 78, Failures: 0, Errors: 0, Skipped: 1
[INFO]
[INF
Aaron Fabbri created HADOOP-13573:
-
Summary: S3Guard: create basic contract tests for MetadataStore
implementations
Key: HADOOP-13573
URL: https://issues.apache.org/jira/browse/HADOOP-13573
Project
Aaron Fabbri created HADOOP-13559:
-
Summary: Remove close() within try-with-resources
Key: HADOOP-13559
URL: https://issues.apache.org/jira/browse/HADOOP-13559
Project: Hadoop Common
Issue
Aaron Fabbri created HADOOP-13476:
-
Summary: CredentialProviderFactory fails at class loading from
libhdfs (JNI)
Key: HADOOP-13476
URL: https://issues.apache.org/jira/browse/HADOOP-13476
Project
On Tue, Aug 2, 2016 at 12:17 AM, Mr rty ff wrote:
>
> Hi I have few questions about implementation of inputstream in S3.
> 1)public synchronized long getPos() throws IOException
> {return (nextReadPos < 0) ? 0 : nextReadPos;}
> Why does it return nextReadPos not pos?
My understanding is:
seek(
Aaron Fabbri created HADOOP-13351:
-
Summary: TestDFSClientSocketSize buffer size tests are flaky
Key: HADOOP-13351
URL: https://issues.apache.org/jira/browse/HADOOP-13351
Project: Hadoop Common
Aaron Fabbri created HADOOP-13230:
-
Summary: s3a's use of fake empty directory blobs does not
interoperate with other s3 tools
Key: HADOOP-13230
URL: https://issues.apache.org/jira/browse/HADOOP-
Aaron Fabbri created HADOOP-12267:
-
Summary: s3a failure due to integer overflow bug in AWS SDK
Key: HADOOP-12267
URL: https://issues.apache.org/jira/browse/HADOOP-12267
Project: Hadoop Common
89 matches
Mail list logo
