Re: zstd compression

2017-08-24 Thread Andrew Wang
Happily, the zstd maintainer responded to my issue, and it's now dual licensed with BSD and GPLv2. We need to upgrade to zstd 1.3.1 to pull this in. I filed this JIRA for tracking: https://issues.apache.org/jira/browse/HADOOP-14805 On Tue, Jul 25, 2017 at 10:27 AM, Andrew Wang wrote: > Thanks S

Re: zstd compression

2017-07-25 Thread Andrew Wang
Thanks Sean, Owen. I've opened an issue on their github here: https://github.com/facebook/zstd/issues/775 I figure it doesn't hurt to ask, particularly if they intend for zstd to be a replacement for the commonly-embedded zlib. On Tue, Jul 25, 2017 at 6:17 AM, Owen O'Malley wrote: > I'd suppor

Re: zstd compression

2017-07-25 Thread Owen O'Malley
I'd support asking Facebook to change it with both my hadoop and orc hats on. .. Owen > On Jul 24, 2017, at 23:43, Sean Busbey wrote: > > Nope. Once I found out HBase's use was compliant as an optional runtime > dependency I stopped looking. > >> On Jul 24, 2017 7:22 PM, "Andrew Wang" wrote:

Re: zstd compression

2017-07-24 Thread Sean Busbey
Nope. Once I found out HBase's use was compliant as an optional runtime dependency I stopped looking. On Jul 24, 2017 7:22 PM, "Andrew Wang" wrote: > I think it'd still be worth asking FB to relicense zstandard. Being able > to bundle it in the release would make it easier to use, since I doubt

Re: zstd compression

2017-07-24 Thread Andrew Wang
I think it'd still be worth asking FB to relicense zstandard. Being able to bundle it in the release would make it easier to use, since I doubt there are zstandard packages in the default OS repos. Sean, have you already filed an issue with zstandard? On Mon, Jul 17, 2017 at 1:30 PM, Jason Lowe

Re: zstd compression

2017-07-17 Thread Jason Lowe
I think we are OK to leave support for the zstd codec in the Hadoop code base. I asked Chris Mattman for clarification, noting that the support for the zstd codec requires the user to install the zstd headers and libraries and then configure it to be included in the native Hadoop build. The Ha

Re: zstd compression

2017-07-17 Thread Sean Busbey
I know that the HBase community is also looking at what to do about our inclusion of zstd. We've had it in releases since late 2016. My plan was to request that they relicense it. Perhaps the Hadoop PMC could join HBase in the request? On Sun, Jul 16, 2017 at 8:11 PM, Allen Wittenauer wrote: > >