Happily, the zstd maintainer responded to my issue, and it's now dual licensed with BSD and GPLv2. We need to upgrade to zstd 1.3.1 to pull this in. I filed this JIRA for tracking:
https://issues.apache.org/jira/browse/HADOOP-14805 On Tue, Jul 25, 2017 at 10:27 AM, Andrew Wang <andrew.w...@cloudera.com> wrote: > Thanks Sean, Owen. I've opened an issue on their github here: > > https://github.com/facebook/zstd/issues/775 > > I figure it doesn't hurt to ask, particularly if they intend for zstd to > be a replacement for the commonly-embedded zlib. > > On Tue, Jul 25, 2017 at 6:17 AM, Owen O'Malley <owen.omal...@gmail.com> > wrote: > >> I'd support asking Facebook to change it with both my hadoop and orc hats >> on. >> >> .. Owen >> >> > On Jul 24, 2017, at 23:43, Sean Busbey <bus...@cloudera.com> wrote: >> > >> > Nope. Once I found out HBase's use was compliant as an optional runtime >> > dependency I stopped looking. >> > >> >> On Jul 24, 2017 7:22 PM, "Andrew Wang" <andrew.w...@cloudera.com> >> wrote: >> >> >> >> I think it'd still be worth asking FB to relicense zstandard. Being >> able >> >> to bundle it in the release would make it easier to use, since I doubt >> >> there are zstandard packages in the default OS repos. >> >> >> >> Sean, have you already filed an issue with zstandard? >> >> >> >> On Mon, Jul 17, 2017 at 1:30 PM, Jason Lowe >> <jl...@yahoo-inc.com.invalid> >> >> wrote: >> >> >> >>> I think we are OK to leave support for the zstd codec in the Hadoop >> code >> >>> base. I asked Chris Mattman for clarification, noting that the >> support for >> >>> the zstd codec requires the user to install the zstd headers and >> libraries >> >>> and then configure it to be included in the native Hadoop build. The >> >>> Hadoop releases are not shipping any zstd code (e.g.: headers or >> libraries) >> >>> nor does it require zstd as a mandatory dependency. Here's what he >> said: >> >>> >> >>> >> >>> On Monday, July 17, 2017 11:07 AM, Chris Mattmann < >> mattm...@apache.org> >> >>> wrote: >> >>> >> >>>> Hi Jason, >> >>>> >> >>>> This sounds like an optional dependency on a Cat-X software. This >> isn’t >> >>> the only type of compression >> >>>> that is allowed within Hadoop, correct? If it is truly optional and >> you >> >>> have gone to that level of detail >> >>>> below to make the user opt in, and if we are not shipping zstd with >> our >> >>> products (source code releases), >> >>>> then this is an acceptable usage. >> >>>> >> >>>> Cheers, >> >>>> Chris >> >>> >> >>> >> >>> So I think we are in the clear with respect to zstd usage as long as >> we >> >>> keep it as an optional codec where the user needs to get the headers >> and >> >>> libraries for zstd and configure it into the native Hadoop build. >> >>> >> >>> Jason >> >>> >> >>> On Monday, July 17, 2017 9:44 AM, Sean Busbey <bus...@cloudera.com> >> >>> wrote: >> >>> >> >>> >> >>> >> >>> I know that the HBase community is also looking at what to do about >> >>> >> >>> our inclusion of zstd. We've had it in releases since late 2016. My >> >>> >> >>> plan was to request that they relicense it. >> >>> >> >>> >> >>> Perhaps the Hadoop PMC could join HBase in the request? >> >>> >> >>> >> >>> On Sun, Jul 16, 2017 at 8:11 PM, Allen Wittenauer >> >>> >> >>> <a...@effectivemachines.com> wrote: >> >>> >> >>>> >> >>> >> >>>> It looks like HADOOP-13578 added Facebook's zstd compression >> >>> codec. Unfortunately, that codec is using the same 3-clause BSD >> (LICENSE >> >>> file) + patent grant license (PATENTS file) that React is using and >> RocksDB >> >>> was using. >> >>> >> >>>> >> >>> >> >>>> Should that code get reverted? >> >>> >> >>>> >> >>> >> >>>> >> >>> >> >>>> >> >>> >> >>>> ------------------------------------------------------------ >> --------- >> >>> >> >>>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org >> >>> >> >>>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org >> >>> >> >>>> >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> >> >>> busbey >> >>> >> >>> >> >>> --------------------------------------------------------------------- >> >>> >> >>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org >> >>> >> >>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org >> >>> >> >>> --------------------------------------------------------------------- >> >>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org >> >>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org >> >>> >> >>> >> >> >> > >
