Hi Fokko, I agree: let's move forward on 0.5.0 release and we can submit 0.5.1 very soon after 0.5.0.
Thanks ! Regards JB On Tue, Sep 12, 2023 at 7:15 PM Driesprong, Fokko <fo...@driesprong.frl> wrote: > > Hey everyone, > > After an issue on Github, I noticed a bug in PyIceberg that the filesystem > isn't being reused. I think there is more room for improvement (both in the > long and short term), but I don't think we should block the release on that > since 0.5.0 is already much faster due to improved Avro parsing, improved IO, > and the previously mentioned bugfix (and one that was merged earlier today). > > I'll cut another PR as soon as #8549 is in. Thanks everyone for the patience! > > Cheers, Fokko > > Op ma 11 sep 2023 om 14:22 schreef Fokko Driesprong <fo...@apache.org>: >> >> Hi Everyone, >> >> I propose that we release the following RC as the official PyIceberg 0.5.0 >> release. A summary of what's included in 0.5.0: >> >> Add gzip metadata support >> PyArrow HDFS support >> Support serverless environments (AWS Lambda) >> Many fixes around Avro performance (PRs 1, 2, 3, 4) >> Remove the upper bound of PyParsing dependency (blocking a PR in Airflow) >> Moving the reading of Avro to Cython (10x speed improvement(!)) >> Support for the SQLCatalog (JDBC in Java) >> Fix support for UUID columns >> Support for adding columns >> Optimize concurrency (follow up on the Support servless environments) >> Bump Pydantic to v2 (improved performance of the JSON (de)serialization) >> A lot of bugfixes! >> >> The commit ID is 3323281045a72f1156d58c261067469e383fb26d >> >> * This corresponds to the tag: pyiceberg-0.5.0rc2 >> (92600935834bdf77ba37ac361338712713549a77) >> * https://github.com/apache/iceberg/releases/tag/pyiceberg-0.5.0rc2 >> * >> https://github.com/apache/iceberg/tree/3323281045a72f1156d58c261067469e383fb26d >> >> The release tarball, signature, and checksums are here: >> >> * https://dist.apache.org/repos/dist/dev/iceberg/pyiceberg-0.5.0rc2/ >> >> You can find the KEYS file here: >> >> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS >> >> Convenience binary artifacts are staged on pypi: >> >> https://pypi.org/project/pyiceberg/0.5.0rc2/ >> >> And can be installed using: pip3 install pyiceberg==0.5.0rc2 >> >> Since a lot has changed due to the release of the wheels (binary Python >> libraries), I've included the following steps to verify the release: >> >> curl https://dist.apache.org/repos/dist/dev/iceberg/KEYS -o KEYS >> gpg --import KEYS >> >> svn checkout >> https://dist.apache.org/repos/dist/dev/iceberg/pyiceberg-0.5.0rc1/ >> /tmp/pyiceberg/ >> >> for name in $(ls /tmp/pyiceberg/pyiceberg-*.whl >> /tmp/pyiceberg/pyiceberg-*.tar.gz) >> do >> gpg --verify ${name}.asc ${name} >> done >> >> cd /tmp/pyiceberg/ >> for name in $(ls /tmp/pyiceberg/pyiceberg-*.whl.asc.sha512 >> /tmp/pyiceberg/pyiceberg-*.tar.gz.asc.sha512) >> do >> shasum -a 512 --check ${name} >> done >> >> tar xzf pyiceberg-0.5.0.tar.gz >> cd pyiceberg-0.5.0 >> >> ./dev/check-license >> >> Please download, verify, and test. >> >> Please vote in the next 72 hours. >> [ ] +1 Release this as PyIceberg 0.5.0 >> [ ] +0 >> [ ] -1 Do not release this because... >> >> Please consider this my +1, I've checked against the docker-spark-iceberg >> notebook, and did some checks. >> >> Kind regards, >> Fokko Driesprong >>
