Re: [VOTE] Release Apache Iceberg 1.3.0 RC0

2023-05-28 Thread Jean-Baptiste Onofré
+1 (non binding) Regards JB On Tue, May 23, 2023 at 10:21 PM Anton Okolnychyi wrote: > > Hi Everyone, > > I propose that we release the following RC as the official Apache Iceberg > 1.3.0 release. > > The commit ID is 7dbdfd33a667a721fbb21c7c7d06fec9daa30b88 > * This corresponds to the tag: apa

Re: 👋 Intro and question for the community

2023-06-06 Thread Jean-Baptiste Onofré
Hi Brian, Can you please describe a bit what you mean by Common Room ? At first glance, it looks like a good idea. However, from Apache standpoint, it has to be approved by the PMC members. Did you request so on the private mailing list ? Regards JB On Thu, May 18, 2023 at 11:44 PM Brian Olsen

Re: 👋 Intro and question for the community

2023-06-06 Thread Jean-Baptiste Onofré
a formal vote around adding this capability if any of them feel that is > necessary. > > On Tue, Jun 6, 2023 at 4:30 AM Jean-Baptiste Onofré wrote: >> >> Hi Brian, >> >> Can you please describe a bit what you mean by Common Room ? >> >> At first

Re: [DISCUSS] June board report

2023-06-15 Thread Jean-Baptiste Onofré
Hi Ryan, It looks good to me. Thanks ! NB: maybe for the next report, we can provide some inputs about community activity (talks/meetups at CommunityOverCode conf, community events, ...). Thanks, Regards JB On Wed, Jun 14, 2023 at 3:51 AM Ryan Blue wrote: > > Hi everyone, > > Here’s our draft

Re: [DISCUSS] June board report

2023-06-19 Thread Jean-Baptiste Onofré
, 2023 at 6:26 PM Ryan Blue wrote: > > Thanks JB. Do you have any community activity to highlight this time? > > On Thu, Jun 15, 2023 at 9:10 AM Jean-Baptiste Onofré > wrote: >> >> Hi Ryan, >> >> It looks good to me. Thanks ! >> >> NB: maybe fo

Re: [VOTE] Release PyIceberg 0.4.0 RC1

2023-06-26 Thread Jean-Baptiste Onofré
+1 (non binding) Regards JB On Mon, Jun 26, 2023 at 11:27 AM Fokko Driesprong wrote: > > Hi Everyone, > > > Excited to start the 0.4.0 PyIceberg release process. The 0.4.0 release is > packed with cool features: > > Support for converting Parquet schemas into Iceberg ones > Support for reading

Re: [VOTE] Release PyIceberg 0.4.0 RC2

2023-06-27 Thread Jean-Baptiste Onofré
+1 (non binding) I did quick tests and it looks good. Thanks! Regards JB On Tue, Jun 27, 2023 at 10:37 PM Fokko Driesprong wrote: > > All, > > > Excited to start the 0.4.0 PyIceberg release process. The 0.4.0 release is > packed with awesome features: > > Support for converting Parquet schemas

[PROPOSAL] Preparing first Apache Iceberg Summit

2023-07-05 Thread Jean-Baptiste Onofré
Hi everyone, I started a discussion on the private mailing list, and, as there are no objections from the PMC members, I'm moving the thread to the dev mailing list. I propose to organize the first Apache Iceberg Summit \o/ For the format, I think the best option is a virtual event with a mix of

Re: [DISCUSS] Apache Iceberg Release 1.3.1

2023-07-06 Thread Jean-Baptiste Onofré
Hi, It sounds good to me to have 1.3.1. Thanks ! Regards JB On Fri, Jul 7, 2023 at 12:53 AM Szehon Ho wrote: > > Hi > > I wanted to start a discussion for whether its the right time for 1.3.1, a > patch release of 1.3.0. It was started based on the issue found by Xiangyang > (@ConeyLiu) : >

Re: [VOTE] Release Apache Iceberg 1.3.1 RC1

2023-07-19 Thread Jean-Baptiste Onofré
+1 (non binding) I checked: - hash & signature are OK - LICENSE, NOTICE look good - maven-rat-plugin checks are OK - build worked Thanks ! Regards JB On Mon, Jul 17, 2023 at 8:00 PM Szehon Ho wrote: > > Hi Everyone, > > I propose that we release the following RC as the official Apache Iceberg

Re: [PROPOSAL] Preparing first Apache Iceberg Summit

2023-07-19 Thread Jean-Baptiste Onofré
lp in the organization and participate to the committees, please let me know. I would like to schedule a meeting with all interested parties. Thanks ! Regards JB On Wed, Jul 5, 2023 at 4:37 PM Jean-Baptiste Onofré wrote: > > Hi everyone, > > I started a discussion on the private mail

Re: [PROPOSAL] Preparing first Apache Iceberg Summit

2023-07-19 Thread Jean-Baptiste Onofré
Jul 19, 2023 at 8:54 AM Russell Spitzer > wrote: >> >> I would love to be involved if possible. I'm a bit short on time though but >> can definitely contribute async time to planning. >> >> On Wed, Jul 19, 2023 at 9:35 AM Jean-Baptiste Onofré >> wrote:

Re: [PROPOSAL] Preparing first Apache Iceberg Summit

2023-07-24 Thread Jean-Baptiste Onofré
>> >>> On Wed, Jul 19, 2023 at 3:00 PM Jay Dave wrote: >>>> >>>> Hello JB: >>>> >>>> I am interested and help in whatever I can. >>>> >>>> Thanks >>>> JD >>>> >

Re: Thoughts on EOL Strategy for Our Releases?

2023-07-25 Thread Jean-Baptiste Onofré
Hi, At Apache, a strong EOL/LTS policy doesn't really exist: anyone can cut a new release on a very old branch as soon as it's voted by at least three binding votes. That said, a lot of Apache projects have EOL/LTS policy defined in the project: - for instance Apache Camel has LTS branches (https

Re: Thoughts on EOL Strategy for Our Releases?

2023-08-01 Thread Jean-Baptiste Onofré
gt; > JB, we do have multiple branches and release older versions from time to time > if there is a serious enough bug to warrant a patch release. > > Ryan > > On Tue, Jul 25, 2023 at 10:37 PM Jean-Baptiste Onofré > wrote: >> >> Hi, >> >> At Apache

Re: [PROPOSAL] Preparing first Apache Iceberg Summit

2023-08-02 Thread Jean-Baptiste Onofré
time. As soon as I have the first proposal completed, I will send an invite to all interested parties. Thanks ! Regards JB On Mon, Jul 24, 2023 at 3:26 PM Jean-Baptiste Onofré wrote: > > Ack. Thanks again all for your interest and willingness to help! > > I will schedule a first pr

Re: [PROPOSAL] Preparing first Apache Iceberg Summit

2023-08-03 Thread Jean-Baptiste Onofré
Hi Chen, ACK, I will include you in the invite for the first meeting. Thanks ! Regards JB On Wed, Aug 2, 2023 at 8:31 PM Chen Qin wrote: > > Love to get involved and present our large scale CDC work. > > Chen > > On Wed, Aug 2, 2023 at 7:00 AM Jean-Baptiste Onofré wr

Re: [PROPOSAL] Preparing first Apache Iceberg Summit

2023-08-20 Thread Jean-Baptiste Onofré
Hi guys, I'm back from vacation and I'm resuming the work on the Iceberg Summit proposal doc. I will share the doc asap. Regards JB On Wed, Jul 5, 2023 at 4:37 PM Jean-Baptiste Onofré wrote: > > Hi everyone, > > I started a discussion on the private mailing list,

Re: [PROPOSAL] Preparing first Apache Iceberg Summit

2023-08-23 Thread Jean-Baptiste Onofré
is summit >> when it happens :) >> >> On Mon, Aug 21, 2023 at 1:38 AM Jean-Baptiste Onofré >> wrote: >> >>> Hi guys, >>> >>> I'm back from vacation and I'm resuming the work on the Iceberg Summit >>> proposal doc. I will

Re: two proposed spec changes

2023-08-28 Thread Jean-Baptiste Onofré
Hi Jacob I agree with 1, it makes sense to use nanosecond precision. IMHO it should be available for both timestamp and timestamptz. For 2, I'm not sure. Let's see what the others think. Regards JB On Wed, Aug 23, 2023 at 10:17 PM Jacob Marble wrote: > > Good afternoon, > > I would like to pro

Re: September board report

2023-09-07 Thread Jean-Baptiste Onofré
Hi Ryan It looks good to me. About the conference (summit/meetup), I will add as soon as we have concrete plans (still working on the Summit proposal doc). Thanks ! Regards JB On Thu, Sep 7, 2023 at 6:30 PM Ryan Blue wrote: > > Hi everyone, > > Here’s my draft for the September Iceberg board r

Re: [VOTE] Release Apache PyIceberg 0.5.0

2023-09-13 Thread Jean-Baptiste Onofré
Hi Fokko, I agree: let's move forward on 0.5.0 release and we can submit 0.5.1 very soon after 0.5.0. Thanks ! Regards JB On Tue, Sep 12, 2023 at 7:15 PM Driesprong, Fokko wrote: > > Hey everyone, > > After an issue on Github, I noticed a bug in PyIceberg that the filesystem > isn't being reus

Re: [DISCUSS] Include Spark 3.5 support in 1.4?

2023-09-13 Thread Jean-Baptiste Onofré
Hi Anton, I think we can increase our release pace (I'm volunteer to deal with releases if it helps). For 1.4, I'm working on removing the deprecated API in AwsProperties (the PR should be there pretty soon). Imho, it would be great to have a regular release pace clearly defined on the website fo

Re: September board report

2023-09-13 Thread Jean-Baptiste Onofré
Hey JB, let me know if you need any help here. > > On Thu, Sep 7, 2023 at 11:01 PM Jean-Baptiste Onofré > wrote: >> >> Hi Ryan >> >> It looks good to me. About the conference (summit/meetup), I will add >> as soon as we have concrete plans (still wo

Re: [VOTE] Release Apache PyIceberg 0.5.0 RC3

2023-09-18 Thread Jean-Baptiste Onofré
+1 (non binding) quickly tested the "legal" part: - signatures/hash has been fixed thanks for that ! - ASF header looks ok - no binaries found in the pyiceberg distribution which is good Thanks ! Regards JB On Wed, Sep 13, 2023 at 2:18 PM Fokko Driesprong wrote: > > Hi Everyone, > > > I propose

Re: [DISCUSS] Spark 3.1 support?

2023-09-21 Thread Jean-Baptiste Onofré
Hi Anton, imho for 1.4.0, we can deprecate/remove Spark 3.1 support. As it's major release, we can remove old version support. Spark 3.1 users can still use Iceberg 1.3.x. That's why I proposed a LTS policy for our users, I will come with a proposal about that. Thanks ! Regards JB On Wed, Sep 2

Re: [DISCUSS] Deprecate Spark 3.2 support?

2023-09-21 Thread Jean-Baptiste Onofré
+1 Regards JB On Thu, Sep 21, 2023 at 12:01 AM Anton Okolnychyi wrote: > > Shall we consider deprecating our Spark 3.2 support? That Spark version is no > longer being maintained by the Spark community and is not under active > development in Iceberg. It was released in October, 2021 and passe

Re: [DISCUSS] Deprecate Spark 3.2 support?

2023-09-21 Thread Jean-Baptiste Onofré
Just to elaborate a bit :) - As Iceberg 1.4.0 is new "major" release, it's good time to deprecate/remove old version support (of Spark and other things) - Spark 3.2 users can still use previous Iceberg version - I will start a discussion about LTS policy with a clear "target" support for our users

Re: Proposal: Introduce deletion vector file to reduce write amplification

2023-09-21 Thread Jean-Baptiste Onofré
Hi Ryan, Thanks for the feedback. Unfortunately, I was not able to join the Iceberg community sync meeting yesterday, I promise I will join the next ones. I think the proposal is very interesting and also the discussion/comments in the document. I agree that some points should be discussed furthe

Re: [DISCUSS] Deprecate Spark 3.2 support?

2023-09-21 Thread Jean-Baptiste Onofré
ansition. Policy is a way of making decisions by algorithm and I don't > think we want to do that here. > > On Thu, Sep 21, 2023 at 1:48 AM Jean-Baptiste Onofré > wrote: >> >> Just to elaborate a bit :) >> >> - As Iceberg 1.4.0 is new "major" rele

Re: [DISCUSS] Deprecate Spark 3.2 support?

2023-09-21 Thread Jean-Baptiste Onofré
Thu, Sep 21, 2023 at 8:52 AM Jean-Baptiste Onofré > wrote: >> >> Hi Ryan, >> >> Yes it makes sense. The way we discuss and decide the Spark versions >> is totally fine. >> >> My proposal was more to clearly announce the Spark/Flink/Java/Python >&

Re: [PROPOSAL] Preparing first Apache Iceberg Summit

2023-09-22 Thread Jean-Baptiste Onofré
via comments. Thanks ! Regards JB On Wed, Aug 23, 2023 at 11:25 PM Jean-Baptiste Onofré wrote: > > It sounds great :) > > I will include a note in the proposal doc about that. > > Regards > JB > > Le mer. 23 août 2023 à 14:14, Brian Olsen a écrit : >> >&g

Re: [PROPOSAL] Preparing first Apache Iceberg Summit

2023-09-26 Thread Jean-Baptiste Onofré
sclose that I'm also associated with a vendor (Tabular). > With that hat on, I think we'd happily sponsor an event like this. But with > my community hat on I want to make sure we plan it out and think carefully. > > Ryan > > On Fri, Sep 22, 2023 at 2:09 AM Jean-Baptis

Re: [PROPOSAL] Preparing first Apache Iceberg Summit

2023-09-26 Thread Jean-Baptiste Onofré
that anyone interested in that topic > could add it to the doc? > > Ryan > > On Tue, Sep 26, 2023 at 12:26 PM Jean-Baptiste Onofré > wrote: > >> Hi Ryan, >> >> My bad: I understood in the previous discussion about the summit that >> we wanted something

Re: [PROPOSAL] Preparing first Apache Iceberg Summit

2023-09-27 Thread Jean-Baptiste Onofré
#x27;t clear is how to > make this happen. Are you planning on updating the proposal to address the > concerns I raised or do you want someone else to pick that up? > > Ryan > > On Tue, Sep 26, 2023 at 11:28 PM Jean-Baptiste Onofré > wrote: > >> I mean that anyone

Re: Proposal to fix the docs - this time it'll be different

2023-09-28 Thread Jean-Baptiste Onofré
Hi Brian Thanks for the update. I will take a look. Regards JB Le ven. 29 sept. 2023 à 07:05, Brian Olsen a écrit : > Hey All, > > I know it's been a while but the first phase of the docs refactor has > landed. I think it's at a decent point for everyone to take a look. To be > clear, this is

Re: [VOTE] Release Apache Iceberg 1.4.0 RC1

2023-09-28 Thread Jean-Baptiste Onofré
+1 (non binding) I checked: - signatures and hash are ok - asf headers are present - no binary in the source distribution - build is ok NB: I’m working on a set of use cases with different data sets but it’s not yet complete. I should have it for next release and be able to compare queries time a

[DISCUSSION] Rename master branch as main for the main repository

2023-09-29 Thread Jean-Baptiste Onofré
Hi guys, The Apache CoC (https://www.apache.org/foundation/policies/conduct) especially contains section 5 about the wording we use. Several Apache projects renamed the master branch to the main branch (Apache Karaf, ActiveMQ, Airflow, ...). As we already use main for go, rust, and python reposito

Re: Migration of PyIceberg to iceberg-python repository

2023-09-29 Thread Jean-Baptiste Onofré
Hi Fokko +1 to move PyIceberg to iceberg-python repo. Can we keep track of log history ? For me, it's not a blocker, we can move even if we lose the history though. Regards JB On Fri, Sep 29, 2023 at 1:25 PM Fokko Driesprong wrote: > > Hey everyone 👋 > > A while ago we discussed that Rust and G

Re: Migration of PyIceberg to iceberg-python repository

2023-09-29 Thread Jean-Baptiste Onofré
Awesome, it looks even better ;) Thanks ! Regards JB On Fri, Sep 29, 2023 at 2:31 PM Fokko Driesprong wrote: > > Hey Ajantha, > > That's a great suggestion. I've followed the steps and created a new PR here: > https://github.com/apache/iceberg-python/pull/3 > > The subdirectory-filter command m

Re: [VOTE] Release Apache Iceberg 1.4.0 RC1

2023-09-29 Thread Jean-Baptiste Onofré
to make > > > table recovery easier when needed. > > > - A new property that's set on the table, indicating zstd > > > compression. > > > - Changes in the exceptions when binding a transform to a column > > > type that is not allowed. &

Re: [VOTE] Release Apache Iceberg 1.4.0 RC2

2023-09-29 Thread Jean-Baptiste Onofré
+1 (non binding) As for RC1, I checked: - signature and hash are OK - ASF headers are there - source distribution doesn't contain binary - build is OK Thanks, Regards JB On Sat, Sep 30, 2023 at 1:25 AM Anton Okolnychyi wrote: > > Hi Everyone, > > I propose that we release the following RC as th

Re: [VOTE] Release Apache Iceberg 1.4.0 RC1

2023-09-30 Thread Jean-Baptiste Onofré
what else we missed. I'd start by looking at > TestSparkV2Filter and TestFilterPushdown classes. > > On 2023/09/30 04:56:07 Jean-Baptiste Onofré wrote: > > Thanks Anton, > > > > Do we have unit tests about filter pushdown ? Maybe worth to add > > something around

Re: Kafka Connect sink

2023-10-02 Thread Jean-Baptiste Onofré
Hi Bryan That’s a great news ! Thanks a lot for the proposal. I will take a look on the PR and existing connector. I’m sure the Iceberg community will be very happy to see this and we will able to add new features and improvements thanks to the community feedback. I would be more than happy to he

Re: [DISCUSSION] Rename master branch as main for the main repository

2023-10-02 Thread Jean-Baptiste Onofré
gt; >>>> +1 >>>> >>>> On 2023/10/02 20:12:37 Bryan Keller wrote: >>>> > Hearty +1 from me >>>> > >>>> > >>>> > >>>> > > On Sep 29, 2023, at 5:37 AM, Brian Olsen >>>> >

Re: Kafka Connect sink

2023-10-02 Thread Jean-Baptiste Onofré
> https://github.com/delta-io/kafka-delta-ingest/blob/main/doc/DESIGN.md > > I am not an expert in this ingestion domain and recently got started. > I hope someone will chime in and we will have detailed analysis over the > design. > > Looking forward to this feature. > >

Re: Kafka Connect sink

2023-10-03 Thread Jean-Baptiste Onofré
ingestion > solutions. > > -Bryan > > > On Oct 2, 2023, at 11:03 PM, Jean-Baptiste Onofré wrote: > > > > From my standpoint, Kafka Connect is interesting to also address > > processing logic without Spark or Flink runtime. Definitely > > interestin

Re: [DISCUSSION] Rename master branch as main for the main repository

2023-10-04 Thread Jean-Baptiste Onofré
1:09 PM Fokko Driesprong wrote: >>> >>> Big +1! >>> >>> Thanks for raising this JB! >>> >>> Kind regards, >>> Fokko >>> >>> Op di 3 okt 2023 om 07:56 schreef Jean-Baptiste Onofré : >>>> >>>> Than

Re: [DISCUSSION] Rename master branch as main for the main repository

2023-10-05 Thread Jean-Baptiste Onofré
at 7:32 AM Jean-Baptiste Onofré wrote: > > Thanks again for your feedback. As we have a consensus, I'm moving forward: > > 1. I will create a PR to update resources to use main instead of > master (mainly the .github/workflows/* files) > 2. I will do a pass on the website/

[PROPOSAL] Regular release pace & some post release actions

2023-10-06 Thread Jean-Baptiste Onofré
Hi guys, I would like to propose some improvements on our release. 1. Predictable & regular release pace We started this discussion quickly on the 1.4.0 vote thread: I think it would be interesting for the community (both our users and also companies leveraging Iceberg in their products) to have

Re: [PROPOSAL] Regular release pace & some post release actions

2023-10-06 Thread Jean-Baptiste Onofré
> 2.3. Yes, we do this periodically. We also need to clean up tags. > 2.4. Go for it. > > As for the release guide, anyone is welcome to submit a pull request and we'd > love to have you contributing. > > Ryan > > On Fri, Oct 6, 2023 at 2:00 AM Jean-Baptiste Onofré

Re: [PROPOSAL] Regular release pace & some post release actions

2023-10-07 Thread Jean-Baptiste Onofré
a few Apache projects do that (Karaf, Camel, ActiveMQ, Subversion, ...), I like this to give "vision" to the community :) Regards JB On Sat, Oct 7, 2023 at 6:59 AM Jean-Baptiste Onofré wrote: > > Hi Ryan, > > For the pace, yes, it's what I saw with the previous rele

Re: [PROPOSAL] Regular release pace & some post release actions

2023-10-07 Thread Jean-Baptiste Onofré
se should be more frequent to fix bugs. > > On Sat, Oct 7, 2023 at 3:20 PM Jean-Baptiste Onofré > wrote: > >> Just to be concrete about "regular & predictable releases pace", the >> proposal is to have one line on https://iceberg.apache.org/releases/ >> l

Re: [PROPOSAL] Regular release pace & some post release actions

2023-10-07 Thread Jean-Baptiste Onofré
ease, an > Iceberg release follows. I like to add *at least* because I see an uptake > in the activity and I think want to release as often as possible, without > introducing too much pressure on testing. > > Cheers, Fokko > > > > Op za 7 okt 2023 om 18:52 schreef Jean-Bap

Re: Migration of PyIceberg to iceberg-python repository

2023-10-09 Thread Jean-Baptiste Onofré
> +1 >>>>>> >>>>>> I checked the discussion thread, and one of the motivations for this >>>>>> separation was to avoid triggering unrelated CI jobs after each change. >>>>>> However, I wonder if it isn't (and will not

Re: [DISCUSSION] Rename master branch as main for the main repository

2023-10-12 Thread Jean-Baptiste Onofré
work with the team to merge it asap. Thanks ! Regards JB On Fri, Sep 29, 2023 at 2:35 PM Jean-Baptiste Onofré wrote: > > Hi guys, > > The Apache CoC (https://www.apache.org/foundation/policies/conduct) > especially contains section 5 about the wording we use. Several Apache >

Re: [DISCUSSION] Rename master branch as main for the main repository

2023-10-12 Thread Jean-Baptiste Onofré
By the way, don't forget to update your local git repo: git fetch --all git checkout main git branch -D master And you are good to go :) Regards JB On Thu, Oct 12, 2023 at 3:37 PM Jean-Baptiste Onofré wrote: > > Hi guys > > I'm pleased to announce that the master bran

Re: [DISCUSSION] Rename master branch as main for the main repository

2023-10-12 Thread Jean-Baptiste Onofré
If you have any issue (with your PR, local repo, etc), please don't hesitate to contact me. Thanks ! Regards JB On Thu, Oct 12, 2023 at 3:41 PM Jean-Baptiste Onofré wrote: > > By the way, don't forget to update your local git repo: > > git fetch --all > git checkout ma

Re: [DISCUSSION] Rename master branch as main for the main repository

2023-10-12 Thread Jean-Baptiste Onofré
2023 at 8:42 AM Jean-Baptiste Onofré > wrote: >> >> By the way, don't forget to update your local git repo: >> >> git fetch --all >> git checkout main >> git branch -D master >> >> And you are good to go :) >> >> Regards >&

Community Meeting Minutes ?

2023-10-12 Thread Jean-Baptiste Onofré
Hi guys, Thanks for the community meeting yesterday, it was super interesting and motivating :) As we say at Apache: "If it didn't happen on the mailing list, it never happened" :) In order to give a chance to anyone in the community to see the topics and participate, it would be great to share t

Re: [DISCUSSION] Should we have a place to host improvement proposals?

2023-10-13 Thread Jean-Baptiste Onofré
Hi Renjie, I like the idea. I would propose to rename https://iceberg.apache.org/roadmap/ as https://iceberg.apache.org/technology/ and put the proposal here with the current status and eventually linking PR. I propose to rework the roadmap page as it doesn't seem up to date and should contain the

Re: Proposal: Introduce deletion vector file to reduce write amplification

2023-10-13 Thread Jean-Baptiste Onofré
Is that a portable format? Do you wrap it at >> > > all in >> > > > >> the file to carry extra metadata? For example, the proposal says >> > > > >> that >> > > a >> > > > >> starting position for a bitmap

[HEADS UP] Apache Iceberg DOAP file registered and update on the release process

2023-10-13 Thread Jean-Baptiste Onofré
Hi guys, We added Apache Iceberg DOAP file on the repo: https://github.com/apache/iceberg/blob/main/doap.rdf I registered the DOAP file in the ASF comdev, and now we are all set: https://projects.apache.org/project.html?iceberg I will submit a PR for the website to update the release process b

Re: [DISCUSS] Apache Iceberg 1.4.1

2023-10-16 Thread Jean-Baptiste Onofré
Hi Eduard It sounds good to me. I have some dep upgrades I think it would be good to include (I'm working on testing/benchmark about that). Thanks to you, we already merged the most important. I'm doing a pass now. In terms of pure bug fix, I don't have anything in mind. Regards JB On Mon, Oct

Re: [VOTE] Release Apache PyIceberg 0.5.1 (RC1)

2023-10-16 Thread Jean-Baptiste Onofré
+1 (non binding) I checked: - hash and signature are good - source distribution is good - run a quick test locally Thanks, Regards JB On Mon, Oct 16, 2023 at 11:28 PM Fokko Driesprong wrote: > > Hi Everyone, > > > I propose that we release the following RC as the official PyIceberg 0.5.1 > rel

Re: [DISCUSSION] Should we have a place to host improvement proposals?

2023-10-17 Thread Jean-Baptiste Onofré
improvement proposals first. We can talk about the exact format if the > community is interested in it later. > > On Fri, Oct 13, 2023 at 4:52 PM Jean-Baptiste Onofré > wrote: >> >> Hi Renjie, >> >> I like the idea. I would propose to rename >> https:

Re: [VOTE] Release Apache Iceberg 1.4.1 RC0

2023-10-18 Thread Jean-Baptiste Onofré
+1 (non binding) I checked: * hashes and signatures are OK * I did quick tests using spark 3.5 I found the following issues that we should fix: * the source distribution contains two binary files (used for tests, empty-puffin-uncompressed.bin and sample-metric-data-uncompressed.bin). Binary files

Re: Kafka Connect sink

2023-10-18 Thread Jean-Baptiste Onofré
Hi Bryan, Any update on this thread ? Can I help somehow ? Thanks, Regards JB On Mon, Oct 2, 2023 at 7:39 PM Bryan Keller wrote: > > Hi all, > > We at Tabular would like to contribute our Kafka Connect Iceberg sink to the > Iceberg project. It would be great to give Iceberg users another optio

Re: [VOTE] Release Apache Iceberg 1.4.1 RC0

2023-10-18 Thread Jean-Baptiste Onofré
with JDK8 > * ran into one test failure, which is reported in > https://github.com/apache/iceberg/issues/8824, but this shouldn't block > the release > * tested with Trino in https://github.com/trinodb/trino/pull/19434 > > On Wed, Oct 18, 2023 at 3:15 PM Jean-Baptiste Onofré

Re: [VOTE] Release Apache Iceberg 1.4.1 RC0

2023-10-19 Thread Jean-Baptiste Onofré
;t block this > release (especially since it's non-binding). This release can proceed once > it garners enough +1 votes. My -1 vote is simply to highlight areas we > could improve in future releases. > > > On Thu, Oct 19, 2023, at 13:11, Jean-Baptiste Onofré wrote: > > H

Re: Kafka Connect sink

2023-10-19 Thread Jean-Baptiste Onofré
addressing comments in the > first PR then following up with more PRs after that one is merged, so stay > tuned for those. > > Thanks, > Bryan > > > > > On Oct 18, 2023, at 7:13 AM, Jean-Baptiste Onofré wrote: > > > > Hi Bryan, > > > > An

Re: Feedback on Iceberg Materialized View Spec

2023-10-24 Thread Jean-Baptiste Onofré
Hi Jan Thanks for the reminder. I will take a look. As proposed by Renjie a few days ago, it would be great to gather/store all document proposals in a central place. If there are no objections, I will prepare a PR for the website about that (with a space listing/linking all proposals). Regards

Re: Feedback on Iceberg Materialized View Spec

2023-10-26 Thread Jean-Baptiste Onofré
on. I want to propose we move > roadmap to GitHub projects. > > Likewise, I feel like somewhere on GitHub might be a better location for this > type of thing. > > Maybe posting these in GitHub issues and adding a proposal label? > > On Tue, Oct 24, 2023 at 9:28 AM Jean-Baptist

Re: Feedback on Iceberg Materialized View Spec

2023-10-26 Thread Jean-Baptiste Onofré
Just to be clear: we can GH Discussions subjects template via .asf.yaml but we have to open a ticket to INFRA to enable it. Regards JB On Thu, Oct 26, 2023 at 1:56 PM Jean-Baptiste Onofré wrote: > > Hi Brian > > I like the idea of GitHub. Why not enabling (in .asf.yml) GitHub >

[PROPOSAL] Improve dev/check-license

2023-10-26 Thread Jean-Baptiste Onofré
Hi guys, During the 1.4.1 vote, we identified some files without ASF headers, more specifically in hidden directories (like .baseline). These files have not been detected by dev/check-license script. The reason is because the script uses apache-rat via java -jar (the Apache RAT CLI), executing th

Re: Feedback on Iceberg Materialized View Spec

2023-10-26 Thread Jean-Baptiste Onofré
still comes back to the value given the cost for the community to > manage it. > > I know I have a lot of thoughts around this and its because I've been down > this road before, but perhaps there's a nuance I'm not seeing yet. > > On Thu, Oct 26, 2023 at 7:15 AM Jean-

Re: Feedback on Iceberg Materialized View Spec

2023-10-26 Thread Jean-Baptiste Onofré
tion from users is > lowering every barrier to entry as well as lowering time to that first hello > world dopamine hit. > > I'm middle millennial and even I find using email for discussion outside of > my mental model/preference but I also see the benefits. > > On Th

Re: Feedback on Iceberg Materialized View Spec

2023-10-26 Thread Jean-Baptiste Onofré
ce/level of engagement or buy-in. >> >> I'm not disagreeing with trying to improve how we communicate and track >> improvements/proposals/etc, but I think we should try to keep the thread on >> topic. >> >> Thanks, >> -Dan >> >> On Thu, Oct

Re: Community Meeting Minutes ?

2023-10-27 Thread Jean-Baptiste Onofré
run this over time. I will share out the last few meeting > minutes in their own threads now. > > On Thu, Oct 12, 2023 at 9:03 AM Jean-Baptiste Onofré > wrote: >> >> Hi guys, >> >> Thanks for the community meeting yesterday, it was super interesting >> a

Re: [VOTE] Release Apache PyIceberg 0.5.1 RC2

2023-10-27 Thread Jean-Baptiste Onofré
+1 (non binding) I checked: - hash and signatures are good - I will check NOTICE (copyright is 2022 and I think some deps are missing there), not release blocker - ASF headers are present - no binary file detected - very quick test Regards JB On Tue, Oct 24, 2023 at 8:48 PM Fokko Driesprong wro

Re: [PROPOSAL] Improve dev/check-license

2023-10-27 Thread Jean-Baptiste Onofré
By the way, as dev/check-license is also used in iceberg-python and iceberg-go repositories (iceberg-rust doesn't have it), maybe I can move forward on new rat release with the fix on hidden directories and update there as well. Regards JB On Thu, Oct 26, 2023 at 5:19 PM Jean-Baptiste O

Re: [PROPOSAL] Improve dev/check-license

2023-10-27 Thread Jean-Baptiste Onofré
27;s fast, correct and > well-maintained. > > Maybe worth take a look. > > On Fri, Oct 27, 2023, at 17:48, Jean-Baptiste Onofré wrote: > > By the way, as dev/check-license is also used in iceberg-python and > > iceberg-go repositories (iceberg-rust doesn't have it)

Re: [PROPOSAL] Improve dev/check-license

2023-10-27 Thread Jean-Baptiste Onofré
apache/iceberg-rust/blob/main/.licenserc.yaml > [3]: > https://github.com/apache/iceberg-rust/blob/94a1c5d7742bc3b2a9ac7c8da20711a5e2578b89/.github/workflows/ci.yml#L38C1-L39C51 > > On Fri, Oct 27, 2023, at 22:17, Jean-Baptiste Onofré wrote: > > Thanks for the heads up Xuanwo. >

Re: [PROPOSAL] Improve dev/check-license

2023-10-27 Thread Jean-Baptiste Onofré
of value > in moving the checks to gradle. That just means that we would need to use a > different framework across the implementations. If there's a way to run > license checks in CI that doesn't have the dot-file limitation, that seems > ideal to me. > > On Fri,

Re: [DISCUSS] Apache Iceberg 1.4.2

2023-10-28 Thread Jean-Baptiste Onofré
Hi Amogh It sounds good to me. I just saw that you already started the release vote. It's a bit short to discuss in this thread, especially during the weekend :) Since it's a regression, you did well to start the vote. We can always do a 1.4.3 including other fixes if needed. Thanks, Regards JB

Re: [VOTE] Release Apache Iceberg 1.4.2 RC0

2023-10-28 Thread Jean-Baptiste Onofré
+1 (non binding) I checked: - hash and signature are good - ASF headers are still missing on some files (.baseline, etc): it has been fixed on main but not cherry picked on 1.4.x branch - still puffin binary files in the source distribution (I will work on a fix about that) - build ok from source

Re: Iceberg Logo Fix and Iceberg Swag Shop

2023-11-01 Thread Jean-Baptiste Onofré
Hi Brian, Good catch. We need to get approval from the PMC, and notify ASF VP Brand Management (Mark Thomas) by sending a message to tradema...@apache.org. We can also be in touch with ASF comdev and marketing teams to help to update the logo and so. I can help with this, don't hesitate to ping

Re: [PROPOSAL] Use Microsoft Style Guide for documentation

2023-11-01 Thread Jean-Baptiste Onofré
Hi Brian I like the proposal, it sounds like a good way to "align" our documentation. Thanks ! Regards JB On Wed, Nov 1, 2023 at 8:20 AM Brian Olsen wrote: > > Hey Iceberg Nation, As I've gone through the Iceberg docs, I've noticed a lot > of inconsistencies with terminology, grammar, and styl

Re: PR: implementation of timestamp_ns and timestamptz_ns

2023-11-07 Thread Jean-Baptiste Onofré
Hi Jacob, Thanks for the update. I will take a look. Regards JB NB: sorry guys, I was offline since last Thursday as we have been hit by Ciaran storm. We still don't have electricity and damages at home. Things should be better in the coming days. On Tue, Nov 7, 2023 at 6:00 PM Jacob Marble wr

Re: Updating the Iceberg table architecture diagram

2023-11-09 Thread Jean-Baptiste Onofré
Hi I think we have to keep it "clear and simple" as possible. I would prefer to have one diagram per spec version (to be clear in the scope). So, I would rather keep the current diagram (working for v1) and add a new one v2 centric. It would be great to have a "side by side" presentation, somethi

Re: Kafka Connect sink

2023-11-09 Thread Jean-Baptiste Onofré
one is merged, so stay > tuned for those. > > Thanks, > Bryan > > > > > On Oct 18, 2023, at 7:13 AM, Jean-Baptiste Onofré wrote: > > > > Hi Bryan, > > > > Any update on this thread ? Can I help somehow ? > > > > Thanks, > >

Re: Ingestion Layer?

2023-11-09 Thread Jean-Baptiste Onofré
> TOP of mind. > > > > On Mon, Oct 2, 2023 at 11:03 PM Jean-Baptiste Onofré > wrote: > >> From my standpoint, Kafka Connect is interesting to also address >> processing logic without Spark or Flink runtime. Definitely >> interesting to have Kafka integration/p

Re: Kafka Connect sink

2023-11-13 Thread Jean-Baptiste Onofré
ve changes to a minimum during the submission process. I will have > the next PR out as soon as the first is merged. > > Bryan > > > On Nov 9, 2023, at 9:30 AM, Jean-Baptiste Onofré wrote: > > > > Hi Bryan > > > > I would like to follow up about kafka connect

[PROPOSAL] Apache Iceberg 1.4.3 release

2023-11-14 Thread Jean-Baptiste Onofré
Hi guys, Avro 1.11.3 has been released, fixing CVE-2023-39410. We already updated to Avro 1.11.3 on main. About CVE, we also already use guava 32.1.3, fixing CVE-2023-2976. As the Avro CVE is classified high (see https://nvd.nist.gov/vuln/detail/CVE-2023-39410), I propose to bump to Avro 1.11.3

Re: [PROPOSAL] Apache Iceberg 1.4.3 release

2023-11-19 Thread Jean-Baptiste Onofré
Hi As there's no objection, I will move forward and prepare the release to vote. I will keep you posted asap. Thanks, Regards JB On Wed, Nov 15, 2023 at 6:11 AM Jean-Baptiste Onofré wrote: > > Hi guys, > > Avro 1.11.3 has been released, fixing CVE-2023-39410. > We alr

Re: [PROPOSAL] Apache Iceberg 1.4.3 release

2023-11-20 Thread Jean-Baptiste Onofré
ards, >> Fokko Driesprong >> >> Op ma 20 nov 2023 om 08:16 schreef Jean-Baptiste Onofré : >>> >>> Hi >>> >>> As there's no objection, I will move forward and prepare the release to >>> vote. >>> >>> I will keep you po

Re: [PROPOSAL] Apache Iceberg 1.4.3 release

2023-11-21 Thread Jean-Baptiste Onofré
Hi We chatted about the 1.4.3 release with Ed. We have few PRs we want to include and as it’s Thanksgiving this week, I will submit the release to vote on Tuesday next week. Regards JB Le lun. 20 nov. 2023 à 17:24, Jean-Baptiste Onofré a écrit : > Thanks Fokko ! > > I'm on t

Re: [PROPOSAL] Apache Iceberg 1.4.3 release

2023-11-22 Thread Jean-Baptiste Onofré
erg, there's no urgency to release 1.4.3. We agreed to wait new fixes for 1.4.3 release. I'm still volunteering to cut the 1.4.3 patch release when ready (I did all the build checks on my machine :)), and I'm doing a pass on GH issues. Thanks ! Regards JB On Tue, Nov 21, 2023 at 8:49 

Re: [PROPOSAL] Apache Iceberg 1.4.3 release

2023-12-02 Thread Jean-Baptiste Onofré
oing a patch release. > > There was another request to backport an issue, I've created a PR: > https://github.com/apache/iceberg/pull/8969#issuecomment-1837286383 > > Kind regards, > Fokko > > Op wo 22 nov 2023 om 18:50 schreef Jean-Baptiste Onofré : >> >>

Re: [PROPOSAL] Apache Iceberg 1.4.3 release

2023-12-04 Thread Jean-Baptiste Onofré
ense to include > thisfix as well to the next patch release? > > Gabor > > On Sun, Dec 3, 2023 at 6:04 AM Jean-Baptiste Onofré wrote: >> >> Hi Fokko, >> >> I know we have some fixes on the fly. So, let me do a new pass on >> issues and backports and I

  1   2   3   4   5   >