Re: Feedback on Iceberg Materialized View Spec

This is one motivation case why I think it's useful to have a central place for improvement proposals. On Fri, Nov 10, 2023 at 21:17 Jan Kaul wrote: > Up until 2 weeks ago the discussion took place in the Github issue but > since then most people joined the discussion in the google doc. Since >

Re: Feedback on Iceberg Materialized View Spec

Up until 2 weeks ago the discussion took place in the Github issue but since then most people joined the discussion in the google doc. Since the google doc seems to have more visibility I would propose to continue the discussion there. I hope that's fine. Best wishes, Jan On 08.11.23 07:09,

Re: Feedback on Iceberg Materialized View Spec

Are there parallel discussions? So far I have been following/commenting on the issue [1], not the Google doc. Can we converge on the issue going forward? Jan, if there are parallel discussions on the doc, could you summarize the open topics from the doc in the issue? Else, we can close the open top

Re: Feedback on Iceberg Materialized View Spec

Hi, Jan: Thanks for the update. Left some comments and let's continue the discussion in doc. On Tue, Nov 7, 2023 at 4:26 PM Jan Kaul wrote: > Hi all, > > thanks again to those who left some comments on the Iceberg Materialized > View spec. The discussions showed that there are still some open q

Re: Feedback on Iceberg Materialized View Spec

Hi all, thanks again to those who left some comments on the Iceberg Materialized View spec. The discussions showed that there are still some open questions that need to be answered. I added a section to the end of the document to highlight the decisions that need to be made. Please have anoth

Re: Feedback on Iceberg Materialized View Spec

Thank you Dan and the others for your helpful comments. I've added some sections to address the points that you mentioned. I'm not really sure what you mean by fail after grace period. I've found a design document for the trino materialized views and tried to incorporate some of the points. I'

Re: Feedback on Iceberg Materialized View Spec

I added a few comments to the doc, but I think there a few other things that probably need to be considered: Do we define behaviors around freshness (fail after grace period or fall back to view definition), what is the expectation for refreshes (manual/just-in-time/lazy), etc. I think there was

Re: Feedback on Iceberg Materialized View Spec

Daniel is right, we deviated :) OK Brian, let's do that. Apologies. Regards JB On Thu, Oct 26, 2023 at 8:40 PM Brian Olsen wrote: > > Agreed, apologies to Jan :). JB, let's discuss this at the sync this Wed, and > after that we can create a new thread if needed. > > On Thu, Oct 26, 2023 at 1:

Re: Feedback on Iceberg Materialized View Spec

Agreed, apologies to Jan :). JB, let's discuss this at the sync this Wed, and after that we can create a new thread if needed. On Thu, Oct 26, 2023 at 1:38 PM Daniel Weeks wrote: > JB and Brian, > > I think we should probably move this discussion to a discuss thread > specifically for the topics

Re: Feedback on Iceberg Materialized View Spec

JB and Brian, I think we should probably move this discussion to a discuss thread specifically for the topics you want to address. We've had a few instances now where the original intent of the thread is redirected to talk about other subjects. I don't feel this is a good approach because, while

Re: Feedback on Iceberg Materialized View Spec

Oh, I don't say we have to provide a user mailing list. Personally, I like mailing list mainly because we have https://lists.apache.org/ where we can browse and search on the mailing lists. A lot of Apache projects are using Slack or Zulip, but in parallel of mailing lists. As we say at Apache: "if

Re: Feedback on Iceberg Materialized View Spec

Yeah, unfortunately there's no way to limit the functionality to only facilitate this. In fact, the product that gets closest to it is GitHub Issues. I believe putting the onus on developers deeply involved in the project makes sense. Expecting users, especially newer users of a newer generation w

Re: Feedback on Iceberg Materialized View Spec

The idea is really to "square" GH Discussion only to roadmap/design proposals. For "user support", more than Slack, I would love to see u...@iceberg.apache.org. So I would distinguish: - the design/spec proposals where we could use GH Discussions. If people use GH Discussion for support questions

Re: Feedback on Iceberg Materialized View Spec

GitHub Discussions could be a solution that we should consider. We used it on the Trino side but still have mixed results with it. On one hand, there's a lot of overlap between creating Issues and Discussions. In fact, GitHub allows you to migrate Issues that only involve discussing a topic, or som

Re: Feedback on Iceberg Materialized View Spec

Just to be clear: we can GH Discussions subjects template via .asf.yaml but we have to open a ticket to INFRA to enable it. Regards JB On Thu, Oct 26, 2023 at 1:56 PM Jean-Baptiste Onofré wrote: > > Hi Brian > > I like the idea of GitHub. Why not enabling (in .asf.yml) GitHub > discussions ? A G

Re: Feedback on Iceberg Materialized View Spec

Hi Brian I like the idea of GitHub. Why not enabling (in .asf.yml) GitHub discussions ? A GitHub Discussion could be a good place to share the doc and exchange both in the doc and in the discussion comments. Regards JB On Thu, Oct 26, 2023 at 1:13 PM Brian Olsen wrote: > > Hey JB, > > I totally

Re: Feedback on Iceberg Materialized View Spec

Hey JB, I totally agree we need a place to centralize this but I'm nit a huge fan of all the lists we currently have going on the site. SSGs are just not an accessible method of storing lists. ( roadmap, blogs, videos, etc..). The roadmap is barely touched for this reason. I want to propose we mo

Re: Feedback on Iceberg Materialized View Spec

Hi, JB: Thanks for replying. I still want to hear what the community thinks, and have a discussion about the format. On Tue, Oct 24, 2023 at 10:27 PM Jean-Baptiste Onofré wrote: > Hi Jan > > Thanks for the reminder. I will take a look. > > As proposed by Renjie a few days ago, it would be grea

Re: Feedback on Iceberg Materialized View Spec

Hi, Jan: Thanks for the proposal. I’ve scanned the doc and left some comments. There are many interesting topics about materialized view, such as incremental mv maintenance, view matching problem. It would be great if the iceberg can support materialized view definition. On Tue, Oct 24, 2023

Re: Feedback on Iceberg Materialized View Spec

Hi Jan Thanks for the reminder. I will take a look. As proposed by Renjie a few days ago, it would be great to gather/store all document proposals in a central place. If there are no objections, I will prepare a PR for the website about that (with a space listing/linking all proposals). Regards

Feedback on Iceberg Materialized View Spec

Hi all, I've created an issue to propose a design for a Materialized View Spec a while ago. After further discussion we reached a first draft for the spec. It would be great if you could have another look at the design and share your feedback.