I added a few comments to the doc, but I think there a few other things that probably need to be considered:
Do we define behaviors around freshness (fail after grace period or fall back to view definition), what is the expectation for refreshes (manual/just-in-time/lazy), etc. I think there was a fair amount of discussion around this when Trino implemented materialized views, so it may be good to revisit those discussions and make sure we're covering the major areas. -Dan On Thu, Oct 26, 2023 at 11:50 AM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Daniel is right, we deviated :) > > OK Brian, let's do that. > > Apologies. > > Regards > JB > > On Thu, Oct 26, 2023 at 8:40 PM Brian Olsen <bitsondata...@gmail.com> > wrote: > > > > Agreed, apologies to Jan :). JB, let's discuss this at the sync this > Wed, and after that we can create a new thread if needed. > > > > On Thu, Oct 26, 2023 at 1:38 PM Daniel Weeks <dwe...@apache.org> wrote: > >> > >> JB and Brian, > >> > >> I think we should probably move this discussion to a discuss thread > specifically for the topics you want to address. > >> > >> We've had a few instances now where the original intent of the thread > is redirected to talk about other subjects. I don't feel this is a good > approach because, while it is on the apache mailing list, the topic of the > thread doesn't reflect the content, so you don't get the right > audience/level of engagement or buy-in. > >> > >> I'm not disagreeing with trying to improve how we communicate and track > improvements/proposals/etc, but I think we should try to keep the thread on > topic. > >> > >> Thanks, > >> -Dan > >> > >> On Thu, Oct 26, 2023 at 9:26 AM Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > >>> > >>> Oh, I don't say we have to provide a user mailing list. Personally, I > >>> like mailing list mainly because we have https://lists.apache.org/ > >>> where we can browse and search on the mailing lists. > >>> A lot of Apache projects are using Slack or Zulip, but in parallel of > >>> mailing lists. As we say at Apache: "if it doesn't happen on the > >>> mailing list, it never happens". > >>> That said I would distinguish: > >>> - for dev, obviously we can use Slack for discussion, community > >>> meetings, etc, but we have to send main topics/discussions on the dev > >>> mailing list. > >>> - for user, I think Slack is good, but I like the user mailing list, > >>> to track/search/async communication as well. > >>> > >>> That's another discussion anyway, let's focus on the design proposals > >>> space: my understanding is that we want to have a space listing all > >>> proposals, for review, tagged as "done" or "in progress". Right ? > >>> I don't think a forum/stack overflow like would help here (it helps > >>> for users, not for dev/technical/design proposals). > >>> > >>> At Apache Beam, we have a similar page as at Iceberg: > >>> https://beam.apache.org/roadmap/ where you can click on roadmap items > >>> for details (https://beam.apache.org/roadmap/portability/). > >>> So, initially, I proposed to update > >>> https://iceberg.apache.org/roadmap/ with proposals (status > >>> "discussion"). As most of the proposals (all ?) come as Google Link, > >>> we can change a bit the look'n feel of this page including the list of > >>> proposals. > >>> > >>> That could be a first move, we can update later. > >>> > >>> Regards > >>> JB > >>> > >>> On Thu, Oct 26, 2023 at 5:54 PM Brian Olsen <bitsondata...@gmail.com> > wrote: > >>> > > >>> > Yeah, unfortunately there's no way to limit the functionality to > only facilitate this. In fact, the product that gets closest to it is > GitHub Issues. > >>> > > >>> > I believe putting the onus on developers deeply involved in the > project makes sense. Expecting users, especially newer users of a newer > generation will use an email list is unlikely, especially if they're in a > discovery mode and figuring out how to solve an issue. A lot of garnering > adoption from users is lowering every barrier to entry as well as lowering > time to that first hello world dopamine hit. > >>> > > >>> > I'm middle millennial and even I find using email for discussion > outside of my mental model/preference but I also see the benefits. > >>> > > >>> > On Thu, Oct 26, 2023 at 10:45 AM Jean-Baptiste Onofré < > j...@nanthrax.net> wrote: > >>> >> > >>> >> The idea is really to "square" GH Discussion only to roadmap/design > proposals. > >>> >> > >>> >> For "user support", more than Slack, I would love to see > >>> >> u...@iceberg.apache.org. > >>> >> > >>> >> So I would distinguish: > >>> >> - the design/spec proposals where we could use GH Discussions. If > >>> >> people use GH Discussion for support questions, then we can move to > GH > >>> >> Issue or direct to the mailing list/slack. > >>> >> - the user "support" should be on user mailing list and/or Slack > >>> >> > >>> >> You have a valid point: GH Discussions could be hard to manage > because > >>> >> most users will use it as a "support forum". > >>> >> > >>> >> My point is really: > >>> >> - we need a central space for design/spec proposals > >>> >> - it has to be on Iceberg community and visible for all > >>> >> > >>> >> Regards > >>> >> JB > >>> >> > >>> >> On Thu, Oct 26, 2023 at 5:30 PM Brian Olsen < > bitsondata...@gmail.com> wrote: > >>> >> > > >>> >> > GitHub Discussions could be a solution that we should consider. > We used it on the Trino side but still have mixed results with it. On one > hand, there's a lot of overlap between creating Issues and Discussions. In > fact, GitHub allows you to migrate Issues that only involve discussing a > topic, or something that can't immediately be tied to any upcoming work to > be a discussion. This keeps the Issue backlog focused on actionable > requests. > >>> >> > > >>> >> > That said, Discussions can become difficult to maintain if no > person or body of people drives it. Of course, the community will drive it > to some degree, especially when it's new and shiny, but GitHub Discussions, > much like Slack, becomes a support channel that encourages the messy human > interactions that help us arrive at a solution. So the question is do we > want to open Discussions knowing that it may become a second support > channel compared to Slack? Would we want to use Discussions in place of > Slack so that there's still a single triage channel? > >>> >> > > >>> >> > I personally lean towards keeping a single real-time > "support-like" channel in the community, otherwise, you will fragment the > attention of the community. Most of what we would need to support the > centralization of proposals can be accomplished with Issues. Slack still > seems to be the dominant interactive system of choice and where we are now > so I wouldn't suggest moving that. I do think this is worth a discussion at > the next sync so I'll add it. > >>> >> > > >>> >> > In full transparency, Tabular is building an Iceberg-focused > Discourse forum (not to be confused with Discord) instance to solve the > problem of centralizing discussions in the community to wiki-style answers > we can link to and having dedicated content curators to those solutions. > Think of it as an Iceberg-specific Stack Overflow with lightened rules to > allow more open discussion. Adding GitHub discussions wouldn't collide with > our goals as it would become another signal that we could use to inform the > answers on our forum. It still comes back to the value given the cost for > the community to manage it. > >>> >> > > >>> >> > I know I have a lot of thoughts around this and its because I've > been down this road before, but perhaps there's a nuance I'm not seeing yet. > >>> >> > > >>> >> > On Thu, Oct 26, 2023 at 7:15 AM Jean-Baptiste Onofré < > j...@nanthrax.net> wrote: > >>> >> >> > >>> >> >> Just to be clear: we can GH Discussions subjects template via > >>> >> >> .asf.yaml but we have to open a ticket to INFRA to enable it. > >>> >> >> > >>> >> >> Regards > >>> >> >> JB > >>> >> >> > >>> >> >> On Thu, Oct 26, 2023 at 1:56 PM Jean-Baptiste Onofré < > j...@nanthrax.net> wrote: > >>> >> >> > > >>> >> >> > Hi Brian > >>> >> >> > > >>> >> >> > I like the idea of GitHub. Why not enabling (in .asf.yml) > GitHub > >>> >> >> > discussions ? A GitHub Discussion could be a good place to > share the > >>> >> >> > doc and exchange both in the doc and in the discussion > comments. > >>> >> >> > > >>> >> >> > Regards > >>> >> >> > JB > >>> >> >> > > >>> >> >> > On Thu, Oct 26, 2023 at 1:13 PM Brian Olsen < > bitsondata...@gmail.com> wrote: > >>> >> >> > > > >>> >> >> > > Hey JB, > >>> >> >> > > > >>> >> >> > > I totally agree we need a place to centralize this but I'm > nit a huge fan of all the lists we currently have going on the site. SSGs > are just not an accessible method of storing lists. ( roadmap, blogs, > videos, etc..). > >>> >> >> > > > >>> >> >> > > The roadmap is barely touched for this reason. I want to > propose we move roadmap to GitHub projects. > >>> >> >> > > > >>> >> >> > > Likewise, I feel like somewhere on GitHub might be a better > location for this type of thing. > >>> >> >> > > > >>> >> >> > > Maybe posting these in GitHub issues and adding a proposal > label? > >>> >> >> > > > >>> >> >> > > On Tue, Oct 24, 2023 at 9:28 AM Jean-Baptiste Onofré < > j...@nanthrax.net> wrote: > >>> >> >> > >> > >>> >> >> > >> Hi Jan > >>> >> >> > >> > >>> >> >> > >> Thanks for the reminder. I will take a look. > >>> >> >> > >> > >>> >> >> > >> As proposed by Renjie a few days ago, it would be great to > >>> >> >> > >> gather/store all document proposals in a central place. > >>> >> >> > >> > >>> >> >> > >> If there are no objections, I will prepare a PR for the > website about > >>> >> >> > >> that (with a space listing/linking all proposals). > >>> >> >> > >> > >>> >> >> > >> Regards > >>> >> >> > >> JB > >>> >> >> > >> > >>> >> >> > >> > >>> >> >> > >> > >>> >> >> > >> On Tue, Oct 24, 2023 at 9:22 AM Jan Kaul > <jank...@mailbox.org.invalid> wrote: > >>> >> >> > >> > > >>> >> >> > >> > Hi all, > >>> >> >> > >> > > >>> >> >> > >> > I've created an issue to propose a design for a > Materialized View Spec a while ago. After further discussion we reached a > first draft for the spec. It would be great if you could have another look > at the design and share your feedback. > >>> >> >> > >> > > >>> >> >> > >> > Here is the google doc: > https://docs.google.com/document/d/1UnhldHhe3Grz8JBngwXPA6ZZord1xMedY5ukEhZYF-A/edit?usp=sharing > >>> >> >> > >> > > >>> >> >> > >> > Thanks in advance, > >>> >> >> > >> > > >>> >> >> > >> > Jan >
