I added a few comments to the doc, but I think there a few other
things that probably need to be considered:
Do we define behaviors around freshness (fail after grace period or
fall back to view definition), what is the expectation for refreshes
(manual/just-in-time/lazy), etc.
I think there was a fair amount of discussion around this when Trino
implemented materialized views, so it may be good to revisit those
discussions and make sure we're covering the major areas.
-Dan
On Thu, Oct 26, 2023 at 11:50 AM Jean-Baptiste Onofré
<j...@nanthrax.net> wrote:
Daniel is right, we deviated :)
OK Brian, let's do that.
Apologies.
Regards
JB
On Thu, Oct 26, 2023 at 8:40 PM Brian Olsen
<bitsondata...@gmail.com> wrote:
>
> Agreed, apologies to Jan :). JB, let's discuss this at the sync
this Wed, and after that we can create a new thread if needed.
>
> On Thu, Oct 26, 2023 at 1:38 PM Daniel Weeks <dwe...@apache.org>
wrote:
>>
>> JB and Brian,
>>
>> I think we should probably move this discussion to a discuss
thread specifically for the topics you want to address.
>>
>> We've had a few instances now where the original intent of the
thread is redirected to talk about other subjects. I don't feel
this is a good approach because, while it is on the apache mailing
list, the topic of the thread doesn't reflect the content, so you
don't get the right audience/level of engagement or buy-in.
>>
>> I'm not disagreeing with trying to improve how we communicate
and track improvements/proposals/etc, but I think we should try to
keep the thread on topic.
>>
>> Thanks,
>> -Dan
>>
>> On Thu, Oct 26, 2023 at 9:26 AM Jean-Baptiste Onofré
<j...@nanthrax.net> wrote:
>>>
>>> Oh, I don't say we have to provide a user mailing list.
Personally, I
>>> like mailing list mainly because we have https://lists.apache.org/
>>> where we can browse and search on the mailing lists.
>>> A lot of Apache projects are using Slack or Zulip, but in
parallel of
>>> mailing lists. As we say at Apache: "if it doesn't happen on the
>>> mailing list, it never happens".
>>> That said I would distinguish:
>>> - for dev, obviously we can use Slack for discussion, community
>>> meetings, etc, but we have to send main topics/discussions on
the dev
>>> mailing list.
>>> - for user, I think Slack is good, but I like the user mailing
list,
>>> to track/search/async communication as well.
>>>
>>> That's another discussion anyway, let's focus on the design
proposals
>>> space: my understanding is that we want to have a space
listing all
>>> proposals, for review, tagged as "done" or "in progress". Right ?
>>> I don't think a forum/stack overflow like would help here (it
helps
>>> for users, not for dev/technical/design proposals).
>>>
>>> At Apache Beam, we have a similar page as at Iceberg:
>>> https://beam.apache.org/roadmap/ where you can click on
roadmap items
>>> for details (https://beam.apache.org/roadmap/portability/).
>>> So, initially, I proposed to update
>>> https://iceberg.apache.org/roadmap/ with proposals (status
>>> "discussion"). As most of the proposals (all ?) come as
Google Link,
>>> we can change a bit the look'n feel of this page including the
list of
>>> proposals.
>>>
>>> That could be a first move, we can update later.
>>>
>>> Regards
>>> JB
>>>
>>> On Thu, Oct 26, 2023 at 5:54 PM Brian Olsen
<bitsondata...@gmail.com> wrote:
>>> >
>>> > Yeah, unfortunately there's no way to limit the
functionality to only facilitate this. In fact, the product that
gets closest to it is GitHub Issues.
>>> >
>>> > I believe putting the onus on developers deeply involved in
the project makes sense. Expecting users, especially newer users
of a newer generation will use an email list is unlikely,
especially if they're in a discovery mode and figuring out how to
solve an issue. A lot of garnering adoption from users is lowering
every barrier to entry as well as lowering time to that first
hello world dopamine hit.
>>> >
>>> > I'm middle millennial and even I find using email for
discussion outside of my mental model/preference but I also see
the benefits.
>>> >
>>> > On Thu, Oct 26, 2023 at 10:45 AM Jean-Baptiste Onofré
<j...@nanthrax.net> wrote:
>>> >>
>>> >> The idea is really to "square" GH Discussion only to
roadmap/design proposals.
>>> >>
>>> >> For "user support", more than Slack, I would love to see
>>> >> u...@iceberg.apache.org.
>>> >>
>>> >> So I would distinguish:
>>> >> - the design/spec proposals where we could use GH
Discussions. If
>>> >> people use GH Discussion for support questions, then we can
move to GH
>>> >> Issue or direct to the mailing list/slack.
>>> >> - the user "support" should be on user mailing list and/or
Slack
>>> >>
>>> >> You have a valid point: GH Discussions could be hard to
manage because
>>> >> most users will use it as a "support forum".
>>> >>
>>> >> My point is really:
>>> >> - we need a central space for design/spec proposals
>>> >> - it has to be on Iceberg community and visible for all
>>> >>
>>> >> Regards
>>> >> JB
>>> >>
>>> >> On Thu, Oct 26, 2023 at 5:30 PM Brian Olsen
<bitsondata...@gmail.com> wrote:
>>> >> >
>>> >> > GitHub Discussions could be a solution that we should
consider. We used it on the Trino side but still have mixed
results with it. On one hand, there's a lot of overlap between
creating Issues and Discussions. In fact, GitHub allows you to
migrate Issues that only involve discussing a topic, or something
that can't immediately be tied to any upcoming work to be a
discussion. This keeps the Issue backlog focused on actionable
requests.
>>> >> >
>>> >> > That said, Discussions can become difficult to maintain
if no person or body of people drives it. Of course, the community
will drive it to some degree, especially when it's new and shiny,
but GitHub Discussions, much like Slack, becomes a support channel
that encourages the messy human interactions that help us arrive
at a solution. So the question is do we want to open Discussions
knowing that it may become a second support channel compared to
Slack? Would we want to use Discussions in place of Slack so that
there's still a single triage channel?
>>> >> >
>>> >> > I personally lean towards keeping a single real-time
"support-like" channel in the community, otherwise, you will
fragment the attention of the community. Most of what we would
need to support the centralization of proposals can be
accomplished with Issues. Slack still seems to be the dominant
interactive system of choice and where we are now so I wouldn't
suggest moving that. I do think this is worth a discussion at the
next sync so I'll add it.
>>> >> >
>>> >> > In full transparency, Tabular is building an
Iceberg-focused Discourse forum (not to be confused with Discord)
instance to solve the problem of centralizing discussions in the
community to wiki-style answers we can link to and having
dedicated content curators to those solutions. Think of it as an
Iceberg-specific Stack Overflow with lightened rules to allow more
open discussion. Adding GitHub discussions wouldn't collide with
our goals as it would become another signal that we could use to
inform the answers on our forum. It still comes back to the value
given the cost for the community to manage it.
>>> >> >
>>> >> > I know I have a lot of thoughts around this and its
because I've been down this road before, but perhaps there's a
nuance I'm not seeing yet.
>>> >> >
>>> >> > On Thu, Oct 26, 2023 at 7:15 AM Jean-Baptiste Onofré
<j...@nanthrax.net> wrote:
>>> >> >>
>>> >> >> Just to be clear: we can GH Discussions subjects
template via
>>> >> >> .asf.yaml but we have to open a ticket to INFRA to
enable it.
>>> >> >>
>>> >> >> Regards
>>> >> >> JB
>>> >> >>
>>> >> >> On Thu, Oct 26, 2023 at 1:56 PM Jean-Baptiste Onofré
<j...@nanthrax.net> wrote:
>>> >> >> >
>>> >> >> > Hi Brian
>>> >> >> >
>>> >> >> > I like the idea of GitHub. Why not enabling (in
.asf.yml) GitHub
>>> >> >> > discussions ? A GitHub Discussion could be a good
place to share the
>>> >> >> > doc and exchange both in the doc and in the discussion
comments.
>>> >> >> >
>>> >> >> > Regards
>>> >> >> > JB
>>> >> >> >
>>> >> >> > On Thu, Oct 26, 2023 at 1:13 PM Brian Olsen
<bitsondata...@gmail.com> wrote:
>>> >> >> > >
>>> >> >> > > Hey JB,
>>> >> >> > >
>>> >> >> > > I totally agree we need a place to centralize this
but I'm nit a huge fan of all the lists we currently have going on
the site. SSGs are just not an accessible method of storing lists.
( roadmap, blogs, videos, etc..).
>>> >> >> > >
>>> >> >> > > The roadmap is barely touched for this reason. I
want to propose we move roadmap to GitHub projects.
>>> >> >> > >
>>> >> >> > > Likewise, I feel like somewhere on GitHub might be a
better location for this type of thing.
>>> >> >> > >
>>> >> >> > > Maybe posting these in GitHub issues and adding a
proposal label?
>>> >> >> > >
>>> >> >> > > On Tue, Oct 24, 2023 at 9:28 AM Jean-Baptiste Onofré
<j...@nanthrax.net> wrote:
>>> >> >> > >>
>>> >> >> > >> Hi Jan
>>> >> >> > >>
>>> >> >> > >> Thanks for the reminder. I will take a look.
>>> >> >> > >>
>>> >> >> > >> As proposed by Renjie a few days ago, it would be
great to
>>> >> >> > >> gather/store all document proposals in a central place.
>>> >> >> > >>
>>> >> >> > >> If there are no objections, I will prepare a PR for
the website about
>>> >> >> > >> that (with a space listing/linking all proposals).
>>> >> >> > >>
>>> >> >> > >> Regards
>>> >> >> > >> JB
>>> >> >> > >>
>>> >> >> > >>
>>> >> >> > >>
>>> >> >> > >> On Tue, Oct 24, 2023 at 9:22 AM Jan Kaul
<jank...@mailbox.org.invalid> wrote:
>>> >> >> > >> >
>>> >> >> > >> > Hi all,
>>> >> >> > >> >
>>> >> >> > >> > I've created an issue to propose a design for a
Materialized View Spec a while ago. After further discussion we
reached a first draft for the spec. It would be great if you could
have another look at the design and share your feedback.
>>> >> >> > >> >
>>> >> >> > >> > Here is the google doc:
https://docs.google.com/document/d/1UnhldHhe3Grz8JBngwXPA6ZZord1xMedY5ukEhZYF-A/edit?usp=sharing
>>> >> >> > >> >
>>> >> >> > >> > Thanks in advance,
>>> >> >> > >> >
>>> >> >> > >> > Jan
