"missing"
current-snapshot-id as well?
I'd really like to hear from other implementers here since
changing all this again would be a lot of
work and I thought we had understanding of Optional == Nullable as
a valid thing in the spec.
On Tue, Feb 1
Also, as an idea, REST catalog services could return an error if those
deprecated properties are being set. Thoughts?
On 18.02.25 16:21, Robert Stupp wrote:
Agree with both Steve's. Personally, I'm okay with removing those
properties - but using the proposed phased approach.
On 1
removing these.
Kind regards,
Fokko
--
Robert Stupp
@snazy
Correcting myself: schema/spec/sort seem to be always present - please
ignore that part in my previous email. The valid values for those fields
however should be defined.
On 18.02.25 14:29, Robert Stupp wrote:
Reality is that Iceberg did write '-1' into current-snapshot-id (and
which is definitely wrong.
I'm not seeing why it would fall back to 0, and I agree, that's wrong.
Would be better IMHO not to break existing implementations /
render existing setups incompatible with Iceberg 1.8.
In my opinion, if this had been caught in an RC
agree that we should have communicated this more
clearly with the release.
Kind regards,
Fokko
Op ma 17 feb 2025 om 12:25 schreef Robert Stupp :
Feels like https://github.com/apache/iceberg/pull/11560 introduced
a behavior change.
snapshot-ID -1 isn't per-se invalid, beca
Central.
Thanks to everyone for contributing!
--
Robert Stupp
@snazy
Hi,
There is a breaking change in 1.8.0 that prevents Iceberg 1.8 clients to talk
to pre-1.8 Iceberg REST services.
The errors manifest as something like:
java.lang.UnsupportedOperationException: Server does not support endpoint: HEAD
/v1/{prefix}/namespaces/{namespace}/views/{view}
at
uot; of merged PRs (or should be
merged soon): default values, REST auth improvements, dependencies
updates, etc.
WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
and PRs we would like to "target" to 1.8.0.
Thoughts ?
Regards
JB
--
Robert Stupp
@snazy
> >>>>
> >>>> We'll leave this open up to Dec 10th to
give people time (as
> >>>> Thanksgiving is this week).
> >>>>
> >>>> Thanks !
> >>>> Regards
> >>>> JB
> >>>>
> >>>
> >>
> >> --
> >> Regards,
> >> Himadri Pal
> >>
> >
>
--
John Zhuge
--
Robert Stupp
@snazy
7;s not a problem to setup authentication.
On a separate note, we can also setup a shared gradle cache (something
like https://iceberg-cache.apache.org/cache/), but that's another
topic :)
Regards
JB
On Mon, Nov 25, 2024 at 12:58 PM Robert Stupp wrote:
It's tricky - have to u
d, including the actual
output of failed tests.
It's convenient to investigate test failures (in our GH Actions) and
Gradle plugins executions.
I'm proposing to export/store our build scans on ASF Develocity.
Thoughts ?
Regards
JB
--
Robert Stupp
@snazy
27;s no objections, the Iceberg PMC should approve the use of
Iceberg and the ASF VP M&P should be notified. I can help on the
paperwork and process again.
- the PMC will appoint two committees (at least selection and
sponsoring committees)
Thoughts ?
Regards
JB
--
Robert Stupp
@snazy
server
implementers and there's no behavioral change for existing
installations.
Please vote on the wording changes in the REST Spec in #10877
<https://github.com/apache/iceberg/pull/10877>.
The vote will remain open for at least 72 hours.
Thanks
Eduard
--
Robert Stupp
@snazy
I do object the plan to make that separation character configurable,
because there is no need to do this (see below), so there's no need for
any code change. Instead having a proper, mostly human readable escaping
mechanism, which is possible, should be implemented with Iceberg REST v2.
Here's
<https://datatracker.ietf.org/doc/html/rfc6570>, which is the
original way we wanted to represent namespaces, but it wasn't
supported (e.g. .../{+namespaces}/tables/{table}). I doubt
it's really worth the effort though, so I feel like a
configurable delimiter makes the most sense.
-Dan
--
Robert Stupp
@snazy
mespaces, but it wasn't supported (e.g.
.../{+namespaces}/tables/{table}). I doubt it's really worth the
effort though, so I feel like a configurable delimiter makes the most
sense.
-Dan
--
Robert Stupp
@snazy
al for guidelines on committing pull requests [1].
Feedback is appreciated.
Given the level of interest in the discussions so far, it seems that the best
path forward is to hold an official vote before merging. I intend to do this
once we appear to have consensus but if the people prefer we can try to avoid
the overhead.
Thanks,
Micah
[1] https://github.com/apache/iceberg/pull/10780
--
Ryan Blue
Databricks
--
Robert Stupp
@snazy
racter. Older clients are already broken, so if they don't
support the property sent by the config route there is no behavior
change.
Ryan
On Thu, Aug 1, 2024 at 9:47 AM Robert Stupp wrote:
How is compatibility with older servers guaranteed?
On 01.08.24 14:59,
because . . .
--
Ryan Blue
--
Robert Stupp
@snazy
the issue with *%1F* being used as the namespace separator.
Eduard
--
Robert Stupp
@snazy
+1 (nb)
On 26.07.24 18:42, Steve Zhang wrote:
+1 (non-binding)
Thanks,
Steve Zhang
On Jul 26, 2024, at 9:15 AM, Amogh Jahagirdar <2am...@gmail.com> wrote:
+1 (non-binding)
--
Robert Stupp
@snazy
me know what everyone thinks. I don't know if we need a
vote on this. I will go ahead to change the time directly if there is
a clear preference here, let's see.
Best,
Jack Ye
--
Robert Stupp
@snazy
+1 (nb) in general, and
+1 (nb) on dropping Java 8 in Iceberg 1.7
On 23.07.24 18:54, Steve Zhang wrote:
+1 (non-binding)
Thanks,
Steve Zhang
On Jul 22, 2024, at 10:13 PM, Ajantha Bhat wrote:
+1 (non-binding)
--
Robert Stupp
@snazy
Apache Iceberg 1.6.0
[ ] +0
[ ] -1 Do not release this because...
Only PMC members have binding votes, but other community members are
encouraged to cast non-binding votes. This vote will pass if there are
3 binding +1 votes and more binding +1 votes than -1 votes.
Thanks
Regards
JB
--
Robert
above. Users have to
re-configure their servers.
Eduard
On Mon, Jul 15, 2024 at 3:00 PM Robert Stupp wrote:
Sorry, I don't understand the two suggestions, especially when
used in combination. Current servers do not send a `capabilities`
field at all. You're suggesti
version 1). A server could send that property via the
config route without having clients to change anything.
On Mon, Jul 15, 2024 at 10:24 AM Robert Stupp wrote:
Hi,
I still have concerns regarding the missing table-spec/view-spec
capabilities. Newer clients can send create/updat
be
>>>> a) tables (version 1) only
>>>> b) the currently existing
functionality as defined in the
REST spec (as version 1)
>>>>
>>>>
>>>> On another note: Including
table-spec / view-spec seems to be
more informative in its nature as
I don't think a client would act
differently right now when seeing
these.
>>>>
>>>> Thanks
>>>> Eduard
>>>>
--
Ryan Blue
Databricks
--
Robert Stupp
@snazy
Apache Iceberg 1.6.0
[ ] +0
[ ] -1 Do not release this because...
Only PMC members have binding votes, but other community members are
encouraged to cast non-binding votes. This vote will pass if there are
3 binding +1 votes and more binding +1 votes than -1 votes.
Thanks,
Regards
JB
--
Robert
Thank you guys, after ~72h I'm closing the vote.
There were (if I counted correctly)
5 binding +1
and
8 non-binding +1
and no -1 or 0 votes.
The vote has passed.
Robert
On 08.07.24 18:15, Robert Stupp wrote:
Hi Everyone,
I propose that we merge PR to "Deprecate oa
om/apache/iceberg/blob/main/format/spec.md#table-metadata-fields> and
the implementation
<https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/TableMetadataParser.java#L111-L112>.
Please vote within the next 72 hours.
Eduard
--
Robert Stupp
@snazy
with JDK 21 passes locally but fails on CI
(where an older JDK is used).
Maybe it's sufficient to mention in the docs that *./gradlew
spotlessApply* needs to be executed locally with an older JDK.
Eduard
--
Robert Stupp
@snazy
spec that
duplicates the table spec and correcting a typo
<https://github.com/apache/iceberg/blob/main/format/spec.md#table-metadata-fields>,
it seems like more of a correction than a substantive change.
On Tue, Jul 9, 2024 at 3:14 AM Robert Stupp wrote:
Hi Eduard,
this needs t
https://github.com/apache/iceberg/pull/10662> to
fix property names for statistics / partition statistics in the REST
spec. I can start a separate VOTE thread if there is agreement around
the proposed Spec change.
Thanks
Eduard
--
Robert Stupp
@snazy
+1
On 08.07.24 18:15, Robert Stupp wrote:
Hi Everyone,
I propose that we merge PR to "Deprecate oauth/tokens endpoint".
The background and overall plan is discussed on this mailing list [2]
and this google doc [3].
Please vote in the next 72 hours.
Robert
[1] https://github.
[2] https://lists.apache.org/thread/twk84xx7v0xy5q5tfd9x5torgr82vv50 and
https://lists.apache.org/thread/wcm9ylm0nbwfrx65n8b1tpjrdhgvcx24 and
https://lists.apache.org/thread/qksh9j9d8h6nt6qrfl47bj76jthddb0p
[3]
https://docs.google.com/document/d/1Xi5MRk8WdBWFC3N_eSmVcrLhk3yu5nJ9x_wC0ec6kVQ
--
Robert Stupp
@snazy
ponse times and reduced load on the underlying storage systems.
Here is the proposal in google doc:
https://docs.google.com/document/d/1MVLwyMQtZ-7jewsQ0PuTvtJbpfl4HCoVdbowMqFTmfc/edit?usp=sharing
Estimated read time: 5 mins
Would really appreciate any feedback on this topic and proposal!
Yu
differently right now when seeing these.
Thanks
Eduard
--
Robert Stupp
@snazy
the service using
the specified authZ mechanism, it is considered trusted. It is
intentionally not an easy process to onboard. I don't know how other
catalog vendors do this, or have similar concepts.
-Jack
On Tue, Jul 2, 2024 at 4:00 AM Robert Stupp wrote:
Just some thoughts abo
n another note: Including *table-spec / view-spec* seems to be more
informative in its nature as I don't think a client would act
differently right now when seeing these.
Thanks
Eduard
--
Robert Stupp
@snazy
differently right now when seeing these.
Thanks
Eduard
--
Robert Stupp
@snazy
However, we believe that *we should at least offer
the right
> > interfaces and set the right standards for how Iceberg catalog
expresses
> > securable objects and how Iceberg catalog users interact with
those objects*,
> > such that (1) users that would like to build IRC can have a
clear guideline
> > of what API constract to implement for managing access to
objects in IRC,
> > and (2) users that are on one IRC product do not need to be
locked-in due
> > to access management aspects.
> >
> > Would really appreciate any feedback on this topic and proposal!
> >
> > Best,
> > Jack Ye
> >
>
--
Robert Stupp
@snazy
ndpoint is
removed from the spec) was raised. The path proposed in the
improvement-doc is that clients can and should configure the currently
existing endpoint explicitly. If a service wants the current
tokens-endpoint, it can still be used.
On 28.06.24 20:44, Robert Stupp wrote:
It'
9:02 AM Daniel Weeks
wrote:
>>
>> I think Robert's approach is a reasonable
compromise here.
>>
>> If we wanted a "per operation/endpoint"
let me know and create an issue on GitHub with the "Iceberg
1.6.0" milestone.
Thanks !
Regards
JB
--
Robert Stupp
@snazy
thub.com/apache/iceberg/pull/10602
--
Robert Stupp
@snazy
It's been a while since the discussion started. There were no objections
to the proposal, so I went ahead and started to implement "M1" of the
document [1].
[1] PR for M1: https://github.com/apache/iceberg/pull/10603
Robert
On 19.06.24 15:11, Robert Stupp wrote:
Alr
h is a reasonable compromise here.
>>
>> If we wanted a "per operation/endpoint" versioning, I think
I'd prefer Micah's OpenAPI spec based approach because it's
more standardized, but I feel adds a lot of client complexity.
On 27.06.24 19:05, Micah Kornfield wrote:
Maybe it pays to prototype the individual end point approach to
demonstrate its relative complexity?
The math is pretty simple: you need to duplicate all endpoints, all
request/response schema types, all tests, duplicate and/or adopt client
code. No
view is how we want to handle
capabilities and backwards compatibility
and what the default capability would be,
since older servers don't know anything
about *capabilities* (in such a case we
could assume that the default capabilities
would be *oauth2* / *tables*).
Are there any other capabilities that we'd
like to include in the list?
Eduard
--
Ryan Blue
Databricks
--
Robert Stupp
@snazy
ut things... We can continue to iterate on it!
There are also some points that I think are a bit controversial. I
intentionally put them in the proposal as a starting point for debates.
Please take a look and let me know what everyone thinks!
Best,
Jack Ye
--
Robert Stupp
@snazy
Thanks Jack for the proposal.
I’m generally +1 on this. There are a few details to clarify, but I suspect
nothing that’s controversial.
> On 24. Jun 2024, at 12:45, Ajantha Bhat wrote:
>
> Thank you, Jack, for your diligent work on this.
>
> This seems essential at the moment.
>
> I would lik
Alright, I've just created the issue:
https://github.com/apache/iceberg/issues/10537
The proposal document is here:
https://docs.google.com/document/d/1Xi5MRk8WdBWFC3N_eSmVcrLhk3yu5nJ9x_wC0ec6kVQ/
Robert
On 29.05.24 22:01, Robert Stupp wrote:
Jack's proposal is pretty much
g/rest/HTTPClient.java#L72>.
It would be nice if we formalize that
in the spec, at least define it as a
generic authorization header.
Best,
Jack Ye
- https://trino.io/docs/current/sql/create-function.html
Snowflake -
https://docs.snowflake.com/en/developer-guide/udf/sql/udf-sql-scalar-functions
Databricks -
https://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-ddl-create-sql-function.html
- Ajantha
--
Ryan Blue
Tabular
--
Robert Stupp
@snazy
c6749
[2] Iceberg pull request 4771 - Core: Add OAuth2 to REST catalog spec -
https://github.com/apache/iceberg/pull/4771
[3] Iceberg pull request 4843 - Spec: Add more context about OAuth2 to
the REST spec - https://github.com/apache/iceberg/pull/4843
--
Robert Stupp
@snazy
Strike that - all good.
On 06.03.24 12:48, Robert Stupp wrote:
-0
JSON (De)serialization of commit-metrics is broken. See
https://github.com/apache/iceberg/issues/9879
On 06.03.24 00:04, Ajantha Bhat wrote:
Hi Everyone,
I propose that we release the following RC as the official Apache
there are 3 binding +1 votes
and more binding
+1 votes than -1 votes.
- Ajantha
--
Robert Stupp
@snazy
From my experience, it’s a big win to have automatic code formatting.
In projectnessie we use automatic code formatting for all languages and haven’t
serious issues with Spotless. It is just nice to not have to bike shed about
whitespaces, line breaks, brackets, etc. It was a bit of discussion,
59 matches
Mail list logo
