Great to hear you're up and running, Marko!
Would you be interested in sharing your JDBC/Posgres metastore? I don't
think we have one yet and it would be great to have a simple one that is
backed by a database.
On Wed, Nov 25, 2020 at 9:06 AM Marko Babic
wrote:
> Thanks for your help + suggesti
Thanks for your help + suggestions, Peter. Thanks for the pointer to
Nessie, Jacques.
To wrap up the thread: I took an afternoon to put together a
Postgres-backed metastore to make sure I understood all the moving pieces
and have some confidence that I know what would go into the migration no
matt
Hi Marco,
See my comments below:
> On Nov 20, 2020, at 19:58, Marko Babic wrote:
>
> Hi Peter. Thanks for responding.
>
> > The command you mention below: `CREATE EXTERNAL TABLE` above an existing
> > Iceberg table will not transfer the "responsibility" of tracking the
> > snapshot to HMS. I
FYI, I would avoid adopting HMS because you need a better catalog. While
the HMS Iceberg catalog is mature, you're adopting something (HMS) that
carries a lot of baggage. I'd look at the other catalogs that are up and
coming if you can.
For example, Nessie (projectnessie.org) was built to provide
Hi Peter. Thanks for responding.
> The command you mention below: `CREATE EXTERNAL TABLE` above an existing
Iceberg table will not transfer the "responsibility" of tracking the
snapshot to HMS. It only creates a HMS external table ...
So my understanding is that the HiveCatalog is basically just
Hi John,
Thanks for the experience report and pointers to resources. :) If we do end
up going down that road it'll be super helpful.
Marko
On Thu, Nov 19, 2020 at 12:29 PM John Clara
wrote:
> Hi,
>
> My team has been using the custom catalog along with atomic metadata
> updates but we never mi
Hi Marko,
The command you mention below: `CREATE EXTERNAL TABLE` above an existing
Iceberg table will not transfer the "responsibility" of tracking the snapshot
to HMS. It only creates a HMS external table which will allow Hive queries to
read the given table. If you want to track the snapshot
Hi,
My team has been using the custom catalog along with atomic metadata
updates but we never migrated existing iceberg tables onto it. We also
haven't turned on integration with the hive catalog, so I'm not sure how
easy it is to plug in there (I think there was some recent work on
that?).
Hi everyone,
At my org we’ve spun up a few Iceberg tables on top of S3 without a
metastore (conscious of the consequences) and we’ve arrived at the point
that we need to support concurrent writes. :) I was hoping to get some
advice as to what the best way to integrate an existing Iceberg table int
