Yufei, answers inline: On Mon, Aug 23, 2021 at 4:06 PM Yufei Gu <flyrain...@gmail.com> wrote:
> @Ryan, how do these properties work with multiple table locations? > > 1. > > write.metadata.path > 2. > > write.folder-storage.path > 3. > > write.object-storage.path > > The current logic with single table location is to honor these properties > on top of table location. In case of multiple roots(table locations), we > cannot mix them. For example, write.object-storage.path and table > location should be in the same region for the DR use case. > I think that we continue to honor those settings and relativize anything that is within the table location. For example, if I have `location=s3://bucket/db.table/` and `write.object-storage.path=s3://bucket/db.table/objects` then the paths would be something like `objects/987ae79d/ts_day=2021-08-18/00000-1-ea98234-1b34-4098-1298f40981.parquet`. If the object storage path was outside the table location, then all paths would be absolute. > One of solutions to keep the similar logic is that we support all > properties for each root, like this > > > 1. Root1: (table location1, metadata path1, folder-storage path1, > object-storage path1) > > 2. Root2: (table location2, metadata path2, folder-storage path2, > object-storage path2) > > 3. Root3: (table location3, null, null, null) > What do you think? > This is something that is up to the catalog/table. Those properties are used by the built-in `LocationProvider` implementations, but there isn't anything that requires using them. You could build a system that does this. > Ryan -- Ryan Blue Tabular
