On 08/04/2019 19:38, Andres Freund wrote:
On 2019-04-08 14:53:53 +0300, Heikki Linnakangas wrote:
On 05/04/2019 23:25, Andres Freund wrote:
- the (optional) bitmap heap scan API - that's fairly intrinsically
block based. An AM could just internally subdivide TIDs in a different
way, but I don't think a bitmap scan like we have would e.g. make a
lot of sense for an index oriented table without any sort of stable
tid.
If an AM doesn't implement the bitmap heap scan API, what happens? Bitmap
scans are disabled?
Yea, the planner doesn't consider them. It just masks the index's
amhasgetbitmap. Seems to be the most reasonable thing to do?
Yep.
Even if an AM isn't block-oriented, the bitmap heap scan API still makes
sense as long as there's some correlation between TIDs and physical
location.
Yea, it could be a non-linear mapping. But I'm honestly not sure how
many non-block oriented AMs with such a correlation there are - I mean
you're not going to have that in say an IOT. And it'd be trivial to just
"fake" a block mapping for an in-memory AM.
Now that Ashwin conveniently posted the ZedStore prototype we started to
hack on [1], I'll point to that as an example :-). It stores data in a
B-tree (or rather, multiple B-trees) on TIDs. So there's very high
correlation between TIDs and physical locality, but it's not block oriented.
Another example would be the "LZ4 Compressed Storage Manager" that
Nikolai envisioned recently [2]. Before we came up with the idea of
using b-trees in ZedStore, we were actually thinking of something very
similar to that. Although that one perhaps still counts as
"block-oriented" as far as the bitmap heap scan API is concerned, as it
still deals with blocks, they're just mapped to different physical
locations.
I'm not sure how an Index-Organized-Table would work, but I think it
would want to just get the whole bitmap, and figure out the best order
to fetch the rows by itself.
PS. Seems that having a table AM API has opened the floodgates for
storage ideas. Nice!
- Heikki
[1]
https://www.postgresql.org/message-id/calfoeiuf-m5jg51mjupm5gn8u396o5sa2af5n97vtraedya...@mail.gmail.com
[2]
https://www.postgresql.org/message-id/flat/11996861554042351%40iva4-dd95b404a60b.qloud-c.yandex.net
