Thank you both for the feedback. To give some background on what I am trying to do. Mostly experimental work right now.
I've built an extension that takes advantage of the copy-on-write properties of ZFS and BTRFS. A tool transforms a data directory into a set of datasets/sub volumes. When a new database is created and the directories are copied, the extension uses the filesystem's ability to create a snapshot/clone of the source dataset/sub volume. As createdb() iterates over the template database's tablespaces and copies the directories to the new one, the copydir() gets invoked (from dbcommands.c). My extension hooks copydir() and instead snapshots the source dir and clones it into the target dir (respecting the recurse flag). If the source dir is not a ZFS data set or BTRFS sub volume, the standard implementation gets invoked and everything is business as usual. As described in the first paragraph, I've built a small tool that transforms all base/* directories in the data directory into ZFS datasets or BTRFS sub volumes. New tablespaces that get created later would have to be ZFS datasets or BTRFS sub volumes as well in order for the extension to work. As explained above, if they are not, the extension will just copy the files. I also hook ProcessUtility in order to clean up the ZFS datasets and BTRFS subvolumes _after_ a database gets deleted. Postgres just handles the deletion of the files and the extension cleans up any dangling ZFS/BTRFS resources. Is there anything that I am missing? My early experiments have been very promising but my experience with Postgres internals is limited. Any help or feedback would be much appreciated. On Mon, Sep 2, 2019 at 11:06 PM Alvaro Herrera <alvhe...@2ndquadrant.com> wrote: > > On 2019-Sep-02, Peter Eisentraut wrote: > > > On 2019-09-02 20:54, Swen Kooij wrote: > > > I've been working on an extension that tightly integrates > > > postgres with underlying filesystem . I need to customize > > > how postgres copies directories for new databases. > > > > Could you share some more details, so we can assess whether that is a > > sensible way to go about it, and what other hooks might be needed? > > It seems either terribly high-level, or terribly low-level, depending on > how you look at it. I wonder to what extent it conflicts with the table > AM work, and with the idea of allowing FDWs to have real relfilenodes. > I wonder if this is just one missing piece that's needed to complete > something of a layer for which other pieces are already satisfied by > other hooks. > > As is and pending further explanation, it seems a bad idea to me. If we > want pluggability here, maybe we need some new abstraction layer. > > -- > Álvaro Herrera https://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
