Re: Read data from Postgres table pages

2024-03-19 Thread Sushrut Shivaswamy
> > > lol, thanks for the inputs Alexander :)!

Re: Read data from Postgres table pages

2024-03-19 Thread Alexander Korotkov
On Tue, Mar 19, 2024 at 4:48 PM Sushrut Shivaswamy wrote: > > If we query the DB directly, is it possible to know which new rows have been > added since the last query? > Is there a change pump that can be latched onto? Please, check this. https://www.postgresql.org/docs/current/logicaldecoding.

Re: Read data from Postgres table pages

2024-03-19 Thread Sushrut Shivaswamy
If we query the DB directly, is it possible to know which new rows have been added since the last query? Is there a change pump that can be latched onto? I’m assuming the page data structs are encapsulated in specific headers which can be used to list / read pages. Why would Postgres need to be

Re: Read data from Postgres table pages

2024-03-19 Thread Alexander Korotkov
On Tue, Mar 19, 2024 at 4:35 PM Sushrut Shivaswamy wrote: > The binary I"m trying to create should automatically be able to read data > from a postgres instance without users having to > run commands for backup / pg_dump etc. > Having access to the appropriate source headers would allow me to rea

Re: Read data from Postgres table pages

2024-03-19 Thread Sushrut Shivaswamy
The binary I"m trying to create should automatically be able to read data from a postgres instance without users having to run commands for backup / pg_dump etc. Having access to the appropriate source headers would allow me to read the data. On Tue, Mar 19, 2024 at 8:03 PM Sushrut Shivaswamy < su

Re: Read data from Postgres table pages

2024-03-19 Thread Sushrut Shivaswamy
I'd like to read individual rows from the pages as they are updated and stream them to a server to create a copy of the data. The data will be rewritten to columnar format for analytics queries. On Tue, Mar 19, 2024 at 7:58 PM Alexander Korotkov wrote: > Hi > > On Tue, Mar 19, 2024 at 4:23 PM Su

Re: Read data from Postgres table pages

2024-03-19 Thread Alexander Korotkov
Hi On Tue, Mar 19, 2024 at 4:23 PM Sushrut Shivaswamy wrote: > I'm trying to build a postgres export tool that reads data from table pages > and exports it to an S3 bucket. I'd like to avoid manual commands like > pg_dump, I need access to the raw data. > > Can you please point me to the postgr