Re: [GENERAL] Performance appending to an array column

2017-09-22 Thread Imre Samu
>I also tried cstore_fdw for this, but my queries >(building a 2-D histogram) were taking 4+ seconds, >compared to 500ms using arrays. > ... > but maybe I could write my own extension Have you checked the new TimescaleDB extension? [ https://github.com/timescale/timescaledb ] "TimescaleDB is pack

Re: [GENERAL] Performance appending to an array column

2017-09-21 Thread Thomas Kellerer
Paul A Jungwirth schrieb am 21.09.2017 um 23:05: but maybe I could write my own extension to load regular files into Postgres arrays, sort of getting the best of both worlds. There is a foreign data wrapper for that: https://github.com/adunstan/file_text_array_fdw but it's pretty old and s

Re: [GENERAL] Performance appending to an array column

2017-09-21 Thread Paul A Jungwirth
> It's going to suck big-time :-(. Ha ha that's what I thought, but thank you for confirming. :-) > We ended up keeping > the time series data outside the DB; I doubt the conclusion would be > different today. Interesting. That seems a little radical to me, but I'll consider it more seriously no

Re: [GENERAL] Performance appending to an array column

2017-09-21 Thread Tom Lane
Paul A Jungwirth writes: > I'm considering a table structure where I'd be continuously appending > to long arrays of floats (10 million elements or more). Keeping the > data in arrays gives me much faster SELECT performance vs keeping it > in millions of rows. > But since these arrays keep growin