Merlin Moncure wrote:
On 8/9/07, Omar Bettin <[EMAIL PROTECTED]> wrote:
Merlin Moncure wrote:
AFAIK, the fastest possible way to get data off the server, skipping
all data and text processing is to write a SPI routine, and stream the
data out locally to the server. I am doing exactly this in a
particular problem that requires high performance and I can tell you
that SPI is fast.
Interesting,do you thing that is possible to implement some send()
in the interface?
SPI is an interface which allows you to make sql calls from C code
(PostgreSQL allows you to link C code compiled as a .so to the server
and call -- see numerous examples in contrib). The routine you need
to exploit is SPI_getbinval which gives you Datum (essentially a
variant) pointing to the internal binary representation of your field.
In theory you could collect the data into a buffer and send() it off
although thats a lot of work IMO. Also, I would only advise this for
fast dumps from a single table (no joins, etc).
why not joins?
I have tried ZeosLib and for me is unusable (too slow), I use a
strong modified PostgresDAC.
I'm suprised -- I know the ZeosLib internals and it's a very thin
layer over libpq. Here is what I suggest:
* turn on statement logging in the server (set log_min_duration_statement)
* make some operations in the app which you suggest are slow -- they
will show up in the log
* 'explain analyze' the query from the psql console
make note of the times and post back (maybe move this thread to the
-performance list)
merlin
So, you aren't agree with the compression...
I have sow the network statistics and in some cases, the network traffic
is very big.
Probably a better setup could increase the performance by a few points
percent but
I think a compressed protocol could increase the communication by 2/3 times.
I think the biggest bottleneck in the whole system is just that.
:..try to get a query from a remote server with a 56k modem! :)
Omar
---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
