Pierre-
I agree with your observation of float <-> text conversion costs, but in
this case Dan is talking about storing the raw float data (ie: 4 bytes
per float) in a bytea array so there is only the conversion from java
float[n] to java byte[4*n] which is not nearly as costly as float <->
t
If the data are stored as a byte array but retrieve into a ResultSet,
the unpacking time goes up by an order of magnitude and the
observed total throughput is 25 MB/s. If the data are stored in a
Postgres float array and unpacked into a byte stream, the
observed throughput is 20 MB/s.
fl
Hello Daniel,
We have the same scenario for the native Java arrays, so we are storing bytea
and doing conversion at the client side, but for the server side SQL, plJava
comes very handy:
No sure how you want to create stored procedures to convert internally but this
is how we do this:
One has
Hi,
My name is Dan and I'm a co-worker of Nick Matheson who initially submitted this question (because the mail group had me blacklisted
for awhile for some reason).
Thank you for all of the suggestions. We were able to improve out bulk read performance from 3 MB/s to 60 MB/s (assuming the da
Hey all-
Glad to know you are still interested... ;)
Didn't mean to leave you hanging, the holiday and all have put some
bumps in the road.
Dan my co-worker might be able to post some more detailed information
here, but here is a brief summary of what I am aware of:
1. We have not tested a
BTW, have you tried prepared statements? bytea is most likely faster (in part)
due to less parsing in the backend. Prepared statements would eliminate that
parsing step.
On Dec 14, 2010, at 10:07 AM, Nick Matheson wrote:
> Hey all-
>
> Glad to know you are still interested... ;)
>
> Didn't me
On 12/14/2010 9:41 AM, Jim Nasby wrote:
On Dec 14, 2010, at 9:27 AM, Andy Colson wrote:
Is this the same thing Nick is working on? How'd he get along?
http://archives.postgresql.org/message-id/4cd1853f.2010...@noaa.gov
So it is. The one I replied to stood out because no one had replied to it
On Dec 14, 2010, at 9:27 AM, Andy Colson wrote:
> Is this the same thing Nick is working on? How'd he get along?
>
> http://archives.postgresql.org/message-id/4cd1853f.2010...@noaa.gov
So it is. The one I replied to stood out because no one had replied to it; I
didn't see the earlier email.
--
On 11/1/2010 9:15 AM, Dan Schaffer wrote:
Hello
We have an application that needs to do bulk reads of ENTIRE Postgres
tables very quickly (i.e. select * from table). We have observed that
such sequential scans run two orders of magnitude slower than observed
raw disk reads (5 MB/s versus 100 MB/
On Nov 1, 2010, at 9:15 AM, Dan Schaffer wrote:
> We have an application that needs to do bulk reads of ENTIRE Postgres tables
> very quickly (i.e. select * from table). We have observed that such
> sequential scans run two orders of magnitude slower than observed raw disk
> reads (5 MB/s versu
Hello
We have an application that needs to do bulk reads of ENTIRE Postgres tables very quickly (i.e. select * from table). We have
observed that such sequential scans run two orders of magnitude slower than observed raw disk reads (5 MB/s versus 100 MB/s). Part
of this is due to the storage
11 matches
Mail list logo
