On Tue, Aug 18, 2009 at 1:48 PM, Greg Stark <gsst...@mit.edu> wrote: > On Tue, Aug 18, 2009 at 6:39 PM, Michael Clark<codingni...@gmail.com> > wrote: > > But it seems pretty crazy that a 140meg bit of data goes to 1.3 gigs. > Does > > that seem a bit excessive? > > From what you posted earlier it looked like it was turning into about > 500M which sounds about right. Presumably either libpq or your code is > holding two copies of it in ram at some point in the process. >
>From what I saw, stopped at this line in my code running through gdb: const char *valC = PQgetvalue(result, rowIndex, i); my mem usage was 300megs. Stepping over this line it went to 1.3 gigs. Unless there is some way to misconfigure something, I can't think how my code could do that. I will profile it and see if I can tell who is holding on to that memory. > 8.5 will have an option to use a denser hex encoding but it will still > be 2x as large as the raw data. > Sweet! > > > I avoided the binary mode because that seemed to be rather confusing when > > having to deal with non-bytea data types. The docs make it sound like > > binary mode should be avoided because what you get back for a datetime > > varies per platform. > > There are definitely disadvantages. Generally it requires you to know > what the binary representation of your data types is and they're not > all well documented or guaranteed not to change in the future. I > wouldn't recommend someone try to decode a numeric or a postgres array > for example. And floating point numbers are platform dependent. But > bytea is a case where it seems more natural to use binary than text > representation. > To do something like this, I guess it would be best for my wrapper to being to detect when I have a bytea column in a table and do 2 fetchs, one in text for all other columns, and one in binary for the bytea column. Is this the best way to handle that do you think? Thanks, Michael.
