Hi Mate,
Thanks for the detailed response. This will help others in the same
situation
Dave Cramer
da...@postgresintl.com
www.postgresintl.com
On Thu, 6 Sep 2018 at 05:03, Mate Varga wrote:
> Hi,
>
> summarizing:
> we had a table that had an OID column, referencing an object in
> pg_largeobj
Hi,
summarizing:
we had a table that had an OID column, referencing an object in
pg_largeobject. This was mapped to a (Java) entity with a byte array field,
annotated with @Lob. The problem was that we were fetching thousands of
these entities in one go, and LOB fetching is not batched by Hibernat
Hi
Can you be more explicit how you fixed the problem ?
Thanks
Dave Cramer
da...@postgresintl.com
www.postgresintl.com
On Thu, 6 Sep 2018 at 03:46, Mate Varga wrote:
> After inlining the data, performance issues have been solved. Thanks for
> the help.
>
> On Mon, Sep 3, 2018 at 9:57 PM Mate
After inlining the data, performance issues have been solved. Thanks for
the help.
On Mon, Sep 3, 2018 at 9:57 PM Mate Varga wrote:
> Thanks,
> 1) we'll try to move stuff out from LOBs
> 2) we might raise a PR for the JDBC driver
>
> Mate
>
> On Mon, 3 Sep 2018, 19:35 Dave Cramer, wrote:
>
>>
>
Thanks,
1) we'll try to move stuff out from LOBs
2) we might raise a PR for the JDBC driver
Mate
On Mon, 3 Sep 2018, 19:35 Dave Cramer, wrote:
>
>
> On Mon, 3 Sep 2018 at 13:00, Mate Varga wrote:
>
>> More precisely: when fetching 10k rows, JDBC driver just does a large
>> bunch of socket read
On Mon, 3 Sep 2018 at 13:00, Mate Varga wrote:
> More precisely: when fetching 10k rows, JDBC driver just does a large
> bunch of socket reads. With lobs, it's ping-pong: one read, one write per
> lob...
>
>
Ok, this is making more sense. In theory we could fetch them all but since
they are LOB's
More precisely: when fetching 10k rows, JDBC driver just does a large bunch
of socket reads. With lobs, it's ping-pong: one read, one write per lob...
On Mon, Sep 3, 2018 at 6:54 PM Mate Varga wrote:
> So I have detailed profiling results now. Basically it takes very long
> that for each blob,
So I have detailed profiling results now. Basically it takes very long that
for each blob, the JDBC driver reads from the socket then it creates the
byte array on the Java side. Then it reads the next blob, etc. I guess this
takes many network roundtrips.
On Mon, Sep 3, 2018 at 5:58 PM Dave Cramer
On Mon, 3 Sep 2018 at 10:48, Mate Varga wrote:
> That's 1690 msec (1.69 seconds, and that is how long it takes to fetch 20k
> (small-ish) rows without LOBs (LOBs are a few lines below on the screenshot)
>
that sound high as well!
Something isn't adding up..
Dave Cramer
da...@postgresintl.com
That's 1690 msec (1.69 seconds, and that is how long it takes to fetch 20k
(small-ish) rows without LOBs (LOBs are a few lines below on the screenshot)
On Mon, Sep 3, 2018 at 4:40 PM Dave Cramer wrote:
> the one you have highlighted ~1.69ms
>
> Dave Cramer
>
> da...@postgresintl.com
> www.postgr
the one you have highlighted ~1.69ms
Dave Cramer
da...@postgresintl.com
www.postgresintl.com
On Mon, 3 Sep 2018 at 10:38, Mate Varga wrote:
> Which frame do you refer to?
>
> On Mon, Sep 3, 2018 at 3:57 PM Dave Cramer wrote:
>
>> Not sure why reading from a socket is taking 1ms ?
>>
>> Dave
Which frame do you refer to?
On Mon, Sep 3, 2018 at 3:57 PM Dave Cramer wrote:
> Not sure why reading from a socket is taking 1ms ?
>
> Dave Cramer
>
> da...@postgresintl.com
> www.postgresintl.com
>
>
> On Mon, 3 Sep 2018 at 09:39, Mate Varga wrote:
>
>> Hi,
>>
>> https://imgur.com/a/ovsJPRv -
Not sure why reading from a socket is taking 1ms ?
Dave Cramer
da...@postgresintl.com
www.postgresintl.com
On Mon, 3 Sep 2018 at 09:39, Mate Varga wrote:
> Hi,
>
> https://imgur.com/a/ovsJPRv -- I've uploaded the profiling info (as an
> image, sorry). It seems this is a JDBC-level problem. I
Hi,
https://imgur.com/a/ovsJPRv -- I've uploaded the profiling info (as an
image, sorry). It seems this is a JDBC-level problem. I understand that the
absolute timing is not meaningful at all because you don't know how large
the resultset is, but I can tell that this is only a few thousands rows +
Hey,
we'll try to test this with pure JDBC versus hibernate. Thanks!
On Mon, Sep 3, 2018 at 11:48 AM Dave Cramer wrote:
>
>
> On Mon, 3 Sep 2018 at 03:55, Mate Varga wrote:
>
>> Basically there's a class with a byte[] field, the class is mapped to
>> table T and the byte field is annotated wi
On Mon, 3 Sep 2018 at 03:55, Mate Varga wrote:
> Basically there's a class with a byte[] field, the class is mapped to
> table T and the byte field is annotated with @Lob so it goes to the
> pg_largeobject table.
>
Ah, so hibernate is in the mix. I wonder if that is causing some challenges
?
>
Basically there's a class with a byte[] field, the class is mapped to table
T and the byte field is annotated with @Lob so it goes to the
pg_largeobject table. The DB is on separate host but relatively close to
the app, and I can reproduce the problem locally as well. One interesting
bit is that tu
On Fri, 31 Aug 2018 at 10:15, Mate Varga wrote:
> I see -- we could try that, though we're mostly using an ORM (Hibernate)
> to do this. Thanks!
>
> On Fri, Aug 31, 2018 at 3:57 PM Dmitry Igrishin wrote:
>
>> пт, 31 авг. 2018 г. в 16:35, Mate Varga :
>> >
>> > Hi,
>> >
>> > we're fetching binary
I see -- we could try that, though we're mostly using an ORM (Hibernate) to
do this. Thanks!
On Fri, Aug 31, 2018 at 3:57 PM Dmitry Igrishin wrote:
> пт, 31 авг. 2018 г. в 16:35, Mate Varga :
> >
> > Hi,
> >
> > we're fetching binary data from pg_largeobject table. The data is not
> very large,
пт, 31 авг. 2018 г. в 16:35, Mate Varga :
>
> Hi,
>
> we're fetching binary data from pg_largeobject table. The data is not very
> large, but we ended up storing it there. If I'm copying the data to a file
> from the psql console, then it takes X time (e.g. a second), fetching it
> through the J
20 matches
Mail list logo
