On Tue, Oct 26, 2021 at 2:31 PM Amit Kapila wrote:
>
> On Fri, Oct 15, 2021 at 1:48 AM Robert Haas wrote:
> >
> > On Tue, Oct 12, 2021 at 10:14 AM Dilip Kumar wrote:
> > > Thanks, yeah now it looks in line with other results.
> >
> > Since it seems there are no remaining concerns here, and we ha
On Fri, Oct 15, 2021 at 1:48 AM Robert Haas wrote:
>
> On Tue, Oct 12, 2021 at 10:14 AM Dilip Kumar wrote:
> > Thanks, yeah now it looks in line with other results.
>
> Since it seems there are no remaining concerns here, and we have
> benchmarking results showing that the patch helps, I have com
On Tue, Oct 12, 2021 at 10:14 AM Dilip Kumar wrote:
> Thanks, yeah now it looks in line with other results.
Since it seems there are no remaining concerns here, and we have
benchmarking results showing that the patch helps, I have committed
the patch.
I wonder whether the new code in shm_mq_send
On Tue, Oct 12, 2021 at 6:41 PM Tomas Vondra
wrote:
>
> On 9/28/21 14:00, Dilip Kumar wrote:
> >
> > I think that would be great, can we just test this specific target
> > where we are seeing a huge dip with the patch, e.g.
> > with 1000 rows, 10 columns and 4 threads, and queue size 64k. In
On Tue, Sep 28, 2021 at 2:49 AM Dilip Kumar wrote:
> 16k 64k 256k1024k
> Head 1232.779804.241134.723901.257
> Patch 1371.4931277.705862.598783.481
>
> So what I have noticed is that in most of th
On Tue, Sep 28, 2021 at 5:21 PM Tomas Vondra
wrote:
> >> Yeah, if it is losing in some cases then it is definitely good to know
> >> the reason, I was just looking into the performance numbers shared by
> >> Tomas in query-results, I can see the worst case is
> >> with 1000 rows, 10 columns an
On 9/28/21 12:53 PM, Amit Kapila wrote:
On Tue, Sep 28, 2021 at 12:19 PM Dilip Kumar wrote:
On Mon, Sep 27, 2021 at 10:52 PM Robert Haas wrote:
And most of the time, that's probably a good bet. But, if you do
somehow hit the losing case repeatedly, then you could see a
significant regressi
On Tue, Sep 28, 2021 at 12:19 PM Dilip Kumar wrote:
>
> On Mon, Sep 27, 2021 at 10:52 PM Robert Haas wrote:
>
> >
> > And most of the time, that's probably a good bet. But, if you do
> > somehow hit the losing case repeatedly, then you could see a
> > significant regression. And that might explai
On Mon, Sep 27, 2021 at 10:52 PM Robert Haas wrote:
>
> And most of the time, that's probably a good bet. But, if you do
> somehow hit the losing case repeatedly, then you could see a
> significant regression. And that might explain Tomas's results.
> Perhaps for some reason they just happen to h
On Mon, Sep 27, 2021 at 1:22 AM Dilip Kumar wrote:
> I have done testing with different batch sizes, 16k (which is the same
> as 1/4 of the queue size with 64k queue size) , 8k, 4k, 2k.
>
> In the attached sheet I have done a comparison of
> 1. head vs patch (1/4 queue size) = execution time reduc
On Sun, Sep 26, 2021 at 11:21 AM Dilip Kumar wrote:
>
> On Sat, Sep 25, 2021 at 2:18 AM Tomas Vondra
> wrote:
> >
> > On 9/24/21 7:08 PM, Robert Haas wrote:
> > > On Fri, Sep 24, 2021 at 3:50 AM Dilip Kumar wrote:
> > >> Tomas, can you share your test script, I would like to repeat the same
> >
On Sat, Sep 25, 2021 at 2:18 AM Tomas Vondra
wrote:
>
> On 9/24/21 7:08 PM, Robert Haas wrote:
> > On Fri, Sep 24, 2021 at 3:50 AM Dilip Kumar wrote:
> >> Tomas, can you share your test script, I would like to repeat the same
> >> test in my environment and with different batching sizes.
For now
On 9/24/21 7:08 PM, Robert Haas wrote:
On Fri, Sep 24, 2021 at 3:50 AM Dilip Kumar wrote:
Tomas, can you share your test script, I would like to repeat the same
test in my environment and with different batching sizes.
I think it's probably the run.sh file attached to
http://postgr.es/m/d76a7
On 9/24/21 1:43 AM, Robert Haas wrote:
On Thu, Sep 23, 2021 at 5:36 PM Tomas Vondra
wrote:
(c) This can't explain the slowdown for cases without any Gather nodes
(and it's ~17%, so unlikely due to binary layout).
Yeah, but none of the modified code would even execute in those cases,
so it's e
On Fri, Sep 24, 2021 at 1:19 AM Dilip Kumar wrote:
> I am looking at the "query-results.ods" file shared by Tomas, with a
> million tuple I do not really see where the patch hurts? because I am
> seeing in most of the cases the time taken by the patch is 60-80%
> compared to the head. And the wor
On Fri, Sep 24, 2021 at 3:50 AM Dilip Kumar wrote:
> Tomas, can you share your test script, I would like to repeat the same
> test in my environment and with different batching sizes.
I think it's probably the run.sh file attached to
http://postgr.es/m/d76a759d-9240-94f5-399e-ae244e5f0...@enterpr
On Fri, Sep 24, 2021 at 1:30 AM Tomas Vondra
wrote:
>
> On 9/23/21 9:31 PM, Robert Haas wrote:
> > On Wed, Sep 8, 2021 at 2:06 AM Dilip Kumar wrote:
> >> But I am attaching both the patches in case you want to play around.
> >
> > I don't really see any reason not to commit 0001. Perhaps some ver
On Fri, Sep 24, 2021 at 2:01 AM Robert Haas wrote:
>
> On Thu, Sep 23, 2021 at 4:00 PM Tomas Vondra
> wrote:
> > I did find some suspicious behavior on the bigger box I have available
> > (with 2x xeon e5-2620v3), see the attached spreadsheet. But it seems
> > pretty weird because the worst affec
On Thu, Sep 23, 2021 at 5:36 PM Tomas Vondra
wrote:
> (c) This can't explain the slowdown for cases without any Gather nodes
> (and it's ~17%, so unlikely due to binary layout).
Yeah, but none of the modified code would even execute in those cases,
so it's either binary layout, something wrong in
On 9/23/21 10:31 PM, Robert Haas wrote:
On Thu, Sep 23, 2021 at 4:00 PM Tomas Vondra
wrote:
I did find some suspicious behavior on the bigger box I have available
(with 2x xeon e5-2620v3), see the attached spreadsheet. But it seems
pretty weird because the worst affected case is with no para
On Thu, Sep 23, 2021 at 4:00 PM Tomas Vondra
wrote:
> I did find some suspicious behavior on the bigger box I have available
> (with 2x xeon e5-2620v3), see the attached spreadsheet. But it seems
> pretty weird because the worst affected case is with no parallel workers
> (so the queue changes sho
On 9/23/21 9:31 PM, Robert Haas wrote:
On Wed, Sep 8, 2021 at 2:06 AM Dilip Kumar wrote:
But I am attaching both the patches in case you want to play around.
I don't really see any reason not to commit 0001. Perhaps some very
minor grammatical nitpicking is in order here, but apart from that
On Wed, Sep 8, 2021 at 2:06 AM Dilip Kumar wrote:
> But I am attaching both the patches in case you want to play around.
I don't really see any reason not to commit 0001. Perhaps some very
minor grammatical nitpicking is in order here, but apart from that I
can't really see anything to criticize
On 9/8/21 8:05 AM, Dilip Kumar wrote:
On Tue, Sep 7, 2021 at 8:41 PM Tomas Vondra
mailto:tomas.von...@enterprisedb.com>>
wrote:
Hi,
The numbers presented in this thread seem very promising - clearly
there's significant potential for improvements. I'll run similar
benchmarks
On Sat, Aug 28, 2021 at 5:04 PM Zhihong Yu wrote:
>
>>
>> * mqh_partial_bytes, mqh_expected_bytes, and mqh_length_word_complete
>>
>> + Sizemqh_send_pending;
>> boolmqh_length_word_complete;
>> boolmqh_counterparty_attached;
>>
>> I wonder if mqh_send_pending sh
On Wed, Sep 8, 2021 at 4:41 PM Dilip Kumar wrote:
Based on various suggestions, I have some more experiments with the patch.
1) I have measured the cache misses count and I see a ~20% reduction
in cache misses with the patch (updating shared memory counter only
after we written certain amount of
On Wed, Sep 8, 2021 at 3:28 PM Tomas Vondra
wrote:
On 9/8/21 9:40 AM, Dilip Kumar wrote:
>
> > Maybe it can be misleading sometimes, but I feel sometimes it is more
> > informative compared to the optimized build where it makes some function
> > inline, and then it becomes really hard to distingu
On Wed, Sep 8, 2021 at 12:03 PM Andres Freund wrote:
> Hi,
>
> On 2021-09-08 11:45:16 +0530, Dilip Kumar wrote:
> > On Wed, Sep 8, 2021 at 3:08 AM Andres Freund wrote:
> >
> >
> > > Looking at this profile made me wonder if this was a build without
> > > optimizations. The pg_atomic_read_u64()/p
Hi,
On 2021-09-08 11:45:16 +0530, Dilip Kumar wrote:
> On Wed, Sep 8, 2021 at 3:08 AM Andres Freund wrote:
>
>
> > Looking at this profile made me wonder if this was a build without
> > optimizations. The pg_atomic_read_u64()/pg_atomic_read_u64_impl() calls
> > should
> > be inlined. And while
On Wed, Sep 8, 2021 at 3:08 AM Andres Freund wrote:
> Looking at this profile made me wonder if this was a build without
> optimizations. The pg_atomic_read_u64()/pg_atomic_read_u64_impl() calls
> should
> be inlined. And while perf can reconstruct inlined functions when using
> --call-graph=dwa
On Tue, Sep 7, 2021 at 8:41 PM Tomas Vondra
wrote:
> Hi,
>
> The numbers presented in this thread seem very promising - clearly
> there's significant potential for improvements. I'll run similar
> benchmarks too, to get a better understanding of this.
>
Thanks for showing interest.
>
> Can you
Hi,
On 2021-08-06 14:00:48 +0530, Dilip Kumar wrote:
> --Setup
> SET parallel_tuple_cost TO 0 -- to test parallelism in the extreme case
> CREATE TABLE t (a int, b varchar);
> INSERT INTO t SELECT i, repeat('a', 200) from generate_series(1,2) as
> i;
> ANALYZE t;
> Test query: EXPLAIN A
Hi,
The numbers presented in this thread seem very promising - clearly
there's significant potential for improvements. I'll run similar
benchmarks too, to get a better understanding of this.
Can you share some basic details about the hardware you used?
Particularly the CPU model - I guess this mi
On Sat, Aug 28, 2021 at 4:29 AM Zhihong Yu wrote:
>
>
> On Sat, Aug 28, 2021 at 12:11 AM Dilip Kumar
> wrote:
>
>> On Tue, Aug 24, 2021 at 8:48 PM Robert Haas
>> wrote:
>>
>>> On Fri, Aug 6, 2021 at 4:31 AM Dilip Kumar
>>> wrote:
>>> > Results: (query EXPLAIN ANALYZE SELECT * FROM t;)
>>> > 1)
On Sat, Aug 28, 2021 at 12:11 AM Dilip Kumar wrote:
> On Tue, Aug 24, 2021 at 8:48 PM Robert Haas wrote:
>
>> On Fri, Aug 6, 2021 at 4:31 AM Dilip Kumar wrote:
>> > Results: (query EXPLAIN ANALYZE SELECT * FROM t;)
>> > 1) Non-parallel (default)
>> > Execution Time: 31627.492 ms
>> >
>> > 2) P
On Tue, Aug 24, 2021 at 8:48 PM Robert Haas wrote:
> On Fri, Aug 6, 2021 at 4:31 AM Dilip Kumar wrote:
> > Results: (query EXPLAIN ANALYZE SELECT * FROM t;)
> > 1) Non-parallel (default)
> > Execution Time: 31627.492 ms
> >
> > 2) Parallel with 4 workers (force by setting parallel_tuple_cost to
On Fri, Aug 6, 2021 at 4:31 AM Dilip Kumar wrote:
> Results: (query EXPLAIN ANALYZE SELECT * FROM t;)
> 1) Non-parallel (default)
> Execution Time: 31627.492 ms
>
> 2) Parallel with 4 workers (force by setting parallel_tuple_cost to 0)
> Execution Time: 37498.672 ms
>
> 3) Same as above (2) but
On Fri, Aug 6, 2021 at 2:00 PM Dilip Kumar wrote:
>
> Experiment #1:
> As part of this experiment, I have modified the sender to keep the
> local copy of "mq_bytes_read" and "mq_bytes_written" in the local mqh
> handle so that we don't need to frequently read/write cache sensitive
> shared memory
Hi,
I have been working on analyzing the performance of sending the tuple
from workers to the Gather using the tuple queue. In the past there
were many off-list discussions around this area, basically, the main
point is that when the "shm_mq" was implemented that time maybe this
was one of the be
39 matches
Mail list logo
