Re: Support "Right Semi Join" plan shapes

2024-01-21 Thread wenhui qiu
Hi vignesh CI saw this path has been passed ( https://cirrus-ci.com/build/6109321080078336),can we push it? Best wish Richard Guo 于2024年1月9日周二 18:49写道: > > On Sun, Jan 7, 2024 at 3:03 PM vignesh C wrote: > >> One of the tests in CFBot has failed at [1] with: >> - Relations: (public.ft1

Re: Support "Right Semi Join" plan shapes

2024-01-23 Thread wenhui qiu
Hi vignesh C Many thanks, I have marked it to "ready for committer" Best wish vignesh C 于2024年1月23日周二 10:56写道: > On Mon, 22 Jan 2024 at 11:27, wenhui qiu wrote: > > > > Hi vignesh CI saw this path has been passed ( > https://cirrus-ci.com/build/6109

Re: Support "Right Semi Join" plan shapes

2024-02-07 Thread wenhui qiu
Hi Alena Rybakina I saw this code snippet also disable mergejoin ,I think it same effect + /* + * For now we do not support RIGHT_SEMI join in mergejoin. + */ + if (jointype == JOIN_RIGHT_SEMI) + { + *mergejoin_allowed = false; + return NIL; + } + Regards Alena Rybakina 于2024年1月30日周二 14:51写道:

Thoughts about NUM_BUFFER_PARTITIONS

2024-02-08 Thread wenhui qiu
HI hackers When I read this text in this document there is a paragraph in it( https://www.interdb.jp/pg/pgsql08/03.html) /* The BufMappingLock is split into partitions to reduce contention in the buffer table (the default is 128 partitions). Each BufMappingLock partition guards a portion of the

Re: Thoughts about NUM_BUFFER_PARTITIONS

2024-02-08 Thread wenhui qiu
, but the result report is in Chinese Best wishes Heikki Linnakangas 于2024年2月8日周四 19:26写道: > On 08/02/2024 12:17, wenhui qiu wrote: > > HI hackers > > When I read this text in this document there is a paragraph in > > it(https://www.interdb.jp/pg/pgsql08/03.html > &

Re: Commitfest Manager for March

2024-03-20 Thread wenhui qiu
Hi Aleksander Alekseev Could you take a look at the patch ( https://commitfest.postgresql.org/47/4284/),How about your opinion Thanks On Tue, 12 Mar 2024 at 21:41, Aleksander Alekseev wrote: > Hi Andrey, > > > > If you need any help please let me know. > > > > Aleksander, I would greatly

Re: Thoughts about NUM_BUFFER_PARTITIONS

2024-02-19 Thread wenhui qiu
Hi Japlin Li Thank you for such important information ! Got it Japin Li 于2024年2月19日周一 10:26写道: > > On Mon, 19 Feb 2024 at 00:56, Tomas Vondra > wrote: > > On 2/18/24 03:30, Li Japin wrote: > >> > >> I find it seems need to change MAX_SIMUL_LWLOCKS if we enlarge the > NUM_BUFFER_PARTITIONS, >

Re: Thoughts about NUM_BUFFER_PARTITIONS

2024-02-19 Thread wenhui qiu
ng this far looks OK. ````````` wenhui qiu 于2024年2月20日周二 09:36写道: > Hi Japlin Li >Thank you for such important information ! Got it > > Japin Li 于2024年2月19日周一 10:26写道: > >> >> On Mon, 19 Feb 2024 at 00:56, Tomas Vondra >> wrote: >> > On 2/18

Re: Thoughts about NUM_BUFFER_PARTITIONS

2024-02-23 Thread wenhui qiu
ter. The default value is 4, which corresponds to 16 partitions, and the maximum is 8. This parameter can only be set in the postgresql.conf file or on the server command line. Best wish On Tue, 20 Feb 2024 at 21:55, Tomas Vondra wrote: > Hi, > > On 2/20/24 03:16, wenhui qiu wrote:

Re: Support "Right Semi Join" plan shapes

2024-03-03 Thread wenhui qiu
HI Richard Now it is starting the last commitfest for v17, can you respond to Alena Rybakina points? Regards On Thu, 8 Feb 2024 at 13:50, wenhui qiu wrote: > Hi Alena Rybakina > I saw this code snippet also disable mergejoin ,I think it same effect > + /* > + * For n

Re: Support "Right Semi Join" plan shapes

2024-03-04 Thread wenhui qiu
Hi Richard Agree +1 ,I think can push now. Richard On Tue, 5 Mar 2024 at 10:44, Richard Guo wrote: > > On Tue, Jan 30, 2024 at 2:51 PM Alena Rybakina > wrote: > >> I have reviewed your patch and I think it is better to add an Assert for >> JOIN_RIGHT_SEMI to the ExecMergeJoin and ExecNest

Re: Support "Right Semi Join" plan shapes

2024-03-05 Thread wenhui qiu
Hi Alena Rybakina For merge join + /* + * For now we do not support RIGHT_SEMI join in mergejoin. + */ + if (jointype == JOIN_RIGHT_SEMI) + { + *mergejoin_allowed = false; + return NIL; + } + Tanks On Wed, 6 Mar 2024 at 04:10, Alena Rybakina wrote: > To be honest, I didn't see it in the code, co

Re: Add 64-bit XIDs into PostgreSQL 15

2023-12-14 Thread wenhui qiu
Hi Maxim Orlov Good news,xid64 has achieved a successful first phase,I tried to change the path status (https://commitfest.postgresql.org/43/3594/) ,But it seems incorrect Maxim Orlov 于2023年12月13日周三 20:26写道: > Hi! > > Just to keep this thread up to date, here's a new version after recent > c

Support "Right Semi Join" plan shapes

2023-12-14 Thread wenhui qiu
Hi Richard Guo I see that the test samples are all (exists) subqueries ,I think semi join should also support ( in) and ( any) subqueries. would you do more test on ( in) and ( any) subqueries? Best whish

Re: Add 64-bit XIDs into PostgreSQL 15

2023-12-15 Thread wenhui qiu
Hi Pavel Borisov Many thanks Best whish Pavel Borisov 于2023年12月15日周五 17:13写道: > Hi, Wenhui! > > On Fri, 15 Dec 2023 at 05:52, wenhui qiu wrote: > >> Hi Maxim Orlov >> Good news,xid64 has achieved a successful first phase,I tried to >>

Re: Transaction timeout

2023-12-20 Thread wenhui qiu
Hi Junwang Zhao Agree +1 Best whish Junwang Zhao 于2023年12月20日周三 10:35写道: > On Wed, Dec 20, 2023 at 9:58 AM Thomas wen > wrote: > > > > Hi Junwang Zhao > > #should we invalidate lock_timeout? Or maybe just document this. > > I think you mean when lock_time is greater than trasaction-tim

Re: Support "Right Semi Join" plan shapes

2023-12-27 Thread wenhui qiu
b.fivethous ) AND i4.f1 = a.tenthous; QUERY PLAN Hash Join Hash Cond: (a.tenthous = i4.f1) -> Seq Scan on tenk1 a Filter: (SubPlan 1) SubPlan 1

Re: Support "Right Semi Join" plan shapes

2024-06-24 Thread wenhui qiu
Hi Japin Li Thank you for your reviewing ,This way the notes are more accurate and complete. Thanks also to the author for updating the patch ,I also tested the new patch ,It looks good to me Regrads Japin Li 于2024年6月25日周二 08:51写道: > On Mon, 24 Jun 2024 at 17:59, Richard Guo wrote: > > T

Linux likely() unlikely() for PostgreSQL

2024-06-30 Thread wenhui qiu
Hi Hackers When I saw this document:https://en.wikipedia.org/wiki/Branch_predictor, I thought of Linux likely() vs unlikely() and thus thought that there are some code segments in src/backend/executor/execMain.c that can be optimized. For example : if (ExecutorStart_hook) (*ExecutorStart_hook) (

Re: Linux likely() unlikely() for PostgreSQL

2024-06-30 Thread wenhui qiu
Hi Tom ,Matthias Thank you for your explanation.Maybe future compilers will be able to do intelligent prediction? Thanks Tom Lane 于2024年6月30日周日 23:23写道: > Matthias van de Meent writes: > > On Sun, 30 Jun 2024, 15:56 wenhui qiu, wrote: > >> if (unlikely(ExecutorRun_h

Re: Support "Right Semi Join" plan shapes

2024-07-04 Thread wenhui qiu
Hi Richard Guo Thank you for updating the patch.Tested on v8 , It looks good to me Thanks Richard Guo 于2024年7月4日周四 17:18写道: > On Fri, Jun 28, 2024 at 3:21 PM Richard Guo > wrote: > > On Fri, Jun 28, 2024 at 2:54 PM Richard Guo > wrote: > > > I've refined this test case further to make

Re: Optimize commit performance with a large number of 'on commit delete rows' temp tables

2024-07-07 Thread wenhui qiu
Hi feichanghong Thanks for updating the patch ,I think could be configured as a GUC parameter,PostgreSQL has too many static variables that are written to death and explicitly stated in the code comments may later be designed as parameters. Now that more and more applications that previously u

Re: Optimize commit performance with a large number of 'on commit delete rows' temp tables

2024-07-07 Thread wenhui qiu
Hi feichanghong I don't think it's acceptable to introduce a patch to fix a problem that leads to performance degradation, or can we take tom's suggestion to optimise PreCommit_on_commit_actions? I think it to miss the forest for the trees Best Regards, feichanghong 于2024年7月8日周一 10:35写道:

Re: Optimize commit performance with a large number of 'on commit delete rows' temp tables

2024-07-07 Thread wenhui qiu
Alibaba Best Regards, feichanghong 于2024年7月8日周一 12:42写道: > Hi wenhui, > > On Jul 8, 2024, at 12:18, wenhui qiu wrote: > > Hi feichanghong > I don't think it's acceptable to introduce a patch to fix a problem > that leads to performance degradation, or

Re: allow changing autovacuum_max_workers without restarting

2024-04-16 Thread wenhui qiu
Agree +1,From a dba perspective, I would prefer that this parameter can be dynamically modified, rather than adding a new parameter,What is more difficult is how to smoothly reach the target value when the setting is considered to be too large and needs to be lowered. Regards On Tue, 16 Apr 202

Re: POC: make mxidoff 64 bits

2024-04-23 Thread wenhui qiu
Hi Maxim Orlov Thank you so much for your tireless work on this. Increasing the WAL size by a few bytes should have very little impact with today's disk performance(Logical replication of this feature wal log is also increased a lot, logical replication is a milestone new feature, and the commun

Re: Support "Right Semi Join" plan shapes

2024-04-28 Thread wenhui qiu
Hi Richard Thank you so much for your tireless work on this,I see the new version of the patch improves some of the comments .I think it can commit in July Thanks On Thu, 25 Apr 2024 at 11:28, Richard Guo wrote: > Here is another rebase with a commit message to help review. I also > twea

Re: Proposal to Enable/Disable Index using ALTER INDEX

2024-09-10 Thread wenhui qiu
Hi Shayon Thank you for your work on this , I think it's great to have this feature implemented ,I checked the doucment on other databases,It seems both MySQL 8.0 and oracle supports it, sql server need to rebuild indexes after disabled,It seems disable the index, it's equivalent to deletin

Re: Can we rely on the ordering of paths in pathlist?

2024-09-17 Thread wenhui qiu
paths that do not meet the criteria,find the unparameterized parallel-safe path with the least total cost and return NULL if it does not exist. + * + */ Thanks wenhui qiu 于2024年7月31日周三 09:21写道: > Hi Richard Guo > Today is the last day of the commitfest cycle.I think this patch >

Re: Can we rely on the ordering of paths in pathlist?

2024-07-25 Thread wenhui qiu
Hi Richard Guo Is it necessary to add some comments here? + if (!innerpath->parallel_safe || + !bms_is_empty(PATH_REQ_OUTER(innerpath))) + continue; + + if (matched_path != NULL && + compare_path_costs(matched_path, innerpath, TOTAL_COST) <= 0) + continue; + + matched_path = innerpath; Richa

Re: Add 64-bit XIDs into PostgreSQL 15

2024-07-25 Thread wenhui qiu
Maybe a setting similar to max_wal_size could be better for that? +1 Thanks Peter Eisentraut 于2024年7月25日周四 21:31写道: > On 25.07.24 13:09, Heikki Linnakangas wrote: > >> However, if there is no more disaster threshold at 2^31, what is the > >> guidance for setting these? Or more radically, why e

Re: Can we rely on the ordering of paths in pathlist?

2024-07-30 Thread wenhui qiu
Hi Richard Guo Today is the last day of the commitfest cycle.I think this patch should be commented ,Except for the comments, I tested it good to me Thanks wenhui qiu 于2024年7月25日周四 16:18写道: > Hi Richard Guo > Is it necessary to add some comments here? > > > +

Re: bgwrite process is too lazy

2024-10-06 Thread wenhui qiu
Hi Andres Freund Thank you for explanation > I doubt that slowdown is caused by bgwriter not being active enough. I suspect > what you're seeing is one or more of: > a) The overhead of doing full page writes (due to increasing the WAL > volume). You could verify whether that's the case by tur

Re: New PostgreSQL Contributors

2024-10-04 Thread wenhui qiu
Congratulations to all! On Fri, 4 Oct 2024 at 22:50, Nathan Bossart wrote: > On Fri, Oct 04, 2024 at 03:55:23PM +0200, Christoph Berg wrote: > > New PostgreSQL Contributors: > > > > * Antonin Houska > > * Ants Aasma > > * Georgios Kokolatos > > * Henrietta Dombrovskaya > > * Ian Lawrence Barwic

Re: bgwrite process is too lazy

2024-10-03 Thread wenhui qiu
mance when checkpoints occur Thanks Andres Freund 于2024年10月4日周五 03:40写道: > Hi, > > On 2024-10-02 18:36:44 +0200, Tomas Vondra wrote: > > On 10/2/24 17:02, Tony Wayne wrote: > > > > > > > > > On Wed, Oct 2, 2024 at 8:14 PM Laurenz Albe > > <mai

optimize the value of vacthresh and anlthresh

2024-11-04 Thread wenhui qiu
Hi hackers A few days ago, I was looking at the sql server documentation and found that sql server has optimized the algorithm related to updating statistics in the 2016 ,version,I think we can also learn from the implementation method of sql server to optimize the problem of automatic vacuum t

Re: POC: make mxidoff 64 bits

2024-10-24 Thread wenhui qiu
HI Maxim Orlov > After a bit of thought, I've realized that to be conservative here is the way to go. >We can reuse a maximum of existing logic. I mean, we can remove offset wraparound "error logic" and reuse "warning logic". But set the threshold for "warning >logic" to a much higher value. For no

Re: Changing the default random_page_cost value

2024-10-24 Thread wenhui qiu
HI Greg Sabino Mullane Another thing is that you simply change the configuration template is not effective, need to modify the DEFAULT_RANDOM_PAGE_COST values { {"random_page_cost", PGC_USERSET, QUERY_TUNING_COST, gettext_noop("Sets the planner's estimate of the cost of a " "nonsequentially fe

Re: New GUC autovacuum_max_threshold ?

2024-11-13 Thread wenhui qiu
HI > In any case, we should do the tests that Robert suggested and/or come up > with a good mathematical model, because we are in the dark at the moment. I think SQL Server has given us great inspiration >I think we should indeed provide a retro-compatible behaviour (so maybe > another GUC after al

Re: New GUC autovacuum_max_threshold ?

2024-11-10 Thread wenhui qiu
QL Server in vacuum analysis . Nathan Bossart 于2024年11月9日周六 23:59写道: > On Sat, Nov 09, 2024 at 10:08:51PM +0800, wenhui qiu wrote: > > Sorry ,I forgot to explain the reason in my last email,In fact, I > > submitted the patch to the community,(frederic.yh...@dalibo.com) tol

Re: [PATCH] Support Int64 GUCs

2024-09-26 Thread wenhui qiu
Hi Alexander I think we need int64 GUCs, due to these parameters( autovacuum_freeze_table_age, autovacuum_freeze_max_age,When a table age is greater than any of these parameters an aggressive vacuum will be performed, When we implementing xid64, is it still necessary to be in the int range?

Re: Patch: Show queries of processes holding a lock

2024-10-01 Thread wenhui qiu
Hi Alexey Orlov Thank you for your work on this path,The lock information is recorded in detail,Easy to trace the lock competition at that time there is a detailed lock competition log,But I have a concern,Frequent calls to this function (pgstat_get_backend_current_activity) in heavy lock cont

Re: bgwrite process is too lazy

2024-10-02 Thread wenhui qiu
24 at 8:14 PM Laurenz Albe > <mailto:laurenz.a...@cybertec.at>> wrote: > > > > On Wed, 2024-10-02 at 16:48 +0800, wenhui qiu wrote: > > > Whenever I check the checkpoint information in a log, most dirty > > pages are written by the checkpoint proces

Re: optimize the value of vacthresh and anlthresh

2024-11-06 Thread wenhui qiu
Hi Frédéric many thanks for your email。 I'll go and see. On Wed, 6 Nov 2024 at 17:47, Frédéric Yhuel wrote: > > > On 11/4/24 09:30, wenhui qiu wrote: > > Hi hackers > > A few days ago, I was looking at the sql server documentation and > > found t

Re: New GUC autovacuum_max_threshold ?

2024-11-06 Thread wenhui qiu
Hi frederic.yhuel > Thank you. FWIW, I would prefer a sub-linear growth, so maybe something > like this > vacthresh = Min(vac_base_thresh + vac_scale_factor * reltuples, > vac_base_thresh + vac_scale_factor * pow(reltuples, 0.7) * 100); > This would give : > * 386M (instead of 5.1 billi

Re: pg_stat_statements: Avoid holding excessive lock

2024-11-08 Thread wenhui qiu
Hi Karina Liskevich > + /* > + * There is no need to hold entry->mutex when reading stats_since and > + * minmax_stats_since for (unlike counters) they are always written > + * while holding pgss->lock exclusively. We are holding pgss->lock > +

Re: Auto Vacuum optimisation

2024-11-27 Thread wenhui qiu
sing the sqrt algorithm ,The main purpose is also to give DBAs the flexibility to make small adjustments,Mainly referencing another discussion as well, trying to stay in line with them On Thu, Nov 28, 2024 at 1:46 PM Japin Li wrote: > On Wed, 27 Nov 2024 at 15:51, wenhui qiu wrote: > > HI

Re: A way to build PSQL 17.1 source on AIX platform

2024-11-19 Thread wenhui qiu
no way ,due to remove aix support, https://commitfest.postgresql.org/50/5003/you can see this link, On Tue, 19 Nov 2024 at 23:26, Raghu Dev Ramaiah wrote: > Hi > > I am getting error while building PSQL 17.1 on AIX box- > > bash-5.1# uname -a > AIX AIXLPHPOM01 1 7 00F8B83A4C00 > > bash-5.1# ./co

Auto Vacuum optimisation

2024-11-18 Thread wenhui qiu
HI hackers and japin li There was an earlier discussion about auto vacuum optimisation( https://www.postgresql.org/message-id/6a2ac9b7-6535-4bb1-8274-0647f7c31c82%40dalibo.com),I referring to the discussion in there, I implemented a parameter selection triggering the calculation of AUTO VACUU

Re: An inefficient query caused by unnecessary PlaceHolderVar

2024-11-27 Thread wenhui qiu
Hi Richard > BTW, since commit cb8e50a4a, we've chosen not to wrap a non-var > expression if it contains Vars/PHVs of the pulled-up subquery and does > not contain non-strict constructs. I wonder if we can apply the same > optimization from this patch to non-var expressions: for a LATERAL > subqu

Re: UUID v7

2024-11-24 Thread wenhui qiu
HI Andrey M. Borodin It's not just mariadb, percona also implements the uuid plugin. https://docs.percona.com/percona-server/8.4/uuid-versions.html#functions-available-in-uuid_vx Thanks Andrey M. Borodin 于2024年11月23日周六 16:21写道: > > > > On 23 Nov 2024, at 10:58, Masahiko Sawada wrote: > >

adjust_limit_rows_costs algorithm

2024-12-23 Thread wenhui qiu
Hi Hackers There should be many people who have encountered the problem of order by col limit 1 without index filtering,Here is an example of my test create extension pg_trgm ; CREATE TABLE public.dba_users ( userid integer primary key, username character varying(64), password character varyin

Re: add vacuum starttime columns

2024-12-30 Thread wenhui qiu
Hi Sami Many people have encountered situations where autovacuum or auto analyze on tables are not triggered in time, leading to suboptimal execution plans and some performance issues. When analyzing such problems, we often need to trace back to when autovacuum or auto analyze was triggered for

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-06 Thread wenhui qiu
g this work! > > On 1/3/2025 9:01 PM, wenhui qiu wrote: > > Hi > > Thank you for your path,NUM_XLOGINSERT_LOCKS increase to 128,I > > think it will be challenged,do we make it guc ? > > > > I noticed there have been some discussions (for example, [1] and i

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-06 Thread wenhui qiu
patch has been fully accepted, we could then investigate the more proper > way of increasing NUM_XLOGINSERT_LOCKS. WDYT? > > On 1/6/2025 4:35 PM, wenhui qiu wrote: > > HI Zhiguo > > Thank you for your reply ,Then you'll have to prove that 128 is the > > optimal va

Re: PoC: history of recent vacuum/checkpoint runs (using new hooks)

2024-12-26 Thread wenhui qiu
mas Vondra wrote: > > > > Hi, > > > > On 12/23/24 07:35, wenhui qiu wrote: > > > Hi Tomas > > > This is a great feature. > > > + /* > > > + * Define (or redefine) custom GUC variables. > > > + */ > > > + DefineCustomI

Re: transaction lost when delete clog file after normal shutdown

2024-12-24 Thread wenhui qiu
Hi Michael Paquier Thank you for the information you provided, Thanks On Tue, 24 Dec 2024 at 13:13, Michael Paquier wrote: > On Tue, Dec 24, 2024 at 09:55:09AM +0800, wenhui qiu wrote: > > However, on the other hand, oracle has many solutions to open the > database > &

Re: PoC: history of recent vacuum/checkpoint runs (using new hooks)

2024-12-26 Thread wenhui qiu
tem can handle). yes, I mean by this is that the maximum value is not friendly to large instances if the setting is small ,In the real production instance , many sub-tables with the same table structure are often encountered On Fri, Dec 27, 2024 at 1:58 AM Tomas Vondra wrote: > On 12/26/24

Re: New GUC autovacuum_max_threshold ?

2025-02-03 Thread wenhui qiu
HI Nathan > I had the opportunity to bring this patch up for discussion at the > developer meeting at FOSDEM PGDay last week [0]. We discussed a subset > of the topics folks have already written about in this thread, and AFAICT > there was general approval among the attendees for proceeding with t

Re: Compression of bigger WAL records

2025-01-30 Thread wenhui qiu
Hi Andery I have a question ,If wal_compression_threshold is set to more than the block size of the wal log, then the FPI is not compressed, and if so, it might make sense to have a maximum value of this parameter that does not exceed the block size of the wal log? Best regards On Thu, Jan

Uncached buffered IO

2025-02-09 Thread wenhui qiu
Hi Hackers. https://lwn.net/Articles/998783/ , is Uncached buffered IO useful for double cache?PostgreSQL's double caching is often a source of complaints. This new Linux kernel feature seems to help mitigate performance jitters caused by file cache reclaim when memory free memory is too small.

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-06 Thread wenhui qiu
c520b9dfd60 > > On Wed, Feb 5, 2025 at 8:54 PM wenhui qiu wrote: > > > > Hi Melanie Plageman > >Thank you for working on this ,Actually, there were two patches aimed > at optimizing vacuum-triggered processes, and one of them reached a > consensus and has been c

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-05 Thread wenhui qiu
Hi Melanie Plageman Thank you for working on this ,Actually, there were two patches aimed at optimizing vacuum-triggered processes, and one of them reached a consensus and has been committed:https://commitfest.postgresql.org/52/5046/ , https://commitfest.postgresql.org/51/5395/, Maybe referring

Re: [PATCH] Support Int64 GUCs

2024-12-10 Thread wenhui qiu
Hi aleksander Didn't you mark this patch as rejected last time? Do we still need to continue this path? Thanks On Sun, Dec 8, 2024 at 10:16 PM Evgeny Voropaev wrote: > Hi hackers! > > Upgraded the "Int64 GUC" patch in order to conform to f3f06b13308e3 > updates. Rebased and tested upon the

Re: Add 64-bit XIDs into PostgreSQL 15

2024-12-16 Thread wenhui qiu
Hi Evgeny xid64 path split several threads ,The current one should be this:( https://www.postgresql.org/message-id/flat/CACG=ezawg7_nt-8ey4akv2w9lculthhknwcawmbgeetnjrj...@mail.gmail.com) ,We can do some tests on path so that can merge earlier Thanks On Tue, Dec 10, 2024 at 7:47 PM Evgeny

Re: Add 64-bit XIDs into PostgreSQL 15

2024-12-17 Thread wenhui qiu
Hi Evgeny > we are causing the target solution to become staled I think the community is not positive enough about xid64,There is also a lot of resistance, such as the wal log becomes larger >I also hope you and the community will support attempts to retain the > whole xid64 solution in the con

Re: PoC: history of recent vacuum/checkpoint runs (using new hooks)

2024-12-22 Thread wenhui qiu
Hi Tomas This is a great feature. + /* + * Define (or redefine) custom GUC variables. + */ + DefineCustomIntVariable("stats_history.size", + "Sets the amount of memory available for past events.", + NULL, + &statsHistorySizeMB, + 1, + 1, + 128, + PGC_POSTMASTER, + GUC_UNIT_MB, + NULL, + NULL,

Re: New GUC autovacuum_max_threshold ?

2025-01-09 Thread wenhui qiu
HI Nathan Frédéric Yhuel On 1/7/25 23:57, Nathan Bossart wrote: > Here is a rebased patch for cfbot. AFAICT we are still pretty far from > consensus on which approach to take, unfortunately. > > For what it's worth, although I would have preferred the sub-linear > growth thing, I'd much rath

Re: POC: make mxidoff 64 bits

2025-01-20 Thread wenhui qiu
HI Maxim > Looks like there is a bit of a pause in the discussion. Here is a small update. Consider v12. > No major changes, rebase to the actual master and a squash of multiple commits to make a > patch set easy to reviewer. > AFAICs, we are reached a consensus on a core patch for switching to 64

Re: Increase NUM_XLOGINSERT_LOCKS

2025-01-22 Thread wenhui qiu
HI Japin Thank you for you test ,It seems NUM_XLOGINSERT_LOCKS 64 is great , I think it doesn't need to grow much,What do you think? Regards On Thu, Jan 23, 2025 at 10:30 AM Japin Li wrote: > On Sat, 18 Jan 2025 at 14:53, Yura Sokolov > wrote: > > Since it seems Andres missed my request

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-03 Thread wenhui qiu
Hi Thank you for your path,NUM_XLOGINSERT_LOCKS increase to 128,I think it will be challenged,do we make it guc ? On Fri, 3 Jan 2025 at 20:36, Yura Sokolov wrote: > 02.01.2025 10:05, Zhou, Zhiguo wrote: > > Hi all, > > > > I am reaching out to solicit your insights and comments on a recent

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-03 Thread wenhui qiu
Hi Sami Thank you for your path,it seems some path monitor vacuum status,Can we synthesize their good ideas together。 On Fri, 3 Jan 2025 at 02:24, Sami Imseih wrote: > Hi, > > After a recent question regarding tracking vacuum start_time in > pg_stat_all_tables [1], it dawned on me that this vie

Re: transaction lost when delete clog file after normal shutdown

2024-12-23 Thread wenhui qiu
>Why would you trust the other 99.999 TB if >something corrupted the clog >file? +1 On Tue, 24 Dec 2024 at 02:06, Jan Wieck wrote: > On 12/23/24 06:01, 章晨曦@易景科技 wrote: > > Yes, of course we can solve this by restoring from backup. > > But if the database volumn is large, say, 100TB or more, the

Re: transaction lost when delete clog file after normal shutdown

2024-12-23 Thread wenhui qiu
ged to salvage the data inside.Because there's a lot of that going on in the real world, and you can't change that by firing the dba. On Tue, Dec 24, 2024 at 9:45 AM wenhui qiu wrote: > >Why would you trust the other 99.999 TB if >something corrupted the clog > >file? >

Re: [Feature Request] INSERT FROZEN to Optimize Large Cold Data Imports and Migrations

2025-02-17 Thread wenhui qiu
HI Sébastien You can check out the email subject:Trigger more frequent autovacuums of heavy insert tables , I think it can alleviate the problem Thanks On Sat, Feb 15, 2025 at 3:13 AM Andres Freund wrote: > Hi, > > On 2025-02-13 10:52:31 +0100, Sébastien wrote: > > Introduce an INSERT FROZ

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-24 Thread wenhui qiu
Hi Melanie > relallvisible. It seems like we should make it consistent. Perhaps we > should just remove it from heap_vacuum_rel(). Then add an assert in > all these places to at least protect development mistakes. I think there's some objection to that. Thanks On Tue, Feb 25, 2025 at 7:35 AM Mel

Anti join confusion

2025-02-23 Thread wenhui qiu
Hi Richard Guo I found this path https://commitfest.postgresql.org/patch/3235/ already supports anti join , But I've found that in many cases it doesn't work.It always uses SubPlan Here's my testing process. ### create table join1 (id integer,name varchar(300),k1 integer); create table

Re: Anti join confusion

2025-02-24 Thread wenhui qiu
Mon, 2025-02-24 at 17:12 +0800, wenhui qiu wrote: > > Do we have plans for NOT IN subquery pull up? > > As mentioned before, that is not possible. > Best practice is to avoid NOT IN with subqueries altogether. > The result is usually not what people expect it to be. >

Re: [Feature Request] INSERT FROZEN to Optimize Large Cold Data Imports and Migrations

2025-02-18 Thread wenhui qiu
HI > Ok you mean that xid64 will remove the need for freezing... it's a way to see things. When xid64 is implemented, there will be no need to trigger vacuum_failsafe_age,it has a long enough time to vacuum freeze, it will have less of an impact on performance,I think that problem may be due to

Re: [Feature Request] INSERT FROZEN to Optimize Large Cold Data Imports and Migrations

2025-02-17 Thread wenhui qiu
priority and speed of the vaccum freeze > opeartion. > > Anyway, there should be a way to insert freeze data other than copy that > does not work with foreign tables. (INSERT into my_table select * from > foreign_table) > > > > Le lun. 17 févr. 2025 à 09:46, wenhui qiu a

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-28 Thread wenhui qiu
Hi > + * It could be the stats were updated manually and relallfrozen > > + * relpages. Clamp relallfrozen to relpages to avoid nonsensical > + * calculations. > + */ > + relallfrozen = Min(relallfrozen, relpages); > + pcnt_unfrozen = 1 - ((float4) relallfrozen / relpages); > + } > + Based on the c

Re: Trigger more frequent autovacuums

2025-03-07 Thread wenhui qiu
Sorry ,A wrong version of debug pcnt_visibletuples ,kindly please check the v3 attachment On Fri, Mar 7, 2025 at 5:37 PM wenhui qiu wrote: > Hi >The more accurate data I've found is tabentry->live_tuples; provides > the second version > > #Here's a simple test I

Re: Trigger more frequent autovacuums

2025-03-07 Thread wenhui qiu
ot;vacthresh: 457850.218750,anlthresh: 228950.109375, the join1 has 2289001.000000 reltuples, pcnt_unfrozen: 1.00, pcnt_visibletuples: 0.877414 ","","autovacuum worker",,0 On Fri, Mar 7, 2025 at 2:22 PM wenhui qiu wrote: > HI Nathan Bossart Melanie Plageman >

Trigger more frequent autovacuums

2025-03-06 Thread wenhui qiu
HI Nathan Bossart Melanie Plageman Firstly, congratulations on the submission of this path: https://commitfest.postgresql.org/patch/5320/ vacthresh = (float4) vac_base_thresh + vac_scale_factor * reltuples; anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples; vacinsthresh = (float

Re: Trigger more frequent autovacuums

2025-03-10 Thread wenhui qiu
suggestion or you provide a patch for a better algorithm,I actually just want to promote these calculation formulas. In fact, I highly admire the solution provided by SQL Server. On Fri, Mar 7, 2025 at 11:48 PM Melanie Plageman wrote: > On Fri, Mar 7, 2025 at 6:19 AM wenhui qiu wr

Re: Use CLOCK_MONOTONIC_COARSE for instr_time when available

2025-03-26 Thread wenhui qiu
> > HI > > > As far as I know, our usage of instr_time really needs the highest > resolution available, because we are usually trying to measure pretty > short intervals. You say that this patch reduces execution time, > and I imagine that's true ... but I wonder if it doesn't do so at > the cost

Re: POC: make mxidoff 64 bits

2025-03-26 Thread wenhui qiu
HI Maxim Orlov Heikki Linnakangas Thank you for working on it,A few more days is a code freeze.It seems like things have been quiet for a while, but I believe implementing xid64 is absolutely necessary. For instance, there’s often concern about performance jitter caused by frequent freezes. If

Re: GSoC 2025 - Looking for Beginner-Friendly PostgreSQL Project

2025-03-26 Thread wenhui qiu
Hi Kruti You can start by reviewing paths , https://commitfest.postgresql.org/53/ ,you can choose a path with fewer changes . Here's my little suggestion. Thanks On Thu, Mar 27, 2025 at 2:15 AM Kruti Popat wrote: > Hello PostgreSQL Developers, > > My name is Kruti Popat, and I am an IT