Re: a pool for parallel worker

2025-03-26 Thread Kirill Reshke
On Wed, 26 Mar 2025, 11:10 Andy Fan, wrote: > > Hi, > >> The boring thing for the pool is it is [dbid + userId] based, which > >> I mean if the dbid or userId is different with the connection in pool, > >> they can't be reused. To reduce the effect of UserId, I think if we can > >> start the poo

Re: a pool for parallel worker

2025-03-25 Thread Andy Fan
Andy Fan writes: > Hi, >>> The boring thing for the pool is it is [dbid + userId] based, which >>> I mean if the dbid or userId is different with the connection in pool, >>> they can't be reused. To reduce the effect of UserId, I think if we can >>> start the pool with a superuser and then swit

Re: a pool for parallel worker

2025-03-25 Thread Andy Fan
Hi, >> The boring thing for the pool is it is [dbid + userId] based, which >> I mean if the dbid or userId is different with the connection in pool, >> they can't be reused. To reduce the effect of UserId, I think if we can >> start the pool with a superuser and then switch the user information

Re: a pool for parallel worker

2025-03-25 Thread Kirill Reshke
r cache, > vfd cache and fork/exit syscall itself. > > I am thinking if we should preallocate (or create lazily) some backends > as a pool for parallel worker. The benefits includes: > > (1) Make the startup cost of a parallel worker lower in fact. > (2) Make the core most su

Re: a pool for parallel worker

2025-03-24 Thread Andy Fan
>> vfd cache and fork/exit syscall itself. >> >> I am thinking if we should preallocate (or create lazily) some backends >> as a pool for parallel worker. The benefits includes: >> >> (1) Make the startup cost of a parallel worker lower in fact. >> (2) Mak

Re: a pool for parallel worker

2025-03-24 Thread James Hunter
exit syscall itself. > > I am thinking if we should preallocate (or create lazily) some backends > as a pool for parallel worker. The benefits includes: > > (1) Make the startup cost of a parallel worker lower in fact. > (2) Make the core most suitable for the cases where executor

a pool for parallel worker

2025-03-11 Thread Andy Fan
(or create lazily) some backends as a pool for parallel worker. The benefits includes: (1) Make the startup cost of a parallel worker lower in fact. (2) Make the core most suitable for the cases where executor need to a new worker to run a piece of plan more. I think this is needed in some data