of
portable macros for the same.
Please let me know on what we think of the above.
Regards,
Nikhil
--
Nikhil Sontakke
2ndQuadrant - PostgreSQL Solutions for the Enterprise
https://www.2ndQuadrant.com/
0002-libpq_batch_tests_community_master.v17.patch
Description: Binary data
0001-libpq_batch
Hi Arseny,
> I hadn't checked whether my concerns where addressed in the latest
> version though.
>
I'd like to believe that the latest patch set tries to address some
(if not all) of your concerns. Can you please take a look and let me
know?
Regards,
Nikhil
--
Nikhil S
A, updated patch set.
Regards,
Nikhil
> regards
>
> --
> Tomas Vondra http://www.2ndQuadrant.com
> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Nikhil Sontakke
2ndQuadrant - PostgreSQL Solutions for the Enterprise
https://www.2ndQuadr
omas Vondra http://www.2ndQuadrant.com
> > PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
>
>
> --
> Nikhil Sontakke http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
>
> --
> Tomas Vondra http://www.2ndQuadrant.com
> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
0001-Cl
e
> +* error out
> + */
> + if (TransactionIdIsValid(CheckXidAlive) &&
> + !TransactionIdIsInProgress(CheckXidAlive) &&
> + !TransactionIdDidCommit(CheckXidAlive))
> + ereport(ERROR,
> + (errcode(ERRCODE_TRANSACTION_ROLLBACK),
> +errmsg("transaction aborted during system
> catalog scan")));
>
> Probably centralize checks in one function? As well as 'We don't expect
> direct calls to heap_fetch...' ones.
>
>
> P.S. Looks like you have torn the thread chain: In-Reply-To header of
> mail [1] is missing. Please don't do that.
>
That wasn't me. I was also annoyed and surprised to see a new email
thread separate from the earlier containing 100 or so messages.
Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
k, added unlikely() checks in the heap_* scan APIs.
Revised patchset attached.
Regards,
Nikhils
> Greetings,
>
> Andres Freund
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
0001-Cleaning-up-of-flags-in-
>
> Hmm, lemme see if we can do it outside of the plugin. But note that a
> plugin might want to decode some 2PC at prepare time and another are
> "commit prepared" time.
>
The test_decoding pg_decode_filter_prepare() API implements a simple
filter strategy now. If the GI
onIdDidCommit(CheckXidAlive)
>
> What do you think?
>
tqual.c does seem to mention this for a non-MVCC snapshot, so might as
well do it this ways. The caching of fetched XID should not make these
checks too expensive anyways.
>
> I think it'd also be good to add assertions to codepaths not g
d. Andres generally doesn't like this
approach :-), but there are no timing/interlocking issues now, and the
sleep just helps us do a concurrent rollback, so it might be ok now,
all things considered. Anyways, it's an additional patch for now.
Comments, feedback appreciated.
Rega
Hi Andres,
>> We can find out if the snapshot is a logical decoding one by virtue of
>> its "satisfies" function pointing to HeapTupleSatisfiesHistoricMVCC.
>
> I think we even can just do something like a global
> TransactionId check_if_transaction_is_alive = InvalidTransactionId;
> and just set
, indeed. We'd need to make the
"being-currently-decoded-XID" visible to these systable_* functions
and then this scheme will work.
Regards,
Nikhils
> Greetings,
>
> Andres Freund
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
, then the catalog read will return us T1 with one
column C1. The catalog scan will NOT error out but will return
metadata which causes the insert-decoding change apply callback to
error out. The point here is that in some cases the catalog scan might
not error out and might return inconsi
Hi Andres,
> So what if we, at the begin / end of cache miss handling, re-check if
> the to-be-decoded transaction is still in-progress (or has
> committed). And we throw an error if that happened. That error is then
> caught in reorderbuffer, the in-progress-xact aborted callback is
> called, an
s that only see command IDs prior to the current one
> so that incompletely-made changes by the last CID aren't seen if an
> abort happens. I think there is a good chance that a full solution
> involves more than one of these things, and maybe some other things I
> haven't thought about. These are ideas, not a plan.
>
I will think more on the above lines and see if we can get something workable..
Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
e
present in the core as of now. I am going to submit it in the upcoming
commitfest.
Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
gid_length.patch
Description: Binary data
to go a little beyond the current "line" variable size.
This caused the strcmp to fail leading to initdb refusing to
initialize any data directories at all.
PFA, a patch which uses MAXPGPATH for the variable size.
Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadr
>> Anyways, will now wait for the next commitfest/opportunity to try to
>> get this in.
>
> It looks like this patch should be in the Needs Review state so I have
> done that and moved it to the next CF.
>
Thanks David,
Regards,
Nikhils
--
Nikhil Sontakke
7jxh43fhqnevhji%40alap3.anarazel.de
Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services
kend to the group, but
> that's not what's happening. We join the group on the first call and
> then we only tweak the decodeLocked flag.
>
True.
>
> 7) I propose minor changes to a couple of comments.
>
Ok, I am looking at your provided patch and incorporating relevant
changes from it. WIll submit a patch set soon.
Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services
cts never need it, and it seems rather wasteful to allocate
> 200B when the rest of the struct has only ~100B. This is particularly
> problematic considering ReorderBufferTXN is not spilled to disk when
> reaching the memory limit. It needs to be allocated ad-hoc only when
> actuall
>
> Quick thought: Should be simple to release lock when interacting with network.
I don’t think this will be that simple. The network calls will typically happen
from inside the plugins and we don’t want to make plugin authors responsible
for that.
> Could also have abort signal lockers.
in terms of
schema changes.
Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services
Hi Petr, Andres and Tomas
>>> Thanks. I think the README is a good start, but I think we also need to
>>> improve the comments, which is usually more detailed than the README.
>>> For example, it's not quite acceptable that LogicalLockTransaction and
>>> LogicalUnlockTransaction have about no com
whole patchset is
> visibly versioned and sorts in the correct order.
>
I did try to use *_Number.patch to convey the sequence, but admittedly
it's pretty lame.
I will re-submit with "git format-patch" soon.
Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQ
pieces and re-submit.
>
> On 2018-02-06 17:50:40 +0530, Nikhil Sontakke wrote:
>> @@ -46,6 +48,9 @@ typedef struct
>> boolskip_empty_xacts;
>> boolxact_wrote_changes;
>> boolonly_loca
tTxn() for such a transaction is my query? I don't see
the need for doing that for skipped transactions..
Will continue to look at this and will add this scenario to the test
cases. Further comments/feedback appreciated.
Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services
e XID is still ok
to decode, read the catalog and unlock. Abort/Commit transaction
processing could take this in EXCLUSIVE mode.
As mentioned above, the plugin API which takes this lock will be smart
enough to be a NOOP if the transaction is not running (i.e we are not
doing 2PC decoding or str
rt handling clean.
I will cook up something along the above lines unless there are any
objections to this approach.
Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services
with cassert enabled and things run fine. Also,
since it only sets the flag for system/user catalog tables and that
too ONLY in the logical decoding environment, it does not cause any
performance issues in normal circumstances.
Regards,
Nikhils
--
Nikhil Sontakke http://www.2nd
ith a separate
>> discussion thread.
>
> Are you working on that as well?
Sure, I was planning to work on that after getting the documentation
for this patch out of the way.
Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services
this needs to be a separate patch along with a separate
discussion thread.
Peter, I will submit a follow-on patch with documentation changes
soon. But this patch is complete IMO, with all the required 2PC
logical decoding functionality.
Comments, feedback is most welcome.
Reg
5, Peter Eisentraut
wrote:
> On 12/7/17 08:31, Peter Eisentraut wrote:
>> On 12/4/17 10:15, Nikhil Sontakke wrote:
>>> PFA, latest patch for this functionality.
>>
>> This probably needs documentation updates for the logical decoding chapter.
>
> You need the attached pa
Another test (actually, functionality) that might come in handy, is to
have a way for DDL to be actually carried out on the subscriber. We
will need something like pglogical.replicate_ddl_command to be added
to the core for this to work. We can add this functionality as a
follow-on separ
ed. This has simplified all that unwanted snapshot push/pop code,
which is nice.
Regards,
Nikhils
On 30 November 2017 at 16:08, Nikhil Sontakke wrote:
> Hi,
>
>
>>> So perhaps better approach would be to not return
>>> HEAPTUPLE_DEAD if the transaction id is newer than t
Hi,
>> So perhaps better approach would be to not return
>> HEAPTUPLE_DEAD if the transaction id is newer than the OldestXmin (same
>> logic we use for deleted tuples of committed transactions) in the
>> HeapTupleSatisfiesVacuum() even for aborted transactions. I also briefly
>> checked HOT pruni
changes, unless it's immediately
> followed by COMMIT PREPARED of the same xid. Because xacts between the two
> cannot see changes the prepared xact made to the catalogs.
>
> - For the same reason we need to ensure that the historic snapshot used to
> decode a 2PC xact that made catalog changes isn't then used for subsequent
> xacts between the prepare and commit. They'd see the uncommitted catalogs of
> the prepared xact.
>
Yes, we will do TeardownHistoricSnapshot and syscache flush as part of
the cleanup for 2PC transactions.
Regards,
Nikhils
> --
> Craig Ringer http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services
ion is dropped or any other
issues. Then this can work fine, IMO.
Thoughts? Holes in the theory? Other issues?
I am attaching my latest and greatest WIP patch with does not contain
any of the above abort handling yet.
Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services
2pc_logical_22_11_17.patch
Description: Binary data
39 matches
Mail list logo
