Hello.
Let me propose methods to clean master off unused gtid domains.
The issue is quite practical, as a couple of references on the subject
line tell. Either of them narrates a scenario of two default domain-id masters
serving to one slave initially by "legacy" non-gtid protocol.
When later the
Kristian, salute.
Let me jump at once to the high-level specification, afterwards I am
remarking on or dwelling into specific parts of the text.
Your last reply made it explicit that you mean totally strict setup
on master (p.(2) of the following list):
KN> (1). We want the master to "forget ab
>> If my concern is practical we may consider *optionally* strict
>> delete domain FLUSH LOGs. The errored out version would maintain a
>
> In that case, I would compare to SET GLOBAL gtid_binlog_state.
Fair enough.
>
> Currently, this is even more restricted, it is only allowed when the binlog
>
Kristian, hello.
Now to the implementation matter,
> The procedure to fix it will then be:
>
> 1. FLUSH BINARY LOGS, note the new GTID position.
>
> 2. Ensure that all slaves are past the problematic point with
> MASTER_GTID_WAIT(). After this, the old errorneous binlog files
> are no
> longer ne
>> 1. Take note of @@global.gtid_binlog_state
>> 2. Ensure that all slaves are past the last event of being deleted domain 'd'
>> 3. PURGE BINARY LOGS DELETE DELETE 'd'
>>
>> The effect of the last step would include purging all the binary log
>> files plus a planned implicit FLUSH LOGS discard
Kristian, Sergey, hello.
While I'm still studying the Mariadb BGC (binlog group commit) let me
jump in with few comments.
> Sergey Petrunia writes:
>
>> == Some background ==
>>
>> "group commit with binlog" feature needs to accomplish two goals:
>>
>> 1. Keep the binlog and the storage engine i
Kristian,
> andrei.el...@pp.inet.fi writes:
>
>>> 2. To make START TRANSACTION WITH CONSISTENT SNAPSHOT actually correctly
>>> synchronise snapshots between multiple storage engines (MySQL does not have
>>> this, I think).
>>
>> (Offtopic, but anyway what it is? Multi-engine transaction with this
Hello, Simon, Kristian.
(The mail was meant to be sent out yesterday, but it got stuck in
my outgoing box).
> Simon Mudd writes:
>
>> ids. Obviously once all appropriate bin logs have been purged
>> (naturally by other means) then no special processing will be needed.
>
> Right. Hence my origina
Hello, everybody, Kristian.
>>> 1. Take note of @@global.gtid_binlog_state
>>> 2. Ensure that all slaves are past the last event of being deleted domain
>>> 'd'
>>> 3. PURGE BINARY LOGS DELETE DELETE 'd'
While I was selecting between two candidates where to settle the new
option to discard the
> Andrei Elkin writes:
>
>> And really why not
>>
>> 3. SET @@GLOBAL.gtid_binlog_state=list-without-d;
>
> The main issue I see with this if the master is actively adding new
> transactions (in other domains than d). It will be hard for the user
> to know
&g
Kristian Nielsen writes:
> andrei.el...@pp.inet.fi writes:
>
>> Then a function to discard a domain term would do:
>>
>> SET @@gtid_binlog_state=gtid_discard_domain(@@gtid_binlog_state,'d')
>>
>> While this time it would be new object introduced still it's of var
>> setting semantics and might b
Simon, Kristian, salute.
> Simon, thanks for your detailed answer.
>
> I see your point on having access to powerful tools when they are needed,
> even when such tools can be dangerous when used incorrectly. It reminds me
> of the old "goto considered harmful" which I never agreed with.
>
> It occ
Hello.
I've completed with the patch which passes few tests.
However I had to make one concession with regard to replication.
Actually..
>>> So I see the DELETE DOMAIN (MariaDB) or “remove old UUID” (MySQL)
>>> type request
>>> to be one that means the master will only pretend that it can serve
Simon, hello.
> Hi Andrei,
>
>> On 25 Sep 2017, at 20:17, andrei.el...@pp.inet.fi wrote:
>
> ...
>> the "vanilla" FLUSH-LOGS is not binlogged by decision commented in
>> reload_acl_and_cache():
>
> In the normal case I think it makes sense to not trigger another flush
> and to not binlog the comma
Kristian, Simon, hello.
Replication side of FLUSH BINARY LOGS DELETE_DOMAIN_ID is actually bound
to another requirement specified in early mails.
The command is successful only *after* the user has run
PURGE BINARY LOGS to 'the-first-log-free-of-old-domains'
which is not replicated. Therefore
Kristian, thanks for more remarks!
>>> If you “forget" the domain on the upstream server what happens if
>>> there
>>> are downstream slaves? I think you’ll break replication if they
>>> disconnect
>>> from this box and try to reconnect. Their GTID information will no
>>> longer match.
>>> IMO an
Kristian, hello.
The patch is ready for review and can be located on bb-10.1-andrei,
https://github.com/MariaDB/server/pull/460
(Ignore https://github.com/MariaDB/server/pull/459 which specified 10.2
base by mistake)
In case you won't be able to, I'll find replacement, no worries.
Have a good ti
t has been requested a number of
> times. You might consider taking over this task from Jira, which (if I
> understand the description correctly) you have basically solved (if with a
> different/better syntax):
>
> https://jira.mariadb.org/browse/MDEV-9241
>
> - Kristian
Kristian, hello.
The patch is polished to address your comments and Ian's editorial work.
I apologize for a possible invinient delay with the final version. It's here:
https://github.com/MariaDB/server/pull/460/commits/56b000b2e7d9c4dec61429f0ff7affe9c75409ca
as well as appended to the end of th
Kristian, howdy.
Thanks for a simple CHANGE MASTER ... IGNORE_SERVER_IDS
that you remind us about! (This time evaded myself alone :-))
It perfectly covers a cluster circular case.
What motivated me to consider this option for looking for duplicates
also in gtid_binlog_pos was the following observ
Sachin, hello.
I've read through your latest patch as well as did so to the current
mail thread.
On previous rounds there were few notes raised by Kristian. I see that
they got addressed.
I got few requests of myself as we spoke yesterday
1. Let's count the transactional groups as well for con
Kristian, hello.
Thanks for looking at fixes so promptly!
> andrei.el...@pp.inet.fi writes:
>
>> commit 3cebb54e6387a7eace1757c82ed0efd6e11590b9
>> Author: Andrei Elkin
>> Date: Fri Feb 9 15:00:23 2018 +0200
>>
>> MDEV-12746 rpl.rpl_parallel_
Hello.
Kristian Nielsen writes:
> andrei.el...@pp.inet.fi writes:
>
>> First the parent errros out goes to `finish_event_group()' but it's
>> possible it does not have yet the child in its `subsequent_commits_list'
>
>> So I understood so far that the retrying worker needs to check
>> `stop_on_
Kristian,
The improved patch is here for more comments if you will have.
Cheers,
Andrei
commit 564a4efc817f9f6c54da00987e4aa5164f4c0d97
Author: Andrei Elkin
Date: Fri Feb 9 15:00:23 2018 +0200
MDEV-12746 rpl.rpl_parallel_optimistic_nobinlog fails committing
out of
Sachin, howdy.
While the latest patch is pretty I still see potential to improve it,
please find A[0-9]+ prefixed annotations below.
Thank you for addressing earlier comments!
> Hi Andrei, Kristian !
>
> Sorry for late reply (I was on vacation , and there was
> very less 10.3 sprint)
>
> On Sat,
Howdy, Monty.
First, to congrat you and the Ali colleagues on this nice
piece of functionality! Doubtlessly the idea is generic and we shall see
creative followups at some time.
Now having MDEV-15782 we can identify some immediate issues, as well as
ideas how to cope with them.
It was a surprise
Kristian, Sergey, hello!
> Sergey Petrunia writes:
>
>> == Symptoms ==
>> When one runs a parallel slave (mode=conservative) and replicates DML for
> Nice analysis. Normally it is guaranteed to be safe to start the next
> transaction in parallel with the commit step of a prior. But as you
> de
Sergei, hello.
> Hi, andrei.elkin!
>
> On May 30, andrei.el...@pp.inet.fi wrote:
>> revision-id: d9913834cebe4ed8494b3d76762cd882788a8fd5
>> (mariadb-10.1.33-28-gd9913834ceb)
>> parent(s): c1698e8dc50f0c342c1a6886426ba1d43396cd1e
>> author: Andrei Elkin
>&g
Sergei,
> Hi, Andrei!
>
> On Jun 12, andrei.el...@pp.inet.fi wrote:
>> >
>> > Did you already test that your commit actually fixes the issue?
>>
>> Like I said, I did not try after we found it's NFS binlog. I've done it
>> right now, but my attempt lasted minutes while in the user's case
>> apart
Hello, Kristian.
I read through your analysis of the case to agree with
the plan and its motivation:
MDEV-6608> Generally, it is a very bad idea to destroy data during
crash recovery ... try an approach that leaves the crashed
binlog file intact, and instead ensures t
Sachin, hello.
> Hi Everyone!
>
> Suppose this case
>
> CREATE USER
> user1@localhost IDENTIFIED BY 'BsG9#9.cem#!85',
> user2@localhost IDENTIFIED BY 'x';
>
> user2 has too short passowrd which will give error (if we use security plugin)
>
> IN the case of *mariadb* we will create user1 and wo
84f8810d0e64c4c
>> (mariadb-10.1.34-8-g9b453c1ebf4)
>> parent(s): c09a8b5b36edb494e2bcc93074c06e26cd9f2b92
>> author: Andrei Elkin
>> committer: Andrei Elkin
>> timestamp: 2018-09-07 20:36:16 +0300
>> message:
>>
>> MDEV-10963 Fragmented BIN
98f286f6b
> author: Andrei Elkin
> committer: Andrei Elkin
> timestamp: 2018-10-03 15:42:12 +0300
> message:
>
> MDEV-17346 parallel slave start and stop races to workers disappeared
>
> The bug appears as a slave SQL thread hanging in
> rpl_parallel_thread_pool::get_thre
Kristian, salute.
> Hi Andrei!
>
> Can you review this patch?
Thanks for taking care of this critical and complicated issue!
I studied the patch to have understand its idea of basically
implementing a roll-back to the slave gtid state as the slave
transaction participant resource.
This would be
The 10.1 patch is good.
But one question,
> Hi Andrei!
>
> Can you review this patch?
...
>> @@ -646,9 +658,9 @@ rpl_slave_state::record_gtid(THD *thd, const rpl_gtid
>> *gtid, uint64 sub_id,
>>/* `break' does not work inside DBUG_EXECUTE_IF */
>>g
Sergei, howdy.
> Hi, Andrei!
>
> On Sep 21, andrei.el...@pp.inet.fi wrote:
>
>> > Please, make it
>> >
>> > BINLOG CONCAT(@binlog_fragment_0, @binlog_fragment_1)
>> >
>> > that'll work with no questions asked, everybody understands what it
>> > means. The parser doesn't need to accept an arbitra
Alexander, Sachin, hello.
> Hi Andrei,
>
> We need your help on:
>
> MDEV-10362 mysqlbinlog verbose output cannot be decoded
>
>
> The problem is that "mysqlbinlog -vvv" writes
> comments right inside the BINLOG statement,
> between base64 chunks:
>
>
> BINLOG '
> kxiEVxMBLQAAAMUBABMAA
still like more the explicit cleanup though with your optimization
below.
>
> Note, we cannot auto-reset variables in the syntax with CONCAT(),
> DEFRAGMENT() or whatever, because functions cannot do it to their
> arguments.
>
> On Oct 18, Andrei Elkin wrote:
>> revision-i
Sergei, good morning.
>> > created again, may be more than once. Freeing @a and @b and the
>> > concatenated copy as soon as possible will save over 6GB in RAM
>> > usage.
...
> 8. free variables
>
> And I suggest to do it like
>
> 1. allocate >3GB for both variables
> 2. allocate >3GB for a combi
time
>zone-related system variables to their previous values.
>
>
> Should the option be around (but it's gone)
>
> SET ONE_SHOT @binlog_fragment_$i=`value`
>
> would be it.
>
> I still like more the explicit cleanup though with your optimization
> below.
Kristian, salute!
> Thanks for the interesting idea and detailed description, Andrei! I have
> written some initial comments inline, below:
It was a great piece of input, few notes are taken. I stumbled at one
point though where I struggled to find out what made you apparently
misled (see your ex
Kristian,
> andrei.el...@pp.inet.fi writes:
>
>> I am yet to check closer thd_rpl_deadlock_check(), perhaps that's why
>
> Oh.
>
> I definitely think you should get well acquainted with existing parallel
> replication in MariaDB to undertake a project as ambitious as this one.
> There are a number
Let me expand on a couple of points in the last reply.
> Kristian,
>
>> andrei.el...@pp.inet.fi writes:
>>
>>> I am yet to check closer thd_rpl_deadlock_check(), perhaps that's why
>>
>> Oh.
>>
>> I definitely think you should get well acquainted with existing parallel
>> replication in MariaDB to
Actually the optimistic statement scheduling is restricted
to the Write_log_events/INSERT-Query_log_event types.
Unlike inter-transaction conflicts which can be cured through
proper victimization and retry this time it's the intra- case
where retrying won't help when the 2nd event is separated from
Salute, Kristian.
> andrei.el...@pp.inet.fi writes:
>
>> We can notice that in the pipeline method any attempt to start waiting for a
>> granted lock
>> to another branch needs interception and erroring out.
>> We are bound to this because the grated lock is to be released only after
>> 2pc.
>
>
Kristian, hello.
The patch is great and instructive in many ways.
Thanks!
There is something to improve in the test organization, like
to base two tests of
> storage/rocksdb/mysql-test/rocksdb_rpl/t/mdev12179.test
> storage/tokudb /mysql-test/tokudb_rpl /t/mdev12179.test
on a common parent.
I
Salute, Kristian!
> Hi Andrei,
>
> Thanks for review! I rebased the patch on 10.4, ran it through another
> buildbot run, and pushed it to 10.4.
>
> I think with this patch I'll close MDEV-12147, ok?
Committed :-)
>
> I wrote up the below documentation, I'm planning on adding it to the
> knowled
(cache, file, SIZE_T_MAX);
> }
>
why is the preference specifically?
I looked around to find something like
https://en.wikipedia.org/wiki/Compile_time_function_execution
... but I somewhat doubt you meant that. Or did you really?
Now to my acknowledgments and comments...
>
&
Sergei,
One correction to just sent reply,
in fact I've removed this:
>> --- a/sql/log_event.h
>> +++ b/sql/log_event.h
>> @@ -749,6 +749,7 @@ typedef struct st_print_event_info
>> that was printed. We cache these so that we don't have to print
>> them if they are unchanged.
>>*/
>
Привет, Алексей!
Я просматриваю патч. Где-нить через 2-3 часа должны быть комментарии от
меня. Можно будет созвониться. И сразу - полагаю, ты сейчас прямо
усердно трудишься над тестами (в патче не присутствующими).
Andrei.
holyf...@askmonty.org (Alexey Botchkov) writes:
> revision-id: b428e822d
Aleksey, hello.
> Hi, Sergei,
>
> Thank you for observations! This task is in progress. While doing it I found
> out that RBR doesn't replicate timestamp fractions.
> That is: it gets seconds from event timestamp marker and adds fractions from
> system time. This causes bugs in System Versioning
Sujatha, hello.
The patches look good though I have a concern in ..
> [
> https://jira.mariadb.org/browse/MDEV-17260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=126440#comment-126440
> ]
>
> Sujatha Sivakumar commented on MDEV-17260:
> ---
Sujatha, hello.
The patch looks good, I only have a suggestion to the test for
better readability, within the quoted patch below.
Thanks for a good piece of work!
Cheers,
Andrei
>commit 8ae7a5eb03725441393c353e4e203f80b666c035
>Author: Sujatha Sivakumar
>Date: Tue Apr 23 19:47:59 2019 +05
Sachin, howdy.
> revision-id: 647c22692ad1fe466eb14b749520cc520807d4fa
> (mariadb-10.1.38-114-g647c22692ad)
> parent(s): 6c5e4c9bc0d9ac30f7ec7ee334630bacb58687ba
> author: Sachin
> committer: Sachin
> timestamp: 2019-04-23 17:58:03 +0530
> message:
>
> MDEV-17588 replicate-do filters cause errors
Sujatha, salute!
Thanks for a detailed description! I would add up to it a minor piece
> revision-id: 53d0e9217943719b806ef9fa1cac0a699df4839c
> (mariadb-10.1.39-31-g53d0e921794)
> parent(s): 47637a3dd13d19e897a7cbfd1499f1bf3b2fdb2a
> author: Sujatha
> committer: Sujatha
> timestamp: 2019-05-17
Sujatha, howdy.
Thank you for spelling it out for me!
> Hello Andrei,
...
>>>
>>> Analysis:
>>> =
>>> Enabling "replicate_annotate_row_events" on slave, Tells the slave to write
>>> annotate rows events received from the master to its own binary log. The
>>> received annotate events are a
Sachin, salute!
The patch looks good, but I think it misses some verbosity in few
places, please read on.
> revision-id: 321a43d9ac586034790407dc519c72e90b3bff81
> (mariadb-10.1.39-54-g321a43d9ac5)
> parent(s): b003b0c934cf6b59358e31144c4f69cf34622fb8
> author: Sachin
> committer: Sachin
> times
Sachin, hello.
The patch is alright.
Thanks for a good piece of work!
Andrei
> revision-id: 5bfc8128c8381e9347589aa0e27cd4fbd66ef19d
> (mariadb-10.1.39-57-g5bfc8128c83)
> parent(s): 2e0fb93e6f3359ffe146f309e32c59922400b7c4
> author: Sachin
> committer: Sachin
> timestamp: 2019-06-12 15:58:44 +0
Sachin, salve.
> revision-id: 8b545d3d41416772203cb98f06243873a4f0d8f7
> (mariadb-10.4.5-24-g8b545d3d414)
> parent(s): e35676f5557d68c7b51ba47aa73dcdf72eafa436
> author: Sachin
> committer: Sachin
> timestamp: 2019-05-31 12:02:37 +0530
> message:
>
> MDEV-7409 On RBR, extend the PROCESSLIST info
Sujatha, howdy.
Thanks for a good piece of work!
The fact that it's actually an upstream contribution only pleases more :-),
and to that regard we need a commit id to refer to (and to the people
behind that work :-)).
> revision-id: fb619954867232b3a733213a2703b2a425dff8e9
> (mariadb-5.5.64-19-g
>> I would also raise another but relevant topic of maintaining
>>
>>gtid_"executed"_pos
>>
>> which is an union of all GTID executed regardless of their arrival
>> method. E.g some of foreign (to the recipient server) domains gtid:s may
>
>> The master potentially could be (made) interested in
Sujatha, Kristian, howdy.
Sorry, I managed to sent my view without actually cc-ing Kristian, as claimed.
X-From-Line: nobody Wed Nov 27 19:27:51 2019
From: Andrei Elkin
To: sujatha
Cc: ,
Subject: Re: f72427f463d: MDEV-20923:UBSAN: member access within address
Let me
ent(s): 77f4a1938f2a069174e07b3453fa0f99ba171a2e
author: Sujatha
committer: Andrei Elkin
timestamp: 2020-03-04 14:45:01 +0200
message:
MDEV-21469: Implement crash-safe logging of the user XA
Description: Make XA PREPARE, XA COMMIT and XA ROLLBACK statements crash-safe.
Implementation:
In order to ensure consistent replicati
Привет, Сергей!
О чем этот "суппорт", не обьяснишь одной фразой?
Спасибо!
Андрей
psergey writes:
> revision-id: 5d0e4ce291ae940feaa8398435158dc56a3db3c4 ()
> parent(s): d504887718112e211544beca0e6651d5477466e1
> author: Sergei Petrunia
> committer: Sergei Petrunia
> timestamp: 2020-03-12 10:2
t;
> +, -. *, /, ++, --, <, >, ==, !=
`%' - the reminder is very popular. `/' I read would result in a
quotient (the whole part of devision).
>
> To implement expressions I want to use Byson to make it easier to rewrite
> parser if needed
>
> I would be glad
Nice work, Sergei!
Just the same
> I like how it sounds, just please don't destroy `--manual-gdb`, I am using it
> for debugging from IDE
+1
>
> --
> Yours truly,
> Nikita Malyavin
>
Cheers,
Andrei
___
Mailing list: https://launchpad.net/~maria-deve
Howdy Kristian,
> Hi Sachin, Andrei,
>
> I'm looking at test cases for START ALTER / Lag-free Alter on Slave. And I
> happened to notice something in test case rpl.rpl_start_alter_bugs:
>
> --connection master
> set global binlog_alter_two_phase=true;
> # ...
> ALTER TABLE t1 DROP PRIMARY
67 matches
Mail list logo
