Hi, andrei.elkin! On Oct 18, andrei.el...@pp.inet.fi wrote: > Sergei, > > > Hi, Andrei! > > > > First, when I run it I got: > > > > Warning: 112 bytes lost at 0x7f212f26a670, allocated by T@0 at > > sql/sql_binlog.cc:68, sql/sql_binlog.cc:181, sql/sql_parse.cc:5625, > > sql/sql_parse.cc:7465, sql/sql_parse.cc:1497, sql/sql_parse.cc:1124, > > sql/sql_connect.cc:1330, sql/sql_connect.cc:1243 > > > > so you have a memory leak with your thd->lex->comment.str > > Turns out LEX cleaned up at the end of decoded event execution. Thanks > for alerting! > > Btw, do you compile with clang or gcc? Asking seeing my stack unresolved.
gcc. It should not matter, I think. You need to build with -g and have addr2line in the path. > > Also, I'm thinking whether BINLOG (in this last form) could > > automatically clear its variables @a and @b. Without an explicit SET > > @a=NULL. It's a bit weird to have such a side effect. But it'll help > > to save memory - the event size it over 3GB, right? So you'll have > > it split in two variables. Then you create a concatenated copy (it's > > over 3GB more), then you decode it (already more than 9GB in > > memory), and then it's executed where copies of huge blob values are > > created again, may be more than once. Freeing @a and @b and the > > concatenated copy as soon as possible will save over 6GB in RAM > > usage. > > Well, in the regular case of replaying all three phases of > SET, BINLOG, UNSET > time between the last two is not really big. It's not a question of time, but of how much memory you need *at the same time*. Now you do 1. allocate >3GB for both variables 2. allocate >3GB for a combined concatenated event 3. allocate ~3GB for a base64-decoded event 4. allocate memory for copies of blobs (2*3GB?) --- now you have 15GB allocated 5. free blobs 6. free base64 decoded event 7. free concatenated event 8. free variables And I suggest to do it like 1. allocate >3GB for both variables 2. allocate >3GB for a combined concatenated event 3. free variables 4. allocate ~3GB for a base64-decoded event 5. free concatenated event 6. allocate memory for copies of blobs (2*3GB?) --- now you have 9GB allocated 7. free blobs 8. free base64 decoded event So the peak memory consumption is >6GB less. Regards, Sergei Chief Architect MariaDB and secur...@mariadb.org _______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
