Re: LMDB: issue with mdb_cursor_del

2017-10-21 Thread Леонид Юрьев
2017-10-20 16:34 GMT+03:00 : > Hi Leonid, > >> 1) With MDB_WRITEMAP the mdb_mid2l_append() will be used. >> Internally mdb_mid2l_append() don't check for duplicated, but just >> append a given page number to the list. >> Therefore when database opens with MDB_WRITEMAP this bug leads to >> duplicat

Re: LMDB: issue with mdb_cursor_del

2017-10-21 Thread Леонид Юрьев
Hi, Timur. There two different cases: with MDB_WRITEMAP, and without it. 1) With MDB_WRITEMAP the mdb_mid2l_append() will be used. Internally mdb_mid2l_append() don't check for duplicated, but just append a given page number to the list. Therefore when database opens with MDB_WRITEMAP this bug le

Re: LMDB: issue with mdb_cursor_del

2017-10-20 Thread timur . kristof
Hi Leonid, > 1) With MDB_WRITEMAP the mdb_mid2l_append() will be used. > Internally mdb_mid2l_append() don't check for duplicated, but just > append a given page number to the list. > Therefore when database opens with MDB_WRITEMAP this bug leads to > duplicates inside the dirty-list and then (se

Re: LMDB: issue with mdb_cursor_del

2017-10-19 Thread timur . kristof
On Thu, 2017-10-19 at 08:12 +0200, Hallvard Breien Furuseth wrote: > On 18/10/17 22:44, timur.kris...@gmail.com wrote: > > On Wed, 2017-10-18 at 20:08 +0300, Леонид Юрьев wrote: > > > Sure this will not create problems when working without > > > MDB_WRITEMAP. > > > But in MDB_WRITEMAP mode duplicat

Re: LMDB: issue with mdb_cursor_del

2017-10-18 Thread Hallvard Breien Furuseth
On 18/10/17 22:44, timur.kris...@gmail.com wrote: On Wed, 2017-10-18 at 20:08 +0300, Леонид Юрьев wrote: Sure this will not create problems when working without MDB_WRITEMAP. But in MDB_WRITEMAP mode duplicates in the dirty-list seems may leads some side affects. Therefore in MDBX I had planned

Re: LMDB: issue with mdb_cursor_del

2017-10-18 Thread Леонид Юрьев
2017-10-18 19:40 GMT+03:00 : > Maybe mdb_page_dirty could be allowed to simply accept -1 as a return > value from insert() and that could fix it? Since the page is already > listed in the txn's dirty list, it shouldn't really matter. Or am I > misunderstanding the consequences of this? Sure this

Re: LMDB: issue with mdb_cursor_del

2017-10-18 Thread timur . kristof
On Wed, 2017-10-18 at 20:08 +0300, Леонид Юрьев wrote: > 2017-10-18 19:40 GMT+03:00 : > > Maybe mdb_page_dirty could be allowed to simply accept -1 as a > > return > > value from insert() and that could fix it? Since the page is > > already > > listed in the txn's dirty list, it shouldn't really m

Re: LMDB: issue with mdb_cursor_del

2017-10-18 Thread timur . kristof
On Wed, 2017-10-18 at 06:18 +0200, Hallvard Breien Furuseth wrote: > On 16/10/17 14:56, Hallvard Breien Furuseth wrote: > > On 16. okt. 2017 13:43, Леонид Юрьев wrote: > > > (...) > > > Inside freelist_save() the mt_loose_pages could be merged into > > > the > > > mt_free_pgs. > > > http://www.ope

Re: LMDB: issue with mdb_cursor_del

2017-10-17 Thread Hallvard Breien Furuseth
On 16/10/17 14:56, Hallvard Breien Furuseth wrote: On 16. okt. 2017 13:43, Леонид Юрьев wrote: (...) Inside freelist_save() the mt_loose_pages could be merged into the mt_free_pgs. http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=libraries/liblmdb/mdb.c;h=1bf81ed4c87901991b7cac8

Re: LMDB: issue with mdb_cursor_del

2017-10-17 Thread timur . kristof
On Mon, 2017-10-16 at 13:58 +0200, Hallvard Breien Furuseth wrote: > On 16. okt. 2017 12:51, Howard Chu wrote: > > timur.kris...@gmail.com wrote: > > > I have an app that uses LMDB, and I've experienced an interesting > > > issue: when trying to delete a certain item with mdb_cursor_del, > > > it >

Re: LMDB: issue with mdb_cursor_del

2017-10-17 Thread timur . kristof
On Mon, 2017-10-16 at 11:51 +0100, Howard Chu wrote: > > > > It appears that it couldn't mark a page as dirty. > > Here is the relevant assertion from mdb_page_dirty: > > rc = insert(txn->mt_u.dirty_list, &mid); > > mdb_tassert(txn, rc == 0); // assertion failed > > > > What might I be doing wron

Re: LMDB: issue with mdb_cursor_del

2017-10-16 Thread Hallvard Breien Furuseth
On 16. okt. 2017 13:43, Леонид Юрьев wrote: This is a bug (seems minor), I fixed it in the MDBX few days ago. https://github.com/leo-yuriev/libmdbx/commit/9fd7056b4f02098914d813f38ba1ed2103c6b855#diff-185afae163fe87013315d2545a8b6055R3095 That's rather different from mdb.c. mdb's page_touch()

Re: LMDB: issue with mdb_cursor_del

2017-10-16 Thread Леонид Юрьев
Hi, Howard. This is a bug (seems minor), I fixed it in the MDBX few days ago. https://github.com/leo-yuriev/libmdbx/commit/9fd7056b4f02098914d813f38ba1ed2103c6b855#diff-185afae163fe87013315d2545a8b6055R3095 Inside freelist_save() the mt_loose_pages could be merged into the mt_free_pgs. http://www

Re: LMDB: issue with mdb_cursor_del

2017-10-16 Thread Hallvard Breien Furuseth
On 16. okt. 2017 12:51, Howard Chu wrote: timur.kris...@gmail.com wrote: I have an app that uses LMDB, and I've experienced an interesting issue: when trying to delete a certain item with mdb_cursor_del, it crashed with the following backtrace: https://pastebin.com/7p9wtkj9 Weird backtrace. I

Re: LMDB: issue with mdb_cursor_del

2017-10-16 Thread Howard Chu
timur.kris...@gmail.com wrote: Hi, I have an app that uses LMDB, and I've experienced an interesting issue: when trying to delete a certain item with mdb_cursor_del, it crashed with the following backtrace: https://pastebin.com/7p9wtkj9 It appears that it couldn't mark a page as dirty. Here is

LMDB: issue with mdb_cursor_del

2017-10-16 Thread timur . kristof
Hi, I have an app that uses LMDB, and I've experienced an interesting issue: when trying to delete a certain item with mdb_cursor_del, it crashed with the following backtrace: https://pastebin.com/7p9wtkj9 It appears that it couldn't mark a page as dirty. Here is the relevant assertion from mdb_p