New submission from Joel Klimont <joel.klim...@gmail.com>:
There is a bug in sqlite3/dump.py when wanting to dump databases that use autoincrement in one or more tables. The problem is that the iterdump command assumes that the table "sqlite_sequence" is present in the new database in which the old one is dumped into. >From the sqlite3 documentation: "SQLite keeps track of the largest ROWID using an internal table named "sqlite_sequence". The sqlite_sequence table is created and initialized automatically whenever a normal table that contains an AUTOINCREMENT column is created. The content of the sqlite_sequence table can be modified using ordinary UPDATE, INSERT, and DELETE statements." Source: https://sqlite.org/autoinc.html#the_autoincrement_keyword Example: BEGIN TRANSACTION; CREATE TABLE "posts" ( id int primary key ); INSERT INTO "posts" VALUES(0); CREATE TABLE "tags" ( id integer primary key autoincrement, tag varchar(256) unique, post int references posts ); INSERT INTO "tags" VALUES(NULL, "test", 0); COMMIT; The following code should work but because of the assumption that "sqlite_sequence" exists it will fail: >> import sqlite3 >> cx = sqlite3.connect("test.db") >> for i in cx.iterdump(): >> print i >> cx2 = sqlite3.connect(":memory:") >> query = "".join(line for line in cx.iterdump()) >> cx2.executescript(query) Exception: Traceback (most recent call last): File "/home/test.py", line 10, in <module> cx2.executescript(query) sqlite3.OperationalError: no such table: sqlite_sequence Here is the ouput of cx.iterdrump() BEGIN TRANSACTION; CREATE TABLE "posts" ( id int primary key ); INSERT INTO "posts" VALUES(0); DELETE FROM "sqlite_sequence"; INSERT INTO "sqlite_sequence" VALUES('tags',1); CREATE TABLE "tags" ( id integer primary key autoincrement, tag varchar(256) unique, post int references posts ); INSERT INTO "tags" VALUES(1,'test',0); COMMIT; As you can see the problem is that "DELETE FROM "sqlite_sequence";" and "INSERT INTO "sqlite_sequence" VALUES('tags',1);" are put into the dump before that table even exists. They should be put at the end of the transaction. (Like the sqlite3 command ".dump" does.) Note that some databases that use autoincrement will work as it could be that a table with autoincrement is created before the sqlite_sequence commands are put into the dump. File: https://github.com/python/cpython/blob/master/Lib/sqlite3/dump.py I have already forked the repository, written tests etc. and if you want I will create a pull request. ---------- components: Library (Lib) messages: 326610 nosy: itssme priority: normal severity: normal status: open title: sqlite.iterdump does not work for (most) databases with autoincrement type: behavior versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue34828> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
