https://bugs.kde.org/show_bug.cgi?id=391115
--- Comment #4 from Michał Karwowski <pro...@gmail.com> --- Thanks for 'DebugView' - I didn't know such tool exists. I made more detail research on the problem. To be precise, my total media count from DigiKam database stats is about 108012. (I didn't add all folders to DigiKam) When I run the application I see in a log: [8144] digikam.database: Search result: 1512168 But: "1 512 168" is much more than "108 012"! Strange.. what does it mean? ========================== ========================== Text filter (available on the right) ========================== ========================== The problem with speed appears to be related to "Text filter" which I use. Filtering pictures with that filter takes from 20 seconds to 3 minutes. DebugView doesn't show logs for database on "text filter". Thus now, I believe that's it's not related to database engine, but rather internal filtering system. Filter query: 'tajlandia' took 2 minutes 31 seconds (to display results - 8586 items). Filter query: 'żaba' took 1 minute 25 seconds (to display results - 1862 items). etc. Clearing filter also takes lots of time. I did tests with offline and online albums. By term offline I mean disconnected USB drive with albums. By term online I mean connected USB drive with albums. Measurements I have don after the inilial digikam scan. ========================== ========================== Search (available on the left) ========================== ========================== When I use search I see digikam.database logs on DebugView. This suggests that for searching database query is made in contrast to filtering. To be clear I use: digiKam 5.8.0 with mariadb 10.2.10 winx64 ........................ Query for: "gosia" (164794 results?) ........................ 00001448 21:55:15 [10424] digikam.general: keywords changed to ' "gosia" ' 00001449 21:55:15 [10424] digikam.geoiface: ---- 00001450 21:55:15 [10424] digikam.geoiface: ---- 00001451 21:55:15 [10424] digikam.geoiface: ---- 00001452 21:55:15 [10424] digikam.general: Using 8 CPU core to run threads 00001453 21:55:15 [10424] digikam.general: Action Thread run 1 new jobs 00001454 21:55:15 [10424] digikam.database: " ( ( (Albums.relativePath LIKE ?) OR (Images.name LIKE ?) OR (Images.id IN (SELECT imageid FROM ImageTags WHERE tagid IN (SELECT id FROM Tags WHERE name LIKE ?))) OR (Albums.caption LIKE ?) OR (Albums.collection LIKE ?) OR (Images.id IN (SELECT imageid FROM ImageComments WHERE type=? AND comment LIKE ?)) OR (Images.id IN (SELECT imageid FROM ImageComments WHERE type=? AND comment LIKE ?)) ) ) " 00001455 21:55:15 [10424] digikam.database: Search query: 00001458 21:55:16 [10424] digikam.database: Search result: 164794 # Result is ready but not visible 00001459 21:55:19 [10424] digikam.general: Cancel Main Thread 00001507 21:55:22 [10424] digikam.general: One job is done 00001508 21:55:24 [10424] digikam.general: Cancel Main Thread 00001509 21:55:45 [10424] digikam.geoiface: ---- 00001512 21:55:50 [10424] digikam.general: keywords changed to ' "gosia" ' 00001513 21:55:50 [10424] digikam.general: same keywords as before, ignoring... 00001514 21:55:58 [10424] digikam.general: keywords changed to ' "ekp" ' # application is responsive again ........................ Query for: "ekp" (394240 results?) ........................ 00001514 21:55:58 [10424] digikam.general: keywords changed to ' "ekp" ' 00001515 21:55:58 [10424] digikam.geoiface: ---- 00001516 21:55:58 [10424] digikam.geoiface: ---- 00001517 21:55:58 [10424] digikam.geoiface: ---- 00001518 21:55:58 [10424] digikam.general: Using 8 CPU core to run threads 00001519 21:55:58 [10424] digikam.general: Action Thread run 1 new jobs 00001520 21:55:58 [10424] digikam.database: " ( ( (Albums.relativePath LIKE ?) OR (Images.name LIKE ?) OR (Images.id IN (SELECT imageid FROM ImageTags WHERE tagid IN (SELECT id FROM Tags WHERE name LIKE ?))) OR (Albums.caption LIKE ?) OR (Albums.collection LIKE ?) OR (Images.id IN (SELECT imageid FROM ImageComments WHERE type=? AND comment LIKE ?)) OR (Images.id IN (SELECT imageid FROM ImageComments WHERE type=? AND comment LIKE ?)) ) ) " 00001521 21:55:58 [10424] digikam.database: Search query: 00001524 21:56:00 [10424] digikam.database: Search result: 394240 00001525 21:56:08 [10424] digikam.general: Cancel Main Thread 00001526 21:56:18 [10424] digikam.general: One job is done 00001575 21:56:23 [10424] digikam.geoiface: ---- 00001576 21:56:23 [10424] digikam.general: Cancel Main Thread 00001582 21:57:17 [10424] digikam.general: keywords changed to ' "ekp" ' 00001582 21:57:17 [10424] digikam.general: keywords changed to ' "ekp" ' 00001583 21:57:17 [10424] digikam.general: same keywords as before, ignoring... 00001584 21:57:19 [10424] digikam.general: Request to get thumbnail for "" 00001585 21:57:19 [10424] digikam.general: Video file "" does not exist. 00001586 21:57:19 [10424] digikam.general: Failed to extract video thumbnail for "" 00001587 21:57:19 [10424] digikam.general: Request to get thumbnail for "" 00001588 21:57:19 [10424] digikam.general: Video file "" does not exist. 00001589 21:57:19 [10424] digikam.general: Failed to extract video thumbnail for "" 00001637 21:57:23 [10424] digikam.general: # Interface is responsive again ........................ Query for: "ekpzg" 166474 results?) ........................ 00000072 21:17:20 [2136] digikam.general: Cancel Main Thread 00000074 21:17:30 [2136] digikam.general: One job is done 00000075 21:17:30 [2136] digikam.general: Cancel Main Thread 00000076 21:17:38 [2136] digikam.general: Using 8 CPU core to run threads 00000077 21:17:38 [2136] digikam.general: Action Thread run 1 new jobs 00000078 21:17:38 [2136] digikam.database: " ( ( (Albums.relativePath LIKE ?) OR (Images.name LIKE ?) OR (Images.id IN (SELECT imageid FROM ImageTags WHERE tagid IN (SELECT id FROM Tags WHERE name LIKE ?))) OR (Albums.caption LIKE ?) OR (Albums.collection LIKE ?) OR (Images.id IN (SELECT imageid FROM ImageComments WHERE type=? AND comment LIKE ?)) OR (Images.id IN (SELECT imageid FROM ImageComments WHERE type=? AND comment LIKE ?)) ) ) " 00000079 21:17:38 [2136] digikam.database: Search query: 00000082 21:17:39 [2136] digikam.database: Search result: 166474 00000083 21:17:47 [2136] digikam.geoiface: ---- 00000084 21:17:48 [2136] digikam.general: Cancel Main Thread 00000134 21:18:08 [2136] digikam.general: One job is done 00000135 21:18:08 [2136] digikam.general: keywords changed to ' "ekpzg" ' 00000136 21:18:08 [2136] digikam.general: same keywords as before, ignoring... 00000137 21:18:08 [2136] digikam.general: Cancel Main Thread 00000138 21:18:08 [2136] digikam.geoiface: ---- ........................ Query for "Jurek" - 26488 results ........................ 00001637 21:57:23 [10424] digikam.general: keywords changed to ' "Jurek" ' 00001638 21:57:23 [10424] digikam.geoiface: ---- 00001639 21:57:23 [10424] digikam.geoiface: ---- 00001640 21:57:23 [10424] digikam.geoiface: ---- 00001641 21:57:23 [10424] digikam.general: Using 8 CPU core to run threads 00001642 21:57:23 [10424] digikam.general: Action Thread run 1 new jobs 00001643 21:57:23 [10424] digikam.database: " ( ( (Albums.relativePath LIKE ?) OR (Images.name LIKE ?) OR (Images.id IN (SELECT imageid FROM ImageTags WHERE tagid IN (SELECT id FROM Tags WHERE name LIKE ?))) OR (Albums.caption LIKE ?) OR (Albums.collection LIKE ?) OR (Images.id IN (SELECT imageid FROM ImageComments WHERE type=? AND comment LIKE ?)) OR (Images.id IN (SELECT imageid FROM ImageComments WHERE type=? AND comment LIKE ?)) ) ) " 00001644 21:57:23 [10424] digikam.database: Search query: 00001647 21:57:24 [10424] digikam.database: Search result: 26488 00001648 21:57:24 [10424] digikam.general: Cancel Main Thread 00001650 21:58:10 [10424] digikam.general: One job is done 00001651 21:58:10 [10424] digikam.general: keywords changed to ' "Jurek" ' 00001652 21:58:10 [10424] digikam.general: same keywords as before, ignoring... 00001653 21:58:10 [10424] digikam.general: Cancel Main Thread 00001654 21:58:10 [10424] digikam.geoiface: ---- 00001655 21:58:11 [10424] digikam.general: Request to get thumbnail for "" 00001656 21:58:11 [10424] digikam.general: Video file "" does not exist. 00001657 21:58:11 [10424] digikam.general: Failed to extract video thumbnail for "" 00001658 21:58:19 [10424] digikam.geoiface: ---- ============================== ============================== Summary ============================== ============================== >From this analysis it seems, that. Huge performance issues are not related to SQL queries (which I was wrongly suspecting). Querying small sets of pictures gives results quite fast. In contrast to queries which results in thousands of pictures. Queries are probably quite fast, with 1-second accuracy - queries took 1 second (probably less). Problem is rather related to displaying huge amount of pictures. This is especially visible in when 'text filter' is used. When using "text filter" -probably internal DigiKam system for filtering is used, which slows application down. When I use search - which uses database side filtering. After DigiKam receives results from the database it freezes until a huge subset of pictures is displayed (thousands of pictures). My mistake with blaming database, but performance issues exist. -- You are receiving this mail because: You are watching all bug changes.
