Serhiy Storchaka added the comment: Initially I used 2 x 128 slots. It is enough for single block alphabetic languages. But it was caused significant slow down for Chinese. Increasing the size to 2 x 256 compensates the overhead for Chinese and restores the performance. If it is appropriate that the optimization affects only languages with small alphabets and keeps the performance for Chinese, Japan and Korean roughly unchanged (plus-minus few percents), this size is enough. I we want to optimize also processing texts with Chinese characters, it can be increased to 2 x 512 or 2 x 1024. Further increasing have smaller effect.
The cache of size 2 x 256 slots can increase memory consumption by 50 KiB in worst case, 2 x 1024 -- by 200 KiB. ---------- nosy: +inada.naoki, xiang.zhang _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31484> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
