Zhang Wenbo added the comment: I think I find a case which can reproduct this bug: the UnstructuredTokenList has a ValueTerminal which is long enough(its encoded word is longer than policy.maxlen ) and not at the beginning.
For examples, the following subjects can cause this bug: - 'Re: 我只是一个有那么一点点长的主题而已' - '回复: 回复:【北京大学-2017年招生信息公告】' So, the code can not fold a ValueTerminal correctly. Seems the method `_fold_as_ew` was designed to do this, but it is missing. ---------- Added file: http://bugs.python.org/file46941/test2.py _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27240> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
