[issue22166] test_codecs leaks references

STINNER Victor added the comment: Codec tests don't leak anymore. I lost track of this very old issue. I don't think that anything should be done. Python code base changed a lot since 2014. I close the issue. -- resolution: -> fixed stage: needs patch -> resolved status: open -> clo

[issue22166] test_codecs leaks references

Stéphane Wirtel added the comment: For 3.7 and 3.8, I have just tried with this command to find leaks in the test_codec* but nothing :/ ./python -m test -v -l -R 6:6 -u all \ test_codecencodings_iso2022 \ test_codecs \ test_codecmaps_hk test_codecmaps_tw \ test_codecencodings_t

[issue22166] test_codecs leaks references

Change by Brett Cannon : -- nosy: -brett.cannon ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail

[issue22166] test_codecs leaks references

Change by Serhiy Storchaka : -- assignee: -> serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue22166] test_codecs leaks references

STINNER Victor added the comment: > Do you mind to create a PR Victor? I completely forgot this issue and I don't plan to fix it in the short-term. Feel free to write PR if you want. -- ___ Python tracker

[issue22166] test_codecs leaks references

Serhiy Storchaka added the comment: Do you mind to create a PR Victor? -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue22166] test_codecs leaks references

Change by Serhiy Storchaka : -- versions: +Python 3.7, Python 3.8 -Python 3.4, Python 3.5 ___ Python tracker ___ ___ Python-bugs-lis

[issue22166] test_codecs leaks references

Nick Coghlan added the comment: Victor, would you be willing to take over pushing this one forward? I believe you have a much better idea of what's still needed than I do. -- assignee: ncoghlan -> ___ Python tracker

[issue22166] test_codecs leaks references

Changes by Serhiy Storchaka : -- stage: resolved -> needs patch ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue22166] test_codecs leaks references

Changes by Nick Coghlan : -- assignee: -> ncoghlan ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mai

[issue22166] test_codecs leaks references

Marc-Andre Lemburg added the comment: On 15.09.2014 15:19, STINNER Victor wrote: > > STINNER Victor added the comment: > > IMO test_codecs_fix1.patch is still needed. > > Review of Nick's change: > > +interp = PyThreadState_GET()->interp; > +if (interp->codec_search_path == NULL) { >

[issue22166] test_codecs leaks references

STINNER Victor added the comment: IMO test_codecs_fix1.patch is still needed. Review of Nick's change: +interp = PyThreadState_GET()->interp; +if (interp->codec_search_path == NULL) { +return -1; +} I don't think that this line raises an exception. You should use an asserti

[issue22166] test_codecs leaks references

Changes by Nick Coghlan : -- resolution: -> fixed stage: -> resolved status: open -> closed ___ Python tracker ___ ___ Python-bugs-l

[issue22166] test_codecs leaks references

Roundup Robot added the comment: New changeset fcf45ec7863e by Nick Coghlan in branch '3.4': Issue #22166: clear codec caches in test_codecs https://hg.python.org/cpython/rev/fcf45ec7863e New changeset 322ee2f2e922 by Nick Coghlan in branch 'default': Merge fix for issue #22166 from 3.4 https://

[issue22166] test_codecs leaks references

Nick Coghlan added the comment: Targeted API to forget a particular codec name. Normally we don't care about the caching (as evidenced by the fact it took 14+ years to want this feature). The intent with this API is to be able to selective purge a particular codec when we either care about ge

[issue22166] test_codecs leaks references

Antoine Pitrou added the comment: Well, the point here is to add a private API just for our test suite, not something users would want to call. -- ___ Python tracker ___ ___

[issue22166] test_codecs leaks references

Marc-Andre Lemburg added the comment: On 06.09.2014 21:14, Nick Coghlan wrote: > > This investigation also showed why the original tests that used a > non-unique name didn't work: the codec was getting cached the first time > through and hence not seeing the different codecs registered by other

[issue22166] test_codecs leaks references

Nick Coghlan added the comment: This investigation also showed why the original tests that used a non-unique name didn't work: the codec was getting cached the first time through and hence not seeing the different codecs registered by other tests. MAL, my plan was to provide an API to forget a s

[issue22166] test_codecs leaks references

Serhiy Storchaka added the comment: May be just do not use the unique name? -- nosy: +serhiy.storchaka ___ Python tracker ___ ___ Pyth

[issue22166] test_codecs leaks references

Marc-Andre Lemburg added the comment: On 06.09.2014 15:03, Nick Coghlan wrote: > A "codecs._forget_codec" helper in the C module would be all that was needed > to handle it. Other implementations could then also use that to clear their > own internal cache (if they have one). If you want to ta

[issue22166] test_codecs leaks references

Nick Coghlan added the comment: Yeah, I'm wary of the GC hack as well. It's a nice way to prove that this the internal cache is the problem, but probably not something we want to commit. A "codecs._forget_codec" helper in the C module would be all that was needed to handle it. Other implementa

[issue22166] test_codecs leaks references

Antoine Pitrou added the comment: I'd much rather see a private function somewhere (in the _codecs module) to clear that private interpreter cache. -- ___ Python tracker ___ ___

[issue22166] test_codecs leaks references

Nick Coghlan added the comment: Yep, it looks like the mess with the reference cycles in the tracebacks was just a secondary problem: the real culprit was the internal cache keeping everything else alive. Attached patch just adds clearing encodings._cache and the CPython internal cache to the

[issue22166] test_codecs leaks references

Nick Coghlan added the comment: Ah, I think I've found a possible underlying culprit: there's a separate C level "codec_search_cache" on the interpreter object that is independent of encodings._cache. The attached patch eliminates the reference leaks entirely (using gc.get_referrers() to get

[issue22166] test_codecs leaks references

Nick Coghlan added the comment: Yeah, the unique name was a hack to deal with the fact I couldn't unregister anything. As a simpler short term fix for the reference leak, perhaps it would be worth trying a call to "traceback.clear_frames(caught)" as a new final line in ExceptionChainingTest.a

[issue22166] test_codecs leaks references

Antoine Pitrou added the comment: > We should maybe enhance the codecs API to be able to unregister a codec? That would be nice. At least as a private API in 3.4. -- nosy: +pitrou ___ Python tracker __

[issue22166] test_codecs leaks references

STINNER Victor added the comment: ExceptionChainingTest creates a random codec name. If you change the codec name to a fixed string like "xxx", ExceptionChainingTest.test_raise_by_type() doesn't leak anymore (when test_codecs_fix1.patch is applied), but other tests start to fail. We should m

[issue22166] test_codecs leaks references

STINNER Victor added the comment: Attached: Draft patch to fix some reference leaks in test_codecs. -- title: test_codecs "leaking" references -> test_codecs leaks references Added file: http://bugs.python.org/file36524/test_codecs_fix1.patch ___ Pyth