[Python-Dev] Proposal to deprecate mailcap
A CVE has been opened against mailcap (see https://github.com/python/cpython/issues/68966 for details). I'm not aware of anyone trying to maintain the module and Victor did a search online and didn't find any use of the module in the top 5000 projects on PyPI (see the issue). The module is also under 300 lines of Python code that only ( https://github.com/python/cpython/blob/main/Lib/mailcap.py), so vendoring wouldn't be burdensome. As such, I'm proposing we deprecate mailcap in 3.11 and remove it in 3.13. Any explicit objections? ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/EB2BS4DBWSTBIOPQL5QTBSIOBORWSCMJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proposal to deprecate mailcap
El jue, 14 abr 2022 a las 11:47, Brett Cannon () escribió: > A CVE has been opened against mailcap (see > https://github.com/python/cpython/issues/68966 for details). I'm not > aware of anyone trying to maintain the module and Victor did a search > online and didn't find any use of the module in the top 5000 projects on > PyPI (see the issue). The module is also under 300 lines of Python code > that only (https://github.com/python/cpython/blob/main/Lib/mailcap.py), > so vendoring wouldn't be burdensome. > > As such, I'm proposing we deprecate mailcap in 3.11 and remove it in 3.13. > Any explicit objections? > Agree on deprecating. I searched grep.app and found no significant usage. Do you know why this module wasn't included in PEP 594? Should we do another audit of old modules to deprecate them before they cause problems? > ___ > Python-Dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/[email protected]/message/EB2BS4DBWSTBIOPQL5QTBSIOBORWSCMJ/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/ON7R7LD7OVPOF5XXPXJWE7AKCRAPQ7IN/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proposal to deprecate mailcap
> I searched grep.app and found no significant usage. Maybe someone wants to inform mitmproxy? It's a very popular tool and it comes up using that tool when searching for "import mailcap" using grep.app: https://grep.app/search?q=import%20mailcap https://github.com/mitmproxy/mitmproxy/blob/main/mitmproxy/tools/console/master.py#L2 On Thu, Apr 14, 2022 at 3:06 PM Jelle Zijlstra wrote: > > > El jue, 14 abr 2022 a las 11:47, Brett Cannon () > escribió: > >> A CVE has been opened against mailcap (see >> https://github.com/python/cpython/issues/68966 for details). I'm not >> aware of anyone trying to maintain the module and Victor did a search >> online and didn't find any use of the module in the top 5000 projects on >> PyPI (see the issue). The module is also under 300 lines of Python code >> that only (https://github.com/python/cpython/blob/main/Lib/mailcap.py), >> so vendoring wouldn't be burdensome. >> >> As such, I'm proposing we deprecate mailcap in 3.11 and remove it in >> 3.13. Any explicit objections? >> > > Agree on deprecating. I searched grep.app and found no significant usage. > > Do you know why this module wasn't included in PEP 594? Should we do > another audit of old modules to deprecate them before they cause problems? > > >> ___ >> Python-Dev mailing list -- [email protected] >> To unsubscribe send an email to [email protected] >> https://mail.python.org/mailman3/lists/python-dev.python.org/ >> Message archived at >> https://mail.python.org/archives/list/[email protected]/message/EB2BS4DBWSTBIOPQL5QTBSIOBORWSCMJ/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > ___ > Python-Dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/[email protected]/message/ON7R7LD7OVPOF5XXPXJWE7AKCRAPQ7IN/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/MRW7DDFMD4G4D46AIRO3JICFKK5HSN6X/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proposal to deprecate mailcap
On Thu, Apr 14, 2022 at 12:04 Jelle Zijlstra wrote: > > > El jue, 14 abr 2022 a las 11:47, Brett Cannon () > escribió: > >> Do you know why this module wasn't included in PEP 594? > Probably because it’s not a top level module — it’s inside the email package. Should we do another audit of old modules to deprecate them before they > cause problems? > -- --Guido (mobile) ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/ZMSD2LZ3SRM6E3W3Z2VDEKZIMVLO6VZW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proposal to deprecate mailcap
+1 add it to the 3.11 deprecations and proactively reach out to the mitmproxy owners. (internal code search: aside from mitmproxy I only see a _single_ use of this in our codebase and it was simply convenient but has a clear simpler alternative assuming that ~2008 era code is even still in use) -gps On Thu, Apr 14, 2022 at 11:49 AM Brett Cannon wrote: > A CVE has been opened against mailcap (see > https://github.com/python/cpython/issues/68966 for details). I'm not > aware of anyone trying to maintain the module and Victor did a search > online and didn't find any use of the module in the top 5000 projects on > PyPI (see the issue). The module is also under 300 lines of Python code > that only (https://github.com/python/cpython/blob/main/Lib/mailcap.py), > so vendoring wouldn't be burdensome. > > As such, I'm proposing we deprecate mailcap in 3.11 and remove it in 3.13. > Any explicit objections? > ___ > Python-Dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/[email protected]/message/EB2BS4DBWSTBIOPQL5QTBSIOBORWSCMJ/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/GG2JGVJPKEGS4P5ASSLJGGWUOO2JBCBJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Do you ever use ceval.c's LLTRACE feature?
Hi everyone,
I'm looking to improve the output of the interpreter's LLTRACE feature to make
it more understandable. This "feature" is undocumented, but it prints out the
opcode and oparg of every bytecode instruction executed, along with some info
about stack operations, whenever you've built with Py_DEBUG and the name
`__ltrace__` is defined in the module.
I've found this useful for debugging bits of the compiler and bytecode
interpreter. For example, if I make some tweak that instroduces an off-by-one
error, by default I get a segfault or a rather unhelpful assertion failure at
`assert(EMPTY())` or `assert(STACK_LEVEL() <= frame->f_code->co_stacksize)` or
similar, at best, with no inducation as to where or why that assertion is
failing. But if I enable `__ltrace__` by either setting `__ltrace__=1` in some
module or by manually setting `lltrace=1;` in the c code, I can follow what was
happening in the interpreter just before the crash.
I'd like the output in that scenario to be a bit more helpful. I propose
printing opcode names rather than decimal digits, and printing out the name of
the current function whenever a stack frame begins executing. I also proprose
to print out the full stack contents (almost never very deep) before each
bytecode, rather than printing the state piecemeal at each
PUSH/POP/STACK_ADJUST macro. I opened issue
https://github.com/python/cpython/issues/91462 and PR
https://github.com/python/cpython/pull/91463
I later found that this had been explored before by
https://github.com/python/cpython/issues/69757, and there was a suggestion that
this could be folded into a more generalized bytecode-level tracing feature
that is pluggable with python code, similar to sys.settrace(). I would tend to
think "YAGNI" -- lltrace is a feature for debugging the c internals of the
interpreter, and there are already separate existing features like the `trace`
module for tracing through Python code with different goals. I appreciate the
simplicity of printf statements at the c level -- it feels more trustworthy
than adding a complicated extra feature involving python calls and global
state. It's as if I just littered the code with my own debugging print
statements, but re-usable and better.
I see no documentation anywhere, and there's only one test case in
test_lltrace, just testing that there's no segfault. Looking back through the
git history, I see that the basic `printf("%d: %d, %d\n", ...);` format goes
back to 1990:
https://github.com/python/cpython/blob/3f5da24ea304e674a9abbdcffc4d671e32aa70f1/Python/ceval.c#L696-L710
I'm essentially writing to ask: how do you use lltrace? Does anyone rely on the
particular format of the output? Would some of these improvements be helpful to
you? What else could make it more helpful?
Thanks,
Dennis Sweeney
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/DELHX3N5PCZDWIK2DLU5JDG6JREQ42II/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proposal to deprecate mailcap
El jue, 14 abr 2022 a las 12:21, Damian Shaw () escribió: > > I searched grep.app and found no significant usage. > > Maybe someone wants to inform mitmproxy? > > It's a very popular tool and it comes up using that tool when searching > for "import mailcap" using grep.app: > https://grep.app/search?q=import%20mailcap > > https://github.com/mitmproxy/mitmproxy/blob/main/mitmproxy/tools/console/master.py#L2 > Thanks for catching that! I missed it because I mistakenly searched for '"import mailcap"' in quotes. It looks like mitmproxy isn't vulnerable to the security issue because it only passes a filename from mkstemp() to mailcap, and hopefully mkstemp filenames don't have shell metacharacters in them. However, if we deprecate mailcap mitmproxy will have to change their code. El jue, 14 abr 2022 a las 13:33, Guido van Rossum () escribió: > Probably because it’s not a top level module — it’s inside the email > package. > It's in fact a top-level module. ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/25FNDJBERE5PLBP3VX3JQ7ER2LNE2K2O/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proposal to deprecate mailcap
Whoops, you’re right. I suppose I should have no opinion on whether to deprecate it; I haven’t thought about it for over two decades… On Thu, Apr 14, 2022 at 16:33 Jelle Zijlstra wrote: > > > El jue, 14 abr 2022 a las 12:21, Damian Shaw () > escribió: > >> > I searched grep.app and found no significant usage. >> >> Maybe someone wants to inform mitmproxy? >> >> It's a very popular tool and it comes up using that tool when searching >> for "import mailcap" using grep.app: >> https://grep.app/search?q=import%20mailcap >> >> https://github.com/mitmproxy/mitmproxy/blob/main/mitmproxy/tools/console/master.py#L2 >> > > Thanks for catching that! I missed it because I mistakenly searched for > '"import mailcap"' in quotes. It looks like mitmproxy isn't vulnerable to > the security issue because it only passes a filename from mkstemp() to > mailcap, and hopefully mkstemp filenames don't have shell metacharacters in > them. However, if we deprecate mailcap mitmproxy will have to change their > code. > > El jue, 14 abr 2022 a las 13:33, Guido van Rossum () > escribió: > >> Probably because it’s not a top level module — it’s inside the email >> package. >> > It's in fact a top-level module. > > ___ > Python-Dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/[email protected]/message/25FNDJBERE5PLBP3VX3JQ7ER2LNE2K2O/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido (mobile) ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/AAIXGR57BXDULXPTJJN4LBTVR4KVFPED/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Do you ever use ceval.c's LLTRACE feature?
Hi Dennis,
It's been a while since I've used this code, but in the early days it was
indispensible. I think you have the right ideas on making it more useful
now that we're hacking on the bytecode again. Since lltrace is most useful
when the bytecode interpreter is somehow faulty, it doesn't make much sense
IMO to extend it with Python code, so indeed YAGNI on that proposal.
Since this will be useful only to a small audience, don't put too much
effort in it, but I'm sure that whatever effort you do put in it will be
much appreciated.
--Guido
On Thu, Apr 14, 2022 at 3:29 PM Dennis Sweeney
wrote:
> Hi everyone,
>
> I'm looking to improve the output of the interpreter's LLTRACE feature to
> make it more understandable. This "feature" is undocumented, but it prints
> out the opcode and oparg of every bytecode instruction executed, along with
> some info about stack operations, whenever you've built with Py_DEBUG and
> the name `__ltrace__` is defined in the module.
>
> I've found this useful for debugging bits of the compiler and bytecode
> interpreter. For example, if I make some tweak that instroduces an
> off-by-one error, by default I get a segfault or a rather unhelpful
> assertion failure at `assert(EMPTY())` or `assert(STACK_LEVEL() <=
> frame->f_code->co_stacksize)` or similar, at best, with no inducation as to
> where or why that assertion is failing. But if I enable `__ltrace__` by
> either setting `__ltrace__=1` in some module or by manually setting
> `lltrace=1;` in the c code, I can follow what was happening in the
> interpreter just before the crash.
>
> I'd like the output in that scenario to be a bit more helpful. I propose
> printing opcode names rather than decimal digits, and printing out the name
> of the current function whenever a stack frame begins executing. I also
> proprose to print out the full stack contents (almost never very deep)
> before each bytecode, rather than printing the state piecemeal at each
> PUSH/POP/STACK_ADJUST macro. I opened issue
> https://github.com/python/cpython/issues/91462 and PR
> https://github.com/python/cpython/pull/91463
>
> I later found that this had been explored before by
> https://github.com/python/cpython/issues/69757, and there was a
> suggestion that this could be folded into a more generalized bytecode-level
> tracing feature that is pluggable with python code, similar to
> sys.settrace(). I would tend to think "YAGNI" -- lltrace is a feature for
> debugging the c internals of the interpreter, and there are already
> separate existing features like the `trace` module for tracing through
> Python code with different goals. I appreciate the simplicity of printf
> statements at the c level -- it feels more trustworthy than adding a
> complicated extra feature involving python calls and global state. It's as
> if I just littered the code with my own debugging print statements, but
> re-usable and better.
>
> I see no documentation anywhere, and there's only one test case in
> test_lltrace, just testing that there's no segfault. Looking back through
> the git history, I see that the basic `printf("%d: %d, %d\n", ...);` format
> goes back to 1990:
> https://github.com/python/cpython/blob/3f5da24ea304e674a9abbdcffc4d671e32aa70f1/Python/ceval.c#L696-L710
>
> I'm essentially writing to ask: how do you use lltrace? Does anyone rely
> on the particular format of the output? Would some of these improvements be
> helpful to you? What else could make it more helpful?
>
> Thanks,
> Dennis Sweeney
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/DELHX3N5PCZDWIK2DLU5JDG6JREQ42II/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
