[issue4297] Add error_log attribute to optparse.OptionParser
New submission from Daniel Watkins <[EMAIL PROTECTED]>: I've recently had to subclass optparse.OptionParser, and copy-paste the exit method, just to change where errors were printed to (I needed stdout rather than stderr). I've also had a request from a client to log errors with command-line parsing to a file, rather than to stdout. So, this patch adds an error_log parameter to OptionParser.__init__ which is used instead of stderr internally (and, of course, defaults to stderr). -- components: Library (Lib) files: optparse-error_log.patch keywords: patch messages: 75741 nosy: odd_bloke severity: normal status: open title: Add error_log attribute to optparse.OptionParser type: feature request versions: Python 2.7 Added file: http://bugs.python.org/file11985/optparse-error_log.patch ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4297> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43882] [security] urllib.parse should sanitize urls containing ASCII newline and tabs.
Daniel Watkins added the comment: Hey folks, Thanks for all the work on this: I really appreciate the efforts to keep Python as secure as possible! This change _is_ causing us problems in the cloud-init codebase, which thankfully have been caught by our testing in Ubuntu's development release. This is in a fairly deep part of the codebase, so apologies in advance for the detailed description. TL;DR: cloud-init constructs mirror URLs and then sanitises them by replacing invalid characters with hyphens. With the fix for this bug, urlsplit silently removes (some of) those characters before we can replace them, modifying the output of our sanitisation code, and therefore meaning cloud-init will, albeit in fairly specific corner cases, configure different mirrors if run with a Python including this fix vs. one that precedes it. cloud-init constructs mirror URLs based on applying cloud metadata to user-configured (or default) templates. As we're responsible for constructing these URLs, we also sanitise them before configuring the package manager to use them: specifically, we urlsplit to get the hostname, IDNA-encode (to handle non-ASCII input), replace any invalid URL characters with a "-", and then strip "-" off each part of the hostname (to handle leading/trailing invalid characters), then recombine the URL. The most common case for this is a cloud which specifies values for the variables used in the template with an underscore: http://my_openstack_region.cloud.archive.ubuntu.com/ubuntu causes Apache mirrors with the default "HTTPProtocolOptions Strict" configuration to reject all requests to them (as that's an invalid hostname). In contrast, http://my-openstack-region.cloud.archive.ubuntu.com/ubuntu *is* accepted, so is preferable. (This is important because *.cloud.archive.ubuntu.com exists so that l ocal cloud admins can DNS "hijack" subdomains of it to point at internal servers: even though the Ubuntu mirrors don't reject underscored domains (any longer), this is a landmine waiting for any admins running their own mirrors.) For more background, see the bug where we figured this all out: https://bugs.launchpad.net/cloud-init/+bug/1868232 So, more concretely: if we consider a post-templated URL of http://my\topenstack\tregion.mirror.internal/ubuntu, cloud-init changes from rewriting that to my-openstack-region.mirror.internal (on < 3.9.5) to myopenstackregion.mirror.internal (on 3.9.5+): if, in this notional deployment, an apt mirror is running at (exactly) my-openstack-region.mirror.internal, then new instance deployments will start failing: they won't be able to install packages. This is the sort of breakage that we aim to avoid in cloud-init (because you just _know_ that everyone who deployed this cloud left NotionalCorp years ago, so fixing the configuration to remove these obviously-incorrect tabs is not necessarily trivial). Given the current state of the fix here, it's not clear to me how we could (cleanly) achieve our desired behaviour. We could perform replacement of these characters before invoking `urlsplit` but that would then substitute these characters outside of only the hostname: this is also a change in behaviour. We could substitute those characters with magic strings, perform the split, and then replace them in the non-hostname parts with the original character and in the hostname with hyphens: we've obviously left "cleanly" behind at this point. Another option would be to monkeypatch _UNSAFE_URL_BYTES_TO_REMOVE to an empty list: again, not a solution I'd want to have to support across Python versions! One solution that presents itself to me: add a `strip_insecure_characters: bool = True` parameter. We, in cloud-init, would pass this in as `False`, knowing that we're going to handle those ourselves. Of course, this does leave the door open for API users to keep the current insecure behaviour: if library code (either public or project-internal) were to default to `False`, then the situation is no better than today. For our use case, at least, I think a more restricted solution would work: `url_replacement_char: str = ""`. We'd call `urlsplit(..., url_replacement_char="-")` and the rest of our code would work as it does today: from its POV, there were never these invalid chars in the first place. Thanks once again for the work (and apologies for the wall of text)! Dan -- nosy: +odd_bloke -ned.deily ___ Python tracker <https://bugs.python.org/issue43882> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43882] [security] urllib.parse should sanitize urls containing ASCII newline and tabs.
Daniel Watkins added the comment: (Accidentally dropped Ned from nosy list; apologies!) -- nosy: +ned.deily ___ Python tracker <https://bugs.python.org/issue43882> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30949] Provide assertion functions in unittest.mock
New submission from Daniel Watkins: The convenience assertion methods on mock objects can be easily mistyped and if they are mistyped, they will silently pass. This can be quite user-hostile. Consider the following: >>> example = Mock() >>> example.assert_called_once() >>> example.assert_caled_once_with(...) This will not raise any exceptions, though the first feels natural and the latter is misspelt. To avoid using the methods, one can type: >>> example = Mock() >>> assert example.call_count == 1 >>> assert example.call_args_list == [call(...)] but the meaning of that latter statement is particularly non-obvious. Instead, it would be great if I could import the assertions from mock as functions, and call them with mock as the first argument: >>> from unittest.mock import assert_called_once # This will be an ImportError >>> example = Mock() >>> assert_caled_once_with(example, ...) # A NameError >>> assert_called_once_with(example, ...) # Great success! -- components: Tests messages: 298533 nosy: odd_bloke priority: normal severity: normal status: open title: Provide assertion functions in unittest.mock versions: Python 3.7 ___ Python tracker <http://bugs.python.org/issue30949> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31040] mimetypes.add_type should complain when you give it an undotted ext
New submission from Daniel Watkins:
```
import mimetypes
print(mimetypes.guess_type('foo.manifest'))
mimetypes.add_type('text/plain', 'manifest')
print(mimetypes.guess_type('foo.manifest'))
```
results in:
```
('application/x-ms-manifest', None)
('application/x-ms-manifest', None)
```
I (mistakenly) expected the latter print to give me "('text/plain', None)". It
doesn't because I should have prepended a . to the second add_type argument.
I think add_type should error out when given an extension without a dot-prefix,
because it's extremely unlikely that code that does so is behaving as intended
with the current implementation.
(At the very least, documentation should be updated to make this expectation
clearer.)
--
components: Library (Lib)
messages: 299135
nosy: odd_bloke
priority: normal
severity: normal
status: open
title: mimetypes.add_type should complain when you give it an undotted ext
___
Python tracker
<http://bugs.python.org/issue31040>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31040] mimetypes.add_type should complain when you give it an undotted ext
Changes by Daniel Watkins : -- pull_requests: +2922 ___ Python tracker <http://bugs.python.org/issue31040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31040] mimetypes.add_type should complain when you give it an undotted ext
Changes by Daniel Watkins : -- pull_requests: +2948 ___ Python tracker <http://bugs.python.org/issue31040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31129] RawConfigParser.items() is unusual in taking arguments
New submission from Daniel Watkins: A grep through the codebase shows that RawConfigParser.items() is the only .items() method in the stdlib which accepts arguments. This is annoying as a stdlib user because when I see the arguments being passed to RawConfigParser.items(), I have _no idea_ what they do. Instinctively, I do not expect .items() to take arguments, and I have to go and work out what it does each time. I think it would be both easier to understand, and more consistent with general Pythonic practice, if the two codepaths in RawConfigParser.items() were split in to separate methods; one which takes no arguments (which will continue to behave as it does today when called that way) and one which is named in a way that makes it clearer what it does (and takes arguments). -- components: Library (Lib) messages: 299821 nosy: odd_bloke priority: normal severity: normal status: open title: RawConfigParser.items() is unusual in taking arguments versions: Python 3.7 ___ Python tracker <http://bugs.python.org/issue31129> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31129] RawConfigParser.items() is unusual in taking arguments
Changes by Daniel Watkins : -- pull_requests: +3045 ___ Python tracker <http://bugs.python.org/issue31129> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31129] RawConfigParser.items() is unusual in taking arguments
Daniel Watkins added the comment: Having ironed out my confusion over typing the method, I agree that making the types more obvious is not a compelling argument for this change. That said, I think the current API has been confusing to me in the past, and I think the proposed change is still a worthwhile improvement for users of this module. On Mon, Aug 7, 2017 at 12:08 AM Guido van Rossum wrote: > > Guido van Rossum added the comment: > > I think the proposed change is not worth it. Developments in type checking > (in particular overloading) make it unambiguous what the return type will > be from just a static inspection of the call site. (Given that the _UNSET > value is intended to be private.) See also > https://github.com/python/mypy/issues/3805#issuecomment-320561797 > > -- > nosy: +gvanrossum > > ___ > Python tracker > <http://bugs.python.org/issue31129> > ___ > -- ___ Python tracker <http://bugs.python.org/issue31129> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
