[issue46954] Awaiting multiple times on same task increases memory usage unboundedly
New submission from David M. : Awaiting multiple times on a single task that failed with an exception results in an unbounded increase in memory usage. Enough repeated "await"s of the task can result in an OOM. The same pattern on a task that didn't raise an exception behaves as expected. The attached short script ends up using more than 1GB of memory in less than a minute. -- components: asyncio files: multi_await_exception.py messages: 414739 nosy: asvetlov, davidmanzanares, yselivanov priority: normal severity: normal status: open title: Awaiting multiple times on same task increases memory usage unboundedly versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 Added file: https://bugs.python.org/file50664/multi_await_exception.py ___ Python tracker <https://bugs.python.org/issue46954> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4362] FileIO object in io module
New submission from David M. Beazley <[EMAIL PROTECTED]>:
The FileIO object defined in the new io library has "name" and "mode"
properties. However, attempts to access either value result in an
AttributeError exception. The C source code in _fileio.c doesn't even
implement a name attribute and it uses a different name for mode ("mode"
instead of "_mode" that the property is looking for).
Broken in 2.6 and 3.0rc2.
--
components: Library (Lib)
messages: 76100
nosy: beazley
severity: normal
status: open
title: FileIO object in io module
type: behavior
versions: Python 2.6
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue4362>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4017] Tkinter cannot find Tcl/Tk on Mac OS X
David M. Beazley <[EMAIL PROTECTED]> added the comment: Just a quick comment from the Python training universe--this bug makes it impossible to use Python 2.6 in any kind of Python teaching environment where IDLE tends to be used a lot. I'm having to tell students to stick with Python-2.5.2. -- nosy: +beazley ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4017> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4428] io.BufferedWriter does not observe buffer size limits
New submission from David M. Beazley <[EMAIL PROTECTED]>: The Buffered I/O interface in the io module has the user specify buffer limits such as size and max_buffer_size. The first limit (size) is easy to understand as a buffering threshold at which writes will occur. However, no apparent attempt is made to strictly limit the internal buffer size to max_buffer_size. In BuffererWriter.write(), one of the first operations is self._write_buf.extend(b) which simply extends the buffer by the full data being written. If b happens to be a large string (e.g., megabytes or even the entire contents of a big file), then the internal I/O buffer makes a complete copy of the data, effectively doubling the memory requirements for carrying out the write operation. I suppose most programmers might not notice given that everyone has gigabytes of RAM these days, but you certainly don't see this kind of buffering behavior in the operating system kernel or in the C library. Some patch suggestions (details left to the maintainers of this module): 1. Don't extend self._write_buf by more than the max_buffer_size. fragment = b[:self.max_buffer_size - len(self._write_buf)] self._write_buf.extend(fragment) 2. For large blocking writes, simply carry out the remaining I/O operations in the write() method instead of in the _flush_locked() method. Try to use the original input data b as the data source instead of making copies of it. And if you have to copy the data, don't do it all at once. -- components: Library (Lib) messages: 76408 nosy: beazley severity: normal status: open title: io.BufferedWriter does not observe buffer size limits type: resource usage versions: Python 2.6, Python 2.7, Python 3.0 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4428> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4447] exec inside a function
New submission from David M. Beazley <[EMAIL PROTECTED]>:
Is the following code valid Python 3 or not?
def foo():
x = 1
exec("x = 42")
print(x)# Prints 1 (exec has no effect)
I know there are a variety of issues surrounding exec(), function
bodies, and other matters. Just wondering if this sort of thing is now
forbidden or not.
--
components: Interpreter Core
messages: 76508
nosy: beazley
severity: normal
status: open
title: exec inside a function
type: behavior
versions: Python 3.0
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue4447>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4447] exec inside a function
David M. Beazley <[EMAIL PROTECTED]> added the comment: For what it's worth, I hope this behavior gets well-documented. Thanks. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4447> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4428] io.BufferedWriter does not observe buffer size limits
David M. Beazley <[EMAIL PROTECTED]> added the comment: I agree with previous comments that write() should definitely write all data when in blocking mode. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4428> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4561] Optimize new io library
David M. Beazley <[EMAIL PROTECTED]> added the comment:
I've done some profiling and the performance of reading line-by-line is
considerably worse in Python 3 than in Python 2. For example, this
code:
for line in open("somefile.txt"):
pass
Ran 35 times slower in Python 3.0 than Python 2.6 when I tested it on a
big text file (100 Megabytes). If you disable Unicode by opening the
file in binary mode, it runs even slower.
This slowdown is really unacceptable for anyone who uses Python for
parsing big non-Unicode text files (and I would claim that there are
many such people).
--
nosy: +beazley
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue4561>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4561] Optimize new io library
David M. Beazley <[EMAIL PROTECTED]> added the comment:
Tried this using projects/python/branches/release30-maint and using the
patch that was just attached. With a 66MB input file, here are the
results of this code fragment:
for line in open("BIGFILE):
pass
Python 2.6: 0.67s
Python 3.0: 32.687s (48 times slower)
This is running on a MacBook with a warm disk cache. For what it's
worth, I didn't see any improvement with that patch.
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue4561>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4561] Optimize new io library
David M. Beazley <[EMAIL PROTECTED]> added the comment:
Just as one other followup, if you change the code in the last example
to use binary mode like this:
for line in open("BIG","rb"):
pass
You get the following results:
Python 2.6: 0.64s
Python 3.0: 42.26s (66 times slower)
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue4561>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4561] Optimize new io library
David M. Beazley <[EMAIL PROTECTED]> added the comment: Just checked it with branches/py3k and the performance is the same. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4561> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4561] Optimize new io library
David M. Beazley <[EMAIL PROTECTED]> added the comment: bash-3.2$ uname -a Darwin david-beazleys-macbook.local 9.5.1 Darwin Kernel Version 9.5.1: Fri Sep 19 16:19:24 PDT 2008; root:xnu-1228.8.30~1/RELEASE_I386 i386 bash-3.2$ ./python.exe -c "import sys; print(sys.version)" 3.1a0 (py3k:67609, Dec 6 2008, 08:47:06) [GCC 4.0.1 (Apple Inc. build 5465)] bash-3.2$ ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4561> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4593] Documentation for multiprocessing - Pool.apply()
New submission from David M. Beazley <[EMAIL PROTECTED]>: The documentation for the apply() and apply_async() methods of a Pool object might emphasize that these operations execute func(*args,**kwargs) in only one of the pool workers and that func() is not being executed in parallel on all workers.On first reading, I actually thought it might be the latter (and had to do some experimentation to figure out what was actually happening). -- assignee: georg.brandl components: Documentation messages: 77312 nosy: beazley, georg.brandl severity: normal status: open title: Documentation for multiprocessing - Pool.apply() versions: Python 2.6, Python 2.7, Python 3.0 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4593> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4593] Documentation for multiprocessing - Pool.apply()
David M. Beazley <[EMAIL PROTECTED]> added the comment: Actually, you shouldn't discount the potential usefulness of running apply() in all of the worker nodes. A lot of people coming from parallel programming know about things like global broadcasts, reductions, and so forth. For example, if I wanted to perform a global operation (maybe some kind of configuration) on all workers, I could see doing some kind of global apply() operation to do it. That said, I'm not actually asking for any new functionality. I'd just make it more clear that apply() is not performing a function call on all pool workers. Also, given that apply() blocks, I'm not exactly sure how useful it is in the context of actually performing work in parallel. You might want to emphasize that apply_async() is better suited for that (the only other way I could think of to take advantage of apply() in parallel would be to call it from separate threads in the process that created the pool). ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4593> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4561] Optimize new io library
David M. Beazley added the comment: I wish I shared your optimism about this, but I don't. Here's a short explanation why. The problem of I/O and the associated interface between hardware, the operating system kernel, and user applications is one of the most fundamental and carefully studied problems in all of computer systems. The C library and its associated I/O functionality provide the user- space implementation of this interface. However, if you peel the covers off of the C library, you're going to find a lot of really hairy stuff in there. Examples might include: 1. Low-level optimization related to the system hardware (processor architecture, caching, I/O bus, etc.). 2. Hand-written finely tuned assembly code. 3. Low-level platform-specific system calls such as ioctl(). 4. System calls related to shared memory regions, kernel buffers, etc. (i.e., optimizations that try to eliminate buffer copies). 5. Undocumented vendor-specific "proprietary" system calls (i.e., unknown "magic"). So, you'll have to forgive me for being skeptical, but I just don't think any programmer is going to sit down and bang out a new implementation of buffered I/O that is going to match the performance of what's provided by the C library. Again, I would love to be proven wrong. ___ Python tracker <http://bugs.python.org/issue4561> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4561] Optimize new io library
David M. Beazley added the comment: Good luck with that. Most people who get bright ideas such as "gee, maybe I'll write my own version of X" where "X" is some part of the standard C library pertaining to I/O, end up fighting a losing battle. Of course, I'd love to be proven wrong, but I don't think I will in this case. As for cranking data, that does not necessarily imply heavy-duty CPU processing. Someone might be reading large datafiles simply to perform some kind of data extraction, filtering, minor translation, or other operation that is easily carried out in Python, but where the programs are still I/O bound. For example, the kinds of processing one might otherwise do using awk, sed, perl, etc. ___ Python tracker <http://bugs.python.org/issue4561> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4561] Optimize new io library
David M. Beazley added the comment: I agree with Raymond. For binary reads, I'll go farther and say that even a 10% slowdown in performance would be surprising if not unacceptable to some people. I know that as hard as it might be for everyone to believe, there are a lot of people who crank lots of non- Unicode data with Python. In fact, Python 2.X is pretty good at it. It's fine that text mode now uses Unicode, but if I don't want that, I would certainly expect the binary file modes to run at virtually the same speed as Python 2 (e.g., okay, they work with bytes instead of strings, but is the bytes type really all that different from the old Python 2 str type?). ___ Python tracker <http://bugs.python.org/issue4561> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4686] Exceptions in ConfigParser don't set .args
New submission from David M. Beazley : The ConfigParser module defines a variety of custom exceptions, many of which take more than one argument (e.g., InterpolationError, NoOptionError, etc.). However, none of these exceptions properly set the .args attribute. For example, shouldn't NoOptionError be defined as follows: class NoOptionError(Error): def __init__(self,option,section): Error.__init__(self,"No option %r in section: %r" % (option,section)) self.option = option self.section = section self.args = (option,section) #!! Added this line This is kind of a minor point, but the missing args means that these exceptions don't work properly with programs that need to do fancy kinds of exception processing (i.e., catching errors and reraising them in a different context or process). I can't speak for Python 3.0, but it's my understanding that .args should always be set to the exception arguments. Don't ask how I came across this---it was the source of a really bizarre bug in something I was playing around with. -- components: Library (Lib) messages: 77983 nosy: beazley severity: normal status: open title: Exceptions in ConfigParser don't set .args type: behavior versions: Python 2.1.1, Python 2.1.2, Python 2.2, Python 2.2.1, Python 2.2.2, Python 2.2.3, Python 2.3, Python 2.4, Python 2.5, Python 2.5.3, Python 2.6, Python 2.7, Python 3.0, Python 3.1 ___ Python tracker <http://bugs.python.org/issue4686> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4694] _call_method() in multiprocessing documentation
New submission from David M. Beazley : The documentation for "Proxy Objects" in the multiprocessing module describes a method "_call_method" and gives various examples. The only problem is that the method is actually called "_callmethod" (i.e., no underscore between "call" and "method"). -- assignee: georg.brandl components: Documentation messages: 78038 nosy: beazley, georg.brandl severity: normal status: open title: _call_method() in multiprocessing documentation type: behavior versions: Python 2.6, Python 3.0 ___ Python tracker <http://bugs.python.org/issue4694> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4694] _call_method() in multiprocessing documentation
David M. Beazley added the comment: The _get_value() method is also in error. It's called "_getvalue()" in the source code. ___ Python tracker <http://bugs.python.org/issue4694> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4695] Bad AF_PIPE address in multiprocessing documentation
New submission from David M. Beazley : In the "Address Formats" part of the "Listeners and Clients" section of the documentation for the multiprocessing module, AF_PIPE addresses are described as having this format: r'ServerName\\pipe\\PipeName' However, it is really this: r'\\ServerName\pipe\PipeName' Be careful with raw strings. The documentation is showing the output of repr(), not a properly formed raw string. I verified this fix on Windows XP. -- assignee: georg.brandl components: Documentation messages: 78041 nosy: beazley, georg.brandl severity: normal status: open title: Bad AF_PIPE address in multiprocessing documentation versions: Python 2.6, Python 3.0 ___ Python tracker <http://bugs.python.org/issue4695> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1194378] sendmsg() and recvmsg() for C socket module
David M. Beazley added the comment: Bump. This functionality seems to be needed if anyone is going to be messing around with advanced features of IPv6. As it stands, the socket module in Python 2.6/3.0 is incomplete without this. -- nosy: +beazley ___ Python tracker <http://bugs.python.org/issue1194378> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4744] asynchat documentation needs to be more precise
New submission from David M. Beazley : The documentation for asynchat needs to be more precise in its use of strings vs. bytes. Unless the undocumented use_encoding attribute is set, it seems that all data should be bytes throughout (e.g., the terminator, inputs to push methods, etc.). I have no idea if the use_encoding attribute is officially blessed or not. However, to avoid "magic behavior", I'm guessing that it would be better practice to be explicit in one's use of bytes vs. text rather than having take place in the internals of asynchat. Advice welcome. -- assignee: georg.brandl components: Documentation messages: 78277 nosy: beazley, georg.brandl severity: normal status: open title: asynchat documentation needs to be more precise versions: Python 3.0 ___ Python tracker <http://bugs.python.org/issue4744> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1194378] sendmsg() and recvmsg() for C socket module
David M. Beazley added the comment: Just a followup comment to note that adding support for sendmsg()/recvmsg() is what you need to do "file descriptor passing" between processes on Unix---another technique for writing network servers. ___ Python tracker <http://bugs.python.org/issue1194378> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4758] Python 3.0 internet documentation needs work
New submission from David M. Beazley : I have recently completed a pretty thorough survey of library documentation for Python 3.0 in conjunction with an update I'm making to my book. This issue is not so much a bug as a documentation request. For all of the library modules related to network programming, it would be extremely useful to be much very explicit about what methods work with strings and what methods requires byte. So many of these modules operate on small fragments of data (e.g., send a request, add a header, parse a query string, etc.). Sometimes using a string is okay, sometimes it's not and sadly, it's not often predictable. Part of the problem is that the documentation has been written for a Python 2 world where text strings and binary data were interchangeable. Anyways, this request minimally covers these modules: ftplib smtplib nntplib http.* urllib.* xmlrpc.* socketserver asynchat asyncore If there is interest, I can submit more detailed notes from my own work, but I'm not sure how the documentation maintainer would want this. Separate issue for each? Added as comments here? Please advise. -- assignee: georg.brandl components: Documentation messages: 78361 nosy: beazley, georg.brandl severity: normal status: open title: Python 3.0 internet documentation needs work type: feature request versions: Python 3.0 ___ Python tracker <http://bugs.python.org/issue4758> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4766] email documentation needs to be precise about strings/bytes
New submission from David M. Beazley : Documentation for the email package needs to be more clear about the usage of strings and bytes. In particular: 1. All operations that parse email messages such as message_from_file() or message_from_string() operate on *text*, not binary data. So, the file must be opened in text mode. Strings must be text strings, not binary strings. 2. All operations that set/get the payload of a message operate on byte strings. For example, using m.get_payload() on a Message object returns binary data as a byte string. Opinion: There might be other bug reports about this, but I'm not advocating that the email module should support reading messages from binary mode files or byte strings. Email and MIME were originally developed with the assumption that messages would always be handled as text. Minimally, this assumed that messages would stay intact even if processed as 7-bit ASCII. By extension, everything should still work if processed as Unicode. So, I think the use of text-mode files is entirely consistent with this if you wanted to keep the module "as is." There may be some confusion on this matter because if you're reading or writing email messages (or sending them across a socket), you may encounter messages stored in the form of bytes strings instead of text. People will then wonder why a byte string can't be parsed by this module (especially given that email messages only use character values in the range of 0-127). -- assignee: georg.brandl components: Documentation messages: 78456 nosy: beazley, georg.brandl severity: normal status: open title: email documentation needs to be precise about strings/bytes type: behavior versions: Python 3.0 ___ Python tracker <http://bugs.python.org/issue4766> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4767] email.mime incorrectly documented (or implemented)
New submission from David M. Beazley : The documentation describes classes such as email.mime.MIMEText() email.mime.MIMEMultipart() email.mime.MIMEApplication() etc... However, it's confusing because none of these classes are actually found in email.mime. Suggest either using the full proper name: email.mime.text.MIMEText() Or just using the short name along with a note saying where it's found: MIMEText() Defined in email.mime.text. Further description, blah, blah.. Note: These classes *are* defined in email.mime in Python 2.6. -- assignee: georg.brandl components: Documentation messages: 78458 nosy: beazley, georg.brandl severity: normal status: open title: email.mime incorrectly documented (or implemented) type: behavior versions: Python 3.0 ___ Python tracker <http://bugs.python.org/issue4767> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4768] email.generator.Generator object bytes/str crash - b64encode() bug?
New submission from David M. Beazley :
The email.generator.Generator class does not work correctly message
objects created with binary data (MIMEImage, MIMEAudio, MIMEApplication,
etc.). For example:
>>> from email.mime.image import MIMEImage
>>> data = open("IMG.jpg","rb").read()
>>> m = MIMEImage(data,'jpeg')
>>> s = m.as_string()
Traceback (most recent call last):
File "", line 1, in
File "/tmp/lib/python3.0/email/message.py", line 136, in as_string
g.flatten(self, unixfrom=unixfrom)
File "/tmp/lib/python3.0/email/generator.py", line 76, in flatten
self._write(msg)
File "/tmp/lib/python3.0/email/generator.py", line 101, in _write
self._dispatch(msg)
File "/tmp/lib/python3.0/email/generator.py", line 127, in _dispatch
meth(msg)
File "/tmp/lib/python3.0/email/generator.py", line 155, in
_handle_text
raise TypeError('string payload expected: %s' % type(payload))
TypeError: string payload expected:
>>>
The source of the problem is rather complicated, but here is the gist of
it.
1. Classes such as MIMEAudio and MIMEImage accept raw binary data as
input. This data is going to be in the form of bytes.
2. These classes immediately encode the data using a base64 encoder.
This encoder uses the library function base64.b64encode().
3. base64.b64encode() takes a byte string as input and returns a byte
string as output. So, even after encoding, the payload of the message
is of type 'bytes'
4. When messages are generated, the method Generator._dispatch() is
used. It looks at the MIME main type and subtype and tries to dispatch
message processing to a handler method of the form
'_handle_type_subtype'.If it can't find such a handler, it defaults
to a method _writeBody(). For image and audio types, this is what
happens.
5. _writeBody() is an alias for _handle_text().
6. _handle_text() crashes because it's not expecting a payload of type
'bytes'.
Suggested fix:
I think the library function base64.b64encode() should return a string,
not bytes. The whole point of base64 encoding is to take binary data
and encode it into characters safe for inclusion in text strings.
Other fixes:
Modify the Generator class in email.generator to properly detect bytes
and use a different _handle function for it. For instance, maybe add a
_handle_binary() method.
--
components: Library (Lib)
messages: 78464
nosy: beazley
severity: normal
status: open
title: email.generator.Generator object bytes/str crash - b64encode() bug?
type: crash
versions: Python 3.0
___
Python tracker
<http://bugs.python.org/issue4768>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4769] b64decode should accept strings or bytes
New submission from David M. Beazley :
The whole point of base64 encoding is to safely encode binary data into
text characters. Thus, the base64.b64decode() function should equally
accept text strings or binary strings as input. For example, there is a
reasonable expectation that something like this should work:
>>> x = 'SGVsbG8='
>>> base64.b64decode(x)
b'Hello'
>>>
In Python 3, you get this exception however:
>>> base64.b64decode(x)
Traceback (most recent call last):
File "", line 1, in
File "/tmp/lib/python3.0/base64.py", line 80, in b64decode
raise TypeError("expected bytes, not %s" % s.__class__.__name__)
TypeError: expected bytes, not str
>>>
I realize that there are encoding issues with Unicode strings, but
base64 encodes everything into the first 127 ASCII characters. If the
input to b64decode is a str, just do a encode('ascii') operation on it
and proceed. If that fails, it wasn't valid Base64 to begin with.
I can't think of any real negative impact to making this change as long
as the result is still always bytes. The main benefit is just
simplifying the decoding process for end-users.
See issue 4768.
--
components: Library (Lib)
messages: 78466
nosy: beazley
severity: normal
status: open
title: b64decode should accept strings or bytes
type: behavior
versions: Python 3.0
___
Python tracker
<http://bugs.python.org/issue4769>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4769] b64decode should accept strings or bytes
David M. Beazley added the comment: Note: This problem applies to all of the other decoders/encoders in the base64 too (b16, b32, etc.) ___ Python tracker <http://bugs.python.org/issue4769> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4770] binascii module, crazy error messages, unexpected behavior
New submission from David M. Beazley :
See Issue 4869 for a related bug.
Most of the functions in binascii are meant to go from binary data to
textual representations (hex digits, base64, binhex, etc.). There are
numerous problems:
1. Misleading error messages. For example:
>>> binascii.b2a_base64("Some text")
Traceback (most recent call last):
File "", line 1, in
TypeError: b2a_base64() argument 1 must be string or buffer, not str
>>> binascii.crc32("Some text")
Traceback (most recent call last):
File "", line 1, in
TypeError: crc32() argument 1 must be string or buffer, not str
>>>
Huh? Didn't I just pass a string? Error message should say "argument
1 must be bytes or buffer, not str".
This problem shows up with most of the other encoding functions too
(i.e., b2a_uu).
2. Expected behavior with encoding/decoding.
The functions in this module are going from binary to text. To be
consistent, I think the result of encoding operations such as b2a_uu(),
b2a_base64(), should be strings, not bytes.
Similarly, decoding operations are going from text back to bytes. I
think the input arguments should be allowed to be str (in addition to
bytes if you want).
--
components: Library (Lib)
messages: 78470
nosy: beazley
severity: normal
status: open
title: binascii module, crazy error messages, unexpected behavior
type: behavior
versions: Python 3.0
___
Python tracker
<http://bugs.python.org/issue4770>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4770] binascii module, crazy error messages, unexpected behavior
David M. Beazley added the comment: Given the low-level nature of this module, I can understand the motivation to make it all bytes. However, I'm going to respectfully disagree with that and claim that making binascii all bytes really goes against the whole spirit of what Python 3.0 has tried to do for Unicode. For example, throughout Python, you now have a clean separation between binary data (bytes) and text data (str). Well, it's cleanly separated everywhere except in the binascii module (and base64 module) which, ironically, is all about converting between binary data and text. As it stands now, it's a huge wart IMHO. ___ Python tracker <http://bugs.python.org/issue4770> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4771] Bad examples in hashlib documentation
New submission from David M. Beazley :
The hashlib documentation has incorrect examples showing the use of the
hexdigest() method:
>>> hashlib.sha224(b"Nobody inspects the spammish
repetition").hexdigest()
b'a4337bc45a8fc544c03f52dc550cd6e1e87021bc896588bd79e901e2'
>>>
and this one
>>> h = hashlib.new('ripemd160')
>>> h.update(b"Nobody inspects the spammish repetition")
>>> h.hexdigest()
b'cc4a5ce1b3df48aec5d22d1f16b894a0b894eccc'
>>>
However, the result of h.hexdigest() is of type 'str', not bytes. Actual
output:
>>> h.hexdigest()
'cc4a5ce1b3df48aec5d22d1f16b894a0b894eccc'
>>>
Sure would be nice if that string of hex digits was easy to decode back
into a binary string.
>>> import binascii
>>> b = binascii.a2b_hex(h.hexdigest())
>>>
Hmmm. So *SOME* of the functions in binascii do accept Unicode strings.
See Issue 4470 :-).
--
assignee: georg.brandl
components: Documentation
messages: 78480
nosy: beazley, georg.brandl
severity: normal
status: open
title: Bad examples in hashlib documentation
type: behavior
versions: Python 3.0
___
Python tracker
<http://bugs.python.org/issue4771>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4771] Bad examples in hashlib documentation
David M. Beazley added the comment: The digest() method of hashes does produce bytes (correct). The hexdigest() method produces a string, but it is also shown as producing bytes in the examples. ___ Python tracker <http://bugs.python.org/issue4771> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4773] HTTPMessage not documented and has inconsistent API across 2.6/3.0
New submission from David M. Beazley :
A file-like object u returned by the urlopen() function in both Python
2.6/3.0 has a method info() that returns a 'HTTPMessage' object. For
example:
::: Python 2.6
>>> from urllib2 import urlopen
>>> u = urlopen("http://www.python.org";)
>>> u.info()
>>>
::: Python 3.0
>>> from urllib.request import urlopen
>>> u = urlopen("http://www.python.org";)
>>> u.info()
>>>
So far, so good. HTTPMessage is defined in two different modules, but
that's fine (it's just library reorganization).
Two major problems:
1. There is no documentation whatsoever on HTTPMessage. No description
in the docs for httplib (python 2.6) or http.client (python 3.0).
2. The HTTPMessage object in Python 2.6 derives from mimetools.Message
and has a totally different programming interface than HTTPMessage in
Python 3.0 which derives from email.message.Message. Check it out:
:::Python 2.6
>>> dir(u.info())
['__contains__', '__delitem__', '__doc__', '__getitem__', '__init__',
'__iter__', '__len__', '__module__', '__setitem__', '__str__',
'addcontinue', 'addheader', 'dict', 'encodingheader', 'fp', 'get',
'getaddr', 'getaddrlist', 'getallmatchingheaders', 'getdate',
'getdate_tz', 'getencoding', 'getfirstmatchingheader', 'getheader',
'getheaders', 'getmaintype', 'getparam', 'getparamnames', 'getplist',
'getrawheader', 'getsubtype', 'gettype', 'has_key', 'headers',
'iscomment', 'isheader', 'islast', 'items', 'keys', 'maintype',
'parseplist', 'parsetype', 'plist', 'plisttext', 'readheaders',
'rewindbody', 'seekable', 'setdefault', 'startofbody', 'startofheaders',
'status', 'subtype', 'type', 'typeheader', 'unixfrom', 'values']
:::Python 3.0
>>> dir(u.info())
['__class__', '__contains__', '__delattr__', '__delitem__', '__dict__',
'__doc__', '__eq__', '__format__', '__ge__', '__getattribute__',
'__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__',
'__len__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__',
'__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__',
'__str__', '__subclasshook__', '__weakref__', '_charset',
'_default_type', '_get_params_preserve', '_headers', '_payload',
'_unixfrom', 'add_header', 'as_string', 'attach', 'defects',
'del_param', 'epilogue', 'get', 'get_all', 'get_boundary',
'get_charset', 'get_charsets', 'get_content_charset',
'get_content_maintype', 'get_content_subtype', 'get_content_type',
'get_default_type', 'get_filename', 'get_param', 'get_params',
'get_payload', 'get_unixfrom', 'getallmatchingheaders', 'is_multipart',
'items', 'keys', 'preamble', 'replace_header', 'set_boundary',
'set_charset', 'set_default_type', 'set_param', 'set_payload',
'set_type', 'set_unixfrom', 'values', 'walk']
I know that getting rid of mimetools was desired, but I have no idea if
changing the API on HTTPMessage was intended or not. In any case, it's
one of the only cases in the entire library where the programming
interface to an object radically changes from 2.6 -> 3.0.
I ran into this problem with code that was trying to properly determine
the charset encoding of the byte string returned by urlopen().
I haven't checked whether 2to3 deals with this or not, but it might be
something for someone to look at in their copious amounts of spare time.
--
components: Library (Lib)
messages: 78486
nosy: beazley
severity: normal
status: open
title: HTTPMessage not documented and has inconsistent API across 2.6/3.0
type: behavior
versions: Python 2.6, Python 3.0
___
Python tracker
<http://bugs.python.org/issue4773>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4773] HTTPMessage not documented and has inconsistent API across 2.6/3.0
David M. Beazley added the comment: Verified that 2to3 does not fix this. ___ Python tracker <http://bugs.python.org/issue4773> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4782] json documentation missing load(), loads()
New submission from David M. Beazley : Documentation for the json module in Python 2.6 and Python 3.0 doesn't have any description for load() or loads() even though both functions are used in the examples. -- assignee: georg.brandl components: Documentation messages: 78542 nosy: beazley, georg.brandl severity: normal status: open title: json documentation missing load(), loads() versions: Python 2.6, Python 3.0 ___ Python tracker <http://bugs.python.org/issue4782> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4783] json documentation needs a BAWM (Big A** Warning Message)
New submission from David M. Beazley :
The json module is described as having an interface similar to pickle:
json.dump()
json.dumps()
json.load()
json.loads()
I think it would be a WISE idea to add a huge warning message to the
documentation that these functions should *NOT* be used to serialize or
unserialize multiple objects on the same file stream like pickle. For
example:
f = open("stuff","w")
json.dump(obj1, f)
json.dump(obj2, f)# NO! FLAMING DEATH!
f = open("stuff","r")
obj1 = json.load(f)
obj2 = json.load(f) # NO! EXTRA CRIPSY FLAMING DEATH!
For one, it doesn't work. load() actually reads the whole file into a
big string and tries to parse it as a single object. If there are
multiple objects in the file, you get a nasty exeption. Second, I'm
not even sure this is technically allowed by the JSON spec.
As far as I call tell, concatenating JSON objects together in the same
file falls into the same category as concatenating two HTML documents
together in the same file (something you just don't do).
Related: json.load() should probably not be used on any streaming input
source such as a file wrapped around a socket. The first thing it does
is consume the entire input by calling f.read()---which probably not
what someone is expecting (and it might even cause the whole program to
hang).
--
assignee: georg.brandl
components: Documentation
messages: 78547
nosy: beazley, georg.brandl
severity: normal
status: open
title: json documentation needs a BAWM (Big A** Warning Message)
type: behavior
versions: Python 2.6, Python 3.0
___
Python tracker
<http://bugs.python.org/issue4783>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4785] json.JSONDecoder() strict argument undocumented and potentially confusing
New submission from David M. Beazley :
The strict parameter to JSONDecoder() is undocumented and is confusing
because someone might assume it has something to do with the encoding
parameter or the general handling of parsing errors (which it doesn't).
As far as I can determine by reading the source, strict determines
whether or not JSON strings are allowed to contain literal newlines in
them or not. For example (note: loads() passes its parameters to
JSONDecoder):
>>> s = '{"test":"Hello\nWorld"}'
>>> print(s)
{"test":"Hello
World"}
>>> json.loads(s)
Traceback (most recent call last):
...
File "/tmp/lib/python3.0/json/decoder.py", line 159, in JSONString
return scanstring(match.string, match.end(), encoding, strict)
ValueError: Invalid control character at: line 1 column 14 (char 14)
>>> json.loads(s,strict=False)
{'test': 'Hello\nWorld'}
>>>
Note in this last example how the result has the literal newline
embedded in it when strict is set False.
--
assignee: georg.brandl
components: Documentation
messages: 78550
nosy: beazley, georg.brandl
severity: normal
status: open
title: json.JSONDecoder() strict argument undocumented and potentially confusing
type: behavior
versions: Python 2.6, Python 3.0
___
Python tracker
<http://bugs.python.org/issue4785>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4769] b64decode should accept strings or bytes
David M. Beazley added the comment:
One more followup. The quopri module (which is highly related to
base64 in that quopri and base64 are often used together within MIME)
does accept both unicode and byte strings when decoding. For example,
this works:
>>> quopri.decodestring('Hello World')
b'Hello World'
>>> quopri.decodestring(b'Hello World')
b'Hello World'
>>>
However, the quopri module, like base64, uses byte strings almost
everywhere else. For example, encoding a byte string with quopri still
produces bytes (just like base64)
>>> quopri.encodestring(b'Hello World')
b'Hello World'
>>>
___
Python tracker
<http://bugs.python.org/issue4769>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4783] json documentation needs a BAWM (Big A** Warning Message)
David M. Beazley added the comment: Just consider me to be an impartial outside reviewer. Hypothetically, let's say I'm a Python programmer who knows a thing or two about standard library modules (like pickle), but I'm new to JSON so I come looking at the json module documentation. The documentation tells me it uses the same interface as pickle and marshal (even naming those two modules right off the bat). So, right away, I'm thinking the module probably does all of the usual things that pickle and marshal can do. For instance, serializing multiple objects to the same stream. However, it doesn't work this way and the only way to find out that it doesn't work is to either try it and get an error, or to read the source code and figure it out. I'm not reporting this as an end-user of the json module, but as a Python book author who is trying to get things right and to be precise. I think if you're going to keep the pickle and marshal reference I would add the warning message. Otherwise, I wouldn't mention pickle or marshal at all. ___ Python tracker <http://bugs.python.org/issue4783> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4783] json documentation needs a BAWM (Big A** Warning Message)
David M. Beazley added the comment: Thanks! Hopefully I'm not giving you too much work to do :-). Cheers, Dave ___ Python tracker <http://bugs.python.org/issue4783> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4786] xml.etree.ElementTree module name in Python 3
New submission from David M. Beazley : Not a bug, but a question to developers: Is xml.etree.ElementTree going to be the only standard library module in Python 3.0 that doesn't follow the standard Python 3.0 module naming conventions? (e.g., socketserver, configparser, etc.). Are there any plans to rename it to xml.etree.elementtree? Just curious. -- components: Library (Lib) messages: 78560 nosy: beazley severity: normal status: open title: xml.etree.ElementTree module name in Python 3 versions: Python 3.0 ___ Python tracker <http://bugs.python.org/issue4786> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4820] ctypes.util.find_library incorrectly documented
New submission from David M. Beazley : In the "ctypes reference / Finding shared libraries" section of the ctypes documentation, the find_library() function is described as being located in ctypes.util. However, it's formal description right below that lists it as ctypes.find_library(). -- assignee: georg.brandl components: Documentation messages: 78964 nosy: beazley, georg.brandl severity: normal status: open title: ctypes.util.find_library incorrectly documented versions: Python 3.0 ___ Python tracker <http://bugs.python.org/issue4820> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4831] exec() behavior - revisited
David M. Beazley added the comment: One further followup just to make sure I'm clear. Is it always safe to pass the result of locals() into exec and extract the result as shown in my example? Since I'm writing about this in a book, I just want to make absolutely certain I know what's going on and that I don't tell people something that's completely bogus. Thanks! ___ Python tracker <http://bugs.python.org/issue4831> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4831] exec() behavior - revisited
New submission from David M. Beazley :
Please forgive me, but I'm really trying to wrap my brain around the
behavior of exec() in Python 3. Here's a quote from the documentation:
"In all cases, if the optional parts are omitted, the code is
executed in the current scope."
This is referring to the optional use of the globals/locals parameters
and seems to indicate that if they're omitted the code executes in the
scope where the exec() appeared.
Yet, this code fails:
def foo():
exec("a = 42")
print(a) # NameError: a
Now, I realize that exec() became a function in Python 3. However,
regardless of that, is it really the intent that exec() not be allowed
to ever modify any local variable of a function? In other words, do I
really have to do this?
def foo():
ldict = locals()
exec("a=42",globals(),ldict)
a = ldict['a']
print(a)
I submitted a bug report about this once before and it was immediately
dismissed.
I would appreciate some greater clarity on this matter this go around.
Specifically, what is the approved way to have exec() modify the local
environment of a function?
--
components: Interpreter Core
messages: 79059
nosy: beazley
severity: normal
status: open
title: exec() behavior - revisited
type: behavior
versions: Python 3.0
___
Python tracker
<http://bugs.python.org/issue4831>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4903] binascii.crc32()
New submission from David M. Beazley :
The result of binascii.crc32() is different on the same input in Python
2.6/3.0. For example:
Python 2.6:
>>> binascii.crc32('Hello')
-137262718
Python 3.0:
>>> binascii.crc32(b'Hello')
4157704578
--
components: Library (Lib)
messages: 79524
nosy: beazley
severity: normal
status: open
title: binascii.crc32()
type: behavior
versions: Python 2.6, Python 3.0
___
Python tracker
<http://bugs.python.org/issue4903>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4903] binascii.crc32()
David M. Beazley added the comment: Can someone PLEASE make sure this gets documented someplace. ___ Python tracker <http://bugs.python.org/issue4903> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4903] binascii.crc32()
David M. Beazley added the comment: Placing a note in the standard library documentation would be a start. Just say in Python 3.0 it always returns the result as an unsigned integer whereas in Python 2.6 a 32-bit signed integer is returned. Although the numerical value may differ between versions, the underlying bits are the same. Use crc32() & 0x to get a consistent value (already noted). Note: Not everyone uses checksums in only a packed-binary format. Having the integer value just change across Python versions like that is a real subtle compatibility problem to point out. ___ Python tracker <http://bugs.python.org/issue4903> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14704] NameError Issue in Multiprocessing
New submission from David M. Rogers :
Python Devs,
There is an issue relating to variable lookup using exec from within
multiprocessing's fork()-ed process. I'm attempting to use the forked process
as a generic remote python shell, but exec is unable to reach variables from
within functions. This issue makes it impossible to define a function which
uses un-passed variables defined in the remote process.
The simplest way to reproduce the error is:
--- err.py ---
from multiprocessing import Process, Pipe
def run_remote(con, name):
my_name = name
for i in range(2):
code = con.recv()
exec code
me, he = Pipe()
p = Process(target=run_remote,
args=(he, "Sono Inglese de Gerrards Cross."))
p.start()
me.send("print my_name") # works
me.send("""
def show_name():
print my_name
show_name() # doesn't work
""")
--- end err.py ---
This program prints:
$ python2.6 err.py
Sono Inglese de Gerrards Cross.
Process Process-1:
Traceback (most recent call last):
File "/sw/lib/python2.6/multiprocessing/process.py", line 232, in _bootstrap
self.run()
File "/sw/lib/python2.6/multiprocessing/process.py", line 88, in run
self._target(*self._args, **self._kwargs)
File "err.py", line 7, in run_remote
exec code
File "", line 4, in
File "", line 3, in show_name
NameError: global name 'my_name' is not defined
I'm using Mac OSX (10.6.8) and
Python 2.6.5 (r265:79063, Sep 23 2010, 14:05:02)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
The issue (with the same traceback) also occurs for:
Python 2.7 (r27:82500, Sep 29 2010, 15:34:46)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Using exactly the same set of exec calls locally results in the correct
behavior.
--- noerr.py ---
my_name = "Sono Inglese de Gerrards Cross."
exec "print my_name"
exec """
def show_name():
print my_name
show_name()
"""
--- end noerr.py ---
--
components: None
messages: 159764
nosy: frobnitzem
priority: normal
severity: normal
status: open
title: NameError Issue in Multiprocessing
versions: Python 2.6
___
Python tracker
<http://bugs.python.org/issue14704>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16856] Segfault from calling repr() on a dict with a key whose repr raise an exception
New submission from David M. Cooke:
The following segfaults:
class A(int):
def __repr__(self):
raise Exception()
a = A()
d = {a : 1}
repr(d)
This is with Python 3.3.0, running on Mac OS 10.7.5, from MacPorts:
Python 3.3.0 (default, Sep 29 2012, 08:16:08)
[GCC 4.2.1 Compatible Apple Clang 3.1 (tags/Apple/clang-318.0.58)] on darwin
--
components: Interpreter Core
messages: 178997
nosy: david.m.cooke
priority: normal
severity: normal
status: open
title: Segfault from calling repr() on a dict with a key whose repr raise an
exception
type: crash
versions: Python 3.3
___
Python tracker
<http://bugs.python.org/issue16856>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21830] ssl.wrap_socket fails on Windows 7 when specifying ca_certs
New submission from David M Noriega:
When trying to use python3-ldap package on Windows 7, found I could not get a
TLS connection to work and traced it to its use of ssl.wrap_socket. Trying out
the following simple socket test fails
import socket
import ssl
sock = socket.socket()
sock.connect(("host.name", 636))
ssl = ssl.wrap_socket(sock, cert_reqs=ssl.CERT_REQUIRED,
ca_certs=r"C:path\to\cert\file")
Traceback (most recent call last):
File "", line 1, in
sock = ssl.wrap_socket(sock, cert_reqs=ssl.CERT_REQUIRED,
ca_certs=r"F:\Downloads\csbc-cacert.pem")
File "C:\Python34\lib\ssl.py", line 888, in wrap_socket
ciphers=ciphers)
File "C:\Python34\lib\ssl.py", line 511, in __init__
self._context.load_verify_locations(ca_certs)
ssl.SSLError: unknown error (_ssl.c:2734)
This code works on Windows XP(and of course linux) and I'm able to use
getpeercert()
A workaround I was able to figure out was to use ssl.SSLContext in conjunction
with Windows central certificate store. By first loading my CA cert into the
trusted root cert store, I could use SSLContext.load_default_certs() to create
an ssl socket.
--
components: Windows
messages: 221373
nosy: David.M.Noriega
priority: normal
severity: normal
status: open
title: ssl.wrap_socket fails on Windows 7 when specifying ca_certs
versions: Python 3.4
___
Python tracker
<http://bugs.python.org/issue21830>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21830] ssl.wrap_socket fails on Windows 7 when specifying ca_certs
David M Noriega added the comment: Oops, thats what I get for running with scissors. Yes, the cert file is in pem format. Its the same file in use on my ldap server and all my servers and workstations that authenticate against it. I have an existing python 2.x script using the python-ldap(different from python3-ldap) module that uses this exact same file and works correctly. I've tested with the socket code above on python 2 and 3 and it works on my linux systems and on Windows XP. I only get this error on a Windows 7 system. -- ___ Python tracker <http://bugs.python.org/issue21830> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7322] Socket timeout can cause file-like readline() method to lose data
New submission from David M. Beazley : Consider a socket that has had a file-like wrapper placed around it using makefile() # s is a socket created previously f = s.makefile() Now, suppose that this socket has had a timeout placed on it. s.settimeout(15) If you try to read data from f, but nothing is available. You'll eventually get a timeout. For example: f.readline() # Now, just wait Traceback (most recent call last): File "", line 1, in File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket. py", line 406, in readline data = self._sock.recv(self._rbufsize) socket.timeout: timed out However, now consider the case where you're reading a line of data, but the receiver has only received a partial line and it's waiting for the rest of the data to arrive. For example, type this: f.readline() Now, go to the other end of the socket connection and send a buffer with no newline character. For example, send the message "Hello". Since no newline character has been received, the readline() method will eventually fail with a timeout as before. However, if you now retry the read operation f.readline() and send more data such as the message "World\n", you'll find that the "Hello" message gets lost. In other words, the repeated readline() operation discards any buffers corresponding to previously received line data and just returns the new data. Admittedly this is a corner case, but you probably don't want data to be discarded on a TCP connection even if a timeout occurs. Hope that makes some sense :-). (It helps to try it out). -- components: Library (Lib) messages: 95245 nosy: beazley severity: normal status: open title: Socket timeout can cause file-like readline() method to lose data type: behavior versions: Python 2.6 ___ Python tracker <http://bugs.python.org/issue7322> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4758] Python 3.x internet documentation needs work
David M. Beazley added the comment: An apology on the delay. Things have been rather hectic. Regarding a patch, I don't really have a patch so much as a suggested procedure. Basically, I'm suggesting that the maintainers of the library documentation simply do a quick survey of network related modules and make it clear that many of the operations work on "byte strings" and not "strings." In Python 2.X, you could get away with being a little sloppy, but in Python 3, the bytes/strings distinction becomes much more prominent. If I have time, I might be able to make a specific patch, but it probably wouldn't be until after PyCON sometime. -- ___ Python tracker <http://bugs.python.org/issue4758> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
