Re: Are dicts supposed to raise comparison errors
On Wed, 01 Aug 2018 19:00:27 +0100, Paul Moore wrote: [...] > My point was that it's a *warning*, and as such it's perfectly possible > for a warning to *not* need addressing (other than to suppress or ignore > it once you're happy that doing so is the right approach). And my point was that ignoring warnings is not the right approach. Suppressing them on a case-by-case basis (if possible) is one thing, but a blanket suppression goes to far, for the reasons I gave earlier. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On Wed, 01 Aug 2018 22:14:54 +0300, Serhiy Storchaka wrote: > 01.08.18 21:03, Chris Angelico пише: >> And in any code that does not and cannot run on Python 2, the warning >> about bytes and text comparing unequal is nothing more than a false >> positive. > > Not always. If your code supported Python 2 in the past, or third-party > dependencies supports or supported Python 2, this warning can expose a > real bug. Even if all your and third-party code always was Python 3 > only, the standard library can contain such kind of bugs. > > Several years after the EOL of Python 2.7 and moving all living code to > Python 3 we can ignore bytes warnings as always false positive. Even then, I don't know that we should do that. I do not believe that the EOL of Python 2 will end all confusion between byte strings and text strings. There is ample opportunity for code to accidentally compare bytes and text even in pure Python 3 code, e.g. comparing data read from files reading from files which are supposed to be opened in the same binary/text mode but aren't. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
asyncio: Warning message when waiting for an Event set by AbstractLoop.add_reader
Hello list,
During my attempt to bring asyncio support to the multiprocessing Queue,
I found warning messages when executing my code with asyncio debug
logging enabled.
It seems that awaiting for an Event will make theses messages appears
after the second attempt to wait for the Event.
Here is a code sample to test this (47 lines long):
import asyncio
import threading
import os
import io
import time
import logging
logging.basicConfig()
logging.getLogger('asyncio').setLevel(logging.DEBUG)
fd1, fd2 = os.pipe() # (r, w)
loop = asyncio.get_event_loop()
event = asyncio.Event()
def readfd(fd, size):
buf = io.BytesIO()
handle = fd
remaining = size
while remaining > 0:
chunk = os.read(handle, remaining)
n = len(chunk)
if n == 0:
raise Exception()
buf.write(chunk)
remaining -= n
return buf.getvalue()
def f():
loop.add_reader(fd1, event.set)
loop.run_until_complete(g())
loop.close()
async def g():
while True:
await event.wait()
msg = readfd(fd1, 4)
event.clear()
if msg == b'STOP':
break
print(msg)
if __name__ == '__main__':
t = threading.Thread(target=f)
t.start()
time.sleep(1)
os.write(fd2, b'TST1')
time.sleep(1)
os.write(fd2, b'TST2')
time.sleep(1)
os.write(fd2, b'TST3')
time.sleep(1)
os.write(fd2, b'STOP')
t.join()
I get this output:
DEBUG:asyncio:Using selector: EpollSelector
DEBUG:asyncio:poll took 999.868 ms: 1 events
b'TST1'
b'TST2'
WARNING:asyncio:Executing took 1.001 seconds
INFO:asyncio:poll took 1000.411 ms: 1 events
b'TST3'
WARNING:asyncio:Executing took 1.001 seconds
DEBUG:asyncio:Close <_UnixSelectorEventLoop [CUT]>
I don't get the message, because it seems that the event is awaited
properly. I'm especially bugged, because I get this output from the
actual code I'm writing:
WARNING:asyncio:Executing took 1.000 seconds
DEBUG:asyncio:poll took 0.012 ms: 1 events
Here, it seems that "polling" went for 0.012 ms. But it should have gone
for 1000.0 ms. But in any case, it still works. However, the fact that
polling seems to "return" instantly is a strange fact.
The test code is not producing this difference. In fact, it seems that
polling is working as intended (Polling takes 1000.0 ms and Event is
awaited for 1000.0 ms). But there is still this warning...
Maybe one of you have an insight on this ?
--
https://mail.python.org/mailman/listinfo/python-list
Re: Dealing with errors in interactive subprocess running python interpreter that freeze the process
On Wed, 1 Aug 2018 at 21:17, wrote:
>
> I can run python3 interactively in a subprocess w/ Popen but
> if I sent it text, that throws an exception, the process freezes
> instead of just printing the exception like the normal interpreter..
> why? how fix? Here is my code below.
>
> (I suspect when there is an exception, there is NO output to stdin so that
> the problem is the line below that tries to read from stdin never finishes.
> Maybe I need a different readline that can "survive" when there is no output
> and won't block?)
>
>
>
> import subprocess
>
> interpreter = subprocess.Popen(['python3', '-i'],
>stdin = subprocess.PIPE,
>stdout = subprocess.PIPE,
>stderr = subprocess.PIPE)
>
> while True:
> exp = input(">>> ").encode() + b"\n"
> interpreter.stdin.write(exp)
> interpreter.stdin.flush()
> print(interpreter.stdout.readline().strip())
> interpreter.stdin.close()
> interpreter.terminate()
You're only reading one line from stdout, but an exception is multiple
lines. So the subprocess is still trying to write while you're wanting
to give it input again. This is a classic way to get a deadlock, which
is basically what you're seeing. Add to that the fact that there are
likely IO buffers in the subprocess that mean it's not necessarily
passing output back to you at the exact time you expect it to (and the
subprocess probably has different buffering behaviour when the IO is
to pipes rather than to the console) and it gets complex fast.
As others have mentioned, separate threads for the individual pipes
may help, or if you need to go that far there are specialised
libraries, I believe (pexpect is one, but from what I know it's fairly
Unix-specific, so I'm not very familiar with it).
Sorry, but there's no "simple" answer here for you (although you may
well be able to get something that works well enough for your specific
needs - but it's not obvious from your snippet of code what you're
trying to achieve).
Paul
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 01/08/2018 18:19, Peter Otten wrote: I've looked into the actual code which has # paraparser.py f = isPy3 and asBytes or asUnicode K = list(known_entities.keys()) for k in K: known_entities[f(k)] = known_entities[k] It looks like known_entities starts out with the default string type, i. e. unicode in py3 and bytes in py2. While in py2 the code has no effect in py3 it adds the corresponding keys of type bytes. However, known_entities is then used in HTMLParser.handle_entity_ref(self, name) which passes the name as unicode in py3. > I didn't try, but I would suspect that the module keeps working as expected when you remove the lines quoted above. I did try and all the tests pass in 2.7.x & >=3.3; the commit message says "fix entityref handling" and happened during the python 3.x porting. I suppose there was some issue, but its entirely probable that some later change (eg parser) has fixed the original problem and made this code redundant. If I'm correct running the program with the -b flag has at least helped in cleaning up the code in this case. and I will try running all tests under that flag; it's sure to find more issues. In other cases it might detect sources of bugs, so IMHO it's better to have a close look at and possibly rewrite code that triggers the warning rather than to disable it. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 02/08/2018 08:20, Steven D'Aprano wrote: On Wed, 01 Aug 2018 22:14:54 +0300, Serhiy Storchaka wrote: ... Not always. If your code supported Python 2 in the past, or third-party dependencies supports or supported Python 2, this warning can expose a real bug. Even if all your and third-party code always was Python 3 only, the standard library can contain such kind of bugs. Several years after the EOL of Python 2.7 and moving all living code to Python 3 we can ignore bytes warnings as always false positive. Even then, I don't know that we should do that. I do not believe that the EOL of Python 2 will end all confusion between byte strings and text strings. There is ample opportunity for code to accidentally compare bytes and text even in pure Python 3 code, e.g. comparing data read from files reading from files which are supposed to be opened in the same binary/text mode but aren't. I think I agree; when python 2 is history I can drop all the messing about with bytes in the input and clean up the code a lot. Stuff like svg and xml will still need conversions which are generally unknown in advance. The output will need converting to bytes, but that's fairly standard. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Dealing with errors in interactive subprocess running python interpreter that freeze the process
On Thu, Aug 2, 2018 at 6:46 PM, Paul Moore wrote:
> On Wed, 1 Aug 2018 at 21:17, wrote:
>>
>> I can run python3 interactively in a subprocess w/ Popen but
>> if I sent it text, that throws an exception, the process freezes
>> instead of just printing the exception like the normal interpreter..
>> why? how fix? Here is my code below.
>>
>> (I suspect when there is an exception, there is NO output to stdin so that
>> the problem is the line below that tries to read from stdin never finishes.
>> Maybe I need a different readline that can "survive" when there is no output
>> and won't block?)
>>
>>
>>
>> import subprocess
>>
>> interpreter = subprocess.Popen(['python3', '-i'],
>>stdin = subprocess.PIPE,
>>stdout = subprocess.PIPE,
>>stderr = subprocess.PIPE)
>>
>> while True:
>> exp = input(">>> ").encode() + b"\n"
>> interpreter.stdin.write(exp)
>> interpreter.stdin.flush()
>> print(interpreter.stdout.readline().strip())
>> interpreter.stdin.close()
>> interpreter.terminate()
>
> You're only reading one line from stdout, but an exception is multiple
> lines. So the subprocess is still trying to write while you're wanting
> to give it input again. This is a classic way to get a deadlock, which
> is basically what you're seeing. Add to that the fact that there are
> likely IO buffers in the subprocess that mean it's not necessarily
> passing output back to you at the exact time you expect it to (and the
> subprocess probably has different buffering behaviour when the IO is
> to pipes rather than to the console) and it gets complex fast.
>
> As others have mentioned, separate threads for the individual pipes
> may help, or if you need to go that far there are specialised
> libraries, I believe (pexpect is one, but from what I know it's fairly
> Unix-specific, so I'm not very familiar with it).
Another possibility: If the ONLY thing you're doing with stdout/stderr
is passing them through to the screen, simply don't change them. Let
them remain bound to the console. You can have a pipe for stdin
without also having pipes for the others. But that won't work if you
intend to do any sort of parsing on the returned output.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
image in db to string to actual image
storing images in db is easy but to retrieve them natively how to? (ignore db type, using orm). meaning without PIL / Pillow. type : png like i'd like to have it as string, any idea ? Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius -- https://mail.python.org/mailman/listinfo/python-list
[RELEASED] Python 3.4.9 and Python 3.5.6 are now available
On behalf of the Python development community, I'm happy to announce the availability of Python 3.4.9 and Python 3.5.6. Both Python 3.4 and 3.5 are in "security fixes only" mode. Both versions only accept security fixes, not conventional bug fixes, and both releases are source-only. You can find Python 3.4.9 here: https://www.python.org/downloads/release/python-349/ And you can find Python 3.5.6 here: https://www.python.org/downloads/release/python-356/ We now return you to your pitched debate already in progress, //arry/ -- https://mail.python.org/mailman/listinfo/python-list
Problems with wxPython _core_.pyd on windows98
I have a laptop with windows 98 I use to connect to the OBD2 port on my car. I'm trying to install pyobd. I have a build for Python 2.7 for Windows98 that works but I'm having trouble with running wxPython. I get the following error. C:\pyobd>python pyobd.py Traceback (most recent call last): File "pyobd.py", line 28, in import wx File "C:\Python27\lib\site-packages\wx-3.0-msw\wx\__init__.py", line 45, in from wx_core import * File "C:\Python27\lib\site-packages\wx-3.0-msw\wx\_core.py", line4, in import _core_ Import Error: DLL load failed: A device attached to the system is not functioning. So what is going on? Do these pyd files need to be compiled specifically for win98? If so how do I do that? Does python compile these dlls or do I need a C compiler? Is there an old version of wxPython that will work on windows 98? Thanks -- https://mail.python.org/mailman/listinfo/python-list
Re: image in db to string to actual image
ah those can be used, thanks, and to write the actual image? On Thu, Aug 2, 2018 at 8:20 PM Gilmeh Serda wrote: > On Thu, 02 Aug 2018 17:24:29 +0400, Abdur-Rahmaan Janhangeer wrote: > > > like i'd like to have it as string, any idea ? > > UUEncode / Base64 / yEnc ...? > > -- > Gilmeh > -- > https://mail.python.org/mailman/listinfo/python-list > -- Abdur-Rahmaan Janhangeer https://github.com/abdur-rahmaanj Mauritius -- https://mail.python.org/mailman/listinfo/python-list
beware of linked in - mail used on this list
just an info if you are using the mail you use in this list for linked in you might get surprises apologies if you got a mail from linkedin somewhere Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius -- https://mail.python.org/mailman/listinfo/python-list
Re: Problems with wxPython _core_.pyd on windows98
On Thursday, August 2, 2018 at 12:03:01 PM UTC-4, Wanderer wrote: > I have a laptop with windows 98 I use to connect to the OBD2 port on my car. > I'm trying to install pyobd. I have a build for Python 2.7 for Windows98 that > works but I'm having trouble with running wxPython. I get the following error. > > C:\pyobd>python pyobd.py > Traceback (most recent call last): > File "pyobd.py", line 28, in > import wx > File "C:\Python27\lib\site-packages\wx-3.0-msw\wx\__init__.py", line 45, in > > from wx_core import * > File "C:\Python27\lib\site-packages\wx-3.0-msw\wx\_core.py", line4, in > > import _core_ > Import Error: DLL load failed: A device attached to the system is not > functioning. > > > So what is going on? > > Do these pyd files need to be compiled specifically for win98? If so how do I > do that? Does python compile these dlls or do I need a C compiler? > > Is there an old version of wxPython that will work on windows 98? > > Thanks I got it working with Python 2.5.1 pywin32-207 pyserial 2.4 wxPython2.7 Though I'm still getting a warning API version mismatch for module win32event. This Python has API version 1013, module win32event has version 1012 I don't know what it means. pyobd.py has bug in it line 354 if "OS" in os.environ.keys() : running windows "OS" is not in os.environ.keys() in windows 98, so I changed "OS" to "WINDIR" which is in os environ.keys in windows 98. -- https://mail.python.org/mailman/listinfo/python-list
Re: Dealing with errors in interactive subprocess running python interpreter that freeze the process
> As others have mentioned, separate threads for the individual pipes > may help, or if you need to go that far there are specialised > libraries, I believe (pexpect is one, but from what I know it's fairly > Unix-specific, so I'm not very familiar with it). I'm on Linux so pexpect is a possibility. > Sorry, but there's no "simple" answer here for you (although you may > well be able to get something that works well enough for your specific > needs - but it's not obvious from your snippet of code what you're > trying to achieve). To send and receive text from a subprocess..even when there are exceptions. -- https://mail.python.org/mailman/listinfo/python-list
Re: Dealing with errors in interactive subprocess running python interpreter that freeze the process
> -I think the Python interpreter actually sends its output to stderr, so to > capture it you'd probably want it to go to the same place as stdout, so use > stderr = subprocess.STDOUT Yes that captured the error messages! Thanks! > -You're only reading 1 line out output for each thing, so if 1 command > creates multiple lines of output then you won't be showing them all. Yes. That is the one remaining problem. I tried replacing the readline with read and readlines and both froze/blocked. cs -- https://mail.python.org/mailman/listinfo/python-list
Re: Dealing with errors in interactive subprocess running python interpreter that freeze the process
> subprocess is not meant for interaction through the pipes. That is why, > I have been told, IDLE uses a socket for interaction. Multiprocess is > apparently better suited for interaction without resorting to a socket. So use normal socket on localhost for this? Don't you still need subprocess to launch the interpreter in a separate process? cs -- https://mail.python.org/mailman/listinfo/python-list
Re: Dealing with errors in interactive subprocess running python interpreter that freeze the process
> Another possibility: If the ONLY thing you're doing with stdout/stderr > is passing them through to the screen, simply don't change them. Let > them remain bound to the console. You can have a pipe for stdin > without also having pipes for the others. But that won't work if you > intend to do any sort of parsing on the returned output. I must parse. -- https://mail.python.org/mailman/listinfo/python-list
Re: Dealing with errors in interactive subprocess running python interpreter that freeze the process
On Thu, 2 Aug 2018 at 20:58, wrote: > > Sorry, but there's no "simple" answer here for you (although you may > > well be able to get something that works well enough for your specific > > needs - but it's not obvious from your snippet of code what you're > > trying to achieve). > > To send and receive text from a subprocess..even when there are exceptions. For an arbitrary program as the subprocess? You need some sort of threading in your process, or maybe some sort of async processing with non-blocking reads/writes, as you cannot assume any particular interleaving of input and output on the part of the child. If you control the child process, you can implement a custom protocol to do the communication, and avoid many of the problems that way. If you're trying to "wrap" something like the Python interpreter, it's certainly possible, but it's tricky to get all the edge cases right (I know, I've tried!) and you probably need to run it with the -u flag (or with PYTHONUNBUFFERED set) to avoid IO buffers making your life miserable ;-) Or, as you say you're on Unix, you may be able to use ptys to do this - I gather that's what they are designed for, but as a Windows programmer myself, I know very little about them. Paul -- https://mail.python.org/mailman/listinfo/python-list
Re: beware of linked in - mail used on this list
On Thu, 02 Aug 2018 22:35:10 +0400, Abdur-Rahmaan Janhangeer wrote: > just an info if you are using the mail you use in this list for linked > in you might get surprises > > apologies if you got a mail from linkedin somewhere LinkedIn is a spammer. I frequently get friend requests from people who I don't know from LinkedIn, and most of them don't even know they sent them. I got three from you yesterday. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: beware of linked in - mail used on this list
was supposed to be one i'm aware of but 3 in 1 day ? Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius -- https://mail.python.org/mailman/listinfo/python-list
Re: Dealing with errors in interactive subprocess running python interpreter that freeze the process
On 8/2/2018 3:52 PM, cseber...@gmail.com wrote: subprocess is not meant for interaction through the pipes. That is why, I have been told, IDLE uses a socket for interaction. Multiprocess is apparently better suited for interaction without resorting to a socket. So use normal socket on localhost for this? Don't you still need subprocess to launch the interpreter in a separate process? Yes. And you need to replace sys.stdxxx with file-like objects that redirect streams from and to the socket. And it is not 100% reliable, just 99.99??% reliableC -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: asyncio: Warning message when waiting for an Event set by AbstractLoop.add_reader
Léo El Amri via Python-list writes: > ... > WARNING:asyncio:Executing took 1.000 seconds > ... > But there is still this warning... At your place, I would look at the code responsible for the warning. I assume that it is produced because the waiting time is rather high -- but this is just a guess. -- https://mail.python.org/mailman/listinfo/python-list
