On Mon, Apr 28, 2014 at 12:16 AM, Matthew Pounsett
<matt.pouns...@gmail.com> wrote:
> On Friday, 25 April 2014 10:05:03 UTC-4, Chris Angelico wrote:
>> First culprit I'd look at is the mixing of subprocess and threading.
>> It's entirely possible that something goes messy when you fork from a
>> thread.
>
> I liked the theory, but I've run some tests and can't reproduce the error
> that way. I'm using all the elements in my test code that the real code
> runs, and I can't get the same error. Even when I deliberately break things
> I'm getting a proper exception with stack trace.
>
In most contexts, "thread unsafe" simply means that you can't use the
same facilities simultaneously from two threads (eg a lot of database
connection libraries are thread unsafe with regard to a single
connection, as they'll simply write to a pipe or socket and then read
a response from it). But processes and threads are, on many systems,
linked. Just the act of spinning off a new thread and then forking can
potentially cause problems. Those are the exact sorts of issues that
you'll see when you switch OSes, as it's the underlying thread/process
model that's significant. (Particularly of note is that Windows is
*very* different from Unix-based systems, in that subprocess
management is not done by forking. But not applicable here.)
You may want to have a look at subprocess32, which Ned pointed out. I
haven't checked, but I would guess that its API is identical to
subprocess's, so it should be a drop-in replacement ("import
subprocess32 as subprocess"). If that produces the exact same results,
then it's (probably) not thread-safety that's the problem.
>> Separately: You're attempting a very messy charset decode there. You
>> attempt to decode as UTF-8, errors ignored, and if that fails, you log
>> an error... and continue on with the original bytes. You're risking
>> shooting yourself in the foot there; I would recommend you have an
>> explicit fall-back (maybe re-decode as Latin-1??), so the next code is
>> guaranteed to be working with Unicode. Currently, it might get a
>> unicode or a str.
>
> Yeah, that was a logic error on my part that I hadn't got around to noticing,
> since I'd been concentrating on the stuff that was actively breaking. That
> should have been in an else: block on the end of the try.
>
Ah good. Keeping bytes versus text separate is something that becomes
particularly important in Python 3, so I always like to encourage
people to get them straight even in Py2. It'll save you some hassle
later on.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
