Re: [Python-Dev] cpython (3.3): don't run frame if it has no stack (closes #17669)

2013-04-11 Thread Nick Coghlan
On 11 Apr 2013 07:49, "Antoine Pitrou"  wrote:
>
> On Wed, 10 Apr 2013 23:01:46 +0200 (CEST)
> benjamin.peterson  wrote:
> > http://hg.python.org/cpython/rev/35cb75b9d653
> > changeset:   83238:35cb75b9d653
> > branch:  3.3
> > parent:  83235:172f825d7fc9
> > user:Benjamin Peterson 
> > date:Wed Apr 10 17:00:56 2013 -0400
> > summary:
> >   don't run frame if it has no stack (closes #17669)
>
> Wouldn't it be better with a test?

Benjamin said much the same thing on the issue, but persuading the
interpreter to create a frame without a stack that then gets exposed to
this code path isn't straightforward :P

Cheers,
Nick.

>
> Regards
>
> Antoine.
>
>
>
> >
> > files:
> >   Misc/NEWS   |  2 ++
> >   Objects/genobject.c |  2 +-
> >   2 files changed, 3 insertions(+), 1 deletions(-)
> >
> >
> > diff --git a/Misc/NEWS b/Misc/NEWS
> > --- a/Misc/NEWS
> > +++ b/Misc/NEWS
> > @@ -12,6 +12,8 @@
> >  Core and Builtins
> >  -
> >
> > +- Issue #17669: Fix crash involving finalization of generators using
yield from.
> > +
> >  - Issue #17619: Make input() check for Ctrl-C correctly on Windows.
> >
> >  - Issue #17610: Don't rely on non-standard behavior of the C qsort()
function.
> > diff --git a/Objects/genobject.c b/Objects/genobject.c
> > --- a/Objects/genobject.c
> > +++ b/Objects/genobject.c
> > @@ -178,7 +178,7 @@
> >  PyObject *yf = NULL;
> >  PyFrameObject *f = gen->gi_frame;
> >
> > -if (f) {
> > +if (f && f->f_stacktop) {
> >  PyObject *bytecode = f->f_code->co_code;
> >  unsigned char *code = (unsigned char
*)PyBytes_AS_STRING(bytecode);
> >
> >
>
>
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] cpython (3.3): don't run frame if it has no stack (closes #17669)

2013-04-11 Thread Maciej Fijalkowski
On Thu, Apr 11, 2013 at 12:28 PM, Nick Coghlan  wrote:
>
> On 11 Apr 2013 07:49, "Antoine Pitrou"  wrote:
>>
>> On Wed, 10 Apr 2013 23:01:46 +0200 (CEST)
>> benjamin.peterson  wrote:
>> > http://hg.python.org/cpython/rev/35cb75b9d653
>> > changeset:   83238:35cb75b9d653
>> > branch:  3.3
>> > parent:  83235:172f825d7fc9
>> > user:Benjamin Peterson 
>> > date:Wed Apr 10 17:00:56 2013 -0400
>> > summary:
>> >   don't run frame if it has no stack (closes #17669)
>>
>> Wouldn't it be better with a test?
>
> Benjamin said much the same thing on the issue, but persuading the
> interpreter to create a frame without a stack that then gets exposed to this
> code path isn't straightforward :P
>
> Cheers,
> Nick.

Maybe it's worth understanding in which circumstances this holds?
Please write a test, we'll run into similar issue at some point Im'm
sure.

Cheers,
fijal
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] cpython (3.3): don't run frame if it has no stack (closes #17669)

2013-04-11 Thread Benjamin Peterson
2013/4/11 Maciej Fijalkowski :
> On Thu, Apr 11, 2013 at 12:28 PM, Nick Coghlan  wrote:
>>
>> On 11 Apr 2013 07:49, "Antoine Pitrou"  wrote:
>>>
>>> On Wed, 10 Apr 2013 23:01:46 +0200 (CEST)
>>> benjamin.peterson  wrote:
>>> > http://hg.python.org/cpython/rev/35cb75b9d653
>>> > changeset:   83238:35cb75b9d653
>>> > branch:  3.3
>>> > parent:  83235:172f825d7fc9
>>> > user:Benjamin Peterson 
>>> > date:Wed Apr 10 17:00:56 2013 -0400
>>> > summary:
>>> >   don't run frame if it has no stack (closes #17669)
>>>
>>> Wouldn't it be better with a test?
>>
>> Benjamin said much the same thing on the issue, but persuading the
>> interpreter to create a frame without a stack that then gets exposed to this
>> code path isn't straightforward :P
>>
>> Cheers,
>> Nick.
>
> Maybe it's worth understanding in which circumstances this holds?
> Please write a test, we'll run into similar issue at some point Im'm
> sure.

Probably not. It's related to cyclic GC.



--
Regards,
Benjamin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] cpython: Add fast-path in PyUnicode_DecodeCharmap() for pure 8 bit encodings:

2013-04-11 Thread Serhiy Storchaka

On 09.04.13 23:29, victor.stinner wrote:

http://hg.python.org/cpython/rev/53879d380313
changeset:   83216:53879d380313
parent:  83214:b7f2d28260b4
user:Victor Stinner 
date:Tue Apr 09 21:53:09 2013 +0200
summary:
   Add fast-path in PyUnicode_DecodeCharmap() for pure 8 bit encodings:
cp037, cp500 and iso8859_1 codecs


I deliberately specialized only most typical case in order to reduce 
maintaining cost. Further optimization of two not the most popular 
encodings probably not worth additional 25 lines of code.



___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] cpython: Add fast-path in PyUnicode_DecodeCharmap() for pure 8 bit encodings:

2013-04-11 Thread Victor Stinner
2013/4/11 Serhiy Storchaka :
> On 09.04.13 23:29, victor.stinner wrote:
>>
>> http://hg.python.org/cpython/rev/53879d380313
>> changeset:   83216:53879d380313
>> parent:  83214:b7f2d28260b4
>> user:Victor Stinner 
>> date:Tue Apr 09 21:53:09 2013 +0200
>> summary:
>>Add fast-path in PyUnicode_DecodeCharmap() for pure 8 bit encodings:
>> cp037, cp500 and iso8859_1 codecs
>
> I deliberately specialized only most typical case in order to reduce
> maintaining cost. Further optimization of two not the most popular encodings
> probably not worth additional 25 lines of code.

I did the commit while I was trying to avoid usage of
PyUnicode_READ_CHAR() and PyUnicode_READ() in unicodeobject.c (slow
macros). I was surprised that PyUnicode_DecodeCharmap() has a fast
path for Py_UCS2 mapping but not Py_UCS1 mapping.

After implementing the fast-path, I realized that only a very few codecs use it.

So what do you suggest? Revert the commit to restore the following
"hack" (to only have one fast-path)?

'\ufffe'   ## Widen to UCS2 for optimization

The Py_UCS1 fast-path has a small advantage: decoding cannot fail (no
need to call an error handler).

Victor
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] casefolding in pathlib (PEP 428)

2013-04-11 Thread Guido van Rossum
Hey Antoine,

Some of my Dropbox colleagues just drew my attention to the occurrence
of case folding in pathlib.py. Basically, case folding as an approach
to comparing pathnames is fatally flawed. The issues include:

- most OSes these days allow the mounting of both case-sensitive and
case-insensitive filesystems simultaneously

- the case-folding algorithm on some filesystems is burned into the
disk when the disk is formatted

- case folding requires domain knowledge, e.g. turkish dotless I

- normalization is a mess, even on OSX, where it's better defined than elsewhere

One or more of them may reply-all to this message with more details.

--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] casefolding in pathlib (PEP 428)

2013-04-11 Thread Oleg Broytman
On Thu, Apr 11, 2013 at 02:11:21PM -0700, Guido van Rossum  
wrote:
> - the case-folding algorithm on some filesystems is burned into the
> disk when the disk is formatted

   Into the partition, I guess, not the physical disc?

Oleg.
-- 
 Oleg Broytmanhttp://phdru.name/[email protected]
   Programmers don't die, they just GOSUB without RETURN.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] casefolding in pathlib (PEP 428)

2013-04-11 Thread Antoine Pitrou
On Thu, 11 Apr 2013 14:11:21 -0700
Guido van Rossum  wrote:
> Hey Antoine,
> 
> Some of my Dropbox colleagues just drew my attention to the occurrence
> of case folding in pathlib.py. Basically, case folding as an approach
> to comparing pathnames is fatally flawed. The issues include:
> 
> - most OSes these days allow the mounting of both case-sensitive and
> case-insensitive filesystems simultaneously
> 
> - the case-folding algorithm on some filesystems is burned into the
> disk when the disk is formatted

The problem is that:
- if you always make the comparison case-sensitive, you'll get false
  negatives
- if you make the comparison case-insensitive under Windows, you'll get
  false positives

My assumption was that, globally, the number of false positives in case
(2) is much less than the number of false negatives in case (1).

On the other hand, one could argue that all comparisons should be
case-sensitive *and* the proper way to test for "identical" paths is to
access the filesystem. Which makes me think, perhaps concrete paths
should get a "samefile" method as in os.path.samefile().

Hmm, I think I'm tending towards the latter right now.

Regards

Antoine.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] casefolding in pathlib (PEP 428)

2013-04-11 Thread Robert Collins
On 12 April 2013 09:18, Oleg Broytman  wrote:
> On Thu, Apr 11, 2013 at 02:11:21PM -0700, Guido van Rossum  
> wrote:
>> - the case-folding algorithm on some filesystems is burned into the
>> disk when the disk is formatted
>
>Into the partition, I guess, not the physical disc?

CDROMs - Joliet IIRC - so yes, physical disc.

-Rob
-- 
Robert Collins 
Distinguished Technologist
HP Cloud Services
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] casefolding in pathlib (PEP 428)

2013-04-11 Thread Oleg Broytman
On Fri, Apr 12, 2013 at 09:29:44AM +1200, Robert Collins 
 wrote:
> On 12 April 2013 09:18, Oleg Broytman  wrote:
> > On Thu, Apr 11, 2013 at 02:11:21PM -0700, Guido van Rossum 
> >  wrote:
> >> - the case-folding algorithm on some filesystems is burned into the
> >> disk when the disk is formatted
> >
> >Into the partition, I guess, not the physical disc?
> 
> CDROMs - Joliet IIRC - so yes, physical disc.

   Ah, I've completely forgotten about that one. I was thinking in terms
of filesystems. Thank you for reminding!

Oleg.
-- 
 Oleg Broytmanhttp://phdru.name/[email protected]
   Programmers don't die, they just GOSUB without RETURN.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] casefolding in pathlib (PEP 428)

2013-04-11 Thread Guido van Rossum
On Thu, Apr 11, 2013 at 2:27 PM, Antoine Pitrou  wrote:
> On Thu, 11 Apr 2013 14:11:21 -0700
> Guido van Rossum  wrote:
>> Hey Antoine,
>>
>> Some of my Dropbox colleagues just drew my attention to the occurrence
>> of case folding in pathlib.py. Basically, case folding as an approach
>> to comparing pathnames is fatally flawed. The issues include:
>>
>> - most OSes these days allow the mounting of both case-sensitive and
>> case-insensitive filesystems simultaneously
>>
>> - the case-folding algorithm on some filesystems is burned into the
>> disk when the disk is formatted
>
> The problem is that:
> - if you always make the comparison case-sensitive, you'll get false
>   negatives
> - if you make the comparison case-insensitive under Windows, you'll get
>   false positives
>
> My assumption was that, globally, the number of false positives in case
> (2) is much less than the number of false negatives in case (1).
>
> On the other hand, one could argue that all comparisons should be
> case-sensitive *and* the proper way to test for "identical" paths is to
> access the filesystem. Which makes me think, perhaps concrete paths
> should get a "samefile" method as in os.path.samefile().
>
> Hmm, I think I'm tending towards the latter right now.

Python on OSX has been using (1) for a decade now without major problems.

Perhaps it would be best if the code never called lower() or upper()
(not even indirectly via os.path.normcase()). Then any case-folding
and path-normalization bugs are the responsibility of the application,
and we won't have to worry about how to fix the stdlib without
breaking backwards compatibility if we ever figure out how to fix this
(which I somehow doubt we ever will anyway :-).

Some other issues to be mindful of:

- On Linux, paths are really bytes; on Windows (at least NTFS), they
are really (16-bit) Unicode; on Mac, they are UTF-8 in a specific
normal form (except on some external filesystems).

- On Windows, short names are still supported, making the number of
ways to spell the path for any given file even larger.

--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] casefolding in pathlib (PEP 428)

2013-04-11 Thread Cameron Simpson
On 11Apr2013 14:11, Guido van Rossum  wrote:
| Some of my Dropbox colleagues just drew my attention to the occurrence
| of case folding in pathlib.py. Basically, case folding as an approach
| to comparing pathnames is fatally flawed. The issues include:
| 
| - most OSes these days allow the mounting of both case-sensitive and
| case-insensitive filesystems simultaneously
| 
| - the case-folding algorithm on some filesystems is burned into the
| disk when the disk is formatted
| 
| - case folding requires domain knowledge, e.g. turkish dotless I
| 
| - normalization is a mess, even on OSX, where it's better defined than 
elsewhere

Yes, but what's the use case? Specificly, _why_ are you comparing pathnames?

To my mind case folding is just one mode of filename conflict.
Surely there are others (forbidden characters in some domains, like
colons; names significant only to a certain number of characters;
an so forth).

Thus: what specific problem are you case-folding to address?

On a personal basis I would normally address this kind of thing
with stat(), avoiding any app knowledge about pathname rules: does
this path exist, or are these paths referencing the same file? But
of course that doesn't solve the wider issue with Dropbox, where
the rules differ per platform and where work can take place disparately
on separate hosts.

Imagining Dropbox, I'd guess there's a file tree in the backing store.
What is its policy? Does it allow multiple files differing only by case?
I can imagine that would be bad when the tree is presented on a case
insensitive platform (eg Windows, default MacOSX).

Taking the view that DropBox should avoid that situation (in what
are doubtless several forms), does Dropbox pre-emptively prevent
making files with specific names based on what is already in the
store, or resolve them to the same object (hard link locally, or
simply and less confusingly and more portably, diverting opens to
the existing name like a CI filesystem would)?

What about offline? That suggests that the forbidden modes should
known to the Dropbox app too. Is this the use case for comparing
filenames instead of just doing a stat() to the local filesystem
or to the remote backing store (via a virtual stat, as it were)?

What does Dropbox do if the local app is disabled and a user runs
riot in the Dropbox directory, making conflicting names: allowed
by the local FS but conflicting in the backing store or on other
hosts?

What does Dropbox do if a user makes conflicting files independently
on different hosts, and then syncs?

I just feel you've got a name conflist issue to resolve (and how
that's done is partly just policy), and pathlib which offers some
facilities related to that kind of thing. But a mismatch between
what you actually need to do and what pathlib offers.

Fixing your problem isn't necessarily a bugfix for pathlib.

I think we need to know the wider context.

Cheers,
-- 
Cameron Simpson 

I had a *bad* day. I had to subvert my principles and kowtow to an idiot.
Television makes these daily sacrifices possible. It deadens the inner core
of my being.- Martin Donovan, _Trust_
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] casefolding in pathlib (PEP 428)

2013-04-11 Thread Guido van Rossum
On Thu, Apr 11, 2013 at 4:09 PM, Cameron Simpson  wrote:
> On 11Apr2013 14:11, Guido van Rossum  wrote:
> | Some of my Dropbox colleagues just drew my attention to the occurrence
> | of case folding in pathlib.py. Basically, case folding as an approach
> | to comparing pathnames is fatally flawed. The issues include:
> |
> | - most OSes these days allow the mounting of both case-sensitive and
> | case-insensitive filesystems simultaneously
> |
> | - the case-folding algorithm on some filesystems is burned into the
> | disk when the disk is formatted
> |
> | - case folding requires domain knowledge, e.g. turkish dotless I
> |
> | - normalization is a mess, even on OSX, where it's better defined than 
> elsewhere
>
> Yes, but what's the use case? Specificly, _why_ are you comparing pathnames?

Um, this isn't about Dropbox. This is a warning against the inclusion
of any behavior depending on case folding in pathlib, based on
experience with case folding at Dropbox (both in the client and in the
server).

> To my mind case folding is just one mode of filename conflict.
> Surely there are others (forbidden characters in some domains, like
> colons; names significant only to a certain number of characters;
> an so forth).

Of course.

> Thus: what specific problem are you case-folding to address?

Why Dropbox is folding case really doesn't matter. But we have to deal
with it because users expect unreasonable things, such as having two
files, "readme" and "README", on a Linux box, and then syncing both
files to a box running Windows or OSX. (There are many other edge
cases, most not involving Linux at all.) We can't always os os.stat()
because some of this logic runs on a box where the files don't exist
(e.g. the server, or the Linux box in the above example).

> On a personal basis I would normally address this kind of thing
> with stat(), avoiding any app knowledge about pathname rules: does
> this path exist, or are these paths referencing the same file? But
> of course that doesn't solve the wider issue with Dropbox, where
> the rules differ per platform and where work can take place disparately
> on separate hosts.

You seem to be completely misunderstanding me. I am not asking for
help solving our problem. I am giving advice to avoid baking the wrong
solution to this class of problems into a new stdlib module.

> Imagining Dropbox, I'd guess there's a file tree in the backing store.
> What is its policy? Does it allow multiple files differing only by case?
> I can imagine that would be bad when the tree is presented on a case
> insensitive platform (eg Windows, default MacOSX).

You got the basic idea, but we can't just refuse to sync files that
might be problematic on some other box. Suppose someone is using
Dropbox just as a backup service for their Linux box. They shouldn't
have to worry about case clashes not being backed up.

> Taking the view that DropBox should avoid that situation (in what
> are doubtless several forms), does Dropbox pre-emptively prevent
> making files with specific names based on what is already in the
> store, or resolve them to the same object (hard link locally, or
> simply and less confusingly and more portably, diverting opens to
> the existing name like a CI filesystem would)?

We have lots of different solutions based on the specific situations.

> What about offline? That suggests that the forbidden modes should
> known to the Dropbox app too. Is this the use case for comparing
> filenames instead of just doing a stat() to the local filesystem
> or to the remote backing store (via a virtual stat, as it were)?

Again, please don't try to solve our problem for us.

> What does Dropbox do if the local app is disabled and a user runs
> riot in the Dropbox directory, making conflicting names: allowed
> by the local FS but conflicting in the backing store or on other
> hosts?
>
> What does Dropbox do if a user makes conflicting files independently
> on different hosts, and then syncs?
>
> I just feel you've got a name conflist issue to resolve (and how
> that's done is partly just policy), and pathlib which offers some
> facilities related to that kind of thing. But a mismatch between
> what you actually need to do and what pathlib offers.
>
> Fixing your problem isn't necessarily a bugfix for pathlib.
>
> I think we need to know the wider context.

My reasoning is as follows. If pathlib supports functionality for
checking whether two paths spelled differently point to the same file,
users are going to rely on that functionality. But if the
implementation is based on knowing how and when to case fold, it will
definitely have bugs. So I am proposing to either remove that
functionality, or to implement it by consulting the filesystem. Which
of course has its own set of issues, for example if the file doesn't
exist yet, but there are ways to deal with that too.

--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-

Re: [Python-Dev] casefolding in pathlib (PEP 428)

2013-04-11 Thread Cameron Simpson
On 11Apr2013 16:23, Guido van Rossum  wrote:
| On Thu, Apr 11, 2013 at 4:09 PM, Cameron Simpson  wrote:
| > On 11Apr2013 14:11, Guido van Rossum  wrote:
| > | Some of my Dropbox colleagues just drew my attention to the occurrence
| > | of case folding in pathlib.py. Basically, case folding as an approach
| > | to comparing pathnames is fatally flawed. [...]
| >
| > Yes, but what's the use case? Specificly, _why_ are you comparing pathnames?
| 
| Um, this isn't about Dropbox. This is a warning against the inclusion
| of any behavior depending on case folding in pathlib, based on
| experience with case folding at Dropbox (both in the client and in the
| server).

Ah. That wasn't so apparent to me. I took you to have tripped over
a specific problem that pathlib appeared to be missolving.

I've always viewed path normalisation and its ilk as hazard prone
and very context dependent, so I tend not to do it if I can help
it.

| You seem to be completely misunderstanding me. I am not asking for
| help solving our problem. I am giving advice to avoid baking the wrong
| solution to this class of problems into a new stdlib module.

Ok, fine.

[...snip lots of stuff now not relevant...]
| My reasoning is as follows. If pathlib supports functionality for
| checking whether two paths spelled differently point to the same file,
| users are going to rely on that functionality. But if the
| implementation is based on knowing how and when to case fold, it will
| definitely have bugs. So I am proposing to either remove that
| functionality, or to implement it by consulting the filesystem. Which
| of course has its own set of issues, for example if the file doesn't
| exist yet, but there are ways to deal with that too.

Personally I'd be for removing it, or making the doco quite blunt
about the many possible shortcomings of guessing whether two paths
are the same thing without being able to stat() them.

I'll back out now. Cheers,
-- 
Cameron Simpson 

Having been erased,
The document you're seeking
Must now be retyped.
- Haiku Error Messages 
http://www.salonmagazine.com/21st/chal/1998/02/10chal2.html
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Destructors and Closing of File Objects

2013-04-11 Thread Nikolaus Rath
[ Note: I already asked this on
http://stackoverflow.com/questions/15917502 but didn't get any
satisfactory answers]

Hello,

The description of tempfile.NamedTemporaryFile() says:

,
|  If delete is true (the default), the file is deleted as soon as it is
|  closed.
`

In some circumstances, this means that the file is not deleted after the
program ends. For example, when running the following test under
py.test, the temporary file remains:

,
| from __future__ import division, print_function, absolute_import
| import tempfile
| import unittest2 as unittest
| class cache_tests(unittest.TestCase):
| def setUp(self):
| self.dbfile = tempfile.NamedTemporaryFile()
| def test_get(self):
| self.assertEqual('foo', 'foo')
`

In some way this makes sense, because this program never explicitly
closes the file object. The only other way for the object to get closed
would presumably be in the __del__ destructor, but here the language
references states that "It is not guaranteed that __del__() methods are
called for objects that still exist when the interpreter exits." So
everything is consistent with the documentation so far.

However, I'm confused about the implications of this. If it is not
guaranteed that file objects are closed on interpreter exit, can it
possibly happen that some data that was successfully written to a
(buffered) file object is lost even though the program exits gracefully,
because it was still in the file objects buffer and the file object
never got closed?

Somehow that seems very unlikely and un-pythonic to me, and the open()
documentation doesn't contain any such warnings either. So I
(tentatively) conclude that file objects are, after all, guaranteed to
be closed.

But how does this magic happen, and why can't NamedTemporaryFile() use
the same magic to ensure that the file is deleted? 


Best,

   -Nikolaus

-- 
 »Time flies like an arrow, fruit flies like a Banana.«

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Destructors and Closing of File Objects

2013-04-11 Thread Brian Curtin
On Fri, Apr 12, 2013 at 12:04 AM, Nikolaus Rath  wrote:
> [ Note: I already asked this on
> http://stackoverflow.com/questions/15917502 but didn't get any
> satisfactory answers]

Sorry, but that's not a reason to repost your question to this list.
If you have to ask somewhere else, it would be python-list, aka,
comp.lang.python.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] casefolding in pathlib (PEP 428)

2013-04-11 Thread Dirkjan Ochtman
On Thu, Apr 11, 2013 at 11:27 PM, Antoine Pitrou  wrote:
> Hmm, I think I'm tending towards the latter right now.

You might also want to look at what Mercurial does. As a
cross-platform filesystem-oriented tool, it has some interesting
issues in the department of casefolding.

Cheers,

Dirkjan
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com