Re: Recursive method in class

[ast]

Le 27/09/2019 à 14:26, Jan van den Broek a écrit :

On 2019-09-27, ast  wrote:

Is it feasible to define a recursive method in a class ?
(I don't need it, it's just a trial)

Here are failing codes:


class Test:
  def fib(self, n):
  if n < 2: return n
  return fib(self, n-2) + fib(self, n-1)

   self.fib(...)

[Schnipp]



Yes you are right. I forgot that class methods
don't see class attributes

An other workaround:

 def fib(self, n):
 if n < 2: return n
 return Test.fib(self, n-2) + Test.fib(self, n-1)

It comes from https://www.pythonsheets.com/notes/python-object.html

>>> def fib(self, n):
... if n <= 2:
... return 1
... return fib(self, n-1) + fib(self, n-2)
...
>>> Fib = type('Fib', (object,), {'val': 10,
...   'fib': fib})
>>> f = Fib()
>>> f.val
10
>>> f.fib(f.val)
55

So it means that when classes are constructed explicitely
with type(name, bases, dict_), some methods are transformed
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[Barry Scott]

> On 30 Sep 2019, at 05:40, DL Neil via Python-list  
> wrote:
> 
> Should pathlib reflect changes it has made to the file-system?

I think it should not.

A Path() is the name of a file it is not the file itself. Why should it
track changes in the file system for the name?

Here is code to show why following the rename will break logic:

save_file = pathlib.Path('my-data.txt')

def save( data ):
# backup file
if save_file.exists():
save_file.rename('my-data.backup')

# save data
with save_file.open() as f:
f.write( data )

while True:
save( generate_data() )
time.sleep( interval )

If the rename of the file changed the path the above code will fail.

Barry


> 
> 
> Sample code, below, shows pathlib identifying a data-file and then renaming 
> it. Yet, after the rename operation, pathlib doesn't recognise its own 
> change; whereas the file system does/proves the change was actioned.
> 
> 
> $ touch before.file
> $ python3
> Python 3.7.4 (default, Jul  9 2019, 16:48:28)
> [GCC 8.3.1 20190223 (Red Hat 8.3.1-2)] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import pathlib
> >>> p = pathlib.Path( "before.file" )
> >>> p
> PosixPath('before.file')
> >>> p.name
> 'before.file'
> >>> p.rename( "after.file" )
> >>> p.name
> 'before.file'
> >>> exit()
> $ ls -lah after*
> -rw-rw-r--. 1 dn dn 0 Sep 30 16:05 after.file
> $ ls -lah before*
> ls: cannot access 'before*': No such file or directory
> 
> 
> I would expect pathlib to keep track of changes it has made. Any ideas/good 
> reasons why/why not?
> NB "name" is a r/o attribute.
> -- 
> Regards,
> =dn
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[Peter Otten]

DL Neil via Python-list wrote:

> Should pathlib reflect changes it has made to the file-system?
> 
> 
> Sample code, below, shows pathlib identifying a data-file and then
> renaming it. Yet, after the rename operation, pathlib doesn't recognise
> its own change; whereas the file system does/proves the change was
> actioned.
> 
> 
> $ touch before.file
> $ python3
> Python 3.7.4 (default, Jul  9 2019, 16:48:28)
> [GCC 8.3.1 20190223 (Red Hat 8.3.1-2)] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> import pathlib
>  >>> p = pathlib.Path( "before.file" )
>  >>> p
> PosixPath('before.file')
>  >>> p.name
> 'before.file'
>  >>> p.rename( "after.file" )
>  >>> p.name
> 'before.file'
>  >>> exit()
> $ ls -lah after*
> -rw-rw-r--. 1 dn dn 0 Sep 30 16:05 after.file
> $ ls -lah before*
> ls: cannot access 'before*': No such file or directory
> 
> 
> I would expect pathlib to keep track of changes it has made. Any
> ideas/good reasons why/why not?
> NB "name" is a r/o attribute.

This is certainly not a backwards-compatible change. Consider:

p = pathlib.Path("data.txt")
p.rename("data.txt.bak")
with p.open("w") as f: 
f.write("new stuff") # oops, we are overwriting the backup

Also, it will not necessarily be as obvious as in the above example.
Everywhere you write 

p = q

you produce an occasion to unexpectedly change a file reference. Generally, 
it's easier to reason about immutable objects.

What might be helpful would be to return a new Path object

renamed = original.rename(...)

but as I'm not a pathlib user I have no idea what the use-it to throw-it-
away ratio would be.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Synchronous and Asynchronous callbacks

[Barry Scott]

> On 29 Sep 2019, at 21:41, Eko palypse  wrote:
> 
> Am Sonntag, 29. September 2019 19:18:32 UTC+2 schrieb Barry Scott:
>>> On 29 Sep 2019, at 14:14, Eko palypse  wrote:
>>> 
>>> Unfortunately, I can't make all callbacks synchronous or asynchronous 
>>> because this has negative effects on the application in one way or another.
>> 
>> Surely you can make all callbacks async?
>> You do not have to have them wait, they can complete their work in one call.
>> 
>> sync == async-that-does-not-await
>> 
>> Barry
> 
> Thank you for your answer but I'm sorry I don't get it. 
> What do you mean by I don't have to have them wait?
> 
> Let me explain my understanding, with some pseudo code, how async callbacks
> works and please keep in mind that I'm a novice when it comes to OO designs
> and basic software design strategies/techniques, so I might be totally
> wrong about what I'm doing.

When you said async I thought you meant the specific programming model in Python
that uses "async def", "await" etc. You are using threads.

The problem you are facing is that the UI must not become unresponsive.
Further a background thread cannot call any UI framework functions as the UI
framework can only be called on the main thread.

You can run fast event handers on the UI thread but slow running handlers
must be run on another thread to allow the UI to be responsive.

What I do is have the event handler for a long running task take the 
responsibility
to queue the task onto a background thread. While the task is running it sends 
status
to the UI thread so that UI elements are updated (progress bars, etc).

If you are using PyQt5 you might find the code I used for SCM Workbench 
interesting.

I can write event handlers that look like this:

@thread_switcher
def eventHandler( self, ... ):
# starts in the foreground (UI thread)
self.status.update('Working...')

yield self.switchToBackground
# now on the background thread
self.doSlowTask()

yield self.switchToForeground
# now in the foreground
self.status.update('Done')

The code is in:

https://github.com/barry-scott/scm-workbench/blob/master/Source/Common/wb_background_thread.py

@thread_switcher adds an attribute to the function so the rest of the
code knows this is an event handler that needs to use a background thread.

When I add an event handler I call through a function (wrapWithThreadSwitcher)
that wraps only functions that have been marked with @thread_switcher.

The rest of the code is the machinery to handle moving between threads via the 
generator.

Barry

> 
> An SYNCHRONOUS callback is like this. Event/notification gets fired and
> notification handler calls all registered methods one after the other.
> Something like
> 
> if notification.list_of_methods:
>for method in list_of_methods:
>method()
> 
> whereas an ASYNCHRONOUS callback is more like this. Event/notification gets
> fired and notification handler informs another thread that the event has
> been fired and all registered async callback methods should get executed.
> So something like this
> 
> class EventThread(threading.Thread):
>def __init__(...):
>super()...
>self.event = threading.Event()
>self.kill = False
>...
>def run(self):
>while True:
>self.event.wait()
>self.event.clear()
>if not self.kill:
>for method in self.list_of_methods:
>method()
> 
> et = EventThread()
> if notification.list_of_methods:
>et.event.set()  # run all async methods
>for method in list_of_methods:  # run sync methods
>method())
> 
> 
> So if there is no synchronous callback for that notification then the 
> notification handler would just set the event and return. 
> The EventThread would then call one registered callback after the other.
> Right?
> 
> Using this approach does sound fine from UI point of view as there is
> minimal impact and UI keeps responsive but from code execution point of view
> it sound much more complicated. What if one async callback is slow or buggy?
> This could lead to unpredictable behavior where as in a synchronous execution 
> you immediately know what is causing it. 
> That was the reason I decided not to offer async callbacks until I 
> discovered that other parts, like UI modifications aren't working properly.
> And there are notification which must be called in a synchronous way, like
> the modification event. If you want to intercept a modification safely then 
> it can only be done within a synchronous callback. 
> 
> Does this makes sense to you? Or do I have wrong understanding or, again, an
> wrong assumption how this works?
> 
> Thank you
> Eren
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[Barry Scott]

> On 30 Sep 2019, at 09:55, Peter Otten <__pete...@web.de> wrote:
> 
> DL Neil via Python-list wrote:
> 
>> Should pathlib reflect changes it has made to the file-system?
>> 
>> 
>> Sample code, below, shows pathlib identifying a data-file and then
>> renaming it. Yet, after the rename operation, pathlib doesn't recognise
>> its own change; whereas the file system does/proves the change was
>> actioned.
>> 
>> 
>> $ touch before.file
>> $ python3
>> Python 3.7.4 (default, Jul  9 2019, 16:48:28)
>> [GCC 8.3.1 20190223 (Red Hat 8.3.1-2)] on linux
>> Type "help", "copyright", "credits" or "license" for more information.
> import pathlib
> p = pathlib.Path( "before.file" )
> p
>> PosixPath('before.file')
> p.name
>> 'before.file'
> p.rename( "after.file" )
> p.name
>> 'before.file'
> exit()
>> $ ls -lah after*
>> -rw-rw-r--. 1 dn dn 0 Sep 30 16:05 after.file
>> $ ls -lah before*
>> ls: cannot access 'before*': No such file or directory
>> 
>> 
>> I would expect pathlib to keep track of changes it has made. Any
>> ideas/good reasons why/why not?
>> NB "name" is a r/o attribute.
> 
> This is certainly not a backwards-compatible change. Consider:
> 
> p = pathlib.Path("data.txt")
> p.rename("data.txt.bak")
> with p.open("w") as f: 
>f.write("new stuff") # oops, we are overwriting the backup
> 
> Also, it will not necessarily be as obvious as in the above example.
> Everywhere you write 
> 
> p = q
> 
> you produce an occasion to unexpectedly change a file reference. Generally, 
> it's easier to reason about immutable objects.
> 
> What might be helpful would be to return a new Path object
> 
> renamed = original.rename(...)

I do not think this is useful as in my code I will already have a path object to
use as the rename target.

src = pathlib.Path('/source/data.txt')
dst = pathlib.Path('/backup/data.bak')

src.rename( dst )
print( 'Renamed %s to %s' % (src, dst) )

> 
> but as I'm not a pathlib user I have no idea what the use-it to throw-it-
> away ratio would be.

Barry

> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Synchronous and Asynchronous callbacks

[Eko palypse]

Am Montag, 30. September 2019 11:46:43 UTC+2 schrieb Barry Scott:
> > On 29 Sep 2019, at 21:41, Eko palypse  wrote:
> > 
> > Am Sonntag, 29. September 2019 19:18:32 UTC+2 schrieb Barry Scott:
> >>> On 29 Sep 2019, at 14:14, Eko palypse  wrote:
> >>> 
> >>> Unfortunately, I can't make all callbacks synchronous or asynchronous 
> >>> because this has negative effects on the application in one way or 
> >>> another.
> >> 
> >> Surely you can make all callbacks async?
> >> You do not have to have them wait, they can complete their work in one 
> >> call.
> >> 
> >> sync == async-that-does-not-await
> >> 
> >> Barry
> > 
> > Thank you for your answer but I'm sorry I don't get it. 
> > What do you mean by I don't have to have them wait?
> > 
> > Let me explain my understanding, with some pseudo code, how async callbacks
> > works and please keep in mind that I'm a novice when it comes to OO designs
> > and basic software design strategies/techniques, so I might be totally
> > wrong about what I'm doing.
> 
> When you said async I thought you meant the specific programming model in 
> Python
> that uses "async def", "await" etc. You are using threads.
> 
> The problem you are facing is that the UI must not become unresponsive.
> Further a background thread cannot call any UI framework functions as the UI
> framework can only be called on the main thread.
> 
> You can run fast event handers on the UI thread but slow running handlers
> must be run on another thread to allow the UI to be responsive.
> 
> What I do is have the event handler for a long running task take the 
> responsibility
> to queue the task onto a background thread. While the task is running it 
> sends status
> to the UI thread so that UI elements are updated (progress bars, etc).
> 
> If you are using PyQt5 you might find the code I used for SCM Workbench 
> interesting.
> 
> I can write event handlers that look like this:
> 
> @thread_switcher
> def eventHandler( self, ... ):
>   # starts in the foreground (UI thread)
>   self.status.update('Working...')
> 
>   yield self.switchToBackground
>   # now on the background thread
>   self.doSlowTask()
> 
>   yield self.switchToForeground
>   # now in the foreground
>   self.status.update('Done')
> 
> The code is in:
> 
> https://github.com/barry-scott/scm-workbench/blob/master/Source/Common/wb_background_thread.py
> 
> @thread_switcher adds an attribute to the function so the rest of the
> code knows this is an event handler that needs to use a background thread.
> 
> When I add an event handler I call through a function (wrapWithThreadSwitcher)
> that wraps only functions that have been marked with @thread_switcher.
> 
> The rest of the code is the machinery to handle moving between threads via 
> the generator.
> 
> Barry
> 
> > 
> > An SYNCHRONOUS callback is like this. Event/notification gets fired and
> > notification handler calls all registered methods one after the other.
> > Something like
> > 
> > if notification.list_of_methods:
> >for method in list_of_methods:
> >method()
> > 
> > whereas an ASYNCHRONOUS callback is more like this. Event/notification gets
> > fired and notification handler informs another thread that the event has
> > been fired and all registered async callback methods should get executed.
> > So something like this
> > 
> > class EventThread(threading.Thread):
> >def __init__(...):
> >super()...
> >self.event = threading.Event()
> >self.kill = False
> >...
> >def run(self):
> >while True:
> >self.event.wait()
> >self.event.clear()
> >if not self.kill:
> >for method in self.list_of_methods:
> >method()
> > 
> > et = EventThread()
> > if notification.list_of_methods:
> >et.event.set()  # run all async methods
> >for method in list_of_methods:  # run sync methods
> >method())
> > 
> > 
> > So if there is no synchronous callback for that notification then the 
> > notification handler would just set the event and return. 
> > The EventThread would then call one registered callback after the other.
> > Right?
> > 
> > Using this approach does sound fine from UI point of view as there is
> > minimal impact and UI keeps responsive but from code execution point of view
> > it sound much more complicated. What if one async callback is slow or buggy?
> > This could lead to unpredictable behavior where as in a synchronous 
> > execution you immediately know what is causing it. 
> > That was the reason I decided not to offer async callbacks until I 
> > discovered that other parts, like UI modifications aren't working properly.
> > And there are notification which must be called in a synchronous way, like
> > the modification event. If you want to intercept a modification safely then 
> > it can only be done within a synchronous callback. 
> > 
> > Does this makes sense to you? Or do I have wrong unders

Re: pathlib

[Dan Sommers]

On 9/30/19 4:28 AM, Barry Scott wrote:




On 30 Sep 2019, at 05:40, DL Neil via Python-list  
wrote:

Should pathlib reflect changes it has made to the file-system?


I think it should not.

A Path() is the name of a file it is not the file itself. Why should it
track changes in the file system for the name?


I would have said the same thing, but the docs⁰ disagree:  a
PurePath represents the name of (or the path to) a file, but a
Path represents the actual file.

That said, why doesn't your argument apply to read and write?  I
would certainly expect that writing to a path and then reading
from that same path would return the newly written data.  If I
squint funny, the Path object is tracking the operations on the
file system.

I think I'm actually arguing against some long since made (and
forgotten?) design decisions that can't be changed (dare I say
fixed?) because changing them would break backwards
compatibility.

Yuck.  :-)  And I can absolutely see all sorts of different
expecations not being met and having to be explained by saying
"well, that's the way it works."

⁰ https://docs.python.org/3/library/pathlib.html
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[Chris Angelico]

On Mon, Sep 30, 2019 at 9:54 PM Dan Sommers
<2qdxy4rzwzuui...@potatochowder.com> wrote:
> I would have said the same thing, but the docs⁰ disagree:  a
> PurePath represents the name of (or the path to) a file, but a
> Path represents the actual file.
>
>
> ⁰ https://docs.python.org/3/library/pathlib.html

I don't think it represents the actual file. If it did, equality would
be defined by samefile, NOT by the file name.

>>> from pathlib import Path
>>> import os
>>> open("file1", "w").close()
>>> os.link("file1", "file2")
>>> Path("file1") == Path("file2")
False
>>> Path("file1").samefile(Path("file2"))
True
>>> Path("file1") == Path("file1")
True

It still represents the path to the file, not the file itself, and if
you move something over it, it will see the new file.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Announcing config-path an OS independent configuration file paths

[Barry Scott]

See https://pypi.org/project/config-path/ for documentation.

Install using pip:

python -m pip install config-path

I write tools that run on macOS, Unix and Windows systems.
Each operating system has its own conventions on where config
files should be stored. Following the conventions is not always
straight forward.

After a question raised here on Python users about config file locations
I was inspired to write this module to help with that problem.

config_path library works out which path to use for configuration folders
and files in an operating system independent way.

Each operating system has particular conventions for where an application
is expected to stores it configuration. The information provided to ConfigPath
is used to figure out an appropriate file path or folder path for the 
application's
configuration data.

Supports Windows, macOS and most unix systems using the 'XDG Base Directory 
Specification'.

Example for the "widget" app from "example.com" that uses JSON for its config:

from config_path import ConfigPath
conf_path = ConfigPath( 'example.com', 'widget', '.json' )

path = conf_path.saveFilePath( mkdir=True )
with path.open() as f:
f.write( config_data )

And to read the config:

path = conf_path.readFilePath()
if path is not None:
# path exists and config can be read
with path.open() as f:
config = json.loads( f )
else:
config = default_config

Barry


-- 
https://mail.python.org/mailman/listinfo/python-list

Announcing colour-text and colour-print

[Barry Scott]

See https://pypi.org/project/colour-text/ for documentation
with colourful examples.

Install using pip:

python -m pip install colour-text


Many command line tools now colour there output to make it
easier to pick out important details.

I want to do the same for my commands that I tend to write in
Python, bash and Windows CMD. The solution needs to work
for Unix, macOS and Windows.

Embedding escape sequences I found to be too low level.
And the existing python libraries a bit too verbose.

My solution is to use a simple markup for adding
colour to text. The markup delimited defaults to "<>".
Names of colours include semantic names like, info, error
and em.

Here is an example:

from colour_text import ColourText
ct = ColourText()
ct.initTerminal()

print( ct( "The next section is in green: <>green example<>." ) )

I also want to be able to add colour output to shell scripts and
Windows CMD scripts. When pip installs colour-text it also installs
the colour-print command for use from scripts.

For example:

$ colour-print "<>info Info:<> this is an <>em informational<> message"
$ colour-print "<>error Error: This is an error message<>"

And you can also pass in arguments in a printf or python % style,
only %s is supported.

$ colour-print "<>info Info:<> Home folder is %s" "$HOME"

Barry

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Announcing colour-text and colour-print

[Peter Otten]

Barry Scott wrote:

> See https://pypi.org/project/colour-text/ for documentation
> with colourful examples.
> 
> Install using pip:
> 
> python -m pip install colour-text

Have you considered to spell that color-text? 

In programming the guys who can't spell won ;)


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[Barry Scott]

> On 30 Sep 2019, at 12:51, Dan Sommers <2qdxy4rzwzuui...@potatochowder.com> 
> wrote:
> 
> On 9/30/19 4:28 AM, Barry Scott wrote:
>>> On 30 Sep 2019, at 05:40, DL Neil via Python-list  
>>> wrote:
>>> Should pathlib reflect changes it has made to the file-system?
>> I think it should not.
>> A Path() is the name of a file it is not the file itself. Why should it
>> track changes in the file system for the name?
> 
> I would have said the same thing, but the docs⁰ disagree:  a
> PurePath represents the name of (or the path to) a file, but a
> Path represents the actual file.

I'm not seeing that wording in the python 3.7 pathlib documentation.
Can you quote the exact wording please?

I do see this:

"Pure path objects provide path-handling operations which don’t actually access 
a filesystem."

And:

"Concrete paths are subclasses of the pure path classes. In addition to 
operations provided
by the latter, they also provide methods to do system calls on path objects."

There is no requirement that a Path() names a file that exists even.

> That said, why doesn't your argument apply to read and write?  I
> would certainly expect that writing to a path and then reading
> from that same path would return the newly written data.  If I
> squint funny, the Path object is tracking the operations on the
> file system.

I do not expect that. Consider the time line:

1. with p.open('w') write data
2. external process changes file on disk
3. with p.open('r') read data

How would (3) get the data written at (1) guaranteed?
It will lead to bugs to assume that.

The path object is allowing system calls that need a file's path to be called,
that is all. Beyond that there is no relationship between the pathlib.Path()
objects and files.

> 
> I think I'm actually arguing against some long since made (and
> forgotten?) design decisions that can't be changed (dare I say
> fixed?) because changing them would break backwards
> compatibility.
> 
> Yuck.  :-)  And I can absolutely see all sorts of different
> expecations not being met and having to be explained by saying
> "well, that's the way it works."

I'd suggest that the design is reasonable and If there is misunderstanding that 
its
something that docs could address.

> 
> ⁰ https://docs.python.org/3/library/pathlib.html
> -- 
> https://mail.python.org/mailman/listinfo/python-list

Barry

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Announcing config-path an OS independent configuration file paths

[Paul Moore]

How does this compare to the existing appdirs module on PyPI?

On Mon, 30 Sep 2019 at 13:15, Barry Scott  wrote:
>
> See https://pypi.org/project/config-path/ for documentation.
>
> Install using pip:
>
> python -m pip install config-path
>
> I write tools that run on macOS, Unix and Windows systems.
> Each operating system has its own conventions on where config
> files should be stored. Following the conventions is not always
> straight forward.
>
> After a question raised here on Python users about config file locations
> I was inspired to write this module to help with that problem.
>
> config_path library works out which path to use for configuration folders
> and files in an operating system independent way.
>
> Each operating system has particular conventions for where an application
> is expected to stores it configuration. The information provided to ConfigPath
> is used to figure out an appropriate file path or folder path for the 
> application's
> configuration data.
>
> Supports Windows, macOS and most unix systems using the 'XDG Base Directory 
> Specification'.
>
> Example for the "widget" app from "example.com" that uses JSON for its config:
>
> from config_path import ConfigPath
> conf_path = ConfigPath( 'example.com', 'widget', '.json' )
>
> path = conf_path.saveFilePath( mkdir=True )
> with path.open() as f:
> f.write( config_data )
>
> And to read the config:
>
> path = conf_path.readFilePath()
> if path is not None:
> # path exists and config can be read
> with path.open() as f:
> config = json.loads( f )
> else:
> config = default_config
>
> Barry
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[Dan Sommers]

On 9/30/19 8:40 AM, Barry Scott wrote:
 >
 >
 >> On 30 Sep 2019, at 12:51, Dan Sommers
<2qdxy4rzwzuui...@potatochowder.com> wrote:
 >>
 >> On 9/30/19 4:28 AM, Barry Scott wrote:
  On 30 Sep 2019, at 05:40, DL Neil via Python-list
 wrote:
  Should pathlib reflect changes it has made to the file-system?
 >>> I think it should not.
 >>> A Path() is the name of a file it is not the file itself. Why 
should it

 >>> track changes in the file system for the name?
 >>
 >> I would have said the same thing, but the docs⁰ disagree:  a
 >> PurePath represents the name of (or the path to) a file, but a
 >> Path represents the actual file.
 >
 > I'm not seeing that wording in the python 3.7 pathlib documentation.
 > Can you quote the exact wording please?
 >
 > I do see this:
 >
 > "Pure path objects provide path-handling operations which don’t
actually access a filesystem."
 >
 > And:
 >
 > "Concrete paths are subclasses of the pure path classes. In addition
to operations provided
 > by the latter, they also provide methods to do system calls on path
objects."

That's the wording I read.  I inferred that "path-handling operations
which don't actually access a filesystem" meant an object that didn't
necessarily represent an actual file, and that "provide methods to do
system calls on path objects" did indicate an actual file.  From the
existence of Path.read_bytes, I inferred that at least some Path objects
represent (and operate on) actual existing files.  I've been doing this
for a long time, and I may have read my expecations into those words.

 > There is no requirement that a Path() names a file that exists even.

Agreed.

 >> That said, why doesn't your argument apply to read and write?  I
 >> would certainly expect that writing to a path and then reading
 >> from that same path would return the newly written data.  If I
 >> squint funny, the Path object is tracking the operations on the
 >> file system.
 >
 > I do not expect that. Consider the time line:
 >
 > 1. with p.open('w') write data
 > 2. external process changes file on disk
 > 3. with p.open('r') read data
 >
 > How would (3) get the data written at (1) guaranteed?
 > It will lead to bugs to assume that.

I didn't say anything about a guarantee, or about an external processes.
If I have a single process that writes data to a file and then reads
from that file, I would expect to read what I just wrote.  See the
documentation of Path.read_bytes and Path.write_bytes.  If I throw an
external process, or a networked file system, or multiple layers of
buffering and/or caching into the mix, then all such bets are off.

I think you're making my point about expectations.  :-)

 > The path object is allowing system calls that need a file's path to
be called,
 > that is all. Beyond that there is no relationship between the
pathlib.Path()
 > objects and files.

The documentation of Path.read_bytes and Path.write_bytes say otherwise.

 >> I think I'm actually arguing against some long since made (and
 >> forgotten?) design decisions that can't be changed (dare I say
 >> fixed?) because changing them would break backwards
 >> compatibility.
 >>
 >> Yuck.  :-)  And I can absolutely see all sorts of different
 >> expecations not being met and having to be explained by saying
 >> "well, that's the way it works."
 >
 > I'd suggest that the design is reasonable and If there is
misunderstanding that its
 > something that docs could address.

I'm not disagreeing.  I suspect that we've both worked on enough
different systems to know that not all OSes, file systems, libraries,
and versions and combinations thereof work the same way under all
circumstances (multiple threads, multiple processes, caching, buffering,
etc.).  It's the epitome of YMMV.

Rename is a particularly thorny case because renaming a file, at least
on a POSIX system, is an operation on the directory containing the file
rather than the file itself.
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[Barry Scott]

> On 30 Sep 2019, at 14:20, Dan Sommers <2qdxy4rzwzuui...@potatochowder.com> 
> wrote:
> 
> That's the wording I read.  I inferred that "path-handling operations
> which don't actually access a filesystem" meant an object that didn't
> necessarily represent an actual file, and that "provide methods to do
> system calls on path objects" did indicate an actual file.  From the
> existence of Path.read_bytes, I inferred that at least some Path objects
> represent (and operate on) actual existing files.  I've been doing this
> for a long time, and I may have read my expecations into those words.

pathlib.Path() adds the semantics of operating system file rules to strings.

The addition of open(), read_bytes() etc is a convenience to the programmer.
Some people consider this a mistake other a great feature.

You can write this:

pathlib.Path("name").open()

Which is the same as:

open(pathlib.Path("name"))

Which is the same as:

open("name")

You would not expect str to track the file, why expect Path() to?

Barry

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Announcing config-path an OS independent configuration file paths

[Barry Scott]

> On 30 Sep 2019, at 14:17, Paul Moore  wrote:
> 
> How does this compare to the existing appdirs module on PyPI?
> 

I did not know about appdirs.

It does not seem to have separate read vs. save paths.
Required for XDG specification where a path of config folder is defined.

On 1st run the config may be in a system directory which the user cannot write 
to.
Saving a config file will go into the users config file.
On the 2nd run that users config should be read not the system config.

appdirs covers the location of more type of files, like cache and data.

Barry

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Announcing config-path an OS independent configuration file paths

[Paul Moore]

On Mon, 30 Sep 2019 at 15:51, Barry Scott  wrote:
>
> > On 30 Sep 2019, at 14:17, Paul Moore  wrote:
> >
> > How does this compare to the existing appdirs module on PyPI?
> >
>
> I did not know about appdirs.

Fair enough ;-)

> It does not seem to have separate read vs. save paths.
> Required for XDG specification where a path of config folder is defined.
>
> On 1st run the config may be in a system directory which the user cannot 
> write to.
> Saving a config file will go into the users config file.
> On the 2nd run that users config should be read not the system config.
>
> appdirs covers the location of more type of files, like cache and data.

Interesting - thanks for the comparison, I'll take a detailed look
next time I need this type of functionality.

Paul
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[Dan Sommers]

On 9/30/19 10:33 AM, Barry Scott wrote:



On 30 Sep 2019, at 14:20, Dan Sommers 
<2qdxy4rzwzuui...@potatochowder.com 
> wrote:


That's the wording I read.  I inferred that "path-handling operations
which don't actually access a filesystem" meant an object that didn't
necessarily represent an actual file, and that "provide methods to do
system calls on path objects" did indicate an actual file.  From the
existence of Path.read_bytes, I inferred that at least some Path objects
represent (and operate on) actual existing files.  I've been doing this
for a long time, and I may have read my expecations into those words.


pathlib.Path() adds the semantics of operating system file rules to strings.


pathlib.Path() adds the semantics of OS pathnames to strings


The addition of open(), read_bytes() etc is a convenience to the programmer.
Some people consider this a mistake other a great feature.


Regardless of your opinion of those methods, they exist, and
evidently, they lead to arguably unfounded expectations.

Do they rise to the level of a wart that should be deprecated?

I don't know.

Are they practical in a number of use cases?

Probably.

That was my original point:  a Path object backed by OS- and
filesystem- objects and operations is a great idea, but it's
imperfect and it leads some programmers astray.


You can write this:

pathlib.Path("name").open()

Which is the same as:

open(pathlib.Path("name"))

Which is the same as:

open("name")

You would not expect str to track the file, why expect Path() to?


That's an interesting question.  If you phrase the question like
that, then you're right:  expecting a string to track the content
of a file is a mistake.

In the totality of a Path object that claims to represent paths
to files, including the arguably troublesome read_bytes and
write_bytes methods, and a rename method, however, it's not
unreasonable expect the object to track *itself* when I use *its*
own rename method (backwards compatibility restraints
notwithstanding).

Dan
--
https://mail.python.org/mailman/listinfo/python-list

Delay in python startup.

[Tobiah]

I don't have a lot of information, so here goes a shot in
the dark.  One day I started experiencing a delay when
starting python.  I'm on Ubuntu 16.04.  It takes three
seconds to get a prompt when I type 'python' on the command
line (Python 2.7.12).  When I run a script that imports
packages, it takes longer, up to 17 seconds just to do
the imports.  Python3 is not affected, and is snappy as
expected.

That's all I know.  I'm hoping someone else has seen this.
I'm about ready to wipe the drive and upgrade to 18.04.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Hi how do I import files inside a txt file?

[Tobiah]

On 9/2/19 3:32 AM, Spencer Du wrote:

Hi

How do i import files inside a txt file if they exist in the current
directory?



Once you've read the module names you can use:

  new_module = __import__(modulename)

So you'd read the name from your file into
modulename and import the name contained in
that variable in this way.




--
https://mail.python.org/mailman/listinfo/python-list

Re: Recursive method in class

[Anders Märak Leffler]

What do you mean by transformed? This is probably your understanding
already, but a further consequence of when arguments are evaluated
plus what you said about data attributes is that the fib(self, n - 1)
call will follow the standard LEGB-lookup of whatever "fib" is, from
the point of view of the function object. As far as I know, there is
no transformation of these scopes - either when it comes to creating
the class, or creating the instances. (self is just the instance,
passed as an argument.)

Cf when you change a binding:

>>> def factorial(self, n):
... if not n:
... return 1
... else:
... return n * factorial(self, n - 1)
...
>>> Dummy = type("DummyObject", (object, ), {"factorial" : factorial})
>>> instance = Dummy()
>>> def factorial(self, n):
... print("Hello!")
... return 999
...
>>> instance.factorial(5)   # Where will the call go ("old" or "new" 
>>> factorial?)? Where will possible recursive calls go (and why)?
Hello!
4995

Oh, and as others have pointed out on this list - you/whoever runs the
system sending the mail might want to change the return address.
n...@gmail.com is somewhat consistently classed as spam.



//Anders

PS. We could further complicate this by adding a call to
self.factorial in the new function, but let's not go there. :)


On Mon, Sep 30, 2019 at 9:58 AM ast  wrote:
>
> Le 27/09/2019 à 14:26, Jan van den Broek a écrit :
> > On 2019-09-27, ast  wrote:
> >> Is it feasible to define a recursive method in a class ?
> >> (I don't need it, it's just a trial)
> >>
> >> Here are failing codes:
> >>
> >>
> >> class Test:
> >>   def fib(self, n):
> >>   if n < 2: return n
> >>   return fib(self, n-2) + fib(self, n-1)
> >self.fib(...)
> >
> > [Schnipp]
> >
>
> Yes you are right. I forgot that class methods
> don't see class attributes
>
> An other workaround:
>
>   def fib(self, n):
>   if n < 2: return n
>   return Test.fib(self, n-2) + Test.fib(self, n-1)
>
> It comes from https://www.pythonsheets.com/notes/python-object.html
>
>  >>> def fib(self, n):
> ... if n <= 2:
> ... return 1
> ... return fib(self, n-1) + fib(self, n-2)
> ...
>  >>> Fib = type('Fib', (object,), {'val': 10,
> ...   'fib': fib})
>  >>> f = Fib()
>  >>> f.val
> 10
>  >>> f.fib(f.val)
> 55
>
> So it means that when classes are constructed explicitely
> with type(name, bases, dict_), some methods are transformed
> --
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[Chris Angelico]

On Tue, Oct 1, 2019 at 1:51 AM Dan Sommers
<2qdxy4rzwzuui...@potatochowder.com> wrote:
> In the totality of a Path object that claims to represent paths
> to files, including the arguably troublesome read_bytes and
> write_bytes methods, and a rename method, however, it's not
> unreasonable expect the object to track *itself* when I use *its*
> own rename method (backwards compatibility restraints
> notwithstanding).
>

What if the path is formed from another path, and the other one
renames? Create a Path("/path/to/some/dir") and then file = dir /
"some_file", then rename the directory - should the file insta-reflect
that? Even if you DO have them link magically so that it updates, you
would then have to contend with the fact that, when you rename
directories, you often *want* to slide onto the new dir - that's one
way to replace an entire set of files atomically (which also involves
open_at).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Delay in python startup.

[Chris Angelico]

On Tue, Oct 1, 2019 at 1:56 AM Tobiah  wrote:
>
> I don't have a lot of information, so here goes a shot in
> the dark.  One day I started experiencing a delay when
> starting python.  I'm on Ubuntu 16.04.  It takes three
> seconds to get a prompt when I type 'python' on the command
> line (Python 2.7.12).  When I run a script that imports
> packages, it takes longer, up to 17 seconds just to do
> the imports.  Python3 is not affected, and is snappy as
> expected.
>
> That's all I know.  I'm hoping someone else has seen this.
> I'm about ready to wipe the drive and upgrade to 18.04.
>

Python 2 and Python 3 have completely independent installations, so it
could be a lot of things. First question: Does it take three seconds
*every* time you type 'python', or only the first time? If it's slow
the first time but then fast, it's probably just a matter of disk
caching; running Python 3 doesn't pre-cache the files for Python 2, so
you have to pay the load-time cost anew.

If it's slow every time, though, you may have something messing with
your startup. Try "python -S" and "python -E" see if they're just as
slow. That would be a way to delve into things a bit.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[Dan Sommers]

On 9/30/19 12:51 PM, Chris Angelico wrote:

On Tue, Oct 1, 2019 at 1:51 AM Dan Sommers
<2qdxy4rzwzuui...@potatochowder.com> wrote:

In the totality of a Path object that claims to represent paths
to files, including the arguably troublesome read_bytes and
write_bytes methods, and a rename method, however, it's not
unreasonable expect the object to track *itself* when I use *its*
own rename method (backwards compatibility restraints
notwithstanding).



What if the path is formed from another path, and the other one
renames? Create a Path("/path/to/some/dir") and then file = dir /
"some_file", then rename the directory - should the file insta-reflect
that? Even if you DO have them link magically so that it updates, you
would then have to contend with the fact that, when you rename
directories, you often *want* to slide onto the new dir - that's one
way to replace an entire set of files atomically (which also involves
open_at).


Mu.

The filesystem is its own ultimate arbiter.  It handles (or not)
race conditions, failures, multiple processes, etc. in some
manner, which might not even be reasonable or consistent from one
moment to the next (who among us hasn't debugged a problem that
came down the patch level of NFS running on some remote system?).
The level to which some random object in a Python heap reflects
any given filesystem is, well, arbitrary.

All I'm doing is defending the OP, who was surprised that
renaming a file *using a Path instance* didn't reflect that
operation *in that Path instance*.  I believe that such a
surprise is reasonable; others apparently don't.

Yes, there are a lot of reasons that it is the way it is, and a
lot of reasons not to change it now.  I get that.  If the OP is
still here, then I'd like to think that the OP also gets that.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Delay in python startup.

[Tobiah]

On 9/30/19 9:54 AM, Chris Angelico wrote:

On Tue, Oct 1, 2019 at 1:56 AM Tobiah  wrote:


I don't have a lot of information, so here goes a shot in
the dark.  One day I started experiencing a delay when
starting python.  I'm on Ubuntu 16.04.  It takes three
seconds to get a prompt when I type 'python' on the command
line (Python 2.7.12).  When I run a script that imports
packages, it takes longer, up to 17 seconds just to do
the imports.  Python3 is not affected, and is snappy as
expected.

That's all I know.  I'm hoping someone else has seen this.
I'm about ready to wipe the drive and upgrade to 18.04.



Python 2 and Python 3 have completely independent installations, so it
could be a lot of things. First question: Does it take three seconds
*every* time you type 'python', or only the first time? If it's slow
the first time but then fast, it's probably just a matter of disk
caching; running Python 3 doesn't pre-cache the files for Python 2, so
you have to pay the load-time cost anew.

If it's slow every time, though, you may have something messing with
your startup. Try "python -S" and "python -E" see if they're just as
slow. That would be a way to delve into things a bit.

ChrisA


It was much faster with -S and instantaneous with -E.  I had a directory
in my PYTHONPATH that I mount with sshfs from a server.  For some reason
that mount is very slow.  Thanks for helping me figure this out.


Tobiah
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[DL Neil via Python-list]

On 1/10/19 6:13 AM, Dan Sommers wrote:

On 9/30/19 12:51 PM, Chris Angelico wrote:

On Tue, Oct 1, 2019 at 1:51 AM Dan Sommers

...


All I'm doing is defending the OP, who was surprised that
renaming a file *using a Path instance* didn't reflect that
operation *in that Path instance*.  I believe that such a
surprise is reasonable; others apparently don't.

Yes, there are a lot of reasons that it is the way it is, and a
lot of reasons not to change it now.  I get that.  If the OP is
still here, then I'd like to think that the OP also gets that.



The OP is still 'here' - has been asleep = lazy toad!

Thank you. Yes, that is a reasonable summary. Yes it did catch me by 
surprise - that I couldn't rely upon the instance to reflect 'reality' 
(IMHO).


I see there is a different point-of-view. To adjust my mind-set and 
properly appreciate the discussion, I'd like to re-read the docs with 
that in-mind, first...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Delay in python startup.

[Chris Angelico]

On Tue, Oct 1, 2019 at 3:31 AM Tobiah  wrote:
>
> On 9/30/19 9:54 AM, Chris Angelico wrote:
> > On Tue, Oct 1, 2019 at 1:56 AM Tobiah  wrote:
> >>
> >> I don't have a lot of information, so here goes a shot in
> >> the dark.  One day I started experiencing a delay when
> >> starting python.  I'm on Ubuntu 16.04.  It takes three
> >> seconds to get a prompt when I type 'python' on the command
> >> line (Python 2.7.12).  When I run a script that imports
> >> packages, it takes longer, up to 17 seconds just to do
> >> the imports.  Python3 is not affected, and is snappy as
> >> expected.
> >>
> >> That's all I know.  I'm hoping someone else has seen this.
> >> I'm about ready to wipe the drive and upgrade to 18.04.
> >>
> >
> > Python 2 and Python 3 have completely independent installations, so it
> > could be a lot of things. First question: Does it take three seconds
> > *every* time you type 'python', or only the first time? If it's slow
> > the first time but then fast, it's probably just a matter of disk
> > caching; running Python 3 doesn't pre-cache the files for Python 2, so
> > you have to pay the load-time cost anew.
> >
> > If it's slow every time, though, you may have something messing with
> > your startup. Try "python -S" and "python -E" see if they're just as
> > slow. That would be a way to delve into things a bit.
> >
> > ChrisA
> >
> It was much faster with -S and instantaneous with -E.  I had a directory
> in my PYTHONPATH that I mount with sshfs from a server.  For some reason
> that mount is very slow.  Thanks for helping me figure this out.
>

Cool! That would be it, then - every import had to go through that SSH tunnel.

Happy to help out.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[Barry Scott]

> On 30 Sep 2019, at 16:49, Dan Sommers <2qdxy4rzwzuui...@potatochowder.com> 
> wrote:
> 
> That's an interesting question.  If you phrase the question like
> that, then you're right:  expecting a string to track the content
> of a file is a mistake.



> 
> In the totality of a Path object that claims to represent paths
> to files,

It represents string that *should* in most cases work in the APIs
that work on files. Not the same idea.


> including the arguably troublesome read_bytes and
> write_bytes methods, and a rename method, however, it's not
> unreasonable expect the object to track *itself* when I use *its*
> own rename method (backwards compatibility restraints
> notwithstanding).

"the object" did track the changes its just that "the object" is not
the Path object, it's in the operating system and/or the file system.
For the rename it's the directory that the name is recorded in.

There was an interest talk at this years PYCON UK about the
the errors that people new to python make. Misunderstand syntax
is about 1/3 of the problems, but 2/3 was having the wrong model.

This discussion seems to fall into the "wrong model" world that
leads to a expectation failure.

Have we moved on to how we can improve the situation?

Barry


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Delay in python startup.

[Cameron Simpson]

On 30Sep2019 10:28, Tobiah  wrote:

On 9/30/19 9:54 AM, Chris Angelico wrote:

On Tue, Oct 1, 2019 at 1:56 AM Tobiah  wrote:
It was much faster with -S and instantaneous with -E.  I had a 
directory

in my PYTHONPATH that I mount with sshfs from a server.  For some reason
that mount is very slow.  Thanks for helping me figure this out.


For the future: you can often see something this slow with an strace:

 strace python

Watch what it is doing when it stalls.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[Dan Sommers]

On 9/30/19 3:56 PM, Barry Scott wrote:



On 30 Sep 2019, at 16:49, Dan Sommers 
<2qdxy4rzwzuui...@potatochowder.com 
> wrote:



In the totality of a Path object that claims to represent paths
to files,


It represents string that *should* in most cases work in the APIs
that work on files. Not the same idea.


I'm not sure the documentation supports your view.  Components
of paths are strings, but IMO the Path object represents a file.


including the arguably troublesome read_bytes and
write_bytes methods, and a rename method, however, it's not
unreasonable expect the object to track *itself* when I use *its*
own rename method (backwards compatibility restraints
notwithstanding).


"the object" did track the changes its just that "the object" is not
the Path object, it's in the operating system and/or the file system.
For the rename it's the directory that the name is recorded in.


So which is it?  Does the Path object represent a string, the
name of a file (whether that file exists or not), or the file
itself?  A moment ago, you claimed that a Path object represents
a string that should work in the APIs that work on files.  Now,
you're claiming that the Path object is a proxy for something in
the filesystem.  I don't mean to be combative or confrontational,
but I think that this fuzziness/inconsistency is at or near the
root of the differing expectations.


There was an interest talk at this years PYCON UK about the
the errors that people new to python make. Misunderstand syntax
is about 1/3 of the problems, but 2/3 was having the wrong model.

This discussion seems to fall into the "wrong model" world that
leads to a expectation failure.


On this (that there's something about the model of Path objects
that leads to expectation failures) we agree.  :-)


Have we moved on to how we can improve the situation?


Absolutely.

Documenting the fact that calling rename on a Path object does
not update that object, and at least an acknowledgement of the
backwards compatibility issues, would be a good start.  Does the
same sort of thing happen with Path.chmod as well?

Clarifying what a Path object represents is also in order.  Again,
I note that the top of https://docs.python.org/3/library/pathlib.html
talks about filesystem paths, but a lot of the method descriptions
(e.g., rename) use the phrase "this file," as if Path objects
represent (or are actually proxies for) actual files or an actual
filesystem.
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[DL Neil via Python-list]

On 30/09/19 9:28 PM, Barry Scott wrote:

On 30 Sep 2019, at 05:40, DL Neil via Python-list  
wrote:

Should pathlib reflect changes it has made to the file-system?


I think it should not.


The term "concrete" is applied to Path(), PosixPath(), and WindowsPath() 
- whereas the others are differentiated with the prefix "Pure".


I take "concrete" to mean 'existing in reality or real experience'. 
Thus, I saw the Pure* entities as supporting abstract paths, but the 
concrete entities as representing (and reflecting) real-world (file 
system) entities.


Thus, there is no need for .exists() to be available in the Pure paths, 
but there is when utilising their concrete implementations.


NB .name however is inherited from PurePath()!



A Path() is the name of a file it is not the file itself. Why should it
track changes in the file system for the name?


BUT... Path() does keep track of changes in the file system for other 
attributes! So, why not also name?


(remember that Python (in our code examples) does not know the file by 
its file-name (before OR after rename) but by the instance name, eg 
"save_file", below)




Here is code to show why following the rename will break logic:

save_file = pathlib.Path('my-data.txt')

def save( data ):
# backup file
if save_file.exists():
save_file.rename('my-data.backup')

# save data
with save_file.open() as f:
f.write( data )

while True:
save( generate_data() )
time.sleep( interval )

If the rename of the file changed the path the above code will fail.



That is one use-case, but in the use-case which led to the original 
post, the logic was:


iterate directory-entries,
if fileNM follows an out-of-date naming-convention,
rename the file,
then process further
(logging observations along the way).

Here's a code-snippet illustrating both of the above points:

import pathlib
p = pathlib.Path( "data-file" )
p
# PosixPath('data-file')
p.stat()
os.stat_result(... st_mtime=1569898467, st_ctime=1569898467)
# ... = excised results, leaving two data-points worth observing

with p.open("w") as f:
f.write("new stuff")
# 9

p.stat()
os.stat_result(... st_mtime=1569898572, st_ctime=1569898572)
# hey look, this reflects REAL and CHANGED data from the FS

# using input logic, cf previous example's output logic
p.rename( "new-name" )
with p.open( "r" ) as f: f.readline()
...
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib64/python3.7/pathlib.py", line 1193, in open
opener=self._opener)
  File "/usr/lib64/python3.7/pathlib.py", line 1046, in _opener
return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'data-file'

# hence we cannot follow the use-case

# why? Because .name == 'data-file' but the real-world file is (now) 
called "new-name".


NB I would not attempt to suggest that the logic of the 'write' use-case 
is any more, or any less, valid that that of the 'read' use-case; and 
I'm not arguing with you personally!



Looking at the PEP, it didn't alleviate my confusion because:
(a) <<<
Why an object-oriented API
... form of filesystem handling abstraction
>>>
(b) <<<
Immutability
Path objects are immutable, which makes them hashable and also prevents 
a class of programming errors.

>>>
(c) <<<
Concrete paths API
In addition to the operations of the pure API, concrete paths provide 
additional methods which actually access the filesystem to query or 
mutate information.

>>>

I liked the OOP aims in point (a) but it clearly says "filesystem".

Whereas the logic (mentioned above) of the inherent 'safety' of 
immutable objects is inescapable - point (b), that is incompatible with 
the usage and behavior of a file system. [which is a point made (by 
others), elsewhere)]


Point (c) appears to suggest (as written above) that whereas the Pure 
API can be immutable and separate from any physical file system, the 
Concrete paths will perform actions and thus mutate with the real FS.



Further to my (personal) point about use-cases and debate, if we follow 
the 'recommended' code snippet in the (PSL) manual, the logic is that 
there should be separate paths/Paths (whether for the two (physical) 
files or not) - their close relationship notwithstanding:


>>> p = Path('foo')
>>> p.open('w').write('some text')
9
>>> target = Path('bar')
>>> p.rename(target)
>>> target.open().read()
'some text'

NB utilising two Path instances (cf allowing a string to 'intrude') will 
enrich both of the use-cases we have illustrated!


Also, in subscribing to the rationale that it is better to represent 
paths/file-names with a semantic object, then the above makes good 
sense. (historically there has been debate about whether paths are a 
sub-class of string (a path is a string, but a string may not be a class 
- indeed a string intended to be a path may not be legal within a file 
system) or 

Re: pathlib

[DL Neil via Python-list]

On 1/10/19 1:09 AM, Chris Angelico wrote:

On Mon, Sep 30, 2019 at 9:54 PM Dan Sommers
<2qdxy4rzwzuui...@potatochowder.com> wrote:

I would have said the same thing, but the docs⁰ disagree:  a
PurePath represents the name of (or the path to) a file, but a
Path represents the actual file.


⁰ https://docs.python.org/3/library/pathlib.html


I don't think it represents the actual file. If it did, equality would
be defined by samefile, NOT by the file name.


from pathlib import Path
import os
open("file1", "w").close()
os.link("file1", "file2")
Path("file1") == Path("file2")

False

Path("file1").samefile(Path("file2"))

True

Path("file1") == Path("file1")

True



This example involves a "hard link" in Linux - can't recall if it 
applies under MS-Win...


On a Linux system the 'two' files share the same inode, and thus the 
same 'patch of disk', but one can be deleted without affecting the 
other. Thus, borrowing from the above snippet:


>>> import os
>>> open("file1", "w").close()
>>> os.link("file1", "file2")
>>> f1 = pathlib.Path( "file1" )
>>> f2 = pathlib.Path( "file2" )
>>> f1 == f2
False
### they are not the same path (by name) - due only to the last character

>>> f1.stat()
os.stat_result(st_mode=33204, st_ino=1049466, st_dev=64769, st_nlink=2, 
st_uid=1000, st_gid=1000, st_size=0, st_atime=1569903409, 
st_mtime=1569903409, st_ctime=1569903410)

>>> f2.stat()
os.stat_result(st_mode=33204, st_ino=1049466, st_dev=64769, st_nlink=2, 
st_uid=1000, st_gid=1000, st_size=0, st_atime=1569903409, 
st_mtime=1569903409, st_ctime=1569903410)

>>> f1.samefile( f2 )
True
### but they do 'label' the same file when the path is applied to the FS.

Let's try something similar, but the other-way-around:

>>> f1 = pathlib.Path( "file1" )
>>> f1.stat()
os.stat_result(st_mode=33204, st_ino=1049466, st_dev=64769, st_nlink=1, 
st_uid=1000, st_gid=1000, st_size=0, st_atime=1569903409, 
st_mtime=1569903409, st_ctime=1569903851)

### this path exists in the FS

>>> f2 = pathlib.Path( "not-file1" )
>>> f2.stat()
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib64/python3.7/pathlib.py", line 1168, in stat
return self._accessor.stat(self)
FileNotFoundError: [Errno 2] No such file or directory: 'not-file1'
### this path does not describe a file in the FS

>>> f1.rename( f2 )
### here's a rename operation per the manual!
>>> f1.stat()
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib64/python3.7/pathlib.py", line 1168, in stat
return self._accessor.stat(self)
FileNotFoundError: [Errno 2] No such file or directory: 'file1'
### but f1 does not represent a 'real file' any more

>>> f2.stat()
os.stat_result(st_mode=33204, st_ino=1049466, st_dev=64769, st_nlink=1, 
st_uid=1000, st_gid=1000, st_size=0, st_atime=1569903409, 
st_mtime=1569903409, st_ctime=1569904293)

### f2 is a path which represents a real-world file though
### where have we seen those numbers before?


> It still represents the path to the file, not the file itself, and if
> you move something over it, it will see the new file.

I like this description. Thanks!

That said, maybe pathlib should have stuck with paths/PurePaths, and we 
should have something else (logically separate, but closely-related) 
which manipulates the files themselves?
(preferably OO, and tidying-up/offering an alternative to the morass of 
os, os.path, sys, shutil, and ?others)

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[DL Neil via Python-list]

On 1/10/19 1:40 AM, Barry Scott wrote:




On 30 Sep 2019, at 12:51, Dan Sommers <2qdxy4rzwzuui...@potatochowder.com> 
wrote:

On 9/30/19 4:28 AM, Barry Scott wrote:

On 30 Sep 2019, at 05:40, DL Neil via Python-list  
wrote:
Should pathlib reflect changes it has made to the file-system?

I think it should not.
A Path() is the name of a file it is not the file itself. Why should it
track changes in the file system for the name?


I would have said the same thing, but the docs⁰ disagree:  a
PurePath represents the name of (or the path to) a file, but a
Path represents the actual file.


I'm not seeing that wording in the python 3.7 pathlib documentation.
Can you quote the exact wording please?

I do see this:

"Pure path objects provide path-handling operations which don’t actually access a 
filesystem."

And:

"Concrete paths are subclasses of the pure path classes. In addition to 
operations provided
by the latter, they also provide methods to do system calls on path objects."

There is no requirement that a Path() names a file that exists even.



That is true - of a "path".

However, as soon as said path is applied to the file system, eg .open( 
"r" ) it had better exist or we will disappear into OSError-country.


If we use .open( "w" ) then the very real file is created within the FS, 
at that path.


So, the file does not need to exist for a path to be legal - which is a 
(to me sounds like) PurePath.


However, don't we have to suspend such 'purity' in the "concrete" classes...


That said, why doesn't your argument apply to read and write?  I
would certainly expect that writing to a path and then reading
from that same path would return the newly written data.  If I
squint funny, the Path object is tracking the operations on the
file system.


I do not expect that. Consider the time line:

1. with p.open('w') write data
2. external process changes file on disk
3. with p.open('r') read data

How would (3) get the data written at (1) guaranteed?
It will lead to bugs to assume that.

The path object is allowing system calls that need a file's path to be called,
that is all. Beyond that there is no relationship between the pathlib.Path()
objects and files.


In theory one can separate paths and files. In practice, if Python is to 
access a file it must pass a path to the OpSys/FS.


To be fair, the temporal argument (race condition) applies to any IO 
method unless "locking" is available (it is not, either in pathlib or 
the os and sys libraries.


You cannot .open( 'r' ) unless the actual file exists.

You (probably) won't want to .open( 'w' ) if the actual file exists.

Wouldn't we like a library which helps, in both cases!



I think I'm actually arguing against some long since made (and
forgotten?) design decisions that can't be changed (dare I say
fixed?) because changing them would break backwards
compatibility.


+1



Yuck.  :-)  And I can absolutely see all sorts of different
expecations not being met and having to be explained by saying
"well, that's the way it works."


I'd suggest that the design is reasonable and If there is misunderstanding that 
its
something that docs could address.


Design: +1
Misunderstanding: +1
Docs: +1

Perhaps pathlib encourages one/some to believe that it has been designed 
to do more than it does/set out to do?


Perhaps it does more than it should (by implementing some closer 
relationships with the underlying file system, but not others)?


Perhaps it makes sense for .name to be a data-attribute in the Pure 
classes, but it should be name(), ie a method-attribute in the concrete 
classes? (per other methods subject to change/mutation - see .stat() 
examples in earlier responses)

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[DL Neil via Python-list]

On 1/10/19 3:21 AM, Dan Sommers wrote:

On 9/30/19 8:40 AM, Barry Scott wrote:
  >> On 30 Sep 2019, at 12:51, Dan Sommers
<2qdxy4rzwzuui...@potatochowder.com> wrote:
  >> On 9/30/19 4:28 AM, Barry Scott wrote:
   On 30 Sep 2019, at 05:40, DL Neil via Python-list
 wrote:
   Should pathlib reflect changes it has made to the file-system?
  >>> I think it should not.
  >>> A Path() is the name of a file it is not the file itself. Why 
should it

  >>> track changes in the file system for the name?
  >>
  >> I would have said the same thing, but the docs⁰ disagree:  a
  >> PurePath represents the name of (or the path to) a file, but a
  >> Path represents the actual file.
  >
  > I'm not seeing that wording in the python 3.7 pathlib documentation.
  > Can you quote the exact wording please?
  >
  > I do see this:
  >
  > "Pure path objects provide path-handling operations which don’t
actually access a filesystem."
  >
  > And:
  >
  > "Concrete paths are subclasses of the pure path classes. In addition
to operations provided
  > by the latter, they also provide methods to do system calls on path
objects."

That's the wording I read.  I inferred that "path-handling operations
which don't actually access a filesystem" meant an object that didn't
necessarily represent an actual file, and that "provide methods to do
system calls on path objects" did indicate an actual file.  From the
existence of Path.read_bytes, I inferred that at least some Path objects
represent (and operate on) actual existing files.  I've been doing this
for a long time, and I may have read my expecations into those words.


+1 "Pure" cf "concrete".

The mixture makes it difficult to insist that a Path does not represent 
a file if (some) operations are included.




  > There is no requirement that a Path() names a file that exists even.
Agreed.

  >> That said, why doesn't your argument apply to read and write?  I
  >> would certainly expect that writing to a path and then reading
  >> from that same path would return the newly written data.  If I
  >> squint funny, the Path object is tracking the operations on the
  >> file system.
  >
  > I do not expect that. Consider the time line:
  >
  > 1. with p.open('w') write data
  > 2. external process changes file on disk
  > 3. with p.open('r') read data
  >
  > How would (3) get the data written at (1) guaranteed?
  > It will lead to bugs to assume that.

I didn't say anything about a guarantee, or about an external processes.
If I have a single process that writes data to a file and then reads
from that file, I would expect to read what I just wrote.  See the
documentation of Path.read_bytes and Path.write_bytes.  If I throw an
external process, or a networked file system, or multiple layers of
buffering and/or caching into the mix, then all such bets are off.

I think you're making my point about expectations.  :-)


+1



  > The path object is allowing system calls that need a file's path to
be called,
  > that is all. Beyond that there is no relationship between the
pathlib.Path()
  > objects and files.

The documentation of Path.read_bytes and Path.write_bytes say otherwise.


+1



  >> I think I'm actually arguing against some long since made (and
  >> forgotten?) design decisions that can't be changed (dare I say
  >> fixed?) because changing them would break backwards
  >> compatibility.
  >>
  >> Yuck.  :-)  And I can absolutely see all sorts of different
  >> expecations not being met and having to be explained by saying
  >> "well, that's the way it works."
  >
  > I'd suggest that the design is reasonable and If there is
misunderstanding that its
  > something that docs could address.

I'm not disagreeing.  I suspect that we've both worked on enough
different systems to know that not all OSes, file systems, libraries,
and versions and combinations thereof work the same way under all
circumstances (multiple threads, multiple processes, caching, buffering,
etc.).  It's the epitome of YMMV.

Rename is a particularly thorny case because renaming a file, at least
on a POSIX system, is an operation on the directory containing the file
rather than the file itself.



Thank you @Dan for keeping the conversation going during my night-hours.
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list