Re: Remove directory tree without following symlinks
Steven D'Aprano writes: > I want to remove a directory, including all files and subdirectories under > it, but without following symlinks. I want the symlinks to be deleted, not > the files pointed to by those symlinks. rm -r shouldn't follow symlinks like you mention. -- https://mail.python.org/mailman/listinfo/python-list
RE: Remove directory tree without following symlinks
> From: eryk...@gmail.com
> Date: Fri, 22 Apr 2016 13:28:01 -0500
> Subject: Re: Remove directory tree without following symlinks
> To: python-list@python.org
>
> On Fri, Apr 22, 2016 at 12:39 PM, Albert-Jan Roskam
> wrote:
> > FYI, Just today I found out that shutil.rmtree raises a WindowsError if the
> > dir is read-
> > only (or its contents). Using 'ignore_errors', won't help. Sure, no error
> > is raised, but the
> > dir is not deleted either! A 'force' option would be a nice improvement.
>
> Use the onerror handler to call os.chmod(path, stat.S_IWRITE). For
> example, see pip's rmtree_errorhandler:
>
> https://github.com/pypa/pip/blob/8.1.1/pip/utils/__init__.py#L105
Thanks, that looks useful indeed. I thought about os.chmod, but with os.walk.
That seemed expensive. So I used subprocess.call('rmdir "%s" /s /q' % dirname).
That's Windows only, of course, but aside of that, is using subprocess less
preferable?Fun fact: I used it to remove .svn dirs, just like what is mentioned
in the pip comments :-)
--
https://mail.python.org/mailman/listinfo/python-list
Re: Remove directory tree without following symlinks
On Sat, 23 Apr 2016 06:13 pm, Paul Rubin wrote: > Steven D'Aprano writes: >> I want to remove a directory, including all files and subdirectories >> under it, but without following symlinks. I want the symlinks to be >> deleted, not the files pointed to by those symlinks. > > rm -r shouldn't follow symlinks like you mention. "rm -r" gives me a NameError when I run it in my Python script :-) But seriously, where is that documented? I've read the man page for rm, and it doesn't say anything about treatment of symlinks, nor is there an option to follow/not follow symlinks. So I never trust rm -r unless I know what I'm deleting. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Remove directory tree without following symlinks
Steven D'Aprano wrote: But seriously, where is that documented? I've read the man page for rm, and it doesn't say anything about treatment of symlinks The Linux man page seems to be a bit deficient on this. The BSD version contains this sentence: The rm utility removes symbolic links, not the files referenced by the links. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Remove directory tree without following symlinks
On Sat, 23 Apr 2016 00:56:33 +1000, Steven D'Aprano wrote:
> I want to remove a directory, including all files and subdirectories under
> it, but without following symlinks. I want the symlinks to be deleted, not
> the files pointed to by those symlinks.
Note that this is non-trivial to do securely, i.e. where an adversary has
write permission on any of the directories involved. Due to the potential
for race conditions between checking whether a name refers to a directory
and recursing into it, the process can be tricked into deleting any
directory tree for which it has the appropriate permissions.
The solution requires:
1. That you always chdir() into each directory and remove entries using
their plain filename, rather than trying to remove entries from a
higher-level directory using a relative path.
2. When chdir()ing into each subdirectory, you need to do e.g.:
st1 = os.stat(".")
os.chdir(subdir)
st2 = os.stat("..")
if st1.st_dev != st2.st_dev or st1.st_ino != st2.st_ino:
raise SomeKindOfException()
If the test fails, it means that the directory you just chdir()d into
isn't actually a subdirectory of the one you just left, e.g. because the
directory entry was replaced between checking it and chdir()ing into it.
On Linux, an alternative is to use fchdir() rather than chdir(), which
changes to a directory specified by an open file descriptor for that
directory rather than by name. Provided that the directory was open()ed
without any race condition (e.g. using O_NOFOLLOW), subsequent fstat() and
fchdir() calls are guaranteed to use the same directory regardless of any
filesystem changes.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Remove directory tree without following symlinks
On Sat, Apr 23, 2016 at 4:34 AM, Albert-Jan Roskam
wrote:
>
>> From: eryk...@gmail.com
>> Date: Fri, 22 Apr 2016 13:28:01 -0500
>> On Fri, Apr 22, 2016 at 12:39 PM, Albert-Jan Roskam
>> wrote:
>> > FYI, Just today I found out that shutil.rmtree raises a WindowsError if
>> > the dir is read-only (or its contents). Using 'ignore_errors', won't help.
>> > Sure, no error is raised, but the dir is not deleted either! A 'force'
>> > option
>> > would be a nice improvement.
>>
>> Use the onerror handler to call os.chmod(path, stat.S_IWRITE). For
>> example, see pip's rmtree_errorhandler:
>>
>> https://github.com/pypa/pip/blob/8.1.1/pip/utils/__init__.py#L105
>
> Thanks, that looks useful indeed. I thought about os.chmod, but with
> os.walk. That seemed expensive. So I used subprocess.call('rmdir "%s" /s /q'
> % dirname). That's Windows only, of course, but aside of that, is using
> subprocess less preferable?
I assume you used shell=True in the above call, and not an external
rmdir.exe. There are security concerns with using the shell if you're
not in complete control of the command line.
As to performance, cmd's rmdir wins without question, not only because
it's implemented in C, but also because it uses the stat data from the
WIN32_FIND_DATA returned by FindFirstFile/FindNextFile to check for
FILE_ATTRIBUTE_DIRECTORY and FILE_ATTRIBUTE_READONLY.
On the other hand, Python wins when it comes to working with deeply
nested directories. Paths in cmd are limited to MAX_PATH characters.
rmdir uses DOS 8.3 short names (i.e. cAlternateFileName in
WIN32_FIND_DATA), but that could still exceed MAX_PATH for a deeply
nested tree, or the volume may not even have 8.3 DOS filenames.
shutil.rmtree allows you to work around the DOS limit by prefixing the
path with "\\?\". For example:
>>> subprocess.call(r'rmdir /q/s Z:\Temp\long', shell=True)
The path Z:\Temp\long\aa
a is too long.
0
>>> shutil.rmtree(r'\\?\Z:\Temp\long')
>>> os.path.exists(r'Z:\Temp\long')
False
Using "\\?\" requires a path that's fully qualified, normalized
(backslash only), and unicode (i.e. decode a Python 2 str).
--
https://mail.python.org/mailman/listinfo/python-list
Re: Remove directory tree without following symlinks
On Sat, Apr 23, 2016, at 06:24, Steven D'Aprano wrote: > "rm -r" gives me a NameError when I run it in my Python script :-) > > But seriously, where is that documented? I've read the man page for > rm, and it doesn't say anything about treatment of symlinks, nor is > there an option to follow/not follow symlinks. So I never trust rm -r > unless I know what I'm deleting. The Unix Standard says "For each entry contained in file, other than dot or dot-dot, the four steps listed here (1 to 4) shall be taken with the entry as if it were a file operand. The rm utility shall not traverse directories by following symbolic links into other parts of the hierarchy, but shall remove the links themselves." and "The rm utility removes symbolic links themselves, not the files they refer to, as a consequence of the dependence on the unlink() functionality, per the DESCRIPTION. When removing hierarchies with -r or -R, the prohibition on following symbolic links has to be made explicit." OSX (and I assume other BSDs) says "The rm utility removes symbolic links, not the files referenced by the links." I don't know why GNU rm's documentation doesn't say anything about its treatment of symlinks - maybe it never occurred to anyone at GNU that someone might think it would do anything else. (I'd be less inclined to trust windows' treatment of symlinks, junctions, and other reparse points without doing some experiments) -- https://mail.python.org/mailman/listinfo/python-list
Re: Remove directory tree without following symlinks
On Sat, Apr 23, 2016, at 12:29, Nobody wrote: > On Linux, an alternative is to use fchdir() rather than chdir(), which > changes to a directory specified by an open file descriptor Of course, then there's also the risk of running out of open file descriptors. High-quality implementations of rm will fork to deal with this. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 22 April 2016 at 04:11, Christopher Reimer
wrote:
> On 4/21/2016 7:10 PM, Ethan Furman wrote:
>>> I do plan to incorporate a sanity test in each Piece class to validate
>>> the initial position value. Pawns have 16 specific positions. Bishop,
>>> Knight and Rook each have four specific positions. King and Queen each
>>> have two specific positions. An invalid value will raise an exception.
>>
>> This will make it so you cannot use your PieceFactory for custom setups.
>
> The sanity check won't be in the PieceFactory, but in the Piece class as an
> interface and each Piece subclass will implement the correct positions for
> comparison.
This is still backwards to me. It prevents your classes from being
suitable for restoring a stored game state, not just custom starting
positions (which I think is what Ethan means by custom setups).
If you need to put sanity checking in your initialisation at all it
should be at some higher level, probably in a method named something
like `start_chess_game`. But really I'd suggest just writing tests for
that method to make sure it's doing the right thing.
On the other hand (and I'm sure you've already thought of this) it
probably does make sense to put piece move validation in to each of
the piece classes: you could even have some fun with it...
def move_straight(old, new):
if old[0] == new[0] or old[1] == new[1]:
return True
def move_diag(old, new):
diff_x = abs(old[0] - new[0])
diff_y = abs(old[1] - new[1])
if diff_x == diff_y:
return True
class Piece:
...
def move(self, new):
if self.validate_move(new):
# do the move
...
else: raise IllegalMove('or something')
class Queen(Piece):
def validate_move(self, new):
return any((test(self.position, new) for test in
(move_straight, move_diag)))
Ok I'll stop before I get too carried away... This is completely
untested so bugs will abound I'm sure :)
--
Matt Wheeler
http://funkyh.at
--
https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 4/21/2016 9:46 PM, Ethan Furman wrote: Oh! and Enum!!! ;) OMG! I totally forgot about Enum. Oh, look. Python supports Enum. Now I don't have to roll my own! Hmm... What do we use Enum for? :) Thank you, Chris R. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On Sat, Apr 23, 2016 at 9:01 PM Christopher Reimer < christopher_rei...@icloud.com> wrote: > On 4/21/2016 9:46 PM, Ethan Furman wrote: > > Oh! and Enum!!! ;) > > OMG! I totally forgot about Enum. Oh, look. Python supports Enum. Now I > don't have to roll my own! > > Hmm... What do we use Enum for? :) > You can use Enum in certain circumstances to replace int or str constants. It can help avoid mistyping mistakes and might help your IDE give auto-complete suggestions. I haven't found a good use for them myself, but I'd been mostly stuck in Python 2 until recently. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 4/21/2016 10:25 PM, Stephen Hansen wrote:
Why not, 'color in ("black", "white")'?
Checkers seems popular around here. What if I want to change "white" to
"red," as red and black is a common color scheme for checkers. Do I
change a single constant variable or replace all the occurrences in the
files?
Some of these constants are shortcuts. Instead of writing slice(0, 16)
or slice(48, 64), and getting the two confused, I write
const['board_bottom'] or const['board_top'], respectively, when I want
to pull the correct set of positions from coordinates list.
That said, if you're wanting to share constants across different parts
of your code, use a module.
I did that at first, made it into a dictionary class and added
ConfigParser to the mix. I had a lot of fun putting that one together.
However, as I refactor my code further as I learn new things, it will
probably get changed or removed. Being a research project, I'm willing
to dive into every rabbit hole that I come across to learn Python properly.
Thank you,
Chris R.
--
https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On Sat, Apr 23, 2016 at 7:00 PM, Christopher Reimer wrote: > On 4/21/2016 9:46 PM, Ethan Furman wrote: >> >> Oh! and Enum!!! ;) > > > OMG! I totally forgot about Enum. Oh, look. Python supports Enum. Now I > don't have to roll my own! > > Hmm... What do we use Enum for? :) Python enums are great. Sadly, they're still not quite as awesome as Java enums. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On Sat, Apr 23, 2016 at 9:31 PM Christopher Reimer <
christopher_rei...@icloud.com> wrote:
> On 4/21/2016 10:25 PM, Stephen Hansen wrote:
> >
> > Why not, 'color in ("black", "white")'?
>
> Checkers seems popular around here. What if I want to change "white" to
> "red," as red and black is a common color scheme for checkers. Do I
> change a single constant variable or replace all the occurrences in the
> files?
>
Why so many files? Python can easily support thousands of lines in a file.
If it's just one file any text editor can do a quick find-replace.
That said, it's easy to make some global ``red = 'red'``.
Some of these constants are shortcuts. Instead of writing slice(0, 16)
> or slice(48, 64), and getting the two confused, I write
> const['board_bottom'] or const['board_top'], respectively, when I want
> to pull the correct set of positions from coordinates list.
>
Why hide these things in a dict ``const`` instead of just making them
top-level variables in the module themselves? ``board_top = slice(48,
64)``
> That said, if you're wanting to share constants across different parts
> > of your code, use a module.
>
Or just use one file to keep things easier. But, yes, I agree a module of
constants is appropriate for bigger projects.
--
https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 4/22/2016 1:40 PM, Michael Selik wrote: Frankly, for someone coming from Java, the best advice is to not write any classes until you must. Of course classes in Python are very useful. It's just that your Java habits are unnecessary and often counter-productive. I disagree. I wrote procedural scripts and translated old BASIC games into Python for two years. One day I came across a Python book that described the principles of subclassing from a base class for chess pieces, but there was no code to demonstrate the process. I had no problem creating the Python classes. That's how my research project got started to build a chess engine. If you ever check the academic literature for chess programming, this research project could turn into a lifelong endeavor. I had to confront all the bad habits I brought over Java and change my code to be more Pythonic. This is where I started having fun, learning the tricks and collapsing multi-line code into a single line code. I've learned more about Python in the few weeks than I had in two years of writing procedural scripts and translating BASIC goto hell. Thank you, Chris R. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 4/23/2016 2:33 PM, Matt Wheeler wrote: This is still backwards to me. It prevents your classes from being suitable for restoring a stored game state, not just custom starting positions (which I think is what Ethan means by custom setups). I haven't thought that far about saving the game state. I'm under the impression that pickle (or something else) would save and load the instantiated object of each piece. If that's not the case, I'll change the code then. The sanity check does have an exemption for pawn promotion, where a pawn reaching the other side of the board can be promoted to another piece (typically a queen). I played Sargon II chess on the Commodore 64 for five years as a teenager. The worst game I ever won was when the computer had nine queens on the board. I thought the computer was cheating outrageously but a chess rule book confirmed that promotion was a legit move. https://en.wikipedia.org/wiki/Promotion_(chess) If I need to add an exemption to custom plays, I'll add it then. Right now I'm cleaning up the existing code from all the bad Java habits. Thank you, Chris R. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 4/23/2016 6:29 PM, Ian Kelly wrote: Python enums are great. Sadly, they're still not quite as awesome as Java enums. I remember enums more from C than Java. Although I haven't used them much in either language. I'm planning to immerse myself back into C via Cython. Depending on far I get into this chess program, I might need a speed boost here and there. Thank you, Chris R. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 4/23/2016 6:38 PM, Michael Selik wrote: Why so many files? Python can easily support thousands of lines in a file. If it's just one file any text editor can do a quick find-replace. That goes back to the Java convention of having one class per file. It took a while to convince myself that mixing classes and functions in a single file was okay. Once I finished cleaning up the code for this round, I'll merge four files into one file (~500 lines) and have four files (main.py, display.py, engine.py and utility.py). Or just use one file to keep things easier. But, yes, I agree a module of constants is appropriate for bigger projects. That's the other problem I'm running into. Building a chess engine is a big project. This is probably bigger than the Java XML parser I built from scratch for a college project. I can't seem to find any information on how to build bigger programs. Community college taught me how to program, but it didn't teach me how to go beyond class inheritance. Thank you, Chris R. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 04/23/2016 07:45 PM, Christopher Reimer wrote: > I had to confront all the bad habits I brought over Java and change my > code to be more Pythonic. This is where I started having fun, learning > the tricks and collapsing multi-line code into a single line code. I've > learned more about Python in the few weeks than I had in two years of > writing procedural scripts and translating BASIC goto hell. Procedural programming does not necessarily mean BASIC-style goto hell. Not sure why you would think that. In fact that's not really what procedural programming is about. However, Mr. Selik wasn't advocating procedural programming at all. Not defining a class does not make your code precdural. But using classes does not mean your code is *not* procedural. If you are using an event-driven framework then I will say, yes your code is not procedural. There are many aspects to Pythonic programming, not just OOP. For example using modules to store shared state for your program components is very pythonic, rather than using classes. A module is kind of like a singleton instance, and still is object-oriented by the way (the module is an object). Sadly Java really messed up people by using classes as a namespace mechanism. That was quite a mistake. Really messed with people's expectations of OOP. I would say that pythonic programming involves defining classes when it's appropriate, and not doing so when something else will work just as well and be simpler. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 4/23/2016 7:34 PM, Michael Torrie wrote Procedural programming does not necessarily mean BASIC-style goto hell. Not sure why you would think that. In fact that's not really what procedural programming is about. I mentioned several messages back that I spent two years writing procedural scripts AND translating BASIC games into Python, which can be goto hell if the program has a drop-down structure that makes sense only from following the entry point of the goto statement (see link for example). http://www.atariarchives.org/basicgames/showpage.php?page=9 I try to follow the BASIC program structure as closely as possible, get it working in Python, and use pylint to make my code PEP8-compliant. Pylint frequently complains about exceeding a dozen if branches in the main function. I then create helper functions to reduce the if branches. Sometimes that makes the Python version either shorter or longer than the original BASIC program. Thank you, Chris R. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 04/23/2016 08:32 PM, Christopher Reimer wrote: > That's the other problem I'm running into. Building a chess engine is a > big project. This is probably bigger than the Java XML parser I built > from scratch for a college project. I can't seem to find any information > on how to build bigger programs. Community college taught me how to > program, but it didn't teach me how to go beyond class inheritance. The reason you weren't taught beyond class inheritance is because Java implements organization only through a class hierarchy. Whole generations of Java programmers think that program organization is through classes (a static main method means your code is procedural, by the way). Learn about Python namespaces and how to use them. I guess it just comes with experience, a lot of trial and a lot of error. And checking on how others are doing it. Take a look at examples from the Python standard library, as well as other projects. For example, Beautiful Soup. Not that you have to understand their code, but take a look at how they organize things. In my mind namespaces matter more than files as an organizational tool. Because namespaces are my primary organization, I will put multiple class definitions in the same file (if they belong in the same namespace), so they can be imported from the same module. If the classes and functions get more complicated, I can separate them into their own submodules and wrap them all up in one Python package (like a module but more flexible and has submodules). The nice thing about building with Python is you can start with one thing, like a module, and convert it to a package later on as needed. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 4/23/2016 8:19 PM, Michael Torrie wrote: The reason you weren't taught beyond class inheritance is because Java implements organization only through a class hierarchy. Whole generations of Java programmers think that program organization is through classes (a static main method means your code is procedural, by the way). I never wanted to learn Java in the first place. My community college couldn't afford to renew the Microsoft site license, which local employers required to learn C/C++ in MS Visual Studio, and all flavors of Java got taught for the programming classes instead. I wanted to learn C/C++. I even wanted to learn assembly language, but I was the only student who showed up for the class and it got cancelled. Of course, it probably doesn't help that I got a job in help desk support after I graduated with A.S. degree in Java and never programmed professionally. Thinking like a programmer helped me resolved many IT problems over the last 12 years. My current job in computer security requires tinkering with PowerShell scripts as Python is strictly prohibited in this Windows shop. I have made Python my main programming language at home. Thank you, Chris R. -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On 04/23/2016 09:41 PM, Christopher Reimer wrote: > I never wanted to learn Java in the first place. My community college > couldn't afford to renew the Microsoft site license, which local > employers required to learn C/C++ in MS Visual Studio, and all flavors > of Java got taught for the programming classes instead. I wanted to > learn C/C++. I even wanted to learn assembly language, but I was the > only student who showed up for the class and it got cancelled. > > Of course, it probably doesn't help that I got a job in help desk > support after I graduated with A.S. degree in Java and never programmed > professionally. Thinking like a programmer helped me resolved many IT > problems over the last 12 years. My current job in computer security > requires tinkering with PowerShell scripts as Python is strictly > prohibited in this Windows shop. I have made Python my main programming > language at home. I don't mean to imply that I'm criticizing you for your Java experience! I am critical of Java, though. I'm very glad you've discovered Python and I hope you'll continue to have fun with it. I hope you'll take the advice offered by the others on this thread in stride and hopefully we'll all learn and benefit. I completely agree with you that learning to think like a programmer is so helpful in solving all kinds of problems, especially in IT! -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On Sun, 24 Apr 2016 12:03 pm, Christopher Reimer wrote: > On 4/23/2016 2:33 PM, Matt Wheeler wrote: >> This is still backwards to me. It prevents your classes from being >> suitable for restoring a stored game state, not just custom starting >> positions (which I think is what Ethan means by custom setups). > > I haven't thought that far about saving the game state. I'm under the > impression that pickle (or something else) would save and load the > instantiated object of each piece. If that's not the case, I'll change > the code then. Pickle will do what you are thinking of, but pickle is not secure and involves executing arbitrary code. If you cannot trust the source of the pickle, then you should not use it. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On Sun, 24 Apr 2016 12:34 pm, Michael Torrie wrote: > There are many aspects to Pythonic programming, not just OOP. For > example using modules to store shared state for your program components > is very pythonic, rather than using classes. A module is kind of like a > singleton instance, and still is object-oriented by the way (the module > is an object). I find myself going backwards and forwards on whether or not that is a good idea. I think it depends on whether you are writing a library or an application. If you're writing an application, then storing state in module-level variables works fine. Your application is, effectively, a singleton, and if you try to run it twice, you'll (probably) be running it in two distinct processes, so that's okay. (If you're not, if somehow you perform some sort of trickery where you have a single Python interpreter running your script twice in the same process, then it will break horribly. But if you can do that, you're living on the edge already and can probably deal with it.) But if you're writing a library, then using module state is a Bad Idea. Your library may be used by more than one client in the same process, and they will conflict over each other's library-level state. "Global variables considered harmful." -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
Steven D'Aprano : > On Sun, 24 Apr 2016 12:03 pm, Christopher Reimer wrote: > >> On 4/23/2016 2:33 PM, Matt Wheeler wrote: >>> This is still backwards to me. It prevents your classes from being >>> suitable for restoring a stored game state, not just custom starting >>> positions (which I think is what Ethan means by custom setups). >> >> I haven't thought that far about saving the game state. I'm under the >> impression that pickle (or something else) would save and load the >> instantiated object of each piece. If that's not the case, I'll change >> the code then. > > Pickle will do what you are thinking of, but pickle is not secure and > involves executing arbitrary code. If you cannot trust the source of > the pickle, then you should not use it. You shouldn't use your runtime objects as a storage format. Instead, design the storage objects separately and translate between runtime and storage objects as needed. JSON objects or straight Python dicts are good candidates for overall storage format. JSON would allow for easy interchange between different programming languages if need be. Marko -- https://mail.python.org/mailman/listinfo/python-list
Optimizing Memory Allocation in a Simple, but Long Function
I have been writing a python script to explore Euler's Method of
approximating Euler's Number. I was hoping there might be a way to make
this process work faster, as for sufficiently large eulerSteps, the process
below becomes quite slow and sometimes memory intensive. I'm hoping someone
can give me some insight as to how to optimize these algorithms, or ways I
might decrease memory usage. I have been thinking about finding a way
around importing the math module, as it seems a bit unneeded except as an
easy reference.
## Write a method to approximate Euler's Number using Euler's Method
import math
class EulersNumber():
def __init__(self,n):
self.eulerSteps = n
self.e = self.EulersMethod(self.eulerSteps)
def linearApproximation(self,x,h,d): # f(x+h)=f(x)+h*f'(x)
return x + h * d
def EulersMethod(self, numberOfSteps): # Repeat linear approximation over
an even range
e = 1 # e**0 = 1
for step in range(numberOfSteps):
e = self.linearApproximation(e,1.0/numberOfSteps,e) # if f(x)= e**x,
f'(x)=f(x)
return e
def EulerStepWithGuess(accuracy,guessForN):
n = guessForN
e = EulersNumber(n)
while abs(e.e -math.e) > abs(accuracy):
n +=1
e = EulersNumber(n)
print('n={} \te= {} \tdelta(e)={}'.format(n,e.e,abs(e.e-math.e)))
return e
def EulersNumberToAccuracy(PowerOfTen):
x = 1
theGuess = 1
thisE = EulersNumber(1)
while x <= abs(PowerOfTen):
thisE = EulerStepWithGuess(10**(-1*x),theGuess)
theGuess = thisE.eulerSteps * 10
x += 1
return thisE
Thanks,
Derek
--
https://mail.python.org/mailman/listinfo/python-list
Re: How much sanity checking is required for function inputs?
On Sun, Apr 24, 2016, 1:51 AM Steven D'Aprano wrote: > On Sun, 24 Apr 2016 12:34 pm, Michael Torrie wrote: > > > There are many aspects to Pythonic programming, not just OOP. For > > example using modules to store shared state for your program components > > is very pythonic, rather than using classes. A module is kind of like a > > singleton instance, and still is object-oriented by the way (the module > > is an object). > > I find myself going backwards and forwards on whether or not that is a good > idea. I think it depends on whether you are writing a library or an > application. > > If you're writing an application, then storing state in module-level > variables works fine. Your application is, effectively, a singleton, and if > you try to run it twice, you'll (probably) be running it in two distinct > processes, so that's okay. > > (If you're not, if somehow you perform some sort of trickery where you have > a single Python interpreter running your script twice in the same process, > then it will break horribly. But if you can do that, you're living on the > edge already and can probably deal with it.) > > But if you're writing a library, then using module state is a Bad Idea. > Your > library may be used by more than one client in the same process, and they > will conflict over each other's library-level state. "Global variables > considered harmful." > I think we're giving mixed messages because we're conflating "constants" and globals that are expected to change. In our case here, I think two clients in the same process sharing state might be a feature rather than a bug. Or at least it has the same behavior as the current implementation. > -- https://mail.python.org/mailman/listinfo/python-list
