Re: __main__ vs official module name: distinct module instances
On Sun, 2 Aug 2015 01:53 pm, Cameron Simpson wrote:
> Hi All,
>
> Maybe this should be over in python-ideas, since there is a proposal down
> the bottom of this message. But first the background...
>
> I've just wasted a silly amount of time debugging an issue that really I
> know about, but had forgotten.
:-)
> I have a number of modules which include a main() function, and down the
> bottom this code:
>
> if __name__ == '__main__':
> sys.exit(main(sys.argv))
>
> so that I have a convenient command line tool if I invoke the module
> directly. I typically have tiny shell wrappers like this:
>
> #!/bin/sh
> exec python -m cs.app.maildb -- ${1+"$@"}
I know this isn't really relevant to your problem, but why use "exec python"
instead of just "python"?
And can you explain the -- ${1+"$@"} bit for somebody who knows just enough
sh to know that it looks useful but not enough to know exactly what it
does?
> In short, invoke this module as a main program, passing in the command
> line arguments. Very useful.
>
> My problem?
>
> When invoked this way, the module cs.app.maildb that is being executed is
> actually the module named "__main__".
Yep. Now, what you could do in cs.app.maildb is this:
# untested, but should work
if __name__ = '__main__':
import sys
sys.modules['cs.app.maildb'] = sys.modules[__name__]
sys.exit(main(sys.argv))
*** but that's the wrong solution ***
The problem here is that by the time cs.app.maildb runs, some other part of
cs or cs.app may have already imported it. The trick of setting the module
object under both names can only work if you can guarantee to run this
before importing anything that does a circular import of cs.app.maildb.
The right existing solution is to avoid having the same module do
double-duty as both runnable script and importable module. In a package,
that's easy. Here's your package structure:
cs
+-- __init__.py
+-- app
+-- __init__.py
+-- mailbd.py
and possibly others. Every module that you want to be a runnable script
becomes a submodule with a __main__.py file:
cs
+-- __init__.py
+-- __main__.py
+-- app
+-- __init__.py
+-- __main__.py
+-- mailbd
+-- __init__.py
+-- __mail__.py
and now you can call:
python -m cs
python -m cs.app
python -m cs.app.mailbd
as needed. The __main__.py files look like this:
if __name__ = '__main__':
import cs.app.maildb
sys.exit(cs.app.maildb.main(sys.argv))
or as appropriate.
Yes, it's a bit more work. If your package has 30 modules, and every one is
runnable, that's a lot more work. But if your package is that, um,
intricate, then perhaps it needs a redesign?
The major use-case for this feature is where you have a package, and you
want it to have a single entry point when running it as a script. (That
would be "python -m cs" in the example above.) But it can be used when you
have multiple entry points too.
For a single .py file, you can usually assume that when you are running it
as a stand alone script, there are no circular imports of itself:
# spam.py
import eggs
if __name__ == '__main__':
main()
# eggs.py
import spam # circular import
If that expectation is violated, then you can run into the trouble you
already did.
So...
* you can safely combine importable module and runnable script in
the one file, provided the runnable script functionality doesn't
depend on importing itself under the original name (either
directly or indirectly);
* if you must violate that expectation, the safest solution is to
make the module a package with a __main__.py file that contains
the runnable script portion;
* if you don't wish to do that, you're screwed, and I think that the
best you can do is program defensively by detecting the problem
after the event and bailing out:
# untested
import __main__
import myactualfilename
if os.path.samefile(__main__.__path__, myactualfilename.__path__):
raise RuntimeError
--
Steven
--
https://mail.python.org/mailman/listinfo/python-list
Re: GvR Europython keynote described on lwn.net
On Sun, 2 Aug 2015 04:57 am, Rick Johnson wrote: > On Saturday, August 1, 2015 at 12:45:45 AM UTC-5, Steven D'Aprano wrote: > >> > Yep, even the BDFL is actively developing in 2.7! He's no fool. >> >> Of course not. Dropbox pay him to work on their systems, >> and he wants to keep his job. > > Thanks for confirming my point that Python3 is not worth > developing with for at least the next five years. If Dropbox were using Python 1.5, would you conclude that Python 2 was not worth developing in? I happen to know that at last year's US PyCon there was at least one company still using Python 1.5. If it works for them, and they don't need security updates, why not? But this doesn't mean others should emulate them. As Laura has explained, there is at least one sector of Python users that not only doesn't want new features, but they don't want bug fixes either. They would rather work around bugs themselves, and stability is more important than correctness. If that works for them, great. But it doesn't work for everyone. Just because company X is using 2.7, why does that mean that *you* shouldn't using 3.x? Surely you should make your own decision, based on your own needs. (For the record, Dropbox isn't using Python 2.7. They're using a heavily customized private implementation of Python based on, but not the same as, version 2.7. Whatever benefits they get from using that, I can promise that *you* will not be getting them from the vanilla version of 2.7 available to the public.) >> Are you aware that Dropbox are heavily pushing for static >> type hints in Python 3 as a prerequisite for them porting >> their masses of Python 2 code to Python 3? > > Well then, i hope they are ready to wait at least 10 years > before adopting Python3, because it will take that long to > work out all the kinks! Nonsense. You can already download mypy and start using static type checking in Python today. > Of course, with the cloud service > wars heating up, no one can be sure how long any of them > will survive. Web technology is moving much faster than > Python. > >> That's one of the motives for the masses of effort put >> into PEP 484, and its support PEPs, 482 and 483: > > I do find it flattering that many of my ideas regarding > Python have been implemented: (1) It was me who recommended > "optional type checking" way back around 2008 Don't flatter yourself. People have been suggesting type checking for Python since Python 0.1. > (Heck, you > even agreed that it would be a good idea, but at the time, a > moratorium was preventing new features) (2) The fresher look > of Python.org is a result of my suggestions So you're the one to blame for the horrible new design and the reliance on Javascript to create an overall worse user experience? I'd call the new design a triumph of style over substance, except that the new style is worse than the old one. > (3) The > interactive online console was my idea to compete with the > Ruby equivalent (4) I have pestered tutorial owners to > upgrade their tutorials to py3 compatibility, and many did! > (5) and last but not least, my courage to face down the > trolls has given courage to the shadow lurkers, who now > participate in open discussions on this list, and some have > even moved over to more dangerous grounds like Python-ideas. So you're the one to blame for all the bike-shedding from people who think that copying design principles from PHP is a great idea? Please stop "helping". -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: __main__ vs official module name: distinct module instances
On Sun, Aug 2, 2015 at 5:41 PM, Steven D'Aprano wrote: > * if you don't wish to do that, you're screwed, and I think that the > best you can do is program defensively by detecting the problem > after the event and bailing out: > > # untested > import __main__ > import myactualfilename > if os.path.samefile(__main__.__path__, myactualfilename.__path__): > raise RuntimeError Not sure what __path__ is here, as most of the things in my sys.modules don't have it; do you mean __file__? In theory, it should be possible to skim across sys.modules, looking for a match against __main__, and raising RuntimeError if any is found. -- https://mail.python.org/mailman/listinfo/python-list
Re: __main__ vs official module name: distinct module instances
On Sun, Aug 2, 2015 at 6:16 PM, Chris Angelico wrote: > On Sun, Aug 2, 2015 at 5:41 PM, Steven D'Aprano wrote: >> * if you don't wish to do that, you're screwed, and I think that the >> best you can do is program defensively by detecting the problem >> after the event and bailing out: >> >> # untested >> import __main__ >> import myactualfilename >> if os.path.samefile(__main__.__path__, myactualfilename.__path__): >> raise RuntimeError > > Not sure what __path__ is here, as most of the things in my > sys.modules don't have it; do you mean __file__? In theory, it should > be possible to skim across sys.modules, looking for a match against > __main__, and raising RuntimeError if any is found. Oops, premature send. *In theory* it should be possible to do the above, but whichever attribute you look for, some modules may not have it. How does this play with, for instance, zipimport, where there's no actual file name for the module? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Most pythonic way of rotating a circular list to a canonical point
On Sun, 2 Aug 2015 08:51 am, Lukas Barth wrote:
> On Saturday, August 1, 2015 at 11:37:48 PM UTC+2, Emile van Sebille wrote:
>> Well, it looks to me that I don't know what a 'canonical rotation' is --
>
> That's because it is not defined. ;)
>
> I need a way to rotate one of these lists in a way so that it will produce
> the same output every time, regardless of what the input rotation was.
I'm not convinced that you necessarily do, but for the sake of the argument
suppose you do...
> Example:
>
> [0,1,2,3,4] => [0,1,2,3,4]
> [2,3,4,0,1] => [0,1,2,3,4]
> [3,4,0,1,2] => [0,1,2,3,4]
> ...
How is this different from sorted(the_list)?
Ah, wait, I've just answered my own question...
[0,1,2,4,3] != [0,1,2,3,4]
> It doesn't have to be "[0,1,2,3,4]", it can just as well be [2,3,4,1,0],
> as long as it's always the same.
Keep a cache, and intern the first seen version of the list in the cache.
CACHE = {}
def intern(alist):
t = MyTuple(alist)
if t not in CACHE:
CACHE[t] = alist
return CACHE[t]
# Untested
class MyTuple(tuple):
def __eq__(self, other):
if self is other: return True
if not isinstance(other, MyTuple): return NotImplemented
if len(self) != len(other): return False
d = self+self
return other in d
def __hash__(self):
return hash(frozenset(self))
--
Steven
--
https://mail.python.org/mailman/listinfo/python-list
Most Pythonic way to store (small) configuration
There are a lot of ways to store configuration information: - conf file - xml file - database - json file - and possible a lot of other ways I want to write a Python program to display cleaned log files. I do not think I need a lot of configuration to be stored: - some things relating to the GUI - default behaviour - default directory - log files to display, including some info - At least until where it was displayed Because of this I think a human readable file would be best. Personally I do not find XML very readable. So a conf or json file looks the most promising to me. And I would have a slight preference for a json file. Any comments, thoughts or tips? -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Re: Most Pythonic way to store (small) configuration
On Sun, Aug 2, 2015 at 8:11 PM, Cecil Westerhof wrote:
> Because of this I think a human readable file would be best.
> Personally I do not find XML very readable. So a conf or json file
> looks the most promising to me. And I would have a slight preference
> for a json file.
>
> Any comments, thoughts or tips?
I'd agree with your analysis. XML is not readable; the two best
options would be JSON, if you need the potential for deep structure,
or the simple config file ("INI file") if you don't.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Most pythonic way of rotating a circular list to a canonical point
On Saturday, August 1, 2015 at 1:34:44 PM UTC-7, Lukas Barth wrote: > Hi! > > I have a list of numbers that I treat as "circular", i.e. [1,2,3] and [2,3,1] > should be the same. Now I want to rotate these to a well defined status, so > that I can can compare them. > > If all elements are unique, the solution is easy: find the minimum element, > find its index, then use mylist[:index] + mylist[index:], i.e. the minimum > element will always be at the beginning. > > But say I have [0,1,0,2,0,3]. I can in fact guarantee that no *pair* will > appear twice in that list, i.e. I could search for the minimum, if that is > unique go on as above, otherwise find *all* positions of the minimum, see > which is followed by the smallest element, and then rotate that position to > the front. > > Now that seems an awful lot of code for a (seemingly?) simple problem. Is > there a nice, pythonic way to do this? > > Thanks for enlightening me! [[[Warning: all code untested, sorry!]]] The general problem (no assumptions about the items in the list except that they're all sortable together) is interesting. It doesn't seem like a simple one to me at all. An acceptable criterion would be if you took the lexically smallest value of every rotation of the list; that would yield the same result for every initial rotation. You can do it in like this: anchor = min(range(len(X)),key=lambda i:X[i:]+X[:i]) canonical_X = X[anchor:]+X[:anchor] I doubt this would be that efficient, though, since it happens to construct every single rotation in the process of scanning. However, you'll notice that the minimum lexical value can only occur when the starting point is the minimum value in the list, so you can almost certainly improve performace to find all the minimum values and only test those rotations, especially if the number of minima is small compared to the list size. (However, be sure to timeit for actual evidence rather than my guessing.) min_value = min(X) i = X.index(min_value) while True: start_points.append(i) try: i = X.index(min_value,i+1) except ValueError: break anchor = min(start_points,key=lambda i:X[i:]+X[:i]) canonical_X = X[anchor:]+X[:anchor] Your lists aren't arbitrary, however and they actually seem to be ordered pairs (in which case the first question I have is why not have it be a list of 2-tuples? but I can imagine a few reasons). You say the pairs are known to be unique, so all you have to do is find the index of the minimum pair. One can use zip and slicing to generate 2-tuples from the flat list, whence you can find the minimum. pairs = list(zip(X[0::2],X[1::2])) min_pair = min(pairs) anchor = 2 * pairs.index(min_pair) canonical_X = X[anchor:]+X[:anchor] Carl Banks -- https://mail.python.org/mailman/listinfo/python-list
Re: Most Pythonic way to store (small) configuration
Cecil Westerhof writes: > Because of this I think a human readable file would be best. I agree with that criterion; in the absence of compelling reasons otherwise, human-readable and -editable text is a good default. > Personally I do not find XML very readable. So a conf or json file > looks the most promising to me. And I would have a slight preference > for a json file. XML and JSON should both be considered data serialisation formats only. JSON is human-readable to an extent, but it is quite brittle, and there are no comments permitted in the syntax. So, both XML and JSON should be considered write-only, and produced only for consumption by a computer; they are a poor choice for presenting to a human. The “INI” format as handled by the Python ‘configparser’ module is what I would recommend for a simple flat configuration file. It is more intuitive to edit, and has a conventional commenting format. If it were more complex I might recommend YAML, which is a very widespread, flexible hierarchical format, that still allows fairly natural editing and commenting. Its main downside (as opposed to INI format) is that the Python standard library does not support it. -- \ “Simplicity and elegance are unpopular because they require | `\ hard work and discipline to achieve and education to be | _o__)appreciated.” —Edsger W. Dijkstra | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Most Pythonic way to store (small) configuration
On 02/08/2015 12:54, Ben Finney wrote: Cecil Westerhof writes: Because of this I think a human readable file would be best. The “INI” format as handled by the Python ‘configparser’ module is what I would recommend for a simple flat configuration file. It is more intuitive to edit, and has a conventional commenting format. +1 -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Most Pythonic way to store (small) configuration
On Sunday 2 Aug 2015 13:54 CEST, Ben Finney wrote: > Cecil Westerhof writes: > >> Because of this I think a human readable file would be best. > > I agree with that criterion; in the absence of compelling reasons > otherwise, human-readable and -editable text is a good default. > >> Personally I do not find XML very readable. So a conf or json file >> looks the most promising to me. And I would have a slight >> preference for a json file. > > XML and JSON should both be considered data serialisation formats > only. > > JSON is human-readable to an extent, but it is quite brittle, and > there are no comments permitted in the syntax. > > So, both XML and JSON should be considered write-only, and produced > only for consumption by a computer; they are a poor choice for > presenting to a human. > > The “INI” format as handled by the Python ‘configparser’ module is > what I would recommend for a simple flat configuration file. It is > more intuitive to edit, and has a conventional commenting format. Well, I would use nested data. (A file will have extra fields besides the name.) That is why I was thinking about json. But I will look into it. -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Re: Most pythonic way of rotating a circular list to a canonical point
On 2015-08-01 13:34, Lukas Barth wrote: > I have a list of numbers that I treat as "circular", i.e. [1,2,3] > and [2,3,1] should be the same. Now I want to rotate these to a > well defined status, so that I can can compare them. > > If all elements are unique, the solution is easy: find the minimum > element, find its index, then use mylist[:index] + mylist[index:], > i.e. the minimum element will always be at the beginning. > > But say I have [0,1,0,2,0,3]. I can in fact guarantee that no > *pair* will appear twice in that list, i.e. I could search for the > minimum, if that is unique go on as above, otherwise find *all* > positions of the minimum, see which is followed by the smallest > element, and then rotate that position to the front. Well, you can pull the minimum (or maximum) of all the rotations to get the "canonical" version: def canonical(lst): """return the "canonical" representation of a list""" return min( lst if i == 0 else (lst[i:] + lst[:i]) for i in range(len(lst)) ) which you can determine, then hash once, and store. It's not a cheap operation, but once you've determined the canonical/hash version, then equality-testing becomes an O(1) test if comparing hashes, or O(N) if comparing lists rather than an O(N*2) brute-force test (which could be lower depending on the commonality). def circular_compare1(a, b): if a == b: return True return any(a == (b[i:] + b[:i]) for i in range(1, len(b))) def circular_compare2(a, b): lena = len(a) if lena != len(b): return False return any( all(a[i] == b[(i + offset) % lena] for i in range(lena)) for offset in range(lena) ) for fn in (circular_compare1, circular_compare2): for (a, b), expected in ( (([1,2,3], [1,2,3]), True), # the same (([1,2,3], [3,1,2]), True), # simple rotation (([1,2,3], [1,2,3,4]), False), # mismatched lengths (([0,1,0,2,0,3], [0,2,0,3,0,1]), True), # repeated elements, simple rotation ): result = bool(fn(a, b)) if result == expected: print "PASS %s %r/%r=%s, not %s" % ( fn.__name__, a, b, result, expected ) else: print "FAIL %s %r/%r=%s, not %s" % ( fn.__name__, a, b, result, expected ) ca = canonical(a) cb = canonical(b) print " Canonical A:", ca print " Canonical B:", cb print " Equal:", (ca == cb) -tim -- https://mail.python.org/mailman/listinfo/python-list
Re: Most Pythonic way to store (small) configuration
Cecil Westerhof writes: > Well, I would use nested data. (A file will have extra fields besides > the name.) That is why I was thinking about json. But I will look into > it. An alternative, very similar to JSON but with some good cherries picked from YAML is AXON, which is my preferite these days for this kind of things. http://intellimath.bitbucket.org/axon/ ciao, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. l...@metapensiero.it | -- Fortunato Depero, 1929. -- https://mail.python.org/mailman/listinfo/python-list
Re: Most pythonic way of rotating a circular list to a canonical point
On 08/01/2015 01:34 PM, Lukas Barth wrote: Hi! I have a list of numbers that I treat as "circular", i.e. [1,2,3] and [2,3,1] should be the same. Now I want to rotate these to a well defined status, so that I can can compare them. If all elements are unique, the solution is easy: find the minimum element, find its index, then use mylist[:index] + mylist[index:], i.e. the minimum element will always be at the beginning. But say I have [0,1,0,2,0,3]. I can in fact guarantee that no *pair* will appear twice in that list, i.e. I could search for the minimum, if that is unique go on as above, otherwise find *all* positions of the minimum, see which is followed by the smallest element, and then rotate that position to the front. Now that seems an awful lot of code for a (seemingly?) simple problem. Is there a nice, pythonic way to do this? Thanks for enlightening me! Lukas Let me try to re-state what I see is your actual problem. (Of course I could be wrong...) ;-) Most of the answers you have been getting concentrate on comparisons and hashes, but those are irrelevant to your need -- they come later. What you are trying to do is to uniquely and independently determine the "starting point" for the rotation. That is, which element will be rotated to index 0. Using the minimum is possible, but that fails if that minimum appears more than once in the list -- it does not give a unique solution. For a specific example... if you are given any _one_ of these lists: [1, 5, 0, 3, 0], [5, 0, 3, 0, 1], [0, 3, 0, 1, 5], [3, 0, 1, 5, 0], [0, 1, 5, 0, 3] it can be _independently_ rotated to (for example) [3, 0, 1, 5, 0] That example does not use the minimum as the starting point, but how the starting point can be absolutely and uniquely determined is your fundamental problem. I do not have an answer for that. But I thought this might be a clearer description of what I think you are really looking for. I hope I'm right. ;-) Good luck. -=- Larry -=- -- https://mail.python.org/mailman/listinfo/python-list
Re: GvR Europython keynote described on lwn.net
On Sunday, August 2, 2015 at 3:05:09 AM UTC-5, Steven D'Aprano wrote: > If Dropbox were using Python 1.5, would you conclude that > Python 2 was not worth developing in? No, if Dropbox were using py1.5, i would conclude that it was being managed by monkeys -- since Py1.5 existed before Dropbox was even founded (in 2007). > I happen to know that at last year's US PyCon there was at > least one company still using Python 1.5. If it works for > them, and they don't need security updates, why not? But > this doesn't mean others should emulate them. Same answer, if they were using py1.5 "when it was hot", and it still works for them today, why bother with backwards incompatibility. They can hack 1.5 if they need to, it's open source after all! > As Laura has explained, there is at least one sector of > Python users that not only doesn't want new features, but > they don't want bug fixes either. They would rather work > around bugs themselves, and stability is more important > than correctness. If that works for them, great. But it > doesn't work for everyone. Does the word "reiterate" mean anything to you? > Just because company X is using 2.7, why does that mean > that *you* shouldn't using 3.x? Surely you should make > your own decision, based on your own needs. It's not just *ANY* company Steven, it's Guido's freaking employer! That would imply that even GvR himself is not motivated enough by 3000 to fight for it's adoption. More evidence that py3000 is not ready for mass consumption. > (For the record, Dropbox isn't using Python 2.7. They're > using a heavily customized private implementation of > Python based on, but not the same as, version 2.7. > Whatever benefits they get from using that, I can promise > that *you* will not be getting them from the vanilla > version of 2.7 available to the public.) So what? If i had to guess, i would guess that the hacks are mostly to bring py3000 features to 2.7 without suffering the ill effects that the py3000 recipe of: "excessive backwards incompatibility spicing" has had on the consumption of py3000. We don't like your spicy sauce, Swedish Chef! https://www.youtube.com/watch?v=SGZPlvbhMIg And if you folks think it burns at runtime, just wait until the exceptions are ejected! -- https://mail.python.org/mailman/listinfo/python-list
Re: Most Pythonic way to store (small) configuration
On 02Aug2015 18:51, Cecil Westerhof wrote:
On Sunday 2 Aug 2015 13:54 CEST, Ben Finney wrote:
Cecil Westerhof writes:
Because of this I think a human readable file would be best.
I agree with that criterion; in the absence of compelling reasons
otherwise, human-readable and -editable text is a good default.
[...]
The “INI” format as handled by the Python ‘configparser’ module is
what I would recommend for a simple flat configuration file. It is
more intuitive to edit, and has a conventional commenting format.
Well, I would use nested data. (A file will have extra fields besides
the name.) That is why I was thinking about json. But I will look into
it.
Like others, I also recommend an INI file. You can always move to something
more complex (and harder to edit) if it doesn't work out.
Note that "nested data" does not rule out an INI file unless you mean
"recursive" data or quite a deep nesting of structure. Shallow structure (one
or two levels) is very easy to embed in the INI format.
Eg:
[main]
this = that
size = 10
directory = /path/to/default/dir
[gui]
foreground = green
style = baroque
[*.log]
stuff about log files...
[/path/to/file1]
title = something
param1 = 5
param2 = 6
fields = column2, column7
[column2:/path/to/file1]
format = numeric
scale = logarithmic
[column7:/path/to/file1]
format = roman
scale = linear
See that by being a little tricky about the section names you can incorporate a
fair degree of structure? I would not advocating going too fair down that
rabbit hole, but for basic stuff it works well.
And of course you can put structure within a section:
[foo]
complexity = {'blah': 'blah blah', 'frib': {'frob': 1, 'frab': 2}}
format_width = 10
format_style = right-justified
format_dialect = quoted
i.e. you could put (small) JSON snippet in some fields, or present a few
parameters to describe "format".
There are several ways to do this kind of thing before reaching for more
complex syntaxes. Just don't let it get too weird - that would probably be a
sign you do want to reach for JSON or YAML.
But start with INI. Simple and easy.
Cheers,
Cameron Simpson
A squealing tire is a happy tire.
- Bruce MacInnes, Skip Barber Driving School instructor
--
https://mail.python.org/mailman/listinfo/python-list
Re: GvR Europython keynote described on lwn.net
On Mon, Aug 3, 2015 at 8:34 AM, Rick Johnson wrote: > On Sunday, August 2, 2015 at 3:05:09 AM UTC-5, Steven D'Aprano wrote: > >> If Dropbox were using Python 1.5, would you conclude that >> Python 2 was not worth developing in? > > No, if Dropbox were using py1.5, i would conclude that it was > being managed by monkeys -- since Py1.5 existed before > Dropbox was even founded (in 2007). I have a git repository on my hard disk with commits dating back to August 1995. That's clearly being managed by monkeys, because git repositories can't exist before git was founded (2005), right? Well, actually, this particular repo was started in CVS, then imported from there into SVN, and thence into git more recently. Suppose Dropbox (the company) inherited a codebase from an older company, which itself inherited it from someone else - maybe they could be all set up with a codebase that pre-existed them by a decade. >> Just because company X is using 2.7, why does that mean >> that *you* shouldn't using 3.x? Surely you should make >> your own decision, based on your own needs. > > It's not just *ANY* company Steven, it's Guido's freaking > employer! That would imply that even GvR himself is not > motivated enough by 3000 to fight for it's adoption. More > evidence that py3000 is not ready for mass consumption. Wind the clock back to 2012, when Guido was working for Google. Dropbox wants him. Is he going to refuse the job unless they *first* get onto Py3, or is he going to accept the job with a view to migrating them? The only form of "fight[ing] for it[']s adoption" that you seem to be advocating here is an rms-style "if it isn't what I believe in, let it sink like the Titanic". That's not the only way to encourage something. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: GvR Europython keynote described on lwn.net
On Sunday, August 2, 2015 at 6:25:37 PM UTC-5, Chris Angelico wrote: > Wind the clock back to 2012, when Guido was working for Google. > Dropbox wants him. Is he going to refuse the job unless they *first* > get onto Py3, or is he going to accept the job with a view to > migrating them? Well, i don't know the specifics of why he departed from a global and mature company for a small (although promising) startup, but i do know that there is always the possibility that Google will sweep in and consume Dropbox if it becomes a threat or shows *real* potential. So there is always the possibility that he could be working at Google again; albeit in a "field office". Don't play coy Chris, you know damn good and well how this game is played, and you know that the founders of any startup dream of a buyout so they can sail off into the proverbial sunset. GvR is a pawn in a greater game. But we are all pawns in one form or another; are we not? He is but a small part of an industrious and creative young enterprise that will quickly become consumed by one of the powerful corporations, and for no other reason than to smother its competition in the cradle! Large corporations are where innovations go to die. Where spontaneity is slowly tortured to death inside the confining walls of a cubicle, and magnificent intellectual achievements are converted into cheap, plastic, soulless, *TRINKETS* that are purposefully designed for early failure in order to achieve maximum profitability! -- https://mail.python.org/mailman/listinfo/python-list
Python 3 May Become Relevant Now
rr should have a field day with this one http://nafiulis.me/python-3-may-become-relevant-now.html -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: __main__ vs official module name: distinct module instances
On 02Aug2015 17:41, Steven D'Aprano wrote:
On Sun, 2 Aug 2015 01:53 pm, Cameron Simpson wrote:
Maybe this should be over in python-ideas, since there is a proposal down
the bottom of this message. But first the background...
I've just wasted a silly amount of time debugging an issue that really I
know about, but had forgotten.
:-)
I have a number of modules which include a main() function, and down the
bottom this code:
if __name__ == '__main__':
sys.exit(main(sys.argv))
so that I have a convenient command line tool if I invoke the module
directly. I typically have tiny shell wrappers like this:
#!/bin/sh
exec python -m cs.app.maildb -- ${1+"$@"}
TL;DR: pertinent discussion around my proposal is lower down. First I digress
into Steven's shell query.
I know this isn't really relevant to your problem, but why use "exec python"
instead of just "python"?
Saves a process. Who needs a shell process just hanging around waiting? Think
of this as tail recursion optimisation.
And can you explain the -- ${1+"$@"} bit for somebody who knows just enough
sh to know that it looks useful but not enough to know exactly what it
does?
Ah.
In a modern shell one can just write $@. I prefer portable code.
The more complicated version, which I use everywhere because it is portable,
has to do with the behaviour of the $@ special variable. As you know, $* is the
command line arguments as a single string, which is useless if you need to
preserve them intact. "$@" is the command line arguments correctly quoted.
Unlike every other "$foo" variable, which produces a single string, "$@"
produces all the command line arguments as separate strings. Critical for
passing them correctly to other commands. HOWEVER, if there are no arguments
then "$@" produces a single empty string. Not desired. It is either a very old
bug or a deliberate decision that no "$foo" shall utterly vanish.
Thus this:
${1+"$@"}
Consulting your nearest "man sh" in the PARAMETER SUBSTITUION section you will
see that this only inserts "$@" if there is at least one argument, avoiding the
"$@" => "" with no arguments. It does this by only inserting "$@" if $1 is
defined. Sneaky and reliable.
I believe in a modern shell a _bare_ $@ acts like a correctly behaving "$@"
should have, but I always use the incantation above for portability.
In short, invoke this module as a main program, passing in the command
line arguments. Very useful.
My problem?
When invoked this way, the module cs.app.maildb that is being executed is
actually the module named "__main__".
Yep. Now, what you could do in cs.app.maildb is this:
# untested, but should work
if __name__ = '__main__':
import sys
sys.modules['cs.app.maildb'] = sys.modules[__name__]
sys.exit(main(sys.argv))
Yes, but that is ghastly and complicated. And also relies on the boiler plate
at the bottom knowing the module name.
*** but that's the wrong solution ***
It is suboptimal. "Wrong" seems a stretch.
The problem here is that by the time cs.app.maildb runs, some other part of
cs or cs.app may have already imported it. The trick of setting the module
object under both names can only work if you can guarantee to run this
before importing anything that does a circular import of cs.app.maildb.
That can be done if it takes place in the python interpreter. But there are
side effects which need to be considered.
My initial objective is that:
python -m cs.app.maildb
should import cs.app.maildb under the supplied name instead of "__main__" so
that a recursive import did not instantiate a second module instance. That is,
I think, a natural thing for users to expect from the above command line:
"import cs.app.maildb, run its main program".
On further thought last night I devised the logic below to implement python's
"-m" option:
# pseudocode, with values hardwired for clarity
import sys
M = new_empty_module(name='__main__', qualname='cs.app.maildb')
sys.modules['cs.app.maildb'] = M
M.execfile('/path/to/cs/app/maildb.py') # you know what I mean...
The "qualname" above is an idea I thought of last night to allow introspection
to cope with '__main__' and 'cs.app.maildb' at the same time, somewhat like the
.__qualname__ attribute of a function as recently added to the language; under
this scheme a module would get a __name__ and a __qualname__, normally the
same, but __name__ set to '__main__' for the "main program module situation.
This should sidestep any issues with recursive imports by having the module in
place in sys.modules ahead of the running of its code.
The right existing solution is to avoid having the same module do
double-duty as both runnable script and importable module.
I disagree. Supporting this double duty is, to me, a highly desirable feature.
This is, in fact, a primary purpose of the present standard boilerplate.
I _like_ that: a single file, short and succinct.
In a package,
that's easy. Here's you
Re: __main__ vs official module name: distinct module instances
On 02Aug2015 18:18, Chris Angelico wrote: On Sun, Aug 2, 2015 at 6:16 PM, Chris Angelico wrote: On Sun, Aug 2, 2015 at 5:41 PM, Steven D'Aprano wrote: * if you don't wish to do that, you're screwed, and I think that the best you can do is program defensively by detecting the problem after the event and bailing out: # untested import __main__ import myactualfilename if os.path.samefile(__main__.__path__, myactualfilename.__path__): raise RuntimeError Not sure what __path__ is here, as most of the things in my sys.modules don't have it; do you mean __file__? In theory, it should be possible to skim across sys.modules, looking for a match against __main__, and raising RuntimeError if any is found. Oops, premature send. *In theory* it should be possible to do the above, but whichever attribute you look for, some modules may not have it. How does this play with, for instance, zipimport, where there's no actual file name for the module? To my eyes, badly, which IMO strengthens my case for addressing the situation in the interpreter instead of requiring increasingly complex gyrations on the part of every programmer on the planet:-) Cheers, Cameron Simpson The CBR and ZXR should come with a warning sticker that says 'You are not Mick Doohan, do NOT be a prat' - UK's _BIKE_ magazine -- https://mail.python.org/mailman/listinfo/python-list
Re: Most Pythonic way to store (small) configuration
Cecil Westerhof writes: > On Sunday 2 Aug 2015 13:54 CEST, Ben Finney wrote: > > > So, both XML and JSON should be considered write-only, and produced > > only for consumption by a computer; they are a poor choice for > > presenting to a human. > > Well, I would use nested data. (A file will have extra fields besides > the name.) In that case, your needs are more complex than “store some simple configuration”. You need a human-editable format (so, not JSON and not XML) which handles structured data well, and is widely implemented. For nested configuration data, you would be better served by YAML https://en.wikipedia.org/wiki/YAML>, and the PyYAML library https://pypi.python.org/pypi/PyYAML/> is the one to use. -- \學而不思則罔，思而不學則殆。 | `\(To study and not think is a waste. To think and not study | _o__) is dangerous.) —孔夫子 Confucius (551 BCE – 479 BCE) | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: GvR Europython keynote described on lwn.net
On Mon, 3 Aug 2015 08:34 am, Rick Johnson wrote: >> Just because company X is using 2.7, why does that mean >> that *you* shouldn't using 3.x? Surely you should make >> your own decision, based on your own needs. > > It's not just *ANY* company Steven, it's Guido's freaking > employer! That would imply that even GvR himself is not > motivated enough by 3000 to fight for it's adoption. More > evidence that py3000 is not ready for mass consumption. No, it's evidence that *Dropbox* is not ready for Python 3.x. It tells you ABSOLUTELY NOTHING about the suitability of Python 3.x for other companies and other users. Despite your earlier snarky comment about "reiterate", it is clear that you still don't get it, so I'll say it again. There is nothing wrong with individuals or companies choosing to stay with Python 2.7, or 2.6, or 2.5, or 1.5, if it meets their needs. GvR doesn't need to "fight" for Python 3's adoption. He works for a company where, *right now*, a customised version of 2.7 meets their needs. That doesn't mean that other companies aren't ready for 3.x, and it doesn't mean that Dropbox aren't preparing for 3.x. They are. You might think that they could just turn around tomorrow and say, "Right, as of tomorrow we're using Python 3 for new projects", but that's not how it works when you're a company running Python on hundreds, maybe thousands of production servers, all of which will need to have Python replaced *while running live* before the change over. >> (For the record, Dropbox isn't using Python 2.7. They're >> using a heavily customized private implementation of >> Python based on, but not the same as, version 2.7. >> Whatever benefits they get from using that, I can promise >> that *you* will not be getting them from the vanilla >> version of 2.7 available to the public.) > > So what? If i had to guess, i would guess that the hacks are > mostly to bring py3000 features to 2.7 These would be the features of Python 3 that nobody needs and nobody wants because Python 2 is good enough for everyone? I wouldn't want to guess what the customizations do. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Most Pythonic way to store (small) configuration
On 2015-08-02 21:54, Ben Finney wrote:
> So, both XML and JSON should be considered write-only, and produced
> only for consumption by a computer; they are a poor choice for
> presenting to a human.
>
> The “INI” format as handled by the Python ‘configparser’ module is
> what I would recommend for a simple flat configuration file. It is
> more intuitive to edit, and has a conventional commenting format.
I second Ben's thoughts against XML & JSON -- they *can* be edited by
hand, but put the onus on the user to make perfect XML/JSON. Config
files (".ini") are more forgiving.
However, the .ini format (or at least the stdlib implementation in
ConfigParser.py) is not without its faults, mostly when you read a
file, then write it back out:
- comments and blank lines get lost in the process:
[section]
# set to local configuration
location=path/to/foo
will get written out as
[section]
location=path/to/foo
- the order of options is not preserved:
[section]
thing=1
other=2
may get written back out as
[section]
other=2
thing=1
though this has improved once ConfigParser started attempting to
use an OrderedDict by default for internal storage.
- a single key can only appear once in a section:
[section]
option=one
option=two
gets written back out as
[section]
option=two
- implicit encoding (is it UTF-8, Latin-1, etc?)
When you understand that the underlying internal storage is a dict
(ordered or unordered, depending on availability), a lot of the above
makes sense. But it still makes me wish for the power of git's
config-file format that seems to preserve original config files much
better.
An additional option is using a sqlite database. The sqlite
library is part of the stdlib, and advantages include being a single
file, expandability, consistent/reliable character encoding,
cross-platform portability, and atomicity (utilities that read/write
are blocked from getting/creating incomplete data seen by the other
file).
-tkc
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 May Become Relevant Now
On Sunday, August 2, 2015 at 7:25:37 PM UTC-5, Mark Lawrence wrote: > rr should have a field day with this one [...] You must be nuts if you think i'm going to click that link for an article that was written today, Hahaha! Quote the relevant bits. -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 May Become Relevant Now
On Mon, Aug 3, 2015 at 12:15 PM, Rick Johnson wrote: > On Sunday, August 2, 2015 at 7:25:37 PM UTC-5, Mark Lawrence wrote: >> rr should have a field day with this one [...] > > You must be nuts if you think i'm going to click that link > for an article that was written today, Hahaha! Quote the > relevant bits. How do you know it was written today, if you didn't click it? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: GvR Europython keynote described on lwn.net
On 03/08/2015 02:27, Steven D'Aprano wrote: On Mon, 3 Aug 2015 08:34 am, Rick Johnson wrote: So what? If i had to guess, i would guess that the hacks are mostly to bring py3000 features to 2.7 These would be the features of Python 3 that nobody needs and nobody wants because Python 2 is good enough for everyone? I wouldn't want to guess what the customizations do. Is there actually a definitive list of goodies that were backported from 3.x to 2.6 or 2.7? Of course some bits and pieces are still being backported to 2.7. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Most Pythonic way to store (small) configuration
On Sun, 02 Aug 2015 16:11:14 -0500, Tim Chase wrote:
> On 2015-08-02 21:54, Ben Finney wrote:
>> So, both XML and JSON should be considered write-only, and produced
>> only for consumption by a computer; they are a poor choice for
>> presenting to a human.
[snip]
> I second Ben's thoughts against XML & JSON -- they *can* be edited by
> hand, but put the onus on the user to make perfect XML/JSON. Config
> files (".ini") are more forgiving.
[snip]
> An additional option is using a sqlite database. The sqlite library
> is part of the stdlib, and advantages include being a single file,
> expandability, consistent/reliable character encoding, cross-platform
> portability, and atomicity (utilities that read/write are blocked from
> getting/creating incomplete data seen by the other file).
Well, I have at least some non-zero chance of reading and writing JSON
or XML by hand. Can the same be said for a sqlite database? ;-)
--
https://mail.python.org/mailman/listinfo/python-list
Re: GvR Europython keynote described on lwn.net
On Mon, Aug 3, 2015 at 1:10 PM, Mark Lawrence wrote: > On 03/08/2015 02:27, Steven D'Aprano wrote: >> >> On Mon, 3 Aug 2015 08:34 am, Rick Johnson wrote: >> >>> So what? If i had to guess, i would guess that the hacks are >>> mostly to bring py3000 features to 2.7 >> >> >> These would be the features of Python 3 that nobody needs and nobody wants >> because Python 2 is good enough for everyone? >> >> I wouldn't want to guess what the customizations do. >> > > Is there actually a definitive list of goodies that were backported from 3.x > to 2.6 or 2.7? Of course some bits and pieces are still being backported to > 2.7. You could probably find a definitive list of *language* features that got backported. Trying to find a full list of the *modules* that got backported will be harder, as they're not a part of Python 2.7 as such (the backports are on PyPI instead) - and some of them have slightly different names, too. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
how to determine for using c extension or not ?
Hello everyone. I use bisect module. bisect module developer give us c extension as _bisect. If Python3.3 use _bisect, _bisect override his functions in bisect.py. now, I use id() function to determine for using c extension or not. >>> import bisect >>> id(bisect.bisect) 139679893708880 >>> import _bisect >>> id(_bisect.bisect) 139679893708880 they return 139679893708880 as id. so i believe that i use c extension. My check is correct ? right ? or you have more good idea ? -- https://mail.python.org/mailman/listinfo/python-list
