Re: object.enable() anti-pattern
On 2013-05-08 09:52, Steven D'Aprano wrote: I'm looking for some help in finding a term, it's not Python-specific but does apply to some Python code. This is an anti-pattern to avoid. The idea is that creating a resource ought to be the same as "turning it on", or enabling it, or similar. I don't think the anti-pattern has a name, but it's opposite pattern is named: http://en.wikipedia.org/wiki/Resource_Acquisition_Is_Initialization -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Style question -- plural of class name?
On 2013-05-08 21:20, Roy Smith wrote: FooEntry is a class. How would you describe a list of these in a docstring? "A list of FooEntries" "A list of FooEntrys" "A list of FooEntry's" "A list of FooEntry instances" The first one certainly sounds the best, but it seems wierd to change the spelling of the class name to make it plural. I'm using services like Github more and more to talk about code, so I have taken to adopting its inline markup for `code` when referring to identifiers. Thus, I will often write A list of `FooEntry`s But I don't mind A list of FooEntries Hopefully there isn't also a `FooEntries` class. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: object.enable() anti-pattern
On 2013-05-10 12:00, Steven D'Aprano wrote: But either way, that's fine. You've found an object where it does make sense to have an explicit "make it go" method: first one entity has permission to construct the object, but not to open the underlying file. Another entity has permission to open the underlying file, but not to create the object. I have no idea whether this is a reasonable security design or not, it actually sounds a bit rubbish to me but what do I know? So let's treat it as a reasonable design. As I've said, repeatedly, that's not what I'm talking about. When you DON'T have useful things that can be done with the object before calling "enable", then it is an anti-pattern to require a separate call to "enable" method, and the enable functionality should be moved into the object constructor. If you DO have useful things that can be done, like pass the object to another entity, for security, then that's a whole 'nuther story. I'd be curious to see in-the-wild instances of the anti-pattern that you are talking about, then. I think everyone agrees that entirely unmotivated "enable" methods should be avoided, but I have my doubts that they come up very often. Do programmers have a natural tendency to make an extra, completely unnecessary method? I would think that they have a natural tendency to the opposite. In my experience, everyone has a reason in mind when they follow a pattern/anti-pattern. It is pretty rare that someone just does some specific, nameable thing for no reason at all. There is no need to call out an anti-pattern for which no one has a reason to do it. But there is a continuum of reasons. Some reasons are better than others. Some reasons only apply in a small set of circumstances but seem like they would apply more generally, at least to novice programmers. Programmers can be wrong about what they think the (anti-)pattern actually achieves. The whole point of naming an anti-pattern is to discuss those reasons, show where they are misapplied, where YAGNI, why novices overuse it, other patterns that should be used instead, and also the circumstances where it is actually a good pattern instead. To artificially limit the discussion of the anti-pattern to the trivial, entirely unmotivated case forbids most of the interesting and instructive parts of the conversation. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: object.enable() anti-pattern
On 2013-05-10 15:01, Roy Smith wrote: In article , Robert Kern wrote: I'd be curious to see in-the-wild instances of the anti-pattern that you are talking about, then. I think everyone agrees that entirely unmotivated "enable" methods should be avoided, but I have my doubts that they come up very often. As I mentioned earlier in this thread, this was a common pattern in the early days of C++, when exceptions were a new concept and handled poorly by many compilers (and, for that matter, programmers). There was a school of thought that constructors should never be able to fail (because the only way for a constructor to fail is to throw an exception). The pattern was to always have the constructor succeed, and then either have a way to check to see if the newly-constructed object was valid, or have a separate post-construction initialization step which could fail. See, for example, the isValid() and Exists() calls for RogueWave's RWFile class (http://tinyurl.com/c8kv26g). And also, http://tinyurl.com/cgs6clx. Even today, there are C++ implementations which do not use exceptions. Some are for use in embedded or real-time systems where things need to be strictly time-bound and/or memory-bound. Others are for historical reasons (http://tinyurl.com/6hn4zo). Once people were used to writing "can't fail" constructors in C++, they often continued using that pattern in other languages, where the underlying reasons no longer made sense. Quite possibly, they never even knew the underlying reasons; they were taught, "Constructors must never fail", and assumed it was a universal rule. Right, this is one of the "bad reasons" I talk about later in my message. The authors had a reason in their mind for doing it (they thought it was a universal rule); it was just a bad one. It's more useful to talk about why people thought they should follow that pattern than to just say "there is no reason to do this". This, BTW, is one of my biggest beefs with the classic Gang Of Four pattern book. It presents a bunch of patterns as being universally applicable, when in reality many (if not most) of them are highly C++ specific. BTW, whenever I read things like, "I think everyone agrees", I automatically assume what the writer really meant was, "I, and all the people who agree with me, think". Hah! Fair enough. I actually meant it to emphasize that I thought that Steven was overly reducing his statements to something that was trivially true, sacrificing content for validity. I will suggest that your interpretation of that phrase is more appropriate when the speaker is proposing something of their own rather than (partially) conceding a point. The exaggeration is only self-aggrandizing in the former case. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: object.enable() anti-pattern
On 2013-05-10 16:44, Serhiy Storchaka wrote:
10.05.13 15:19, Robert Kern написав(ла):
I'd be curious to see in-the-wild instances of the anti-pattern that you
are talking about, then.
Many (if not most) GUI frameworks use this pattern.
button = Button("text")
button.setForegroundColor(...)
button.setBackgoundColor(...)
button.setFont(...)
button.setRelief(...)
button.setBorder(...)
button.setWidth(...)
button.setAction(...)
button.setMouseListener(...)
button.place(...)
Another example is running a subprocess in Unix-like systems.
fork()
open/close file descriptors, set limits, etc
exec*()
According to Steven's criteria, neither of these are instances of the
anti-pattern because there are good reasons they are this way. He is reducing
the anti-pattern to just those cases where there is no reason for doing so. That
is why I asked for in-the-wild instances. I should have qualified my sentence to
include "according to your criteria" because people seem to be answering my
query out of that context.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
Re: object.enable() anti-pattern
On 2013-05-11 08:51, Steven D'Aprano wrote: On Fri, 10 May 2013 18:20:34 +0100, Robert Kern wrote: According to Steven's criteria, neither of these are instances of the anti-pattern because there are good reasons they are this way. He is reducing the anti-pattern to just those cases where there is no reason for doing so. But isn't that the case for all anti-patterns? We agree that GOTO is an anti-pattern. That doesn't mean that there aren't valid reasons for using GOTO. Sometimes there are good use-cases for GOTO that outweigh the harm. By calling it an anti-pattern, though, we shift the onus onto the person wanting to use GOTO: justify why you need it, or use something else. Yes, that was the point I was making. You seemed to be defining away the legitimate instances as not instances of the pattern at all because they were legitimate, and that appeared to me to be defeating the purpose of having the discussion. On a related note, I *don't* think it's a good idea to phrase it as "justify why you need it". I don't think that gives very good guidance to a novice when they are given the task of designing something. People can come up with a justification for just about anything, especially when they are only justifying things to themselves. I think it's more important to just talk about the situations where a pattern is useful, and the common situations where people, for whatever reason, *think* that a pattern is useful, but isn't. Naming it a Pattern or Anti-pattern is really just a measure of how bad the latter half of that is compared to the first half, and is less interesting than the discussion itself. That's why I had a bug up my ass about what looked like the exclusion of the "good" uses. It's the good examples that give novices an idea of what a good justification looks like, so they can tell if the justification they are giving themselves measures up. Would you object less if I called it a "code smell" than an "anti- pattern"? If so, I accept your criticism, and call it a code smell: the less temporal coupling your API has, the better. That was not really my objection. I was objecting to the way you appeared to be defining the particular pattern in question. But as we appear to agree on the important matters, I won't press it further. That is why I asked for in-the-wild instances. How about this? http://legacy.thecodewhisperer.com/post/366626867/improving-clarity-by-removing-temporal-coupling There's something about Java mixedCase that makes my eyes glaze, so I'll take your word for it. :-) Another example of temporal coupling is json_decode in PHP: you must follow it by a call to json_last_error, otherwise you have no way of telling whether the json_decode function succeeded or not. http://php.net/manual/en/function.json-last-error.php I suspect that the author might say something about error checking being optional. But yeah, terrible API. :-) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Future standard GUI library
On 2013-05-20 08:00, Terry Jan Reedy wrote: On 5/20/2013 1:04 AM, Vito De Tullio wrote: Terry Jan Reedy wrote: Do you think tkinter is going to be the standard python built-in gui solution as long as python exists? AT the moment, there is nothing really comparable that is a realistic candidate to replace tkinter. FLTK? (http://www.fltk.org/index.php) tkinter is the Python wrapper of the tk library, just as wxpython is the python wrapper of the wx library. I do not see a py-fltk wrapper. It exists, but it's really old. http://pyfltk.sourceforge.net/ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: PyWart: The problem with "print"
On 2013-06-03 05:20, Dan Sommers wrote: On Sun, 02 Jun 2013 23:23:42 -0400, Jason Swails wrote: ... (And yes, a good portion of our code is -still- in Fortran -- but at least it's F90+ :). I am a huge proponent of using the right tool for the job. There is nothing wrong with some well-placed FORTRAN, as long as the PSF No, no. It's the PSU that you have to worrNO CARRIER -- http://mail.python.org/mailman/listinfo/python-list
Re: Interactive interpreter hooks
On 2013-06-03 08:55, Steven D'Aprano wrote: The sys module defines two hooks that are used in the interactive interpreter: * sys.displayhook(value) gets called with the result of evaluating the line when you press ENTER; * sys.excepthook(type, value, traceback) gets called with the details of the exception when your line raises an exception. Is there a way to hook into the interactive interpreter *before* it is evaluated? That is, if I type "len([])" at the prompt and hit ENTER, I want a hook that runs before len([]) is evaluated to 0, so that I get the string "len([])". You will need to write your own REPL for this. Use the code.InteractiveConsole class: http://docs.python.org/2/library/code I recommend source-diving to see what you need to override, but I suspect you can just wrap around the `runsource()` method. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Bools and explicitness [was Re: PyWart: The problem with "print"]
On 2013-06-06 10:45, Chris Angelico wrote: For the "accept any object that has a next() method" sorts of rules, I don't know of any really viable system that does that usefully. The concept of implementing interfaces in Java comes close, but the class author has to declare that it's implementing some named interface. In theory there could be something that deduces the validity from the given structure, but I'm not aware of any language that does this. But it would let you do stuff like this (prototyped in Python): class Integers: def __init__(self): self.value=0 def next(self): self.value+=1 return self.value interface Iterable: next(self) def grab_three_values(Iterable iter): return iter.next(),iter.next(),iter.next() With a language that checks these sorts of things at compile time, it's not a big deal to test. With something fully dynamic like Python, it's probably not worth the effort. But maybe checks like this could be useful to something like Coverity. As Serhiy notes, Go does this, almost exactly as you wrote it (modulo syntax). http://golang.org/doc/effective_go.html#interfaces_and_types -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Bools and explicitness [was Re: PyWart: The problem with "print"]
On 2013-06-06 16:41, Chris Angelico wrote: Anyway, regardless of your language, there's always some criteria that can't be coded. Suppose the valid input for a function were "integers whose square roots are integers but whose cube roots are not". You won't easily get compile-time checking of that. Say that on a Haskell list, and they'll take it as a challenge. :-) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Re-using copyrighted code
On 2013-06-08 22:31, Malte Forkel wrote: Hello, I have written a small utility to locate errors in regular expressions that I want to upload to PyPI. Before I do that, I would like to learn a litte more about the legal aspects of open-source software. What would be a good introductory reading? Larry Rosen's free (open source, even!) book _Open Source Licensing_ is good introductory reading. Larry is an intellectual property lawyer and helped draft the current PSF license. http://www.rosenlaw.com/oslbook.htm -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple algorithm question - how to reorder a sequence economically
On 2013-05-24 14:43, Chris Angelico wrote: On Fri, May 24, 2013 at 11:23 PM, Peter Brooks wrote: Actually, thinking about it, there is probably a source of non-algorithmically-derived 'random' numbers somewhere on the net that would do the job nicely. True entropy is usually provided by a source such as /dev/random (on Unix systems). It's sometimes referred to as "cryptographic" randomness, due to its necessity in secure encryption work. There are various ways to get this in a cross-platform way. os.random() and os.urandom(), particularly. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie: standard way to add version, author and license to a source file?
On 2013-06-13 13:47, Rui Maciel wrote: Is there any PEP that establishes a standard way to specify the version number of a source code file, as well as its authors and what license it's distributed under? As for versions: http://www.python.org/dev/peps/pep-0396/ For licenses and author: http://producingoss.com/en/license-quickstart.html#license-quickstart-applying -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Must we include urllib just to decode a URL-encoded string, when using Requests?
On 2013-06-13 14:05, Dotan Cohen wrote: I am using the Requests module to access remote URLs. Sometimes I need to URL-decode or URL-encode a string (via RFC 3986). Must I import urllib or urllib2 just to use their quote() and unquote() methods? Yes. Do you think there is a problem with doing so? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Must we include urllib just to decode a URL-encoded string, when using Requests?
On 2013-06-13 14:25, Dotan Cohen wrote: On Thu, Jun 13, 2013 at 4:20 PM, Robert Kern wrote: Yes. Do you think there is a problem with doing so? I'm pretty sure that Requests will use either urllib or urllib2, depending on what is available on the server. No, it doesn't. It gets its quote() function from urllib always. I would like to use whatever Requests is currently using, rather than import the other. Can I tell which library Requests is currently using and use that? The only thing I can think that you are talking about is the difference between Python 2 and Python 3. In Python 2, it's urllib.quote() and in Python 3, it's urllib.parse.quote(), but that's a Python-version issue, not something to do with requests, per se. requests does have a compatibility layer, internally, that pastes over those issues, but I don't think that is intended to be a stable public API that you should rely on. You should handle that kind of switch yourself if you care about compatibility across both versions of Python. https://github.com/kennethreitz/requests/blob/master/requests/compat.py#L86 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Eval of expr with 'or' and 'and' within
On 2013-06-14 10:50, Nick the Gr33k wrote:
I started another thread because the last one was !@#$'ed up by irrelevant
replies and was difficult to jeep track.
>>> name="abcd"
>>> month="efgh"
>>> year="ijkl"
>>> print(name or month or year)
abcd
Can understand that, it takes the first string out of the 3 strings that has a
truthy value.
>>> print("k" in (name and month and year))
True
No clue. since the expression in parenthesis returns 'abcd' how can 'k'
contained within 'abcd' ?
>>> print(name and month and year)
ijkl
Seems here is returning the last string out of 3 strings, but have no clue why
Python doing this.
>>> print("k" in (name and month and year))
True
>>>
yes, since expression returns 'ijkl', then the in operator can detect the 'k'
character within the returned string.
This is all iw ant to know.
This is all you need to read:
http://docs.python.org/2/reference/expressions.html#boolean-operations
Note the difference between how "or" and "and" each short-circuit. That is why
the (name or month or year) returns the first truthy value while (name and month
and year) returns the last truthy value. When "or" finds the first truthy value,
it can stop looking since the whole expression must be truthy no matter what the
values are after it. "and" cannot stop looking until it finds a falsy value or
runs out of values to look at.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
Re: Eval of expr with 'or' and 'and' within
On 2013-06-14 18:01, Nick the Gr33k wrote: On 14/6/2013 7:47 μμ, Benjamin Kaplan wrote: In an "and" clause, python returns the first false value or the last value, because that will evaluate to the correct Boolean value. In an "or" clause, python returns the first true value or the last value. When Python finally got a Boolean type, no one wanted to break backwards compatibility for this. This is exactly what i dont understand and thats why i keep asking and people call me an idiot. I just dont understand why it behaves like that. Why return first or last value? because that will evaluate to the correct Boolean value How do you mean? Please elaborate. Please read the link I gave. It explains why. http://docs.python.org/2/reference/expressions.html#boolean-operations -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Don't feed the troll...
On 2013-06-15 03:09, Cameron Simpson wrote: On 15Jun2013 10:42, Ben Finney wrote: | "D'Arcy J.M. Cain" writes: | Even for those who do participate by email, though, your approach is | broken: | > My answer is simple. Get a proper email system that filters out | > duplicates. | | The message sent to the individual typically arrives earlier (since it | is sent straight from you to the individual), and the message on the | forum arrives later (since it typically requires more processing). | | But since we're participating in the discussion on the forum and not in | individual email, it is the later one we want, and the earlier one | should be deleted. They're the same message! (Delivered twice.) Replying to either is equivalent. So broadly I don't care which gets deleted; it works regardless. | So at the point the first message arrives, it isn't a duplicate. The | mail program will show it anyway, because “remove duplicates” can't | catch it when there's no duplicate yet. But it can when the second one arrives. This is true regardless of the delivery order. Ben said that he doesn't use email for this list. Neither do I. We use one of the newsgroup mirrors. If you Cc us, we will get a reply on the newsgroup (where we want it) and a reply in our email (where we don't). The two systems cannot talk to each other to delete the other message. | You do this by using your mail client's “reply to list” function, which | uses the RFC 3696 information in every mailing list message. No need, but a valid option. | Is there any mail client which doesn't have this function? If so, use | your vendor's bug reporting system to request this feature as standard, | and/or switch to a better mail client until they fix that. Sorry, I could have sworn you said you weren't using a mail client for this... He's suggesting that *you* who are using a mail reader to use the "reply to list" functionality or request it if it is not present. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Timsort in Cpython
On 2013-06-15 20:44, alphons...@gmail.com wrote: I'm currently trying to make sense of Python's Timsort function. From the wikipedia page I was told the algorithm is located somewhere here: http://hg.python.org/cpython/file/default/Objects/listobject.c So of all the functions in there, could somebody point to me which one is timsort? listsort() http://hg.python.org/cpython/file/default/Objects/listobject.c#l1896 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Timsort in Cpython
On 2013-06-15 21:21, alphons...@gmail.com wrote: Hey guys, Thanks for the quick reply! So why did they decide to call it listsort in the source instead? Why didn't they keep it as Timsort? This was the first implementation of the algorithm. The algorithm was only colloquially named "Timsort" after it was used in Python. This the naming convention for the C implementation of builtin types' methods in the Python codebase. The C implementation listsort() corresponds with the Python method list.sort(). Similarly, listappend() is list.append(), listpop() is list.pop(), etc. C.f. http://hg.python.org/cpython/file/default/Objects/listobject.c#l2362 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: weird behavior. bug perhaps?
On 2013-06-18 15:23, zoom wrote: Hi, I have a strange problem here. Perhaps someone would care to help me. In the file test.py I have the following code: from scipy import matrix, tile, mean, shape import unittest class TestSequenceFunctions(unittest.TestCase): def setUp(self): self.m = [[1,2],[3,4],[3,4],[3,4]] def test_simplify(self): m = matrix(self.m) print shape(m) print [shape(m)[1],1] print shape(tile(mean(m,1),[shape(m)[1],1]).T) if __name__ == '__main__': unittest.main() (Note that test.py, is just a simplification of my testing file, sufficient to reproduce the weird behavior that I'm about to describe.) If i run it in terminal via "python test.py" command I get the following output: (4, 2) [2, 1] (1, 8) . -- Ran 1 test in 0.000s OK Now comes the funny part. Let's try to run the following code in python interpreter: >>> m = [[1,2],[3,4],[3,4],[3,4]] >>> >>> from scipy import matrix, tile, mean, shape >>> print shape(m) (4, 2) >>> print [shape(m)[1],1] [2, 1] >>> print shape(tile(mean(m,1),[shape(m)[1],1]).T) (4, 2) Note the difference between outputs of: print shape(tile(mean(m,1),[shape(m)[1],1]).T) I mean, WTF? This is definitely not the expected behavior. Anybody knows what just happened here? As rusi noted, the difference between your two snippets is that in one, you converted the list of lists to a matrix object in your test suite but not in your interactive session. Most numpy functions like mean() will convert their arguments to regular numpy.ndarray objects rather than matrix objects. matrix is a subclass of ndarray that adds special behavior: in particular, operations on matrix objects retain their 2D-ness even when an ndarray would flatten down to a 1D array. [~] |1> import numpy as np [~] |2> m = [[1,2],[3,4],[3,4],[3,4]] [~] |3> a = np.array(m) [~] |4> b = np.matrix(m) [~] |5> np.mean(a, axis=1) array([ 1.5, 3.5, 3.5, 3.5]) [~] |6> np.mean(b, axis=1) matrix([[ 1.5], [ 3.5], [ 3.5], [ 3.5]]) [~] |7> np.mean(a, axis=1).shape (4,) [~] |8> np.mean(b, axis=1).shape (4, 1) This will propagate through the rest of your computation. Personally, I recommend avoiding the matrix type. It causes too many problems. Stick to plain ndarrays. You will probably want to ask further numpy questions on the numpy-discussion mailing list: http://www.scipy.org/scipylib/mailing-lists.html -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: n00b question on spacing
On 2013-06-25 01:22, Mark Janssen wrote: On Mon, Jun 24, 2013 at 4:48 PM, alex23 wrote: On 23/06/2013 3:43 AM, Mark Janssen wrote: There was a recent discussion about this (under "implicit string concatenation"). It seems this is a part of the python language specification that was simply undefined. It's part of the language reference, not an accidental artifact: http://docs.python.org/2/reference/lexical_analysis.html#string-literal-concatenation When I say "specification", I mean "specified in the formal notation" (BNF, etc). There is quite a bit of Python's lexical analysis that is specified in places other than the formal notation. That does not mean it is undefined. It is well defined in the lexer code and the documentation. You suggest that a "rule probably should be added to the lexer to make this explicit." That is not necessary. The rule is already there. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for a name for a deployment framework...
On 2013-06-24 13:50, Roy Smith wrote: Without forming any opinion on the software itself, the best advice I can offer is that naming puns are very popular. If you're thinking of this as a fabric replacement, I would go with cloth, textile, material, gabardine, etc. brocade -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: n00b question on spacing
On 2013-06-25 12:48, Chris Angelico wrote: On Tue, Jun 25, 2013 at 9:19 PM, Robert Kern wrote: There is quite a bit of Python's lexical analysis that is specified in places other than the formal notation. That does not mean it is undefined. It is well defined in the lexer code and the documentation. You suggest that a "rule probably should be added to the lexer to make this explicit." That is not necessary. The rule is already there. Be careful; Python is not an implementation-defined language. Python has no "lexer code" - CPython does, and is probably what you're thinking of. No, that's not what I am thinking of. I said that the rule is defined in both code and the documentation. Mark did suggest adding the rule to the lexer (for which he may have been thinking of just CPython, but you can take that up with him), but of course it is already there. I did not suggest that its presence in the lexer code (of any or all implementations) is sufficient, but the point is moot because it is already both explicitly implemented (several times) and clearly documented in the Python language reference. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is the argparse module so inflexible?
On 2013-06-27 17:02, Dave Angel wrote: On 06/27/2013 09:49 AM, Andrew Berg wrote: On 2013.06.27 08:08, Roy Smith wrote: Can you give us a concrete example of what you're trying to do? The actual code I've written so far isn't easily condensed into a short simple snippet. I'm trying to use argparse to handle all the little details of parsing and verifying arguments in the precmd hook for a cmd.Cmd child class. argparse's help system is more sophisticated than cmd's help and does all the work of verifying arguments. The problem I keep running into is that I can't handle any bad input very well. I would have to override every method that catches ArgumentError in order to get a useful exception that I would then handle. If I input something that begins with '-' that isn't recognized, parse_args doesn't even raise the exception; it just quits. No, it raises the SystemExit exception. if you don't catch it, then the program exits. Perhaps it's not clear to you, but sys.exit() just raises the SystemExit exception, as Joshua pointed out. Internally, the parser raises ArgumentError which has some useful pieces of information, specifically, the name of the argument that failed to parse. Unfortunately, it catches that error in parse_known_args(), then formats that information into a message string to pass to the error() method, which by default raises SystemExit with just that message string. It is somewhat difficult to override the parse_known_args() to not lose that information from the ArgumentError because you will have to copy-paste the rest of the code in there. So yes, you can override the error() method or catch SystemExit if all you want is the formatted message string, but I have to agree that there is a missed opportunity to make argparse more widely usable for other command-line like parsing tasks[1]. [1] As an existence proof, I offer you one that I wrote for handling IPython's %magic commands. I just needed the formatted string, so I could just get away with overriding error(). https://github.com/ipython/ipython/blob/master/IPython/core/magic_arguments.py -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Making a pass form cgi => webpy framework
On 2013-06-28 04:38, Νίκος wrote: Στις 28/6/2013 2:08 πμ, ο/η Cameron Simpson έγραψε: Pick a simple framework or templating engine and try it. I have no recommendations to make in this area myself. Can you explain to me the difference of the former and latter? A templating engine takes your data and applies it to templates that you have written to generate the final HTML that is sent to the web browser. A web framework is a library that provides tools and a way of structuring your that makes it easier to write a web application. Web frameworks typically include a templating engine or provide support for working with external templating engines. In addition, web frameworks provide many other services, like routing URLs to specific parts of your code, managing pools of database connections, handling web sessions, validating form data, and connecting your code to many different ways of deploying web applications without having to rewrite your code. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Making a pass form cgi => webpy framework
On 2013-06-28 11:15, Νίκος wrote: Στις 28/6/2013 12:35 μμ, ο/η Robert Kern έγραψε: On 2013-06-28 04:38, Νίκος wrote: Στις 28/6/2013 2:08 πμ, ο/η Cameron Simpson έγραψε: Pick a simple framework or templating engine and try it. I have no recommendations to make in this area myself. Can you explain to me the difference of the former and latter? A templating engine takes your data and applies it to templates that you have written to generate the final HTML that is sent to the web browser. A web framework is a library that provides tools and a way of structuring your that makes it easier to write a web application. Web frameworks typically include a templating engine or provide support for working with external templating engines. In addition, web frameworks provide many other services, like routing URLs to specific parts of your code, managing pools of database connections, handling web sessions, validating form data, and connecting your code to many different ways of deploying web applications without having to rewrite your code. I see, your explanation started to make things clearer to me. What is the easiest and simplest web framework you advise me to use? Also please provide a well written and simple tutorial i can read upon. Thank you. I will not advise you to do anything. I will point you to a web framework that I happen to like, but you will need to read its documentation to decide if it is right for you. I cannot provide any help in installing or using it. http://flask.pocoo.org/ http://flask.pocoo.org/docs/tutorial/ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Problems with subclassing enum34
On 2013-06-28 11:48, Thomas Heller wrote: trying out the enum34 module. What I want to create is a subclass of enum.Enum that is also based on ctypes.c_int so that I can better use enum instances in ctypes api calls. When I do this, I get a metaclass conflict: >>> class MyEnum(ctypes.c_int, enum.Enum): ...FOOBAR = 0 ... Traceback (most recent call last): File "", line 1, in TypeError: Error when calling the metaclass bases metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases >>> When I do this, it does not work either: >>> class MyEnum_meta(type(ctypes.c_int), type(enum.Enum)): ... pass enum.EnumMeta uses super() in its __new__() implementation but _ctypes.PyCSimpleType doesn't. Thus, only _ctypes.PyCSimpleType.__new__() gets a chance to run. Switching the order of the two might work. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Problems with subclassing enum34
On 2013-06-28 16:32, Thomas Heller wrote: Am 28.06.2013 17:25, schrieb Thomas Heller: Robert Kern: enum.EnumMeta uses super() in its __new__() implementation but _ctypes.PyCSimpleType doesn't. Thus, only _ctypes.PyCSimpleType.__new__() gets a chance to run. Switching the order of the two might work. Robert found the problem but I'm unsure if there is a solution. Also I'm unsure whether this is a bug in ctypes or in enum or if they are simply incompatible. I forgot to mention that switching the order of metaclasses didn't work. You may also need to manually deal with the conflict between Enum.__new__() and c_int.__new__(). -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: ? get negative from prod(x) when x is positive integers
On 2013-06-28 16:26, Vincent Davis wrote: @Joshua "You are using numpy.prod()" Wow, since sum([1,2,3,4]) worked I tried prod([1,2,3,4]) and got the right answer so I just used that. Confusing that it would use numpy.prod(), I realize now there is no python prod(). At no point do I "import numpy" in my code. The seems to be a result of using ipython, or at least how I am using it "ipython notebook --pylab inline". The --pylab option will do the following import: from matplotlib.pyplot import * That includes a "from numpy import *" in there. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: python adds an extra half space when reading from a string or list
On 2013-06-30 18:24, Νίκος wrote: Στις 29/6/2013 8:00 μμ, ο/η Mark Lawrence έγραψε: Why this when the approach to Nick the Incompetant Greek has been to roll out the red carpet? Your mother is incompetent who raised a brat like you. That is not acceptable behavior on this list. Please keep the gratuitous insults offlist. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: python adds an extra half space when reading from a string or list
On 2013-06-30 21:14, Νίκος wrote: Στις 30/6/2013 10:58 μμ, ο/η Robert Kern έγραψε: On 2013-06-30 18:24, Νίκος wrote: Στις 29/6/2013 8:00 μμ, ο/η Mark Lawrence έγραψε: Why this when the approach to Nick the Incompetant Greek has been to roll out the red carpet? Your mother is incompetent who raised a brat like you. That is not acceptable behavior on this list. Please keep the gratuitous insults offlist. Ι'm sorry but please put yourself into my shows, where in multiple threads i'm still discussed and being insulted as a troll and as an idiot and incompetent and stuff like that by lots of people who instead of actually replying to my posts they deviate and have endless conversation about me. Enough is enough. Iam not a troll, neither incompetent. Period. You may ask people to stop their insulting comments. Do not engage by returning insults yourself. Not on the list at least. I don't care what you send by private email. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: python adds an extra half space when reading from a string or list
On 2013-06-30 22:57, Joshua Landau wrote: On 30 June 2013 20:58, Robert Kern wrote: On 2013-06-30 18:24, Νίκος wrote: Στις 29/6/2013 8:00 μμ, ο/η Mark Lawrence έγραψε: Why this when the approach to Nick the Incompetant Greek has been to roll out the red carpet? Your mother is incompetent who raised a brat like you. That is not acceptable behavior on this list. Please keep the gratuitous insults offlist. As much as you are right, this argument was started by Mark. If you reprimand anyone (other threads being ignored) it should be him. Reacting only to Nick, even though what Nick said was undue, implies that you agree with Mark's actions. Remember that Nick is as much a human as all of us, he is bound to have his feelings hurt when so many people pick on him -- whether they are justified or not. I hereby reprimand both Mark and Nikos. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: indexerror: list index out of range??
On 2013-06-29 16:52, Joshua Landau wrote: On 29 June 2013 15:30, Mark Lawrence wrote: On 29/06/2013 14:44, Dave Angel wrote: Since you're using the arrogant and buggy GoogleGroups, this http://wiki.python.org/moin/GoogleGroupsPython. Please don't make comments like this, you'll upset the Python Mailing List Police. *doesn't understand* Mark frequently makes similar comments (in content and tone) to people who come here using Google Groups. Presumably, he has received criticism for this (mostly on tone grounds, I imagine), either on or off list. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Need explanation of this error
On 2013-07-01 10:13, preri...@gmail.com wrote: Hi, I'm new to Python and trying to run a already written code. Can someone please explain the error below? And if possible, how do I resolve this? Traceback (most recent call last): File "c:\Project_1\regression_1.py", line 7, in from sklearn import metrics, cross_validation, linear_model File "c:\Python27\lib\site-packages\sklearn\metrics\__init__.py", line 31, in from . import cluster File "c:\Python27\lib\site-packages\sklearn\metrics\cluster\__init__.py", line 8, in from .supervised import adjusted_mutual_info_score File "c:\Python27\lib\site-packages\sklearn\metrics\cluster\supervised.py", li ne 19, in from .expected_mutual_info_fast import expected_mutual_information File "expected_mutual_info_fast.pyx", line 10, in init sklearn.metrics.cluster .expected_mutual_info_fast (sklearn\metrics\cluster\expected_mutual_info_fast.c: 4886) File "c:\Python27\lib\site-packages\scipy\special\__init__.py", line 529, in < module> from ._ufuncs import * ImportError: DLL load failed: The specified module could not be found. This particular module incorporates FORTRAN subroutines. My guess is that whoever compiled your scipy binary did it in such a way that the FORTRAN standard library was being linked in as a DLL instead of statically. This DLL is not present on your system. Windows is trying to find it, but failing. How did you install scipy? If you used a prebuilt binary installer, can you please link to the exact one that you used? Try using depends.exe to find out what DLL it is looking for. http://www.dependencywalker.com/ The file that you want to check in depends.exe: c:\Python27\lib\site-packages\scipy\special\_ufuncs.pyd -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: File exists but Python says 'not found'.
On 2013-07-01 11:47, preri...@gmail.com wrote:
I'm running this code that reads 2 csv files (one of them is train.csv). The
code gives an error saying 'file not does not exist'. However, the file does
exists in the same location as the .py file. Can someone please help me on
this. Thanks!
How are you running the code? If you are doing pandas.read_csv('train.csv'), the
file must be in the current working directory of the running process. The
location of the .py file contain the code is not relevant. For example, if you
are using an IDE to run the script, you may need to configure how it runs the
script to pick a particular directory for it to run in.
Code Output-->
Reading dataset...
Traceback (most recent call last):
File "c:\Project_1\regression_2.py", line 163, in
main(**args)
File "c:\Project_1\regression_2.py", line 80, in main
train_data = pd.read_csv(train)
Since the filename of regression_2.py in the traceback is fully-qualified, I
expect that you are running the program from something other than the
c:\Project_1\ directory.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
Re: OSError [Errno 26] ?!?!
On 2013-07-02 14:00, Chris “Kwpolska” Warrick wrote: On Tue, Jul 2, 2013 at 2:39 PM, Νίκος wrote: Please suggest an editor that has built in rsync ability so to immediately upload my cgi-scripts when i hit save in the text editor. CGI? Is this 2000? Nobody uses that wording these days. He is indeed using actual, bona fide CGI scripts. It's not just an antiquated wording for "web app". -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Important features for editors
On 2013-07-04 10:14, Νίκος wrote: If you guys want to use it i can send you a patch for it. I know its illegal thing to say but it will help you use it without buying it. Please do not use this forum to make such offers. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Explain your acronyms (RSI?)
On 2013-07-06 20:38, Terry Reedy wrote: "rms has crippling RSI" (anonymous, as quoted by Skip). I suspect that 'rms' = Richard M Stallman (but why lower case? to insult him?). http://stallman.org/ """ "Richard Stallman" is just my mundane name; you can call me "rms". """ But Skip mentions 'worse for wrists'. So RSI must be a physical rather than mental condition. Does 'I' instead stand for Inoperability?, Instability?, or what? Let us try Google. Type in RSI and it offers 'RSI medications' as a choice. "RSI wrist" would probably have been a wiser start. Not all medical conditions have medication associated with them. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Geo Location extracted from visitors ip address
On 2013-07-06 09:41, Νίκος Gr33k wrote: Στις 6/7/2013 11:30 πμ, ο/η Chris Angelico έγραψε: On Sat, Jul 6, 2013 at 6:01 PM, � Gr33k wrote: Is there any way to pinpoint the visitor's exact location? Yes. You ask them to fill in a shipping address. They may still lie, or they may choose to not answer, but that's the best you're going to achieve without getting a wizard to cast Scrying. No, no registration requirements. you know when i go to maps.google.com its always find my exact city of location and not just say Europe/Athens. and twitter and facebook too both of them pinpoint my _exact_ location. How are they able to do it? We need the same way. They use client-side JavaScript. This is a relatively new API available in most, but not all, recent browsers. This information will not be available to your CGI script. You will have to generate HTML with the proper JavaScript to get the geolocation (if the user allows it) and then send it back to your server through a different CGI script (or web application endpoint). http://diveintohtml5.info/geolocation.html -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Stack Overflow moderator “animuson”
On 2013-07-10 10:52, Joshua Landau wrote: On 10 July 2013 10:12, Ian Kelly wrote: On Wed, Jul 10, 2013 at 2:46 AM, Mats Peterson wrote: Then they would have full control of this list and what gets pos Ahhh so this is pos, right? Telling the truth? Interesting. I don't know what you mean by that, but since the joke appears to have flown over your head, I'll explain it. Steven's "pos" was clearly mea What? I don't understand. Look, it's perfectly obvi -- http://mail.python.org/mailman/listinfo/python-list
Re: could you change PYPI downloads number for not-uploaded packages?
On 2013-07-22 16:44, dmitre...@gmail.com wrote: Hi all, could you change PYPI downloads number for not-uploaded packages from zeros to real posivive numbers? For example, my projects download links are binded to my website , and thus people see misleading zeros, e.g. https://pypi.python.org/pypi/openopt Downloads (All Versions): 0 downloads in the last day 0 downloads in the last week Or, even better, taking into account that some people install packages from subversion/git/etc repository, invoke "+1" when someone runs "python setup.py install" (or "develop") (provided internet connection is present) The maintenance and development of PyPI is discussed on the Distutils-SIG. Please bring your concerns there. http://www.python.org/community/sigs/current/distutils-sig/ In short, if you want to have download counts, you will need to host your package downloads from PyPI itself. There is no good way for PyPI to count downloads from any other source. What you might want to ask for instead is to have the download count not shown when the packages are not hosted on PyPI. That would be a reasonable change that I think the PyPI team would accept. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Frustrating circular bytes issue
On 6/26/12 5:30 PM, J wrote:
This is driving me batty... more enjoyment with the Python3
"Everything must be bytes" thing... sigh...
I have a file that contains a class used by other scripts. The class
is fed either a file, or a stream of output from another command, then
interprets that output and returns a set that the main program can
use... confusing, perhaps, but not necessarily important.
The class is created and then called with the load_filename method:
def load_filename(self, filename):
logging.info("Loading elements from filename: %s", filename)
file = open(filename, "rb", encoding="utf-8")
return self.load_file(file, filename)
I get this with Python 3.2:
Traceback (most recent call last):
File "bytes_unicode.py", line 32, in
d.load_filename(__file__)
File "bytes_unicode.py", line 6, in load_filename
file = open(filename, "rb", encoding="utf-8")
ValueError: binary mode doesn't take an encoding argument
Are you sure you are copy-pasting the code that is actually running? Can you
reduce your test case down to a small self-contained example that runs and
demonstrates the problem? I suspect that there is some other detail that is
causing things to fail. The following code works fine for me:
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import re
class Dummy(object):
""" This is a dummy file.
éµ∫é∂∂é∂ üñîçø∂é
"""
def load_filename(self, filename):
file = open(filename, "r", encoding="utf-8")
return self.load_file(file, filename)
def load_file(self, file, filename=""):
for string in self._reader(file):
print(string)
if not string:
break
def _reader(self, file, size=4096, delimiter=r"\n{2,}"):
buffer_old = ""
while True:
buffer_new = file.read()
print(type(buffer_new))
if not buffer_new:
break
lines = re.split(delimiter, buffer_old + buffer_new)
buffer_old = lines.pop(-1)
for line in lines:
yield line
yield buffer_old
if __name__ == '__main__':
d = Dummy()
d.load_filename(__file__)
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
Re: question about numpy, subclassing, and a DeprecationWarning
On 6/27/12 10:02 PM, Jason Swails wrote: Hello, I'm running into an unexpected issue in a program I'm writing, and I was hoping someone could provide some clarification for me. I'm trying to subclass numpy.ndarray (basically create a class to handle a 3D grid). When I instantiate a numpy.ndarray, everything works as expected. When I call numpy.ndarray's constructor directly within my subclass, I get a deprecation warning about object.__init__ not taking arguments. Presumably this means that ndarray's __init__ is somehow (for some reason?) calling object's __init__... This is some sample code: >>> import numpy as np >>> class derived(np.ndarray): ... def __init__(self, stuff): ... np.ndarray.__init__(self, stuff) ... >>> l = derived((2,3)) __main__:3: DeprecationWarning: object.__init__() takes no parameters >>> l derived([[ 8.87744455e+159, 6.42896975e-109, 5.56218818e+180], [ 1.79996515e+219, 2.41625066e+198, 5.15855295e+307]]) >>> Am I doing something blatantly stupid? Is there a better way of going about this? I suppose I could create a normal class and just put the grid points in a ndarray as an attribute to the class, but I would rather subclass ndarray directly (not sure I have a good reason for it, though). Suggestions on what I should do? numpy.ndarray does not have its own __init__(), just a __new__(). It's __init__() is the same as object.__init__(), which takes no arguments. [~] |3> np.ndarray.__init__ is object.__init__ True There is no need to call np.ndarray.__init__() explicitly. http://docs.scipy.org/doc/numpy/user/basics.subclassing.html#a-brief-python-primer-on-new-and-init You will also want to ask numpy questions on the numpy mailing list. http://www.scipy.org/Mailing_Lists Personally, I recommend not subclassing ndarray at all. It rarely works out well. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: lambda in list comprehension acting funny
On 7/11/12 9:21 PM, John Ladasky wrote: Exactly. It's threads like these which remind me why I never use lambda. I would rather give a function an explicit name and adhere to the familiar Python syntax, despite the two extra lines of code. I don't even like the name "lambda". It doesn't tell you what it is (unless you're John McCarthy), a function that you won't re-use and so you don't really need to give it a persistent name. I haven't seen any lambdas in any Python library code, or in any of the third-party modules I use (numpy, matplotlib, Biopython). Do they exist? Because I have not been forced to do so, I haven't retained a space in the top drawer of my programming brain for lambda. I count 162 uses in the Python standard library, 69 uses in numpy, 108 in matplotlib, and 238 uses in Biopython. Adding in the unit tests for each would add significantly to those counts. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: type(None)()
On 8/16/12 2:56 PM, Ian Kelly wrote: On Thu, Aug 16, 2012 at 6:47 AM, Hans Mulder wrote: On 8/08/12 04:14:01, Steven D'Aprano wrote: NoneType raises an error if you try to create a second instance. bool just returns one of the two singletons (doubletons?) again. py> type(None)() Traceback (most recent call last): File "", line 1, in TypeError: cannot create 'NoneType' instances Why is that? Why doesn't it just return an existing instance of the type, like bool, int, str and other built-in non-mutable types do? Because unlike those other types there is no use case for that. It's simpler to raise an error. What are the use cases for the empty-argument versions of bool(), int(), float(), and str()? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: [CGI] Why is HTML not rendered?
On 8/17/12 2:27 PM, Gilles wrote:
Hello
I'm learning how to call Python scripts through the different
solutions available.
For some reason, this CGI script that I found on Google displays the
contents of the variable but the HTML surrounding it is displayed
as-is by the browser instead of being rendered:
--
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
# enable debugging
import cgitb
cgitb.enable()
import cgi
form = cgi.FieldStorage()
# get a value from the form
value = form.getvalue("dummy")
print "Content-Type: text/plain;charset=utf-8"
print
# print a document
print "You typed: %s" % (
cgi.escape(value),
)
--
Here's the output:
--
You typed: test
--
Could this be due to the script itself, or some server configuration?
By using "Content-Type: text/plain", you told the browser to treat it like plain
text instead of HTML. Use text/html instead.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
Re: Private methods
On 10/9/12 2:59 PM, D.M. Procida wrote: Mark Lawrence wrote: On 09/10/2012 14:24, D.M. Procida wrote: What exactly is the point of a private method? Why or when would I want to use one? Daniele Hardly a Python question but using a search engine could have got you here, and rather faster :) http://stackoverflow.com/questions/2620699/why-private-methods-in-the-ob ject-oriented Thanks. Sometimes I prefer to talk to real people on Usenet than do web searches. Just my preference. That's understandable, but the real people on Usenet who will answer your questions usually prefer that you do a web search first, for a variety of reasons. http://www.catb.org/esr/faqs/smart-questions.html#before Anyway, one of the answers on that page explains that public methods are interfaces to a class, that other things might rely on, and private ones are for its own internal logic, that other things don't need to care about. In Python, using an underscore is simply a convention to note that a method is private - it doesn't actually hide it from other things - correct? This is correct. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Private methods
On 10/10/12 12:51 AM, Steven D'Aprano wrote: On Tue, 09 Oct 2012 11:08:13 -0600, Ian Kelly wrote: On Tue, Oct 9, 2012 at 8:08 AM, Demian Brecht wrote: A single underscore semantically means private. A double underscore will name mangle the function such that it's only accessible strictly by name through the class that it's define in. Note that you *can* still access it if you understand how name mangling works. Nothing in Python is truly private. I tend to view name mangling as being more for avoiding internal attribute collisions in complex inheritance structures than for designating names as private. Really? I tend to view name mangling as a waste of time, and complex inheritance structures as something to avoid. Whatever you may think of the use case, it was the motivating reason why it was put into the language: http://docs.python.org/reference/lexical_analysis.html#reserved-classes-of-identifiers -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: serialization and versioning
On 10/12/12 11:42 AM, Neal Becker wrote: I wonder if there is a recommended approach to handle this issue. Suppose objects of a class C are serialized using python standard pickling. Later, suppose class C is changed, perhaps by adding a data member and a new constructor argument. It would see the pickling protocol does not directly provide for this - but is there a recommended method? I could imagine that a class could include a class __version__ property that might be useful - although I would further expect that it would not have been defined in the original version of class C (but only as an afterthought when it became necessary). You don't need to add anything to the actual attributes on the class. Just define your __getstate__() dict to include a '__version__' item. You can treat the absence of '__version__' in the dict that __setstate__() receives as implying "version 0". -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: [on topic] Re: readline trick needed
On 10/16/12 12:27 PM, Steven D'Aprano wrote: On Tue, 16 Oct 2012 10:30:01 +0200, Peter Otten wrote: Steven D'Aprano wrote: I'm working with the readline module, and I'm trying to set a key combination to process the current command line by calling a known function, *and* enter the command line. Something along the lines of: * execute function spam() in some context where it can access the current command line as a string * enter the command line Function spam() may or may not modify the command line. (P.S. I'm aware of IPython, I want to get this working in the standard CPython interpreter.) If IPython does what you want why don't you have a look at the source? Well, I was hoping for a pure Python solution, rather than having to troll through who knows how many thousands of lines of code in a language I can barely read. Are you confusing IPython, the pure Python REPL for CPython, for IronPython, the C# implementation of Python? https://github.com/ipython/ipython -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: OT Questions
On 10/17/12 11:05 PM, Dwight Hutto wrote: On Wed, Oct 17, 2012 at 5:31 AM, Chris Angelico wrote: On Wed, Oct 17, 2012 at 5:27 PM, Dwight Hutto wrote: On Wed, Oct 17, 2012 at 2:06 AM, Demian Brecht wrote: I can't ascertain what your strengths are as I don't work with you on a daily basis (one of the many benefits of working with people smarter than you ;)). Doubt that, unless they have 160+ I.Q.'s(been seeing psychiatrists since I was 13). I'm very secure in my childlike intellectualism. A high IQ just proves ability to score well on IQ tests. On the whole, your statement strikes me as reminiscent of Sheldon Cooper's insistence that "I'm not crazy, my mother had me tested!". Someone insulted my intelligence, and stated how they worked with much smarter people...this was just a confidence statement that I'm intelligent as well, so don't get uppity with me. No, you misread his sentence. That's not at all what he was saying. He was saying that one of the benefits that a person may get from working with people smarter than said person is that they can ascertain said person's strengths. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Aggressive language on python-list
On 10/18/12 6:43 AM, David Hutto wrote: On Thu, Oct 18, 2012 at 1:29 AM, Steven D'Aprano wrote: David, While I acknowledge and appreciate your efforts to be less aggressive on this list, I think you have crossed a line by forwarding the contents of an obviously personal email containing CLEARLY PRIVATE MATTERS to a public list without permission, without even anonymising it. I get that it was a in a thread, and we;'re always told to respond all, unless otherwise asked, and they didn't directly ask, so I responded back to the list like the etiquette dictates. I know that you have apologized for this later in the email, and I appreciate that, but I would like to explicitly state some of the expectations of etiquette for this list. I don't mean to chastise excessively. I'm afraid that you were either misinformed, or you misinterpreted what you were told. When someone sends you an email that is *only addressed to you*, you should not forward that to the list without getting explicit permission. It is possible that someone just forgot to include the list, but it's also quite likely that they meant it only for you, particularly when it is of a more personal nature. Etiquette dictates that you should not assume that they meant to include the list. If you are in doubt, you must ask. This rule trumps others if you think there is a conflict in interpretation. If you do make a private response, it is always a good idea to explicitly state so, but the lack of such a statement is not an excuse for the recipient to make the email public. The default assumption must be that they meant to send it to exactly those people they actually sent it to. Thank you for listening. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: turn list of letters into an array of integers
On 10/24/12 1:03 PM, 8 Dihedral wrote: The list in python is a list of valid python objects. For the number crunching part, please use arrays in numarray and scipy. Your bot's database is laughably out of date. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Negative array indicies and slice()
On 10/31/12 8:16 PM, Andrew Robinson wrote: On 10/31/2012 02:20 PM, Ian Kelly wrote: On Wed, Oct 31, 2012 at 7:42 AM, Andrew Robinson wrote: Then; I'd note: The non-goofy purpose of slice is to hold three data values; They are either numbers or None. These *normally* encountered values can't create a memory loop. So, FOR AS LONG, as the object representing slice does not contain an explicit GC pair; I move that we mandate (yes, in the current python implementation, even as a *fix*) that its named members may not be assigned any objects other than None or numbers eg: Lists would be forbidden Since functions, and subclasses, can be test evaluated by int( the_thing_to_try ) and *[] can too, generality need not be lost for generating nothing or numbers. PEP 357 requires that anything implementing the __index__ special method be allowed for slicing sequences (and also that __index__ be used for the conversion). For the most part, that includes ints and numpy integer types, but other code could be doing esoteric things with it. I missed something... (but then that's why we're still talking about it...) Reading the PEP, it notes that *only* integers (or longs) are permitted in slice syntax. (Overlooking None, of course... which is strange...) The PEP gives the only exceptions as objects with method "__index__". Automatically, then, an empty list is forbidden (in slice syntax). However, What you did, was circumvent the PEP by passing an empty list directly to slice(), and avoiding running it through slice syntax processing. Why do you think it is forbidden by the syntax? [~] |1> class A(object): ..> def __getitem__(self, key): ..> return key ..> [~] |2> a = A() [~] |3> a[[]:] slice([], None, None) The PEP is a little unclear and refers to a state of the Python interpreter that no longer exists. At the time, I think __getslice__() was still not deprecated, and it did require ints (or after the PEP, __index__able objects). __getslice__() is now deprecated in favor of __getitem__() where you can interpret slice objects with arbitrary objects to your heart's content. Arbitrary objects *are* definitely allowed by the slice syntax (how could the syntax know what is an int and what is another kind of object?). Most objects that interpret slices, especially the builtin sequence types, do require __index__able objects (or None). So, what's the psychology behind allowing slice() to hold objects which are not converted to ints/longs in the first place? In numpy, we (ab)use this freedom for some convenient notation in special objects. We have a couple of grid-making convenience objects: [~] |5> numpy.mgrid[1.5:2.5:0.1] array([ 1.5, 1.6, 1.7, 1.8, 1.9, 2. , 2.1, 2.2, 2.3, 2.4]) This syntax uses the start:stop:step notation to make a float range. If we use an imaginary integer in the "step" slot, mgrid will interpret it as the number of items requested instead of the step. [~] |6> numpy.mgrid[1.5:2.5:11j] array([ 1.5, 1.6, 1.7, 1.8, 1.9, 2. , 2.1, 2.2, 2.3, 2.4, 2.5]) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: how to perform word sense disambiguation?
On 11/1/12 5:14 AM, nachiket wrote: an initial part of my project involves assigning sense to each word in sentence. I came across this tool called wordnet. do share your views You can get access to Wordnet and a wide variety of useful tools in NLTK: http://nltk.org/ http://nltk.org/book/ch06.html http://nltk.org/api/nltk.classify.html -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Organisation of python classes and their methods
On 11/2/12 8:20 AM, Martin Hewitson wrote:
On 2, Nov, 2012, at 09:00 AM, Peter Otten <__pete...@web.de> wrote:
Martin Hewitson wrote:
Dear list,
I'm relatively new to Python and have googled and googled but haven't
found a reasonable answer to this question, so I thought I'd ask it here.
I'm beginning a large Python project which contains many packages, modules
and classes. The organisation of those is clear to me.
Now, the classes can contain many methods (100s of data analysis methods)
which operate on instances of the class they belong to. These methods can
be long and complex. So if I put these methods all in the module file
inside the class, the file will get insanely long. Reading on google, the
answer is usually "refactor", but that really doesn't make sense here.
It's just that the methods are many, and each method can be a long piece
of code. So, is there a way to put these methods in their own files and
have them 'included' in the class somehow? I read a little about mixins
but all the solutions looked very hacky. Is there an official python way
to do this? I don't like having source files with 100's of lines of code
in, let alone 1000's.
You googled, found the right answer ("refactor"), didn't like it and are now
looking to cure the symptoms of the original problem?
Seriously, a good editor can deal with a long source file, but a class with
hundreds of methods will bring trouble to any old brain.
Well, here we disagree. Suppose I have a class which encapsulates time-series
data. Below is a list of the absolute minimum methods one would have to process
that data. That's close to 100 already before even having any specialised
methods for dealing with the data. Each of these methods will have maybe 20
lines of documentation. That's 2000 lines already. And what if someone wants to
extend that class to add their own processing methods? It would a maintenance
nightmare for them to edit the actual class file, which they would then have to
repeat each time a new version of the 'official' class file is released.
Do not make them methods of the time-series class. Make functions that take a
time-series object. Then you can organize the functions in separate modules to
your heart's content and import them all into a single convenience namespace.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
Re: Negative array indicies and slice()
On 11/2/12 8:57 AM, Andrew Robinson wrote: Hi Ian, I apologize for trying your patience with the badly written code example. All objects were meant to be ThirdParty(), the demo was only to show how a slice() filter could have been applied for the reasons PEP357 made index() to exist. eg: because numpy items passed to __getitems__ via slice syntax [::] were illegal values. PEP 357 is the one who specifically mentioned Numpy types -- which is the only reason I used the name in the example; I could have just as well used a string. I am fully aware of what numpy does -- I have used it; modified the fortran interfaces underneath, etc. The index() method, however, affects *all* list objects in Python, not just Numpy's -- correct? Please forget that PEP 357 mentions slices at all. The motivation for the __index__() method (not index()) goes far beyond slices. I'm not really sure why they are given such a prominent place in the PEP. Let me try to lay out the motivation more clearly. numpy has objects that represent integers but cannot be subclasses of the Python int or long objects because their internal representations are different. These are the width-specific types: uint8, int16, int64, etc. Before __index__() was introduced, all indexing operations in the builtin Python sequence types strictly checked for int or long objects and rejected other objects. We wanted to provide a generic method that third party types could implement to say, "Yes, I really am an integer, here is my value in a canonical representation you can understand." We could not use __int__() for this purpose because it has additional semantics, namely conversion from not-integers to integers. This is why floats are mentioned; they do not generally represent integers but they do define an __int__() method for their conversion to ints via the floor() function. Generally, they should be rejected as indices. With the __index__() method, we have a solution: int16 and the rest get __index__() methods and float doesn't. This is used where an integer index or offset is needed, not just in slices. List indices, file.seek(), mmap.mmap(), etc. The change to use PyIndex_Check() instead of PyInt_Check() was not very difficult or extensive. Even if you were to change the slicing API for your other reasons, __index__() would still be needed. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Organisation of python classes and their methods
On 11/2/12 10:21 AM, Peter Otten wrote: Martin Hewitson wrote: On 2, Nov, 2012, at 09:40 AM, Mark Lawrence wrote: 20 lines of documentation per method? As far as I'm concerned that's not a smell, that's a stink. Wow, I don't think I've ever been criticised before for writing too much documentation :) I guess we have different end users. This is not a set of classes for other developers to use: it's a set of classes which creates a data analysis environment for scientists to use. They are not programmers, and expect the algorithms to be documented in detail. While I would never discourage thorough documentation you may be better off with smaller docstrings and the details in an external document. Python projects typically use rst-files processed by sphinx. http://pypi.python.org/pypi/Sphinx/ In the science/math community, we tend to build the Sphinx API reference from the thorough, authoritative docstrings. We like having complete docstrings because we are frequently at the interactive prompt. We tend to have broad APIs, so having a single source of documentation and not repeating ourselves is important. http://docs.scipy.org/doc/numpy/reference/index.html http://docs.scipy.org/doc/scipy/reference/index.html http://www.sagemath.org/doc/reference/ http://docs.sympy.org/0.7.2/modules/index.html http://scikit-learn.org/stable/modules/classes.html -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Organisation of python classes and their methods
On 11/2/12 10:48 AM, Mark Lawrence wrote: On 02/11/2012 08:45, Martin Hewitson wrote: On 2, Nov, 2012, at 09:40 AM, Mark Lawrence wrote: On 02/11/2012 08:08, Martin Hewitson wrote: Even if one takes reasonable numbers: 20 methods, each method has 20 lines of documentation, then we immediately have 400 lines in the file before writing a line of code. It would seem much more natural to me to have these methods in their own file, grouped nicely in sub-directories. But it seems this is not the python way. Sigh. Thanks for your thoughts, Martin 20 lines of documentation per method? As far as I'm concerned that's not a smell, that's a stink. Wow, I don't think I've ever been criticised before for writing too much documentation :) I guess we have different end users. This is not a set of classes for other developers to use: it's a set of classes which creates a data analysis environment for scientists to use. They are not programmers, and expect the algorithms to be documented in detail. Martin You've completely missed the point. 99% of the time if you can't write down what a method does in at most half a dozen lines, the method is screaming out to be refactored. Rightly or wrongly you've already rejected that option, although I suspect that rightly is nearer the mark in this case on the grounds that practicality beats purity. You've completely missed the context. These are not really complicated methods doing lots of things all at once such that can be refactored to simpler methods. The docstrings are not just glorified comments for other developers reading the source code. They are the online documentation for non-programmer end-users who are using the interactive prompt as an interactive data analysis environment. Frequently, they not only have to describe what it's doing, but also introduce the whole concept of what it's doing, why you would want to do such a thing, and provide examples of its use. That's why they are so long. For example: http://docs.scipy.org/doc/numpy/reference/generated/numpy.fft.fft.html -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: word sense disambiguation
On 11/2/12 10:49 AM, nachiket wrote: hello, do you know how to perform word sense disambiguation. Input:- sentence Output:- Sense tagged words. You've asked this already, and I have pointed you to NLTK. http://nltk.org/ http://nltk.org/book/ch06.html http://nltk.org/api/nltk.classify.html -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Proper place for everything
On 11/2/12 11:20 AM, Jason Benjamin wrote: Anybody know of the appropriate place to troll and flame about various Python related issues? I'm kind of mad about some Python stuff and I need a place to vent where people may or may not listen, but at at least respond. Thought this would be a strange question, but I might as well start somewhere. There are plenty of good, free blog hosting options. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Web Frameworks Excessive Complexity
On 20/11/2012 17:41, Andriy Kornatskyy wrote: Cyclomatic (or conditional) complexity is a metric used to indicate the complexity of a source code. Excessive complexity is something that is beyond recommended level of 10 (threshold that points to the fact the source code is too complex and refactoring is suggested). Here is a list of web frameworks examined: bottle, cherrypy, circuits, django, flask, pyramid, pysi, tornado, turbogears, web.py, web2py and wheezy.web. Cyclomatic complexity tells you nothing that counting lines of code doesn't already. http://www.scirp.org/Journal/PaperInformation.aspx?paperID=779 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Web Frameworks Excessive Complexity
On 20/11/2012 19:46, Andriy Kornatskyy wrote: Robert, Thank you for the comment. I do not try relate CC with LOC. Instead pointing to excessive complexity, something that is beyond recommended threshold, a subject to refactoring in respective web frameworks. Those areas are likely to be potential source of bugs (e.g. due to low code coverage with unit tests) thus have certain degree of interest to both: end users and framework developers. Did you read the paper? I'm not suggesting that you compare CC with LoC; I'm suggesting that you don't use CC as a metric at all. The research is fairly conclusive that CC doesn't measure what you think it measures. The source of bugs is not excessive complexity in a method, just excessive lines of code. LoC is much simpler, easier to understand, and easier to correct than CC. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Web Frameworks Excessive Complexity
On 20/11/2012 20:22, Andriy Kornatskyy wrote: Robert, I respect your point of view and it definitely make sense to me. I personally do not have a problem to understand CC but agree, method LoC is easier to understand. Regardless the path your choose in your next refactoring (based on method CC, LoC) it gives your better product. No, refactoring based on CC does not give you a better product, except by accident. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Web Frameworks Excessive Complexity
On 21/11/2012 01:43, Steven D'Aprano wrote: On Tue, 20 Nov 2012 20:07:54 +0000, Robert Kern wrote: The source of bugs is not excessive complexity in a method, just excessive lines of code. Taken literally, that cannot possibly the case. def method(self, a, b, c): do_this(a) do_that(b) do_something_else(c) def method(self, a, b, c): do_this(a); do_that(b); do_something_else(c) It *simply isn't credible* that version 1 is statistically likely to have twice as many bugs as version 2. Over-reliance on LOC is easily gamed, especially in semicolon languages. Logical LoC (executable LoC, number of statements, etc.) is a better measure than Physical LoC, I agree. That's not the same thing as cyclomatic complexity, though. Also, the relationship between LoC (of either type) and bugs is not linear (at least not in the small-LoC regime), so you are certainly correct that it isn't credible that version 1 is likely to have twice as many bugs as version 2. No one is saying that it is. Besides, I think you have the cause and effect backwards. I would rather say: The source of bugs is not lines of code in a method, but excessive complexity. It merely happens that counting complexity is hard, counting lines of code is easy, and the two are strongly correlated, so why count complexity when you can just count lines of code? No, that is not the takeaway of the research. More code correlates with more bugs. More cyclomatic complexity also correlates with more bugs. You want to find out what causes bugs. What the research shows is that cyclomatic complexity is so correlated with LoC that it is going to be very difficult, or impossible, to establish a causal relationship between cyclomatic complexity and bugs. The previous research that just correlated cyclomatic complexity to bugs without controlling for LoC does not establish the causal relationship. Keep in mind that something like 70-80% of published scientific papers are never replicated, or cannot be replicated. Just because one paper concludes that LOC alone is a better metric than CC doesn't necessary make it so. But even if we assume that the paper is valid, it is important to understand just what it says, and not extrapolate too far. This paper is actually a replication. It is notable for how comprehensive it is. The paper makes various assumptions, takes statistical samples, and uses models. (Which of course *any* such study must.) I'm not able to comment on whether those models and assumptions are valid, but assuming that they are, the conclusion of the paper is no stronger than the models and assumptions. We should not really conclude that "CC has no more predictive power than LOC". The right conclusion is that one specific model of cyclic complexity, McCabe's CC, has no more predictive power than LOC for projects written in C, C++ and Java. How does that apply to Python code? Well, it's certainly suggestive, but it isn't definitive. More so than the evidence that CC is a worthwhile measure, for Python or any language. It's also important to note that the authors point out that in their samples of code, they found very high variance and large numbers of outliers: [quote] Modules where LOC does not predict CC (or vice-versa) may indicate an overly-complex module with a high density of decision points or an overly- simple module that may need to be refactored. [end quote] So *even by the terms of this paper*, it isn't true that CC has no predictive value over LOC -- if the CC is radically high or low for the LOC, that is valuable to know. Is it? What is the evidence that excess, unpredicted-by-LoC CC causes (or even correlates with) bugs? The paper points that out as a target for future research because no one has studied it yet. It may turn out to be a valid metric, but one that has a very specific utility: identifying a particular hotspot. Running CC over whole projects to compare their "quality", as the OP has done, is not a valid use of even that. LoC is much simpler, easier to understand, and easier to correct than CC. Well, sure, but do you really think Perl one-liners are the paragon of bug-free code we ought to be aiming for? *wink* No, but introducing more statements and method calls to avoid if statements isn't either. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Web Frameworks Excessive Complexity
On 21/11/2012 11:02, Andriy Kornatskyy wrote: Robert, You would never get a better product by accident. The meaning of better product might differ from team to team but you can not ignore excessive complexity. Earlier or later you get back to that code and refactor it, thus existence of such fact was driven by your intention to make it a bit better (easier to understand, to support, to cover with unit tests, etc), with a team of 20 heads you can get even further: the whole team adherence. So those drops make the overall picture better. This is what you, as a software developer, donate to what the final better product become. I think you may be misinterpreting the English idiom. I don't mean that your finger slips and randomly types out better code. I mean that by focusing on CC as a metric for improvement, you may very well end up improving the code, but it's not because you reduced the CC of the code. It's because of all of those *other* things that you talk about. Those are the things that should drive your refactoring, not CC, because they actually do cause improved code. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Web Frameworks Excessive Complexity
On 21/11/2012 12:17, Andriy Kornatskyy wrote: Agreed. I think we have pretty much the same point of view on this. All these metrics advise you... this is again depends how you look at this. If you are a new comer to a project, you usually spend some time on code review, talk to people, read docs if any. The qa tools for static code analysis give you an initial picture, how it fits with your own vision, etc. Convince or accept? No, we don't have the same point of view on this. I think that using metrics that have no evidence for their utility is a misleading distraction. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Web Frameworks Excessive Complexity
On 21/11/2012 12:47, Andriy Kornatskyy wrote: Hm... what serves an evidence purpose for you? Well-done empirical studies, like the one I gave you. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Numpy outlier removal
On 07/01/2013 15:20, Oscar Benjamin wrote: On 7 January 2013 05:11, Steven D'Aprano wrote: On Mon, 07 Jan 2013 02:29:27 +, Oscar Benjamin wrote: On 7 January 2013 01:46, Steven D'Aprano wrote: On Sun, 06 Jan 2013 19:44:08 +, Joseph L. Casale wrote: I'm not sure that this approach is statistically robust. No, let me be even more assertive: I'm sure that this approach is NOT statistically robust, and may be scientifically dubious. Whether or not this is "statistically robust" requires more explanation about the OP's intention. Not really. Statistics robustness is objectively defined, and the user's intention doesn't come into it. The mean is not a robust measure of central tendency, the median is, regardless of why you pick one or the other. Okay, I see what you mean. I wasn't thinking of robustness as a technical term but now I see that you are correct. Perhaps what I should have said is that whether or not this matters depends on the problem at hand (hopefully this isn't an important medical trial) and the particular type of data that you have; assuming normality is fine in many cases even if the data is not "really" normal. "Having outliers" literally means that assuming normality is not fine. If assuming normality were fine, then you wouldn't need to remove outliers. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: [Offtopic] Line fitting [was Re: Numpy outlier removal]
On 08/01/2013 06:35, Chris Angelico wrote: On Tue, Jan 8, 2013 at 1:06 PM, Steven D'Aprano wrote: given that weather patterns have been known to follow cycles at least that long. That is not a given. "Weather patterns" don't last for thirty years. Perhaps you are talking about climate patterns? Yes, that's what I meant. In any case, debate about global warming is quite tangential to the point about statistical validity; it looks quite significant to show a line going from the bottom of the graph to the top, but sounds a lot less noteworthy when you see it as a half-degree increase on about (I think?) 30 degrees, and even less when you measure temperatures in absolute scale (Kelvin) and it's half a degree in three hundred. Why on Earth do you think that the distance from nominal surface temperatures to freezing much less absolute 0 is the right scale to compare global warming changes against? You need to compare against the size of global mean temperature changes that would cause large amounts of human suffering, and that scale is on the order of a *few* degrees, not hundreds. A change of half a degree over a few decades with no signs of slowing down *should* be alarming. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: [Offtopic] Line fitting [was Re: Numpy outlier removal]
On 08/01/2013 20:14, Chris Angelico wrote: On Wed, Jan 9, 2013 at 2:55 AM, Robert Kern wrote: On 08/01/2013 06:35, Chris Angelico wrote: ... it looks quite significant to show a line going from the bottom of the graph to the top, but sounds a lot less noteworthy when you see it as a half-degree increase on about (I think?) 30 degrees, and even less when you measure temperatures in absolute scale (Kelvin) and it's half a degree in three hundred. Why on Earth do you think that the distance from nominal surface temperatures to freezing much less absolute 0 is the right scale to compare global warming changes against? You need to compare against the size of global mean temperature changes that would cause large amounts of human suffering, and that scale is on the order of a *few* degrees, not hundreds. A change of half a degree over a few decades with no signs of slowing down *should* be alarming. I didn't say what it should be; Actually, you did. You stated that "a ~0.6 deg increase across ~30 years [is h]ardly statistically significant". Ignoring the confusion between statistical significance and practical significance (as external criteria like the difference between the nominal temp and absolute 0 or the right criteria that I mentioned has nothing to do with statistical significance), you made a positive claim that it wasn't significant. I gave three examples. You gave negligently incorrect ones. Whether your comments were on topic or not, you deserve to be called on them when they are wrong. And as I said, this is not the forum to debate climate change; I was just using it as an example of statistical reporting. Three types of lies. FUD is a fourth. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: importing module versus using function?
On 1/31/12 3:08 PM, gujax wrote: Hi, I am confused on this quite bad!! If I have this typed in interactive python: import numpy def dummy(): y=numpy.arange(1,2,0.1) return y and then s = dummy() s array[1. , 1.1, 1.2] it works. But if I have a module called example.py, whose code is def dummy(): y=numpy.arange(1,2,0.1) return y and now if I do the following: import numpy >from example import * s=dummy() The above sequence does not work. It gives an error saying global variable numpy not defined. I understand that when I import a module it gets imported after getting compiled and therefore, the module should have a statement "import numpy" at its start. But what if I just want to use the function 'dummy'? You need to import numpy in dummy.py. When a function needs to look up a name, it goes to the module's namespace in which the function is defined, not one of the many namespaces where the function is called from. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Numeric root-finding in Python
On 2/12/12 6:41 AM, Steven D'Aprano wrote: This is only peripherally a Python problem, but in case anyone has any good ideas I'm going to ask it. I have a routine to calculate an approximation of Lambert's W function, and then apply a root-finding technique to improve the approximation. This mostly works well, but sometimes the root-finder gets stuck in a cycle. I don't have any advice for fixing your code, per se, but I would just grab mpmath and use their lambertw function: http://mpmath.googlecode.com/svn/trunk/doc/build/functions/powers.html#lambert-w-function -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Numerical Linear Algebra in arbitrary precision
On 2/17/12 6:09 AM, Tim Roberts wrote: Ken wrote: Brand new Python user and a bit overwhelmed with the variety of packages available. Any recommendation for performing numerical linear algebra (specifically least squares and generalized least squares using QR or SVD) in arbitrary precision? I've been looking at mpmath but can't seem to find much info on built in functions except for LU decomposition/solve. It is been my experience that numpy is the best place to start with requests like this, although I don't know whether it will actually solve your specific tasks: http://docs.scipy.org/doc/numpy/reference/routines.linalg.html This will not do arbitrary-precision, though. We use the double- and single-precision routines from LAPACK. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Python recursive tree, linked list thingy
On 3/8/12 9:12 AM, Enrico Franchi wrote: Wanderer wrote: How do you handle this sort of thing in Python? I believe that the best thing to do is a Union-Find algorithm. Another term this problem is finding the "connected components". Here is some code from Stefan van der Walt for this: http://mentat.za.net/source/connected_components.tar.bz2 http://mentat.za.net/cgi-bin/hgwebdir.cgi/ccomp -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Raise X or Raise X()?
On 3/12/12 10:37 AM, Jean-Michel Pichavant wrote: bvdp wrote: Which is preferred in a raise: X or X()? I've seen both. In my specific case I'm dumping out of a deep loop: try: for ... for ... for ... if match: raise StopInteration() else ... except StopInteration: print "found it" I prefer the raise X() version, it fulfils the zen of python : "Special cases aren't special enough to break the rules. There should be one-- and preferably only one --obvious way to do it." I still wonder why they've added the class raise form, on which purpose. The class raise form used to be the only way to raise exceptions. To pass an argument, there was special syntax: raise Exception, "some message" This syntax has been done away with in Python 3 in favor of regular calling conventions. Python 3 still allows bare classes, though. I also prefer to always raise instances of exceptions rather than bare exception classes. It simplifies the mental model. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: concatenate function
On 3/13/12 2:35 PM, ferreirafm wrote: Hi List, I've coded three functions that I would like to concatenate. I mean, run them one after another. The third function depends on the results of the second function, which depends on the results of the first one. When I call one function after another, python runs them at the same time causing obvious errors messages. I've tried to call one of them from inside another but no way. Any clues are appreciated. Complete code goes here: http://ompldr.org/vZDB4OQ Just to clarify, the Python functions are indeed running consecutively, not concurrently. Your Python functions write scripts and then use subprocess.call() to make qsub (an external program) to submit those scripts to a job queue. What you are calling a "function" in your post are these scripts. Please don't call them "functions". It's confusing. Python is not running these scripts concurrently. Your job queue is. subprocess.call() will wait until qsub returns. However, qsub just submits the script to the job queue; it does not wait until the job is completed. Most qsub-using job queues can be set up to make jobs depend on the completion of other jobs. You will need to read the documentation of your job queue to figure out how to do this. Once you figure out the right arguments to give to qsub, your Python code is already more or less correct. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: concatenate function
On 3/13/12 3:59 PM, ferreirafm wrote: Hi Robert, Thanks for you kind replay and I'm sorry for my semantic mistakes. Indeed, that's what I'm doing: qsub-ing different cshell scripts. Certainly, that's not the best approach and the only problem. It's not a problem to write out a script and have qsub run it. That's a perfectly fine thing to do. You need to read the documentation for your job queue to find out the right arguments to give to qsub to make it wait until the first job finishes before executing the second job. This is not a Python problem. You just need to find the right flags to give to qsub. Alternately, you could just make a single .qsub script running all three of your programs in a single job instead of making three separate .qsub scripts. I've unsuccessfully tried to set an os.environ and call qsub from it. However, subprocess.Popen seems not accept to run "qsub" over a second program. Do you have a over come to this issue? Code goes here: http://ompldr.org/vZDB5YQ When you report a problem, you should copy-and-paste the output that you got and also state the output that you expected. I have no idea what you mean when you say "subprocess.Popen seems not accept to run "qsub" over a second program." -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: concatenate function
On 3/13/12 6:01 PM, ferreirafm wrote:
Robert Kern-2 wrote
When you report a problem, you should copy-and-paste the output that you
got and
also state the output that you expected. I have no idea what you mean when
you
say "subprocess.Popen seems not accept to run "qsub" over a second
program."
Code goes here:
http://ompldr.org/vZDB5YQ
stdout:
$ no_name.py --toplist top_percent.list
Traceback (most recent call last):
File "/home6/psloliveira/ferreirafm/bin/no_name.py", line 73, in
main()
File "/home6/psloliveira/ferreirafm/bin/no_name.py", line 68, in main
comb_slt(toplist)
File "/home6/psloliveira/ferreirafm/bin/no_name.py", line 55, in comb_slt
subprocess.Popen([cmd, options], env=qsub_env)
File "/share/apps/python/lib/python2.7/subprocess.py", line 679, in
__init__
errread, errwrite)
File "/share/apps/python/lib/python2.7/subprocess.py", line 1228, in
_execute_child
raise child_exception
OSError: [Errno 13] Permission denied
You need to use a command list like this:
['qsub', 'combine_silent.linuxgccrelease', '-database',
'/home6/psloliveira/rosetta_database/', ...]
The program to run ("qsub", not "qsub combine_silent.linuxgccrelease") and each
individual argument must be a separate string in the list. You cannot combine
them together with spaces. The reason you get a "Permission denied" error is
that it tried to find an executable file named "qsub
combine_silent.linuxgccrelease" and, obviously, could not.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
Re: Enchancement suggestion for argparse: intuit type from default
On 3/15/12 5:59 AM, Cameron Simpson wrote: On 15Mar2012 12:22, Ben Finney wrote: | Roy Smith writes: |> I'll admit I hadn't considered that, but I don't see it as a major |> problem. The type intuition could be designed to only work for types |> other than NoneType. | | −1, then. It's growing too many special cases, and is no longer simple | to describe, so that indicates it's probably a bad idea. If `type` is not supplied and `default` is present and not None, `type` shall be the type of `default`. That seems straightforward to me. It's a single sentence, easy to read and understand, and potentially saves a lot of code verbiage (gratuitous type= prarameters). I say "gratuitous" because unless `default` is a sentinel for "no option supplied", the `type` should always match type(default). Or am I wrong about that? Yes. Not all type(default) types can be called with a string to produce a valid value. Note that "type=" is really a misnomer. argparse doesn't really want a type object there; it wants a converter function that takes a string to an object. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
On 3/15/12 2:30 PM, Kiuhnm wrote: On 3/15/2012 15:23, Duncan Booth wrote: Kiuhnm wrote: BTW, aren't those ':' redundant? They are required by the grammar, but in a sense you are correct. You could modify Python's grammar to make the colons optional and still keep it unambiguous but that would make it harder for other tools (such as text editors or indeed humans) to understand. Sorry, but I can't see how it would make it harder for humans to understand. Are there particular situations you're referring to? There were usability studies done on one of Python's indentation-based ancestors, ABC. Those studies found, empirically, that having the colons helped people read and understand the code faster. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
On 3/15/12 2:55 PM, Kiuhnm wrote: On 3/15/2012 15:28, Tim Golden wrote: http://docs.python.org/faq/design.html#why-are-colons-required-for-the-if-while-def-class-statements The second one is slightly easier to read because it's syntax-highlighted. Was that on purpose? No, it's an unintended side effect. The (automated) syntax highlighting was added to the FAQ much, much later than that entry was written. The syntax highlighting tool does not recognize the first example as Python, so it does not apply Python syntax highlighting to it. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Why not use juxtaposition to indicate function application
On 3/16/12 12:45 PM, Ray Song wrote: I confess i've indulged in Haskell and found f a more readable than f(a) And why aren't functions curried (partially applied function is another function which takes the rest arguments) by default? Python isn't a strongly functional language. We just don't do partial function application all that frequently to make it a language feature. Leaving out an argument is a common enough mistake, though, and using curry-by-default would postpone the error and make for even more inscrutable error messages. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: avoid import short-circuiting
On 3/16/12 4:49 PM, Andrea Crotti wrote: I started the following small project: https://github.com/AndreaCrotti/import-tree because I would like to find out what exactly depends on what at run-time, using an import hook. It works quite well for small examples but the main problem is that once a module is imported it's added to sys.modules and then it doesn't go through the import hook anymore. I tried to mess around with sys.modules but it might not be a good idea, and it leads to easy infinite loops. Is there a good way to achieve this? I guess I could do the loop detection myself, but that should not be too hard.. You want to monkeypatch __builtin__.__import__() instead. It always gets called. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: avoid import short-circuiting
On 3/16/12 10:04 PM, Andrea Crotti wrote: On 03/16/2012 05:19 PM, Robert Kern wrote: On 3/16/12 4:49 PM, Andrea Crotti wrote: I started the following small project: https://github.com/AndreaCrotti/import-tree because I would like to find out what exactly depends on what at run-time, using an import hook. It works quite well for small examples but the main problem is that once a module is imported it's added to sys.modules and then it doesn't go through the import hook anymore. I tried to mess around with sys.modules but it might not be a good idea, and it leads to easy infinite loops. Is there a good way to achieve this? I guess I could do the loop detection myself, but that should not be too hard.. You want to monkeypatch __builtin__.__import__() instead. It always gets called. Seems like a good idea :) My first attempt failes though def full(module): from __builtin__ import __import__ ls = [] orig = __import__ def my_import(name): ls.append(name) orig(name) __import__ = my_import __import__(module) __import__ = orig return ls it imports only the first element and doesn't import the dependencies.. Any hints? You need to replace it in __builtin__. Don't forget to handle all of the arguments. import __builtin__ orig_import = __builtin__.__import__ all_imports = [] def my_import(*args, **kwds): module = orig_import(*args, **kwds) # Get the fully-qualified module name from the module object itself # instead of trying to compute it yourself. all_imports.append(module.__name__) return module __builtin__.__import__ = my_import For extra points, make a context manager that hooks and then unhooks your custom importer. You may also want to take a look at an import profiler that I once made: http://www.enthought.com/~rkern/cgi-bin/hgwebdir.cgi/import_profiler/file/tip/import_profiler.py#l23 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: avoid import short-circuiting
On 3/16/12 11:14 PM, Andrea Crotti wrote: Very nice thanks, here it is class ImportMock: def _my_import(self, *args, **kwargs): self.ls.append(args[0]) self.orig(*args, **kwargs) There's a bug here. You need to return the module object you got from calling self.orig(). By the way, you really should follow my example of getting the .__name__ from the module object instead of the argument in order to properly account for relative imports inside packages. __import__() will be passed the relative name, not the fully-qualified name. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
On 3/19/12 12:53 PM, Neil Cerutti wrote: I still think sentence fragments before a colon introducing a list often looks bad, and may be taken for an error. But it doesn't always look bad, and I didn't think about it enough. One of my English teachers suggested a rule that seems to accord (descriptively) with the uses the "look good" and "look bad" to me: don't use a colon to separate a transitive verb from its objects. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Distribution
On 3/20/12 11:21 AM, Ben Finney wrote: "prince.pangeni" writes: I am doing a simulation project using Python. In my project, I want to use some short of distribution to generate requests to a server. What is a distribution? That term already means something in Python jargon, and it doesn't match the rest of your use case. So what do you mean by “distribution”? Maybe we can find a less confusing term. Judging from the context, he means a probability distribution. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Distribution
On 3/20/12 4:31 AM, prince.pangeni wrote:
Hi all,
I am doing a simulation project using Python. In my project, I want
to use some short of distribution to generate requests to a server.
The request should have two distributions. One for request arrival
rate (should be poisson) and another for request mix (i.e. out of the
total requests defined in request arrival rate, how many requests are
of which type).
Example: Suppose the request rate is - 90 req/sec (generated using
poisson distribution)
Just a note on terminology to be sure we're clear: a Poisson *distribution*
models the number of arrivals in a given time period if the events are from a
Poisson *process* with a given mean rate. To model the inter-event arrival
times, you use an exponential distribution. If you want to handle events
individually in your simulation, you will need to use the exponential
distribution to figure out the exact times for each. If you are handling all of
the events in each second "in bulk" without regard to the exact times or
ordering within that second, then you can use a Poisson distribution.
at time t and we have 3 types of requests (i.e.
r1, r2, r2). The request mix distribution output should be similar to:
{r1 : 50 , r2 : 30 , r3 : 10} (i.e. out of 90 requests - 50 are of r1
type, 30 are of r2 type and 10 are of r3 type).
As I an new to python distribution module, I am not getting how to
code this situation. Please help me out for the same.
I am going to assume that you want to handle each event independently. A basic
strategy is to keep a time variable starting at 0 and use a while loop until the
time reaches the end of the simulation time. Increment it using a draw from the
exponential distribution each loop. Each iteration of the loop is an event. To
determine the kind of event, you will need to draw from a weighted discrete
distribution. What you want to do here is to do a cumulative sum of the weights,
draw a uniform number from 0 to the total sum, then use bisect to find the item
that matches.
import bisect
import random
# Use a seeded PRNG for repeatability. Use the methods on the Random
# object rather than the functions in the random module.
prng = random.Random(1234567890)
avg_rate = 90.0 # reqs/sec
kind_weights = [50.0, 30.0, 10.0]
kind_cumsum = [sum(kind_weights[:i+1]) for i in range(len(kind_weights))]
kind_max = kind_cumsum[-1]
max_time = 10.0 # sec
t = 0.0 # sec
events = [] # (t, kind)
while t < max_time:
dt = prng.expovariate(avg_rate)
u = prng.uniform(0.0, kind_max)
kind = bisect.bisect_left(kind_cumsum, u)
events.append((t, kind))
t += dt
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
Re: random number
On 3/26/12 8:50 AM, Grzegorz Staniak wrote:
On 26.03.2012, Steven D'Aprano wroted:
How can we generate a 6 digit random number from a given number ?
what about this?
given_number=123456
def rand_given_number(x):
... s = list(str(x))
... random.shuffle(s)
... return int(''.join(s))
...
print (rand_given_number(given_number))
653421
That's not very random. In fact, it is *terrible* as a random number
generator.
But isn't it what the OP requested, i.e. "6 digit random number
*from a given number*"? That is, a random permutation of the set
of its digits?
I would consider that to be a very odd interpretation of that request. But it
*is* an extraordinarily vague request. I'm not sure if even the OP knows what he
wants. I suspect he really wants something like a hash.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
Re: random number
On 3/26/12 10:45 AM, Nikhil Verma wrote:
Hi
Thanks Michael I want exactly wanted this. Great
def random_number(id)
...characters = list(string.ascii_lowercase +string.ascii_uppercase
+string.digits)
I used this this earlier and tried then by using choice .
This is great.
Note that the id parameter is not used by the function at all. If you call this
function multiple times with the same input, you will get different results each
time. Is that what you want? What role did you expect the id parameter to play?
On Mon, Mar 26, 2012 at 2:54 PM, Michael Poeltl mailto:michael.poe...@univie.ac.at>> wrote:
It's still not quite clear to me what role 'id' is playing ... so let's
check this one;
and Steven, who is maybe more experienced than I am will help us ufrther
>>> import random, string
>>> def random_number(id):
... characters = list(string.ascii_lowercase +
... string.ascii_uppercase +
... string.digits)
... coll_rand = []
... for i in range(6):
... random.shuffle(characters)
... coll_rand.append(characters[0])
... return ''.join(coll_rand)
...
>>> id = 5
>>> print (random_number(id))
puMHCr
>>>
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there any difference between print 3 and print '3' in Python ?
On 3/26/12 12:45 PM, redstone-c...@163.com wrote:
I know the print statement produces the same result when both of these two
instructions are executed ,I just want to know Is there any difference between
print 3 and print '3' in Python ?
Yes, there is a difference, but not much.
[~]
|6> import dis
[~]
|7> dis.disassemble(compile('print 3', '', 'exec'))
1 0 LOAD_CONST 0 (3)
3 PRINT_ITEM
4 PRINT_NEWLINE
5 LOAD_CONST 1 (None)
8 RETURN_VALUE
[~]
|8> dis.disassemble(compile('print "3"', '', 'exec'))
1 0 LOAD_CONST 0 ('3')
3 PRINT_ITEM
4 PRINT_NEWLINE
5 LOAD_CONST 1 (None)
8 RETURN_VALUE
As you can see, the only difference is in the first instruction. Both of these
put the object that you specified by the literal onto the stack. The difference
is that one is the int object specified by the literal 3 and the other is the
str object specified by the literal "3". Both of these objects happen to give
the same __str__ output, so that's what gets printed.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
Re: Puzzled by FiPy's use of "=="
On 3/26/12 12:47 PM, André Roberge wrote: In FiPy (a finite volume PDE solver), equations are "magically" set up as eqX = TransientTerm() == ExplicitDiffusionTerm(coeff=D) and solved via eqX.solve(...) How can eqX be anything than True or False?... This must be via a redefinition of "==" but I can't see how that is done. I did look at many of the source files, thinking that it must be via a redefinition of "__eq__" somewhere but with no luck. Any pointers would be appreciated. It's in the root base class Term: http://matforge.org/fipy/browser/trunk/fipy/terms/term.py#L374 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about collections.defaultdict
On 3/26/12 2:33 PM, Steven W. Orr wrote: I created a new class called CaseInsensitiveDict (by stealing from code I found on the web, thank you very much). The new class inherits from dict. It makes it so that if the key has a 'lower' method, it will always access the key using lower I'd like to change the place where I previously declared a dict self.lookup = defaultdict(list) so that the new code will allow this new dict to be used instead. But then I realized I may have painted myself into a small corner: Is there a way to use defaultdict so that I can override what *kind* of dict it will use? No. I would like the value to still be a list be default, but it seems like I can't tell defaultdict to use *my* new dict. Do I give up on defaultdict? Assuming that your CaseInsensitiveDict subclasses from dict or UserDict, it's relatively easy to make a subclass of your CaseInsensitiveDict act like a defaultdict. Just implement the __missing__(key) method appropriately (and modify the constructor to take the callable, of course). http://docs.python.org/library/stdtypes.html#dict http://docs.python.org/library/collections.html#collections.defaultdict.__missing__ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about collections.defaultdict
On 3/26/12 4:33 PM, Steven W. Orr wrote: On 3/26/2012 9:44 AM, Robert Kern wrote: On 3/26/12 2:33 PM, Steven W. Orr wrote: I created a new class called CaseInsensitiveDict (by stealing from code I found on the web, thank you very much). The new class inherits from dict. It makes it so that if the key has a 'lower' method, it will always access the key using lower I'd like to change the place where I previously declared a dict self.lookup = defaultdict(list) so that the new code will allow this new dict to be used instead. But then I realized I may have painted myself into a small corner: Is there a way to use defaultdict so that I can override what *kind* of dict it will use? No. I would like the value to still be a list be default, but it seems like I can't tell defaultdict to use *my* new dict. Do I give up on defaultdict? Assuming that your CaseInsensitiveDict subclasses from dict or UserDict, it's relatively easy to make a subclass of your CaseInsensitiveDict act like a defaultdict. Just implement the __missing__(key) method appropriately (and modify the constructor to take the callable, of course). http://docs.python.org/library/stdtypes.html#dict http://docs.python.org/library/collections.html#collections.defaultdict.__missing__ I'm not quite getting what you're telling me, but I'm sure you have the right idea. Here's the beginning of my class: class CaseInsensitiveDict(dict): def __init__(self, init=None): if isinstance(init, (dict, list, tuple)): for kk, vv in init.items(): self[self.key_has_lower(kk)] = vv It sounds like you want me to subclass defaultdict to create something like this? class CaseInsensitiveDictDef(defaultdict): def __init__(self, init=None): super(CaseInsensitiveDictDef, self).__init__(list) self.__missing__ = list I think I'm way off base. I'm not clear on what the calling sequence is for defaultdict or how to get it to use my CaseInsensitiveDict instead of regular dict. Can you help? You need to make a subclass of CaseInsensitiveDict, implement the __missing__(key) method, and override the __init__() method to take the factory function as an argument instead of data. defaultdict is just a subclass of dict that does this. class CaseInsensitiveDictDef(CaseInsensitiveDict): def __init__(self, default_factory): super(CaseInsensitiveDictDef, self).__init__() self.default_factory = default_factory def __missing__(self, key): return self.default_factory() -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Threads on google groups not on gmane?
On 3/31/12 1:47 AM, Mark Lawrence wrote: I went onto google groups to do a search and saw three threads (there may be more) that I've never seen on gmane, which I read via thunderbird on windows. The titles are "Is programming art or science", "breezypythongui: A New Toolkit for Easy GUIs in Python" and "weird behaviour: pygame plays in shell but not in script". Is anyone else seeing the same thing? I also don't see these on GMane. It's possible that they are getting caught in one of GMane's several levels of spam filtering. http://gmane.org/spam.php -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Gotcha's?
On 4/5/12 3:15 PM, John Posner wrote: On 4/4/2012 7:32 PM, Chris Angelico wrote: Don't know if it's what's meant on that page by the += operator, Yes, it is. a=([1],) a[0].append(2) # This is fine [In the following, I use the term "name" rather loosely.] The append() method attempts to modify the object whose name is "a[0]". That object is a LIST, so the attempt succeeds. a[0]+=[3] # This is not. The assignment attempts to modify the object whose name is "a". That object is a TUPLE, so the attempt fails. This might be a surprise, but I'm not sure it deserves to be called a wart. The wart is not that it fails, but that it does not fail atomically. The list inside the tuple gets modified even though an exception is raised for the statement as a whole. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On 4/20/12 8:10 PM, dmitrey wrote: I have spent some time searching for a bug in my code, it was due to different work of "is" with () and []: () is () True The empty tuple is unique, immutable, and common so the Python runtime optimizes this by reusing the same object, much like small integers are interned. This is not something your code should rely on, though. [] is [] False Empty lists are mutable and may be modified differently, so they cannot be the same object. Every time you use an empty list literal, it must create a new object. (Python 2.7.2+ (default, Oct 4 2011, 20:03:08) [GCC 4.6.1] ) Is this what it should be or maybe yielding unified result is better? If your code is relying on the difference, or a lack of one, it's buggy. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
