Re: Converting a string to a number by using INT (no hash method)
Τη Τρίτη, 22 Ιανουαρίου 2013 10:40:39 μ.μ. UTC+2, ο χρήστης John Gordon έγραψε: > In Ferrous Cranus > writes: > > > > > May i sent you my code by mail so for you see whats wrong and > > > http://superhost.gr produces error? > > > > I tried going to that address and got some error output. I noticed this > > in the error dump: > > > > 186 if cursor.rowcount == 0: > > 187 cursor.execute( '''INSERT INTO visitors(pin, host > >, hits, useros, browser, date) VALUES(%s, %s, %s, %s, %s)''', (pin, hos > >t, 1, useros, browser, date) ) > > > > The INSERT statement gives six column names but only five placeholders (%s) > > in the VALUES clause. > > > > Perhaps that's the problem? Excatly Gordon, i missed the extra placeholder(%s) when i was adding a new useros column. I also used a 5-digit number. Now my website finally works as intended. Just visit the following links plz. -- 1. http://superhost.gr 2. http://superhost.gr/?show=log 3. http://i.imgur.com/3Hcz1uP.png (this displays the database's column 'pin', a 5-digit number acting as a filepath indicator. I guess i won't be needing column 'page' anymore) 4. http://i.imgur.com/kRwzLp3.png (this is the detailed page information associated to 'pin' column indicator instead of something like '/home/nikos/public_html/index.html' Isn't it a nice solution? -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquely identifying each & every html template
Τη Τετάρτη, 23 Ιανουαρίου 2013 1:55:07 π.μ. UTC+2, ο χρήστης Michael Torrie έγραψε: > You refuse to use a database in a way that it was designed to be used. > > If you're unwilling to identify a file based on name, path, and > > contents, then you're only solution is to use a database to associate a > > particular file with an id. When you move or rename the file you have > > to update the database. That's the only answer we can give you. table counters: pin page - hosts table visitors: pin host hits useros browser date 1. Who is going to create the 'pin' ? the database automatically when i'am inserting a new record from python? 2. How's the 'pin' going to be associated with 'page' ? 3. if 'page' renames/moves then i have to manually edit the counter database table to find the specific 'page' record and upadate it by hand? > I in no way intend anything I have said to be condescending. It's just > > that those who are honestly trying to help you are getting frustrated. > > And possibly you will interpret this frustration as hostility or > > elitism, which it really is not. They are getting frustrated because they do not want to follow the logic i'am imposing. -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquely identifying each & every html template
Τη Τρίτη, 22 Ιανουαρίου 2013 11:08:57 μ.μ. UTC+2, ο χρήστης Dennis Lee Bieber έγραψε: > On Mon, 21 Jan 2013 22:49:31 -0800 (PST), Ferrous Cranus > > declaimed the following in > > gmane.comp.python.general: > > > > > You are right, it cannot be done. > > > > > > So i have 2 options . > > > > > > Either identify an .html file from its "filepath" or from its "hash". > > > > > > Which method do you advice me to utilize? > > > > The hash identifies based upon the file contents -- any edit to the > > file will result in a different hash (and if the hash system isn't large > > enough, you may even encounter collisions where two or more files have > > the same hash). However, moving or renaming the file should still > > produce the same hash. > > > > Path/name at least lets the file contents be edited. Anything that > > changes the path/name will be seen as a new file. > > > > Which condition is most useful to your needs: allowing free content > > edits while keeping the counter tied to the end URL; or tying the > > counter to a fixed page but letting the URL to that page change. The best would be to: Allow free content edits, while the URL to that page may also change. -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
Am 23.01.2013 05:06, schrieb Isaac Won: I have tried to use different interpolation methods with Scipy. My code seems just fine with linear interpolation, but shows memory error with quadratic. I am a novice for python. I will appreciate any help. > #code f = open(filin, "r") Check out the "with open(...) as f" syntax. for columns in ( raw.strip().split() for raw in f ): For the record, this first builds a sequence and then iterates over that sequence. This is not very memory-efficient, try this instead: for line in f: columns = line.strip().split() Concerning the rest of your problems, there is lots of code and the datafile missing. However, there is also too much of it, try replacing the file with generated data and remove everything from the code that is not absolutely necessary. Good luck! Uli -- http://mail.python.org/mailman/listinfo/python-list
Re: Any algorithm to preserve whitespaces?
I am in a problem.
words = line.split(' ')
preserve whitespaces but the problem is it writes an additional line
after every line.
And:
words = line.split()
works as I expect (does not adds addition line after every line) but
does not preserves whitespaces.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Increase value in hash table
On Jan 23, 3:34 pm, Chris Rebert wrote:
> On Jan 22, 2013 11:31 PM, "moonhkt" wrote:
>
>
>
>
>
>
>
>
>
>
>
> > Hi Al
>
> > I have Data file have below
>
> > Data file
> > V1
> > V2
> > V3
> > V4
> > V4
> > V3
>
> > How to using count number of data ?
>
> > Output
> > V1 = 1
> > V2 = 1
> > V3 =2
> > V4 = 2
>
> Construct a frequency table using collections.Counter:
>
> http://docs.python.org/2.7/library/collections.html#collections.Counter
What is problem for below ?
#!/usr/bin/env python
# Python hash {}
# Python Lists []
global age
karry = "ER"
k1 = "EU"
age = {}
age[karry] = 3
age[k1] = 5
def ABC():
global age
global karry
i = 0
a = "A B"
karry = a.split()
age[karry[0]] += 1
ABC()
for key in age:
print key, age[key]
Result
ex_array.py
Traceback (most recent call last):
File "ex_array.py", line 21, in
ABC()
File "ex_array.py", line 18, in ABC
age[karry[0]] += 1
KeyError: 'A'
--
http://mail.python.org/mailman/listinfo/python-list
Re: Uniquely identifying each & every html template
On Jan 23, 7:12 pm, Ferrous Cranus wrote: > They are getting frustrated because they do not want to follow the logic i'am > imposing. No, it's because you're an obvious troll and you're wasting everyone's time. Now be a good lad and sod off. -- http://mail.python.org/mailman/listinfo/python-list
Re: Any algorithm to preserve whitespaces?
Santosh Kumar wrote:
> I am in a problem.
>
> words = line.split(' ')
>
> preserve whitespaces but the problem is it writes an additional line
> after every line.
Strip off the newline at the end of the line with:
line = line.rstrip("\n")
words = line.split(" ")
--
http://mail.python.org/mailman/listinfo/python-list
Re: Uniquely identifying each & every html template
On 23/01/2013 09:37, alex23 wrote: On Jan 23, 7:12 pm, Ferrous Cranus wrote: They are getting frustrated because they do not want to follow the logic i'am imposing. No, it's because you're an obvious troll and you're wasting everyone's time. Now be a good lad and sod off. What an appalling lack of manners, it's sod off please :) -- Cheers. Mark Lawrence -- http://mail.python.org/mailman/listinfo/python-list
Re: Importing class from another file
Hi, Kevin Holleran wrote: Traceback (most recent call last): File "testing.py", line 1, in from Sub_Dir.My_Class import * ImportError: No module named Sub_Dir.My_Class Make sure, the script you execute by passing it to the python interpreter is in the parent directory of Sub_Dir. Additionaly put an empty file called __init__.py (double underscores!) in Sub_Dir if you don't already have. This is necessary to use Sub_Dir as a package and import modules from it. For more details see: http://docs.python.org/3.3/tutorial/modules.html#packages I have played around a bit with the calls (removing the My_Class in the q_api assignment to instantiate the object, etc). When using "from...import" on the module that contains the class, you can use the class without the package identifier: q_api = My_Class_Connector(string1,string2) For your information: I think beginner questions like this one should be asked on the tutor list: http://mail.python.org/mailman/listinfo/tutor -- http://mail.python.org/mailman/listinfo/python-list
Re: Retrieving the full command line
On 23/01/2013 03:58, Steven D'Aprano wrote: > Currently, if I have a package __main__.py that prints sys.argv, I get > results like this: > > steve@runes:~$ python3.3 /home/steve/python/testpackage/__main__.py ham > spam eggs > ['/home/steve/python/testpackage/__main__.py', 'ham', 'spam', 'eggs'] > > > which is correct, that's what I gave on the command line. But: > > steve@runes:~$ python3.3 -m testpackage ham spam eggs > ['/home/steve/python/testpackage/__main__.py', 'ham', 'spam', 'eggs'] > > > The second example is lying. It should say: > > ['-m testpackage', 'ham', 'spam', 'eggs'] Thanks for the input, Steven & Oscar. Apologies for the confusion over my initial example. Of course, -m runs something on sys.path, not something in the filesystem as such. I confused myself because, running on Windows where the current directory is on the path, I sit in c:\path\to and do python -mapp Now I look harder, this discussion is basically issue14208: http://bugs.python.org/issue14208 so I'll probably go and contribute over there. TJG -- http://mail.python.org/mailman/listinfo/python-list
Re: Retrieving the full command line
On 23 January 2013 03:58, Steven D'Aprano wrote: > On Wed, 23 Jan 2013 00:53:21 +, Oscar Benjamin wrote: > >> On 22 January 2013 23:46, Steven D'Aprano >> wrote: [SNIP] >>> >> The purpose of the -m option is that you can run a script that is >> located via the Python import path instead of an explicit file path. The >> idea is that if '/path/to/somewhere' is in sys.path then: >> python -m script arg1 arg2 >> is equivalent to >> python /path/to/somewhere/script.py arg1 arg2 >> >> If Python didn't modify sys.argv then 'script.py' would need to be >> rewritten to understand that sys.argv would be in a different format >> when it was invoked using the -m option. > > I don't think that it would be in a different format. Normally people > only care about sys.argv[1:], the actual arguments. argv[0], the name of > the script, already comes in multiple formats: absolute or relative paths. > > Currently, if I have a package __main__.py that prints sys.argv, I get > results like this: > > steve@runes:~$ python3.3 /home/steve/python/testpackage/__main__.py ham > spam eggs > ['/home/steve/python/testpackage/__main__.py', 'ham', 'spam', 'eggs'] > > > which is correct, that's what I gave on the command line. But: > > steve@runes:~$ python3.3 -m testpackage ham spam eggs > ['/home/steve/python/testpackage/__main__.py', 'ham', 'spam', 'eggs'] > > > The second example is lying. It should say: > > ['-m testpackage', 'ham', 'spam', 'eggs'] I don't know why you would expect this. I imagined that you would want ['-m', 'testpackage', 'ham', 'spam', 'eggs'] If the two were combined into one string I would expect it to at least be a valid argument list: ['-mtestpackage', 'ham', 'spam', 'eggs'] > > > If you are one of the few people who care about argv[0], then you are > already dealing with the fact that the name of the executable script is > not always an absolute path and therefore can vary greatly from one call > to another. Hell, if you are on a system with soft links, the name of the > script in the command line is not even necessarily the name of the > module. So there's not much more effort involved in dealing with one > extra case: Unless I've missed something sys.argv[0] is always a valid path to the script. Whether it is absolute or not shouldn't matter. For imported modules the path is available from __name__. For a script that is executed rather than imported __name__ == "__main__" but the path is accessible from sys.argv[0]. If you are one of those people who cares about sys.argv[0] then this is probably the value that you wanted it to contain. If it were important for sys.argv to show how exactly the script was located and executed, then why not also include the 'python3.3' command line argument (the real argv[0])? sys.argv emulates the argv that e.g. a C program would get. The real command line used is not exactly the same since a Python script is not a directly executable binary, so Python processes the argument list before passing it through. In the OP's case, the script is never invoked without -m so the following works: cmdline = [sys.executable, '-m', __package__] + sys.argv[1:] In the more general case it would be better to have an API specifically for this purpose. Oscar -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On 23 January 2013 08:55, Ulrich Eckhardt wrote: > Am 23.01.2013 05:06, schrieb Isaac Won: > >> I have tried to use different interpolation methods with Scipy. My >> code seems just fine with linear interpolation, but shows memory >> error with quadratic. I am a novice for python. I will appreciate any >> help. > [SNIP] > > > Concerning the rest of your problems, there is lots of code and the datafile > missing. However, there is also too much of it, try replacing the file with > generated data and remove everything from the code that is not absolutely > necessary. Also please copy paste the actual error message rather than paraphrasing it. Oscar -- http://mail.python.org/mailman/listinfo/python-list
Re: Increase value in hash table
On 23 January 2013 07:26, moonhkt wrote:
> Hi Al
>
> I have Data file have below
>
> Data file
> V1
> V2
> V3
> V4
> V4
> V3
>
> How to using count number of data ?
>
> Output
> V1 = 1
> V2 = 1
> V3 =2
> V4 = 2
>
>
>
> # Global Veriable
> printque = {}
> in def have below
>
> printque[val] = printque[val] + 1
>
> I have below error
> File "xprintlogchk.py", line 78, in chklog
> printque[val] = printque[val] + 1
> KeyError: 'nan'
You can't retrieve the value of printque[val] if you haven't yet added
an entry with the key val to the dict. Try this:
if val not in printque:
printque[val] = 1
else:
printque[val] = printque[val] + 1
Oscar
--
http://mail.python.org/mailman/listinfo/python-list
Re: Uniquely identifying each & every html template
Τη Τετάρτη, 23 Ιανουαρίου 2013 11:37:29 π.μ. UTC+2, ο χρήστης alex23 έγραψε: > On Jan 23, 7:12 pm, Ferrous Cranus wrote: > > > They are getting frustrated because they do not want to follow the logic > > i'am imposing. > > > > No, it's because you're an obvious troll and you're wasting everyone's > > time. > > > > Now be a good lad and sod off. 1. I'am a troll because i want to generate a solution in a specific way? 2. Am' i not wasting my time trying to reply to everybody? -- http://mail.python.org/mailman/listinfo/python-list
Re: Any algorithm to preserve whitespaces?
On 01/23/2013 04:20 AM, Santosh Kumar wrote:
I am in a problem.
words = line.split(' ')
preserve whitespaces but the problem is it writes an additional line
after every line.
Think about what you said. It might be clearer if you wrote:
"but the problem is it doesn't strip off the newline (which is whitespace)."
You might want to fix it by doing an rstrip(), as Peter said, or you
might want to check if the last character is \n, and delete it if so.
Or you might want to fix the other logic where you use the reconstituted
line, making sure it doesn't add an extra newline to a line that already
has one.
Best answer depends on whether there might be other whitespace at the
end of the line, and on whether you consider the newline part of the
last field on the line.
Chances are that Peter's response is the one you want, but I had to
point out that without a spec, we're really just guessing. For another
example, suppose that some of the words in the file are separated by
tabs. If so, perhaps you'd better rethink the whole split logic.
--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list
Re: Increase value in hash table
On Wed, 23 Jan 2013 10:12:25 +, Oscar Benjamin wrote: > You can't retrieve the value of printque[val] if you haven't yet added > an entry with the key val to the dict. Try this: > > if val not in printque: > printque[val] = 1 > else: > printque[val] = printque[val] + 1 Another way of writing that is: printque[val] = printque.get(val, 0) + 1 -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Understanding while...else...
they wouldnt be equivalent if #staff in version 1 did not cointain "break" statement and this is common mistake On Wed, Jan 23, 2013 at 1:39 AM, Oscar Benjamin wrote: > On 22 January 2013 23:41, Terry Reedy wrote: > > On 1/22/2013 3:09 PM, Ethan Furman wrote: > >> > >> On 01/22/2013 09:44 AM, Terry Reedy wrote: > >>> > [SNIP] > >>> The else clause is executed if and when the condition is false. > >>> Now use a real Python while statement to do the *same > >>> thing*. > >>> > >>> while n > 0: > >>>n -= 1 > >>> else: > >>>n = None > >> > >> > >> I understand how it works (although it did take a while for it to sink > >> in); my gripe, and probably why it is misunderstood so often, is that > >> nine times out of ten when I /want/ to use a while-else or for-else I > >> only want the true/false check /once/, at the beginning of the loop. > > > > > > I do not understand what you are saying. There already is only one > > true/false check, at the beginning of the loop. If you only want the > check > > *performed* once, you would use if-else. But I presume you know this. > > I think he meant that he would use the else clause more often if it > had the semantics so that the two blocks below were equivalent: > > # Version 1 > while condition: > # stuff > else: > # other stuff > > # Version 2 > if condition: > while condition: > # stuff > else: > # other stuff > > So he wants a convenient way to execute code only if the loop > performed zero iterations. I think that often when people are confused > about the else clause on while loops it is because they expect this > behaviour (which would also be useful). The same confusion arises with > for loops where people expect the else clause to execute if the > iterable was empty so that these would be equivalent: > > # Version 1 > for x in iterable: > # stuff > else: > # other stuff > > # Version 2 > iterated = False > for x in iterable: > iterated = True > # stuff > if not iterated: > # other stuff > > > Oscar > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re: Understanding while...else...
> > # Version 1 > while condition: > # stuff > else: > # other stuff > > # Version 2 > if condition: > while condition: > # stuff > else: > # other stuff > they wouldnt be equivalent if #staff in version did not cointain "break" statement and this is common mistake On Wed, Jan 23, 2013 at 12:03 PM, René Klačan wrote: > they wouldnt be equivalent if #staff in version 1 did not cointain "break" > statement and this is common mistake > > > On Wed, Jan 23, 2013 at 1:39 AM, Oscar Benjamin < > oscar.j.benja...@gmail.com> wrote: > >> On 22 January 2013 23:41, Terry Reedy wrote: >> > On 1/22/2013 3:09 PM, Ethan Furman wrote: >> >> >> >> On 01/22/2013 09:44 AM, Terry Reedy wrote: >> >>> >> [SNIP] >> >>> The else clause is executed if and when the condition is false. >> >>> Now use a real Python while statement to do the *same >> >>> thing*. >> >>> >> >>> while n > 0: >> >>>n -= 1 >> >>> else: >> >>>n = None >> >> >> >> >> >> I understand how it works (although it did take a while for it to sink >> >> in); my gripe, and probably why it is misunderstood so often, is that >> >> nine times out of ten when I /want/ to use a while-else or for-else I >> >> only want the true/false check /once/, at the beginning of the loop. >> > >> > >> > I do not understand what you are saying. There already is only one >> > true/false check, at the beginning of the loop. If you only want the >> check >> > *performed* once, you would use if-else. But I presume you know this. >> >> I think he meant that he would use the else clause more often if it >> had the semantics so that the two blocks below were equivalent: >> >> # Version 1 >> while condition: >> # stuff >> else: >> # other stuff >> >> # Version 2 >> if condition: >> while condition: >> # stuff >> else: >> # other stuff >> >> So he wants a convenient way to execute code only if the loop >> performed zero iterations. I think that often when people are confused >> about the else clause on while loops it is because they expect this >> behaviour (which would also be useful). The same confusion arises with >> for loops where people expect the else clause to execute if the >> iterable was empty so that these would be equivalent: >> >> # Version 1 >> for x in iterable: >> # stuff >> else: >> # other stuff >> >> # Version 2 >> iterated = False >> for x in iterable: >> iterated = True >> # stuff >> if not iterated: >> # other stuff >> >> >> Oscar >> -- >> http://mail.python.org/mailman/listinfo/python-list >> > > -- http://mail.python.org/mailman/listinfo/python-list
Re: Increase value in hash table
On Wed, 23 Jan 2013 10:12:25 +, Oscar Benjamin wrote: > You can't retrieve the value of printque[val] if you haven't yet added > an entry with the key val to the dict. Try this: > > if val not in printque: > printque[val] = 1 > else: > printque[val] = printque[val] + 1 Another way of writing that is: printque[val] = printque.get(val, 0) + 1 -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Increase value in hash table
On Wed, 23 Jan 2013 10:12:25 +, Oscar Benjamin wrote: > You can't retrieve the value of printque[val] if you haven't yet added > an entry with the key val to the dict. Try this: > > if val not in printque: > printque[val] = 1 > else: > printque[val] = printque[val] + 1 Another way of writing that is: printque[val] = printque.get(val, 0) + 1 -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Increase value in hash table
On Wed, 23 Jan 2013 10:12:25 +, Oscar Benjamin wrote: > You can't retrieve the value of printque[val] if you haven't yet added > an entry with the key val to the dict. Try this: > > if val not in printque: > printque[val] = 1 > else: > printque[val] = printque[val] + 1 Another way of writing that is: printque[val] = printque.get(val, 0) + 1 -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Multiple postings [was Re: Increase value in hash table]
Steven D'Aprano wrote: [snip content] Arrgggh, it's happened again. Sorry for the multiple posts folks, I *swear* I only sent it once. Trying this time with a different news client. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Understanding while...else...
On 23 January 2013 11:03, René Klačan wrote: > On Wed, Jan 23, 2013 at 1:39 AM, Oscar Benjamin > wrote: You missed off an important piece of context in your post: >> I think he meant that he would use the else clause more often if it >> had the semantics so that the two blocks below were equivalent: The key word in that sentence is "if". >> # Version 1 >> while condition: >> # stuff >> else: >> # other stuff >> >> # Version 2 >> if condition: >> while condition: >> # stuff >> else: >> # other stuff > > they wouldnt be equivalent if #staff in version did not cointain "break" > statement and this is common mistake I realise that they are not equivalent. My point was that some people expect, or would prefer, different behaviour so that those two *would* be equivalent (assuming that evaluating "condition" doesn't have side effects). Oscar -- http://mail.python.org/mailman/listinfo/python-list
Re: Else statement executing when it shouldnt
On Tue, 22 Jan 2013 17:28:35 -0800 (PST) alex23 wrote: > On Jan 23, 1:48 am, Thomas Boell wrote: > > I must say, that's bound to be confusing for anyone who knows any > > language other than Python (or none, even). Syntax like that is "an > > accident waiting to happen"... > > No, ignorantly expecting every language to conform to every other is > the pending accident. Using a keyword that has a well-understood meaning in just about every other programming language on the planet *and even in English*, redefining it to mean something completely different, and then making the syntax look like the original, well-understood meaning -- that's setting a trap out for users. The feature isn't bad, it's just very, very badly named. -- http://mail.python.org/mailman/listinfo/python-list
Re: Understanding while...else...
- Original Message - > Several people have trouble understanding Python's while-else and > for-else constructs. It is actually quite simple agreed on the last part. [snip long story] Did you just try to make it simple by showing the compiled code ? I'm quite not sure about that... JM -- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. -- http://mail.python.org/mailman/listinfo/python-list
Oportunidade: Programador Python Pleno / RJ
Arpex Capital seleciona para uma de suas startups: Programador Python Pleno Descrição: Programador para Web Crawler Python - Experiência mínima de 1 ano em python - Conhecimento de padrões de projeto - Sólido conhecimento OO - Ser auto-gerenciável - Conhecimento de SQL Desejável: - Ter github - Ter interesse por NPL Já ter feito um scraper/crawler/parser - Saber qualquer NoSQL Local de Trabalho: Botafogo/RJ Graduação Completa em Ciência da Computação e/ou afins; Os interessados deverão enviar CV com PRENTENSÃO SALARIAL para kgar...@arpexcapital.com.br , mencionando no assunto Programador Python/Botafogo. -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquely identifying each & every html template
On Wed, Jan 23, 2013 at 5:29 AM, Ferrous Cranus wrote: > Τη Τετάρτη, 23 Ιανουαρίου 2013 11:37:29 π.μ. UTC+2, ο χρήστης alex23 > έγραψε: > > On Jan 23, 7:12 pm, Ferrous Cranus wrote: > > > > > They are getting frustrated because they do not want to follow the > logic i'am imposing. > > > > > > > > No, it's because you're an obvious troll and you're wasting everyone's > > > > time. > > > > > > > > Now be a good lad and sod off. > > 1. I'am a troll because i want to generate a solution in a specific way? > 2. Am' i not wasting my time trying to reply to everybody? > This is so weird. Your time is not wasted, because your time is not valuable. You aren't even a person. You are a fiction. If you were real, someone would beat you to death, and you would stop annoying people. I don't understand the troll thing. Its really anti social. I like the web, but this is so strange > -- > http://mail.python.org/mailman/listinfo/python-list > -- Joel Goldstick http://joelgoldstick.com -- http://mail.python.org/mailman/listinfo/python-list
Converting a number back to it's original string (that was hashed to generate that number)
Now my website finally works as intended. Just visit the following links plz.
--
1. http://superhost.gr
2. http://superhost.gr/?show=log
3. http://i.imgur.com/89Eqmtf.png (this displays the database's column 'pin',
a 5-digit number acting as a filepath indicator)
4. http://i.imgur.com/9js4Pz0.png (this is the detailed html page's
information associated to 'pin' column indicator instead of something like
'/home/nikos/public_html/index.html'
Isn't it a nice solution?
I beleive it is.
but what happens when: http://superhost.gr/?show=stats
I just see the number that correspons to a specific html page and hence i need
to convert that number back to its original string.
# ==
# generating an 5-digit integer based on filepath, for to identify the current
html page
# ==
pin = int( htmlpage.encode("hex"), 16 ) % 10
Now i need the opposite procedure. Will hex.decode(number) convert back to the
original string?
I think not because this isnt a hash procedure.
But how can it be done then?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Parse a Wireshark pcap file
The import "from scapy.all import *" does work for me with macports and 10.6.8 When I installed the scapy port, I did see that macports installed the py27-libdnet package as well. On Wed, Jan 23, 2013 at 1:24 AM, Dennis Lee Bieber wrote: > On Tue, 22 Jan 2013 22:43:24 -0500, Kevin Holleran > declaimed the following in gmane.comp.python.general: > > > > > Mac OS 10.8.2 > > Python v.2.7 > > I downloaded from the sourceforge site, then tried to install with > MacPorts > > when some dependencies were failing. I then downloaded & installed > > pcapy-0.10.6 when that dependency still failed. That solved that but I > > received the dnet error: > > > > from scapy.all import conf > > File "/Library/Python/2.7/site-packages/scapy/all.py", line 16, in > > > > from arch import * > > File "/Library/Python/2.7/site-packages/scapy/arch/__init__.py", line > 75, > > in > > from bsd import * > > File "/Library/Python/2.7/site-packages/scapy/arch/bsd.py", line 12, in > > > > from unix import * > > File "/Library/Python/2.7/site-packages/scapy/arch/unix.py", line 20, > in > > > > from pcapdnet import * > > File "/Library/Python/2.7/site-packages/scapy/arch/pcapdnet.py", line > > 160, in > > import dnet > > ImportError: No module named dnet > > > > So I downloaded and compiled libdnet-1.11 with a: > > $ sudo ./configure && make > > > > I see it compile fine & the libraries have been installed to: > > /usr/local/sbin/dnet > > > > However, python can't find it... I am not clear on how to point Python > > there... > > > "libdnet" is likely a shared object binary... What I /think/ you > are > missing is the Python library that interfaces with that binary... > > Could http://pypi.python.org/pypi/dnet answer the question? > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/ > > -- > http://mail.python.org/mailman/listinfo/python-list > -- John Evans -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquely identifying each & every html template
Τη Τετάρτη, 23 Ιανουαρίου 2013 2:03:51 μ.μ. UTC+2, ο χρήστης Joel Goldstick έγραψε: > On Wed, Jan 23, 2013 at 5:29 AM, Ferrous Cranus wrote: > > Τη Τετάρτη, 23 Ιανουαρίου 2013 11:37:29 π.μ. UTC+2, ο χρήστης alex23 έγραψε: > > > > On Jan 23, 7:12 pm, Ferrous Cranus wrote: > > > > > > > They are getting frustrated because they do not want to follow the logic > > > i'am imposing. > > > > > > > > > > > > No, it's because you're an obvious troll and you're wasting everyone's > > > > > > time. > > > > > > > > > > > > Now be a good lad and sod off. > > > > 1. I'am a troll because i want to generate a solution in a specific way? > > 2. Am' i not wasting my time trying to reply to everybody? > > > > This is so weird. Your time is not wasted, because your time is not > valuable. You aren't even a person. You are a fiction. If you were real, > someone would beat you to death, and you would stop annoying people. I > don't understand the troll thing. Its really anti social. > > > > I like the web, but this is so strange > > -- > > http://mail.python.org/mailman/listinfo/python-list > > > > > -- > > > Joel Goldstick > http://joelgoldstick.com Who told you that my time is NOT valuable and yours is? And yes iam an actual person. -- http://mail.python.org/mailman/listinfo/python-list
Re: Converting a number back to it's original string (that was hashed to generate that number)
Ferrous Cranus writes:
> pin = int( htmlpage.encode("hex"), 16 ) % 10
>
> Now i need the opposite procedure.
As already said several times by different persons in this thread, there
is no way you can get the original string that originated a particular
“pin”: the function you are using is “lossy”, that is, information gets
lost in order to reduce a BIG string into a SMALL five-digits integer
number.
> Will hex.decode(number) convert back to the original string?
NO. As people demonstrated you, you are going to meet collisions very
fast, if you insist going this way (even you thought a “smarter” way to
get a checksum out of your string by using a different weight for the
single characters, there is still high chances of collisions, not
counting the final “modulo” operation). Once you get such a collision,
there is not enough information in that single tiny number to get back a
single string that generated it.
Imagine that, instead of using an integer checksum of your full path,
you “shrink” it by replacing each name in the path with its starting
letter, that is:
/home/ferrous/public_html/index.html => /h/f/p/i
That is evidently way shorter of the original, but you LOST information,
and you cannot write code in any language that eventually reproduce the
original.
The only way out is either use the fullpath as the primary key of your
table, or using a mapping table with a bi-directional univoke mapping
between any single fullpath to the corresponding "short" integer value.
ciao, lele.
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
l...@metapensiero.it | -- Fortunato Depero, 1929.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Converting a number back to it's original string (that was hashed to generate that number)
please don't feed the troll. cu, Kurt -- http://mail.python.org/mailman/listinfo/python-list
Re: Converting a number back to it's original string (that was hashed to generate that number)
Please DON'T tell me to save both the pin <=> filepath and associate them (that can be done by SQL commands, i know) I will not create any kind of primary/unique keys to the database. I will not store the filepath into the database, just the number which indicates the filepath(html page). Also no external table associating fielpaths and numbers. i want this to be solved only by Python Code, not database oriented. That is: I need to be able to map both ways, in a one to one relation, 5-digit-integer <=> string int( hex ( string ) ) can encode a string to a number. Can this be decoded back? I gues that can also be decoded-converted back because its not losing any information. Its encoding, not compressing. But it's the % modulo that breaks the forth/back association. So, the question is: HOW to map both ways, in a one to one relation, (5-digit-integer <=> string) without losing any information? -- http://mail.python.org/mailman/listinfo/python-list
Re: Else statement executing when it shouldnt
Thomas Boell writes: > Using a keyword that has a well-understood meaning in just about > every other programming language on the planet *and even in > English*, redefining it to mean something completely different, and > then making the syntax look like the original, well-understood > meaning -- that's setting a trap out for users. > > The feature isn't bad, it's just very, very badly named. I believe it would read better - much better - if it was "for/then" and "while/then" instead of "for/else" and "while/else". I believe someone didn't want to introduce a new keyword for this, hence "else". -- http://mail.python.org/mailman/listinfo/python-list
Re: Converting a number back to it's original string (that was hashed to generate that number)
On 01/23/2013 08:38 AM, Ferrous Cranus wrote: Please DON'T tell me to save both the pin <=> filepath and associate them (that can be done by SQL commands, i know) I will not create any kind of primary/unique keys to the database. I will not store the filepath into the database, just the number which indicates the filepath(html page). Also no external table associating fielpaths and numbers. i want this to be solved only by Python Code, not database oriented. That is: I need to be able to map both ways, in a one to one relation, 5-digit-integer <=> string int( hex ( string ) ) can encode a string to a number. Can this be decoded back? I gues that can also be decoded-converted back because its not losing any information. Its encoding, not compressing. But it's the % modulo that breaks the forth/back association. So, the question is: HOW to map both ways, in a one to one relation, (5-digit-integer <=> string) without losing any information? Simple. Predefine the 100,000 legal strings, and don't let the user use anything else. One way to do that would be to require a path string of no more than 5 characters, and require them all to be of a restricted alphabet of 10 characters. (eg. the alphabet could be 0-9, which is obvious, or it could be ".aehilmpst" (no uppercase, no underscore, no digits, no non-ascii, etc.) In the realistic case of file paths or URLs, it CANNOT be done. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Parse a Wireshark pcap file
On Wed, Jan 23, 2013 at 7:25 AM, John Evans wrote: > The import "from scapy.all import *" does work for me with macports and > 10.6.8 When I installed the scapy port, I did see that macports installed > the py27-libdnet package as well. > > > > > > On Wed, Jan 23, 2013 at 1:24 AM, Dennis Lee Bieber > wrote: > >> On Tue, 22 Jan 2013 22:43:24 -0500, Kevin Holleran >> declaimed the following in gmane.comp.python.general: >> >> > >> > Mac OS 10.8.2 >> > Python v.2.7 >> > I downloaded from the sourceforge site, then tried to install with >> MacPorts >> > when some dependencies were failing. I then downloaded & installed >> > pcapy-0.10.6 when that dependency still failed. That solved that but I >> > received the dnet error: >> > >> > from scapy.all import conf >> > File "/Library/Python/2.7/site-packages/scapy/all.py", line 16, in >> > >> > from arch import * >> > File "/Library/Python/2.7/site-packages/scapy/arch/__init__.py", line >> 75, >> > in >> > from bsd import * >> > File "/Library/Python/2.7/site-packages/scapy/arch/bsd.py", line 12, >> in >> > >> > from unix import * >> > File "/Library/Python/2.7/site-packages/scapy/arch/unix.py", line 20, >> in >> > >> > from pcapdnet import * >> > File "/Library/Python/2.7/site-packages/scapy/arch/pcapdnet.py", line >> > 160, in >> > import dnet >> > ImportError: No module named dnet >> > >> > So I downloaded and compiled libdnet-1.11 with a: >> > $ sudo ./configure && make >> > >> > I see it compile fine & the libraries have been installed to: >> > /usr/local/sbin/dnet >> > >> > However, python can't find it... I am not clear on how to point Python >> > there... >> > >> "libdnet" is likely a shared object binary... What I /think/ you >> are >> missing is the Python library that interfaces with that binary... >> >> Could http://pypi.python.org/pypi/dnet answer the question? >> -- >> Wulfraed Dennis Lee Bieber AF6VN >> wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/ >> >> -- >> http://mail.python.org/mailman/listinfo/python-list >> > > > > -- > John Evans > > -- > http://mail.python.org/mailman/listinfo/python-list > > I downloaded scapy manually since for some reason, after using macports, it wouldn't find the package at all. I am also trying to install libdnet manually as mentioned above, so after ./configure && make I go into the python directory & do a python setup.py install, which generates a bunch of warnings & the following two errors: /dnet.c:2729:4: error: assignment to cast is illegal, lvalue casts are not supported ((PyObject*)__pyx_v_next) = Py_None; Py_INCREF(((PyObject*)__pyx_v_next)); ~^~~~ ~ ./dnet.c:2741:6: error: assignment to cast is illegal, lvalue casts are not supported ((PyObject *)__pyx_v_next) = __pyx_3; ~^ ~ Thanks again for any help. Need to get all this working for this mini-project and also because I am starting a SANS class that leverages scapy quite a bit... Kevin -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiple postings [was Re: Increase value in hash table]
On Jan 23, 3:54 pm, Steven D'Aprano wrote: > Steven D'Aprano wrote: > > I *swear* I only sent it once. Now Now Steven! Good boys dont swear. > Arrgggh, it's happened again. Sorry for the multiple posts folks... > Trying this time with a different news client. Its a law of the universe called karma. Thou shalt double triple quadruple post for each GG user thou tickest off. And the choice for instant liberation is always there: Use GG!! -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On Tuesday, January 22, 2013 10:06:41 PM UTC-6, Isaac Won wrote: > Hi all, > > > > I have tried to use different interpolation methods with Scipy. My code seems > just fine with linear interpolation, but shows memory error with quadratic. I > am a novice for python. I will appreciate any help. > > > > #code > > f = open(filin, "r") > > for columns in ( raw.strip().split() for raw in f ): > > a.append(columns[5]) > > x = np.array(a, float) > > > > > > not_nan = np.logical_not(np.isnan(x)) > > indices = np.arange(len(x)) > > interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic') > > > > p = interp(indices) > > > > > > The number of data is 31747. > > > > Thank you, > > > > Isaac I really appreciate to both Ulich and Oscar. To Oscar My actual error message is: File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__ self._spline = splmake(x,oriented_y,order=order) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake coefs = func(xk, yk, order, conds, B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest u,s,vh = np.dual.svd(B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd full_matrices=full_matrices, overwrite_a = overwrite_a) MemoryError -- Thank you, Hoonill -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On Wednesday, January 23, 2013 4:08:13 AM UTC-6, Oscar Benjamin wrote: > On 23 January 2013 08:55, Ulrich Eckhardt > > wrote: > > > Am 23.01.2013 05:06, schrieb Isaac Won: > > > > > >> I have tried to use different interpolation methods with Scipy. My > > >> code seems just fine with linear interpolation, but shows memory > > >> error with quadratic. I am a novice for python. I will appreciate any > > >> help. > > > > > [SNIP] > > > > > > > > > Concerning the rest of your problems, there is lots of code and the datafile > > > missing. However, there is also too much of it, try replacing the file with > > > generated data and remove everything from the code that is not absolutely > > > necessary. > > > > Also please copy paste the actual error message rather than paraphrasing it. > > > > > > Oscar I really appreciate to both Ulich and Oscar. To Oscar My actual error message is: File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__ self._spline = splmake(x,oriented_y,order=order) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake coefs = func(xk, yk, order, conds, B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest u,s,vh = np.dual.svd(B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd full_matrices=full_matrices, overwrite_a = overwrite_a) MemoryError -- Thank you, Isaac -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On Wednesday, January 23, 2013 4:08:13 AM UTC-6, Oscar Benjamin wrote: > On 23 January 2013 08:55, Ulrich Eckhardt > > > > > Am 23.01.2013 05:06, schrieb Isaac Won: > > > > > >> I have tried to use different interpolation methods with Scipy. My > > >> code seems just fine with linear interpolation, but shows memory > > >> error with quadratic. I am a novice for python. I will appreciate any > > >> help. > > > > > [SNIP] > > > > > > > > > Concerning the rest of your problems, there is lots of code and the datafile > > > missing. However, there is also too much of it, try replacing the file with > > > generated data and remove everything from the code that is not absolutely > > > necessary. > > > > Also please copy paste the actual error message rather than paraphrasing it. > > > > > > Oscar I really appreciate to both Ulich and Oscar. To Oscar My actual error message is: File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__ self._spline = splmake(x,oriented_y,order=order) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake coefs = func(xk, yk, order, conds, B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest u,s,vh = np.dual.svd(B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd full_matrices=full_matrices, overwrite_a = overwrite_a) MemoryError -- Thank you, Hoonill -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquely identifying each & every html template
On Jan 23, 8:29 pm, Ferrous Cranus wrote: > 1. I'am a troll because i want to generate a solution in a specific way? No, it's because you have the name & behaviour of a known type of troll: As has been pointed out: http://redwing.hutman.net/~mreed/warriorshtm/ferouscranus.htm "Ferrous Cranus is utterly impervious to reason, persuasion and new ideas, and when engaged in battle he will not yield an inch in his position regardless of its hopelessness. Though his thrusts are decisively repulsed, his arguments crushed in every detail and his defenses demolished beyond repair he will remount the same attack again and again with only the slightest variation in tactics." Sound familiar? -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On 23 January 2013 14:28, Isaac Won wrote: > On Wednesday, January 23, 2013 4:08:13 AM UTC-6, Oscar Benjamin wrote: > > To Oscar > My actual error message is: > File > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", > line 311, in __init__ > self._spline = splmake(x,oriented_y,order=order) > File > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", > line 809, in splmake > coefs = func(xk, yk, order, conds, B) > File > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", > line 530, in _find_smoothest > u,s,vh = np.dual.svd(B) > File > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", > line 91, in svd > full_matrices=full_matrices, overwrite_a = overwrite_a) > MemoryError Are you sure that's the *whole* error message? The traceback only refers to the scipy modules. I can't see the line from your code that is generating the error. Oscar -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On Wednesday, January 23, 2013 2:55:14 AM UTC-6, Ulrich Eckhardt wrote: > Am 23.01.2013 05:06, schrieb Isaac Won: > > > I have tried to use different interpolation methods with Scipy. My > > > code seems just fine with linear interpolation, but shows memory > > > error with quadratic. I am a novice for python. I will appreciate any > > > help. > > > > > > #code > > > f = open(filin, "r") > > > > Check out the "with open(...) as f" syntax. > > > > > > > for columns in ( raw.strip().split() for raw in f ): > > > > For the record, this first builds a sequence and then iterates over that > > sequence. This is not very memory-efficient, try this instead: > > > > for line in f: > > columns = line.strip().split() > > > > > > Concerning the rest of your problems, there is lots of code and the > > datafile missing. However, there is also too much of it, try replacing > > the file with generated data and remove everything from the code that is > > not absolutely necessary. > > > > Good luck! > > > > Uli Hi Ulich, I tried to change the code following your advice, but it doesn't seem to work still. My adjusted code is: a = [] with open(filin, "r") as f: for line in f: columns = line.strip().split() a.append(columns[5]) x = np.array(a, float) not_nan = np.logical_not(np.isnan(x)) indices = np.arange(len(x)) interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic') p = interp(indices) - And full error message is: interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic') File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__ self._spline = splmake(x,oriented_y,order=order) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake coefs = func(xk, yk, order, conds, B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest u,s,vh = np.dual.svd(B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd full_matrices=full_matrices, overwrite_a = overwrite_a) MemoryError --- Could you give me some advice for this situation? Thank you always, Isaac -- http://mail.python.org/mailman/listinfo/python-list
Re: Else statement executing when it shouldnt
On Wed, Jan 23, 2013 at 8:35 AM, Jussi Piitulainen wrote: >> The feature isn't bad, it's just very, very badly named. > > I believe it would read better - much better - if it was "for/then" > and "while/then" instead of "for/else" and "while/else". That's always been my opinion too. I'd remember how the construct worked if it was for/then (and while/then). Since seeing for/else always makes my brain lock up for a few seconds when I'm reading code, I don't bother using it. Jerry -- http://mail.python.org/mailman/listinfo/python-list
Re: Any algorithm to preserve whitespaces?
Yes, Peter got it right. Now, how can I replace: script, givenfile = argv with something better that takes argv[1] as input file as well as reads input from stdin. By input from stdin, I mean that currently when I do `cat foo.txt | capitalizr` it throws a ValueError error: Traceback (most recent call last): File "/home/santosh/bin/capitalizr", line 16, in script, givenfile = argv ValueError: need more than 1 value to unpack I want both input methods. -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On Wednesday, January 23, 2013 8:40:54 AM UTC-6, Oscar Benjamin wrote: > On 23 January 2013 14:28, Isaac Won wrote: > > > On Wednesday, January 23, 2013 4:08:13 AM UTC-6, Oscar Benjamin wrote: > > > > > > To Oscar > > > My actual error message is: > > > File > > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", > > line 311, in __init__ > > > self._spline = splmake(x,oriented_y,order=order) > > > File > > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", > > line 809, in splmake > > > coefs = func(xk, yk, order, conds, B) > > > File > > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", > > line 530, in _find_smoothest > > > u,s,vh = np.dual.svd(B) > > > File > > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", > > line 91, in svd > > > full_matrices=full_matrices, overwrite_a = overwrite_a) > > > MemoryError > > > > Are you sure that's the *whole* error message? The traceback only > > refers to the scipy modules. I can't see the line from your code that > > is generating the error. > > > > > > Oscar Dear Oscar, Following is full error message after I adjusted following Ulich's advice: interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic') File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__ self._spline = splmake(x,oriented_y,order=order) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake coefs = func(xk, yk, order, conds, B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest u,s,vh = np.dual.svd(B) File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd full_matrices=full_matrices, overwrite_a = overwrite_a) MemoryError -- Thank you, Hoonill -- http://mail.python.org/mailman/listinfo/python-list
Re: Using filepath method to identify an .html page
On 01/23/2013 12:25 AM, Ferrous Cranus wrote: > > Using that method ABC.html and CBA.html now have different values > because each letter position's value gets bumped up increasingly from > left to right. You have run this little "hash" algorithm on a whole bunch of files, say C:\windows\system32 right? And how many collisions did you get? You've already rejected using the file path or url as a key because it could change. Why are you wanting to do this hash based on the file's path or url anyway? -- http://mail.python.org/mailman/listinfo/python-list
Re: Increase value in hash table
Works.
prndev = line.split()
# print line
for key in prndev :
if key in 'lpr':
val = prndev[5].replace("-P","")
if val not in printque:
printque[val] = 1
else:
printque[val] = printque[val] + 1
if key in "/dev/null":
val='null'
if val not in printque:
printque[val] = 1
else:
printque[val] = printque[val] + 1
On Wed, Jan 23, 2013 at 6:12 PM, Oscar Benjamin
wrote:
> On 23 January 2013 07:26, moonhkt wrote:
>> Hi Al
>>
>> I have Data file have below
>>
>> Data file
>> V1
>> V2
>> V3
>> V4
>> V4
>> V3
>>
>> How to using count number of data ?
>>
>> Output
>> V1 = 1
>> V2 = 1
>> V3 =2
>> V4 = 2
>>
>>
>>
>> # Global Veriable
>> printque = {}
>> in def have below
>>
>> printque[val] = printque[val] + 1
>>
>> I have below error
>> File "xprintlogchk.py", line 78, in chklog
>> printque[val] = printque[val] + 1
>> KeyError: 'nan'
>
> You can't retrieve the value of printque[val] if you haven't yet added
> an entry with the key val to the dict. Try this:
>
> if val not in printque:
> printque[val] = 1
> else:
> printque[val] = printque[val] + 1
>
>
> Oscar
--
moonhkt
GMT+8
--
http://mail.python.org/mailman/listinfo/python-list
Re: Converting a number back to it's original string (that was hashed to generate that number)
Τη Τετάρτη, 23 Ιανουαρίου 2013 3:58:45 μ.μ. UTC+2, ο χρήστης Dave Angel έγραψε: > On 01/23/2013 08:38 AM, Ferrous Cranus wrote: > > > Please DON'T tell me to save both the pin <=> filepath and associate them > > (that can be done by SQL commands, i know) > > > I will not create any kind of primary/unique keys to the database. > > > I will not store the filepath into the database, just the number which > > indicates the filepath(html page). > > > Also no external table associating fielpaths and numbers. > > > i want this to be solved only by Python Code, not database oriented. > > > > > > > > > That is: I need to be able to map both ways, in a one to one relation, > > 5-digit-integer <=> string > > > > > > int( hex ( string ) ) can encode a string to a number. Can this be decoded > > back? I gues that can also be decoded-converted back because its not losing > > any information. Its encoding, not compressing. > > > > > > But it's the % modulo that breaks the forth/back association. > > > > > > So, the question is: > > > > > > HOW to map both ways, in a one to one relation, (5-digit-integer <=> > > string) without losing any information? > > > > > > > Simple. Predefine the 100,000 legal strings, and don't let the user use > > anything else. One way to do that would be to require a path string of > > no more than 5 characters, and require them all to be of a restricted > > alphabet of 10 characters. (eg. the alphabet could be 0-9, which is > > obvious, or it could be ".aehilmpst" (no uppercase, no underscore, no > > digits, no non-ascii, etc.) > > > > In the realistic case of file paths or URLs, it CANNOT be done. OK, its not doable. I'll stop asking for it. CHANGE of plans. i will use the database solution which is the most easy wau to do it: # insert new page record in table counters or update it if already exists try: cursor.execute( '''INSERT INTO counters(page, hits) VALUES(%s, %s) ON DUPLICATE KEY UPDATE hits = hits + 1''', (htmlpage, 1) ) except MySQLdb.Error, e: print ( "Query Error: ", sys.exc_info()[1].excepinfo()[2] ) # update existing visitor record if same pin and same host found try: cursor.execute( '''UPDATE visitors SET hits = hits + 1, useros = %s, browser = %s, date = %s WHERE pin = %s AND host = %s''', (useros, browser, date, page, host)) except MySQLdb.Error, e: print ( "Error %d: %s" % (e.args[0], e.args[1]) ) # insert new visitor record if above update did not affect a row if cursor.rowcount == 0: cursor.execute( '''INSERT INTO visitors(hits, host, useros, browser, date) VALUES(%s, %s, %s, %s, %s)''', (1, host, useros, browser, date) ) I can INSERT a row to the table "counter" I cannot UPDATE or INSERT into the table "visitors" without knowing the "pin" primary key number the database created. Can you help on this please? -- http://mail.python.org/mailman/listinfo/python-list
Re: Increase value in hash table
On Jan 23, 11:33 pm, moonhk wrote:
> Works.
>
> prndev = line.split()
> # print line
> for key in prndev :
> if key in 'lpr':
> val = prndev[5].replace("-P","")
> if val not in printque:
> printque[val] = 1
> else:
> printque[val] = printque[val] + 1
> if key in "/dev/null":
> val='null'
> if val not in printque:
> printque[val] = 1
> else:
> printque[val] = printque[val] + 1
>
> On Wed, Jan 23, 2013 at 6:12 PM, Oscar Benjamin
>
>
>
>
>
>
>
>
>
> wrote:
> > On 23 January 2013 07:26, moonhkt wrote:
> >> Hi Al
>
> >> I have Data file have below
>
> >> Data file
> >> V1
> >> V2
> >> V3
> >> V4
> >> V4
> >> V3
>
> >> How to using count number of data ?
>
> >> Output
> >> V1 = 1
> >> V2 = 1
> >> V3 =2
> >> V4 = 2
>
> >> # Global Veriable
> >> printque = {}
> >> in def have below
>
> >> printque[val] = printque[val] + 1
>
> >> I have below error
> >> File "xprintlogchk.py", line 78, in chklog
> >> printque[val] = printque[val] + 1
> >> KeyError: 'nan'
>
> > You can't retrieve the value of printque[val] if you haven't yet added
> > an entry with the key val to the dict. Try this:
>
> > if val not in printque:
> > printque[val] = 1
> > else:
> > printque[val] = printque[val] + 1
>
> > Oscar
>
> --
> moonhkt
> GMT+8
Tried below works
a = "A B"
karry = a.split()
age[karry[0]] = age.get(karry[0], 100) + 1
age[karry[1]] = age.get(karry[1], 0) + 1
age[karry[1]] = age.get(karry[1], 0) + 1
Result
A 101
B 2
--
http://mail.python.org/mailman/listinfo/python-list
Re: Using filepath method to identify an .html page
Τη Τετάρτη, 23 Ιανουαρίου 2013 5:25:36 μ.μ. UTC+2, ο χρήστης Michael Torrie έγραψε: > On 01/23/2013 12:25 AM, Ferrous Cranus wrote: > > > > > > Using that method ABC.html and CBA.html now have different values > > > because each letter position's value gets bumped up increasingly from > > > left to right. > > > > You have run this little "hash" algorithm on a whole bunch of files, say > > C:\windows\system32 right? And how many collisions did you get? > > > > You've already rejected using the file path or url as a key because it > > could change. Why are you wanting to do this hash based on the file's > > path or url anyway? No, its inevitable, something must remain the same. Filepath *must* be used. Can you transliterate this code to Python code please? -- http://mail.python.org/mailman/listinfo/python-list
Re: Parse a Wireshark pcap file
It looks like there was some very recent reorganization of the scapy packaging on macports, see http://lists.macosforge.org/pipermail/macports-dev/2013-January/021620.html Did you have an updated port tree when you installed? If not, I'd suggest uninstalling whatever port you installed, resync with "port selfupdate", then install the scapy port again. You should see not only the libdnet port installed, but also py27-libdnet, which is the missing glue you need for interfacing with libdnet. On Wed, Jan 23, 2013 at 9:01 AM, Kevin Holleran wrote: > > On Wed, Jan 23, 2013 at 7:25 AM, John Evans wrote: > >> The import "from scapy.all import *" does work for me with macports and >> 10.6.8 When I installed the scapy port, I did see that macports installed >> the py27-libdnet package as well. >> >> >> >> >> >> On Wed, Jan 23, 2013 at 1:24 AM, Dennis Lee Bieber > > wrote: >> >>> On Tue, 22 Jan 2013 22:43:24 -0500, Kevin Holleran >>> declaimed the following in gmane.comp.python.general: >>> >>> > >>> > Mac OS 10.8.2 >>> > Python v.2.7 >>> > I downloaded from the sourceforge site, then tried to install with >>> MacPorts >>> > when some dependencies were failing. I then downloaded & installed >>> > pcapy-0.10.6 when that dependency still failed. That solved that but I >>> > received the dnet error: >>> > >>> > from scapy.all import conf >>> > File "/Library/Python/2.7/site-packages/scapy/all.py", line 16, in >>> > >>> > from arch import * >>> > File "/Library/Python/2.7/site-packages/scapy/arch/__init__.py", >>> line 75, >>> > in >>> > from bsd import * >>> > File "/Library/Python/2.7/site-packages/scapy/arch/bsd.py", line 12, >>> in >>> > >>> > from unix import * >>> > File "/Library/Python/2.7/site-packages/scapy/arch/unix.py", line >>> 20, in >>> > >>> > from pcapdnet import * >>> > File "/Library/Python/2.7/site-packages/scapy/arch/pcapdnet.py", line >>> > 160, in >>> > import dnet >>> > ImportError: No module named dnet >>> > >>> > So I downloaded and compiled libdnet-1.11 with a: >>> > $ sudo ./configure && make >>> > >>> > I see it compile fine & the libraries have been installed to: >>> > /usr/local/sbin/dnet >>> > >>> > However, python can't find it... I am not clear on how to point Python >>> > there... >>> > >>> "libdnet" is likely a shared object binary... What I /think/ you >>> are >>> missing is the Python library that interfaces with that binary... >>> >>> Could http://pypi.python.org/pypi/dnet answer the question? >>> -- >>> Wulfraed Dennis Lee Bieber AF6VN >>> wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/ >>> >>> -- >>> http://mail.python.org/mailman/listinfo/python-list >>> >> >> >> >> -- >> John Evans >> >> -- >> http://mail.python.org/mailman/listinfo/python-list >> >> > > I downloaded scapy manually since for some reason, after using macports, > it wouldn't find the package at all. > > I am also trying to install libdnet manually as mentioned above, so after > ./configure && make I go into the python directory & do a python setup.py > install, which generates a bunch of warnings & the following two errors: > > /dnet.c:2729:4: error: assignment to cast is illegal, lvalue casts are not > supported > ((PyObject*)__pyx_v_next) = Py_None; > Py_INCREF(((PyObject*)__pyx_v_next)); > ~^~~~ ~ > ./dnet.c:2741:6: error: assignment to cast is illegal, lvalue casts are > not supported > ((PyObject *)__pyx_v_next) = __pyx_3; > ~^ ~ > > > Thanks again for any help. Need to get all this working for this > mini-project and also because I am starting a SANS class that leverages > scapy quite a bit... > > Kevin > > -- John Evans -- http://mail.python.org/mailman/listinfo/python-list
RE: Failed to import a "pyd: File When python intepreter embed in C++ project
> I create a pyd File named "testPyd" with boostPython,and then I import the > testPyd module into "test.py", it works perfect! > But when I embeded the python interpreter into my C++ project and run the > "test.py", it comes out a "ImportErr: no module named testPyd". > It has confused me for two days and I googled for long time,but I can't find > the answer! > Anybody here can help me ? > Thank you! > Ah, that sounds familiar. I have a small bit of experience with Boost. I could be wrong, because once you start mixing it up with Boost all sorts of weird things can happen (especially on a MS compiler, because no one tests for Windows, let alone a pay-for compiler) but my experience has shown that if you get that specific import error, what it actually means is just that the PYD import failed for ANY reason. It has nothing to do with the name or that the PYD couldn't be found. Just that somewhere, at some time during import, it failed to fully load. In my use a lot of times it was either that a DLL that module depended on wasn't in the path, or my favorite kicker, that some compile/link flags between the PYD and the Python interpreter don't match well enough. (Usually from mixing debug and release builds together.) From what I've seen anyway, the Python interpreter really doesn't like being built in a traditional debug mode, so I always do a release build of it. It's a little inconvenient, but in the linker flags you can still set your PYDs to generate debug information even in release builds, so you can still run the debugger on them when you attach to the process of the python interpreter EXE. And especially be sure to use the RELEASE runtime library flag (such as /MD) instead of the debug flag (such as /MDd). That's as much as I know anyway. Though depending, if you add any new templates/libraries into Boost (such as for NumPy ndarray), you also may need to use the /DBOOST_ALL_NO_LIB compiler macro on an MS compiler because MS doesn't adhere to template standards correctly and you often end up with multiply-defined functions if you don't use that macro. If I remember correctly. (It's been a while.) That and you may be picking up variable length arrays out of your teeth, replacing chunks of code with the use of new and delete operators. No one tests for Microsoft and the MS compiler is way behind in adhering to C/C++ standards, and VLAs pop up a lot. Hopefully something in all of this helped. Boost can be ... daunting. I get it, in theory. But in practice it often hurts my head. ;) Sincerely, Arah Leonard -- http://mail.python.org/mailman/listinfo/python-list
RE: Using filepath method to identify an .html page
> "his quote string is Cyrillic"? > > If you're referring to the "Τη Τρίτη, 22 Ιανουαρίου 2013 6:23:16 μ.μ. > UTC+2, ο χρήστης Leonard, Arah έγραψε", that's Greek. > Cyrillic or not, it's all Greek to me. ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: Using filepath method to identify an .html page
Τη Τετάρτη, 23 Ιανουαρίου 2013 6:33:11 μ.μ. UTC+2, ο χρήστης Leonard, Arah
έγραψε:
> > "his quote string is Cyrillic"?
>
> >
>
> > If you're referring to the "Τη Τρίτη, 22 Ιανουαρίου 2013 6:23:16 μ.μ.
>
> > UTC+2, ο χρήστης Leonard, Arah έγραψε", that's Greek.
>
> >
>
>
> Cyrillic or not, it's all Greek to me. ;)
=
my @i = split(//,$url); # put each letter in it's own bin
my $j=0; # Initailize our
my $k=1; # hashing increment values
my @m=(); # workspace
foreach my $n(@i){
my $q=ord($n); # ASCII for character
$k += $j; # Increment our hash offset
$q += $k; # add our "old" value
$j = $k;# store that.
push @m,$q; # save the offsetted value
}
my $hashval=0; #initialize our hash value
# Generate that
map { $hashval = ($hashval + $_) % 10} @m;
=
Is this the solution i seek to turn an 'absolute path' <=> '5-digit number' in
a bi-directional way?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On 23 January 2013 14:57, Isaac Won wrote: > On Wednesday, January 23, 2013 8:40:54 AM UTC-6, Oscar Benjamin wrote: >> On 23 January 2013 14:28, Isaac Won wrote: >> [SNIP] > > Following is full error message after I adjusted following Ulich's advice: > > interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic') > File > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", > line 311, in __init__ > self._spline = splmake(x,oriented_y,order=order) > File > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", > line 809, in splmake > coefs = func(xk, yk, order, conds, B) > File > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", > line 530, in _find_smoothest > u,s,vh = np.dual.svd(B) > File > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", > line 91, in svd > full_matrices=full_matrices, overwrite_a = overwrite_a) > MemoryError Where is the new code? You should show full working code (with the import statements) and the full error that is generated by exactly that code. If possible you should also write code that someone else could run even without having access to your data files. If you did that in your first post, you'd probably have an answer to your problem by now. Here is a version of your code that many people on this list can test straight away: import numpy as np from scipy.interpolate import interp1d x = np.array(31747 * [0.0], float) indices = np.arange(len(x)) interp = interp1d(indices, x, kind='quadratic') Running this gives the following error: ~$ python tmp.py Traceback (most recent call last): File "tmp.py", line 5, in interp = interp1d(indices, x, kind='quadratic') File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py", line 308, in __init__ self._spline = splmake(x,oriented_y,order=order) File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py", line 805, in splmake B = _fitpack._bsplmat(order, xk) MemoryError Unless I've misunderstood how this function is supposed to be used, it just doesn't really seem to work for arrays of much more than a few hundred elements. Oscar -- http://mail.python.org/mailman/listinfo/python-list
PDF/A with pyPDF
Hi all, I wonder if pyPDF module can manage PDF/A format of PDF... Can anyone tell me? thank you!! === Davide Scatto Resp. Gestionale Gulliver GMG SISTEMI SRL Via Belluno, 43-45 30035 Mirano (VE) e-mail: softw...@gmgsistemi.it Tel.: 041/5703131 (r.a.) Fax: 041/5703006 P.I. e C.F. 02938960271 Reg. Imp. VE 02938960271 R.E.A. C.C.I.A.A. VE n. 251319 Cap. Soc. Euro 50.000 i.v. === INFORMAZIONI STRETTAMENTE CONFIDENZIALI: L'invio di questa e-mail è destinato solo ad uso personale o a enti sopra nominati e potrebbe contenere informazioni riservate, coperte da segreto professionale, e non soggette a divulgazione ai sensi di legge. Se non ne siete i corretti destinatari, con la presente siete informati che non vi è assolutamente permessa alcuna divulgazione, copia, distribuzione, o altro uso delle informazioni in essa contenute. Se per errore avete ricevuto questo messaggio, Vi chiedo cortesemente di informarci immediatamente al nostro indirizzo di posta elettronica. Grazie-- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquely identifying each & every html template
On 01/23/2013 05:29 AM, Ferrous Cranus wrote: 1. I'am a troll because i want to generate a solution in a specific way? 2. Am' i not wasting my time trying to reply to everybody? 1) your username is obviously a pseudonym, and identified by multiple websites as a troll. Further, your actions conform to the description on at least one of those websites. 2) Your replies are mostly of the forms: a) but I want a car that gets 300 miles to the gallon b) I know you explained that, but would you write all the code for me c) I haven't read any of the literature on computer science, so please build me a perpetual motion machine. Never mind there's a better way to solve my problem, I don't want to hear it d) I'll ignore any message that I don't understand 3) The time that the OP spends ostensibly trying to get help is time he's spending for his own benefit. The time that all the experienced people on here are spending has been mostly to try to help the OP. That's a big difference. Your last code sample was in perl, and while I can read it (and have worked in it when it was absolutely necessary) I choose not to. But just fixing a hash so it doesn't confuse abc with cba will NOT take away the very real probability of collisions. Those calculations I did for you were for the ideal hash. Most are much worse than that. And increasing the range of pin from 1 to 10 will reduce the chances of collision, but not by nearly enough. It still only takes about 150 average samples to get a collision likelihood of over 10% You think it's an accident that md5 size is roughly equivalent to 39 decimal digits? Or that the ones that haven't been proven insecure are much larger than that? The sha512 hash is roughly equivalent to 154 decimal digits. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error with quadratic interpolation
On Wednesday, January 23, 2013 10:51:43 AM UTC-6, Oscar Benjamin wrote:
> On 23 January 2013 14:57, Isaac Won wrote:
>
> > On Wednesday, January 23, 2013 8:40:54 AM UTC-6, Oscar Benjamin wrote:
>
> >> On 23 January 2013 14:28, Isaac Won wrote:
>
> >>
>
> [SNIP]
>
> >
>
> > Following is full error message after I adjusted following Ulich's advice:
>
> >
>
> > interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')
>
> > File
> > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
> > line 311, in __init__
>
> > self._spline = splmake(x,oriented_y,order=order)
>
> > File
> > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
> > line 809, in splmake
>
> > coefs = func(xk, yk, order, conds, B)
>
> > File
> > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
> > line 530, in _find_smoothest
>
> > u,s,vh = np.dual.svd(B)
>
> > File
> > "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py",
> > line 91, in svd
>
> > full_matrices=full_matrices, overwrite_a = overwrite_a)
>
> > MemoryError
>
>
>
> Where is the new code? You should show full working code (with the
>
> import statements) and the full error that is generated by exactly
>
> that code. If possible you should also write code that someone else
>
> could run even without having access to your data files. If you did
>
> that in your first post, you'd probably have an answer to your problem
>
> by now.
>
>
>
> Here is a version of your code that many people on this list can test
>
> straight away:
>
>
>
> import numpy as np
>
> from scipy.interpolate import interp1d
>
> x = np.array(31747 * [0.0], float)
>
> indices = np.arange(len(x))
>
> interp = interp1d(indices, x, kind='quadratic')
>
>
>
> Running this gives the following error:
>
>
>
> ~$ python tmp.py
>
> Traceback (most recent call last):
>
> File "tmp.py", line 5, in
>
> interp = interp1d(indices, x, kind='quadratic')
>
> File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py",
>
> line 308, in __init__
>
> self._spline = splmake(x,oriented_y,order=order)
>
> File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py",
>
> line 805, in splmake
>
> B = _fitpack._bsplmat(order, xk)
>
> MemoryError
>
>
>
> Unless I've misunderstood how this function is supposed to be used, it
>
> just doesn't really seem to work for arrays of much more than a few
>
> hundred elements.
>
>
>
>
>
> Oscar
Thank you Oscar for your help and advice.
I agree with you. So, I tried to find the way to solve this problem.
My full code adjusted is:
from scipy.interpolate import interp1d
import numpy as np
import matplotlib.pyplot as plt
with open(filin, "r") as f:
for line in f:
columns = line.strip().split()
a.append(columns[5])
x = np.array(a, float)
not_nan = np.logical_not(np.isnan(x))
indices = np.arange(len(x))
interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')
p = interp(indices)
k = np.arange(31747)
plt.subplot(211)
plt.plot(k, p)
plt.xlabel('Quadratic interpolation')
plt.subplot(212)
plt.plot(k, x)
plt.show()
-
Whole error message was:
Traceback (most recent call last):
File "QI1.py", line 22, in
interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')
File
"/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
line 311, in __init__
self._spline = splmake(x,oriented_y,order=order)
File
"/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
line 809, in splmake
coefs = func(xk, yk, order, conds, B)
File
"/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py",
line 530, in _find_smoothest
u,s,vh = np.dual.svd(B)
File
"/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py",
line 91, in svd
full_matrices=full_matrices, overwrite_a = overwrite_a)
MemoryError
--
Thank you again Oscar,
Isaac
--
http://mail.python.org/mailman/listinfo/python-list
RE: Using filepath method to identify an .html page
> =
> my @i = split(//,$url); # put each letter in it's own bin
> my $j=0; # Initailize our
> my $k=1; # hashing increment values
> my @m=(); # workspace
> foreach my $n(@i){
>my $q=ord($n); # ASCII for character
>$k += $j; # Increment our hash offset
>$q += $k; # add our "old" value
>$j = $k;# store that.
>push @m,$q; # save the offsetted value
> }
>
> my $hashval=0; #initialize our hash value # Generate that map { $hashval =
> ($hashval + $_) % 10} @m;
> =
>
>
> Is this the solution i seek to turn an 'absolute path' <=> '5-digit number'
> in a bi-directional way?
>
1) There is NO solution to turn a complete path into a 4-digit number in a
bi-directional way, no matter what language you write it in. Nor does adding
one more digit make it any more plausible. It is NOT possible. Which is why
EVERYONE keeps telling you that. The only way to store a complete path in a
unique and bi-directional way is to STORE THE COMPLETE PATH. Even if you
compress the path data in some way, you would still need to store the complete
path.
2) Within reason and sanity, any use of a modulus operator to chop a large
checksum value into a small value means that the results are not unique and not
reversible. This was plainly stated from the beginning, and is still true no
matter how many programming languages you write it in.
3) This is a Python-specific resource and that's not even Python code. What
next? Javascript? Ada? Fortran? COBOL? 8-bit x86 assembly with minimal
comments written in Esperanto?
4) The novelty of the entertainment resulting from this perversity has waned,
even for me. The educational aspect to novice programmers has likewise run
dry. I've now officially grown bored of your game and am joining everyone else
who already has already gotten off of this kiddie ride. Congratulations on
beating a dead horse into mince-meat and successfully milking the one-uddered
cow until the pale is full. I hope that you enjoyed your meal.
Or to borrow a phrase, "I say GOOD DAY, sir!"
--
http://mail.python.org/mailman/listinfo/python-list
Re: Using filepath method to identify an .html page
On 23/01/2013 18:19, Leonard, Arah wrote: 3) This is a Python-specific resource and that's not even Python code. What next? Javascript? Ada? Fortran? COBOL? 8-bit x86 assembly with minimal comments written in Esperanto? Please can we have CORAL 66 mentioned on the odd occasion. 4) The novelty of the entertainment resulting from this perversity has waned, even for me. The educational aspect to novice programmers has likewise run dry. I've now officially grown bored of your game and am joining everyone else who already has already gotten off of this kiddie ride. Congratulations on beating a dead horse into mince-meat and successfully milking the one-uddered cow until the pale is full. I hope that you enjoyed your meal. Pail not pale :) Or to borrow a phrase, "I say GOOD DAY, sir!" Or madam? -- Cheers. Mark Lawrence -- http://mail.python.org/mailman/listinfo/python-list
Search log for string and display hourly/daily report
I need to search a log file for a specific string (Successfully Sent) and report the number of instances in the last hour (from when executed) and total for the day so far (midnight till the time executed). Can anyone provide any examples of such a program or get me started? -- http://mail.python.org/mailman/listinfo/python-list
Re: Search log for string and display hourly/daily report
On 01/23/13 13:05, spe...@gmail.com wrote: I need to search a log file for a specific string (Successfully Sent) and report the number of instances in the last hour (from when executed) and total for the day so far (midnight till the time executed). Can anyone provide any examples of such a program or get me started? You'd have to specify additional details on how the log-file is formatted, presumably how it's delimited, and what the format of the corresponding time-stamp is. Something like "I have a tab-delimited log file where the 4th column is the timestamp in the format '-MM-DD HH:mm:ss' and the 18th column is the status. I want to search for items where the status contains 'Successfully Sent' and then further filter them by (1) events in the last hour, and (2) all events today" -tkc -- http://mail.python.org/mailman/listinfo/python-list
urllib2 FTP Weirdness
Python 2.7.3 on linux
This has me fairly stumped. It looks like
urllib2.urlopen("ftp://some.ftp.site/path";).read()
will either immediately return '' or hang indefinitely. But
response = urllib2.urlopen("ftp://some.ftp.site/path";)
response.read()
works fine and returns what is expected. This is only an issue with urllib2,
vanilla urllib doesn't do it.
The site I first noticed it on is private, but I can reproduce it with
"ftp://ftp2.census.gov/";.
I've tested the equivalent code on Python 3.2.3 and get the same results,
except that one time I got a socket error (may have been a spurious network
blip, though).
I'm at a loss as to how that could even work differently. My only guess is that
by not having a reference to the addinfourl response object, something
important is getting garbage collected or closed... that seems like a stretch,
though. Is this a urllib2 bug, or am I crazy?
-Nick Cash
--
http://mail.python.org/mailman/listinfo/python-list
Re: Search log for string and display hourly/daily report
In <4f952e77-258d-47ae-9d76-a86daa8ac...@googlegroups.com> spe...@gmail.com
writes:
> I need to search a log file for a specific string (Successfully Sent) and
> report the number of instances in the last hour (from when executed) and
> total for the day so far (midnight till the time executed). Can anyone
> provide any examples of such a program or get me started?
from datetime import datetime, timedelta
from time import mktime, strptime
now = datetime.now()
midnight = now.replace(hour=0, minute=0, second=0, microsecond=0)
one_hour_ago = now - timedelta(hours=1)
daily_instances = 0
hourly_instances = 0
with open('myfile.log') as logfile:
for line in logfile:
if 'Successfully Sent' in line:
time_string = line[0:19]
struct = strptime(time_string, "%Y-%m-%dT%H:%M:%S")
log_time = datetime.fromtimestamp(mktime(struct))
if log_time > midnight:
daily_instances += 1
if log_time > one_hour_ago:
hourly_instances += 1
print "Instances in the last hour: ", hourly_instances
print "Instances since midnight: ", daily_instances
This code assumes that log lines begin with a timestamp similar to
"2013-01-23T09:27:01". If the timestamp is in a different format, or
occurs elsewhere in the line, you'll have to adjust for that.
--
John Gordon A is for Amy, who fell down the stairs
gor...@panix.com B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"
--
http://mail.python.org/mailman/listinfo/python-list
Re: Any algorithm to preserve whitespaces?
On 01/23/2013 07:56 AM, Santosh Kumar wrote: > Yes, Peter got it right. > > Now, how can I replace: > > script, givenfile = argv > > with something better that takes argv[1] as input file as well as > reads input from stdin. > > By input from stdin, I mean that currently when I do `cat foo.txt | > capitalizr` it throws a ValueError error: > > Traceback (most recent call last): > File "/home/santosh/bin/capitalizr", line 16, in > script, givenfile = argv > ValueError: need more than 1 value to unpack > > I want both input methods. That's up to your program logic to do. Check to see if the arguments have been provided, and if not, open sys.stdin. It's quite common for command-line utilities to do this, but most of them use an explicit parameter '-' to indicate that you want the command to use standard-in. Again, you can code this any way you want. Personally I use one of the standard library command-line argument parsing modules, like optparse, but there are others that may be better. -- http://mail.python.org/mailman/listinfo/python-list
Re: Parse a Wireshark pcap file
On 2013-01-23, Kevin Holleran wrote: > Is there a way to parse out a wireshark pcap file and extract key value > pairs from the data? You can use pylibpcap to read pcap files (or to capture live data). I'm afraid I don't know what "parse out" or "extract key value pairs" means. pylibpcap doesn't have access to any of wireshark's packet disecter plugins, if that's what you're after. > I am illustrated a sniff of some traffic and why it needs utilize > HTTPS instead of HTTP but I was hoping to run the pcap through a > python script and just output some interesting key value pairs To what does "key value pairs" refer? -- Grant Edwards grant.b.edwardsYow! I am a traffic light, at and Alan Ginzberg kidnapped gmail.commy laundry in 1927! -- http://mail.python.org/mailman/listinfo/python-list
Arent these snippets equivalent?
Is this:
while True:
data = fp.read(4096)
if not data:
break
...
not equivalent to this:
data = fp.read (4096)
while data:
...{handle the chunk here}
data = fp.read (4096)
Heres the article that sparked this question:
http://wordaligned.org/articles/pythons-lesser-known-loop-control
--
http://mail.python.org/mailman/listinfo/python-list
Re: Arent these snippets equivalent?
In Coolgg
writes:
> Is this:
> while True:
> data = fp.read(4096)
> if not data:
> break
> ...
> not equivalent to this:
> data = fp.read (4096)
> while data:
> ...{handle the chunk here}
> data = fp.read (4096)
It looks equivalent to me (in terms of control flow).
But in the second example the fp.read() statement is duplicated, which is
undesirable. It would be all too easy for a maintenance programmer to go
into the code a year from now and change the first one but miss the second
one.
--
John Gordon A is for Amy, who fell down the stairs
gor...@panix.com B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"
--
http://mail.python.org/mailman/listinfo/python-list
Re: Any algorithm to preserve whitespaces?
Santosh Kumar wrote:
> Yes, Peter got it right.
>
> Now, how can I replace:
>
> script, givenfile = argv
>
> with something better that takes argv[1] as input file as well as
> reads input from stdin.
>
> By input from stdin, I mean that currently when I do `cat foo.txt |
> capitalizr` it throws a ValueError error:
>
> Traceback (most recent call last):
> File "/home/santosh/bin/capitalizr", line 16, in
> script, givenfile = argv
> ValueError: need more than 1 value to unpack
>
> I want both input methods.
You can use argparse and its FileType:
import argparse
import sys
parser = argparse.ArgumentParser()
parser.add_argument("infile", type=argparse.FileType("r"), nargs="?",
default=sys.stdin)
args = parser.parse_args()
for line in args.infile:
print line.strip().title() # replace with your code
As this has the small disadvantage that infile is opened immediately I tend
to use a slight variation:
import argparse
import sys
from contextlib import contextmanager
@contextmanager
def xopen(filename):
if filename is None or filename == "-":
yield sys.stdin
else:
with open(filename) as instream:
yield instream
parser = argparse.ArgumentParser()
parser.add_argument("infile", nargs="?")
args = parser.parse_args()
with xopen(args.infile) as instream:
for line in instream:
print line.strip().title()
--
http://mail.python.org/mailman/listinfo/python-list
Re: Search log for string and display hourly/daily report
On 01/23/2013 12:05 PM, spe...@gmail.com wrote: > I need to search a log file for a specific string (Successfully Sent) > and report the number of instances in the last hour (from when > executed) and total for the day so far (midnight till the time > executed). Can anyone provide any examples of such a program or get > me started? Take a look at this very interesting presentation/document: http://www.dabeaz.com/generators/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Arent these snippets equivalent?
On Thu, Jan 24, 2013 at 8:56 AM, Coolgg wrote:
> Is this:
>
> while True:
> data = fp.read(4096)
> if not data:
> break
> ...
>
> not equivalent to this:
>
> data = fp.read (4096)
> while data:
> ...{handle the chunk here}
> data = fp.read (4096)
They should do the same thing, but there's one critical difference in
the second: Edits to something that's really part of the loop now have
to be done twice, at the bottom of the loop and *before the loop*.
It's a violation of the principle Don't Repeat Yourself.
Personally, I'd much rather have a 'while' condition that does
assignment, but that's something Python's unlikely ever to do.
There've been various proposals to make that possible, but ultimately
the only way to make that work is for assignment to be an expression,
which is right up there alongside braces defining blocks.
(Wonder when we'll see "from __future__ import assignment_expression"
implemented...)
The 'break' method is the most common. Assuming you're doing something
as simple as the above, with a single function call and a clear
condition, it's pretty readable. Compare:
while (data = fp.read(4096))
{
... imagine about 20 lines here
}
and
while True:
data = fp.read(4096)
if not data: break
... imagine those same 20 lines, ported to Python
The critical parts of your while loop are in those first three lines.
It's the same goal as a C-style for loop - you can see everything you
need right up there at the loop header. All you have to do is
understand that the "loop header" is three lines long now.
With the second form of the loop, though, the loop header is down at
the bottom of the loop too. It's less clear. Granted, this might be
how a compiler lays it out in memory, but programmers shouldn't have
to read it that way.
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
Re: Increase value in hash table
On 01/23/2013 10:39 AM, moonhkt wrote:
On Jan 23, 11:33 pm, moonhk wrote:
Works.
For some definition of 'works"
prndev = line.split()
# print line
for key in prndev :
if key in 'lpr':
This test will fire if key is the letter "l", or the letter "p", or the
letter "r". Probably not what you want. Suggest you change it to
if key == "lpr":
val = prndev[5].replace("-P","")
if val not in printque:
printque[val] = 1
else:
printque[val] = printque[val] + 1
if key in "/dev/null":
ditto here
val='null'
if val not in printque:
printque[val] = 1
else:
printque[val] = printque[val] + 1
Of course, I don't know what prndev actually looks like, so I could be
wrong as well. But I doubt it "works" as written.
--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list
Re: Using filepath method to identify an .html page
On 01/23/2013 01:36 PM, Mark Lawrence wrote: On 23/01/2013 18:19, Leonard, Arah wrote: 3) This is a Python-specific resource and that's not even Python code. What next? Javascript? Ada? Fortran? COBOL? 8-bit x86 assembly with minimal comments written in Esperanto? Please can we have CORAL 66 mentioned on the odd occasion. 4) The novelty of the entertainment resulting from this perversity has waned, even for me. The educational aspect to novice programmers has likewise run dry. I've now officially grown bored of your game and am joining everyone else who already has already gotten off of this kiddie ride. Congratulations on beating a dead horse into mince-meat and successfully milking the one-uddered cow until the pale is full. I hope that you enjoyed your meal. Pail not pale :) Or to borrow a phrase, "I say GOOD DAY, sir!" Or madam? or 'script kiddie' -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Understanding while...else...
On 1/22/2013 1:15 PM, Jean-Michel Pichavant wrote: - Original Message - Several people have trouble understanding Python's while-else and for-else constructs. It is actually quite simple agreed on the last part. [snip long story] Did you just try to make it simple by showing the compiled code ? I'm quite not sure about that... The dis output was an optional appendix for anyone who doubted the correctly of my claimed python-level equivalence ;-). -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Arent these snippets equivalent?
In article , Chris Angelico wrote: > Personally, I'd much rather have a 'while' condition that does > assignment, but that's something Python's unlikely ever to do. > There've been various proposals to make that possible, but ultimately > the only way to make that work is for assignment to be an expression, > which is right up there alongside braces defining blocks. while getchar() as c: putchar(c) That would give people (including me) the use case they're after most of the time (call a function, assign the return value, and test it). It's way less klunky than: while True: c = getchar() if c: break putchar() It wouldn't require assignment as an expression, or braces, or any new keywords. It would also be quite analogous to try: blah() except BogusThing as ex: whatever() in both cases, the effect is "perform some action, grab a value which resulted from that, and if it passes some test, make it available in the next block bound to a name". -- http://mail.python.org/mailman/listinfo/python-list
Re: Understanding while...else...
On 1/22/2013 7:39 PM, Oscar Benjamin wrote: On 22 January 2013 23:41, Terry Reedy wrote: On 1/22/2013 3:09 PM, Ethan Furman wrote: On 01/22/2013 09:44 AM, Terry Reedy wrote: [SNIP] The else clause is executed if and when the condition is false. Now use a real Python while statement to do the *same thing*. while n > 0: n -= 1 else: n = None I understand how it works (although it did take a while for it to sink in); my gripe, and probably why it is misunderstood so often, is that nine times out of ten when I /want/ to use a while-else or for-else I only want the true/false check /once/, at the beginning of the loop. I do not understand what you are saying. There already is only one true/false check, at the beginning of the loop. If you only want the check *performed* once, you would use if-else. But I presume you know this. I think he meant that he would use the else clause more often if it had the semantics so that the two blocks below were equivalent: # Version 1 while condition: # stuff else: # other stuff # Version 2 if condition: while condition: # stuff else: # other stuff So he wants a convenient way to execute code only if the loop performed zero iterations. I think that often when people are confused about the else clause on while loops it is because they expect this behaviour (which would also be useful). Thank you for the clarification. If 'condition' has side effects, or if one is really bothered by computing it twice, Version 2 could be re-written as if condition: while True: stuff() if not condition: break else: other_stuff() > The same confusion arises with for loops where people expect the else clause to execute if the iterable was empty so that these would be equivalent: # Version 1 for x in iterable: # stuff else: # other stuff # Version 2 iterated = False for x in iterable: iterated = True # stuff if not iterated: # other stuff I see. I guess people who want version 2 will have to write it explicitly. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Arent these snippets equivalent?
On 01/23/2013 03:56 PM, Coolgg wrote:
> Is this:
>
> while True:
> data = fp.read(4096)
> if not data:
> break
> ...
>
> not equivalent to this:
>
> data = fp.read (4096)
> while data:
> ...{handle the chunk here}
> data = fp.read (4096)
>
> Heres the article that sparked this question:
> http://wordaligned.org/articles/pythons-lesser-known-loop-control
There is at least one potentially-critical difference: what happens if
there is a 'continue' statement in the "..." part. The upper loop will
set data again, while the lower one will not.
So if what you mean is "are they equivalent no matter what legal Python
code you put in the ...", no, they aren't.
Evan
signature.asc
Description: OpenPGP digital signature
--
http://mail.python.org/mailman/listinfo/python-list
XML/XHTML/HTML differences, bugs... and howto
Good day :), I've been exploring XML parsers in python; particularly: xml.etree.cElementTree; and I'm trying to figure out how to do it incrementally, for very large XML files -- although I don't think the problems are restricted to incremental parsing. First problem: I've come across an issue where etree silently drops text without telling me; and separate. I am under the impression that XHTML is a subset of XML (eg:defined tags), and that once an HTML file is converted to XHTML, the body of the document can be handled entirely as XML. If I convert a (partial/contrived) html file like: This is example bold text. to XHTML, I might do --right or wrong-- (1): This is example bold text. or, alternate difference: (2): " This is example bold text. " But, when I parse with etree, in example (1) both "This is an example" and "text." are dropped; The missing text is part of the start, or end event tags, in the incrementally parsed method. Likewise: In example (2), only "text" gets dropped. So, etree is silently dropping all text following a close tag, but before another open tag happens. Q: Isn't XML supposed to error out when invalid xml is parsed? Is there a way in etree to recover/access the dropped text? If not -- is the a python library issue, or the underlying expat.so, etc. library. Secondly; I have an XML file which will grow larger than memory on a target machine, so here's what I want to do: Given a source XML file, and a destination file: 1) iteratively scan part of the source tree. 2) Optionally Modify some of scanned tree. 3) Write partial scan/tree out to the destination file. 4) Free memory of no-longer needed (partial) source XML. 5) continue scanning a new section of the source file... eg: goto step 1 until source file is exhausted. But, I don't see a way to write portions of an XML tree, or iteratively write a tree to disk. How can this be done? :) Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Arent these snippets equivalent?
On Thu, Jan 24, 2013 at 9:47 AM, Roy Smith wrote: > while getchar() as c: >putchar(c) > > That would give people (including me) the use case they're after most of > the time (call a function, assign the return value, and test it). It's > way less klunky than: > > while True: >c = getchar() >if c: > break >putchar() > > It wouldn't require assignment as an expression, or braces, or any new > keywords. I believe this was discussed recently (on python-ideas?). It's nice in its simplest form, but doesn't cover all possibilities. My point about braces was that, like assignment-expressions, it's a feature that Python will not be implementing. Fundamentally against the "push" of the language. If you want C, you know where to get it. (There are other options, of course; ECMAScript and Pike come to mind. But you know what I mean.) ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquely identifying each & every html template
On Wed, Jan 23, 2013 at 11:38 PM, Dave Angel wrote: > You think it's an accident that md5 size is roughly equivalent to 39 decimal > digits? Or that the ones that haven't been proven insecure are much larger > than that? The sha512 hash is roughly equivalent to 154 decimal digits. Proving a hash function secure or not is orthogonal to its length. You could have a cryptographically secure hash function that produces a single byte; you'd get collisions pretty often, but that's understood. Conversely, you could have an insecure hash that produces a value several orders of magnitude longer than SHA512. Look at this: def big_long_hash(val): return sum(bytes(str(val),"utf-8"))*12345678901234567890 But longer hashes do reduce the chance of collisions, by the fundamental rules of mathematics. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Arent these snippets equivalent?
On 01/23/13 16:47, Roy Smith wrote:
while getchar() as c:
putchar(c)
That would give people (including me) the use case they're after most of
the time (call a function, assign the return value, and test it). It's
way less klunky than:
while True:
c = getchar()
if c:
# I presume you mean "if not c:" here.
break
putchar()
I was a pretty strong advocate early in one of these long threads,
and for the simple cases, it's some attractive syntactic sugar.
However, I found that it quickly blossomed into a lot of really ugly
edge cases (multiple tests, multiple results, checking for "is None"
vs. false'ness or some other condition such as "< 0"). I found that
it was pretty easy to create a generator-wrapper for this:
def getter(fn):
while True:
val = fn()
if not val: break
yield val
# DB example
cursor = conn.cursor()
for row in getter(lambda: cursor.fetchmany()):
do_something(row)
# your getchar example
for c in getter(getchar):
do_something_else(c)
This allowed me to have both the readability and customized tests
(and the ability to return multiple values). It could be expanded with
def getter(fn, is_at_end=lambda v: not v):
while True:
val = fn()
if is_at_end(val): break
yield val
which would even allow you to do things like
for line in getter(file("foo.txt"), lambda s: s.find("xxx") < 0):
print "This line has 'xxx' in it:"
print line
and those felt a lot more pythonic than any of the proposals I saw
on the list.
-tkc
--
http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 FTP Weirdness
On Thu, Jan 24, 2013 at 7:07 AM, Nick Cash
wrote:
> Python 2.7.3 on linux
>
> This has me fairly stumped. It looks like
> urllib2.urlopen("ftp://some.ftp.site/path";).read()
> will either immediately return '' or hang indefinitely. But
> response = urllib2.urlopen("ftp://some.ftp.site/path";)
> response.read()
> works fine and returns what is expected. This is only an issue with urllib2,
> vanilla urllib doesn't do it.
>
> The site I first noticed it on is private, but I can reproduce it with
> "ftp://ftp2.census.gov/";.
Confirmed on 2.6.5 on Windows, fwiw. This is extremely weird. Possibly
it's some kind of race condition??
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
Re: Uniquely identifying each & every html template
On 01/23/2013 06:25 PM, Chris Angelico wrote: On Wed, Jan 23, 2013 at 11:38 PM, Dave Angel wrote: You think it's an accident that md5 size is roughly equivalent to 39 decimal digits? Or that the ones that haven't been proven insecure are much larger than that? The sha512 hash is roughly equivalent to 154 decimal digits. Proving a hash function secure or not is orthogonal to its length. You could have a cryptographically secure hash function that produces a single byte; you'd get collisions pretty often, but that's understood. Conversely, you could have an insecure hash that produces a value several orders of magnitude longer than SHA512. Look at this: def big_long_hash(val): return sum(bytes(str(val),"utf-8"))*12345678901234567890 But longer hashes do reduce the chance of collisions, by the fundamental rules of mathematics. I certainly can't disagree that it's easy to produce a very long hash that isn't at all secure. But I would disagree that longer hashes *automatically* reduce chances of collision. Anyway, about cryptographically ... OK, I'd like to learn here. I thought that "cryptographically secure" meant that it was infeasible to take a given message and make an innocuous change to it (such as adding a trailer of whatever size) and from that produce a predetermined hash value. Obviously "infeasible" will change over time. But if my definition is even close, then wouldn't it be a necessary (not sufficient) condition that the hash be at least some certain size. It is that size I was trying to impress on the OP. Wikipedia - http://en.wikipedia.org/wiki/Cryptographic_hash_function seems to say that there are four requirements. it is easy to compute the hash value for any given message it is infeasible to generate a message that has a given hash it is infeasible to modify a message without changing the hash it is infeasible to find two different messages with the same hash Seems to me a small hash wouldn't be able to meet the last 3 conditions. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquely identifying each & every html template
On Thu, Jan 24, 2013 at 11:09 AM, Dave Angel wrote: > I certainly can't disagree that it's easy to produce a very long hash that > isn't at all secure. But I would disagree that longer hashes > *automatically* reduce chances of collision. Sure. But by and large, longer hashes give you a better chance at avoiding collisions. Caveat: I am not a cryptography expert. My statements are based on my own flawed understanding of what's going on. I use the stuff but I don't invent it. > Wikipedia - http://en.wikipedia.org/wiki/Cryptographic_hash_function > > seems to say that there are four requirements. > it is easy to compute the hash value for any given message > it is infeasible to generate a message that has a given hash > it is infeasible to modify a message without changing the hash > it is infeasible to find two different messages with the same hash > > Seems to me a small hash wouldn't be able to meet the last 3 conditions. True, but the definition of "small" is tricky. Of course the one-byte hash I proposed isn't going to be difficult to break, since you can just brute-force a bunch of message changes until you find one that has the right hash. But it's more about the cascade effect - that any given message has equal probability of having any of the possible hashes. Make a random change, get another random hash. So for a perfect one-byte hash, you have exactly one chance in 256 of getting any particular hash. By comparison, a simple/naive hash that just XORs together all the byte values fails these checks. Even if you take the message 64 bytes at a time (thus producing a 512-bit hash), you'll still be insecure, because it's easy to predict what hash you'll get after making a particular change. This property of the hash doesn't change as worldwide computing power improves. A hashing function might go from being "military-grade security" to being "home-grade security" to being "two-foot fence around your property", while still being impossible to predict without brute-forcing. But when an algorithm is found that generates collisions faster than the hash size indicates, it effectively reduces the hash size to the collision rate - MD5 is 128-bit, but (if I understand the Wikipedia note correctly) a known attack cuts that to 20.96 bits of "real hash size". So MD5 is still better than a perfect 16-bit hash, but not as good as a perfect 32-bit hash. (And on today's hardware, that's not good enough.) http://en.wikipedia.org/wiki/Collision_resistant ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 FTP Weirdness
Nick Cash wrote:
> Python 2.7.3 on linux
>
> This has me fairly stumped. It looks like
> urllib2.urlopen("ftp://some.ftp.site/path";).read()
> will either immediately return '' or hang indefinitely. But
> response = urllib2.urlopen("ftp://some.ftp.site/path";)
> response.read()
> works fine and returns what is expected. This is only an issue with
> urllib2, vanilla urllib doesn't do it.
>
> The site I first noticed it on is private, but I can reproduce it with
> "ftp://ftp2.census.gov/";.
Then why not give that in your example, to make running your code
easier? :-)
I cannot reproduce the problem:
py> import urllib2
py> x = urllib2.urlopen("ftp://ftp2.census.gov/";).read()
py> len(x)
5550
Works fine for me using Python 2.7.2 on Linux. I cannot see how the two
snippets you give could possibly be different. If you are using a proxy,
what happens if you bypass it?
If you can reproduce this at will, with and without proxy, with multiple
sites, then I suppose it is conceivable that it could be some sort of bug.
But I wouldn't bet on it.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 FTP Weirdness
On 24/01/13 00:58:04, Chris Angelico wrote:
> On Thu, Jan 24, 2013 at 7:07 AM, Nick Cash
> wrote:
>> Python 2.7.3 on linux
>>
>> This has me fairly stumped. It looks like
>> urllib2.urlopen("ftp://some.ftp.site/path";).read()
>> will either immediately return '' or hang indefinitely. But
>> response = urllib2.urlopen("ftp://some.ftp.site/path";)
>> response.read()
>> works fine and returns what is expected. This is only an issue with urllib2,
>> vanilla urllib doesn't do it.
>>
>> The site I first noticed it on is private, but I can reproduce it with
>> "ftp://ftp2.census.gov/";.
>
> Confirmed on 2.6.5 on Windows, fwiw. This is extremely weird.
It works fine with 2.7.3 on my Mac.
> Possibly it's some kind of race condition??
If urllib2 is using active mode FTP, then a firewall on your box
could explain what you're seeing. But then, that's why active
mode is hardly used these days.
Hope this helps,
-- HansM
--
http://mail.python.org/mailman/listinfo/python-list
Re: Any algorithm to preserve whitespaces?
Santosh Kumar wrote: > Yes, Peter got it right. Peter? Which Peter? What's "it" that he got right? You have deleted all context from your post, so I have no idea what you are talking about. And whatever program you are using to post is stripping out threading information, so I can't tell what post you are replying to. Please take careful note of the posting conventions used by the experienced regulars on this forum, and copy their style. That is for your benefit as well as ours. > Now, how can I replace: > > script, givenfile = argv > > with something better that takes argv[1] as input file as well as > reads input from stdin. > > By input from stdin, I mean that currently when I do `cat foo.txt | > capitalizr` it throws a ValueError error: > > Traceback (most recent call last): > File "/home/santosh/bin/capitalizr", line 16, in > script, givenfile = argv > ValueError: need more than 1 value to unpack > > I want both input methods. The usual convention in Unix and Linux is that if the file name is "-", read from stdin instead. Something like this, untested: givenfile = sys.argv[1] if givenfile == '-': data = sys.stdin.read() else: data = open(givenfile).read() Adding error checking etc. is left as an exercise. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Uniquely identifying each & every html template
On 01/23/2013 07:39 PM, Chris Angelico wrote: On Thu, Jan 24, 2013 at 11:09 AM, Dave Angel wrote: I certainly can't disagree that it's easy to produce a very long hash that isn't at all secure. But I would disagree that longer hashes *automatically* reduce chances of collision. Sure. But by and large, longer hashes give you a better chance at avoiding collisions. Caveat: I am not a cryptography expert. My statements are based on my own flawed understanding of what's going on. I use the stuff but I don't invent it. Wikipedia - http://en.wikipedia.org/wiki/Cryptographic_hash_function seems to say that there are four requirements. it is easy to compute the hash value for any given message it is infeasible to generate a message that has a given hash it is infeasible to modify a message without changing the hash it is infeasible to find two different messages with the same hash Seems to me a small hash wouldn't be able to meet the last 3 conditions. True, but the definition of "small" is tricky. Of course the one-byte hash I proposed isn't going to be difficult to break, since you can just brute-force a bunch of message changes until you find one that has the right hash. But it's more about the cascade effect - that any given message has equal probability of having any of the possible hashes. Make a random change, get another random hash. So for a perfect one-byte hash, you have exactly one chance in 256 of getting any particular hash. By comparison, a simple/naive hash that just XORs together all the byte values fails these checks. Even if you take the message 64 bytes at a time (thus producing a 512-bit hash), you'll still be insecure, because it's easy to predict what hash you'll get after making a particular change. This property of the hash doesn't change as worldwide computing power improves. A hashing function might go from being "military-grade security" to being "home-grade security" to being "two-foot fence around your property", while still being impossible to predict without brute-forcing. But when an algorithm is found that generates collisions faster than the hash size indicates, it effectively reduces the hash size to the collision rate - MD5 is 128-bit, but (if I understand the Wikipedia note correctly) a known attack cuts that to 20.96 bits of "real hash size". So MD5 is still better than a perfect 16-bit hash, but not as good as a perfect 32-bit hash. (And on today's hardware, that's not good enough.) http://en.wikipedia.org/wiki/Collision_resistant ChrisA Thanks. I've read a lot about encryption and data compression (two overlapping fields), and done some amateurish work (first time was 1975) that was just to keep something secret, not to be especially secure. I find the field fascinating, but have never needed to do anything particularly secure for a real product. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Search log for string and display hourly/daily report
On Wednesday, January 23, 2013 4:25:59 PM UTC-5, John Gordon wrote:
> In <4f952e77-258d-47ae-9d76-a86daa8ac...@googlegroups.com> spek06 writes:
>
>
>
> > I need to search a log file for a specific string (Successfully Sent) and
>
> > report the number of instances in the last hour (from when executed) and
>
> > total for the day so far (midnight till the time executed). Can anyone
>
> > provide any examples of such a program or get me started?
>
>
>
> from datetime import datetime, timedelta
>
> from time import mktime, strptime
>
>
>
> now = datetime.now()
>
> midnight = now.replace(hour=0, minute=0, second=0, microsecond=0)
>
> one_hour_ago = now - timedelta(hours=1)
>
>
>
> daily_instances = 0
>
> hourly_instances = 0
>
>
>
> with open('myfile.log') as logfile:
>
> for line in logfile:
>
> if 'Successfully Sent' in line:
>
> time_string = line[0:19]
>
> struct = strptime(time_string, "%Y-%m-%dT%H:%M:%S")
>
> log_time = datetime.fromtimestamp(mktime(struct))
>
>
>
> if log_time > midnight:
>
> daily_instances += 1
>
>
>
> if log_time > one_hour_ago:
>
> hourly_instances += 1
>
>
>
> print "Instances in the last hour: ", hourly_instances
>
> print "Instances since midnight: ", daily_instances
>
>
>
>
>
> This code assumes that log lines begin with a timestamp similar to
>
> "2013-01-23T09:27:01". If the timestamp is in a different format, or
>
> occurs elsewhere in the line, you'll have to adjust for that.
>
>
>
> --
>
> John Gordon A is for Amy, who fell down the stairs
>
> B is for Basil, assaulted by bears
>
> -- Edward Gorey, "The Gashlycrumb Tinies"
Thanks John, I think this will definitely help get me started!
--
http://mail.python.org/mailman/listinfo/python-list
Re: Any algorithm to preserve whitespaces?
On 01/23/2013 07:49 PM, Steven D'Aprano wrote: Santosh Kumar wrote: Yes, Peter got it right. Peter? Which Peter? What's "it" that he got right? You have deleted all context from your post, so I have no idea what you are talking about. Right. And whatever program you are using to post is stripping out threading information, so I can't tell what post you are replying to. You're not entirely right here. Santosh's message threads correctly to mine when I look with Thunderbird. And mine is parallel to one by Peter Otten, who suggested rstrip() to get rid of the extra newline. About 10% of your posts show up as top-level (starting new threads), even though I know you're careful. So there seem to be more than one threading protocol, and the multiple protocols are fighting each other. I'd love to see a spec that I could use to (manually?) check whether the threads are right or not. the relevant timestamps (at least as seen from USA EST zone) are Santosh at 4:20 am Peter Otten at 4:46 am DaveA at 5:34 am Santosh at 9:56 am Steven D'Aprano at 7:49 pm But your message was a reply to Santosh's 9:56 am message. (I'm deleting the rest, because I'm not responding to the commandline parsing question) -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Arent these snippets equivalent?
On 1/23/2013 6:29 PM, Tim Chase wrote:
On 01/23/13 16:47, Roy Smith wrote:
while getchar() as c:
putchar(c)
That would give people (including me) the use case they're after most of
the time (call a function, assign the return value, and test it). It's
way less klunky than:
while True:
c = getchar()
if c:
# I presume you mean "if not c:" here.
break
putchar()
I was a pretty strong advocate early in one of these long threads, and
for the simple cases, it's some attractive syntactic sugar. However, I
found that it quickly blossomed into a lot of really ugly edge cases
(multiple tests, multiple results, checking for "is None" vs. false'ness
or some other condition such as "< 0"). I found that it was pretty easy
to create a generator-wrapper for this:
def getter(fn):
while True:
val = fn()
if not val: break
yield val
# DB example
cursor = conn.cursor()
for row in getter(lambda: cursor.fetchmany()):
do_something(row)
# your getchar example
for c in getter(getchar):
do_something_else(c)
This allowed me to have both the readability and customized tests (and
the ability to return multiple values). It could be expanded with
def getter(fn, is_at_end=lambda v: not v):
while True:
val = fn()
if is_at_end(val): break
yield val
which would even allow you to do things like
for line in getter(file("foo.txt"), lambda s: s.find("xxx") < 0):
print "This line has 'xxx' in it:"
print line
and those felt a lot more pythonic than any of the proposals I saw on
the list.
I agree. To me, the beauty of iterators and for loops is that they
separate production of the items of a collection from the processing of
the same items. The two processes are often quite independent, and
separating them clearly allows us to mix and match. For instance, when
summing numbers, the internal details of producing the numbers does not
matter.
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: Failed to import a "pyd: File When python intepreter embed in C++ project
On Thursday, January 24, 2013 12:28:58 AM UTC+8, Leonard, Arah wrote: > > I create a pyd File named "testPyd" with boostPython,and then I import the > > testPyd module into "test.py", it works perfect! > > > But when I embeded the python interpreter into my C++ project and run the > > "test.py", it comes out a "ImportErr: no module named testPyd". > > > It has confused me for two days and I googled for long time,but I can't > > find the answer! > > > Anybody here can help me ? > > > Thank you! > > > > > > > Ah, that sounds familiar. I have a small bit of experience with Boost. > I could be wrong, because once you start mixing it up with Boost all sorts > of weird things can happen (especially on a MS compiler, because no one tests > for Windows, let alone a pay-for compiler) but my experience has shown that > if you get that specific import error, what it actually means is just that > the PYD import failed for ANY reason. It has nothing to do with the name or > that the PYD couldn't be found. Just that somewhere, at some time during > import, it failed to fully load. > > > > In my use a lot of times it was either that a DLL that module depended > on wasn't in the path, or my favorite kicker, that some compile/link flags > between the PYD and the Python interpreter don't match well enough. (Usually > from mixing debug and release builds together.) > > > > From what I've seen anyway, the Python interpreter really doesn't like > being built in a traditional debug mode, so I always do a release build of > it. It's a little inconvenient, but in the linker flags you can still set > your PYDs to generate debug information even in release builds, so you can > still run the debugger on them when you attach to the process of the python > interpreter EXE. And especially be sure to use the RELEASE runtime library > flag (such as /MD) instead of the debug flag (such as /MDd). > > > > That's as much as I know anyway. Though depending, if you add any new > templates/libraries into Boost (such as for NumPy ndarray), you also may need > to use the /DBOOST_ALL_NO_LIB compiler macro on an MS compiler because MS > doesn't adhere to template standards correctly and you often end up with > multiply-defined functions if you don't use that macro. If I remember > correctly. (It's been a while.) > > > > That and you may be picking up variable length arrays out of your > teeth, replacing chunks of code with the use of new and delete operators. No > one tests for Microsoft and the MS compiler is way behind in adhering to > C/C++ standards, and VLAs pop up a lot. > > > > Hopefully something in all of this helped. Boost can be ... daunting. > I get it, in theory. But in practice it often hurts my head. ;) > > > > Sincerely, > > Arah Leonard Dear Lenoard, Thanks for your hearty assistance, and it finally works.I just change my RELEASE Run Time Library flag to MD, then the results come out. Before I receive your mail, I googled for kinds of solutions and tried to solve this problem, but none of them works.Yesterday, someones said that the python module in *pyd Files* should be initialized before they can be embed into C++, and a function named init should be called before importing the module, I was confused with this for a couple of hours until I saw your mail. I am new to boost.python.If I propose any questions that seems naive to you the other days, any suggestion from you would be more than welcome! Thanks again for your help to make me out of my wrong way! Sincerely, George Liu -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 FTP Weirdness
On Thu, 24 Jan 2013 01:45:31 +0100, Hans Mulder wrote:
> On 24/01/13 00:58:04, Chris Angelico wrote:
>> On Thu, Jan 24, 2013 at 7:07 AM, Nick Cash
>> wrote:
>>> Python 2.7.3 on linux
>>>
>>> This has me fairly stumped. It looks like
>>> urllib2.urlopen("ftp://some.ftp.site/path";).read()
>>> will either immediately return '' or hang indefinitely. But
>>> response = urllib2.urlopen("ftp://some.ftp.site/path";)
>>> response.read()
>>> works fine and returns what is expected. This is only an issue with
>>> urllib2, vanilla urllib doesn't do it.
>>>
>>> The site I first noticed it on is private, but I can reproduce it with
>>> "ftp://ftp2.census.gov/";.
>>
>> Confirmed on 2.6.5 on Windows, fwiw. This is extremely weird.
>
> It works fine with 2.7.3 on my Mac.
>
>> Possibly it's some kind of race condition??
>
> If urllib2 is using active mode FTP, then a firewall on your box could
> explain what you're seeing. But then, that's why active mode is hardly
> used these days.
Explain please?
I cannot see how the firewall could possible distinguish between using a
temporary variable or not in these two snippets:
# no temporary variable hangs, or fails
urllib2.urlopen("ftp://ftp2.census.gov/";).read()
# temporary variable succeeds
response = urllib2.urlopen("ftp://ftp2.census.gov/";)
response.read()
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Decrease for loop by one
I'm using this code with Sikuli so thats why I have click() for x in range(0,10): decimal_value = random.randint(1,12) if myList.count(decimal_value) < 1: egg = 'A%d.png' % (decimal_value) egg = wait(egg) click(egg.getCenter().offset(random.randint(-10,10), random.randint(-10,10))) myList.append(decimal_value) else: print x x-1 = x print x I made an array to check if the random integer already exists and then I send it to the else statement at which point I want to decrease x by 1 so that it doesn't count as one of the loops. In other languages this works but it just comes up with the output: 3 2 9 8 10 9 -- http://mail.python.org/mailman/listinfo/python-list
Re: Retrieving the full command line
On Wed, 23 Jan 2013 10:01:24 +, Oscar Benjamin wrote: > On 23 January 2013 03:58, Steven D'Aprano > wrote: >> On Wed, 23 Jan 2013 00:53:21 +, Oscar Benjamin wrote: >> >>> On 22 January 2013 23:46, Steven D'Aprano >>> wrote: [SNIP] >>> The purpose of the -m option is that you can run a script that is >>> located via the Python import path instead of an explicit file path. >>> The idea is that if '/path/to/somewhere' is in sys.path then: >>> python -m script arg1 arg2 >>> is equivalent to >>> python /path/to/somewhere/script.py arg1 arg2 >>> >>> If Python didn't modify sys.argv then 'script.py' would need to be >>> rewritten to understand that sys.argv would be in a different format >>> when it was invoked using the -m option. >> >> I don't think that it would be in a different format. Normally people >> only care about sys.argv[1:], the actual arguments. argv[0], the name >> of the script, already comes in multiple formats: absolute or relative >> paths. >> >> Currently, if I have a package __main__.py that prints sys.argv, I get >> results like this: >> >> steve@runes:~$ python3.3 /home/steve/python/testpackage/__main__.py ham >> spam eggs >> ['/home/steve/python/testpackage/__main__.py', 'ham', 'spam', 'eggs'] >> >> >> which is correct, that's what I gave on the command line. But: >> >> steve@runes:~$ python3.3 -m testpackage ham spam eggs >> ['/home/steve/python/testpackage/__main__.py', 'ham', 'spam', 'eggs'] >> >> >> The second example is lying. It should say: >> >> ['-m testpackage', 'ham', 'spam', 'eggs'] > > I don't know why you would expect this. I imagined that you would want > > ['-m', 'testpackage', 'ham', 'spam', 'eggs'] No. argv[0] is intended to be the script being called, argv[1:] for the arguments to the script. Given the two choices: 1) Break every Python script that expects argv[1:] to be the arguments to the script, forcing them to decide whether they should look at argv[1:] or argv[2:] according to whether or not argv[0] == '-m'; or 2) don't break anything, but make a very small addition to the semantics of argv[0] (was: "the path to the script", add "or -m and the name of module/package") that won't break anyone's code; there's practically no choice in the matter. > If the two were combined into one string I would expect it to at least > be a valid argument list: > > ['-mtestpackage', 'ham', 'spam', 'eggs'] Okay, fair point. I didn't consider that. Note however that there is an ambiguity between calling "python -mspam" and calling a script literally named "-mspam". But that same ambiguity exists in the shell, so I don't consider it a problem. You cannot call a script named -mspam unless you use something like this "python ./-mspam". >> If you are one of the few people who care about argv[0], then you are >> already dealing with the fact that the name of the executable script is >> not always an absolute path and therefore can vary greatly from one >> call to another. Hell, if you are on a system with soft links, the name >> of the script in the command line is not even necessarily the name of >> the module. So there's not much more effort involved in dealing with >> one extra case: > > Unless I've missed something sys.argv[0] is always a valid path to the > script. Whether it is absolute or not shouldn't matter. Sure. But if you care about argv[0] (say, you want to pull out the name of the script at runtime, instead of hard-coding it), then you need to be aware that you could be given an absolute path, a relative path, a bare script name, or the path of a softlink to the file you actually care about. Adding one more trivially simple case is not a large burden. People hardly ever care about argv[0]. At least, I don't think I ever have. But the OP does, and Python mangling argv[0] is causing him grief because it lies, claiming to have called the __main__.py of his package directly when in fact he called it with -m. > For imported > modules the path is available from __name__. For a script that is > executed rather than imported __name__ == "__main__" but the path is > accessible from sys.argv[0]. If you are one of those people who cares > about sys.argv[0] then this is probably the value that you wanted it to > contain. I'm wary about guessing what people "probably" want, and therefore lying about what they actually got. That's DWIM coding, and that almost always ends in tears. > If it were important for sys.argv to show how exactly the script was > located and executed, then why not also include the 'python3.3' command > line argument (the real argv[0])? sys.argv emulates the argv that e.g. a > C program would get. The real command line used is not exactly the same > since a Python script is not a directly executable binary, so Python > processes the argument list before passing it through. Also a good point. To some degree, we're constrained by backwards compatibility -- there's only so much change we can do
Re: Decrease for loop by one
On Thu, Jan 24, 2013 at 3:39 PM, Milter Skyler wrote: > I made an array to check if the random integer already exists and then I send > it to the else statement at which point I want to decrease x by 1 so that it > doesn't count as one of the loops. In other languages this works... A Python 'for' loop doesn't behave like that; you can't just edit the loop counter. But take a step back: what are you actually trying to accomplish? As I see it, you're trying to pick ten unique random values in the range 1-12 inclusive. Try this: random.sample(range(1,13),10) # note that range() stops *before* its second arg That'll give you a list of those integers, and it's a lot safer and more reliable than the try-fail-retry method. Your loop can become: for decimal_value in random.sample(range(1,13),10): egg = 'A%d.png' % (decimal_value) egg = wait(egg) click(egg.getCenter().offset(random.randint(-10,10), random.randint(-10,10))) By the way, this line simply won't work: x-1 = x I think you mean: x = x-1 But I strongly suggest you copy and paste directly rather than retype; it's less error-prone. Hope that helps! ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Arent these snippets equivalent?
On Wednesday, January 23, 2013 6:38:54 PM UTC-8, Terry Reedy wrote:
> On 1/23/2013 6:29 PM, Tim Chase wrote:
>
> > On 01/23/13 16:47, Roy Smith wrote:
>
> >> while getchar() as c:
>
> >> putchar(c)
>
> >>
>
> >> That would give people (including me) the use case they're after most of
>
> >> the time (call a function, assign the return value, and test it). It's
>
> >> way less klunky than:
>
> >>
>
> >> while True:
>
> >> c = getchar()
>
> >> if c:
>
> > # I presume you mean "if not c:" here.
>
> >>break
>
> >> putchar()
>
> >
>
> > I was a pretty strong advocate early in one of these long threads, and
>
> > for the simple cases, it's some attractive syntactic sugar. However, I
>
> > found that it quickly blossomed into a lot of really ugly edge cases
>
> > (multiple tests, multiple results, checking for "is None" vs. false'ness
>
> > or some other condition such as "< 0"). I found that it was pretty easy
>
> > to create a generator-wrapper for this:
>
> >
>
> >def getter(fn):
>
> > while True:
>
> >val = fn()
>
> >if not val: break
>
> >yield val
>
> >
>
> ># DB example
>
> >cursor = conn.cursor()
>
> >for row in getter(lambda: cursor.fetchmany()):
>
> > do_something(row)
>
> >
>
> ># your getchar example
>
> >for c in getter(getchar):
>
> > do_something_else(c)
>
> >
>
> > This allowed me to have both the readability and customized tests (and
>
> > the ability to return multiple values). It could be expanded with
>
> >
>
> >def getter(fn, is_at_end=lambda v: not v):
>
> > while True:
>
> >val = fn()
>
> >if is_at_end(val): break
>
> >yield val
>
> >
>
> > which would even allow you to do things like
>
> >
>
> >for line in getter(file("foo.txt"), lambda s: s.find("xxx") < 0):
>
> > print "This line has 'xxx' in it:"
>
> > print line
>
> >
>
> > and those felt a lot more pythonic than any of the proposals I saw on
>
> > the list.
>
>
>
> I agree. To me, the beauty of iterators and for loops is that they
>
> separate production of the items of a collection from the processing of
>
> the same items. The two processes are often quite independent, and
>
> separating them clearly allows us to mix and match. For instance, when
>
> summing numbers, the internal details of producing the numbers does not
>
> matter.
>
>
>
> --
>
> Terry Jan Reedy
Thanks for all the perspectives everyone. I was just curious about the
functional equivalence and I got what I needed.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Retrieving the full command line
On Thu, Jan 24, 2013 at 3:49 PM, Steven D'Aprano wrote: > Note however that there is an ambiguity between calling "python -mspam" > and calling a script literally named "-mspam". But that same ambiguity > exists in the shell, so I don't consider it a problem. You cannot call a > script named -mspam unless you use something like this "python ./-mspam". Another spanner for your works: "python -- -mspam" succeeds. That sets argv[0] to '-mspam'. > People hardly ever care about argv[0]. At least, I don't think I ever > have. But the OP does, and Python mangling argv[0] is causing him grief > because it lies, claiming to have called the __main__.py of his package > directly when in fact he called it with -m. Usually when I reference argv[0], $0, or any equivalent, it's for a usage display - eg: USAGE: $0 [opts] infile [outfile] --fooFooify the file --barBurn your computer to the ground So I don't particularly care about symlinks or relative paths (if it worked once, it'll probably work another time). But ambiguities may be an issue. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
