Latest Jobs for Freshers and Experienced
www.fresherjobway.com fresherjobway.com/0-1-year-software-developer-at-pc-solutions-pvt-ltd-delhi/#more-311 http://fresherjobway.com/kent-technology-pvt-ltd-walkin-call-center-executive-business-development-executive-experience-mumbai-7th-to-20th-december-2012/#more-308 fresherjobway.com/national-institute-of-technology-agartala-walkin-asst-professors-freshers-agartala-10th-december-2012/#more-306 fresherjobway.com/central-salt-marine-chemicals-research-institute-walkin-project-fellow-freshers-bhavnagar-10th-january-2013/#more-303 fresherjobway.com/central-glass-and-ceramic-research-institute-cgcri-walkin-jrf-freshers-kolkata-14th-december-2012/#more-301 -- http://mail.python.org/mailman/listinfo/python-list
Re: python 3.3 urllib.request
On 8/12/12 07:20:55, Terry Reedy wrote:
> On 12/7/2012 12:27 PM, Hans Mulder wrote:
>> On 7/12/12 13:52:52, Steeve C wrote:
>>> hello,
>>>
>>> I have a python3 script with urllib.request which have a strange
>>> behavior, here is the script :
>>>
>>> +
>>>
>>> #!/usr/bin/env python3
>>> # -*- coding: utf-8 -*-
>>>
>>> import urllib.request
>>> import sys, time
>>>
>>>
>>> url = 'http://google.com'
>>>
>>> def make_some_stuff(page, url):
>>> sys.stderr.write(time.strftime("%d/%m/%Y %H:%M:%S -> page from \"")
>>> + url + "\"\n")
>>> sys.stderr.write(str(page) + "\"\n")
>>> return True
>>>
>>> def get_page(url):
>>> while 1:
>>> try:
>>> page = urllib.request.urlopen(url)
>>> yield page
>>>
>>> except urllib.error.URLError as e:
>>> sys.stderr.write(time.strftime("%d/%m/%Y %H:%M:%S ->
>>> impossible to access to \"") + url + "\"\n")
>>> time.sleep(5)
>>> continue
>>>
>>> def main():
>>> print('in main')
>>> for page in get_page(url):
>>> make_some_stuff(page, url)
>>> time.sleep(5)
>>>
>>> if __name__ == '__main__':
>>> main()
>>> +
>>>
>>>
>>> if the computer is connected on internet (with an ethernet connection
>>> for example) and I run this script, it works like a charme :
>>> - urllib.request.urlopen return the page
>>> - make_some_stuff write in stderr
>>> - when the ethernet cable is unplug the except block handle the error
>>> while the cable is unplug, and when the cable is pluged
>>> back urllib.request.urlopen return the page and make_some_stuff write in
>>> stderr
>>>
>>> this is the normal behavior (for me, imho).
>>>
>>> but if the computer is not connected on internet (ethernet cable
>>> unpluged) and I run this script, the except block handle the error
>>> (normal), but when I plug the cable, the script continue looping
>>> and urllib.request.urlopen never return the page (so, it always
>>> go to the except block)
>>>
>>> What can I do to handle that ?
> Don't do that '-).
>> On my laptop, your script works as you'd hope: if I plug in the
>> network cable, then the next urllib request sometimes fails, but
>> the request after that succeeds.
>> This is using Python 3.3 on MacOS X 10.5.
>> What version are you running?
>>
>> What happens if you start the script with the network cable
>> plugged in, then unplug it when the first request has succeeded,
>> and then plug it in again when the next request has failed?
> I believe he said that that worked.
You're right: he said that.
> But unplugging cables is not a good idea ;-)
>
> I remember when it was recommended that all cables be plugged in and the
> the connected devices turned on when the computer was turned on and when
> devices might not be recognized unless plugged in and on when the
> computer was booted or rebooted. In other words, ports were scanned once
> as part of the boot process and adding a device required a reboot.
I also remember the time when that was true. But these day, many
devices are designed to be plugged in with the computer running,
and the OS continuously scans for new devices.
> It certainly was not that long ago when I had to reboot after the
> Internet Service went down and the cable modem had to reset.
That's a configuration problem: when the cable modem is reset, your
computer needs to rerun its "network up" script to renew its DHCP lease.
If it isn't configured to do that automatically, and you don't know
how to run it manually, then rebooting may be your only option.
This is a common problem on desktop computers (where losing the
connection to the cable modem is rare). Laptops are typically
configured to deal with connection appearing and disappearing
on both the wired and the wireless interface.
> Ethernet and usb ports and modern OSes are more forgiving. But it does
> not surprise me if on some systems something has to be presence at
> process startup to even be visible to the process.
His system may be caching the outcome of the IP address lookup.
If that's the case, I'd expect different error messages, depending
on whether the first lookup succeeded or not. But since his script
carefully avoids printing the exception message, it's hard to tell.
> I believe this is all beyond Python's control. So the only thing to do
> might be to change hardware and/or OS or have the program restart itself
> if it gets repeated errors.
I think that it would deped on the error message. If the error is
"Network is unreachable" or "No route to host", then sleeping and
trying again might work. If the error is "nodename nor servname
provided, or not known", then the script would have to restart itself.
Hope this helps,
-- HansM
--
http://mail.python.org/mailman/listinfo/python-list
Help with Singleton SafeConfigParser
I am trying to create a Singleton SafeConfigParser object to use across all the
various scripts in this application.
I tried a Singleton pattern found on the web:
class Singleton(object):
def __new__(cls):
if not hasattr(cls, '_inst'):
print "Creating Singleton Object"
cls._inst = super(Singleton, cls).__new__(cls)
else:
print "Returning established Singleton"
return cls._inst
import ConfigParser
class Options(ConfigParser.SafeConfigParser, Singleton):
def __init__(self):
print "Initialing Options"
ConfigParser.SafeConfigParser.__init__(self)
self.add_section('test')
And this doesn't work because it calls the __init__ method every time I create
the Options object:
O = Options()
print O
O.set('test','start','True')
print "from O", O.options('test')
P = Options()
print P
print "from P", P.options('test')
This results in:
Creating Singleton Object
Initialing Options
<__main__.Options object at 0x02BF4C50>
from O ['start']
Returning established Singleton
Initialing Options
<__main__.Options object at 0x02BF4C50>
from P []
I have seen older posts in this group that talk about using modules as
singletons, but this, unless I misunderstand, requires me to code the entire
API for SafeConfigParser in the module:
import ConfigParser
class Options(ConfigParser.SafeConfigParser):
def __init__(self):
ConfigParser.SafeConfigParser.__init__(self)
self.readfp(open('defaults.cfg'))
self.read(['local.txt', 'local.cfg'])
def save(self):
with open('local.txt','w') as f:
self.write(f)
__options = Options()
def set(section, name, value):
return self.__options.set(section, name, value)
def options(section):
return self.__options.options
# And so on
This seems incredibly wasteful, and to introspect my options I get a module,
not a SafeConfigParser object, so I'm wondering if there is a different way to
handle this?
Josh
--
http://mail.python.org/mailman/listinfo/python-list
Re: Help with Singleton SafeConfigParser
Josh English wrote:
> I have seen older posts in this group that talk about using modules as
singletons, but this, unless I misunderstand, requires me to code the entire
API for SafeConfigParser in the module:
>
>
> import ConfigParser
>
>
> class Options(ConfigParser.SafeConfigParser):
> def __init__(self):
> ConfigParser.SafeConfigParser.__init__(self)
> self.readfp(open('defaults.cfg'))
> self.read(['local.txt', 'local.cfg'])
>
> def save(self):
> with open('local.txt','w') as f:
> self.write(f)
>
> __options = Options()
>
> def set(section, name, value):
> return self.__options.set(section, name, value)
>
> def options(section):
> return self.__options.options
>
> # And so on
>
>
> This seems incredibly wasteful, and to introspect my options I get a
module, not a SafeConfigParser object, so I'm wondering if there is a
different way to handle this?
Two underscores trigger name mangling only in a class, not in a module.
Don't try to hide the Options instance:
# module config.py
import ConfigParser
class Options(ConfigParser.SafeConfigParser):
... # as above
options = Options()
Then use it elsewhere:
from config import options
options.set("mysection", "myoption", "myvalue")
All but the first import will find the module in the cache (sys.modules) and
therefore the same Options instance will be used. Voilà your no-nonsense
singleton.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Help with Singleton SafeConfigParser
On Saturday, December 8, 2012 9:40:07 AM UTC-8, Peter Otten wrote:
>
>
>
> Two underscores trigger name mangling only in a class, not in a module.
>
> Don't try to hide the Options instance:
>
>
>
> # module config.py
>
> import ConfigParser
>
>
>
> class Options(ConfigParser.SafeConfigParser):
>
> ... # as above
>
>
>
> options = Options()
>
>
>
> Then use it elsewhere:
>
> from config import options
>
> options.set("mysection", "myoption", "myvalue")
>
>
>
> All but the first import will find the module in the cache (sys.modules) and
>
> therefore the same Options instance will be used. Voilà your no-nonsense
>
> singleton.
Ah. I was over-thinking again. I couldn't find an example of this anywhere, and
when I saw the tirades against Singletons they mentioned "use modules" but,
well, I haven't had my morning coffee yet. I shouldn't even be trying this sort
of thing until then.
Thank you for the simple answer.
Josh
--
http://mail.python.org/mailman/listinfo/python-list
Re: Confused compare function :)
On 2012-12-08 07:17, Chris Angelico wrote: On Sat, Dec 8, 2012 at 6:01 PM, Terry Reedy wrote: Unfortunately, catching exceptions may be and often is as slow as the redundant check and even multiple redundant checks. It depends on how often you're going to catch and how often just flow through. In Python, as in most other modern languages, exceptions only cost you when they get thrown. The extra check, though, costs you in the normal case. That's where the .get method comes in handy: MISSING = object() ... value = my_dict.get(key, MISSING) if value is not MISSING: ... It could be faster if the dict often doesn't contain the key. -- http://mail.python.org/mailman/listinfo/python-list
Re: regex walktrough
On 2012-12-08 17:48, rh wrote:
Look through some code I found this and wondered about what it does:
^(?P[0-9A-Za-z-_.//]+)$
Here's my walk through:
1) ^ match at start of string
2) ?P if a match is found it will be accessible in a variable
salsipuedes
3) [0-9A-Za-z-_.//] this is the one that looks wrong to me, see below
4) + one or more from the preceeding char class
5) () the grouping we want returned (see #2)
6) $ end of the string to match against but before any newline
more on #3
the z-_ part looks wrong and seems that the - should be at the start
of the char set otherwise we get another range z-_ or does the a-z
preceeding the z-_ negate the z-_ from becoming a range? The "."
might be ok inside a char set. The two slashes look wrong but maybe
it has some special meaning in some case? I think only one slash is
needed.
I've looked at pydoc re, but it's cursory.
Python itself will help you:
>>> re.compile(r"^(?P[0-9A-Za-z-_.//]+)$", flags=re.DEBUG)
at at_beginning
subpattern 1
max_repeat 1 65535
in
range (48, 57)
range (65, 90)
range (97, 122)
literal 45
literal 95
literal 46
literal 47
literal 47
at at_end
Inside the character set: "0-9", "A-Z" and "a-z" are ranges; "-", "_",
"." and "/" are literals. Doubling the "/" is unnecessary (it has no
special meaning). "-" is a literal because it immediately follows a
range, so it can't be defining another range (if it immediately
followed a literal and wasn't immediately followed by an unescaped "]"
then it would, so r"[a-]" is the same as r"[a\-]").
As for "(?P...)", it won't be accessible in a variable
"salsipuedes", but will be accessible as a named group in the match
object:
>>> m = re.match(r"(?P[a-z]+)", "xyz")
>>> m.group("foo")
'xyz'
--
http://mail.python.org/mailman/listinfo/python-list
Re: regex walktrough
On 8/12/12 18:48:13, rh wrote:
> Look through some code I found this and wondered about what it does:
> ^(?P[0-9A-Za-z-_.//]+)$
>
> Here's my walk through:
>
> 1) ^ match at start of string
> 2) ?P if a match is found it will be accessible in a
> variable salsipuedes
I wouldn't call it a variable. If m is a match-object produced
by this regex, then m.group('salsipuedes') will return the part
that was captured.
I'm not sure, though, why you'd want to define a group that
effectively spans the whole regex. If there's a match, then
m.group(0) will return the matching substring, and
m.group('salsipuedes') will return the substring that matched
the parenthesized part of the pattern and these two substrings
will be equal, since the only bits of the pattern outside the
parenthesis are zero-width assertions.
> 3) [0-9A-Za-z-_.//] this is the one that looks wrong to me, see below
> 4) + one or more from the preceeding char class
> 5) () the grouping we want returned (see #2)
> 6) $ end of the string to match against but before any newline
>
> more on #3
> the z-_ part looks wrong and seems that the - should be at the start
> of the char set otherwise we get another range z-_ or does the a-z
> preceeding the z-_ negate the z-_ from becoming a range?
The latter: a-z is a range and block the z-_ from being a range.
Consequently, the -_ bit matches only - and _.
> The "." might be ok inside a char set.
It is. Most special characters lose their special meaning
inside a char set.
> The two slashes look wrong but maybe it has some special meaning
> in some case? I think only one slash is needed.
You're correct: there's no special meaning and only one slash
is needed. But then, a char set is a set and duplcates are
simply ignored, so it does no harm.
Perhaps the person who wrote this was confusing slashes and
backslashes.
> I've looked at pydoc re, but it's cursory.
That's one way of putting it.
Hope this helps,
-- HansM
--
http://mail.python.org/mailman/listinfo/python-list
Re: Help with Singleton SafeConfigParser
On 08/12/2012 17:48, Josh English wrote:
On Saturday, December 8, 2012 9:40:07 AM UTC-8, Peter Otten wrote:
Two underscores trigger name mangling only in a class, not in a module.
Don't try to hide the Options instance:
# module config.py
import ConfigParser
class Options(ConfigParser.SafeConfigParser):
... # as above
options = Options()
Then use it elsewhere:
from config import options
options.set("mysection", "myoption", "myvalue")
All but the first import will find the module in the cache (sys.modules) and
therefore the same Options instance will be used. Voilà your no-nonsense
singleton.
Ah. I was over-thinking again. I couldn't find an example of this anywhere, and when I
saw the tirades against Singletons they mentioned "use modules" but, well, I
haven't had my morning coffee yet. I shouldn't even be trying this sort of thing until
then.
Thank you for the simple answer.
Josh
For the benefit of the OP and others, if you want to gain more knowledge
about patterns in Python such as the Singleton, I suggest you use your
favourite search engine to find "Alex Martelli Python patterns".
--
Cheers.
Mark Lawrence.
--
http://mail.python.org/mailman/listinfo/python-list
Running a Python app on a remote server and displaying the output files
I have a Python 2.7 script at https://github.com/jhsu802701/dopplervalueinvesting . When I run the screen.py script locally, the end result is a new screen-output sub-directory (within the root directory) and a results.csv file within it. What I'm trying to do is put this script on a remote server, run this screen.py script every night, and make the results.csv file publicly readable. I've tried to do this on Google App Engine, but I can't get it to work. The Google App Engine tutorial revolves around trying to dynamically create a web site, and I haven't been able to figure out how to make anything other than an index.html file in the root directory work. HOW DO I MAKE OTHER FILES PUBLICLY READABLE? Is Google App Engine the way to go, or am I barking up the wrong tree? I understand that another route is using WebFaction, a web hosting provider that offers a whole Linux system. (Running my app on my current web host, MDDHosting, is not an option because lxml is not available without a much more costly VPS.) In summary, my questions are: 1. How do I run my Python script in Google App Engine and make the output results.csv file publicly available? 2. If Google App Engine isn't the solution for me, should I use WebFaction? (I already tried Heroku, and it didn't work for me.) 3. What are my other options? I'm willing to pay for a solution, but only if I get web hosting as well. (I'm not willing to pay for MDDHosting for my dopplervalueinvesting.com web site AND another host for running my script.) -- http://mail.python.org/mailman/listinfo/python-list
Issue with seeded map generation
Hey, all! I've managed to get my project to a semi-playable state (everything functions, if not precisely the way I'd like it to). One small issue is that when the player movs from one level to the next, the items and monsters in the previous level all 'reset' and return to the positions they had when the level was seeded. I've puzzled over (and attempted) quite a few workarounds, and had no success. I don't want to pickle the entire level (that would be overkill for what I need), but I want to update the item/monster locations so the player can drop an item and come back to it later. Should I add something to the 'drop_item' function, or call soemthing in make_map? -- http://mail.python.org/mailman/listinfo/python-list
Parsing in Python
Dear Group, I am looking at a readymade tool to resolve anaphora, and I am looking a Python based one. I checked NLTK. It has DRT parser. But I do not like that. In other parsers you have to insert grammar. But I am looking for a completely built in. If anyone can kindly suggest. Regards, Subhabrata. -- http://mail.python.org/mailman/listinfo/python-list
Re: Issue with seeded map generation
On 12/08/2012 04:32 PM, Graham Fielding wrote: Hey, all! > > I've managed to get my project to a semi-playable state (everything functions, if not precisely the way I'd like it to). One small issue is that when the player movs from one level to the next, the items and monsters in the previous level all 'reset' and return to the positions they had when the level was seeded. > > I've puzzled over (and attempted) quite a few workarounds, and had no success. I don't want to pickle the entire level (that would be overkill for what I need), but I want to update the item/monster locations so the player can drop an item and come back to it later. > > Should I add something to the 'drop_item' function, or call soemthing in make_map? > > How many levels do you have and how much does each take up in memory? It might be ok to to simply save the level in memory under its number; if the map takes up a lot of space you can make objects take up less memory by using __slots__, and then change levels by doing something like this: def change_level(num): self.levels[self.current_lvl] = self.levelmap if num in self.levels: self.levelmap = self.levels[num] else: [handle new level creation] self.current_lvl = num -m -- http://mail.python.org/mailman/listinfo/python-list
Re: Issue with seeded map generation
On 8/12/12 22:32:22, Graham Fielding wrote: > Hey, all! > > I've managed to get my project to a semi-playable state (everything > functions, if not precisely the way I'd like it to). One small issue is > that when the player moves from one level to the next, the items and > monsters in the previous level all 'reset' and return to the positions > they had when the level was seeded. > > I've puzzled over (and attempted) quite a few workarounds, and had no > success. I don't want to pickle the entire level (that would be > overkill for what I need), but I want to update the item/monster > locations so the player can drop an item and come back to it later. > > Should I add something to the 'drop_item' function, or call something > in make_map? I think pickling the entire level would be the sensible thing to do. The alternative would be to keep track of everything that changed on the level and redo all those changes of the player returns to the level. That's a lot of work. Moreover, everytime you add a feature to the game, you'd have to extend your keep_track() and redo() functions, so they can redo the new thing, too. Hope this helps, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: regex walktrough
On 8/12/12 23:19:40, rh wrote: > I reduced the expression too. Now I wonder why re.DEBUG doesn't unroll > category_word. Some other re flag? he category word consists of the '_' character and the characters for which .isalnum() return True. On my system there are 102158 characters matching '\w': >>> sum(1 for i in range(sys.maxunicode+1) ... if re.match(r'\w', chr(i))) 102158 >>> You wouldn't want to see the complete list. -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: regex walktrough
On 8/12/12 23:57:48, rh wrote: > Not sure if the \w sequence includes the - or the . or the / > I think it does not. You guessed right: >>> [ c for c in 'x-./y' if re.match(r'\w', c) ] ['x', 'y'] >>> So x and y match \w and -, . and / do not. Hope this helps, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: Running a Python app on a remote server and displaying the output files
On Sun, Dec 9, 2012 at 7:22 AM, Jason Hsu wrote: > I have a Python 2.7 script at > https://github.com/jhsu802701/dopplervalueinvesting . When I run the > screen.py script locally, the end result is a new screen-output sub-directory > (within the root directory) and a results.csv file within it. > > What I'm trying to do is put this script on a remote server, run this > screen.py script every night, and make the results.csv file publicly readable. Sounds like a cron job and a web server. > I've tried to do this on Google App Engine, but I can't get it to work. The > Google App Engine tutorial revolves around trying to dynamically create a web > site, and I haven't been able to figure out how to make anything other than > an index.html file in the root directory work. HOW DO I MAKE OTHER FILES > PUBLICLY READABLE? I don't know Google App Engine, but the normal way to do these things is to simply drop a file into your web server's designated root - for instance, /var/www/results.csv would be accessible as http://yourserver/results.csv - just put it next to index.html. But you may need to configure it, if it's set to a more secure default setup that allows only html files. > 1. How do I run my Python script in Google App Engine and make the output > results.csv file publicly available? > 2. If Google App Engine isn't the solution for me, should I use WebFaction? > (I already tried Heroku, and it didn't work for me.) > 3. What are my other options? > > I'm willing to pay for a solution, but only if I get web hosting as well. > (I'm not willing to pay for MDDHosting for my dopplervalueinvesting.com web > site AND another host for running my script.) Here's a possible solution for you. Make your screen.py web-accessible (maybe with a password or IP address check, if you don't want it public) as your means of regenerating the file, and create another one that returns it. You can then have a cron job on any other computer in the world - your own home box, for instance - that runs: wget http://yourserver/screen.py You can store the csv file as an actual file, or in a database, or whatever's convenient. You just need the other script to be able to find it. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: regex walktrough
On 2012-12-08 23:27, Hans Mulder wrote: On 8/12/12 23:19:40, rh wrote: I reduced the expression too. Now I wonder why re.DEBUG doesn't unroll category_word. Some other re flag? he category word consists of the '_' character and the characters for which .isalnum() return True. On my system there are 102158 characters matching '\w': That would be because you're using Python 3, where strings are Unicode. sum(1 for i in range(sys.maxunicode+1) ... if re.match(r'\w', chr(i))) 102158 You wouldn't want to see the complete list. The number of such codepoints depends on which version of Unicode is being supported (Unicode is evolving all the time). -- http://mail.python.org/mailman/listinfo/python-list
Re: regex walktrough
On 2012-12-08 23:34, Hans Mulder wrote: On 8/12/12 23:57:48, rh wrote: Not sure if the \w sequence includes the - or the . or the / I think it does not. You guessed right: [ c for c in 'x-./y' if re.match(r'\w', c) ] ['x', 'y'] So x and y match \w and -, . and / do not. This is shorter: >>> re.findall(r'\w', 'x-./y') ['x', 'y'] But remember that r"\w" is more than just r"[A-Za-z0-9_]" (unless you're using ASCII). -- http://mail.python.org/mailman/listinfo/python-list
Re: Issue with seeded map generation
On Sat, Dec 8, 2012 at 2:32 PM, Graham Fielding wrote: > Hey, all! > > I've managed to get my project to a semi-playable state (everything > functions, if not precisely the way I'd like it to). One small issue is > that when the player movs from one level to the next, the items and monsters > in the previous level all 'reset' and return to the positions they had when > the level was seeded. > > I've puzzled over (and attempted) quite a few workarounds, and had no > success. I don't want to pickle the entire level (that would be overkill > for what I need), but I want to update the item/monster locations so the > player can drop an item and come back to it later. Make the level generation process two-step. Step 1, build the map. Step 2, populate it with items and monsters. When the level is left, save the state and positions of the items and monsters in it along with the seed. When re-entering a level, do only step 1 of the process, and then repopulate it with the saved items and monsters. Of course, if you ever want to introduce features that can actually modify the map, then you'll need to save and reload the modifications as well. -- http://mail.python.org/mailman/listinfo/python-list
Re: Confused compare function :)
On Thursday, 6 December 2012 17:44:17 UTC+5:30, Chris Angelico wrote: > On Thu, Dec 6, 2012 at 10:47 PM, Steven D'Aprano > > wrote: > > > Not so. Which one is faster will depend on how often you expect to fail. > > > If the keys are nearly always present, then: > > > > > > try: > > > do_stuff(mydict[k]) > > > except KeyError: > > > pass > > > > > > will be faster. Setting up a try block is very fast, about as fast as > > > "pass", and faster than "if k in mydict". > > > > > > But if the key is often missing, then catching the exception will be > > > slow, and the "if k in mydict" version may be faster. It depends on how > > > often the key is missing. > > > > > > > Setting up the try/except is a constant time cost, while the > > duplicated search for k inside the dictionary might depend on various > > other factors. In the specific case of a Python dictionary, the > > membership check is fairly cheap (assuming you're not the subject of a > > hash collision attack - Py3.3 makes that a safe assumption), but if > > you were about to execute a program and wanted to first find out if it > > existed, that extra check could be ridiculously expensive, eg if the > > path takes you on a network drive - or, worse, on multiple network > > drives, which I have had occasion to do! > > > > ChrisA Not really. I remember a bug saying that only 256 hashes were required of known texts and then the randomization becomes useless. -- http://mail.python.org/mailman/listinfo/python-list
Re: Confused compare function :)
On Sun, Dec 9, 2012 at 2:07 PM, Ramchandra Apte wrote: > Not really. I remember a bug saying that only 256 hashes were required of > known texts and then the randomization becomes useless. That requires that someone be able to get you to hash some text and give back the hash. In any case, even if you _are_ dealing with the worst-case hash collision attack, all it does is stop a Python dictionary from being an exception to the general principle. If you're doing a lookup in, say, a tree, then checking if the element exists and then retrieving it means walking the tree twice - O(log n) if the tree's perfectly balanced, though a splay tree would be potentially quite efficient at that particular case. But there's still extra cost to the check. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Confused compare function :)
On Sun, 09 Dec 2012 14:22:21 +1100, Chris Angelico wrote: > On Sun, Dec 9, 2012 at 2:07 PM, Ramchandra Apte > wrote: >> Not really. I remember a bug saying that only 256 hashes were required >> of known texts and then the randomization becomes useless. > > That requires that someone be able to get you to hash some text and give > back the hash. In any case, even if you _are_ dealing with the > worst-case hash collision attack, all it does is stop a Python > dictionary from being an exception to the general principle. Dictionaries never were an exception to the general principle. Regardless of how cheap or expensive it is, whether it is a constant cost or a potentially unbound expense or somewhere in between, the "look" in "look before you leap" tests has some cost. # Look Before You Leap (LBYL) if condition(): do_this() else: do_that() # Easier to Ask Forgiveness than Permission (EAFP) try: do_this() except ConditionFailed: do_that() Depending on the cost, and how often you have to pay it, either LBYL or EAFP will be cheaper. Sometimes you can predict which one will be cheaper -- "most of the time, the key will be present" -- sometimes you can't. But there's always a choice, and the choice is not always the right choice. But... why are we focusing only on the cost? What about *correctness*? LBYL suffers from an even more critical problem, never mind the cost of the check. That problem is the gap in time between the check and when you go to do the actual work. In the fraction of a second between checking condition and calling do_this(), the situation might have changed. The file that did exist has now been deleted by another program. The key was in the dict, and now another thread has deleted it. The network was up, and now its down. These sorts of things are extremely hard to diagnose, extremely hard to prevent, and extremely hard to replicate. They lead to race conditions and "Time Of Check to Time Of Use" bugs. Many of the more tricky, subtle security vulnerabilities are due to TOCTOU bugs. So, very often, it isn't enough to Look Before You Leap. You *still* have to be prepared to Ask Forgiveness: # not good enough if os.path.exists(filename): f = open(filename) else: handle_missing_file() # should be if os.path.exists(filename): try: f = open(filename) except (IOError, OSError): handle_missing_file() else: handle_missing_file() But in that case, what's the point of the initial check? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
