Re: can't get urllib2 or httplib to work with tor & privoxy

2011-05-12 Thread Bob Fnord
Chris Angelico wrote: > On Tue, May 10, 2011 at 4:20 AM, Bob Fnord wrote: > > Both methods give me a 503 error... > > As a networking geek, my first thought would be to fire up a tiny > little "snoop server" and see what, exactly, the two methods are > doin

can't get urllib2 or httplib to work with tor & privoxy

2011-05-09 Thread Bob Fnord
Here's my python code: import httplib, urllib2 proxy_handler = {'http' : 'localhost:8118', 'https' : 'localhost:8118'} def connect_u2(url = 'http://ipid.shat.net/iponly/'):, proxied = urllib2.ProxyHandler(proxy_handler) opnr = urllib2.build_opener(proxied) opnr.addhe

Re: my computer is allergic to pickles

2011-03-11 Thread Bob Fnord
Miki Tebeka wrote: > > >From looking at the shelve info in the library reference, I get > > the impression it's tricky to change the values in the dict for > > existing keys and be sure they get changed on disk. > You can use writeback=True or call sync at the right places. > > > > How can you

Re: my computer is allergic to pickles

2011-03-11 Thread Bob Fnord
Peter Otten <__pete...@web.de> wrote: > Bob Fnord wrote: > > I started by using cPickle to save the instance of the class that > > contained this dict, but the pickling process started to write > > the file but ate so much memory that my computer (4 GB RAM) > &g

Re: my computer is allergic to pickles

2011-03-09 Thread Bob Fnord
Terry Reedy wrote: > On 3/7/2011 4:50 AM, Bob Fnord wrote: > > > I want a portable data file (can be moved around the filesystem > > or copied to another machine and used), > > Used only by Python or by other software? just Python > > Would a database in a f

Re: my computer is allergic to pickles

2011-03-09 Thread Bob Fnord
"Martin P. Hellwig" wrote: > On 05/03/2011 01:56, Bob Fnord wrote: > > > Any comments, suggestions? > > > No but I have a bunch of pseudo-questions :-) > > What version of python are you using? How about your OS and bitspace > (32/64)? Have you also

Re: my computer is allergic to pickles

2011-03-09 Thread Bob Fnord
Miki Tebeka wrote: > > Or, which situations does shelve suit better and which does > > marshal suit better? > shelve ease of use and the fact it uses the disk to store objects makes it a > good choice if you have a lot of object, each with a unique string key (and a > tuple of strings can be co

Re: my computer is allergic to pickles

2011-03-07 Thread Bob Fnord
MRAB wrote: > On 05/03/2011 01:56, Bob Fnord wrote: > > I'm using python to do some log file analysis and I need to store > > on disk a very large dict with tuples of strings as keys and > > lists of strings and numbers as values. > > > > I started by usi

Re: my computer is allergic to pickles

2011-03-06 Thread Bob Fnord
GSO wrote: > On 5 March 2011 02:14, MRAB wrote: > ... > >> Any comments, suggestions? > >> > > You obviously can't feed your computer pickles then. > > How about a tasty tidbit of XML? Served up in a main dish of DOM, or > serially if preferred? Well, right now it takes three lines to save t

Re: my computer is allergic to pickles

2011-03-06 Thread Bob Fnord
Miki Tebeka wrote: > > I'm using python to do some log file analysis and I need to store > > on disk a very large dict with tuples of strings as keys and > > lists of strings and numbers as values. > I recommend that you'll use the shelve module. It stores data on disk and is > more memory effic

my computer is allergic to pickles

2011-03-04 Thread Bob Fnord
I'm using python to do some log file analysis and I need to store on disk a very large dict with tuples of strings as keys and lists of strings and numbers as values. I started by using cPickle to save the instance of the class that contained this dict, but the pickling process started to write th