Pickle Problem
Hello,
I am new to using Python and am looking at exporting some of my code
into a seperate document.
The code I am using for the pickle is:
file = open('testdoc.txt', 'w')
pickle.dump(res1.total_results_available,file)
pickle.dump(res2.total_results_available,file)
pickle.dump(res3.total_results_available,file)
file.close()
res1.total_results_available and others are simply integers which are
recalled from the Yahoo Search API and they print fine cmd or console
but when trying to pickle them they are displayed like this:
I14
.I15200
.I86000
.
But in console simply printing these attributes I get:
14
15200
86000
Can anyone help?
Many thanks,
Fabien
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pickle Problem
On Mar 3, 10:34 am, Chris Rebert wrote:
> On Tue, Mar 3, 2009 at 1:52 AM, Fab86 wrote:
> > Hello,
>
> > I am new to using Python and am looking at exporting some of my code
> > into a seperate document.
>
> > The code I am using for the pickle is:
>
> > file = open('testdoc.txt', 'w')
>
> > pickle.dump(res1.total_results_available,file)
> > pickle.dump(res2.total_results_available,file)
> > pickle.dump(res3.total_results_available,file)
> > file.close()
>
> > res1.total_results_available and others are simply integers which are
> > recalled from the Yahoo Search API and they print fine cmd or console
> > but when trying to pickle them they are displayed like this:
>
> > I14
> > .I15200
> > .I86000
> > .
>
> That's the contents of testdoc.txt after your program has written data
> to it using pickle. With pickle's default settings, the version of the
> format it uses (specifically, the oldest version) looks like that
> (integers in base-10 prefixed with 'I', entries separated by a newline
> and a period). It's *NOT* intended NOR guaranteed to be human-readable
> (indeed, with alternate settings it uses a binary format which is
> /unintelligible/ to the causal viewer). For human-readable
> serialization, use the `json` module (among other possibilities).
>
>
>
> > But in console simply printing these attributes I get:
>
> > 14
> > 15200
> > 86000
>
> That's normal. Pickle reads in the data from the file and deserializes
> it back into the proper Python objects.
>
>
>
> > Can anyone help?
>
> There's no problem to be solved here, just some explaining in order to
> deepen your understanding.
>
> Cheers,
> Chris
>
> --
> Shameless self-promotion:http://blog.rebertia.com
Thank you for your reply.
Are you saying to take a look at the python json module then?
I am wanting to store the integers in a file so that I can then run it
through some software without having to edit it. Will json enable me
to do this?
Thanks again,
Fabien
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pickle Problem
Fantastic, just what I was looking for Andrew.
Many thanks,
Fabien
On Mar 3, 11:50 am, "andrew cooke" wrote:
> Fab86 wrote:
> > I am wanting to store the integers in a file so that I can then run it
> > through some software without having to edit it. Will json enable me
> > to do this?
>
> no. ignore json - it is for something else entirely.
>
> all you need to do is to write the numbers out to a file:
>
> f = open('file.txt', 'w')
> f.write('%d\n' % 123)
> f.write('%d\n' % 456)
> f.close()
>
> for an explanation of the "%"
> seehttp://docs.python.org/library/stdtypes.html#index-1680
>
> the '\n' is a newline character so that after each number a new line is
> started.
>
> andrew
>
> > Thanks again,
>
> > Fabien
> > --
> >http://mail.python.org/mailman/listinfo/python-list
>
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pickle Problem
Thanks, this seems like a simpler way to do it.
I plan on recording 200 values to this file from the outcome of 200
Yahoo searches. Is there any type of loop I can make to do this or do
I have to have a line like "print >> f, res1.total_results_available"
200 times?
Regards,
Fabien
On Mar 3, 12:00 pm, "andrew cooke" wrote:
> maybe the following are simpler as they use print
>
> if you are using python 2.6 or python 3:
>
> >>> from __future__ import print_function
> >>> f = open('myfile.txt', 'w')
> >>> print(123, file=f)
> >>> print(456, file=f)
> >>> f.close()
>
> alternatively, in python 2.5 or 2.6:
>
> >>> f = open('myfile.txt', 'w')
> >>> print >> f, 123
> >>> print >> f, 456
> >>> f.close()
>
> andrew
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pickle Problem
I am getting res1 and res2 etc from this code:
srch1 = WebSearch(app_id=YahooKey)
srch1.query = "avoir site:.al"
res1 = srch1.parse_results()
srch2 = WebSearch(app_id=YahooKey)
srch2.query = "avoir site:.fr"
res2 = srch2.parse_results()
After identifying res1, I then use the total_results_available class
which saves res1 as an integer.
This is happening 200 times.
How could I make a for loop to do this?
Thanks,
Fabien
On Mar 3, 12:21 pm, odeits wrote:
> On Mar 3, 4:16 am, Fab86 wrote:
>
>
>
> > Thanks, this seems like a simpler way to do it.
>
> > I plan on recording 200 values to this file from the outcome of 200
> > Yahoo searches. Is there any type of loop I can make to do this or do
> > I have to have a line like "print >> f, res1.total_results_available"
> > 200 times?
>
> > Regards,
>
> > Fabien
>
> > On Mar 3, 12:00 pm, "andrew cooke" wrote:
>
> > > maybe the following are simpler as they use print
>
> > > if you are using python 2.6 or python 3:
>
> > > >>> from __future__ import print_function
> > > >>> f = open('myfile.txt', 'w')
> > > >>> print(123, file=f)
> > > >>> print(456, file=f)
> > > >>> f.close()
>
> > > alternatively, in python 2.5 or 2.6:
>
> > > >>> f = open('myfile.txt', 'w')
> > > >>> print >> f, 123
> > > >>> print >> f, 456
> > > >>> f.close()
>
> > > andrew
>
> How are you getting res1, res2? in most cases you could just use a for
> loop
>
> for res in results:
> print >> f, res.total_results_available
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pickle Problem
On Mar 3, 1:02 pm, MRAB wrote: > Fab86 wrote: > > I am getting res1 and res2 etc from this code: > > > srch1 = WebSearch(app_id=YahooKey) > > srch1.query = "avoir site:.al" > > res1 = srch1.parse_results() > > > srch2 = WebSearch(app_id=YahooKey) > > srch2.query = "avoir site:.fr" > > res2 = srch2.parse_results() > > > After identifying res1, I then use the total_results_available class > > which saves res1 as an integer. > > > This is happening 200 times. > > > How could I make a for loop to do this? > > langs = ["al", "fr"] > for lang in langs: > srch = WebSearch(app_id=YahooKey) > srch.query = "avoir site:.%s" % lang > res = srch.parse_results() > print >> f, res.total_results_available > > You might be able to move the "WebSearch" line out of the loop. > > Anyway, have you read through a tutorial? thanks for this, works perfectly just as I wanted it to. Again, thanks to everyone who has helped me. Fabien -- http://mail.python.org/mailman/listinfo/python-list
Re: Pickle Problem
On Mar 3, 6:48 pm, "Gabriel Genellina" wrote: > En Tue, 03 Mar 2009 16:39:43 -0200, Fab86 escribió: > > > I am having a bit on an issue getting my program to work. The online > > database which I am trying to contact keep timing out meaning I can > > not carry out my 200 searches without being interupted. > > > I believe that the solution would be to include an exception handler > > with a timer delay if this error occurs so wait a bit then carry on > > however having spent hours looking at this I can not get my head > > around how to do such a thing. > > Exactly. How to handle exceptions:http://docs.python.org/tutorial/errors.html > > Use time.sleep() to wait for some > time:http://docs.python.org/library/time.html#time.sleep > > -- > Gabriel Genellina Thanks for that I will give it a read. Do I need to know what error I am getting? IDLE is giving me this: Traceback (most recent call last): File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\test6.py", line 13, in res = srch.parse_results() File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\yahoo\search \__init__.py", line 765, in parse_results xml = self.get_results() File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\yahoo\search \__init__.py", line 738, in get_results stream = self.open() File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\yahoo\search \__init__.py", line 723, in open raise SearchError(err) SearchError: service temporarily unavailable [C:28] -- http://mail.python.org/mailman/listinfo/python-list
Re: Pickle Problem
On Mar 3, 8:59 pm, "Gabriel Genellina" wrote: > En Tue, 03 Mar 2009 16:50:25 -0200, Fab86 escribió: > > > > > On Mar 3, 6:48 pm, "Gabriel Genellina" wrote: > >> En Tue, 03 Mar 2009 16:39:43 -0200, Fab86 > >> escribió: > > >> > I am having a bit on an issue getting my program to work. The online > >> > database which I am trying to contact keep timing out meaning I can > >> > not carry out my 200 searches without being interupted. > > >> > I believe that the solution would be to include an exception handler > >> > with a timer delay if this error occurs so wait a bit then carry on > >> > however having spent hours looking at this I can not get my head > >> > around how to do such a thing. > > >> Exactly. How to handle > >> exceptions:http://docs.python.org/tutorial/errors.html > > >> Use time.sleep() to wait for some > >> time:http://docs.python.org/library/time.html#time.sleep > > > Thanks for that I will give it a read. > > > Do I need to know what error I am getting? > > Usually it's best to be as specific as possible. > > > > > IDLE is giving me this: > > > Traceback (most recent call last): > > File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\test6.py", line > > 13, in > > res = srch.parse_results() > > File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\yahoo\search > > \__init__.py", line 765, in parse_results > > xml = self.get_results() > > File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\yahoo\search > > \__init__.py", line 738, in get_results > > stream = self.open() > > File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\yahoo\search > > \__init__.py", line 723, in open > > raise SearchError(err) > > SearchError: service temporarily unavailable [C:28] > > That means that, in line 13 of test6.py [first line in the stack trace], > after several function calls [intermediate lines in the stack trace] > leading to function open in the search package [last line of the stack > trace], a SearchError exception was raised. > If you want to catch that exception: > > try: > ... some code ... > except SearchError: > ... code to handle the exception ... > > How to "spell" exactly the exception name should appear in the > documentation; might be yahoo.SearchError, or yahoo.search.SearchError, or > yahoo.errors.SearchError, or similar. > > -- > Gabriel Genellina I have been trying except SearchError: however I get the error: Traceback (most recent call last): File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\timeDelay.py", line 19, in except SearchError: NameError: name 'SearchError' is not defined I have searched all documents for terms along the lines of searcherror but am finding nothing.. Any ideas? -- http://mail.python.org/mailman/listinfo/python-list
Re: Pickle Problem
On Mar 4, 12:00 am, MRAB wrote: > Fab86 wrote: > > On Mar 3, 8:59 pm, "Gabriel Genellina" wrote: > >> En Tue, 03 Mar 2009 16:50:25 -0200, Fab86 escribió: > > >>> On Mar 3, 6:48 pm, "Gabriel Genellina" wrote: > >>>> En Tue, 03 Mar 2009 16:39:43 -0200, Fab86 > >>>> escribió: > >>>>> I am having a bit on an issue getting my program to work. The online > >>>>> database which I am trying to contact keep timing out meaning I can > >>>>> not carry out my 200 searches without being interupted. > >>>>> I believe that the solution would be to include an exception handler > >>>>> with a timer delay if this error occurs so wait a bit then carry on > >>>>> however having spent hours looking at this I can not get my head > >>>>> around how to do such a thing. > >>>> Exactly. How to handle > >>>> exceptions:http://docs.python.org/tutorial/errors.html > >>>> Use time.sleep() to wait for some > >>>> time:http://docs.python.org/library/time.html#time.sleep > >>> Thanks for that I will give it a read. > >>> Do I need to know what error I am getting? > >> Usually it's best to be as specific as possible. > > >>> IDLE is giving me this: > >>> Traceback (most recent call last): > >>> File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\test6.py", line > >>> 13, in > >>> res = srch.parse_results() > >>> File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\yahoo\search > >>> \__init__.py", line 765, in parse_results > >>> xml = self.get_results() > >>> File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\yahoo\search > >>> \__init__.py", line 738, in get_results > >>> stream = self.open() > >>> File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\yahoo\search > >>> \__init__.py", line 723, in open > >>> raise SearchError(err) > >>> SearchError: service temporarily unavailable [C:28] > >> That means that, in line 13 of test6.py [first line in the stack trace], > >> after several function calls [intermediate lines in the stack trace] > >> leading to function open in the search package [last line of the stack > >> trace], a SearchError exception was raised. > >> If you want to catch that exception: > > >> try: > >> ... some code ... > >> except SearchError: > >> ... code to handle the exception ... > > >> How to "spell" exactly the exception name should appear in the > >> documentation; might be yahoo.SearchError, or yahoo.search.SearchError, or > >> > >> yahoo.errors.SearchError, or similar. > > > I have been trying except SearchError: however I get the error: > > > Traceback (most recent call last): > > File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\timeDelay.py", > > line 19, in > > except SearchError: > > NameError: name 'SearchError' is not defined > > > I have searched all documents for terms along the lines of searcherror > > but am finding nothing.. > > > Any ideas? > > It's defined in the module you imported to get the search functionality. I imported: from yahoo.search.web import WebSearch However there is nothing re SearchError in that doc or in the .py. I can only find a reference to SearchError in the __init__ file as a class called SearchError -- http://mail.python.org/mailman/listinfo/python-list
Re: Pickle Problem
On Mar 4, 1:40 am, "Gabriel Genellina" wrote: > En Tue, 03 Mar 2009 23:11:30 -0200, Fab86 escribió: > > > > > On Mar 4, 12:00 am, MRAB wrote: > >> Fab86 wrote: > >> > On Mar 3, 8:59 pm, "Gabriel Genellina" wrote: > >> >> How to "spell" exactly the exception name should appear in the > >> >> documentation; might be yahoo.SearchError, or > >> yahoo.search.SearchError, or > >> >> yahoo.errors.SearchError, or similar. > > >> > I have been trying except SearchError: however I get the error: > > >> > Traceback (most recent call last): > >> > File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\timeDelay.py", > >> > line 19, in > >> > except SearchError: > >> > NameError: name 'SearchError' is not defined > > >> > I have searched all documents for terms along the lines of searcherror > >> > but am finding nothing.. > > >> It's defined in the module you imported to get the search functionality. > > > I imported: > > from yahoo.search.web import WebSearch > > > However there is nothing re SearchError in that doc or in the .py. > > > I can only find a reference to SearchError in the __init__ file as a > > class called SearchError > > The __init__.py indicates a package > <http://docs.python.org/tutorial/modules.html#packages> > You didn't tell the package name (the name of the directory containing > __init__.py) so this is somewhat generic. If the package name is foo, use: > from foo import SearchError > If foo is a subpackage under bar, use: > from bar.foo import SearchError > > It *might* be: > from yahoo.search.web import WebSearch > or perhaps: > from yahoo.search import WebSearch > > You can enter those lines in the interactive interpreter to discover the > right form. (This really ought to have been documented) > > -- > Gabriel Genellina Ok, I managed to import the correct error class (was in a non expected place) Just as I thought I was finished, I encountered a final problem. I am running a while loop which is constantly adding search results to a file, like this (print >> f, res.total_results_available). I have put an exception in that if it times out, it simply tries again until its finished. The problem is that the loop re-write all new results continuing on from the previous searches. I would like to somehow delete all in that file and start again. I thought this could simply be achieved by putting f.close() in the exception and then it re-writes it however I am getting this error: Traceback (most recent call last): File "/home/csunix/scs5fjnh/FYProj/Python/pYsearch-3.1/test9.py", line 17, in print >> f, res.total_results_available ValueError: I/O operation on closed file Is there another way rather than closing the file? Is it possible to delete all within the file? Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: Pickle Problem
On Mar 4, 2:49 pm, MRAB wrote: > Fab86 wrote: > > On Mar 4, 1:40 am, "Gabriel Genellina" wrote: > >> En Tue, 03 Mar 2009 23:11:30 -0200, Fab86 escribió: > > >>> On Mar 4, 12:00 am, MRAB wrote: > >>>> Fab86 wrote: > >>>>> On Mar 3, 8:59 pm, "Gabriel Genellina" wrote: > >>>>>> How to "spell" exactly the exception name should appear in the > >>>>>> documentation; might be yahoo.SearchError, or > >>>> yahoo.search.SearchError, or > >>>>>> yahoo.errors.SearchError, or similar. > >>>>> I have been trying except SearchError: however I get the error: > >>>>> Traceback (most recent call last): > >>>>> File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\timeDelay.py", > >>>>> line 19, in > >>>>> except SearchError: > >>>>> NameError: name 'SearchError' is not defined > >>>>> I have searched all documents for terms along the lines of searcherror > >>>>> but am finding nothing.. > >>>> It's defined in the module you imported to get the search functionality. > >>> I imported: > >>> from yahoo.search.web import WebSearch > >>> However there is nothing re SearchError in that doc or in the .py. > >>> I can only find a reference to SearchError in the __init__ file as a > >>> class called SearchError > >> The __init__.py indicates a package > >> <http://docs.python.org/tutorial/modules.html#packages> > >> You didn't tell the package name (the name of the directory containing > >> __init__.py) so this is somewhat generic. If the package name is foo, use: > >> from foo import SearchError > >> If foo is a subpackage under bar, use: > >> from bar.foo import SearchError > > >> It *might* be: > >> from yahoo.search.web import WebSearch > >> or perhaps: > >> from yahoo.search import WebSearch > > >> You can enter those lines in the interactive interpreter to discover the > >> right form. (This really ought to have been documented) > > >> -- > >> Gabriel Genellina > > > Ok, I managed to import the correct error class (was in a non expected > > place) > > > Just as I thought I was finished, I encountered a final problem. I am > > running a while loop which is constantly adding search results to a > > file, like this (print >> f, res.total_results_available). I have put > > an exception in that if it times out, it simply tries again until its > > finished. The problem is that the loop re-write all new results > > continuing on from the previous searches. I would like to somehow > > delete all in that file and start again. > > > I thought this could simply be achieved by putting f.close() in the > > exception and then it re-writes it however I am getting this error: > > > Traceback (most recent call last): > > File "/home/csunix/scs5fjnh/FYProj/Python/pYsearch-3.1/test9.py", > > line 17, in > > print >> f, res.total_results_available > > ValueError: I/O operation on closed file > > > Is there another way rather than closing the file? Is it possible to > > delete all within the file? > > You could do: > > f.seek(0) > f.truncate() > > I hope you don't just discard all the results you've got so far and then > start from the beginning again. MRAB, yes I do, thats the only way I can I get it to work. I wouldnt know how to make the program read the expection and then carry of from the last search term. Is that even possible? -- http://mail.python.org/mailman/listinfo/python-list
While loop
Hello, I am currently working on my program which send queries to Yahoo and then saves them into a flatfile. The problem I have is that I need to conduct 200 searches and Yahoo typically times out during the search with an error. I have caught the error and told it to time.sleep(10) however I can only think of deleting the flatfile and starting again hoping that it will not time out again. Is it possible to get the program to catch the exception, wait 10 seconds, then carry of from where it was rather than starting again? If so, could someone please inform me how to do this? Thanks, Fabien -- http://mail.python.org/mailman/listinfo/python-list
Re: While loop
On Mar 5, 5:23 pm, Marco Mariani wrote:
> Fab86 wrote:
> > Is it possible to get the program to catch the exception, wait 10
> > seconds, then carry of from where it was rather than starting again?
>
> something like this? probably works in PASCAL as well :)
>
> > i=0
> > while i < len(stuff):
> > try:
> > do_with(stuff[i])
> > except SomeError:
> > sleep(10)
> > continue
> > i+=1
>
>
using sleep and then continue just makes the search start from the
first search term like before.. Would it be easier to understand if I
posted sections of my code?
i = 0
k = open('blah', 'w')
domains = ["au", "ca", "nl", "be", "...]
srch = WebSearch(app_id=YahooKey)
while i<200:
try:
for domain in domains:
srch.query = "test site:.%s" % domain
res = srch.parse_results()
print >> k, res.total_results_available
i = i + 1
except SearchError:
(I currently close then reopen document here then restart i to 0)
Any ideas?
--
http://mail.python.org/mailman/listinfo/python-list
Re: While loop
On Mar 5, 5:49 pm, Fab86 wrote:
> On Mar 5, 5:23 pm, Marco Mariani wrote:
>
>
>
> > Fab86 wrote:
> > > Is it possible to get the program to catch the exception, wait 10
> > > seconds, then carry of from where it was rather than starting again?
>
> > something like this? probably works in PASCAL as well :)
>
> > > i=0
> > > while i < len(stuff):
> > > try:
> > > do_with(stuff[i])
> > > except SomeError:
> > > sleep(10)
> > > continue
> > > i+=1
>
> using sleep and then continue just makes the search start from the
> first search term like before.. Would it be easier to understand if I
> posted sections of my code?
>
> i = 0
> k = open('blah', 'w')
> domains = ["au", "ca", "nl", "be", "...]
> srch = WebSearch(app_id=YahooKey)
>
> while i<200:
> try:
> for domain in domains:
> srch.query = "test site:.%s" % domain
> res = srch.parse_results()
> print >> k, res.total_results_available
> i = i + 1
>
> except SearchError:
>
> (I currently close then reopen document here then restart i to 0)
>
> Any ideas?
All sorted now, thanks.
--
http://mail.python.org/mailman/listinfo/python-list
