I'm running a script which uses pydal outside of the web2py environment, in
the following manner:
from gluon import DAL, Field
from gluon.validators import IS_LENGTH, IS_IN_SET, IS_EMPTY_OR, IS_URL,
IS_INT_IN_RANGE
db = DAL(.., pool_size = 1, check_reserved=['all'], lazy_tables = False,
fake_migrate_all = False)
db.define_table(...)
After the script acquires data from an external source, it attempts to
insert the data into a DB on a remote server:
for idx, result_dict in enumerate(results_dicts[:]):
query__url = (db.t_blog_sources.f_url == result_dict['f_url'])
# avoid duplicates
if not db(query__url).select(db.t_blog_sources.id, limitby=(0,1)):
db.t_blog_sources.insert(**db.t_blog_sources._filter_fields(
result_dict))
if not idx % 50: # commit on every 50 items
db.commit()
db.commit()
If "results_dicts" (a list of the dicts to be inserted) is relatively small
(i.e. a few items), the inserts are usually performed withoutout a problem.
However, if "results_dicts" is fairly large, the following error almost
invariably occurs:
Traceback (most recent call last):
File "get_data.py", line 355, in <module>
if not db(query__url).select(db.t_blog_sources.id, limitby=(0,1)):
File "C:\Python27\lib\site-packages\gluon\dal.py", line 8787, in select
return adapter.select(self.query,fields,attributes)
File "C:\Python27\lib\site-packages\gluon\dal.py", line 1615, in select
return self._select_aux(sql,fields,attributes)
File "C:\Python27\lib\site-packages\gluon\dal.py", line 1580, in
_select_aux
self.execute(sql)
File "C:\Python27\lib\site-packages\gluon\dal.py", line 1693, in execute
return self.log_execute(*a, **b)
File "C:\Python27\lib\site-packages\gluon\dal.py", line 1687, in
log_execute
ret = self.cursor.execute(*a, **b)
psycopg2.DatabaseError: SSL SYSCALL error: Software caused connection abort
I can't check the logs on the remote server, because the admins have
disabled DB logging for efficiency reasons. Any ideas as to what could be
going on here, and how to fix it?
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
