Thank you, Val K, I think this helped me lot. Good idea: to separate the problem - 1) system+scheduler, 2) served python task
I have replaced the python task with a) cycle with sleep (CPU low), b) cycle with pass (CPU >> 100%) and in both cases there was no problem with scheduler. So I know that the problem is in the scheduled task. Or in the postgres database (connections from manager will die!). But how to debug this, when scheduler - run well on development machine, - gives no info at all (traceback,..) and looks like the crashed worker is still running? :( I have isolated it more and started the task directly, without scheduler ... ./web2py.py -M -S codex2020 -R applications/codex2020/scripts/import_codex_direct.py and inside the script I call: <scheduler_task.function_name>(<scheduler_task.vars>) Now in few minutes I receive really the python error. It looks like the strange errors (listed bellow) have really something with the postgres access. I think reason is the too old contrib/pg8000 driver. It is something like listed here (pg8000+web2py): https://github.com/mfenniak/pg8000/issues/73 I have installed the current one pg8000 and it looks well, without or with scheduler. Thank you for help. Mirek --------------------------------------- File "applications/codex2020/modules/dal_import.py", line 223, in update_or_insert_owned_book db.owned_book[row.id] = owned_book File "/home/www-data/web2py/gluon/packages/dal/pydal/objects.py", line 589, in __setitem__ .update(**self._filter_fields(value)) is None: File "/home/www-data/web2py/gluon/packages/dal/pydal/objects.py", line 2250, in update ret = db._adapter.update(table, self.query, row.op_values()) File "/home/www-data/web2py/gluon/packages/dal/pydal/adapters/base.py", line 523, in update raise e error: unpack_from requires a buffer of at least 5 bytes File "applications/codex2020/modules/dal_utils.py", line 22, in answer_by_ean return db(db.answer.ean == ean).select(*flds).first() File "/home/www-data/web2py/gluon/packages/dal/pydal/objects.py", line 2211, in select return adapter.select(self.query, fields, attributes) File "/home/www-data/web2py/gluon/packages/dal/pydal/adapters/base.py", line 762, in select return self._select_aux(sql, fields, attributes, colnames) File "/home/www-data/web2py/gluon/packages/dal/pydal/adapters/base.py", line 718, in _select_aux rows = self._select_aux_execute(sql) File "/home/www-data/web2py/gluon/packages/dal/pydal/adapters/base.py", line 712, in _select_aux_execute self.execute(sql) File "/home/www-data/web2py/gluon/packages/dal/pydal/adapters/__init__.py", line 67, in wrap return f(*args, **kwargs) File "/home/www-data/web2py/gluon/packages/dal/pydal/adapters/postgres.py", line 190, in execute return super(PostgrePG8000, self).execute(*args, **kwargs) File "/home/www-data/web2py/gluon/packages/dal/pydal/adapters/__init__.py", line 67, in wrap return f(*args, **kwargs) File "/home/www-data/web2py/gluon/packages/dal/pydal/adapters/base.py", line 412, in execute rv = self.cursor.execute(command, *args[1:], **kwargs) File "/home/www-data/web2py/gluon/contrib/pg8000/core.py", line 906, in execute self._c.execute(self, operation, args) File "/home/www-data/web2py/gluon/contrib/pg8000/core.py", line 1940, in execute self.handle_messages(cursor) File "/home/www-data/web2py/gluon/contrib/pg8000/core.py", line 2084, in handle_messages self._close() File "/home/www-data/web2py/gluon/contrib/pg8000/core.py", line 1760, in _close raise OperationalError(str(e)) OperationalError: [Errno 32] Broken pipe Dne středa 16. srpna 2017 23:21:21 UTC+2 Val K napsal(a): > > Hi, Mirek! > It's hard to say something, because heaven knows what does `codex2020` > look like (are there any outgoing db-connections/url-requests/sub > processes) and what is the DB where the scheduler is mounted (Postgres?). > Exclusion method is universal - just make workers to dig something like > Fibonacci numbers from fence to lunch with smoke breaks for a minute - if > they will done the work then scheduler is OK > > > On Wednesday, August 16, 2017 at 6:32:12 PM UTC+3, Mirek Zvolský wrote: >> >> I have a task about 15 minutes long which run great on localhost with >> command >> python web2py.py -a "q" -K codex2020 -X >> >> In production, Debian9+systemd+nginx+uwsgi I have >> - emperor.uwsgi.service for web server, >> - web2py-scheduler.service for tasks, where >> ExecStart=/usr/bin/python /home/www-data/web2py/web2py.py -K >> codex2020,codex2020,codex2020 >> >> Task starts well, but after about 4 mins >> - workers are restarted (they have later different job id's), >> - scheduler_task has 1 RUNNING record >> - scheduler_run has 2 RUNNING records, first one with earlier worker, >> second one with (new) restarted worker. >> There is no information (traceback or so) in the records, and all records >> stay as RUNNING. >> >> The task is repeated from beginning. >> I have found this is (I think) because the systemd service restarts the >> task after the crash. >> >> So when I run without the systemd service, just with the shell command >> nohup /usr/bin/python /home/www-data/web2py/web2py.py -K >> codex2020,codex2020,codex2020 & >> then the 2nd record in scheduler_run is not created and the task stops. >> >> Both records in scheduler_task and scheduler_run stay as RUNNING without >> any crash info (no traceback, times_failed=0,..). >> "jobs" command output before and after the crash is: >> jobs >> [1]+ Running nohup /usr/bin/python >> /home/www-data/web2py/web2py.py -K codex2020,codex2020,codex2020 & >> root@mzDebian:/# jobs >> [1]+ Done nohup /usr/bin/python >> /home/www-data/web2py/web2py.py -K codex2020,codex2020,codex2020 >> >> I show the postgres db content using DBeaver manager and after the task >> crash I MUST RECONNECT. THE CONNECTION HANGS. >> So maybe it has nothing to do with web2py/scheduler. >> >> Any idea what could I try? >> Thanks, Mi >> > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
