Iterating generator from C

[sven . suursoho]

Does messing with signal handlers and longjmp affect Python
interpreter?

I'm trying to find solution for problem, described in
http://groups.google.com/group/comp.lang.python/browse_thread/thread/98cbae94ca4beefb/9d4d96fd0dd9fbc3
and came up with test application. It works well but i'm not sure it is
ok for long-running python interpreter?

#include 
#include 
#include 

const char *py_source =
"def fn():\n"
"yield 1\n";

jmp_buf env;
enum _ { detect, no, yes } has_buggy_gen_iternext = detect;

static sighandler_t old_abrt = SIG_DFL;

void
signal_handler (int sig)
{
  if (sig == SIGABRT)
  longjmp(env, 1);
}


int
main (int argc, char *argv[])
{
  Py_Initialize();

  PyObject *globals = PyDict_New();

  // insert function code into interpreter
  PyObject *code = PyRun_String(py_source, Py_file_input, globals,
NULL);
  Py_DECREF(code);

  // compile call to the function
  code = Py_CompileString("fn()", "", Py_eval_input);

  // do call
  PyObject *gen = PyEval_EvalCode((PyCodeObject *)code, globals, NULL);
  gen = PyObject_GetIter((PyObject *)gen);

  // detect if we are using bad Python interpreter
  if (has_buggy_gen_iternext == detect) {
  if (setjmp(env) == 0)
  // first time, set signal handler
  old_abrt = signal(SIGABRT, signal_handler);
  else {
  // jumped here from signal handler -- bad Python
  has_buggy_gen_iternext = yes;
  signal(SIGABRT, old_abrt);
  }
  }

  if (has_buggy_gen_iternext == yes)
  printf("generators are disabled\n");
  else {
  // iterate result
  PyObject *item;
  while ((item = PyIter_Next(gen))) {
  printf("> %ld\n", PyInt_AsLong(item));
  Py_DECREF(item);
  }

  if (has_buggy_gen_iternext == detect) {
  // ok, restore old signal handler
  has_buggy_gen_iternext = no;
  signal(SIGABRT, old_abrt);
  }
  }

  Py_DECREF(gen);
  Py_DECREF(code);
  Py_DECREF(globals);
  
  Py_Finalize();
  return 0;
}

-- 
http://mail.python.org/mailman/listinfo/python-list

Iterating generator from C

[Sven Suursoho]

Hi,


I am working on project that has embedded python interpreter to run  
user-specified python procedures. Those procedures might return any  
iterable object with set of result data -- basically everything for which  
iter() returns valid object (list, tuple, dict, iterator etc)

It works ok, except generator under Python 2.4 with debugging enabled (see  
http://sourceforge.net/tracker/index.php?func=detail&aid=1483133&group_id=5470&atid=105470).

Is there any way to rewrite following program to handle returned generator  
without hitting this bug?
Error handling is omitted for clarity.

#include 

const char *py_source =
 "def fn():\n"
 "   yield 1\n";

int main ()
{
   Py_Initialize();

   PyObject *globals = PyDict_New();

   // insert function code into interpreter
   PyObject *code = PyRun_String(py_source, Py_file_input, globals, NULL);
   Py_DECREF(code);

   // do call
   code = Py_CompileString("fn()", "", Py_eval_input);
   PyObject *gen = PyEval_EvalCode((PyCodeObject *)code, globals, NULL);

   // iterate result
   PyObject *item;
   while ((item = PyIter_Next(gen))) {
   printf("> %ld\n", PyInt_AsLong(item));
   Py_DECREF(item);
   }

   Py_DECREF(gen);
   Py_DECREF(code);
   Py_DECREF(globals);

   Py_Finalize();
   return 0;
}


Br,
-- 
Sven Suursoho
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Iterating generator from C

[Sven Suursoho]

Mon, 15 May 2006 16:53:12 +0300, Christian Tismer <[EMAIL PROTECTED]>:

>>   I am working on project that has embedded python interpreter to run   
>> user-specified python procedures. Those procedures might return any   
>> iterable object with set of result data -- basically everything for  
>> which  iter() returns valid object (list, tuple, dict, iterator etc)
>>  It works ok, except generator under Python 2.4 with debugging enabled  
>> (see  
>> http://sourceforge.net/tracker/index.php?func=detail&aid=1483133&group_id=5470&atid=105470).
>>  Is there any way to rewrite following program to handle returned  
>> generator  without hitting this bug?
>
> I found this bug as well, and I think the fix should be
> back-ported.
> This problem can only show up when you are comiling a C
> extension, anyway.
> Why don't you just apply the fix and compile your own?
> It is just a wrong assertion, anyway.

Unfortunately, this is not an option because I can't control used  
environment: I'm trying to improve PostgreSQL's stored procedure language  
PL/Python and this software can be used everywhere.

At first I tried to block using generators if Py_DEBUG is defined. But  
this wouldn't work because of another, overlapping bug in Fedora Core 4's  
RPM packaging system (didn't check other versions) -- it disables Py_DEBUG  
but strips -DNDEBUG from compiler's command-line i.e. no Py_DEBUG and  
still active asserts().

See:
http://archives.postgresql.org/pgsql-patches/2006-05/msg00042.php
http://archives.postgresql.org/pgsql-patches/2006-05/msg00105.php


-- 
Sven Suursoho
-- 
http://mail.python.org/mailman/listinfo/python-list