On 2022-09-30 17:02, Jen Kris wrote:
Thanks very much for your detailed reply. I have a few followup
questions.
You said, “Some functions return an object that has already been
incref'ed ("new reference"). This occurs when it has either created a
new object (the refcount will be 1) or has returned a pointer to an
existing object (the refcount will be > 1 because it has been
incref'ed). Other functions return an object that hasn't been
incref'ed. This occurs when you're looking up something, for example,
looking at a member of a list or the value of an attribute.”
In the official docs some functions show “Return value: New reference”
and others do not. Is there any reason why I should not just INCREF
on every new object, regardless of whether it’s a new reference or
not, and DECREF when I am finished with it? The answer at
https://stackoverflow.com/questions/59870703/python-c-extension-need-to-py-incref-a-borrowed-reference-if-not-returning-it-to
says “With out-of-order execution, the INCREF/DECREF are basically
free operations, so performance is no reason to leave them out.”
Doing so means I don’t have to check each object to see if it needs to
be INCREF’d or not, and that is a big help.
It's OK to INCREF them, provided that you DECREF them when you no longer
need them, and remember that if it's a "new reference" you'd need to
DECREF it twice.
Also:
What is a borrowed reference, and how does it effect reference
counting? According to
https://jayrambhia.com/blog/pythonc-api-reference-counting, “Use
Py_INCREF on a borrowed PyObject pointer you already have. This
increments the reference count on the object, and obligates you to
dispose of it properly.” So I guess it’s yes, but I’m confused by
“pointer you already have.”
A borrowed reference is when it hasn't been INCREFed.
You can think of INCREFing as a way of indicating ownership, which is
often shared ownership (refcount > 1). When you're borrowing a
reference, you're using it temporarily, but not claiming ownership. When
the last owner releases its ownership (DECREF reduces the refcount to
0), the object can be garbage collected.
When, say, you lookup an attribute, or get an object from a list with
PyList_GetItem, it won't have been INCREFed. You're using it
temporarily, just borrowing a reference.
What does it mean to steal a reference? If a function steals a
reference does it have to decref it without incref (because it’s stolen)?
When function steals a reference, it's claiming ownership but not
INCREFing it.
Finally, you said:
if (pMod_random == 0x0){
PyErr_Print();
Leaks here because of the refcount
Assuming pMod_random is not null, why would this leak?
It's pName_random that's the leak.
PyUnicode_FromString("random") will either create and return a new
object for the string "random" (refcount == 1) or return a reference to
an existing object (refcount > 1). You need to DECREF it before
returning from the function.
Suppose it created a new object. You call the function, it creates an
object, you use it, then return from the function. The object still
exists, but there's no reference to it. Now call the function again. It
creates another object, you use it, then return from the function. You
now have 2 objects with no reference to them.
Thanks again for your input on this question.
Jen
Sep 29, 2022, 17:33 by pyt...@mrabarnett.plus.com:
On 2022-09-30 01:02, MRAB wrote:
On 2022-09-29 23:41, Jen Kris wrote:
I just solved this C API problem, and I’m posting the
answer to help anyone else who might need it.
[snip]
What I like to do is write comments that state which variables
hold a reference, followed by '+' if it's a new reference
(incref'ed) and '?' if it could be null. '+?' means that it's
probably a new reference but could be null. Once I know that it's
not null, I can remove the '?', and once I've decref'ed it (if
required) and no longer need it, I remobe it from the comment.
Clearing up references, as soon as they're not needed, helps to
keep the number of current references more manageable.
int64_t Get_LibModules(int64_t * return_array) {
PyObject * pName_random = PyUnicode_FromString("random");
//> pName_random+?
if (!pName_random) {
PyErr_Print();
return 1;
}
//> pName_random+
PyObject * pMod_random = PyImport_Import(pName_random);
//> pName_random+ pMod_random+?
Py_DECREF(pName_random);
//> pMod_random+?
if (!pMod_random) {
PyErr_Print();
return 1;
}
//> pMod_random+
PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed");
//> pMod_random+ pAttr_seed?
if (!pAttr_seed) {
Py_DECREF(pMod_random);
PyErr_Print();
return 1;
}
//> pMod_random+ pAttr_seed
PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random,
"randrange");
//> pMod_random+ pAttr_seed pAttr_randrange?
Py_DECREF(pMod_random);
//> pAttr_seed pAttr_randrange?
if (!pAttr_randrange) {
PyErr_Print();
return 1;
}
//> pAttr_seed pAttr_randrange
return_array[0] = (int64_t)pAttr_seed;
return_array[1] = (int64_t)pAttr_randrange;
return 0;
}
int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) {
PyObject * value_ptr = PyLong_FromLong(value_1);
//> value_ptr+?
if (!!value_ptr) {
PyErr_Print();
return 1;
}
//> value_ptr+
PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed,
value_ptr, NULL);
//> value_ptr+ p_seed_calc+?
Py_DECREF(value_ptr);
//> p_seed_calc+?
if (!p_seed_calc) {
PyErr_Print();
return 1;
}
//> p_seed_calc+
Py_DECREF(p_seed_calc);
return 0;
}
int64_t C_API_12(PyObject * pAttr_randrange, Py_ssize_t value_1) {
PyObject * value_ptr = PyLong_FromLong(value_1);
//> value_ptr+?
if (!value_ptr) {
PyErr_Print();
return 1;
}
//> value_ptr+
PyObject * p_randrange_calc =
PyObject_CallFunctionObjArgs(pAttr_randrange, value_ptr, NULL);
//> value_ptr+ p_randrange_calc+?
Py_DECREF(value_ptr);
//> p_randrange_calc+?
if (!p_randrange_calc) {
PyErr_Print();
return 1;
}
//Prepare return values
//> p_randrange_calc+
return_val = PyLong_AsLong(p_randrange_calc);
Py_DECREF(p_randrange_calc);
return return_val;
}
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
