junrushao commented on code in PR #593: URL: https://github.com/apache/tvm-ffi/pull/593#discussion_r3581677138
########## python/tvm_ffi/cython/tvm_ffi_python_object.h: ########## @@ -0,0 +1,1032 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* + * \file tvm_ffi_python_object.h + * \brief PyObject-tying state machine: binds one Python wrapper to one C++ FFI object + * ("chandle") for the object's lifetime so identity is stable (``a.x is a.x``, + * stable ``id()`` across drop+refetch, ``f(x) is x`` for FFI returns). + * + * Split out of tvm_ffi_python_helpers.h. The design overview is the banner comment below. + */ +#ifndef TVM_FFI_PYTHON_OBJECT_H_ +#define TVM_FFI_PYTHON_OBJECT_H_ + +#include <Python.h> +#include <tvm/ffi/c_api.h> +#include <tvm/ffi/memory.h> + +// Define here to avoid dependencies on non-c headers for now +#ifndef TVM_FFI_INLINE +#if defined(_MSC_VER) +#define TVM_FFI_INLINE [[msvc::forceinline]] inline +#else +#define TVM_FFI_INLINE [[gnu::always_inline]] inline +#endif +#endif + +// Managed-dict (`__slots__ = ("__dict__",)` without an explicit dictoffset) +// is a CPython 3.11+ feature. On 3.9/3.10 such types instead use a regular +// ``tp_dictoffset != 0``, which the inactive-eligibility check catches anyway, +// so defining the flag as 0 here yields the correct (no-op) behavior. +#ifndef Py_TPFLAGS_MANAGED_DICT +#define Py_TPFLAGS_MANAGED_DICT 0 +#endif + +#include <atomic> +#include <cassert> +#include <cstring> +#include <utility> + +// ``_Interlocked*`` intrinsics for the MSVC arm of the spin-lock leaves below. <intrin.h>, not +// <windows.h>, to keep min/max etc. macros out of the Cython TU. +#if defined(_MSC_VER) && defined(Py_GIL_DISABLED) +#include <intrin.h> +#endif + +//================================================================================ Review Comment: Some extra documentation to help understand the state and state transition: ```markdown “Wrapper” means the Python object that represents a native TVM-FFI object. It is not the native object itself. For example: ```python x = outer.x ``` Conceptually: ```text Python wrapper W Native TVM-FFI object H +----------------------+ +----------------------+ | PyObject header | | refcounts | | ob_refcnt | | type_index | | ob_type | | actual object fields | | CObject.chandle --------------------> +----------------------+ +----------------------+ | ^ | +--------- tagged_pyobj in header <------+ ``` `W` is typically an instance of a Cython `CObject` subclass—such as a container, reflected dataclass, `Function`, or another registered TVM-FFI Python class. It provides Python identity, methods, properties, and Python reference counting. Its `chandle` points to the actual native object. The native object contains the real TVM-FFI data and has TVM-FFI’s strong/weak reference counts. ## What “canonical wrapper” means Previously, the same native handle could be represented by several Python objects: ```python a = outer.x b = outer.x a.same_as(b) # True: same native chandle a is b # False: two Python wrappers ``` The new mechanism chooses one Python wrapper as the canonical representative: ```python a = outer.x b = outer.x a is b # True ``` The native allocation’s `tagged_pyobj` field remembers the address of that canonical wrapper. Crucially, this is a raw pointer, not a Python reference. Merely storing `W` in `tagged_pyobj` does not increment `W.ob_refcnt` and therefore does not keep `W` alive. ## Pointer tagging `W | 1` means that the implementation converts the aligned pointer to an integer and uses its unused low bits as flags. For example, if: ```text W = 0x1000 ``` then: ```text Active: 0x1000 Inactive: 0x1001 # W | 1 InTransit: 0x1003 # W | 1 | 2 Locked: 0x1004 # W | 4 ``` `0x1001` and `0x1003` are not valid `PyObject*` values to dereference. The implementation must mask off the tag bits to recover `0x1000`. There are actually four semantic lifecycle states. `Locked` is a temporary synchronization overlay. ## Detached ```text tagged_pyobj = NULL ``` The native object exists, but it currently has no canonical Python wrapper. Typical situations: - The native object was created but has not yet been returned to Python. - A wrapper was moved from and its binding was detached. - An ineligible wrapper died and was genuinely freed. - The object came from a Python-aware allocator but has not been wrapped yet. The next ordinary FFI return creates a fresh wrapper and transitions: ```text Detached -> Active ``` For native objects not allocated by the Python allocator—such as some static objects—there is no `tagged_pyobj` field at all. Those objects use the old fresh-wrapper behavior. ## Active ```text tagged_pyobj = W ``` `W` is a live Python object. Usually: ```text W.ob_refcnt > 0 W.chandle == H ``` The wrapper’s `chandle` owns one strong native reference to `H`. The native header’s pointer back to `W` owns no Python reference. When another FFI call returns `H`: 1. Find `W` through `tagged_pyobj`. 2. Increment `W`’s Python reference count. 3. Drop the redundant native reference supplied by the FFI return. 4. Return `W`. That produces: ```python identity(x) is x ``` Possible exits from Active include: ```text Active -> Inactive last Python reference disappears, but another native owner keeps H alive Active -> Detached move, ineligible-wrapper death, or explicit rebinding Active -> freed wrapper and native object both reach end of life through the InTransit handshake ``` ## Inactive ```text tagged_pyobj = W | 1 ``` The Python wrapper’s lifetime has ended, but its raw allocation is retained. At this point: - `W.ob_refcnt` reached zero. - Cython/CPython deallocation has run. - `W` is no longer a valid Python object. - It is untracked from cyclic GC. - It no longer owns a strong native reference. - Its memory has not been passed to `PyObject_GC_Del`. - The native object remains alive because something else owns it. For example: ```python outer = Outer(Inner(42)) w = outer.x old_id = id(w) del w ``` The Python wrapper can become Inactive, but the native `Inner` remains alive because the native `Outer` field still holds it. Later: ```python w2 = outer.x assert id(w2) == old_id ``` The implementation reinitializes the cached memory: ```text raw cached storage -> zero Cython fields -> PyObject_Init -> restore ob_refcnt and ob_type -> PyObject_GC_Track -> restore chandle ``` This transitions: ```text Inactive -> Active ``` It is best understood as a new Python object lifetime created at the same address—not resurrection of the old live object. If the native object dies before another fetch, its deleter frees both the cached wrapper storage and the native allocation. ## InTransit ```text tagged_pyobj = W | 1 | 2 ``` InTransit is a short-lived teardown handshake. There are two potentially competing cleanup paths: - CPython is destroying `W`. - TVM-FFI is destroying `H`. The difficult case is that dropping `W`’s native reference might itself destroy `H`: ```text W reaches Python refcount zero -> W.__dealloc__() -> TVMFFIObjectDecRef(H) -> possibly destroys H immediately ``` Before calling `TVMFFIObjectDecRef`, the wrapper publishes InTransit. It acts like a baton between `tp_free` and the native `delete_space` callback. ### Native object survives Another native owner still references `H`: ```text Active -> Inactive | InTransit -> native DecRef, but H survives -> tp_free sees InTransit -> clears InTransit and retains W's storage -> Inactive ``` ### Wrapper owned the last native reference The DecRef destroys `H` reentrantly: ```text Active -> Inactive | InTransit -> native DecRef destroys H -> delete_space sees InTransit -> clears it and defers freeing -> tp_free sees it already cleared -> tp_free frees both W storage and H allocation ``` The rule is effectively: ```text The first teardown participant clears the baton and defers. The second participant performs the final free. ``` This prevents both participants from freeing the same memory, while also preventing each from assuming the other will do it. ## Locked ```text tagged_pyobj has bit 4 set ``` Locked is not a fifth lifecycle state. It is a free-threaded-Python synchronization bit that can overlay any semantic state: ```text Detached + Locked: 0x4 Active + Locked: W | 4 Inactive + Locked: W | 1 | 4 InTransit + Locked: W | 1 | 2 | 4 ``` When code acquires the lock, it atomically sets bit 4. It then reasons about the underlying state after removing that bit. Unlocking publishes the new semantic state without bit 4. This protects against races such as: ```text Thread A reads W from the native header Thread B decrements W's Python refcount to zero Thread B begins deallocating W Thread A attempts to Py_INCREF freed memory ``` On free-threaded Python, an Active lookup therefore: 1. Locks the tagged word. 2. Calls `PyUnstable_TryIncRef(W)`. 3. If `W` is still alive, returns it. 4. If its refcount has already reached zero, waits for deallocation to transition the state. 5. Retries as Inactive or Detached. On normal GIL builds, bit 4 is never used; the GIL already serializes these operations. The overall state flow is therefore: ```text move / ineligible death +------------------------------+ | v Detached ------> Active ------> InTransit ------> Detached/freed ^ | | | | | native object survives | | v | +----------> Inactive | | +---------------------------------+ native object returned again Locked temporarily overlays any state on free-threaded Python. ``` ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
