asfimport opened a new issue, #20265:
URL: https://github.com/apache/arrow/issues/20265
When registering extension types as is now possible in the R bindings, it
looks as though we cannot register an extension type in R and Python at the
same time:
```R
# apache/arrow@master
library(arrow, warn.conflicts = FALSE)
library(reticulate)
# this is a virtualenv with pyarrow installed against the same commit
use_virtualenv(
"/Users/deweydunnington/Desktop/rscratch/pyarrow-dev",
required = TRUE
)
pa <- import("pyarrow")
pa[["__version__"]]
#> [1] "9.0.0.dev131+g8a36f0f6c"
py_run_string("
import pyarrow as pa
class TestExtensionType(pa.ExtensionType):
def __init__(self):
super().__init__(pa.int32(), 'arrow.test_type')
def __arrow_ext_serialize__(self):
return b''
@classmethod
def __arrow_ext_deserialize__(cls, storage_type, serialized):
return cls()
pa.register_extension_type(TestExtensionType())
")
arrow::register_extension_type(
arrow::new_extension_type(int32(), "arrow.test_type")
)
#> Error: Key error: A type extension with name arrow.test_type already
defined
```
I also get a segfault if I try to surface a Python type into R (probably
because the R bindings mistakenly assume that if `type.id() == Type::EXTENSION`
then it is safe to cast to our own `ExtensionType` C++ subclass that implements
R-specific things.
This came about because the 'geoarrow' Python and 'geoarrow' R packages both
register a number of extension type definitions.
- geoarrow's Python registration:
https://github.com/jorisvandenbossche/python-geoarrow/blob/main/src/geoarrow/extension_types.py#L108-L117
- geoarrow's R registration:
https://github.com/paleolimbot/geoarrow/blob/master/R/pkg-arrow.R#L208-L223
I can also force an interaction if I build GDAL against the same Arrow
that the arrow R package is linked against and attempt to load a Feather file
saved with an extension type using the sf package. I will attempt to recreate
that interaction as well in both R and Python.
I don't know enough about linking to know to what extent this is linked to
my own development setup/build of the R package, although I think there are at
least some environments where a shared library is picked up first by the R
config script (fedora36, for example). It does look like my own R package build
is dynamically linking to libarrow.dylib.
**Reporter**: [Dewey
Dunnington](https://issues.apache.org/jira/browse/ARROW-16688) / @paleolimbot
<sub>**Note**: *This issue was originally created as
[ARROW-16688](https://issues.apache.org/jira/browse/ARROW-16688). Please see
the [migration documentation](https://github.com/apache/arrow/issues/14542) for
further details.*</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
