mesejo opened a new issue, #17745: URL: https://github.com/apache/datafusion/issues/17745
### Is your feature request related to a problem or challenge? The Python errors are badly mangled in datafusion-python, see https://github.com/apache/datafusion-python/issues/1226. Basically, when going from Python -> Rust -> Python, the original Python stack trace is lost. We could use DataFusionError::External to store the original PyErr, as demonstrated in this [POC](https://github.com/mesejo/arrow-datafusion-python/tree/fix/unmangle-errors), but I think the right place for this transformation is the From trait. ### Describe the solution you'd like A solution would be to change the [From trait](https://github.com/apache/datafusion/blob/main/datafusion/common/src/pyarrow.rs#L29) defined in pyarrow.rs to ```rust use pyo3::exceptions::{PyException, PyNotImplementedError}; use pyo3::prelude::PyErr; use pyo3::types::{PyAnyMethods, PyList}; use pyo3::{Bound, FromPyObject, IntoPyObject, PyAny, PyObject, PyResult, Python}; use crate::{DataFusionError, ScalarValue}; impl From<DataFusionError> for PyErr { fn from(err: DataFusionError) -> PyErr { match err { DataFusionError::External(boxed) => match boxed.downcast::<PyErr>() { Ok(py_err) => *py_err, Err(original_boxed) => PyException::new_err(original_boxed.to_string()), }, DataFusionError::NotImplemented(message) => PyNotImplementedError::new_err(message), _ => PyException::new_err(err.to_string()), } } } ``` ### Describe alternatives you've considered _No response_ ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
