issues  

Messages by Thread

• [jira] [Updated] (SPARK-61157) Column operations capture their call site with a failing IPython import on every call, which makes them slow without ipykernel Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61157) Column operations capture their call site with a failing IPython import on every call, which makes them slow without ipykernel ASF GitHub Bot (Jira) via issues
• [jira] [Updated] (SPARK-61156) createDataFrame of local rows with a year-month or calendar interval raises the wrong error Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61156) createDataFrame of local rows with a year-month or calendar interval raises the wrong error ASF GitHub Bot (Jira) via issues
• [jira] [Assigned] (SPARK-61148) Leave a builder option of None unset, check the destination of copyFromLocalToFs, and raise NO_ACTIVE_SESSION for the requests of a stopped session, as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61155) Unpickle the DataFrames that foreachBatch functions capture on the Python workers of the server, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61155) Unpickle the DataFrames that foreachBatch functions capture on the Python workers of the server, in the Python client as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Created] (SPARK-61154) Raise VALUE_NOT_BETWEEN for a literal of an int beyond the range of a long, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61154) Raise VALUE_NOT_BETWEEN for a literal of an int beyond the range of a long, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61154) Raise VALUE_NOT_BETWEEN for a literal of an int beyond the range of a long, in the Python client as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Resolved] (SPARK-61148) Leave a builder option of None unset, check the destination of copyFromLocalToFs, and raise NO_ACTIVE_SESSION for the requests of a stopped session, as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61147) Refer to the column of withMetadata and pivot through the DataFrame, so a missing column is CANNOT_RESOLVE_DATAFRAME_COLUMN as in PySpark Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61150) Plan IDs are per session, so a column of another session's DataFrame resolves to a node of this one Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61153) Run the queries of progress handlers and hooks, and keep an execution reattachable when a hook adds reattach options, as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61153) Run the queries of progress handlers and hooks, and keep an execution reattachable when a hook adds reattach options, as PySpark does Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61153) Run the queries of progress handlers and hooks, and keep an execution reattachable when a hook adds reattach options, as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Updated] (SPARK-61152) mapInPandas, mapInArrow and the grouped and cogrouped applyIn functions accept a non-struct return type ASF GitHub Bot (Jira) via issues
  • [jira] [Updated] (SPARK-61152) mapInPandas, mapInArrow and the grouped and cogrouped applyIn functions accept a non-struct return type Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61152) mapInPandas, mapInArrow and the grouped and cogrouped applyIn functions accept a non-struct return type Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61144) Repartition by range without a number of partitions in repartitionByRange of columns, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61149) Retry by the policy past the bound of RetryExceptions, take SESSION_NOT_FOUND and a response already received in reattach, and bound the config read of an error, as PySpark does Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61149) Retry by the policy past the bound of RetryExceptions, take SESSION_NOT_FOUND and a response already received in reattach, and bound the config read of an error, as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Created] (SPARK-61151) Vendor pyspark.sql.streaming.listener, of the listener and its event and progress classes, in the Python client as PySpark has it Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61151) Vendor pyspark.sql.streaming.listener, of the listener and its event and progress classes, in the Python client as PySpark has it Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61151) Vendor pyspark.sql.streaming.listener, of the listener and its event and progress classes, in the Python client as PySpark has it ASF GitHub Bot (Jira) via issues
• [jira] [Updated] (SPARK-61150) Plan IDs are per session, so a column of another session's DataFrame resolves to a node of this one Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61150) Plan IDs are per session, so a column of another session's DataFrame resolves to a node of this one ASF GitHub Bot (Jira) via issues
• [jira] [Created] (SPARK-61149) Retry by the policy past the bound of RetryExceptions, take SESSION_NOT_FOUND and a response already received in reattach, and bound the config read of an error, as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61148) Leave a builder option of None unset, check the destination of copyFromLocalToFs, and raise NO_ACTIVE_SESSION for the requests of a stopped session, as PySpark does ASF GitHub Bot (Jira) via issues
  • [jira] [Updated] (SPARK-61148) Leave a builder option of None unset, check the destination of copyFromLocalToFs, and raise NO_ACTIVE_SESSION for the requests of a stopped session, as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61144) Repartition by range without a number of partitions in repartitionByRange of columns, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61148) Leave a builder option of None unset, check the destination of copyFromLocalToFs, and raise NO_ACTIVE_SESSION for the requests of a stopped session, as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61146) Create DataFrames of pandas frames whose struct fields repeat names in the Python client, as PySpark does Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61146) Create DataFrames of pandas frames whose struct fields repeat names in the Python client, as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Updated] (SPARK-61147) Refer to the column of withMetadata and pivot through the DataFrame, so a missing column is CANNOT_RESOLVE_DATAFRAME_COLUMN as in PySpark ASF GitHub Bot (Jira) via issues
  • [jira] [Updated] (SPARK-61147) Refer to the column of withMetadata and pivot through the DataFrame, so a missing column is CANNOT_RESOLVE_DATAFRAME_COLUMN as in PySpark Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61114) Run the full matrix in sampled post-merge CI Ruifeng Zheng (Jira) via issues
• [jira] [Resolved] (SPARK-61114) Run the full matrix in sampled post-merge CI Ruifeng Zheng (Jira) via issues
• [jira] [Resolved] (SPARK-61145) Calls from several Python threads on one DataFrameWriter, DataFrameWriterV2 or DataStreamWriter fail with "Already borrowed" Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61145) Calls from several Python threads on one DataFrameWriter, DataFrameWriterV2 or DataStreamWriter fail with "Already borrowed" Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61147) Refer to the column of withMetadata and pivot through the DataFrame, so a missing column is CANNOT_RESOLVE_DATAFRAME_COLUMN as in PySpark Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-60805) CREATE FUNCTION rejects function bodies that call system- or catalog-qualified functions ASF GitHub Bot (Jira) via issues
• [jira] [Updated] (SPARK-61145) Calls from several Python threads on one DataFrameWriter, DataFrameWriterV2 or DataStreamWriter fail with "Already borrowed" ASF GitHub Bot (Jira) via issues
  • [jira] [Updated] (SPARK-61145) Calls from several Python threads on one DataFrameWriter, DataFrameWriterV2 or DataStreamWriter fail with "Already borrowed" Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61145) Calls from several Python threads on one DataFrameWriter, DataFrameWriterV2 or DataStreamWriter fail with "Already borrowed" Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61089) Allow UDTF AnalyzeResult subclasses in the Rust Python client Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61144) Repartition by range without a number of partitions in repartitionByRange of columns, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61141) Run Rust WASM UDFs once per Arrow batch in the Spark Connect Rust client Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61144) Repartition by range without a number of partitions in repartitionByRange of columns, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61144) Repartition by range without a number of partitions in repartitionByRange of columns, in the Python client as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Created] (SPARK-61143) Decode the Arrow batches that the server compresses with zstd or lz4 in the Rust client, as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61122) Re-export the exceptions of pyspark.errors from pyspark.sql.utils, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61142) Make the functions of pyspark.sql.functions those of pyspark.sql.functions.builtin, which pickle by reference, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61142) Make the functions of pyspark.sql.functions those of pyspark.sql.functions.builtin, which pickle by reference, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61142) Make the functions of pyspark.sql.functions those of pyspark.sql.functions.builtin, which pickle by reference, in the Python client as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Updated] (SPARK-61143) Decode the Arrow batches that the server compresses with zstd or lz4 in the Rust client, as PySpark does Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61143) Decode the Arrow batches that the server compresses with zstd or lz4 in the Rust client, as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Resolved] (SPARK-61117) Call the hooks of SparkSession.Hook before each execution, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-60991) Show a DataFrame as the server formats it, with truncate and vertical, in Spark Connect Rust as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61082) Fix UnsafeHashedRelation build failure when many small relations are alive in one task Dongjoon Hyun (Jira) via issues
• [jira] [Commented] (SPARK-60123) Reading ORC tables with collated STRING columns fails Wayne Wang (Jira) via issues
• [jira] [Created] (SPARK-60878) Fix wrong results by dynamic partition pruning in group-based UPDATE Dongjoon Hyun (Jira) via issues
• [jira] [Assigned] (SPARK-61038) Take the triggers of DataStreamWriter.trigger, including realTime, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61092) Take the arguments of withMetadata, withWatermark and randomSplit, and the attributes of a DataFrame, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-60858) Close remaining Spark 4.3.0 client API gaps in the Rust drop-in ASF GitHub Bot (Jira) via issues
• [jira] [Updated] (SPARK-60834) Run master post-merge builds every five commits Ruifeng Zheng (Jira) via issues
• [jira] [Assigned] (SPARK-60724) Document the order of the calls of native data sources and add is_log_enabled Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-60919) Make Row hashable like a tuple in the Spark Connect Rust drop-in ASF GitHub Bot (Jira) via issues
• [jira] [Resolved] (SPARK-60720) Use Arrow columns for supported pandas string input in Spark Connect Rust Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-60927) Keep the keys of map values typed and in order in Spark Connect Rust Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-60735) Reject a native data source package that a session adds when Spark cannot use it Kousuke Saruta (Jira) via issues
• [jira] [Resolved] (SPARK-60849) Fix wrong results by ConstantFolding for size of a map with duplicate keys Dongjoon Hyun (Jira) via issues
• [jira] [Resolved] (SPARK-60739) Fix truncated results and unreleased executions in the Rust client's reattachable execution, and four drop-in parity gaps Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-60960) Keep executionInfo per DataFrame, and the operation id of every ExecutePlan error, in Spark Connect Rust as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-60726) Add an API to read a partition of a native data source as an Arrow C stream Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61016) Move post-merge CI sampling into the merge script Ruifeng Zheng (Jira) via issues
• [jira] [Created] (SPARK-60882) Accept more expressions that can not fail in `ExprUtils.canEvaluateUnconditionally` Kousuke Saruta (Jira) via issues
• [jira] [Created] (SPARK-60923) Upgrade the scheduled no-GIL Python build to Python 3.15 Ruifeng Zheng (Jira) via issues
• [jira] [Resolved] (SPARK-60662) Share field names across pandas input rows in Spark Connect Rust Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-60854) Fix `release-build.sh` finalize failing for the first release of a series Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-60958) Accept url in SparkSession.builder.remote() in Spark Connect Rust as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-60891) Upgrade `MySQL` JDBC driver to `26.10.0` Dongjoon Hyun (Jira) via issues
• [jira] [Updated] (SPARK-61016) Move post-merge CI sampling into the merge script ASF GitHub Bot (Jira) via issues
  • [jira] [Updated] (SPARK-61016) Move post-merge CI sampling into the merge script Ruifeng Zheng (Jira) via issues
  • [jira] [Updated] (SPARK-61016) Move post-merge CI sampling into the merge script Ruifeng Zheng (Jira) via issues
• [jira] [Updated] (SPARK-61135) Parse the JSON of GeometryType and GeographyType, such as bare geometry and geography(SPHERICAL), in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-56180) `build/mvn` prefer to download tgz from maven central Dongjoon Hyun (Jira) via issues
• [jira] [Updated] (SPARK-60887) Preserve option key casing in SessionState.newHadoopConfWithOptions ASF GitHub Bot (Jira) via issues
• [jira] [Updated] (SPARK-60972) Simplify DataFrameReader plan construction ASF GitHub Bot (Jira) via issues
  • [jira] [Updated] (SPARK-60972) Simplify DataFrameReader plan construction Ruifeng Zheng (Jira) via issues
• [jira] [Updated] (SPARK-60867) Run the GitHub Pages deployment daily instead of on every master push ASF GitHub Bot (Jira) via issues
• [jira] [Created] (SPARK-60917) Reduce cached artifact upload copies and accelerate SHA-256 in Spark Connect Rust DB Tsai (Jira) via issues
• [jira] [Resolved] (SPARK-61136) Collect the decimals of a negative scale in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-60973) Remove unused `VariableAssignmentUtils` from `sql/core` Dongjoon Hyun (Jira) via issues
• [jira] [Assigned] (SPARK-60986) Take datetime.timedelta values in lit() and createDataFrame in Spark Connect Rust as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61040) Parse the DDL strings of DataType.fromDDL and of the return types of UDFs on the server in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-60982) Build an array of a list or a NumPy array in lit() in Spark Connect Rust as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-60870) Run master documentation deployments every ten commits ASF GitHub Bot (Jira) via issues
• [jira] [Resolved] (SPARK-60841) LIKE on UTF8_LCASE strings can silently return wrong results after optimizer rewrites James Xu (Jira) via issues
• [jira] [Resolved] (SPARK-60993) Send the optional arguments of the SQL functions of the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-60964) Retry each reattach, and release executions that fail at once, in Spark Connect Rust as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-60997) Apply the per-RPC deadlines of PySpark Connect in the Rust client, and reattach an execution past its deadline Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-60949) Access struct fields as attributes of a Column in Spark Connect Rust as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-60997) Apply the per-RPC deadlines of PySpark Connect in the Rust client, and reattach an execution past its deadline Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-60634) Give copied UUID expressions independent random generators Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-60997) Apply the per-RPC deadlines of PySpark Connect in the Rust client, and reattach an execution past its deadline Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61131) Fix wrong results of decimal in ANSI mode when the quotient overflows ASF GitHub Bot (Jira) via issues
• [jira] [Resolved] (SPARK-60986) Take datetime.timedelta values in lit() and createDataFrame in Spark Connect Rust as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-59589) Hand Expand's varying output columns to the consumer as globals Yang Jie (Jira) via issues
• [jira] [Assigned] (SPARK-61041) Raise SESSION_NOT_SAME for the DataFrames of other sessions in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61055) Return the status and the exception of a StreamingQuery, and check the timeouts of its waits, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61113) Add DataFrame.__arrow_c_stream__, the Arrow C stream of a DataFrame, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61050) Fix `ArithmeticException` of decimal remainder, `pmod` and `div` when the integral quotient has more than 39 digits Dongjoon Hyun (Jira) via issues
• [jira] [Resolved] (SPARK-61138) Make str of a VariantVal its JSON, and createDataFrame reject rows that are VariantVals, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61117) Call the hooks of SparkSession.Hook before each execution, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61104) Convert pandas input without a schema as PySpark's Arrow conversion does in the Python client Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61016) Move post-merge CI sampling into the merge script Ruifeng Zheng (Jira) via issues
• [jira] [Assigned] (SPARK-61136) Collect the decimals of a negative scale in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61138) Make str of a VariantVal its JSON, and createDataFrame reject rows that are VariantVals, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-60933) Make every DataType pickle round-trip in the Spark Connect Rust drop-in Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-60957) Import pyspark.sql.conversion, and PandasUDFType from pyspark.sql.functions, in Spark Connect Rust as in PySpark Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61008) Get or create the sessions of SparkSession.builder of the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61122) Re-export the exceptions of pyspark.errors from pyspark.sql.utils, in the Python client as PySpark does ASF GitHub Bot (Jira) via issues
  • [jira] [Updated] (SPARK-61122) Re-export the exceptions of pyspark.errors from pyspark.sql.utils, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-60968) Treat explicit None arguments of SQL functions in Spark Connect Rust as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61008) Get or create the sessions of SparkSession.builder of the Python client as PySpark does ASF GitHub Bot (Jira) via issues
  • [jira] [Updated] (SPARK-61008) Get or create the sessions of SparkSession.builder of the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61141) Run Rust WASM UDFs once per Arrow batch in the Spark Connect Rust client Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61141) Run Rust WASM UDFs once per Arrow batch in the Spark Connect Rust client ASF GitHub Bot (Jira) via issues
• [jira] [Resolved] (SPARK-61035) Send the resource profile of mapInPandas and mapInArrow in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61141) Run Rust WASM UDFs once per Arrow batch in the Spark Connect Rust client Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61043) Take the arguments of count and sum_distinct, and warn of sumDistinct, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-60194) Improve type annotation for StreamingQueryListener sessions Tian Gao (Jira) via issues
• [jira] [Assigned] (SPARK-61035) Send the resource profile of mapInPandas and mapInArrow in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-60315) Filter raises an error with whole-stage codegen off but not on when an earlier conjunct is null Dongjoon Hyun (Jira) via issues
• [jira] [Assigned] (SPARK-61133) Add the private helpers of pyspark.sql.types, such as _infer_schema, _merge_type and _make_type_verifier, in the Python client as PySpark has them Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61134) Make DayTimeIntervalType and YearMonthIntervalType check their fields, and the types without parameters their arguments, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-60922) Construct GeometryType and GeographyType in Spark Connect Rust as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-60992) Fix marking a succeeded task as failed when a post-status-update listener fails Dongjoon Hyun (Jira) via issues
• [jira] [Created] (SPARK-61017) Support configurable intermediate storage levels in iterative classifiers Ruifeng Zheng (Jira) via issues
• [jira] [Resolved] (SPARK-60808) Runtime bloom filter built from a non-deterministic creation side can drop join rows Peter Toth (Jira) via issues
• [jira] [Resolved] (SPARK-61133) Add the private helpers of pyspark.sql.types, such as _infer_schema, _merge_type and _make_type_verifier, in the Python client as PySpark has them Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61110) Add DataFrame._repr_html_, the HTML table of eager evaluation in notebooks, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61110) Add DataFrame._repr_html_, the HTML table of eager evaluation in notebooks, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61069) Collect TIMESTAMP_NTZ values, distant timestamps and year-month intervals in the Rust client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61104) Convert pandas input without a schema as PySpark's Arrow conversion does in the Python client Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-60745) Avoid row materialization for Arrow-backed pandas string input in Spark Connect Rust Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61079) Convert pandas input with an explicit schema as PySpark's Arrow conversion does in createDataFrame of the Rust client Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61116) Port the Read, Range and SQL plans, Column.to_plan, and the type and conversion helpers of the connect import path, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-60974) Accept a DDL string schema in mapInPandas, applyInPandas and the other map functions in Spark Connect Rust as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61136) Collect the decimals of a negative scale in the Python client as PySpark does ASF GitHub Bot (Jira) via issues
  • [jira] [Updated] (SPARK-61136) Collect the decimals of a negative scale in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61122) Re-export the exceptions of pyspark.errors from pyspark.sql.utils, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61115) Add the module-level names and the connect modules that the Python client lacks, as PySpark 4.3.0 has them Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61115) Add the module-level names and the connect modules that the Python client lacks, as PySpark 4.3.0 has them ASF GitHub Bot (Jira) via issues
• [jira] [Created] (SPARK-60945) from_protobuf/to_protobuf can corrupt a shared options map when the plan is canonicalized concurrently Lihang Shen (Jira) via issues
• [jira] [Resolved] (SPARK-61124) Check the arguments of nearestByJoin, and take them, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61135) Parse the JSON of GeometryType and GeographyType, such as bare geometry and geography(SPHERICAL), and add their _from_crs, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61135) Parse the JSON of GeometryType and GeographyType, such as bare geometry and geography(SPHERICAL), and add their _from_crs, in the Python client as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Created] (SPARK-61140) Send the renames of DataFrame.withColumnsRenamed in their order, in the Rust core and the Python client, as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61140) Send the renames of DataFrame.withColumnsRenamed in their order, in the Rust core and the Python client, as PySpark does Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61140) Send the renames of DataFrame.withColumnsRenamed in their order, in the Rust core and the Python client, as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Assigned] (SPARK-61126) Make DataFrame.pandas_api take index_col, and DataFrame.plot the PySparkPlotAccessor, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61126) Make DataFrame.pandas_api take index_col, and DataFrame.plot the PySparkPlotAccessor, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61034) Send the output schema of transformWithState in the Python client as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Assigned] (SPARK-61121) Take a float frame bound past its threshold as unbounded in rowsBetween and rangeBetween, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61129) Make SparkSession and DataFrame picklable, of the active session, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61135) Parse the JSON of GeometryType and GeographyType, such as bare geometry and geography(SPHERICAL), in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61113) Add DataFrame.__arrow_c_stream__, the Arrow C stream of a DataFrame, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61129) Make SparkSession and DataFrame picklable, of the active session, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61138) Make str of a VariantVal its JSON, and createDataFrame reject rows that are VariantVals, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61138) Make str of a VariantVal its JSON, and createDataFrame reject rows that are VariantVals, in the Python client as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Resolved] (SPARK-61120) Add DataFrame._session, SparkSession._client, and the methods and results of the client that vendored modules use, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61119) Set the element type of the array literals of the Rust core, as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61138) Make str of a VariantVal its JSON in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61139) Fix wrong results when `ConstantPropagation` or `InferFiltersFromConstraints` replaces a `FLOAT` or `DOUBLE` zero with the other signed zero Kousuke Saruta (Jira) via issues
• [jira] [Updated] (SPARK-61138) Make str of a VariantVal its JSON in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61134) Make DayTimeIntervalType and YearMonthIntervalType check their fields, and the types without parameters their arguments, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61134) Make DayTimeIntervalType and YearMonthIntervalType check their fields, and the types without parameters their arguments, in the Python client as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Updated] (SPARK-61137) Make StructType give back the same StructField objects in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
  • [jira] [Updated] (SPARK-61137) Make StructType give back the same StructField objects in the Python client as PySpark does ASF GitHub Bot (Jira) via issues
• [jira] [Created] (SPARK-61137) Make StructType give back the same StructField objects in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61136) Collect the decimals of a negative scale in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61125) Pickle the handler of a UDTF when it is used, and raise UDTF_SERIALIZATION_ERROR, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61125) Pickle the handler of a UDTF when it is used, and raise UDTF_SERIALIZATION_ERROR, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61120) Add DataFrame._session, SparkSession._client, and the methods and results of the client that vendored modules use, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61128) Make ArrayType, MapType, StructField and StructType give back and pickle the user-defined types they are made of, and StructField picklable, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Created] (SPARK-61118) Add the CLI modules of pyspark.pipelines, in the Python client as PySpark has them Hyukjin Kwon (Jira) via issues
• [jira] [Comment Edited] (SPARK-55211) Create an optimizer rule to choose when to use transpiled Monica G (Jira) via issues
• [jira] [Created] (SPARK-61117) Call the hooks of SparkSession.Hook before each execution, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61123) Take a Python Enum as the literal of its value in lit, Column operators and functions, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61118) Add the CLI modules of pyspark.pipelines, and the user context extensions of the client, in the Python client as PySpark has them ASF GitHub Bot (Jira) via issues
  • [jira] [Updated] (SPARK-61118) Add the CLI modules of pyspark.pipelines, and the user context extensions of the client, in the Python client as PySpark has them Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61127) Make the rows of DataFrame.collect and toLocalIterator honor spark.sql.execution.pyspark.binaryAsBytes in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61128) Make ArrayType, MapType, StructField and StructType give back and pickle the user-defined types they are made of, and StructField picklable, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61127) Make the rows of DataFrame.collect and toLocalIterator honor spark.sql.execution.pyspark.binaryAsBytes in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-61123) Take a Python Enum as the literal of its value in lit, Column operators and functions, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Assigned] (SPARK-61124) Check the arguments of nearestByJoin, and take them, in the Python client as PySpark does Hyukjin Kwon (Jira) via issues
• [jira] [Resolved] (SPARK-60704) Avoid repeated Decimal type lookup for explicit decimal local input in Spark Connect Rust Hyukjin Kwon (Jira) via issues
• [jira] [Updated] (SPARK-61127) Make the rows of DataFrame.collect and toLocalIterator honor spark.sql.execution.pyspark.binaryAsBytes in the Python client as PySpark does ASF GitHub Bot (Jira) via issues

• Earlier messages
• Later messages