[
https://issues.apache.org/jira/browse/SPARK-56173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-56173:
-----------------------------
Description: (was: Enable V2 file write by default.
# Fix cache invalidation: switch from `recacheByPlan` to `recacheByPath` for
`FileTable` in `DataSourceV2Strategy.refreshCache`, matching V1 behavior. Add
`fileIndex.refresh()` after write.
# Skip `checkPartitioningMatchesV2Table` for `FileTable` (partitioning is
inferred from disk, empty for new paths).
# Delete `FallBackFileSourceV2` and remove from analyzer rule chains. Remove
`V2_FILE_WRITE_ENABLED` config.
# Regression fixes: `FileTable.columns()` restores NOT NULL constraints from
catalog schema; `FileFormatDataWriter.writeAll` wraps errors with
`TASK_WRITE_FAILED`; `V2SessionCatalog.loadTable` sets `catalogTable` on
`FileTable`. )
> flag flip, and delete FallBackFileSourceV2
> ------------------------------------------
>
> Key: SPARK-56173
> URL: https://issues.apache.org/jira/browse/SPARK-56173
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.2.0
> Reporter: Yang Jie
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
