gabotechs opened a new pull request, #49: URL: https://github.com/apache/datafusion-iceberg/pull/49
## Summary Organize the DataFusion integration by domain so reads, writes, catalog code, metadata providers, and table providers have clear module boundaries. Keep and focused on module declarations and public re-exports. ## Validation - - running 98 tests test error::tests::maps_non_iceberg_datafusion_errors_to_unexpected ... ok test error::tests::encodes_iceberg_errors_with_native_datafusion_variants ... ok test error::tests::roundtrips_iceberg_error_kind_and_message ... ok test catalog::schema::tests::test_deregister_nonexistent_table_returns_none ... ok test read::predicate::tests::test_predicate_conversion_with_date_cast ... ok test read::predicate::tests::test_predicate_conversion_with_isnan ... ok test read::predicate::tests::test_predicate_conversion_with_cast ... ok test read::predicate::tests::test_predicate_conversion_with_complex_binary_expr ... ok test read::predicate::tests::test_predicate_conversion_with_binary ... ok test read::predicate::tests::test_predicate_conversion_with_and_condition_unsupported ... ok test read::predicate::tests::test_predicate_conversion_with_and_condition ... ok test read::predicate::tests::test_predicate_conversion_with_isnan_abs ... ok test read::predicate::tests::test_predicate_conversion_with_complex_binary_expr_unsupported ... ok test read::predicate::tests::test_predicate_conversion_with_and_condition_both_unsupported ... ok test read::predicate::tests::test_predicate_conversion_with_ilike ... ok test read::predicate::tests::test_predicate_conversion_with_isnan_and_other_complex_condition ... ok test read::predicate::tests::test_predicate_conversion_with_isnan_and_other_condition ... ok test read::predicate::tests::test_predicate_conversion_with_like_complex_pattern ... ok test read::predicate::tests::test_predicate_conversion_with_like_empty_prefix ... ok test read::predicate::tests::test_predicate_conversion_with_like_and_other_conditions ... ok test read::predicate::tests::test_predicate_conversion_with_isnan_nested_expr ... ok test catalog::schema::tests::test_register_table_with_data_fails ... ok test read::predicate::tests::test_predicate_conversion_with_like_special_characters ... ok test read::predicate::tests::test_predicate_conversion_with_isnan_negation ... ok test read::predicate::tests::test_predicate_conversion_with_like_underscore_wildcard ... ok test read::predicate::tests::test_predicate_conversion_with_like_starts_with ... ok test read::predicate::tests::test_predicate_conversion_with_like_no_wildcard ... ok test catalog::schema::tests::test_deregister_table_succeeds ... ok test read::predicate::tests::test_predicate_conversion_with_like_unicode ... ok test catalog::schema::tests::test_register_duplicate_table_fails ... ok test read::predicate::tests::test_predicate_conversion_with_isnan_additive ... ok test read::predicate::tests::test_predicate_conversion_with_isnan_multiplicative ... ok test catalog::schema::tests::test_register_empty_table_succeeds ... ok test read::predicate::tests::test_scalar_value_to_datum_boolean ... ok test read::predicate::tests::test_scalar_value_to_datum_binary ... ok test read::predicate::tests::test_scalar_value_to_datum_timestamp ... ok test read::predicate::tests::test_predicate_conversion_with_one_and_expr_supported ... ok test read::predicate::tests::test_predicate_conversion_with_not_like_starts_with ... ok test read::predicate::tests::test_predicate_conversion_with_not_isnan ... ok test read::predicate::tests::test_predicate_conversion_with_or_condition_supported ... ok test read::predicate::tests::test_predicate_conversion_with_or_condition_unsupported ... ok test read::predicate::tests::test_predicate_conversion_with_single_condition_rev ... ok test read::predicate::tests::test_predicate_conversion_with_isnan_unsupported_arg ... ok test table_provider::factory::tests::test_schema_of_created_table ... ok test table_provider::catalog_table_provider::tests::test_scan_rejects_out_of_range_projection ... ok test read::predicate::tests::test_predicate_conversion_with_single_unsupported_condition ... ok test table_provider::static_table_provider::tests::test_limit_pushdown_static_provider ... ok test table_provider::static_table_provider::tests::test_no_limit_pushdown ... ok test table_provider::factory::tests::test_schema_of_created_external_table_sql ... ok test table_provider::catalog_table_provider::tests::test_catalog_backed_provider_rejects_non_append_op ... ok test table_provider::catalog_table_provider::tests::test_catalog_backed_provider_creation ... ok test table_provider::catalog_table_provider::tests::test_limit_pushdown_catalog_backed_provider ... ok test write::commit::tests::test_iceberg_commit_exec_rejects_multiple_input_partitions ... ok test table_provider::catalog_table_provider::tests::test_insert_plan_fanout_enabled_no_sort ... ok test table_provider::static_table_provider::tests::test_static_provider_from_table ... ok test table_provider::catalog_table_provider::tests::test_insert_plan_fanout_disabled_has_sort ... ok test write::commit::tests::test_iceberg_commit_exec_empty_insert ... ok test table_provider::catalog_table_provider::tests::test_physical_input_schema_consistent_with_logical_input_schema ... ok test table_provider::catalog_table_provider::tests::test_catalog_backed_provider_scan ... ok test read::predicate::tests::test_predicate_conversion_with_single_condition ... ok test write::project::tests::test_partition_calculator_basic ... ok test write::project::tests::test_nested_partition ... ok test write::project::tests::test_partition_expr_evaluate ... ok test write::project::tests::test_partition_expr_differs_on_schema_with_shared_id ... ok test table_provider::static_table_provider::tests::test_static_provider_from_snapshot ... ok test write::project::tests::test_partition_expr_rebuilds_from_its_retained_parts ... ok test write::project::tests::test_partition_expr_equality_is_value_based ... ok test write::project::tests::test_partition_expr_with_projection ... ok test write::project::tests::test_try_new_error_paths ... ok test table_provider::static_table_provider::tests::test_static_provider_rejects_writes ... ok test write::exec::tests::test_iceberg_write_exec_advertises_result_schema ... ok test write::repartition::tests::test_bucket_aware_partitioning ... ok test write::repartition::tests::test_none_distribution_mode_fallback ... ok test write::repartition::tests::test_partition_column_with_temporal_transforms_uses_round_robin ... ok test write::repartition::tests::test_partition_count_validation ... ok test write::project::tests::test_schema_validation_mismatched_schemas ... ok test table_provider::static_table_provider::tests::test_static_provider_scan ... ok test write::repartition::tests::test_datafusion_repartitioning_integration ... ok test write::exec::tests::test_iceberg_write_exec ... ok test write::repartition::tests::test_mixed_transforms_use_hash_partitioning ... ok test write::repartition::tests::test_partition_column_with_identity_transforms_uses_hash ... ok test write::project::tests::test_schema_validation_matching_schemas ... ok test write::project::tests::test_schema_validation_with_metadata_differences ... ok test write::repartition::tests::test_combined_partition_and_bucket_strategy ... ok test write::repartition::tests::test_repartition_explicit_partitions ... ok test write::repartition::tests::test_range_only_partitions_use_round_robin ... ok test write::repartition::tests::test_repartition_unpartitioned_table ... ok test write::repartition::tests::test_repartition_zero_partitions_fails ... ok test write::repartition::tests::test_schema_ref_convenience_method ... ok test write::sort::tests::test_sort_by_partition_missing_column ... ok test write::commit::tests::test_datafusion_execution_partitioned_source ... ok test write::sort::tests::test_sort_by_partition_basic ... ok test write::sort::tests::test_sort_by_partition_multi_field ... ok test write::commit::tests::test_iceberg_commit_exec ... ok test table_provider::catalog_table_provider::tests::test_catalog_backed_provider_insert ... ok test write::task_writer::tests::test_task_writer_unpartitioned ... ok test write::task_writer::tests::test_task_writer_partitioned_clustered ... ok test write::task_writer::tests::test_task_writer_partitioned_fanout ... ok test result: ok. 98 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.06s running 11 tests test test_provider_list_schema_names ... ok test test_provider_list_table_names ... ok test test_provider_plan_stream_schema ... ok test test_table_projection ... ok test test_table_predict_pushdown ... ok test test_metadata_table ... ok test test_insert_into ... ok test test_insert_into_partitioned ... ok test test_insert_into_nested ... ok test test_plan_nodes_are_inspectable ... ok test test_scan_after_schema_evolution_reads_provider_columns ... ok test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.11s running 3 tests test catalog::tests::test_parse_unsupported_catalog_type ... ok test catalog::tests::test_catalog_names ... ok test catalog::tests::test_parse_memory_catalog ... ok test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s running 10 tests test engine::tests::test_deserialize_engine_config ... ok test engine::tests::test_deserialize_engine_config_with_catalog ... ok test engine::tests::test_deserialize_catalog_config ... ok test schedule::tests::test_deserialize_invalid_engine_type ... ok test schedule::tests::test_deserialize_missing_engine_type ... ok test schedule::tests::test_deserialize_missing_step_fields ... ok test schedule::tests::test_deserialize_schedule_config ... ok test schedule::tests::test_deserialize_multiple_steps ... ok test schedule::tests::test_deserialize_with_catalog_config ... ok test engine::tests::test_load_datafusion ... ok test result: ok. 10 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s running 10 tests test schedule: df_show_tables.toml ... ok test schedule: df_encrypted_insert_round_trip.toml ... ok test schedule: df_drop_table.toml ... ok test schedule: df_binary_predicate_pushdown.toml ... ok test schedule: df_create_table.toml ... ok test schedule: df_boolean_predicate_pushdown.toml ... ok test schedule: df_scan_all_types.toml ... ok test schedule: df_timestamp_predicate_pushdown.toml ... ok test schedule: df_basic_queries.toml ... ok test schedule: df_insert_and_like.toml ... ok test result: ok. 10 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.23s running 7 tests test crates/datafusion/src/write/repartition.rs - write::repartition::repartition (line 78) ... ignored test crates/datafusion/src/write/task_writer.rs - write::task_writer::TaskWriter (line 49) ... ignored test crates/datafusion/src/write/task_writer.rs - write::task_writer::TaskWriter<B>::close (line 255) ... ignored test crates/datafusion/src/write/task_writer.rs - write::task_writer::TaskWriter<B>::try_new (line 110) ... ignored test crates/datafusion/src/write/task_writer.rs - write::task_writer::TaskWriter<B>::write (line 178) ... ignored test crates/datafusion/src/table_provider/factory.rs - table_provider::factory::IcebergTableProviderFactory (line 41) ... ok test crates/datafusion/src/write/project.rs - write::project::PartitionExpr::try_new (line 124) ... ok test result: ok. 2 passed; 0 failed; 5 ignored; 0 measured; 0 filtered out; finished in 0.52s all doctests ran in 5.98s; merged doctests compilation took 1.90s running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s - -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
