tobixdev opened a new issue, #14010:
URL: https://github.com/apache/datafusion/issues/14010
### Describe the bug
Trying to insert into a MemTable with zero partitions causes and index out
of bounds error when executing.
Backtrace:
```
index out of bounds: the len is 0 but the index is 0
thread 'datasource::memory::tests::test_insert_into_zero_partition' panicked
at datafusion/core/src/datasource/memory.rs:373:24:
index out of bounds: the len is 0 but the index is 0
stack backtrace:
0: rust_begin_unwind
at
/rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panicking.rs:665:5
1: core::panicking::panic_fmt
at
/rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/panicking.rs:74:14
2: core::panicking::panic_bounds_check
at
/rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/panicking.rs:276:5
3: <usize as core::slice::index::SliceIndex<[T]>>::index_mut
at
/rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/slice/index.rs:307:14
4: core::slice::index::<impl core::ops::index::IndexMut<I> for
[T]>::index_mut
at
/rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/slice/index.rs:28:9
5: <alloc::vec::Vec<T,A> as core::ops::index::IndexMut<I>>::index_mut
at
/rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/alloc/src/vec/mod.rs:2924:9
6: <datafusion::datasource::memory::MemSink as
datafusion_physical_plan::insert::DataSink>::write_all::{{closure}}
```
### To Reproduce
The following test case produces said error:
```
// Test inserting a batch into a MemTable without any partitions
#[tokio::test]
async fn test_insert_into_zero_partition() -> Result<()> {
// Create a new schema with one field called "a" of type Int32
let schema = Arc::new(Schema::new(vec![Field::new("a",
DataType::Int32, false)]));
// Create a new batch of data to insert into the table
let batch = RecordBatch::try_new(
schema.clone(),
vec![Arc::new(Int32Array::from(vec![1, 2, 3]))],
)?;
// Run the experiment and expect an error
let experiment_result = experiment(schema, vec![],
vec![vec![batch.clone()]])
.await
.unwrap_err();
// Ensure that there is a descriptive error message
assert_eq!(
"Error during planning: Cannot insert into MemTable with zero
partitions.",
experiment_result.strip_backtrace()
);
Ok(())
}
```
### Expected behavior
I think there are multiple ways to address this.
1. An error during planning with a descriptive message.
2. An error during execution with a descriptive message.
3. Automatically create a single partition during insertion
4. Do not allow creating MemTables without any partitions
From my point of view, 1. is the preferred solution because this allows
creating "Empty MemTables that cannot become non-empty".
However, I am not really familiar with the code base so opinions may vary.
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
