Omega359 commented on issue #15394:
URL: https://github.com/apache/datafusion/issues/15394#issuecomment-2764244381
I've been trying to find where to resolve this issue but my understanding of
the core of DF is currently too limited to uncover a solution. I've created a
test though that exhibits the issue
```Rust
#[tokio::test]
async fn union_by_name_literal_is_null_and_not_null() -> Result<()> {
let str_array_1 = StringArray::from(vec![Some("a1")]);
let str_array_2 = StringArray::from(vec![Some("a2")]);
let str_array_3 = StringArray::from(vec![Some("b1")]);
let batch_1 = RecordBatch::try_from_iter(vec![
("a", Arc::new(str_array_1) as ArrayRef),
])?;
let batch_2 = RecordBatch::try_from_iter(vec![
("a", Arc::new(str_array_2) as ArrayRef),
("b", Arc::new(str_array_3) as ArrayRef),
])?;
let ctx = SessionContext::new();
ctx.register_batch("union_batch_1", batch_1)?;
ctx.register_batch("union_batch_2", batch_2)?;
let df1 = ctx.table("union_batch_1").await?;
let df2 = ctx.table("union_batch_2").await?;
let batches = df2.union_by_name(df1)?.collect().await?;
let schema = batches[0].schema();
for batch in batches {
// Verify schema is the same for all batches
if !schema.contains(&batch.schema()) {
return Err(DataFusionError::Internal(
format!(
"Schema mismatch. Previously had\n{:#?}\n\nGot:\n{:#?}",
&schema,
batch.schema()
),
));
}
}
Ok(())
}
```
Above is based upon https://github.com/apache/datafusion/pull/15489
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
