alamb commented on PR #14286:
URL: https://github.com/apache/datafusion/pull/14286#issuecomment-2654986803
> Is there extra work I'd need to do to get this to work with the
`ParquetSink`? This isn't a full reproducer but just a quick copy of the
relevant section of the `do_put_statement_ingest()` method in my flightsql
server.
>
> I get
>
> > flight_sql: writing data to disk
> > thread 'tokio-runtime-worker' panicked at
/Users/dja/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5:
> > A Tokio 1.x context was found, but timers are disabled. Call
`enable_time` on the runtime builder to enable timers.
>
> ```rust
> let data_sink = ParquetSink::new(file_sink_config, table_options);
>
> let record_batch_stream =
>
FlightRecordBatchStream::new_from_flight_data(flight_data_stream.map_err(|e|
e.into()));
>
> let stream = Box::pin(RecordBatchStreamAdapter::new(
> schema,
> record_batch_stream.map_err(|e|
DataFusionError::External(Box::new(e))),
> ));
>
> // This displays "IoObjectStore"
> info!(
> "{}",
> self.datafusion_state
> .runtime_env()
> .object_store(object_store_url.as_ref())
> .unwrap()
> );
>
> info!("writing data to disk");
> let rows_written = self
> .dedicated_rt
> .spawn(async move { data_sink.write_all(stream,
&datafusion.task_ctx()).await })
> .await
> .map_err(|e| Status::data_loss(format!("error processing
stream: {e}")))?
> .map_err(|e| Status::data_loss(format!("data write failed:
{e}")))?;
> info!("wrote {rows_written} rows");
> ```
Maybe you need to run `data_sink.write_all(stream,
&datafusion.task_ctx()).await })` somehow on the iO thread? I am nto sure what
it is doing
What is the panic error message?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
