andygrove commented on code in PR #2834: URL: https://github.com/apache/datafusion-ballista/pull/2834#discussion_r4530823183
########## README.md: ########## @@ -143,6 +144,23 @@ Ballista supports two execution modes, selected with `ballista.execution.mode`. Pipelined execution (experimental) runs a job the way [Trino](https://trino.io) does: every task is scheduled at once and stages stream record batches to each other over Arrow Flight, with no intermediate data written to disk, so results reach the client sooner, at the cost of a failed task failing the whole job. It suits short, latency-sensitive queries on clusters with room for the whole job. See the [Pipelined Execution Guide](docs/source/user-guide/pipelined-execution.md) for details. +## One Engine for ETL and Interactive Queries + +A lakehouse aims to merge the data warehouse and the ETL platform, but it is often still served by two engines: Spark +for batch ETL and Trino for interactive analytics. Ballista's two execution modes let one engine serve both, from the Review Comment: Good point, thanks. I reworded the section to describe the two workload categories (long ETL jobs that need to survive task failures, and short analytical queries that need low latency) instead of naming Spark and Trino. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
