alamb opened a new pull request, #18085: URL: https://github.com/apache/datafusion/pull/18085
Draft as I still need to polish it up (fix test normalization), but putting up what I have for feedback ## Which issue does this PR close? - part of https://github.com/apache/datafusion/issues/17207 ## Rationale for this change As suggested by @BlakeOrth in https://github.com/apache/datafusion/pull/18045#issuecomment-3403692516 here is an attempt to improve the output of datafusion object store trace profiling: ## What changes are included in this PR? Update the output format when `\object_store_profiling summary` is set Current format ```sql Summaries: Get count: 2 duration min: 0.024603s duration max: 0.031946s duration avg: 0.028274s size min: 8 B size max: 34322 B size avg: 17165 B size sum: 34330 B ``` New format (after this PR): ```sql DataFusion CLI v50.2.0 > \object_store_profiling summary ObjectStore Profile mode set to Summary > select count(*) from 'https://datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_1.parquet'; +----------+ | count(*) | +----------+ | 1000000 | +----------+ 1 row(s) fetched. Elapsed 0.548 seconds. Object Store Profiling Instrumented Object Store: instrument_mode: Summary, inner: HttpStore Summaries: +-----------+----------+-----------+-----------+-----------+-----------+ | Operation | Metric | min | max | avg | sum | +-----------+----------+-----------+-----------+-----------+-----------+ | Get | duration | 0.040982s | 0.047195s | 0.044088s | 0.088176s | | Get | size | 8 B | 34322 B | 17165 B | 34330 B | +-----------+----------+-----------+-----------+-----------+-----------+ ``` ## Are these changes tested? Yes ## Are there any user-facing changes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
