BlakeOrth opened a new pull request, #18016:
URL: https://github.com/apache/datafusion/pull/18016
## Which issue does this PR close?
This does not fully close, but is an incremental building block component
for:
- https://github.com/apache/datafusion/issues/17207
The full context of how this code is likely to progress can be seen in the
POC for this effort:
- https://github.com/apache/datafusion/pull/17266
## Rationale for this change
This change implements actual instrumentation for individual `GET` requests
through an `InstrumentedObjectStore` and renders that information for users
when profiling has been enabled.
## What changes are included in this PR?
- Adds request instrumentation to GET operations when using the
InstrumentedObjectStore in datafusion-cli
- Adds RequestDetails type to store and format details about instrumented
requests
- Adds tests for new functionality
## Are these changes tested?
Yes, aside from the specific functionality in `PrintOptions`, however all
methods executed there are tested via unit tests and functional test output can
be seen below:
```sql
DataFusion CLI v50.1.0
> \object_store_profiling enabled
ObjectStore Profile mode set to Enabled
> CREATE EXTERNAL TABLE hits
STORED AS PARQUET
LOCATION
'https://datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_1.parquet';
0 row(s) fetched.
Elapsed 0.208 seconds.
Object Store Profiling
Instrumented Object Store: instrument_mode: Enabled, inner: HttpStore
2025-10-10T23:21:49.102730971+00:00 operation=Get duration=0.022583s size=8
range: bytes=174965036-174965043
path=hits_compatible/athena_partitioned/hits_1.parquet
2025-10-10T23:21:49.125493437+00:00 operation=Get duration=0.024233s
size=34322 range: bytes=174930714-174965035
path=hits_compatible/athena_partitioned/hits_1.parquet
> \object_store_profiling disabled
ObjectStore Profile mode set to Disabled
> CREATE EXTERNAL TABLE hits2
STORED AS PARQUET
LOCATION
'https://datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_2.parquet';
0 row(s) fetched.
Elapsed 0.274 seconds.
>
```
## Are there any user-facing changes?
Yes? Clearly there is new user-facing output generated, however there are no
new user options and/or breaking changes introduced in this PR.
cc @alamb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
