Andrey Yarovoy created HDDS-17714:
-------------------------------------
Summary: OFS getFileStatus should not refresh pipelines or return
block locations
Key: HDDS-17714
URL: https://issues.apache.org/jira/browse/HDDS-17714
Project: Apache Ozone
Issue Type: Improvement
Components: Ozone Filesystem
Reporter: Andrey Yarovoy
*Description*
`BasicRootedOzoneFileSystem.getFileStatus` sends `GetFileStatus` with
`headOp=false`. For a file, OM then:
- calls `refresh(fileKeyInfo)`, which looks up every block's container pipeline
(`KeyManagerImpl.getOzoneFileStatusFSO`);
- optionally sorts the datanodes;
- returns the full `KeyLocationList`.
The client parses it (`OmKeyInfo.getFromProtobuf` →
`OmKeyLocationInfo.getFromProtobuf` streams), builds a `BlockLocation[]`, and
returns a
`LocatedFileStatus`.
HDFS `getFileStatus` returns no block locations; callers that need them call
`getFileBlockLocations` or `listLocatedStatus`. Most
`getFileStatus` callers use only length, type, mtime and owner. HBase is the
extreme case. Every `StoreScanner.trySwitchToStreamRead`
reopens each store file, and `StoreFileInfo.createReaderContext` calls
`fs.getFileStatus(path)` only for `getLen()`.
*Follow-up to HDDS-15678 / HDDS-15877*
- **HDDS-15678** built the server side. With `headOp=true`, OM skips
`refresh`/`sortDatanodes`, and
`OzoneManagerRequestHandler.getOzoneFileStatus` clears `KeyLocationList` from
the response. On the client it switched ofs
`isFile`/`isDirectory` to head ops.
- **HDDS-15877** did the same for o3fs and the remaining *type-only* callers
(trash roots, shell `DeleteKeyHandler`).
- Both deliberately left the public `FileSystem.getFileStatus(Path)` on
`headOp=false`, because its result is a `LocatedFileStatus` that
`getFileBlockLocations(FileStatus)` reads. That is the call HBase makes on
every store-file reopen, and still the call that pays the refresh and
returns all block locations.
This Jira finishes the job for `getFileStatus(Path)`. It returns a plain
`FileStatus` (like HDFS), and `getFileBlockLocations` fetches the
locations only when they are actually requested.
*Evidence (HBase YCSB on EC, RegionServer async-profiler, 6×60s per arm)*
- `getFileStatus` is 12.5–14.6% of busy RS handler time, all spent waiting on
OM.
- Client-side status conversion is 1.2–1.6% of RS CPU (`toFileStatusAdapter`,
`getBlockLocations`, `OmKeyInfo.getFromProtobuf` for the status).
Parsing the `GetFileStatusResponse` `KeyLocationList` is ~3% of the reopen
subtree.
- OM-side cost (`refresh` + response encode) is not visible from RS profiles.
**Measure on OM before and after.**
*Proposed fix*
1. `BasicRootedOzoneFileSystem.getFileStatus(Path)` (and
`BasicOzoneFileSystem`) call the adapter with `headOp=true`. They return a plain
`FileStatus` (no `BlockLocation[]`). Length, mtime, owner, replication, EC
policy and encryption flags come from the same `KeyInfo` fields as
today.
2. `getFileBlockLocations(FileStatus, start, len)`: when the status is not a
`LocatedFileStatus` and is a file, fetch a full status (`headOp=false`)
and compute the range from it, instead of `super.getFileBlockLocations` (which
returns a fake `localhost` location).
3. Override `getFileBlockLocations(Path, start, len)` to fetch the full status
directly, so a path-based call stays at **1** RPC.
4. `listStatus` / `listLocatedStatus` / `listFiles` are unchanged; they keep
their own location handling.
5. Optional escape hatch: `ozone.fs.getfilestatus.include.block.locations`
(default `false`) restores today's `LocatedFileStatus` from
`getFileStatus`.
{code:java}
// BasicRootedOzoneFileSystem
public FileStatus getFileStatus(Path f) throws IOException {
return TracingUtil.executeInNewSpan("ofs getFileStatus",
() -> convertFileStatus(getFileStatusAdapter(f, !includeBlockLocations)));
}
{code}
*Old OM compatibility*
An OM without HDDS-15678's response stripping still returns locations on a head
op, but they are unrefreshed (no pipeline). The client must
ignore `KeyLocationList` whenever it sent `headOp=true`, whatever the OM
version. With that rule no version gate is needed. Old OMs still
skip `refresh` on head ops (that check predates HDDS-15678).
*Compatibility / side effects*
- **Behavior change:** `getFileStatus` no longer returns a `LocatedFileStatus`.
- A caller that casts the result to `LocatedFileStatus` breaks; in-tree, check
`ozonefs` tests and the `BasicOzoneFileSystem` copy.
- A caller that does `getFileStatus` then `getFileBlockLocations(status)` now
pays 2 OM RPCs instead of 1. MapReduce/Spark/Hive split
planning uses `listLocatedStatus`, so it is unaffected; one-off single-file
planners pay +1 RPC.
- No proto change. No new OM version.
*Tests*
- Unit (adapter, Mockito): `getFileStatus` sends `headOp=true`. The result has
the correct length/mtime/owner/EC fields and is not a
`LocatedFileStatus`. `getFileBlockLocations(status)` sends exactly one
`GetFileStatus` with `headOp=false` and returns real locations.
`getFileBlockLocations(path)` sends exactly one RPC.
- Integration: `TestRootedOzoneFileSystem` / `TestOzoneFileSystem` locality
tests still return real datanode hosts. EC and Ratis keys,
encrypted keys.
- Benchmark: `getFileStatus` latency/throughput A/B on a multi-block EC key
(n=3 per arm, sequential), with OM CPU per call.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
