[
https://issues.apache.org/jira/browse/HDDS-10634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devesh Kumar Singh updated HDDS-10634:
--------------------------------------
Description:
New API:
api/v1/namespace/listKeys?startPrefix=/volume1/obs-bucket/&count=105
h5. Default values of API parameters if not provided:
# replicationType - empty/null, so effectively will list all types of keys.
# creationTime - empty string and filter will not be applied, so list out keys
irrespective of age.
# keySize - 0 bytes, which means all keys greater than zero bytes will be
listed, effectively all.
# startPrefix - /
# count - 1000
# offset - 0 -> which means records starting with 1st record.
h5. Behavior of API:
h5. For OBS bucket - list out *_count_* number of keys on the provided
path.
This API will implement pagination support using *count params.*
h3. Get List of All Keys:
GET /api/v1/namespace/listKeys
API params:
# replicationType
# creationTime in "MM-dd-yyyy HH:mm:ss" string format.
# startPrefix
# count
# keySize
# offset
*Input Request:*
api/v1/namespace/listKeys?startPrefix=/volume1/obs-bucket/&count=105
*Output Response:*
** {
"status": "OK",
"path": "/volume1/obs-bucket/",
"size": 62914560,
"sizeWithReplica": -1,
"subPathCount": 6,
"subPaths": [
{ "key": true, "path": "key1", "size":
10485760, "sizeWithReplica": -1, "isKey": true,
"replicationType": "RATIS", "creationTime": "1234567",
"modificationTime": "4567259" }
,
{ "key": true, "path": "key1/key2", "size":
10485760, "sizeWithReplica": -1, "isKey": true }
,
{ "key": true, "path": "key1/key2/key3",
"size": 10485760, "sizeWithReplica": -1, "isKey": true,
"replicationType": "RATIS", "creationTime": "1234567",
"modificationTime": "4567259" }
,
{ "key": true, "path": "key4", "size":
10485760, "sizeWithReplica": -1, "isKey": true,
"replicationType": "RATIS", "creationTime": "1234567",
"modificationTime": "4567259" }
,
{ "key": true, "path": "key5", "size":
10485760, "sizeWithReplica": -1, "isKey": true,
"replicationType": "RATIS", "creationTime": "1234567",
"modificationTime": "4567259" }
,
{ "key": true, "path": "key6", "size":
10485760, "sizeWithReplica": -1, "isKey": true,
"replicationType": "RATIS", "creationTime": "1234567",
"modificationTime": "4567259" }
],
"sizeDirectKey": 62914560
}
# Output response will have total count based on filters provided in request,
which tells how many total keys in the start prefix path. This will help client
to continue request for next page.
# Will provide one more param -> {{{}offset{}}}. So e.g. if client provides
{{offset}} as {{0}} and count as {{{}10{}}}, then server will give first {{10}}
records. And if client provides {{offset}} as {{100}} and count as {{{}50{}}},
then server will skip first {{100}} records and provides next {{50}} records.
This mean, API will apply filters like *{{replicationType}}* , *{{keySize}}*
and *{{creationTime}}* on set of records returned after *{{startPrefix}}* and
number of records returned based on *offset* and *count* value specified.
Similar behavior will be for *LEGACY* buckets
Similar behavior will be for *FSO* buckets except the difference of additional
recursive flag in request to list all keys recursively.
was:
New API:
api/v1/namespace/listKeys?startPrefix=/volume1/obs-bucket/&count=105
h5. Default values of API parameters if not provided:
# replicationType - RATIS
# creationTime - empty string and filter will not be applied, so list out keys
irrespective of age.
# keySize - 0 bytes, which means all keys greater than zero bytes will be
listed, effectively all.
# startPrefix - /
# count - 1000
h5. Behavior of API:
h5. For OBS bucket - list out *_count_* number of keys on the provided
path.
This API will implement pagination support using *count params.*
h3. Get List of All Keys:
GET /api/v1/namespace/listKeys
API params:
# replicationType
# creationTime in "MM-dd-yyyy HH:mm:ss" string format.
# startPrefix
# count
# keySize
*Input Request:*
api/v1/namespace/listKeys?startPrefix=/volume1/obs-bucket/&count=105
*Output Response:*
** {
"status": "OK",
"path": "/volume1/obs-bucket/",
"size": 62914560,
"sizeWithReplica": -1,
"subPathCount": 6,
"subPaths": [
{ "key": true, "path": "key1", "size":
10485760, "sizeWithReplica": -1, "isKey": true,
"replicationType": "RATIS", "creationTime": "1234567",
"modificationTime": "4567259" }
,
{ "key": true, "path": "key1/key2", "size":
10485760, "sizeWithReplica": -1, "isKey": true }
,
{ "key": true, "path": "key1/key2/key3",
"size": 10485760, "sizeWithReplica": -1, "isKey": true,
"replicationType": "RATIS", "creationTime": "1234567",
"modificationTime": "4567259" }
,
{ "key": true, "path": "key4", "size":
10485760, "sizeWithReplica": -1, "isKey": true,
"replicationType": "RATIS", "creationTime": "1234567",
"modificationTime": "4567259" }
,
{ "key": true, "path": "key5", "size":
10485760, "sizeWithReplica": -1, "isKey": true,
"replicationType": "RATIS", "creationTime": "1234567",
"modificationTime": "4567259" }
,
{ "key": true, "path": "key6", "size":
10485760, "sizeWithReplica": -1, "isKey": true,
"replicationType": "RATIS", "creationTime": "1234567",
"modificationTime": "4567259" }
],
"sizeDirectKey": 62914560
}
Working of API will be similar to what we have in CLI (ozone sh key list
--start=key4 --length=5 /volume1/obs-bucket/).
This mean, API will apply filters like *{{replicationType}}* , *{{keySize}}*
and *{{creationTime}}* on set of records returned after *{{startPrefix}}*
(exclusive) and number of records returned equal to *count* value specified.
> Recon - listKeys API for listing of OBS , FSO and Legacy bucket keys with
> filters
> ---------------------------------------------------------------------------------
>
> Key: HDDS-10634
> URL: https://issues.apache.org/jira/browse/HDDS-10634
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: Ozone Recon
> Reporter: Devesh Kumar Singh
> Assignee: Devesh Kumar Singh
> Priority: Major
> Labels: pull-request-available
>
> New API:
> api/v1/namespace/listKeys?startPrefix=/volume1/obs-bucket/&count=105
> h5. Default values of API parameters if not provided:
> # replicationType - empty/null, so effectively will list all types of keys.
> # creationTime - empty string and filter will not be applied, so list out
> keys irrespective of age.
> # keySize - 0 bytes, which means all keys greater than zero bytes will be
> listed, effectively all.
> # startPrefix - /
> # count - 1000
> # offset - 0 -> which means records starting with 1st record.
>
> h5. Behavior of API:
> h5. For OBS bucket - list out *_count_* number of keys on the provided
> path.
> This API will implement pagination support using *count params.*
> h3. Get List of All Keys:
> GET /api/v1/namespace/listKeys
> API params:
> # replicationType
> # creationTime in "MM-dd-yyyy HH:mm:ss" string format.
> # startPrefix
> # count
> # keySize
> # offset
> *Input Request:*
>
> api/v1/namespace/listKeys?startPrefix=/volume1/obs-bucket/&count=105
> *Output Response:*
> ** {
> "status": "OK",
> "path": "/volume1/obs-bucket/",
> "size": 62914560,
> "sizeWithReplica": -1,
> "subPathCount": 6,
> "subPaths": [
>
> { "key": true, "path": "key1", "size":
> 10485760, "sizeWithReplica": -1, "isKey": true,
> "replicationType": "RATIS", "creationTime": "1234567",
> "modificationTime": "4567259" }
> ,
>
> { "key": true, "path": "key1/key2",
> "size": 10485760, "sizeWithReplica": -1, "isKey":
> true }
> ,
>
> { "key": true, "path": "key1/key2/key3",
> "size": 10485760, "sizeWithReplica": -1, "isKey":
> true, "replicationType": "RATIS", "creationTime":
> "1234567", "modificationTime": "4567259" }
> ,
>
> { "key": true, "path": "key4", "size":
> 10485760, "sizeWithReplica": -1, "isKey": true,
> "replicationType": "RATIS", "creationTime": "1234567",
> "modificationTime": "4567259" }
> ,
>
> { "key": true, "path": "key5", "size":
> 10485760, "sizeWithReplica": -1, "isKey": true,
> "replicationType": "RATIS", "creationTime": "1234567",
> "modificationTime": "4567259" }
> ,
>
> { "key": true, "path": "key6", "size":
> 10485760, "sizeWithReplica": -1, "isKey": true,
> "replicationType": "RATIS", "creationTime": "1234567",
> "modificationTime": "4567259" }
> ],
> "sizeDirectKey": 62914560
> }
>
> # Output response will have total count based on filters provided in
> request, which tells how many total keys in the start prefix path. This will
> help client to continue request for next page.
> # Will provide one more param -> {{{}offset{}}}. So e.g. if client provides
> {{offset}} as {{0}} and count as {{{}10{}}}, then server will give first
> {{10}} records. And if client provides {{offset}} as {{100}} and count as
> {{{}50{}}}, then server will skip first {{100}} records and provides next
> {{50}} records.
> This mean, API will apply filters like *{{replicationType}}* , *{{keySize}}*
> and *{{creationTime}}* on set of records returned after *{{startPrefix}}*
> and number of records returned based on *offset* and *count* value specified.
> Similar behavior will be for *LEGACY* buckets
> Similar behavior will be for *FSO* buckets except the difference of
> additional recursive flag in request to list all keys recursively.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
