[
https://issues.apache.org/jira/browse/HDDS-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sreeja updated HDDS-12581:
--------------------------
Description:
Log files should be parsed using multiple threads, where each thread processes
one file at a time. The number of threads should be dynamically adjusted (can
be given in the command). During the parsing process, batch updates should be
performed on the database. It should be ensured that only one thread performs
the database updates at a time while other threads are blocked during this
operation.
Also create a ozone debug CLI command for the same.
was:
Log files should be parsed using multiple threads, where each thread processes
one file at a time. The number of threads should be dynamically adjusted based
on the total number of files, with a maximum limit to optimize resource usage.
During the parsing process, batch updates should be performed on the database.
It should be ensured that only one thread performs the database updates at a
time while other threads are blocked during this operation.
Also create a ozone debug CLI command for the same.
> Multi-threaded Log File Parsing with Batch Updates to DB
> --------------------------------------------------------
>
> Key: HDDS-12581
> URL: https://issues.apache.org/jira/browse/HDDS-12581
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sreeja
> Assignee: Sreeja
> Priority: Major
> Labels: pull-request-available
>
> Log files should be parsed using multiple threads, where each thread
> processes one file at a time. The number of threads should be dynamically
> adjusted (can be given in the command). During the parsing process, batch
> updates should be performed on the database. It should be ensured that only
> one thread performs the database updates at a time while other threads are
> blocked during this operation.
> Also create a ozone debug CLI command for the same.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
