[
https://issues.apache.org/jira/browse/HDDS-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sreeja updated HDDS-12581:
--------------------------
Description:
Log files should be parsed using multiple threads, where each thread processes
one file at a time. The number of threads should be dynamically adjusted based
on the total number of files, with a maximum limit to optimize resource usage.
During the parsing process, batch updates should be performed on the database.
It should be ensured that only one thread performs the database updates at a
time while other threads are blocked during this operation.
Also create a ozone debug CLI command for the same.
was:Log files should be parsed using multiple threads, where each thread
processes one file at a time. The number of threads should be dynamically
adjusted based on the total number of files, with a maximum limit to optimize
resource usage. During the parsing process, batch updates should be performed
on the database. It should be ensured that only one thread performs the
database updates at a time while other threads are blocked during this
operation.
> Multi-threaded Log File Parsing with Batch Updates to DB
> --------------------------------------------------------
>
> Key: HDDS-12581
> URL: https://issues.apache.org/jira/browse/HDDS-12581
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sreeja
> Assignee: Sreeja
> Priority: Major
>
> Log files should be parsed using multiple threads, where each thread
> processes one file at a time. The number of threads should be dynamically
> adjusted based on the total number of files, with a maximum limit to optimize
> resource usage. During the parsing process, batch updates should be performed
> on the database. It should be ensured that only one thread performs the
> database updates at a time while other threads are blocked during this
> operation.
> Also create a ozone debug CLI command for the same.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
