[
https://issues.apache.org/jira/browse/KUDU-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17957044#comment-17957044
]
ASF subversion and git services commented on KUDU-3364:
-------------------------------------------------------
Commit 87580082ff5f248576fe20d8fcea08d825d00a99 in kudu's branch
refs/heads/branch-1.18.x from kedeng
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=87580082f ]
[util] Remove SchedulerThread code and tests
This removes the SchedulerThread implementation and its associated tests.
The TestSubmitAfterShutdown case has proven unstable in ARM+TSAN
environments. The code was originally introduced in KUDU-3364 but has
never been used outside of tests, and the original author appears to
have no plans to maintain it.
Removing this unused code simplifies the codebase and improves test
stability. The removal was suggested by Alexey.
Change-Id: I544488158db84dca580694e884b56d5c546d89c1
Reviewed-on: http://gerrit.cloudera.org:8080/22987
Tested-by: Alexey Serbin <ale...@apache.org>
Reviewed-by: Alexey Serbin <ale...@apache.org>
(cherry picked from commit 7a3868519f1eed9617ee0e4c1a20154b55f85063)
Conflicts:
src/kudu/util/threadpool.h
Reviewed-on: http://gerrit.cloudera.org:8080/22996
Reviewed-by: Abhishek Chennaka <achenn...@cloudera.com>
> Add TimerThread to ThreadPool to support a category of problem
> --------------------------------------------------------------
>
> Key: KUDU-3364
> URL: https://issues.apache.org/jira/browse/KUDU-3364
> Project: Kudu
> Issue Type: New Feature
> Reporter: Yuqi Du
> Assignee: Yuqi Du
> Priority: Minor
> Attachments: image-2022-06-01-11-02-20-888.png,
> image-2022-06-01-11-03-10-272.png
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> h1. Scenanios
> In general, I am talking about a category of problem.
> There are some periodic tasks or automatically triggered scheduling tasks in
> kudu.
> For example, automatic rebalance of cluster data, some GC task and compaction
> tasks.
> Their implementation is by kudu Thread, maybe std::thread or ThreadPool, the
> really task internally periodic scheduled or internally strategy to trigge
> execution.
> They are all internal, we cann't do some.
> In fact, we need a method our control to trigge the above types of actions.
> In general, I am talking about a category of problem.
> Some scenarios is significant.
> Below is examples:
>
> h2. data rebalance
> There are two rebalance ways:
> 1. enable auto rebalance
> 2. use rebalance tool 1.14 before.
> The two ways maybe exist some conflicts at opeations race, because rebalance
> tool' logic is a litte complex at tool and auto rebalance is running at
> master.
> In future, auto rebalance at master will become very steady and become the
> main way for data rebalance. And at the same time, admin opers need a
> external trigger the rebalance just like auto rebalance.
> But, now auto rebalance is running in a thread and by time period.
> Although we can add a api for MasterService, but the api is synchronize, and
> will cose very much, we need a asynchronized method to trigger the rebalance.
> h2. auto compaction
> Another example is auto compaction,
> I have found compaction strategy is not always valid, so maybe we need a
> method controlled by admin users to triggle compaction.
> If we can do a RowSetInCompaction, we need not restart the kudu cluster.
> h1.
> h1. My Solution
> Add a timer in ThreadPool. This timer is a worker thread that schedules tasks
> to the specified thread according to time.
> We can limit only SERIAL ThreadPoolToken can enable TimerThread.
> Pseudo code expresses my intention：
> {code:java}
> //代码占位符
> class TimerThread {
> class Task {
> ThreadPoolToken token;
> std::function<void()> f;
> };
>
> void Schedule(Task task, int delay_ms) {
> tasks_.insert(...);
> }
> void RunLoop() {
> while (...) {
> SleepFor(100ms);
> tasks = FindTasks();
> for (auto task : tasks) {
> token = task.token;
> token->Submit(task.f);
> tasks_.erase...
> }
> }
> }
> scoped_refptr<Thread> thread_;
> std::multimap<MonoTime, Task> tasks;
> };
> class ThreadPool{
> ...
> TimerThread* timer_;
> ...
> };
> class ThreadPoolToken {
> void Scheduler();
> };{code}
> This scheme can be compatible with the previous ThreadPool, and timer is
> nullptr by default.
> For periodic tasks, We can use a Control ThreadPool with timer to refact some
> codes to make them more clear, to avoid the problem of too many single
> threads in the past.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
