Jackie-Jiang commented on code in PR #11315:
URL: https://github.com/apache/pinot/pull/11315#discussion_r1290771548
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotHelixTaskResourceManager.java:
##########
@@ -118,25 +117,19 @@ public synchronized Set<String> getTaskTypes() {
*
* @param taskType Task type
*/
- public void ensureTaskQueueExists(String taskType) {
+ public synchronized void ensureTaskQueueExists(String taskType) {
String helixJobQueueName = getHelixJobQueueName(taskType);
WorkflowConfig workflowConfig =
_taskDriver.getWorkflowConfig(helixJobQueueName);
-
if (workflowConfig == null) {
// Task queue does not exist
LOGGER.info("Creating task queue: {} for task type: {}",
helixJobQueueName, taskType);
// Set full parallelism
// Don't allow overlap job assignment so that we can control number of
concurrent tasks per instance
- JobQueue jobQueue = new JobQueue.Builder(helixJobQueueName)
- .setWorkflowConfig(new
WorkflowConfig.Builder().setParallelJobs(Integer.MAX_VALUE).build()).build();
+ JobQueue jobQueue = new
JobQueue.Builder(helixJobQueueName).setWorkflowConfig(
+ new
WorkflowConfig.Builder().setParallelJobs(Integer.MAX_VALUE).build()).build();
_taskDriver.createQueue(jobQueue);
}
-
- // Wait until task queue context shows up
- while (_taskDriver.getWorkflowContext(helixJobQueueName) == null) {
Review Comment:
This was added in #1894 as a workaround to a Helix bug. Since we upgraded to
latest Helix version, this workaround is no longer needed. Other changes were
already removed, but this one was not removed because we need to fix the task
status API as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
