GitHub user dosubot[bot] added a comment to the discussion: how superset worker
pods are used and where to configure to utilise workloads in helm
<!-- Answer -->
To verify if the Superset worker pods are being utilized, you can check their
status and logs using Kubernetes commands. For example, use `kubectl get pods`
to see if the pods are running, and `kubectl logs <pod-name>` to view the logs
and check if tasks are being processed.
To enable the Celery worker in Helm, you need to configure the `values.yaml`
file with the following settings:
```yaml
supersetWorker:
replicas:
enabled: true
replicaCount: 1
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 100
targetCPUUtilizationPercentage: 80
podDisruptionBudget:
enabled: false
minAvailable: 1
maxUnavailable: 1
command:
- "/bin/sh"
- "-c"
- ". {{ .Values.configMountPath }}/superset_bootstrap.sh; celery
--app=superset.tasks.celery_app:app worker"
forceReload: false
initContainers:
- name: wait-for-postgres-redis
image: "{{ .Values.initImage.repository }}:{{ .Values.initImage.tag }}"
imagePullPolicy: "{{ .Values.initImage.pullPolicy }}"
envFrom:
- secretRef:
name: "{{ tpl .Values.envFromSecret . }}"
command:
- /bin/sh
- -c
- dockerize -wait "tcp://$DB_HOST:$DB_PORT" -wait
"tcp://$REDIS_HOST:$REDIS_PORT" -timeout 120s
extraContainers: []
deploymentAnnotations: {}
deploymentLabels: {}
affinity: {}
livenessProbe:
exec:
command:
- "celery"
- "inspect"
- "ping"
initialDelaySeconds: 120
periodSeconds: 60
timeoutSeconds: 60
successThreshold: 1
failureThreshold: 3
containerSecurityContext: {}
podAnnotations: {}
```
This configuration sets up the Superset worker pods with a Celery worker
command, initializes containers to wait for dependencies like PostgreSQL and
Redis, and includes a liveness probe using the `celery inspect ping` command to
ensure the worker is responsive
<sup>[[1]](https://github.com/apache/superset/blob/master/helm/superset/values.yaml)</sup><sup>[[2]](https://github.com/apache/superset/blob/master/helm/superset/README.md)</sup>.
<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
---
<sup>Help Dosu learn! Give it feedback: [Great
Response](https://app.dosu.dev/response-feedback/3b2fd57f-aa48-41a6-a2af-c830616e9e40?feedback_type=great_response)
| [Irrelevant
Answer](https://app.dosu.dev/response-feedback/3b2fd57f-aa48-41a6-a2af-c830616e9e40?feedback_type=irrelevant_answer)
| [Incorrect
Sources](https://app.dosu.dev/response-feedback/3b2fd57f-aa48-41a6-a2af-c830616e9e40?feedback_type=incorrect_sources)
| [Too
Verbose](https://app.dosu.dev/response-feedback/3b2fd57f-aa48-41a6-a2af-c830616e9e40?feedback_type=too_verbose)
|
[Hallucination](https://app.dosu.dev/response-feedback/3b2fd57f-aa48-41a6-a2af-c830616e9e40?feedback_type=hallucination)
| [Bug
Report](https://app.dosu.dev/response-feedback/3b2fd57f-aa48-41a6-a2af-c830616e9e40?feedback_type=bug_report)
|
[Other](https://app.dosu.dev/response-feedback/3b2fd57f-aa48-41a6-a2af-c830616e9e40?feedback_type=other)</sup>
GitHub link:
https://github.com/apache/superset/discussions/33006#discussioncomment-13028182
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
