[
https://issues.apache.org/jira/browse/AIRFLOW-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aditya Vishwakarma updated AIRFLOW-3545:
----------------------------------------
Description:
Prometheus, which is a common way to instrument services on Kubernetes, uses a
pull based mechanism to fetch metrics. This involves a service exposing a
`/metrics` endpoint. This endpoint is scraped every 30 secs by prometheus to
collect metrics.
This requires a port to be specified in the generated Pod config. Something
like below.
{code:java}
// Sample Pod Spec
apiVersion: v1
kind: Job
metadata:
name: batch-job
spec:
ports:
- name: metrics
port: 9091 # port to fetch metrics from
protocol: TCP
targetPort: 9091
{code}
Currently KubernetesPodOperator doesn't have any options to open ports like
this.
Is it possible to have an option to do this?
was:
Prometheus, which is a common way to instrument services on Kubernetes, uses a
pull based mechanism to fetch metrics. Which generally involves a service
exposing a `/metrics` endpoints. This endpoint is scraped every 30 secs by
prometheus to collect metrics.
This requires a port to be specified in the generated Pod config. Something
like below.
{code:java}
// Sample Pod Spec
apiVersion: v1
kind: Job
metadata:
name: batch-job
spec:
ports:
- name: metrics
port: 9091 # port to fetch metrics from
protocol: TCP
targetPort: 9091
{code}
Currently KubernetesPodOperator doesn't have any options to open ports like
this.
Is it possible to have an option to do this?
> Can't use Prometheus or other pull based instrumentation systems to monitor
> Tasks launched on Kubernetes
> --------------------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-3545
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3545
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Aditya Vishwakarma
> Priority: Major
>
> Prometheus, which is a common way to instrument services on Kubernetes, uses
> a pull based mechanism to fetch metrics. This involves a service exposing a
> `/metrics` endpoint. This endpoint is scraped every 30 secs by prometheus to
> collect metrics.
> This requires a port to be specified in the generated Pod config. Something
> like below.
> {code:java}
> // Sample Pod Spec
> apiVersion: v1
> kind: Job
> metadata:
> name: batch-job
> spec:
> ports:
> - name: metrics
> port: 9091 # port to fetch metrics from
> protocol: TCP
> targetPort: 9091
> {code}
> Currently KubernetesPodOperator doesn't have any options to open ports like
> this.
> Is it possible to have an option to do this?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
