I found out the root cause after that. We have duplicated API with sama
name so the one I added metrics is not seen. Legacy code drive me crazy.
About those out-of-date metrics, I delete the db through the admin-api and
those metrics are cleared. Thanks for looking into that.
On Wednesday, January 3, 2024 at 5:11:38 p.m. UTC-5 Brian Candler wrote:
> I suggest you take Prometheus entirely out of the picture, and scrape your
> metrics endpoint directly using curl:
>
> curl localhost:8881/metrics
>
> See if your expected custom metric is there. If it's not, then that's why
> Prometheus isn't showing it, and you need to debug your Python code.
>
> On Wednesday 3 January 2024 at 20:44:12 UTC Honsen Clock wrote:
>
>> Issue Description:
>> I have a Python Flask app and I wanna add a new metric
>> "by_path_counter_doc_quality" for an API, which I added by:
>> ```Python
>> # app.py
>>
>> by_path_counter = metrics.counter(
>> 'by_path_counter_doc_quality', 'Request count by request paths',
>> labels={'path': lambda: request.path, 'method': lambda:
>> request.method}
>> )
>> ...
>> @app.route("/document_quality/transformation", methods=["POST"])
>> @by_path_counter
>> def run_comp_metrics():
>> # code
>> ```
>> But when I start prometheus server and monitor, I cannot find this
>> metrics on the metrics list. The flask app itself is working and using
>> metric "flask_http_request_total" can find the requests.
>> [image: prometheus-flask-app.png]
>> - Moreover, I notice that the metrics list on the "metrics explorer" tab
>> shows out-of-dated metrics. See the attached image, "by_path_counter" and
>> "in_progress" were the metrics I added in previous scrape_config job.
>>
>> [image: prometheus-metrics list.png]
>>
>> My prometheus.yml:
>> ```yaml
>> # my global config
>> global:
>> scrape_interval: 15s
>> evaluation_interval: 15s
>>
>> alerting:
>> alertmanagers:
>> - static_configs:
>> - targets:
>> # - alertmanager:9093
>>
>> rule_files:
>> # - "first_rules.yml"
>> # - "second_rules.yml"
>>
>> scrape_configs:
>> - job_name: "doc-quality"
>> static_configs:
>> - targets: ["localhost:8881"]
>> ```
>> Some context about the issue:
>> In my prometheus.yaml, I created a job "flask_app", with metrics called
>> "by_path_counter" and "in_progress". The metrics works well. So I wanna
>> applied the same way to the current new job "doc-quality" and new metrics
>> "by_path_counter_doc_quality".
>>
>> My questions:
>> - What's the root cause for above issue? Why my new metric is now shown?
>> - Are there any ways that I can reset/refresh the metrics? Maybe clean
>> the time series DB? If so, how to to that.
>>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/5256b926-2904-491a-b501-f277fe18156bn%40googlegroups.com.
