1 year ago
#279758
KNCK
Prometheus Alert manager not printing {{ $labels.instance }} vaule
We have multiple AWS accounts and network access is configured between two AWS accounts and service discovery is working with node-exporter. I have Prometheus configuration with some of the rules configured for the docker containers and now I have added one of the rules similar to the existing one to check if by mistakenly same container is launched in another AWS account and below is the rule. for exiting rules, {{ $labels.instance }} is printing in Alerts email, but not for the new rule which I have written newly
Scrape config for labels:
- job_name: 'aws-conatiners'
scheme: http
ec2_sd_configs:
- region: {{region}}
port: 8181
relabel_configs:
- source_labels: [__meta_ec2_tag_Name]
target_label: instance
The new rule which I have created to check if more than one container is running:
# Alert to check if more than one instance is running for backendapi service
- alert: multiple_instances_are_running
expr: sum(container_last_seen{name=~"backendapi"}) > 1
for: 5m
labels:
severity: critical
annotations:
summary: "More than one Instance (instance {{ $labels.instance }}) is running"
description: "More than one Instance (instance {{ $labels.instance }}) is running for 5 minutes."
Can someone please check and help me to get the instance name printed in alert emails
amazon-web-services
docker
prometheus
prometheus-alertmanager
alerts
0 Answers
Your Answer