Configuring Grafana Webhook and Alert Notifications

---

1. Open Grafana Dashboard

• Log in to Grafana.

• Navigate to Alerting → Contact Points.

---

2. Create a Webhook Contact Point

• Click New Contact Point.

• Enter a name (e.g., custom-webhook-alert).

• Select Webhook as the integration type.

Webhook Configuration

• URL → This is the endpoint where Grafana will send alerts. You must provide a reachable API URL (for example, your backend service or webhook receiver).

Example:

http://your-server:50100/webhook

Optional Fields

• HTTP Method → POST (default).

• Authentication → Configure if your API requires authentication.

• Custom Headers → Add headers if required by your API.

Example header:

Content-Type: application/json

---

3. Configure Custom Payload (Optional)

Grafana allows you to customize the JSON payload sent to your webhook.

Example Payload

{
  "alert": "{{ .Title }}",
  "status": "{{ .Status }}",
  "message": "{{ .Message }}",
  "labels": "{{ .Labels }}"
}

---

4. Save the Contact Point

• Click Save Contact Point.

• Your webhook notification channel is now ready to use.

---

5. Create an Alert Rule

• Go to Alerting → Alert Rules.

• Click New Alert Rule.

• Select a data source (e.g., Prometheus).

• Write a query for the metric you want to monitor.

Example:

(
  count by (SnapshotID, WorkloadID, SnapshotName, status) (
    mysql_trilio_snapshots_status{status="successful"}
  )
)
unless
(
  count by (SnapshotID, WorkloadID, SnapshotName, status) (
    mysql_trilio_snapshots_status{status="successful"} offset 5m
  )
)

---

6. Define Alert Conditions

Example condition:

WHEN query(A) > 0
FOR 5m

This means the alert will trigger if the condition remains true for 5 minutes.

---

7. Attach the Webhook Contact Point

• Under Configure Notifications, select the contact point you created (custom-webhook-alert).

• Save the alert rule.

---

8. Test the Webhook

• Go to Contact Points.

• Select your webhook.

• Click Test.

Grafana will send a sample JSON payload to your configured endpoint URL.

---

Example Webhook Payload Sent by Grafana (Simplified)

{
  "status": "resolved",
  "alert": "test-snapshot-status",
  "snapshot_id": "ffe2ff4b-5122-42be-8e25-b61a0a6699c1",
  "snapshot_name": "snapshot",
  "workload_id": "7166933a-bb0b-4988-9188-e73e5a1cebb5",
  "result": "successful",
  "message": "Snapshot completed successfully"
}

---

Example Python Webhook Receiver (Flask)

Prerequisites for Example Webhook Server

• Python must be installed on the system.

• The pip package manager must be available.

• Install the Flask Python module using the following command: pip3 install flask.

• All commands must be executed on the system where the webhook server will run.

from flask import Flask, request
import json

app = Flask(__name__)

@app.route('/')
def index():
    return 'Hello, World!'

@app.route('/webhook', methods=["POST"])
def webhook():
    alertData = request.json

    # Save alert data to a file (for debugging/logging)
    with open("test.json", "w") as out_file:
        json.dump(alertData, out_file, indent=4)

    return 'ok'

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=50100, debug=True)

---

How to Run and test the Webhook Server quickly

1. Save the above Python code in a file (e.g., server1.py).

2. Test the webhook:

curl -X POST http://localhost:50100/webhook \
-H "Content-Type: application/json" \
-d '{"status":"test","message":"hello webhook"}'

---

Expected Result

• A file named test.json will be created.

• It will contain the received webhook payload.

---

1. Run the following command on your host machine:

python3 server1.py

1. Once the server starts, it will be available at:

http://localhost:50100/webhook

Use this URL in Grafana as your webhook endpoint.

---

Example Server Output

 * Running on http://127.0.0.1:50100
 * Running on http://<your-ip>:50100
172.22.0.2 - - [17/Mar/2026 07:34:28] "POST /webhook HTTP/1.1" 200 -

This confirms that Grafana successfully sent an alert to your webhook.

Note: Similarly, if you follow the above steps and create a Grafana alert rule (for example, using the MySQL snapshot query), the webhook will receive the actual alert payload. The details will be stored in test.json, confirming that Grafana successfully sent the alert to your webhook.

---