A validator that fails silently is worse than one that fails loudly, because a loud failure gets fixed and a silent one gets discovered by users, by a missed attestation report, or by a slashing or downtime penalty that has already happened. Monitoring dashboards do not fix that problem on their own: a dashboard only helps if a human is looking at it at the moment it turns red. Alerts fix it, by pushing the failure to a human, on a channel the human actually reads, with enough detail to act. Telegram is a good alert channel for small teams for a simple reason: it is already on the operator's phone, it supports bots with a simple HTTP API, and it can carry a short, actionable message without a paid paging tool.
This post covers the alerts that earn a page, the pipeline that delivers them, and the Telegram bot setup that makes it work, plus the failure mode most alert setups miss: the alert system itself going quiet. It assumes you already have a node and basic monitoring, as covered in the Grafana monitoring post. If you do not, build the monitoring first: alerts without metrics are just guesses sent to your phone.
The five alerts that earn a page
Alert on failures that cost money, uptime, or trust, not on every wobble. First, sync lag: the node is behind the chain tip by more than your threshold, or it has stopped advancing at all, because a lagging validator misses duties and a lagging RPC endpoint serves stale data. Second, missed duties: attestations or proposals missed in the last few slots, because a single miss is a warning and a pattern is an incident. Third, process and peer health: the client process is down, peer count has collapsed, or the consensus and execution clients cannot talk to each other, because any of those stops the node even when the server itself is up.
Fourth, disk and resource pressure: disk use above your alert threshold, disk growth faster than expected, or memory and CPU saturation that precedes a crash, because disk-full is one of the most common and most preventable node failures, and the pruning post covers the fix once the alert fires. Fifth, the heartbeat: no metrics at all for a set period, because a monitoring system that stops reporting looks identical to a healthy system with no alerts, and that silence is the most dangerous state of all. Every other alert assumes metrics are flowing. The heartbeat alert proves they are.
The pipeline: metrics to message
A workable pipeline has four stages, and each stage has one job. The node exposes metrics: sync status, peer count, attestation results, disk use, and process health, through its client metrics endpoint or an exporter. A collector, such as Prometheus, scrapes those metrics on a schedule and stores them, so an alert can be based on a trend, not a single sample. An alerting layer, such as Grafana alerting or Alertmanager, evaluates rules against the stored metrics and decides when a condition has held long enough to page a human. A notifier, in this case a Telegram bot, delivers the message. Keeping those stages separate matters, because when an alert is wrong, you need to know whether the metric was wrong, the rule was wrong, or the delivery failed, and a single tangled script hides all three.
Write alert rules with a duration, not just a threshold. A node that lags for ten seconds during a client restart is not an incident. A node that lags for five minutes is. The same applies to peer count and missed duties: a short blip is noise, a sustained pattern is signal. Set two severities and keep them apart: critical alerts page the on-call operator immediately, and informational alerts go to a separate channel or a daily summary, where they cannot train the operator to ignore the critical channel. Alert fatigue is not a personality flaw. It is what happens when a system pages a human for noise, and the fix is fewer, better alerts, not a more disciplined human.
Set up the Telegram bot
Telegram's Bot API is an HTTP interface, and the setup has three steps. First, create a bot through BotFather in Telegram, with the newbot command, and BotFather will reply with a token. Treat that token like a password: store it in a secret manager or an environment variable, never commit it to a repository, and rotate it if it ever leaks, because anyone with the token can control the bot. Second, find the chat ID for the channel or group that should receive alerts, by messaging the bot and reading the chat identifier from the bot updates response. Third, send a test message through the API, using the sendMessage method with the chat ID and the text, and confirm it arrives before you wire any real alert to it.
# Test message only - replace the placeholders, never commit a real token:
curl -X POST https://api.telegram.org/botYOUR_BOT_TOKEN/sendMessage -H "Content-Type: application/json" -d '{"chat_id": "YOUR_CHAT_ID", "text": "Test alert: validator monitoring is live."}'
Once the test works, wire the notifier to the alerting layer, not to the node directly. Grafana alerting and Alertmanager both support webhook notifiers, and a small webhook receiver can translate the alert payload into a Telegram message, with the alert name, the node, the value that crossed the threshold, and the time it started. Keep the message short and actionable: what broke, which node, how long it has been broken, and the first check to run. A message that only says that something is wrong, without saying what or where, creates a second incident: the operator logging in to find out what the first incident was. Include a link to the dashboard or runbook when you have one, and include a resolved message when the condition clears, so the operator knows the incident is over without guessing.
Test the alert, then test the silence
An alert pipeline that has never fired is untested, however correct it looks. Test it deliberately: stop a test node, or raise a test threshold, and confirm the message arrives, the right person sees it, and the resolved message follows when the node recovers. Then test the failure mode that matters most: stop the metrics pipeline itself, and confirm the heartbeat alert fires, telling you that no data has arrived. If the heartbeat test produces silence, your alert system has a single point of failure, and you have found it in a test instead of in a real outage, which is the entire point of testing.
Finally, review alerts monthly and delete the ones that never fire, retune the ones that fire too often, and promote the ones that caught a real incident. An alert set is a living system: chains upgrade, thresholds drift, and a rule that was right last quarter may be noise this quarter. If alerting, monitoring, and on-call coverage are taking operator time away from running nodes, node operations covers managed monitoring and upgrades, starting with a free infra audit. Bring your five critical alerts, your on-call rota, and your last silent failure: those three facts decide whether Telegram alerting is enough, or whether you need a full paging setup behind it.
Disclaimer: This post is educational only, not financial advice. Figures are listing or project data as of 2026-11-02 and may change - always confirm version numbers, requirements, and dates in the official docs before acting on them.