REAL-WORLD USE CASE | APRIL 13, 2025
The NAS Died. My WhatsUp Gold Server Died With It. WhatsUp Gold 360 Still Alerted Me.
A real home-lab failure shows why always-on external monitoring and cloud-originated notifications matter when the local monitoring stack becomes part of the outage.
On April 13, 2025, the NAS providing NFS storage to several virtual machines in my Proxmox lab locked up and halted. A reboot did not bring the array back quickly enough, so I began restoring the affected NFS-backed workloads from backup before the array had finished initializing.
The storage outcome was stark:
- A virtual machine protected by ZFS replication did not miss a beat.
- Virtual machines that depended on the NFS storage became unavailable.
- My main WhatsUp Gold server was one of the virtual machines on the failed NFS share.
That last point changed the incident from an ordinary storage outage into a test of the monitoring architecture itself. The system that normally reported failures inside the environment had become part of the failure.
Yet the incident was not silent. WhatsUp Gold 360 caught the loss of visibility and alerted me.
"My main WUG Server was on the NFS storage that died!"
The Blind Spot: Local Monitoring Can Fail With the Environment
An on-premises monitoring system needs its own operating system, database, compute, storage, network path, and notification services to remain available. When one of those shared dependencies fails, the monitor can disappear along with the infrastructure it was supposed to report on.
Local notification systems often depend on the same Internet connection or email services they are expected to report as unavailable. That can delay notification and extend an outage.
My home-lab failure was the same pattern with a different dependency. The Internet connection was not the original cause; shared storage was. But the result was identical from the monitoring server's perspective: it could no longer speak for the environment.
Who monitors the monitoring system when its own infrastructure fails?
How WhatsUp Gold 360 Keeps an Independent Signal
WhatsUp Gold 360 uses a tenant-specific Cloud Connector deployed in the monitored environment. The connector communicates with the cloud service and performs heartbeat, ping, and network path health operations. The cloud can then send email or SMS through configured communication channels when the connector becomes unavailable or a rule is triggered.
We call this an out-of-band notification path. In practical terms, the alert is generated by the cloud service rather than by the local WhatsUp Gold server or its local SMTP path. If the on-premises monitor is down, the cloud still has a signal to evaluate: either the connector continues reporting the problem, or its expected heartbeat disappears.
The independent signal path
| Layer | Role in the alert path |
|---|---|
| Cloud Connector | Runs inside the monitored environment and performs heartbeat, ping, and network path health checks toward the WhatsUp Gold 360 cloud. |
| WhatsUp Gold 360 | Tracks connector reachability, last updates, availability, latency, packet loss, and route information. |
| Communication channel | Uses cloud services to send email or SMS when a connector becomes unavailable or a configured rule is triggered. |
What WhatsUp Gold 360 Detected - and What It Did Not
WhatsUp Gold 360 did not inspect the NAS and announce that the array had locked. It detected that an expected signal from the environment had stopped or that a monitored component was no longer reporting.
A missing heartbeat can be caused by several different failures:
- An Internet connection outage
- A power failure
- A connector or virtual machine failure
- A hypervisor or host problem
- A storage outage
- A broader site-level failure
The alert told me that the environment required attention. Investigation then established that the NAS and NFS storage were the root cause. That distinction between detection and diagnosis makes the use case more credible, not less: detection must happen before root-cause analysis can begin.
More Than a Binary Up-or-Down Check
The WhatsUp Gold 360 dashboard can provide context around a missing heartbeat, including connector state, uptime, Internet connection events, availability, latency, packet loss, and expandable network path health details
"Latency over time" and "network path taken over time" can help distinguish a sudden local failure from recurring WAN instability, path changes, or an upstream provider problem. It is also useful when documenting an ISP issue or justifying additional bandwidth or provider changes.
Why This Matters Beyond a Home Lab
The same failure pattern appears in branch offices, retail sites, schools, clinics, factories, and managed-service environments. A local WhatsUp Gold server may monitor switches, wireless access points, applications, hosts, storage, and the WAN - while running on the same local virtualization or storage platform.
If that platform, the site power, or the internet circuit fails, the local monitor may no longer be able to send its own alert. A cloud-observed heartbeat gives central IT an independent way to learn that the site is no longer visible instead of waiting for users or customers to report the outage.
That is the business value behind the technical design: faster detection, a more dependable notification path, less time spent wondering whether a remote site is truly online, and better evidence for troubleshooting recurring connection problems.
Architecture Lessons From the Failure
Separate Failure Domains
Where practical, do not place the primary monitoring server, its database, the Cloud Connector, and the only usable backups on the same host or storage platform. The connector does not have to survive every incident, but the overall design should ensure that either it remains available or its disappearance becomes a cloud-visible event.
Treat a Missing Heartbeat as a Site-Level Event
A missing heartbeat may be an Internet problem, but it may also represent the loss of an entire virtualization platform, storage system, monitoring stack, or physical location. If the internet connection can't be confirmed, route the alert to someone who can investigate the broader infrastructure.
Test the Failure Path Deliberately
Do not wait for a real storage failure to learn whether the cloud notification path works. A controlled test - stopping a test connector, isolating its outbound path, or scheduling a maintenance exercise - can confirm that the expected email or SMS arrives and that the escalation procedure is clear.
Protect the Management Plane
The ZFS-replicated virtual machine continued operating while the NFS-backed workloads did not. That does not make one storage design universally correct, but it does show why monitoring, backup, identity, and management systems deserve independent recovery options. They are the tools needed when everything else is already broken.
From heartbeat monitoring to a broader cloud layer
WhatsUp Gold 360 starts with the immediate resilience problem demonstrated by this incident, then extends the same cloud layer into centralized visibility and, eventually, more intelligent incident workflows. The next cool part is you can add WhatsUp Gold servers to sync with your connector. This enables you to view your on-premise monitoring data from anywhere without directly exposing your WhatsUp Gold server to the internet. With additional functionalities planned to launch in the near future, prepare yourself to take advantage by deploying the WhatsUp Gold 360 connector virtual appliance today!
The Failure Was Serious. It Was Not Silent.
The NAS failed. The NFS storage hosting several virtual machines stopped working. My main WhatsUp Gold server went down with it. I restored the affected workloads from backup while the ZFS-replicated virtual machine continued operating.
Most importantly, WhatsUp Gold 360 alerted me that the environment had stopped reporting. It did not prevent the failure, and it did not replace the investigation. It provided the independent warning that allowed the investigation to begin.
That is what Internet connection and heartbeat monitoring should provide: not a promise that infrastructure will never fail, but confidence that a major failure will not remain invisible.
Because sometimes the most important system to monitor is the monitoring system itself.
For the full Proxmox architecture, HA recovery results, and April 13 storage-failure update, see Surviving a Real-World Host Crash with Proxmox HA.