To check whether your node is synchronized with the leader, there's a built-in endpoint at port 5000 called /node-sync. It returns a boolean flag true or false to indicate whether your node is caught up to the latest block, on par with the majority node-runners.
This endpoint can be used to feed into your favorite dashboard metrics (Grafana, CloudWatch, etc.) and trigger alarms (Healthchecks.io, UptimeRobot, PagerDuty, webhooks into Discord, etc.) that can to help monitor your node operations. If you're running a node service to host the RPC endpoint on port 9500, this will be a great way to check the health of your nodes behind a load balancer and take unhealthy nodes out of rotation.
Prometheus & Grafana
Prerequisites
Golang
Make sure you have Golang installed. For instructions on how to install here click here. To make sure it is installed run correctly:
goversion
Output example:
goversiongo1.16.3linux/amd64
Check Metrics
Make sure metrics are being exposed. For this check you can run:
You can now load Prometheus on your browser: http://Your_IP_Address:9090. You can run up or any other query to see if everthing is working as expected:
Setup Grafana
Install
Follow instructions here on how to download and install Grafana for your OS.
Make sure to enable and start Grafana server.
Once started, you should be able to load Grafana on your browser: http://Your_IP_Address:3000. Default user and password is admin.
Add Prometheus Datasource
Go to http://Your_IP_Address:3000/datasourcesand add Prometheu as datasource:
Import Dashboard
Go to http://Your_IP_Address:3000/dashboard/import and load the json file below: