5. [Advanced] - how to create snapDB on your server

If you running validator and an extra node together, you can use extra node to create a snanDB snaphot - smaller DB(120GB+) with only validator info - neither old state nor old transaction history.

SnapDB snapshots be the core team

Please note, you can still find the snapDB from the core team by the link, but if your own you are welcome to proceed.

Requirements

• an extra Harmony node

• time -> snapDB creation is taking around 2 days

• extra space:

  • current DB around 200->500 GB

  • snapDB will be around 120+ GB

Creating a snapDB snapshot

You need to do the following to get the fresh snapDB:

1. Stop currently running harmony service/container/any other way how you run Harmony node, e.g. systemd service:

sudo systemctl stop harmony.service
# we don't want some server reboot to start normal harmony service
sudo systemctl disable harmony.service

1. Place the following bash script `check-and-run-snapdb.sh` nearby the binary.

Reason : the dumpdb feature may crash, fortunately, at restart of the process the dump will start where it left off

Note: KakashiDB batch writhing and account lines will produce you 90GB of logs without providing you any extra information => so let's just ignore these lines.

#!/bin/bash
​
# make sure the folder where the new snapDB is is created
mkdir -p /data/snapdb/new_snapdb_creation
​
# Specify the name of the program you want to monitor
program_name="harmony"
​binary_path="/put_yours_here"
db_path="/put_yours_here" # by default it is the same folder as for binary
snapdb_path="/put_yours_here"

# Use pgrep to get the number of PIDs for the specified program
pid_count=$(pgrep -cx "$program_name")
​
if [ "$pid_count" -eq 1 ]; then
    echo "$program_name is running all good"
else
    "$binary_path"/harmony dumpdb \
    "$db_path" "$snapdb_path" | \
     grep -v "KakashiDB batch writhing\|account" >> "$binary_path"/newsnapdb.log &
fi

1. Put the check-and-run-snapdb.sh in the cron to rerun every 5 minutes via crontab -e

Note: by default SHELL in crontab is set to /bin/sh and all relative path like ~/ wouldn't work here, use full path

*/5 * * * * bash /data/snapdb/check-and-run-snapdb.sh

1. Time to time check the logs:

less +F +G /your_path_for_logs/newsnapdb.log

1. What will you see in the logs when snapdb creation have been finished:

Note, this process will take around 2 days

===dumpMain===
head-block: 48393536 0x03cd3c3093f4299a4d70721c2d70192080e82a42401b72a1cdc425751ce8ac12
start copying...
Dump completed!
KakashiDB Close

1. Comment the crontab entry from the 3rd step via crontab -e

#*/5 * * * * bash /data/snapdb/check-and-run-snapdb.sh

1. Move your old DB and name it like backup, and make a :

db_path="/put_yours_here"
snapdb_path="/put_yours_here/"
mv "$db_path" "$db_path"_backup
mv "$snapdb_path" "$db_path"

1. Start your harmony process again:

sudo systemctl start harmony.service
sudo systemctl enable harmony.service

1. Wait till the node is synced with the latest HEAD block, it will take some small amount of time around 5-15 minutes - node needs to fetch 2+ day blocks.

2. If everything good - remove old DB from the server, check the step 7:

Note: running rm -rf is dangerous, please double check what are you about to delete!

rm -rf /your_path_to_DB_backup/