The more I use Redis server, the more I enjoy it.
During the last year, I saw a lot of good system administrators using HashiCorp Consul to store information for their shell scripts.
As I am concerned, I am not a big fan of Consul because it can be a real gasworks to deploy and configure.
I prefer to use Redis server on my workstation to store configuration information.
Introduction
In the world of FreeBSD system administration, automation and scripting are essential tools for efficiently managing and maintaining systems.
Many administrators have adopted HashiCorp Consul for managing key-value pairs and coordinating their scripts, but Redis offers a simpler and more lightweight alternative.
Redis is a powerful in-memory data structure store that can easily be used for storing configuration data, managing locks, and tracking script execution status.
In this article, I will demonstrate how you can use Redis in conjunction with shell scripting to manage backup tasks on your FreeBSD system.
Why Redis Over Consul?
HashiCorp Consul is indeed a robust tool, especially when you need a full-fledged service discovery and configuration management system.
However, its complexity can be overkill for smaller tasks or personal use cases.
Redis, on the other hand, is much simpler to set up and use, making it a great alternative when your needs are more straightforward, such as storing key-value pairs for shell scripts.
Redis excels in scenarios where you need quick access to configuration data or need to manage state information for your scripts.
Its simplicity, speed, and versatility makes it an excellent choice for system administrators looking for a lightweight solution.
Adding the List of Users to Redis
Before running the backup script, we need to populate Redis with the list of users whose home directories need to be backed up.
This can be done easily using Redis CLI commands.
Here's how you can add users to the backup_users
set in Redis:
Populating the User List in Redis
To add users to the Redis set, you can use the sadd
command, which adds one or more members to a set stored at a given key.
In our case, the key is backup_users
. Here’s how you can do it:
# Add individual users
redis-cli sadd backup_users myuser1
redis-cli sadd backup_users myuser2
redis-cli sadd backup_users myuser3
# Verify the users were added
redis-cli smembers backup_users
The sadd
command adds user1
, user2
, and user3
to the backup_users
set.
You can add as many users as you need by repeating the command with different usernames.
Managing the User List
You can manage the list of users directly from the Redis CLI or using scripts. To remove a user from the backup list, use the srem
command:
redis-cli srem backup_users myuser3
To clear the entire list of users:
redis-cli del backup_users
These commands give you full control over the list of users to be backed up, allowing you to dynamically manage who gets backed up based on your needs.
Shell Scripting with Redis: A Practical Example
Let's walk through a practical example where we use Redis to manage a shell script that backs up user directories on a FreeBSD system.
The Scenario
We want to create a script that performs the following tasks:
Fetch a list of users whose home directories need to be backed up.
Back up each user’s home directory to a tarball (TGZ file).
Use a lock mechanism to ensure that the script doesn't run multiple times concurrently.
Store the status of the last backup (success or failure) in Redis.
Setting Up Redis on FreeBSD
First, you need to have Redis installed and running on your FreeBSD system.
You can install Redis using the FreeBSD package management system (pkg
):
sudo pkg install redis
Ensure Redis starts on boot:
sudo sysrc redis_enable=yes
Start the Redis server:
sudo service redis start
The Shell Script
Here’s the shell script that achieves our goal:
#!/bin/sh
# Variables
REDIS_CLI="/usr/local/bin/redis-cli"
LOCK_KEY="backup_lock"
USERS_KEY="backup_users"
BACKUP_STATUS_KEY="backup_status"
BACKUP_DIR="/backup"
# Check if another instance of the script is running
lock_status=$($REDIS_CLI get $LOCK_KEY)
if [ "$lock_status" = "locked" ]; then
echo "Backup script is already running. Exiting."
exit 1
fi
# Set the lock
$REDIS_CLI set $LOCK_KEY "locked"
# Fetch the list of users from Redis
users=$($REDIS_CLI smembers $USERS_KEY)
# Perform backup
for user in $users; do
user_home="/home/$user"
if [ -d "$user_home" ]; then
tar -czf $BACKUP_DIR/${user}_backup.tgz -C $user_home .
if [ $? -eq 0 ]; then
$REDIS_CLI hset $BACKUP_STATUS_KEY $user "success"
echo "Backup for $user completed successfully."
else
$REDIS_CLI hset $BACKUP_STATUS_KEY $user "failure"
echo "Backup for $user failed."
fi
else
echo "User $user does not have a home directory. Skipping."
fi
done
# Remove the lock
$REDIS_CLI del $LOCK_KEY
exit 0
Explanation of the Script
Lock Mechanism: The script first checks if the
backup_lock
key in Redis is set to "locked".
If it is, this means the script is already running, so it exits to prevent multiple instances from running simultaneously.
If not, it sets thebackup_lock
key to "locked" to prevent other instances from starting.User List: The script fetches a list of users to back up from Redis using the
smembers
command, which retrieves all members of a Redis set stored under thebackup_users
key.Backup Process: For each user, the script checks if the user's home directory exists. If it does, it creates a tarball (
tgz
file) of the directory.
The backup is stored in the/backup
directory with the format{user}_backup.tgz
.Backup Status: After each backup, the script updates the
backup_status
key in Redis with the result of the backup (either "success" or "failure").Lock Removal: Finally, the script removes the lock key in Redis, allowing future executions of the script.
Conclusion
Redis provides a lightweight and flexible solution for storing configuration data and managing state information in shell scripts.
While HashiCorp Consul is a powerful tool, Redis is often more than sufficient for many system administration tasks.
In this example, we used Redis to manage a backup script, demonstrating how it can be effectively employed to prevent race conditions, store configuration data, and track script execution status.
By integrating Redis into your shell scripts, you can simplify your system administration tasks and avoid the overhead of more complex tools like Consul. Redis's speed, simplicity, and ease of use makes it a valuable addition to any FreeBSD administrator’s toolkit.