How to Configure Consul KV Using Docker

Consul by HashiCorp is a service networking platform that includes a distributed key-value store. This tutorial will guide you through setting up Consul using Docker and demonstrate how to use its key-value store functionality.

Prerequisites

• Docker installed on your system
• Basic understanding of Docker commands
• Terminal/command-line access

Step 1 — Creating a Docker Network

First, create a dedicated network for Consul:

docker network create consul

Step 2 — Starting the Consul Server

Launch a Consul server container in development mode:

docker run -d \
  --name=consul-server \
  --network=consul \
  -p 8500:8500 \
  -p 8600:8600/udp \
  hashicorp/consul agent -server -ui -node=server-1 -bootstrap-expect=1 -client=0.0.0.0

Command Explanation:

• -d: Run container in background
• --name=consul-server: Name the container
• --network=consul: Use the network we created
• -p 8500:8500: HTTP API and Web UI port
• -p 8600:8600/udp: DNS interface port
• -server: Run in server mode
• -ui: Enable web interface
• -bootstrap-expect=1: Single server setup
• -client=0.0.0.0: Allow external connections

Step 3 — Accessing the Consul UI

Access the Consul web interface at:

http://localhost:8500

Step 4 — Using the Key-Value Store

Setting Key-Value Pairs

Use the Consul CLI within the container to set values:

docker exec consul-server consul kv put redis/config/minconns "1"
docker exec consul-server consul kv put redis/config/maxconns "25"

Retrieving Values

Get a specific value:

docker exec consul-server consul kv get redis/config/minconns

Get all values under a prefix:

docker exec consul-server consul kv get -recurse redis/config

Deleting Values

Delete a specific key:

docker exec consul-server consul kv delete redis/config/minconns

Delete all keys under a prefix:

docker exec consul-server consul kv delete -recurse redis/config

Step 5 — Working with JSON Values

Store complex configurations using JSON:

docker exec consul-server consul kv put web/config '{"id": "web_01", "port": 8080, "environment": "production"}'

Retrieve and format JSON data:

docker exec consul-server consul kv get web/config | jq .

Step 6 — Using Consul KV with Environment Variables

Create a script to load configuration:

#!/bin/bash
# get-config.sh
export REDIS_MIN_CONNS=$(docker exec consul-server consul kv get redis/config/minconns)
export REDIS_MAX_CONNS=$(docker exec consul-server consul kv get redis/config/maxconns)
echo "Configuration loaded:"
echo "REDIS_MIN_CONNS=$REDIS_MIN_CONNS"
echo "REDIS_MAX_CONNS=$REDIS_MAX_CONNS"

Practical Usage Tips

Organizing Keys

Follow these best practices for key organization:

• Use forward slashes (/) to create hierarchies
• Include application name in the path
• Add environment information when needed
• Example: myapp/production/database/connection-string

Backup and Restore

Backup all KV pairs:

docker exec consul-server consul kv export > consul-backup.json

Restore from backup:

cat consul-backup.json | docker exec -i consul-server consul kv import -

Security Considerations

• Never store sensitive data without encryption
• Use ACLs in production environments
• Regularly backup your KV store
• Monitor access logs for unauthorized attempts

Troubleshooting Common Issues

Connection Refused

docker logs consul-server

Permission Denied

docker exec consul-server consul acl bootstrap

Next Steps

After setting up your Consul KV store, consider:

• Implementing service discovery
• Setting up ACLs for security
• Configuring Consul clusters for high availability
• Integrating with your application's configuration management

With this setup, you now have a working Consul key-value store that can be used for distributed configuration management. Remember to implement proper security measures before using in a production environment.