Skip to content

Commit f4dea3b

Browse files
FizzadarFizzadar
committed
Add FoundationDB deploy example
1 parent 5ff73b7 commit f4dea3b

13 files changed

Lines changed: 346 additions & 2 deletions

File tree

‎.github/workflows/test.yaml‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,13 @@ jobs:
99
matrix:
1010
include:
1111
- dir: full-deploys/python-web-app
12-
deploy-file: deploy.py
12+
deploy-file: deploy_app.py
1313
inventory-file: inventories/docker.py
14+
15+
- dir: full-deploys/foundationdb-cluster
16+
deploy-file: deploy_foundationdb.py
17+
inventory-file: inventories/docker.py
18+
1419
runs-on: ubuntu-latest
1520
steps:
1621
- uses: actions/checkout@v3
@@ -19,10 +24,19 @@ jobs:
1924
python-version: "3.12"
2025
- uses: docker/setup-buildx-action@v2
2126
- run: pip install pyinfra --pre
27+
2228
- run: ./docker-start.sh
2329
working-directory: ${{ matrix.dir }}
30+
2431
- name: Run pyinfra deploy
2532
run: pyinfra -y ${{ matrix.inventory-file }} ${{ matrix.deploy-file }}
2633
working-directory: ${{ matrix.dir }}
34+
if: ${{ !runner.debug }}
35+
36+
- name: Run pyinfra deploy (debug mode)
37+
run: pyinfra -y -vvv ${{ matrix.inventory-file }} ${{ matrix.deploy-file }}
38+
working-directory: ${{ matrix.dir }}
39+
if: ${{ runner.debug }}
40+
2741
- run: ./docker-stop.sh
2842
working-directory: ${{ matrix.dir }}

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
.vagrant*
2+
settings.local.json

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ Simple: deploys two servers: one database and one web running a Python app fetch
1313

1414
### [`foundationdb-cluster`](./full-deploys/foundationdb-cluster)
1515

16-
Advanced: sets up a five node FoundationDB cluster from scratch. Uses runtime callbacks to bootstrap.
16+
Advanced: sets up a five node FoundationDB cluster from scratch. Uses nested operations (callbacks) to bootstrap.
1717

1818
## Snippets
1919

‎full-deploys/foundationdb-cluster/README.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Example: FoundationDB Cluster
2+
3+
This example deploys a 5-node FoundationDB cluster using pyinfra and Docker containers. Usage:
4+
5+
```sh
6+
# Start Docker containers
7+
./docker-start.sh
8+
9+
# Run pyinfra against them
10+
pyinfra inventories/docker.py deploy_foundationdb.py
11+
12+
# Verify the cluster is health
13+
ssh -p 9022 -i ../../.docker/insecure_private_key pyinfra@localhost
14+
fdbcli --exec "status details"
15+
16+
# Delete Docker containers
17+
./docker-stop.sh
18+
```
19+
20+
## File Layout
21+
22+
```
23+
foundationdb-cluster/
24+
├── deploy_foundationdb.py # Main deployment entry point
25+
├── tasks/
26+
│ ├── install.py # Package download and installation
27+
│ ├── configure.py # Config files and service management
28+
│ └── bootstrap.py # Cluster initialization
29+
├── templates/
30+
│ └── foundationdb.conf.j2 # fdbmonitor configuration template
31+
├── group_data/
32+
│ └── all.py # Shared settings for all hosts
33+
└── inventories/
34+
└── docker.py # Docker inventory with host data
35+
```

‎full-deploys/foundationdb-cluster/deploy_foundationdb.py‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
"""
2+
FoundationDB Cluster Deployment
3+
4+
Deploys a 5-node FoundationDB cluster with:
5+
- 3 coordinator nodes (nodes 1-3) for quorum
6+
- 5 storage nodes (all nodes)
7+
- Triple redundancy mode
8+
9+
Usage:
10+
pyinfra inventories/docker.py deploy_foundationdb.py
11+
"""
12+
13+
from os import path
14+
15+
from pyinfra import host, local
16+
from pyinfra.operations import apt
17+
18+
# Update apt cache (with 1 hour cache time to avoid unnecessary updates)
19+
apt.packages(
20+
name="Update apt cache",
21+
packages=["wget"],
22+
cache_time=3600,
23+
update=True,
24+
)
25+
26+
# Install FoundationDB packages
27+
local.include(
28+
filename=path.join("tasks", "install.py"),
29+
)
30+
31+
# Configure FoundationDB (config files, cluster file, service)
32+
local.include(
33+
filename=path.join("tasks", "configure.py"),
34+
)
35+
36+
# Bootstrap cluster (only on the designated bootstrap node)
37+
if host.data.get("is_bootstrap_node"):
38+
local.include(
39+
filename=path.join("tasks", "bootstrap.py"),
40+
)

‎full-deploys/foundationdb-cluster/docker-start.sh‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
#!/usr/bin/env bash
2+
3+
set -euo pipefail
4+
5+
source "$(realpath "$(realpath "$(dirname "${BASH_SOURCE[0]}")")/../../utils.sh")"
6+
ensure_test_container
7+
8+
export DOCKER_TEST_NETWORK_NAME="pyinfra-examples-foundationdb-cluster"
9+
10+
echo "Create Docker network..."
11+
docker network create "$DOCKER_TEST_NETWORK_NAME"
12+
13+
echo "Starting Docker containers..."
14+
run_test_container pyinfra-example-foundationdb-node-1 -p 9022:22 -p 4500:4500
15+
run_test_container pyinfra-example-foundationdb-node-2 -p 9023:22
16+
run_test_container pyinfra-example-foundationdb-node-3 -p 9024:22
17+
run_test_container pyinfra-example-foundationdb-node-4 -p 9025:22
18+
run_test_container pyinfra-example-foundationdb-node-5 -p 9026:22
19+
20+
echo
21+
echo "Doker containers are now ready to run the pyinfra deploy, you can do this by running:"
22+
echo
23+
echo " pyinfra inventories/docker.py deploy_foundationdb.py"
24+
echo
25+
echo "Once complete, don't forget to remove the Docker containers and network using the ./docker-stop.sh script!"

‎full-deploys/foundationdb-cluster/docker-stop.sh‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
#!/usr/bin/env bash
2+
3+
set -euo pipefail
4+
5+
source "$(realpath "$(realpath "$(dirname "${BASH_SOURCE[0]}")")/../../utils.sh")"
6+
7+
export DOCKER_TEST_NETWORK_NAME="pyinfra-examples-foundationdb-cluster"
8+
9+
docker rm -f pyinfra-example-foundationdb-node-1
10+
docker rm -f pyinfra-example-foundationdb-node-2
11+
docker rm -f pyinfra-example-foundationdb-node-3
12+
docker rm -f pyinfra-example-foundationdb-node-4
13+
docker rm -f pyinfra-example-foundationdb-node-5
14+
docker network rm "$DOCKER_TEST_NETWORK_NAME"

‎full-deploys/foundationdb-cluster/group_data/all.py‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
fdb_version = "7.3.61"
2+
fdb_port = 4500
3+
fdb_coordinators = [
4+
f"pyinfra-example-foundationdb-node-1.pyinfra-examples-foundationdb-cluster:{fdb_port + 1}",
5+
f"pyinfra-example-foundationdb-node-2.pyinfra-examples-foundationdb-cluster:{fdb_port + 1}",
6+
f"pyinfra-example-foundationdb-node-3.pyinfra-examples-foundationdb-cluster:{fdb_port + 1}",
7+
]
8+
9+
fdb_cluster_name = "docker"
10+
fdb_cluster_id = "pyinfra123456789" # Random cluster identifier
11+
fdb_redundancy_mode = "triple"
12+
fdb_storage_engine = "ssd-redwood-1"

‎full-deploys/foundationdb-cluster/inventories/docker.py‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
inventory = (
2+
# Individual host list with host-specific data
3+
[
4+
(
5+
"node-1",
6+
{"ssh_port": 9022, "is_coordinator": True, "is_bootstrap_node": True},
7+
),
8+
("node-2", {"ssh_port": 9023, "is_coordinator": True}),
9+
("node-3", {"ssh_port": 9024, "is_coordinator": True}),
10+
("node-4", {"ssh_port": 9025}),
11+
("node-5", {"ssh_port": 9026}),
12+
],
13+
# Shared data for all the hosts in the group
14+
{
15+
"_sudo": True, # use sudo for all operations
16+
# SSH details matching the Docker container started in ./docker-start.sh
17+
"ssh_hostname": "localhost",
18+
"ssh_user": "pyinfra",
19+
"ssh_key": "../../.docker/insecure_private_key",
20+
"ssh_known_hosts_file": "/dev/null",
21+
# This is insecure, don't use in production!
22+
"ssh_strict_host_key_checking": "off",
23+
},
24+
)

‎full-deploys/foundationdb-cluster/tasks/bootstrap.py‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
"""
2+
FoundationDB cluster bootstrap tasks.
3+
Initializes the cluster configuration (runs only on bootstrap node).
4+
"""
5+
6+
import json
7+
8+
from pyinfra import host
9+
from pyinfra.operations import python, server
10+
11+
12+
def bootstrap_cluster():
13+
"""
14+
Initialize the FoundationDB cluster if not already configured.
15+
"""
16+
17+
# Note: this command will take a while to execute because it has to timeout trying to get the
18+
# status, I have yet to find a quick way to do this check.
19+
check_status = server.shell(
20+
name="Check cluster status",
21+
commands="fdbcli --no-status --exec 'status json' || true",
22+
)
23+
24+
status_json = json.loads(check_status.stdout)
25+
status_client = status_json.get("client", {})
26+
27+
# Note this is, so far, the best way I can tell to determine if FDB cluster is bootstrapped,
28+
# despite that for production I'd recommend any configure new commands are executed by hand.
29+
if (
30+
status_client.get("coordinators", {}).get("quorum_reachable") is True
31+
and status_client.get("database_status", {}).get("available") is False
32+
):
33+
redundancy_mode = host.data.fdb_redundancy_mode
34+
storage_engine = host.data.fdb_storage_engine
35+
server.shell(
36+
name="Configure cluster",
37+
commands=f"fdbcli --no-status --exec 'configure new {redundancy_mode} {storage_engine}'",
38+
)
39+
40+
41+
# Use python.call to execute the bootstrap function at runtime
42+
# This ensures proper state checking during deployment
43+
python.call(
44+
name="Bootstrap FoundationDB cluster",
45+
function=bootstrap_cluster,
46+
)

0 commit comments

Comments
 (0)