diff --git a/docker-compose/nested-spire/README.md b/docker-compose/nested-spire/README.md index 626a228b..ebd3a2ac 100644 --- a/docker-compose/nested-spire/README.md +++ b/docker-compose/nested-spire/README.md @@ -34,21 +34,21 @@ Before proceeding, review the following system requirements: This tutorial's `nested-spire` main directory contains three subdirectories, one for each of the SPIRE deployments: `root`, `nestedA` and `nestedB`. These directories hold the configuration files for the SPIRE Servers and Agents. They will also contain the private keys and certificates created to attest the Agents on the Servers with the [x509pop Node Attestor](https://github.com/spiffe/spire/blob/main/doc/plugin_server_nodeattestor_x509pop.md) plugin. Private keys and certificates are created at the initialization of the scenario using a Go application, the details of which are out of the scope of this tutorial. -## Create a Shared Directory +## The Shared Socket Directory -The first thing needed is a local directory that will be volume mounted on the services to share the Workload API between the root SPIRE Agent and its nested SPIRE Servers. This tutorial uses `.../spire-tutorials/docker-compose/nested-spire/sharedRootSocket` as the shared directory. +We use a shared directory that will be volume mounted on the services to share the Workload API between the root SPIRE Agent and its nested SPIRE Servers. This tutorial uses a named volume in its docker-compose configuration to share the Workload API socket. ## Configuring Root SPIRE Deployment Configuration files for [root-server](root/server/server.conf) and [root-agent](root/agent/agent.conf) have not been changed from the default `server.conf` and `agent.conf` files, but it's worth noting the location defined to bind the workload API socket by the SPIRE Agent: `socket_path ="/opt/spire/sockets/workload_api.sock"`. This path will be used later to configure a volume to share the Workload API with the nested SPIRE Servers. -We define all the services for the tutorial in the [docker-compose.yaml](docker-compose.yaml) file. In the `root-agent` service definition we mount the `/opt/spire/sockets` directory from the SPIRE Agent container on the new local directory `sharedRootSocket`. In the next section, when defining the nested SPIRE Server services, we'll use this directory to mount the `root-agent` socket on the SPIRE Server containers. +We define all the services for the tutorial in the [docker-compose.yaml](docker-compose.yaml) file. In the `root-agent` service definition we mount the `/opt/spire/sockets` directory from the SPIRE Agent container on the named `spire-sockets` volume. In the next section, when defining the nested SPIRE Server services, we'll use this directory to mount the `root-agent` socket on the SPIRE Server containers. ```console services: # Root root-server: - image: ghcr.io/spiffe/spire-server:1.5.1 + image: ghcr.io/spiffe/spire-server:1.11.2 hostname: root-server volumes: - ./root/server:/opt/spire/conf/server @@ -56,12 +56,12 @@ We define all the services for the tutorial in the [docker-compose.yaml](docker- root-agent: # Share the host pid namespace so this agent can attest the nested servers pid: "host" - image: ghcr.io/spiffe/spire-agent:1.5.1 + image: ghcr.io/spiffe/spire-agent:1.11.2 depends_on: ["root-server"] hostname: root-agent volumes: # Share root agent socket to be accessed by nestedA and nestedB servers - - ./sharedRootSocket:/opt/spire/sockets + - spire-sockets:/opt/spire/sockets - ./root/agent:/opt/spire/conf/agent - /var/run/:/var/run/ command: ["-config", "/opt/spire/conf/agent/agent.conf"] @@ -85,7 +85,7 @@ The configuration file for the [nestedA-server](./nestedA/server/server.conf) in } ``` -The Docker Compose definition for the `nestedA-server` service in the [docker-compose.yaml](docker-compose.yaml) file mounts the new local directory `sharedRootSocket` as a volume. Remember from the previous section that the `root-agent` socket is mounted on that directory. That way the `nestedA-server` can access the `root-agent` workload API and fetch its SVID. +The Docker Compose definition for the `nestedA-server` service in the [docker-compose.yaml](docker-compose.yaml) file mounts the `spire-sockets` named volume. Remember from the previous section that the `root-agent` socket is mounted on that directory. That way the `nestedA-server` can access the `root-agent` workload API and fetch its SVID. ```console nestedA-server: @@ -97,8 +97,8 @@ The Docker Compose definition for the `nestedA-server` service in the [docker-co # label to attest nestedA-server against root-agent - org.example.name=nestedA volumes: - # Add root agent socket - - ./shared/rootSocket:/opt/spire/sockets + # Add root agent socket + - spire-sockets:/opt/spire/sockets - ./nestedA/server:/opt/spire/conf/server command: ["-config", "/opt/spire/conf/server/server.conf"] ``` diff --git a/docker-compose/nested-spire/docker-compose.yaml b/docker-compose/nested-spire/docker-compose.yaml index 8b4faa4a..4c707094 100644 --- a/docker-compose/nested-spire/docker-compose.yaml +++ b/docker-compose/nested-spire/docker-compose.yaml @@ -2,7 +2,7 @@ version: '3' services: # Root root-server: - image: ghcr.io/spiffe/spire-server:1.5.1 + image: ghcr.io/spiffe/spire-server:1.11.2 hostname: root-server volumes: - ./root/server:/opt/spire/conf/server @@ -10,20 +10,20 @@ services: root-agent: # Share the host pid namespace so this agent can attest the nested servers pid: "host" - image: ghcr.io/spiffe/spire-agent:1.5.1 + image: ghcr.io/spiffe/spire-agent:1.11.2 depends_on: ["root-server"] hostname: root-agent volumes: # Share root-agent socket to be accessed by nested servers - - ./sharedRootSocket:/opt/spire/sockets + - spire-sockets:/opt/spire/sockets - ./root/agent:/opt/spire/conf/agent - - /var/run/:/var/run/ + - /var/run/docker.sock:/var/run/docker.sock command: ["-config", "/opt/spire/conf/agent/agent.conf"] # NestedA nestedA-server: # Share the host pid namespace so this server can be attested by the root agent pid: "host" - image: ghcr.io/spiffe/spire-server:1.5.1 + image: ghcr.io/spiffe/spire-server:1.11.2 hostname: nestedA-server labels: # label to attest server against root-agent @@ -31,21 +31,21 @@ services: depends_on: ["root-server","root-agent"] volumes: # Add root-agent socket - - ./sharedRootSocket:/opt/spire/sockets + - spire-sockets:/opt/spire/sockets - ./nestedA/server:/opt/spire/conf/server command: ["-config", "/opt/spire/conf/server/server.conf"] nestedA-agent: - image: ghcr.io/spiffe/spire-agent:1.5.1 + image: ghcr.io/spiffe/spire-agent:1.11.2 hostname: nestedA-agent depends_on: ["nestedA-server"] volumes: - ./nestedA/agent:/opt/spire/conf/agent - - /var/run/:/var/run/ + - /var/run/docker.sock:/var/run/docker.sock command: ["-config", "/opt/spire/conf/agent/agent.conf"] nestedB-server: # Share the host pid namespace so this server can be attested by the root agent pid: "host" - image: ghcr.io/spiffe/spire-server:1.5.1 + image: ghcr.io/spiffe/spire-server:1.11.2 hostname: nestedB-server depends_on: ["root-server","root-agent"] labels: @@ -53,14 +53,24 @@ services: - org.example.name=nestedB-server volumes: # Add root-agent socket - - ./sharedRootSocket:/opt/spire/sockets + - spire-sockets:/opt/spire/sockets - ./nestedB/server:/opt/spire/conf/server command: ["-config", "/opt/spire/conf/server/server.conf"] nestedB-agent: - image: ghcr.io/spiffe/spire-agent:1.5.1 + image: ghcr.io/spiffe/spire-agent:1.11.2 hostname: nestedB-agent depends_on: ["nestedB-server"] volumes: - ./nestedB/agent:/opt/spire/conf/agent - - /var/run/:/var/run/ + - /var/run/docker.sock:/var/run/docker.sock command: ["-config", "/opt/spire/conf/agent/agent.conf"] + +volumes: + # Allows macs to have functional domain sockets between spire agents and + # servers by avoiding a volume mount to the host filesystem and instead + # creating a tmpfs volume to be used as a socket directory. + spire-sockets: + driver: local + driver_opts: + type: tmpfs + device: tmpfs diff --git a/docker-compose/nested-spire/scripts/set-env.sh b/docker-compose/nested-spire/scripts/set-env.sh index df2ff980..49f1d70f 100755 --- a/docker-compose/nested-spire/scripts/set-env.sh +++ b/docker-compose/nested-spire/scripts/set-env.sh @@ -31,6 +31,20 @@ fingerprint() { openssl x509 -in "$1" -outform DER | openssl sha1 -r | awk '{print $1}' } +wait-for-server() { + for ((i=1;i<=30;i++)); do + if ! docker-compose -f "${PARENT_DIR}"/docker-compose.yaml exec -T $1 /opt/spire/bin/spire-server healthcheck; then + log "Waiting for $1 to start..." + sleep 1 + else + log "${green}$1 is up and running.${norm}" + return 0 + fi + done + log "${red}$1 did not start.${norm}" + return 1 +} + check-entry-is-propagated() { # Check at most 30 times that the agent has successfully synced down the workload entry. # Wait one second between checks. @@ -57,10 +71,6 @@ fi sed -i.bak "s#\#container_id_cgroup_matchers#container_id_cgroup_matchers#" "${PARENT_DIR}"/root/agent/agent.conf sed -i.bak "s#CGROUP_MATCHERS#$CGROUP_MATCHERS#" "${PARENT_DIR}"/root/agent/agent.conf -# create a shared folder for root agent socket to be accessed by nestedA and nestedB servers -mkdir -p "${PARENT_DIR}"/sharedRootSocket - - # Starts root SPIRE deployment log "Generate certificates for the root SPIRE deployment" setup "${PARENT_DIR}"/root/server "${PARENT_DIR}"/root/agent @@ -68,6 +78,9 @@ setup "${PARENT_DIR}"/root/server "${PARENT_DIR}"/root/agent log "Start root server" docker-compose -f "${PARENT_DIR}"/docker-compose.yaml up -d root-server +wait-for-server root-server +docker-compose -f "${PARENT_DIR}"/docker-compose.yaml exec -T root-server /opt/spire/bin/spire-server bundle show > "${PARENT_DIR}"/root/agent/bootstrap.crt + log "bootstrapping root-agent." docker-compose -f "${PARENT_DIR}"/docker-compose.yaml exec -T root-server /opt/spire/bin/spire-server bundle show > "${PARENT_DIR}"/root/agent/bootstrap.crt @@ -103,6 +116,7 @@ setup "${PARENT_DIR}"/nestedA/server "${PARENT_DIR}"/nestedA/agent log "Starting nestedA-server.." docker-compose -f "${PARENT_DIR}"/docker-compose.yaml up -d nestedA-server +wait-for-server nestedA-server log "bootstrapping nestedA agent..." docker-compose -f "${PARENT_DIR}"/docker-compose.yaml exec -T nestedA-server /opt/spire/bin/spire-server bundle show > "${PARENT_DIR}"/nestedA/agent/bootstrap.crt @@ -117,6 +131,7 @@ setup "${PARENT_DIR}"/nestedB/server "${PARENT_DIR}"/nestedB/agent log "Starting nestedB-server.." docker-compose -f "${PARENT_DIR}"/docker-compose.yaml up -d nestedB-server +wait-for-server nestedB-server log "bootstrapping nestedB agent..." docker-compose -f "${PARENT_DIR}"/docker-compose.yaml exec -T nestedB-server /opt/spire/bin/spire-server bundle show > "${PARENT_DIR}"/nestedB/agent/bootstrap.crt