diff --git a/README.md b/README.md index cec22be..23d798e 100644 --- a/README.md +++ b/README.md @@ -4,147 +4,126 @@ A repository to help with deployment of the OpenUTM toolset in your cloud. ## Steps -This example is based on **DigitalOcean** cloud, it uses the `kustomize` tool and kubectl so you can use it with your cloud with it. +This example is based on **DigitalOcean** cloud, it uses the `kustomize` tool and `kubectl` so you can use it with your cloud with it. -### πŸš€ Before You Begin (3-step process) +### πŸš€ Before You Begin To ensure a smooth deployment process, follow these three steps: -1. πŸ“„ **Understand the Architecture** - Familiarize yourself with the system's architecture by reviewing the [.env file documentation](system-architecture-env-file-documentation.md). This will give you a clear picture of how the components interact. +1. πŸ“„ **Understand the Architecture** + Familiarize yourself with the system's architecture by reviewing the [.env file documentation](system-architecture-env-file-documentation.md). This will give you a clear picture of how the components interact. -2. πŸ”‘ **Deploy and Connect** - Once the `.env` files are created, deploy the systems. Since Flight Passport acts as the authentication bridge between the backend and frontend, you'll need to update the environment files with the correct variables after deployment. Reapply these updated variables as detailed in Step 3. +2. πŸ”‘ **Deploy and Connect** + Once the `.env` files are created, deploy the systems. Since Flight Passport acts as the authentication bridge between the backend and frontend, you'll need to update the environment files with the correct variables after deployment. Reapply these updated variables as detailed in Step 3. -3. πŸ”— **Link Components** - Check out the [environment files documentation](linking-spotlight-blender-via-passport.md) to properly link Spotlight and Blender via Passport. This step ensures seamless integration between all components. +3. πŸ”— **Link Components** + Check out the [environment files documentation](linking-spotlight-blender-via-passport.md) to properly link Spotlight and Blender via Passport. This step ensures seamless integration between all components. -πŸ’¬ **Need Help?** +πŸ’¬ **Need Help?** If you run into any issues, feel free to reach out to us on Discord. We're here to help! ### Pre-requisites 1. Installed and configured `doctl`, [link](https://docs.digitalocean.com/reference/doctl/how-to/install/) 2. Created Kubernetes Cluster, [link](https://docs.digitalocean.com/products/kubernetes/how-to/create-clusters/) -3. Created Load Balancer, [link](https://docs.digitalocean.com/products/kubernetes/how-to/add-load-balancers/) -4. You can connect to your Cluster, [link](https://docs.digitalocean.com/products/kubernetes/how-to/connect-to-cluster/) -5. Your domain sub zone points to DigitalOcean DNS, [link](https://docs.digitalocean.com/products/networking/dns/getting-started/dns-registrars/) +3. You can connect to your Cluster, [link](https://docs.digitalocean.com/products/kubernetes/how-to/connect-to-cluster/) +4. Your domain sub zone points to DigitalOcean DNS, [link](https://docs.digitalocean.com/products/networking/dns/getting-started/dns-registrars/) +5. `openssl` installed (for generating keys) ### Your deployment **NOTE**: We will assume your sub-domain is `test.example.com`, and contact email is `test@example.com` - these need to be customized -1. Create A and CNAME records for your domain on the DigitalOcean NS - -```bash -# SETUP env variables -export DOMAIN_NAME="test.example.com" -export ACME_CONTACT_EMAIL="test@example.com" -export LOAD_BALANCER_IP=$(doctl compute load-balancer list --format IP --no-header) - -# SETUP A and CNAME -doctl compute domain records list $DOMAIN_NAME -doctl compute domain delete $DOMAIN_NAME -f -doctl compute domain create $DOMAIN_NAME -doctl compute domain records create $DOMAIN_NAME \ - --record-type "A" --record-name "$DOMAIN_NAME." \ - --record-data "$LOAD_BALANCER_IP" \ - --record-ttl "30" -doctl compute domain records create $DOMAIN_NAME \ - --record-type "CNAME" --record-name "*" \ - --record-data "$DOMAIN_NAME." \ - --record-ttl "30" -doctl compute domain records list $DOMAIN_NAME -``` - -2. Install `ingress-nginx` - -```bash -helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx -helm repo update ingress-nginx -helm search repo ingress-nginx -helm install ingress-nginx ingress-nginx/ingress-nginx \ - --namespace ingress-nginx \ - --create-namespace -``` - -3. Install `cert-manager` - -```bash -helm repo add jetstack https://charts.jetstack.io -helm repo update jetstack -helm search repo jetstack -helm install cert-manager jetstack/cert-manager \ - --namespace cert-manager \ - --create-namespace \ - --set installCRDs=true -``` - -4. Edit your `.env` files from `env.examples` folder, and place them in this structure: - -```bash -└── kustomize - Β Β  β”œβ”€β”€ blender - Β Β  β”‚Β Β  └── .env.blender - Β Β  β”œβ”€β”€ passport - Β Β  β”‚Β Β  └── .env.passport - Β Β  └── spotlight - Β Β  └── .env.spotlight -``` - -5. Generate the OIDC key and deploy your applications - -```bash -openssl genrsa -out kustomize/passport/oidc.key 4096 -kubectl apply -k kustomize/ -``` - -6. Create personal access token with full access to modify `domain`, [link](https://docs.digitalocean.com/reference/api/create-personal-access-token/), and create kubernetes secret from it - -```bash -export DO_API_TOKEN=__YOUR TOKEN HERE__ -kubectl create secret generic "digitalocean-dns" \ - --from-literal=access-token="$DO_API_TOKEN" --namespace=openutm -``` - -7. Edit `generate-from-templates.sh` and customize the following env variables - -```bash -export DOMAIN_NAME="test.example.com" -export ACME_CONTACT_EMAIL="test@example.com" -``` - -8. Run the file to generate customized yaml files from templates - -```bash -./generate-from-templates.sh -``` - -9. Create `Issuer` - -```bash -kubectl apply -f issuer.yaml -``` - -10. Create root and wildcard `Certificate` - -```bash -kubectl apply -f certificate-root.yaml -kubectl apply -f certificate-wcard.yaml -``` - -11. Create `Ingress` - -```bash -kubectl apply -f ingress.yaml -``` - -It will take some time for all components to settle and acquire certificates. After that, your apps should be accessible under the following domains with trusted certificates: - -- `https://blender.$DOMAIN_NAME` -- `https://spotlight.$DOMAIN_NAME` -- `https://passport.$DOMAIN_NAME` +1. Install `caddy-ingress-controller` + + ```bash + # SETUP env variables + export ACME_CONTACT_EMAIL="test@example.com" + + helm repo add caddy-ingress-controller https://caddyserver.github.io/ingress/ + helm repo update + helm install caddy-ingress-controller caddy-ingress-controller/caddy-ingress-controller \ + --namespace caddy-system \ + --create-namespace \ + --version 1.3.0 \ + --set ingressController.config.email="$ACME_CONTACT_EMAIL" \ + --set ingressController.config.acmeCA="https://acme-v02.api.letsencrypt.org/directory" + ``` + +2. Create A and CNAME records for your domain on the DigitalOcean NS + + **Note**: Wait for the Load Balancer to be assigned an IP address. You can check this with `kubectl get svc -n caddy-system caddy-ingress-controller`. + + ```bash + # SETUP env variables + export DOMAIN_NAME="test.example.com" + + # Get the Load Balancer IP from the installed service + export LOAD_BALANCER_IP=$(kubectl get svc -n caddy-system caddy-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}') + + # SETUP A and CNAME + doctl compute domain records list $DOMAIN_NAME + doctl compute domain delete $DOMAIN_NAME -f + doctl compute domain create $DOMAIN_NAME + doctl compute domain records create $DOMAIN_NAME \ + --record-type "A" --record-name "$DOMAIN_NAME." \ + --record-data "$LOAD_BALANCER_IP" \ + --record-ttl "30" + doctl compute domain records create $DOMAIN_NAME \ + --record-type "CNAME" --record-name "*" \ + --record-data "$DOMAIN_NAME." \ + --record-ttl "30" + doctl compute domain records list $DOMAIN_NAME + ``` + +3. Create your `.env` files from the examples: + + ```bash + cp env.examples/.blender.env.example kustomize/blender/.env.blender + cp env.examples/.passport.env.example kustomize/passport/.env.passport + cp env.examples/.spotlight.env.example kustomize/spotlight/.env.spotlight + ``` + + Then edit these new files with your specific configuration (database passwords, secrets, etc.). + +4. Generate the OIDC key and deploy your applications + + > **Note:** Blender and Spotlight containers may restart or fail health checks initially until they are linked with valid Passport credentials (see "Configuring Passport" section below). + + ```bash + openssl genrsa -out kustomize/passport/oidc.key 4096 + kubectl apply -k kustomize/ + ``` + +5. Generate `Ingress` manifest + + Export your domain name and run the generation script. This will replace placeholders in `templates/` and create `ingress.yaml`. + + ```bash + export DOMAIN_NAME="test.example.com" + ./generate-from-templates.sh + ``` + +6. Create `Ingress` + + ```bash + kubectl apply -f ingress.yaml + ``` + +7. Verify the Deployment + + Check that all pods are running and healthy: + + ```bash + kubectl get pods -n openutm + ``` + + It will take some time for all components to settle and acquire certificates. After that, your apps should be accessible under the following domains with trusted certificates: + + - `https://blender.$DOMAIN_NAME` + - `https://spotlight.$DOMAIN_NAME` + - `https://passport.$DOMAIN_NAME` ### Configuring Passport and subsequently Blender and Spotlight -The first step is to configure Flight Passport and once Flight Passport is up and running you will have to update the variables for Blender and Spotlight and re-deploy. Use the [constructing environment files](constructing_environment_files.md) to login to Passport and generate variables for Spotlight and Blender. Once these are setup, they need to be reapplied to the cluster. \ No newline at end of file +The first step is to configure Flight Passport and once Flight Passport is up and running you will have to update the variables for Blender and Spotlight and re-deploy. Use the [linking documentation](linking-spotlight-blender-via-passport.md) to login to Passport and generate variables for Spotlight and Blender. Once these are setup, they need to be reapplied to the cluster. diff --git a/generate-from-templates.sh b/generate-from-templates.sh index c4f4ea4..b8acbd1 100755 --- a/generate-from-templates.sh +++ b/generate-from-templates.sh @@ -1,7 +1,6 @@ export DOMAIN_NAME="test.example.com" -export ACME_CONTACT_EMAIL="test@example.com" -for file_name in issuer certificate-wcard certificate-root ingress; do +for file_name in ingress; do ( echo "cat <${file_name}.yaml" cat templates/${file_name}-template.yaml diff --git a/kustomize/blender/beat.yaml b/kustomize/blender/beat.yaml index 9b8c940..eb3d096 100644 --- a/kustomize/blender/beat.yaml +++ b/kustomize/blender/beat.yaml @@ -19,7 +19,7 @@ spec: spec: containers: - name: beat - image: blender + image: flight-blender command: - ./entrypoints/with-database/entrypoint-beat.sh # imagePullPolicy: IfNotPresent @@ -32,6 +32,8 @@ spec: env: - name: POSTGRES_HOST value: "db-blender" + - name: POSTGRES_PORT + value: "5432" - name: PGDATA value: "/var/lib/postgresql/data/pgdata" - name: POSTGRES_DB @@ -50,7 +52,7 @@ spec: key: POSTGRES_PASSWORD name: env-secrets-blender - name: DATABASE_URL - value: "pgsql://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@$(POSTGRES_HOST):5432/$(POSTGRES_DB)" + value: "pgsql://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@$(POSTGRES_HOST):$(POSTGRES_PORT)/$(POSTGRES_DB)" - name: SECRET_KEY valueFrom: secretKeyRef: @@ -153,4 +155,3 @@ spec: secretKeyRef: key: DJANGO_LOG_LEVEL name: env-secrets-blender - diff --git a/kustomize/blender/blender.yaml b/kustomize/blender/blender.yaml index 1e8cdd1..418d768 100644 --- a/kustomize/blender/blender.yaml +++ b/kustomize/blender/blender.yaml @@ -22,7 +22,7 @@ spec: serviceAccountName: blender-svc containers: - name: blender - image: blender + image: flight-blender command: - ./entrypoints/with-database/entrypoint-prod.sh # imagePullPolicy: IfNotPresent @@ -38,6 +38,8 @@ spec: env: - name: POSTGRES_HOST value: "db-blender" + - name: POSTGRES_PORT + value: "5432" - name: PGDATA value: "/var/lib/postgresql/data/pgdata" - name: POSTGRES_DB @@ -56,7 +58,7 @@ spec: key: POSTGRES_PASSWORD name: env-secrets-blender - name: DATABASE_URL - value: "pgsql://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@$(POSTGRES_HOST):5432/$(POSTGRES_DB)" + value: "pgsql://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@$(POSTGRES_HOST):$(POSTGRES_PORT)/$(POSTGRES_DB)" - name: SECRET_KEY valueFrom: secretKeyRef: @@ -164,5 +166,3 @@ spec: volumes: - name: tmp emptyDir: {} - - diff --git a/kustomize/blender/db-blender.yaml b/kustomize/blender/db-blender.yaml index 0206252..38ac49a 100644 --- a/kustomize/blender/db-blender.yaml +++ b/kustomize/blender/db-blender.yaml @@ -34,6 +34,8 @@ spec: value: "/var/lib/postgresql/data/pgdata" - name: POSTGRES_HOST value: "db-blender" + - name: POSTGRES_PORT + value: "5432" - name: POSTGRES_DB valueFrom: secretKeyRef: diff --git a/kustomize/blender/kustomization.yaml b/kustomize/blender/kustomization.yaml index 41361db..95b21d7 100644 --- a/kustomize/blender/kustomization.yaml +++ b/kustomize/blender/kustomization.yaml @@ -4,7 +4,7 @@ kind: Kustomization namespace: openutm images: - - name: blender + - name: flight-blender newName: openutm/flight-blender:latest diff --git a/kustomize/blender/worker.yaml b/kustomize/blender/worker.yaml index 35485b7..d978549 100644 --- a/kustomize/blender/worker.yaml +++ b/kustomize/blender/worker.yaml @@ -19,7 +19,7 @@ spec: spec: containers: - name: worker - image: blender + image: flight-blender command: - ./entrypoints/with-database/entrypoint-celery.sh # imagePullPolicy: IfNotPresent @@ -32,6 +32,8 @@ spec: env: - name: POSTGRES_HOST value: "db-blender" + - name: POSTGRES_PORT + value: "5432" - name: PGDATA value: "/var/lib/postgresql/data/pgdata" - name: POSTGRES_DB @@ -50,7 +52,7 @@ spec: key: POSTGRES_PASSWORD name: env-secrets-blender - name: DATABASE_URL - value: "pgsql://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@$(POSTGRES_HOST):5432/$(POSTGRES_DB)" + value: "pgsql://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@$(POSTGRES_HOST):$(POSTGRES_PORT)/$(POSTGRES_DB)" - name: SECRET_KEY valueFrom: secretKeyRef: @@ -158,4 +160,3 @@ spec: volumes: - name: tmp emptyDir: {} - diff --git a/kustomize/passport/db-passport.yaml b/kustomize/passport/db-passport.yaml index 0ecf395..2015a6a 100644 --- a/kustomize/passport/db-passport.yaml +++ b/kustomize/passport/db-passport.yaml @@ -34,6 +34,8 @@ spec: value: "/var/lib/postgresql/data/pgdata" - name: POSTGRES_HOST value: "db-passport" + - name: POSTGRES_PORT + value: "5432" - name: POSTGRES_DB valueFrom: secretKeyRef: diff --git a/kustomize/passport/passport.yaml b/kustomize/passport/passport.yaml index 677b24e..60cd4f2 100644 --- a/kustomize/passport/passport.yaml +++ b/kustomize/passport/passport.yaml @@ -38,6 +38,8 @@ spec: env: - name: POSTGRES_HOST value: "db-passport" + - name: POSTGRES_PORT + value: "5432" - name: PGDATA value: "/var/lib/postgresql/data/pgdata" - name: POSTGRES_DB @@ -56,7 +58,7 @@ spec: key: POSTGRES_PASSWORD name: env-secrets-passport - name: DATABASE_URL - value: "pgsql://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@$(POSTGRES_HOST):5432/$(POSTGRES_DB)" + value: "pgsql://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@$(POSTGRES_HOST):$(POSTGRES_PORT)/$(POSTGRES_DB)" - name: DJANGO_SECRET_KEY valueFrom: secretKeyRef: @@ -135,4 +137,3 @@ spec: volumes: - name: tmp emptyDir: {} - diff --git a/templates/certificate-root-template.yaml b/templates/certificate-root-template.yaml deleted file mode 100644 index ba2d315..0000000 --- a/templates/certificate-root-template.yaml +++ /dev/null @@ -1,19 +0,0 @@ -apiVersion: cert-manager.io/v1 -kind: Certificate -metadata: - name: letsencrypt-nginx-root - namespace: openutm - # Cert-Manager will put the resulting Secret in the same Kubernetes namespace as the Certificate. -spec: - # Secret name to create, where the private key and certificate should be stored. - secretName: letsencrypt-nginx-root-tls - # What Issuer to use for getting the certificate. - issuerRef: - name: letsencrypt-nginx-issuer - kind: Issuer - group: cert-manager.io - # Common name to be used on the Certificate. - commonName: "${DOMAIN_NAME}" - # List of DNS subjectAltNames to be set on the Certificate. - dnsNames: - - "${DOMAIN_NAME}" diff --git a/templates/certificate-wcard-template.yaml b/templates/certificate-wcard-template.yaml deleted file mode 100644 index 008c1da..0000000 --- a/templates/certificate-wcard-template.yaml +++ /dev/null @@ -1,19 +0,0 @@ -apiVersion: cert-manager.io/v1 -kind: Certificate -metadata: - name: letsencrypt-nginx-wcard - namespace: openutm - # Cert-Manager will put the resulting Secret in the same Kubernetes namespace as the Certificate. -spec: - # Secret name to create, where the private key and certificate should be stored. - secretName: letsencrypt-nginx-wcard-tls - # What Issuer to use for getting the certificate. - issuerRef: - name: letsencrypt-nginx-issuer - kind: Issuer - group: cert-manager.io - # Common name to be used on the Certificate. - commonName: "*.${DOMAIN_NAME}" - # List of DNS subjectAltNames to be set on the Certificate. - dnsNames: - - "*.${DOMAIN_NAME}" diff --git a/templates/ingress-template.yaml b/templates/ingress-template.yaml index ee8238d..b5ee23a 100644 --- a/templates/ingress-template.yaml +++ b/templates/ingress-template.yaml @@ -3,18 +3,8 @@ kind: Ingress metadata: name: ingress-apps namespace: openutm - annotations: - nginx.ingress.kubernetes.io/ssl-redirect: "true" - cert-manager.io/issuer: "letsencrypt-nginx-issuer" - acme.cert-manager.io/dns01-edit-in-place: "true" - labels: - app: nginx spec: - ingressClassName: nginx - tls: - - hosts: - - "*.${DOMAIN_NAME}" - secretName: letsencrypt-nginx-wcard-ingress + ingressClassName: caddy rules: - host: "blender.${DOMAIN_NAME}" http: @@ -53,17 +43,10 @@ metadata: name: ingress-root namespace: openutm annotations: - nginx.ingress.kubernetes.io/ssl-redirect: "true" - cert-manager.io/issuer: "letsencrypt-nginx-issuer" - nginx.ingress.kubernetes.io/permanent-redirect: https://spotlight.${DOMAIN_NAME} - labels: - app: nginx + caddy.ingress.kubernetes.io/config: | + redir https://spotlight.${DOMAIN_NAME}{uri} permanent spec: - ingressClassName: nginx - tls: - - hosts: - - "${DOMAIN_NAME}" - secretName: letsencrypt-nginx-root-ingress + ingressClassName: caddy rules: - host: "${DOMAIN_NAME}" http: diff --git a/templates/issuer-template.yaml b/templates/issuer-template.yaml deleted file mode 100644 index cd54a77..0000000 --- a/templates/issuer-template.yaml +++ /dev/null @@ -1,29 +0,0 @@ -apiVersion: cert-manager.io/v1 -kind: Issuer -metadata: - name: letsencrypt-nginx-issuer - namespace: openutm -spec: - # ACME issuer configuration: - # email - the email address to be associated with the ACME account (make sure it's a valid one). - # server - the URL used to access the ACME server’s directory endpoint. - # privateKeySecretRef - Kubernetes Secret to store the automatically generated ACME account private key. - acme: - email: "${ACME_CONTACT_EMAIL}" - server: https://acme-v02.api.letsencrypt.org/directory - privateKeySecretRef: - name: letsencrypt-nginx-issuer - # List of challenge solvers that will be used to solve ACME challenges for the matching domains. - solvers: - # Use the DigitalOcean DNS API to manage DNS01 challenge records. - - dns01: - digitalocean: - # Kubernetes secret that contains the DO API token . - # Must be in the same namespace as the Issuer CRD. - tokenSecretRef: - name: digitalocean-dns - key: access-token - selector: - dnsZones: - - "${DOMAIN_NAME}" - - "*.${DOMAIN_NAME}"