Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 153 additions & 5 deletions .github/CICD.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The ODE monorepo uses GitHub Actions for continuous integration and deployment.
#### Triggers

- **Push to `main`**: Builds and publishes release images
- **Push to `develop`**: Builds and publishes pre-release images
- **Push to `dev`**: Builds and publishes pre-release images
- **Push to feature branches**: Builds and publishes branch-specific images
- **Pull Requests**: Builds but does not publish (validation only)
- **Manual Dispatch**: Allows manual triggering with optional version tag
Expand All @@ -37,7 +37,7 @@ Images are published to **GitHub Container Registry (GHCR)**:
| Branch/Event | Tags Generated | Description |
|--------------|----------------|-------------|
| `main` | `latest`, `main-{sha}` | Latest stable release |
| `develop` | `develop`, `develop-{sha}` | Development pre-release |
| `dev` | `dev`, `dev-{sha}` | Development pre-release |
| Feature branches | `{branch-name}`, `{branch-name}-{sha}` | Feature-specific builds |
| Pull Requests | `pr-{number}` | PR validation builds (not pushed) |
| Manual with version | `v{version}`, `v{major}.{minor}`, `latest` | Versioned release |
Expand Down Expand Up @@ -76,7 +76,7 @@ docker pull ghcr.io/opendataensemble/synkronus:v1.0.0
### Pull Development Build

```bash
docker pull ghcr.io/opendataensemble/synkronus:develop
docker pull ghcr.io/opendataensemble/synkronus:dev
```

### Pull Feature Branch Build
Expand Down Expand Up @@ -129,6 +129,155 @@ echo $GITHUB_TOKEN | docker login ghcr.io -u USERNAME --password-stdin
3. Select **synkronus**
4. View all published tags and their details

## Code Formatting & Linting

All workflows enforce formatting and linting checks before builds. Ensure your code is properly formatted before pushing to avoid CI failures.

### Go Projects (synkronus, synkronus-cli)

#### Format Go Code

Format all Go files in a project:

```bash
# For synkronus
cd synkronus
go fmt ./...

# Or using gofmt directly
gofmt -s -w .

# For synkronus-cli
cd synkronus-cli
go fmt ./...
```

#### Check Go Formatting (without modifying files)

```bash
# Check if files are formatted
gofmt -s -l .

# View what would change
gofmt -s -d .
```

#### Run Linting

```bash
# Install golangci-lint (if not already installed)
# macOS: brew install golangci-lint
# Linux: curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.54.2

# Run linting for synkronus
cd synkronus
golangci-lint run

# Run linting for synkronus-cli
cd synkronus-cli
golangci-lint run
```

### TypeScript/JavaScript Projects

#### Synkronus Portal

```bash
cd synkronus-portal

# Format code
npm run format

# Check formatting (no writes)
npm run format:check

# Run linting
npm run lint
```

#### Formulus (React Native)

```bash
cd formulus

# Format code
npm run format

# Check formatting (no writes)
npm run format:check

# Run linting
npm run lint

# Run linting with auto-fix
npm run lint:fix
```

#### Formulus Formplayer (React Web)

```bash
cd formulus-formplayer

# Format code
npm run format

# Check formatting (no writes)
npm run format:check

# Run linting
npm run lint

# Run linting with auto-fix
npm run lint:fix
```

### What CI Enforces

#### Go Projects

- **gofmt**: All Go files must be formatted. CI will fail if any files are unformatted.
- **golangci-lint**: Runs comprehensive linting checks using the `.golangci.yml` configuration.

#### TypeScript/JavaScript Projects

- **Prettier**: All source files must be formatted according to Prettier rules.
- **ESLint**: Code must pass ESLint checks with no errors.

### Pre-commit Checklist

Before pushing your code, run these commands:

```bash
# For Go projects
cd synkronus # or synkronus-cli
go fmt ./...
golangci-lint run

# For TypeScript/JavaScript projects
cd synkronus-portal # or formulus, or formulus-formplayer
npm run format:check
npm run lint
```

### Auto-formatting with Git Hooks (Optional)

You can set up a pre-commit hook to automatically format code:

```bash
# Create .git/hooks/pre-commit
cat > .git/hooks/pre-commit << 'EOF'
#!/bin/bash
# Format Go files
find . -name "*.go" -not -path "./vendor/*" -exec gofmt -s -w {} \;
# Format TypeScript/JavaScript files
cd synkronus-portal && npm run format
cd ../formulus && npm run format
cd ../formulus-formplayer && npm run format
EOF

chmod +x .git/hooks/pre-commit
```

## Troubleshooting

### Build Fails on Push
Expand Down Expand Up @@ -163,7 +312,7 @@ echo $GITHUB_TOKEN | docker login ghcr.io -u USERNAME --password-stdin
### For Deployments

1. **Pin versions in production**: Use specific version tags, not `latest`
2. **Test pre-releases**: Use `develop` tag for staging environments
2. **Test pre-releases**: Use `dev` tag for staging environments
3. **Monitor image sizes**: Keep images lean for faster deployments
4. **Use health checks**: Always configure health checks in deployments

Expand All @@ -175,7 +324,6 @@ Potential improvements to the CI/CD pipeline:
- [ ] Implement security scanning (Trivy, Snyk)
- [ ] Add deployment to staging environment
- [ ] Create release notes automation
- [ ] Add Slack/Discord notifications
- [ ] Implement rollback mechanisms
- [ ] Add performance benchmarking

Expand Down
32 changes: 20 additions & 12 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -169,28 +169,27 @@ jobs:
- name: Download dependencies
run: go mod download

- name: Run gofmt (warning)
- name: Run gofmt
run: |
UNFORMATTED_FILES=$(git ls-files '*.go' | xargs gofmt -s -l)

if [ -n "$UNFORMATTED_FILES" ]; then
echo "⚠️ Code is not formatted. Consider running 'go fmt ./...' or 'gofmt -s -w .'"
echo "Code is not formatted. Please run 'go fmt ./...' or 'gofmt -s -w .'"
echo "Unformatted files:"
echo "$UNFORMATTED_FILES"
echo "Diffs:"
echo "$UNFORMATTED_FILES" | xargs gofmt -s -d
exit 1
else
echo "✅ All Go files are formatted."
fi

- name: Run golangci-lint (if available)
run: |
if command -v golangci-lint &> /dev/null; then
golangci-lint run
else
echo "golangci-lint not installed, skipping..."
fi
continue-on-error: true
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: v2.0.0
working-directory: ./synkronus
args: --timeout=5m

- name: Run tests
run: go test -v -race -coverprofile=coverage.out ./...
Expand Down Expand Up @@ -226,20 +225,29 @@ jobs:
- name: Download dependencies
run: go mod download

- name: Run gofmt (warning)
- name: Run gofmt
run: |
UNFORMATTED_FILES=$(git ls-files '*.go' | xargs gofmt -s -l)

if [ -n "$UNFORMATTED_FILES" ]; then
echo "⚠️ Code is not formatted. Consider running 'go fmt ./...' or 'gofmt -s -w .'"
echo "Code is not formatted. Please run 'go fmt ./...' or 'gofmt -s -w .'"
echo "Unformatted files:"
echo "$UNFORMATTED_FILES"
echo "Diffs:"
echo "$UNFORMATTED_FILES" | xargs gofmt -s -d
exit 1
else
echo "✅ All Go files are formatted."
fi

- name: Run golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: v2.0.0
working-directory: ./synkronus-cli
args: --timeout=5m
args: --timeout=5m

- name: Run tests
run: go test -v -race -coverprofile=coverage.out ./...

Expand Down
53 changes: 53 additions & 0 deletions .github/workflows/synkronus-cli.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,62 @@ on:
types: [published]

jobs:
format-check:
name: Check Go formatting
if: github.event_name != 'release'
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.24.x'
cache-dependency-path: synkronus-cli/go.sum

- name: Check formatting
working-directory: synkronus-cli
run: |
UNFORMATTED_FILES=$(git ls-files '*.go' | xargs gofmt -s -l)

if [ -n "$UNFORMATTED_FILES" ]; then
echo "❌ Code is not formatted. Please run 'go fmt ./...' or 'gofmt -s -w .'"
echo "Unformatted files:"
echo "$UNFORMATTED_FILES"
echo ""
echo "Diffs:"
echo "$UNFORMATTED_FILES" | xargs gofmt -s -d
exit 1
else
echo "✅ All Go files are formatted."
fi

lint:
name: Lint with golangci-lint
if: github.event_name != 'release'
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.24.x'
cache-dependency-path: synkronus-cli/go.sum

- name: golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: v2.0.0
working-directory: synkronus-cli
args: --timeout=5m

build-cli:
name: Build synkronus-cli (CI)
if: github.event_name != 'release'
needs: [format-check, lint]
runs-on: ubuntu-latest

strategy:
Expand Down
27 changes: 27 additions & 0 deletions .github/workflows/synkronus-docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,33 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.24'
cache-dependency-path: synkronus/go.sum

- name: Check Go formatting
working-directory: ./synkronus
run: |
UNFORMATTED_FILES=$(git ls-files '*.go' | xargs gofmt -s -l)

if [ -n "$UNFORMATTED_FILES" ]; then
echo "❌ Code is not formatted. Please run 'go fmt ./...' or 'gofmt -s -w .'"
echo "Unformatted files:"
echo "$UNFORMATTED_FILES"
exit 1
else
echo "✅ All Go files are formatted."
fi

- name: Run golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: v2.0.0
working-directory: ./synkronus
args: --timeout=5m

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
Expand Down
23 changes: 23 additions & 0 deletions .github/workflows/synkronus-portal-docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,29 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: |
synkronus-portal/package-lock.json
packages/tokens/package-lock.json

- name: Install dependencies
run: |
cd packages/tokens && npm ci
cd ../components && npm install || true
cd ../../synkronus-portal && npm ci

- name: Run ESLint
working-directory: synkronus-portal
run: npm run lint

- name: Check Prettier formatting
working-directory: synkronus-portal
run: npm run format:check

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
Expand Down
Loading
Loading