diff --git a/README.md b/README.md index 67b0e1a..7688ab5 100644 --- a/README.md +++ b/README.md @@ -19,10 +19,10 @@ notifications. Deploy as AWS Lambda, standard HTTP server, or container. ### Prerequisites -* GitHub App ([setup guide](#github-app-setup)) * Go ≥ 1.24 -* **Optional**: Okta API Service app for group sync -* **Optional**: Slack app for notifications +* GitHub App ([setup guide](docs/github-app-setup.md)) +* **Optional**: Okta API Service app ([setup guide](docs/okta-setup.md)) +* **Optional**: Slack app ([setup guide](docs/slack-setup.md)) ### Deployment Options @@ -176,36 +176,14 @@ Map Okta groups to GitHub teams using JSON rules: Okta-synced teams and sends Slack notifications. Enabled by default when sync is enabled. -## Okta Setup +## Integration Setup -Create an API Services application in Okta Admin Console: +Detailed setup guides for each integration: -1. **Applications** → **Create App Integration** → **API Services** -2. Name: `github-bot-api-service` -3. **Client Credentials**: - - Authentication: **Public key / Private key** - - Generate and download private key (PEM format) - - Note the Client ID -4. **Okta API Scopes**: Grant `okta.groups.read` and `okta.users.read` - -Use the Client ID and private key in your environment variables. - -## GitHub App Setup - -Create a GitHub App in your organization settings: - -1. **Developer settings** → **GitHub Apps** → **New GitHub App** -2. **Basic info**: - - Name: `github-ops-app` - - Webhook URL: Your API Gateway URL - - Webhook secret: Generate and save for `APP_GITHUB_WEBHOOK_SECRET` -3. **Permissions**: - - Repository: Pull requests (Read), Contents (Read) - - Organization: Members (Read & write), Administration (Read) -4. **Events**: Subscribe to Pull request, Team, Membership -5. Generate and download private key (`.pem` file) -6. Install app to your organization -7. Note: **App ID**, **Installation ID** (from install URL), **Private key** +- [GitHub App Setup](docs/github-app-setup.md) - Create and install the GitHub + App with required permissions +- [Okta Setup](docs/okta-setup.md) - Configure API Services app for group sync +- [Slack Setup](docs/slack-setup.md) - Create Slack app for notifications ## Development @@ -243,15 +221,63 @@ CMD ["/server"] ## How It Works -**Okta Sync**: EventBridge triggers sync → Fetch Okta groups → Apply rules → -Update GitHub teams → Detect orphaned users → Send Slack reports. Automatically -reconciles when external team changes are detected. Only syncs ACTIVE Okta -users, skips external collaborators, and prevents mass removal during outages -via safety threshold. Orphaned user detection identifies org members not in any -synced teams. +``` + ┌─────────────────────────────────────────────────┐ + │ github-ops-app │ + │ │ + ┌──────────────┐ │ ┌───────────────────────────────────────────┐ │ + │ GitHub │ webhooks │ │ Webhook Handler │ │ + │ │───────────────▶ • PR merge events │ │ + │ • PR merge │ │ │ • Team membership changes │ │ + │ • Team edit │ │ │ • Signature verification │ │ + └──────────────┘ │ └─────────────┬─────────────────────────────┘ │ + │ │ │ + │ ▼ │ + │ ┌─────────────────────────────────────────┐ │ + │ │ PR Compliance Check │ │ + ┌──────────────┐ │ │ • Branch protection verification │────────┐ + │ Okta │ │ │ • Required checks validation │ │ │ + │ │ │ │ • Bypass detection │ │ │ + │ • Groups │◀──────────────┴─────────────────────────────────────────┘ │ │ + │ • Users │ │ │ │ + └──────────────┘ │ ┌─────────────────────────────────────────┐ │ │ + │ │ │ Okta Sync Engine │ │ │ + │ │ │ • Match groups via rules │ │ │ + └─────────────────────────▶ • Create/update GitHub teams │ │ │ + │ │ • Sync team membership │ │ │ + │ │ • Orphaned user detection │────────┤ + │ │ • Safety threshold protection │ │ │ + ┌──────────────┐ │ └─────────────────────────────────────────┘ │ │ + │ GitHub │ │ │ │ │ + │ Teams API │◀─────────────────────────────────────────────────────────────┘ │ + │ │ │ │ │ + │ • Teams │ └─────────────────────────────────────────────────┘ │ + │ • Members │ │ + └──────────────┘ │ + ┌──────────────┐ │ + │ Slack │◀─────────────────────────────────────┘ + │ │ Notifications + │ • Alerts │ • PR violations + │ • Reports │ • Sync reports + └──────────────┘ • Orphaned users +``` + +### Okta Sync Flow + +1. **Trigger**: Scheduled cron/EventBridge or team membership webhook +2. **Fetch**: Query Okta groups matching configured rules +3. **Match**: Apply sync rules to map Okta groups → GitHub teams +4. **Sync**: Add/remove GitHub team members (ACTIVE Okta users only) +5. **Safety**: Abort if removal ratio exceeds threshold (default 50%) +6. **Report**: Send Slack notification with changes and orphaned users + +### PR Compliance Flow -**PR Compliance**: Webhook on PR merge → Verify signature → Check branch -protection rules → Detect bypasses → Notify Slack if violations found. +1. **Receive**: GitHub webhook on PR merge to monitored branch +2. **Verify**: Validate webhook signature (HMAC-SHA256) +3. **Check**: Query branch protection rules and required status checks +4. **Detect**: Identify bypasses (admin override, missing reviews, failed checks) +5. **Notify**: Send Slack alert with violation details ## Troubleshooting diff --git a/docs/github-app-setup.md b/docs/github-app-setup.md new file mode 100644 index 0000000..360bab7 --- /dev/null +++ b/docs/github-app-setup.md @@ -0,0 +1,165 @@ +# GitHub App Setup + +This guide walks through creating and installing a GitHub App for github-ops-app. + +## Prerequisites + +- GitHub organization with admin access + +## Step 1: Create the GitHub App + +1. Navigate to your organization's settings: + - Go to `https://github.com/organizations/YOUR_ORG/settings/apps` + - Or: **Organization** → **Settings** → **Developer settings** → **GitHub Apps** + +2. Click **New GitHub App** + +3. Fill in the basic information: + + | Field | Value | + |-----------------------|-------------------------------------------------| + | GitHub App name | `github-ops-app` (must be unique across GitHub) | + | Homepage URL | Your organization's URL or repo URL | + | Webhook > Webhook URL | Leave blank for now | + | Webhook > Secret | Generate a strong secret (save this for later) | + | Webhook > Active | **Uncheck** to disable webhooks initially | + + > **Note**: Disable webhooks during creation since you may not know your + > endpoint URL until after deployment. You'll configure webhooks and + > subscribe to events in [Step 7](#step-7-configure-webhook-and-events). + +4. Under **Permissions**, set the following: + - Repository Permissions + - Contents: Read + - Read branch protection rules + - Pull requests: Read + - Access PR details for compliance + - Organization Permissions + - Administration: Read + - Read organization settings + - Members: Read/Write + - Manage team membership + +4. Under Set installation scope: + - Where can this GitHub App be installed?: Only on this account + +5. Click **Create GitHub App** + +## Step 2: Generate Private Key + +After creating the app: + +1. Scroll to **Private keys** section +2. Click **Generate a private key** +3. Save the downloaded `.pem` file securely +4. This file is used for `APP_GITHUB_APP_PRIVATE_KEY` or + `APP_GITHUB_APP_PRIVATE_KEY_PATH` + +## Step 3: Note Your App ID + +On the app's settings page, find and save: + +- **App ID** - numeric ID displayed near the top (e.g., `123456`) + +## Step 4: Install the App + +1. In the left sidebar, click **Install App** +2. Select your organization +3. Choose repository access: + - **All repositories** - recommended for org-wide PR compliance + - **Only select repositories** - if limiting scope +4. Click **Install** + +## Step 5: Get Installation ID + +After installation, you'll be redirected to a URL like: +``` +https://github.com/organizations/YOUR_ORG/settings/installations/12345678 +``` + +The number at the end (`12345678`) is your **Installation ID**. + +Alternatively, use the GitHub API: +```bash +# List installations (requires app JWT authentication) +curl -H "Authorization: Bearer YOUR_JWT" \ + https://api.github.com/app/installations +``` + +## Step 6: Configure Environment Variables + +Set these environment variables in your deployment: + +```bash +# Required GitHub configuration +APP_GITHUB_APP_ID=123456 +APP_GITHUB_INSTALLATION_ID=12345678 +APP_GITHUB_ORG=your-org-name +APP_GITHUB_WEBHOOK_SECRET=your-webhook-secret + +# Private key (choose one method) +APP_GITHUB_APP_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY----- +... +-----END RSA PRIVATE KEY-----" + +# Or use a file path +APP_GITHUB_APP_PRIVATE_KEY_PATH=/path/to/private-key.pem + +# Or use AWS SSM parameter +APP_GITHUB_APP_PRIVATE_KEY=arn:aws:ssm:us-east-1:123456789:parameter/github-bot/private-key +``` + +## Step 7: Configure Webhook and Events + +After deploying your server, configure and enable webhooks: + +1. Go to your GitHub App settings: + `https://github.com/organizations/YOUR_ORG/settings/apps/YOUR_APP` +2. On the **General** tab, under **Webhook**: + - Set **Webhook URL** to your endpoint: + - Lambda: `https://xxx.execute-api.region.amazonaws.com/webhooks` + - Server: `https://your-domain.com/webhooks` + - Check **Active** to enable webhooks + - Click **Save changes** +3. Go to the **Permissions & events** tab +4. Scroll to **Subscribe to events** and check: + - [x] **Pull request** - PR open, close, merge events + - [x] **Team** - Team creation, deletion, changes + - [x] **Membership** - Team membership changes +5. Click **Save changes** + +## Verification + +Test your setup: + +1. **Webhook delivery**: Check **Settings** → **Developer settings** → + **GitHub Apps** → your app → **Advanced** → **Recent Deliveries** + +2. **Create a test PR**: Open and merge a PR to a monitored branch to verify + webhook reception + +3. **Check logs**: Verify your application receives and processes the webhook + +## Troubleshooting + +### Webhook signature verification failed + +- Verify `APP_GITHUB_WEBHOOK_SECRET` matches the secret in GitHub App settings +- Check for whitespace or encoding issues in the secret + +### 401 Unauthorized from GitHub API + +- Verify the private key matches the one generated for this app +- Check that the app is installed on the target organization +- Ensure `APP_GITHUB_INSTALLATION_ID` is correct + +### Missing permissions error + +- Re-check the app's permission settings +- After changing permissions, organization admins may need to re-approve + +### Webhook not received + +- Verify the webhook URL is accessible from the internet +- Check the webhook URL doesn't have a trailing slash mismatch +- Review recent deliveries in GitHub App settings for error details diff --git a/docs/okta-setup.md b/docs/okta-setup.md new file mode 100644 index 0000000..3f7b26b --- /dev/null +++ b/docs/okta-setup.md @@ -0,0 +1,221 @@ +# Okta Setup + +This guide walks through creating an Okta API Services application for +github-ops-app to sync Okta groups with GitHub teams. + +## Prerequisites + +- Okta organization with admin access +- Super Admin or Application Admin role + +## Step 1: Create API Services Application + +1. Log in to your **Okta Admin Console** +2. Navigate to **Applications** → **Applications** +3. Click **Create App Integration** +4. Select **API Services** and click **Next** + + > API Services apps use OAuth 2.0 client credentials flow with no user + > context, ideal for server-to-server integrations. + +5. Enter application name: `github-ops-app` (or similar) +6. Click **Save** + +## Step 2: Configure Client Authentication + +After creating the app: + +1. Go to the **General** tab +2. Under **Client Credentials**, click **Edit** +3. Set **Client authentication** to **Public key / Private key** +4. Click **Save** + +## Step 3: Generate Key Pair + +### Option A: Generate in Okta (Recommended) + +1. Under **PUBLIC KEYS**, click **Add Key** +2. Click **Generate new key** +3. Click **Download PEM** to save the private key +4. Click **Save** + +The downloaded file contains your private key for `APP_OKTA_PRIVATE_KEY`. + +### Option B: Generate Your Own Key + +```bash +# Generate private key +openssl genpkey -algorithm RSA -out okta-private-key.pem -pkeyopt rsa_keygen_bits:2048 + +# Extract public key in JWK format (for Okta) +# You'll need to convert PEM to JWK - use a tool like: +# https://8gwifi.org/jwkconvertfunctions.jsp +# Or use the node jose library +``` + +Then upload the public JWK to Okta under **PUBLIC KEYS** → **Add Key**. + +## Step 4: Note Client ID + +On the **General** tab, find and save: + +- **Client ID** - alphanumeric string (e.g., `0oa1abc2def3ghi4j5k6`) + +## Step 5: Grant API Scopes + +1. Go to the **Okta API Scopes** tab +2. Grant the following scopes: + + | Scope | Purpose | + |--------------------|-------------------------------| + | `okta.groups.read` | Read group names and members | + | `okta.users.read` | Read user profiles | + +3. Click **Grant** for each scope + +These scopes allow read-only access to groups and users - no write access to +Okta is required. + +## Step 6: Identify Your Okta Domain + +Your Okta domain is the URL you use to access the admin console: + +- Production: `your-org.okta.com` +- Preview/Dev: `your-org.oktapreview.com` or `dev-123456.okta.com` + +Use the domain without `https://` prefix for `APP_OKTA_DOMAIN`. + +## Step 7: Configure User Profile Field + +The app needs to map Okta users to GitHub usernames. Determine which Okta user +profile field contains GitHub usernames: + +| Common Fields | Description | +|--------------------|------------------------------------------| +| `login` | Okta username (often email) | +| `email` | User's email address | +| `githubUsername` | Custom field (recommended) | +| `nickName` | Sometimes used for GitHub username | + +### Adding a Custom GitHub Username Field (Recommended) + +1. Go to **Directory** → **Profile Editor** +2. Select **Okta** (or your user profile) +3. Click **Add Attribute** +4. Configure: + - Data type: `string` + - Display name: `GitHub Username` + - Variable name: `githubUsername` + - Description: `User's GitHub username for team sync` +5. Click **Save** + +Then set `APP_OKTA_GITHUB_USER_FIELD=githubUsername`. + +## Step 8: Prepare Okta Groups + +Ensure your Okta groups follow a naming convention that can be matched by sync +rules: + +**Example naming conventions:** + +| Pattern | Example Groups | +|----------------------|----------------------------------------------| +| `github-{team}` | `github-engineering`, `github-platform` | +| `gh-eng-{team}` | `gh-eng-frontend`, `gh-eng-backend` | +| `Team - {name}` | `Team - Platform`, `Team - Security` | + +Groups can be: +- Okta groups (manually managed) +- Groups synced from Active Directory +- Groups from other identity providers + +## Step 9: Configure Environment Variables + +```bash +# Required Okta configuration +APP_OKTA_DOMAIN=your-org.okta.com +APP_OKTA_CLIENT_ID=0oa1abc2def3ghi4j5k6 +APP_OKTA_GITHUB_USER_FIELD=githubUsername + +# Private key (choose one method) +APP_OKTA_PRIVATE_KEY="-----BEGIN PRIVATE KEY----- +... +-----END PRIVATE KEY-----" + +# Or use a file path +APP_OKTA_PRIVATE_KEY_PATH=/path/to/okta-private-key.pem + +# Or use AWS SSM parameter +APP_OKTA_PRIVATE_KEY=arn:aws:ssm:us-east-1:123456789:parameter/github-bot/okta-key +``` + +## Step 10: Configure Sync Rules + +Define how Okta groups map to GitHub teams: + +```bash +APP_OKTA_SYNC_RULES='[ + { + "name": "engineering-teams", + "enabled": true, + "okta_group_pattern": "^github-eng-.*", + "github_team_prefix": "eng-", + "strip_prefix": "github-eng-", + "sync_members": true, + "create_team_if_missing": true + } +]' +``` + +See the main README for complete sync rule documentation. + +## Verification + +Test your Okta configuration: + +```bash +# Test OAuth token retrieval (manual verification) +# The app will automatically authenticate on startup + +# Check app logs for: +# - "okta client initialized" +# - No authentication errors during sync +``` + +Trigger a sync and verify: +1. POST to `/scheduled/okta-sync` endpoint +2. Check logs for groups discovered and teams synced +3. Verify GitHub team memberships match Okta groups + +## Troubleshooting + +### Authentication failed / Invalid client + +- Verify `APP_OKTA_CLIENT_ID` matches the Client ID in Okta +- Check the private key is the one generated for this specific app +- Ensure the key format is correct (PEM with proper headers) + +### No groups found + +- Verify `okta.groups.read` scope is granted +- Check your sync rule patterns match actual group names +- Test the regex pattern against your group names + +### Users not syncing + +- Verify `okta.users.read` scope is granted +- Check `APP_OKTA_GITHUB_USER_FIELD` points to a valid profile field +- Ensure users have the GitHub username field populated +- Only `ACTIVE` users are synced - suspended users are skipped + +### Rate limiting + +Okta has API rate limits. If you hit limits: +- Reduce sync frequency +- The app handles rate limit responses gracefully + +### Permission denied errors + +- API Services apps need explicit scope grants +- Check that scopes were granted (not just requested) +- Super Admin role may be required to grant certain scopes diff --git a/docs/slack-setup.md b/docs/slack-setup.md new file mode 100644 index 0000000..b571495 --- /dev/null +++ b/docs/slack-setup.md @@ -0,0 +1,243 @@ +# Slack Setup + +This guide walks through creating a Slack app for github-ops-app notifications. + +## Prerequisites + +- Slack workspace with admin access (or ability to request app approval) +- Channel where notifications will be posted + +## Step 1: Create Slack App + +### Option A: Use App Manifest (Recommended) + +1. Go to [api.slack.com/apps](https://api.slack.com/apps) +2. Click **Create New App** +3. Select **From an app manifest** +4. Select your workspace +5. Paste the manifest from `assets/slack/manifest.json`: + + ```json + { + "display_information": { + "name": "GitHub Bot", + "description": "GitHub automation bot with Okta sync and PR compliance monitoring", + "background_color": "#24292e" + }, + "features": { + "bot_user": { + "display_name": "GitHub Bot", + "always_online": false + } + }, + "oauth_config": { + "scopes": { + "bot": [ + "channels:join", + "channels:read", + "chat:write.public", + "chat:write" + ] + } + }, + "settings": { + "org_deploy_enabled": false, + "socket_mode_enabled": false, + "token_rotation_enabled": false + } + } + ``` + +6. Click **Create** + +### Option B: Manual Setup + +1. Go to [api.slack.com/apps](https://api.slack.com/apps) +2. Click **Create New App** → **From scratch** +3. Enter app name: `GitHub Bot` +4. Select your workspace +5. Click **Create App** + +Then continue to configure OAuth scopes manually (Step 2). + +## Step 2: Configure OAuth Scopes + +If you used the manifest, scopes are pre-configured. Otherwise: + +1. Go to **OAuth & Permissions** in the sidebar +2. Scroll to **Scopes** → **Bot Token Scopes** +3. Add the following scopes: + + | Scope | Purpose | + |---------------------|----------------------------------------------| + | `chat:write` | Post messages to channels bot is member of | + | `chat:write.public` | Post to public channels without joining | + | `channels:read` | View basic channel info | + | `channels:join` | Join public channels | + +## Step 3: Install to Workspace + +1. Go to **OAuth & Permissions** +2. Click **Install to Workspace** +3. Review permissions and click **Allow** + +If your workspace requires admin approval: +- Submit the app for approval +- Wait for workspace admin to approve +- Return to install after approval + +## Step 4: Get Bot Token + +After installation: + +1. Go to **OAuth & Permissions** +2. Copy the **Bot User OAuth Token** + - Starts with `xoxb-` + - Example: `xoxb-1234567890-...` + +This is your `APP_SLACK_TOKEN`. + +## Step 5: Get Channel ID + +You need the channel ID (not the name) for `APP_SLACK_CHANNEL`. + +### Method 1: From Slack UI + +1. Open Slack in a browser or desktop app +2. Right-click the channel name +3. Select **View channel details** (or **Open channel details**) +4. Scroll to the bottom - Channel ID is shown (e.g., `C01ABC2DEFG`) + +### Method 2: From Channel Link + +1. Right-click the channel name +2. Select **Copy link** +3. The link contains the channel ID: + `https://workspace.slack.com/archives/C01ABC2DEFG` + +### Method 3: Using Slack API + +```bash +curl -H "Authorization: Bearer xoxb-your-token" \ + "https://slack.com/api/conversations.list?types=public_channel,private_channel" \ + | jq '.channels[] | select(.name=="your-channel-name") | .id' +``` + +## Step 6: Add Bot to Private Channels + +For private channels, you must invite the bot: + +1. Open the private channel +2. Type `/invite @GitHub Bot` (or your bot's display name) +3. Or click the channel name → **Integrations** → **Add apps** + +Public channels work without invitation when using `chat:write.public`. + +## Step 7: Configure Environment Variables + +```bash +# Required Slack configuration +APP_SLACK_TOKEN=xoxb-1234567890-... +APP_SLACK_CHANNEL=C01ABC2DEFG +``` + +For AWS deployments, use SSM parameters: + +```bash +APP_SLACK_TOKEN=arn:aws:ssm:us-east-1:123456789:parameter/github-bot/slack-token +``` + +## Step 8: Customize App Appearance (Optional) + +Make notifications more recognizable: + +1. Go to **Basic Information** +2. Under **Display Information**: + - **App name**: `GitHub Bot` (or your preference) + - **Short description**: Brief description of the bot + - **App icon**: Upload a custom icon (recommended: GitHub logo variant) + - **Background color**: `#24292e` (GitHub dark) or your brand color + +## Verification + +Test your Slack configuration: + +```bash +# Test message posting +curl -X POST https://slack.com/api/chat.postMessage \ + -H "Authorization: Bearer xoxb-your-token" \ + -H "Content-Type: application/json" \ + -d '{ + "channel": "C01ABC2DEFG", + "text": "GitHub Bot test message" + }' +``` + +Expected response: +```json +{ + "ok": true, + "channel": "C01ABC2DEFG", + "ts": "1234567890.123456", + "message": { ... } +} +``` + +## Notification Types + +The bot sends these notification types: + +| Event | Description | +|-----------------------|------------------------------------------------| +| PR Compliance Alert | PR merged bypassing branch protection | +| Okta Sync Report | Summary of team membership changes | +| Orphaned Users Alert | Org members not in any synced teams | +| Sync Error | Errors during Okta sync process | + +## Troubleshooting + +### "not_in_channel" error + +- Bot needs to be in the channel to post +- For private channels: `/invite @GitHub Bot` +- For public channels: Ensure `chat:write.public` scope is granted + +### "invalid_auth" error + +- Token may be expired or revoked +- Regenerate token: **OAuth & Permissions** → **Reinstall to Workspace** +- Verify token starts with `xoxb-` + +### "channel_not_found" error + +- Verify channel ID is correct (not channel name) +- Channel IDs start with `C` (public), `G` (private), or `D` (DM) +- Check the channel still exists + +### "missing_scope" error + +- Add the required scope in **OAuth & Permissions** +- Reinstall the app after adding scopes + +### Messages not appearing + +- Check the channel ID is correct +- Verify bot is in the channel (for private channels) +- Check app logs for API response errors +- Test with curl command above to isolate the issue + +### Rate limiting + +Slack has rate limits (typically 1 message per second per channel). The app +handles rate limits gracefully, but if you see delays: +- Notifications are queued and retried +- Consider consolidating notifications for high-volume events + +## Security Considerations + +- **Token security**: Store `APP_SLACK_TOKEN` securely (environment variable, + SSM parameter, or secrets manager) +- **Channel access**: Bot can only post to channels it has access to +- **No incoming webhooks**: This setup uses bot tokens, not incoming webhooks, + providing better security and audit trails +- **Minimal scopes**: Only request scopes actually needed