[New Feature]: Map Cognito to Airflow Authorization Roles

[LucaCinquini]

Working on a local installation, Ramesh has demonstrated how Cognito groups can be mapped to Airflow groups by overriding the class FabAirflowSecurityManagerOverride in webserver_config.py. We need to port those changes to an SPS deployment onto MCP.

[nikki-t]

Following this PR: #226

I added in the webserver_config.py to airflow/config and then modified the Helm chart values (values.tmpl.yaml) to include the file plus the webserver_config.py file and required Cognito data (e.g., client id, client secret, etc.). I then tried to deploy these changes to unity-nikki-1 and it got stuck on creating the helm release resource.

After destroying and re-applying the Terraform (several times), I ran into the same issue and intermittent timeouts (context deadline exceeded). So I destroyed the EKS, Karpenter, and Airflow deployments and am not having issues getting things back to a normal, known state.

I also ran the following Helm commands to try to debug the helm chart:

# Render chart from template and debug
helm template airflow apache-airflow/airflow --values values.tmpl.yaml --debug 

# Complete a dry run installation and debug
helm install airflow apache-airflow/airflow --dry-run --debug

I didn't see anything obviously wrong with the Helm chart. So I will keep working at isolating the issue.

[nikki-t]

I needed to update the TFVARS file for the recent Airflow component changes we made. I was able to add the webserver_config.py file to the unity-nikki-1 deployment.

Branch: https://github.com/unity-sds/unity-sps/tree/376-airflow-cognito

I was able to log in with Cognito the first time. Then I clicked on the Sign in with Cognito link and it returns a 404 error message. I have sent a message to Ramesh to try to troubleshoot.

[LucaCinquini]

We decided to not pursue this ticket any further at this time because Cognito-Airflow integration is not on the critical path of MDPS-MAAP integration.