Skip to content

Conversation

@baogorek
Copy link
Collaborator

@baogorek baogorek commented Sep 6, 2025

Summary

  • Expands Python version support to include 3.12 alongside 3.13
  • Ensures compatibility with Google Colab and other Python 3.12 environments
  • Makes the Matching module optional when R dependencies are unavailable
  • Python 3.12 runs a minimal smoke test focused on QRF functionality (what PolicyEngine actually uses)
  • Python 3.13 continues to run the full test suite

Motivation

As described in #110, Python 3.12 support is needed for:

  • Google Colab compatibility: Colab's latest Python version is 3.12
  • Wider adoption: Many organizations still use Python 3.12
  • PolicyEngine compatibility: PolicyEngine only uses the QRF model, not R/Matching features

Changes Made

Configuration Updates

  • Updated pyproject.toml to allow Python >=3.12,<3.14
  • Added py312 to Black formatter targets

CI/CD Strategy

  • Python 3.12: Runs minimal smoke test (test_smoke_qrf.py) that verifies QRF works as PolicyEngine uses it
    • No R/rpy2 dependencies installed
    • No Matching module tests
    • Avoids complex parallel processing issues
  • Python 3.13: Runs full test suite with all features including R/Matching support

Code Changes

  • Made Matching module imports optional with proper fallback handling
  • Fixed autoimpute to handle single quantile predictions returning DataFrame
  • Made rpy2-dependent imports in utils module optional

Why Minimal Testing for Python 3.12?

After analysis, we found that PolicyEngine only uses:

  • The QRF model class
  • Basic fit() and predict() methods
  • No R/Matching functionality
  • No cross-validation or complex features

Running the full test suite for Python 3.12 was causing issues with R/rpy2 integration that aren't relevant to the actual use case. The minimal smoke test approach verifies what matters while avoiding unnecessary complexity.

Test Plan

✅ Python 3.12 smoke test passes (QRF basic functionality)
✅ Python 3.13 full test suite passes
✅ Formatting and linting checks pass
✅ Package installs successfully with Python 3.12

Closes #110

🤖 Generated with Claude Code

baogorek and others added 9 commits September 6, 2025 08:41
- Expand Python version support to include 3.12 alongside 3.13
- Update CI/CD workflows to test against both Python versions
- Make Matching module optional for environments without R dependencies
- Fix single quantile prediction returning DataFrame instead of Dict

Closes #110

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Wrap statmatch_hotdeck import in try/except to handle missing rpy2
- Fixes test failures on Python 3.12 when R dependencies are not installed
- Allows the package to work without R/rpy2 for non-matching functionality
- Set n_jobs=1 when CI environment variable is set
- Also disable parallel processing when using Matching model
- Fixes segfaults in CI when joblib workers try to use R/rpy2
- Python 3.12 now runs only a minimal smoke test for QRF functionality
- Python 3.13 continues to run the full test suite with R/Matching support
- Removes unnecessary R dependencies and parallel processing fixes for Python 3.12
- Focuses Python 3.12 testing on what PolicyEngine actually needs (just QRF)
@baogorek baogorek requested a review from MaxGhenis September 6, 2025 22:21
@MaxGhenis MaxGhenis merged commit 5853e80 into main Sep 7, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for Python 3.12

3 participants