Modernize

.github/workflows/deploy.yml

@@ -0,0 +1,111 @@
+name: Build and Deploy
+
+on:
+  push:
+    branches: ['**']
+  pull_request:
+    branches: [master]
+
+jobs:
+  build-and-deploy:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Setup R
+        uses: r-lib/actions/setup-r@v2
+        with:
+          r-version: '4.3'
+
+      - name: Install R packages
+        run: |
+          R -e "install.packages(c('ggplot2', 'mgcv'), repos='https://cran.rstudio.com/')"
+
+      - name: Verify R installation
+        run: |
+          echo "R_HOME=$R_HOME"
+          R --version
+          R -e "library(ggplot2); library(mgcv); cat('R packages loaded successfully\n')"
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+
+      - name: Install uv
+        run: |
+          curl -LsSf https://astral.sh/uv/install.sh | sh
+          echo "$HOME/.cargo/bin" >> $GITHUB_PATH
+
+      - name: Install Python dependencies
+        run: |
+          uv pip install -r requirements.txt --system
+
+      - name: Verify Python packages
+        run: |
+          python -c "import pandas, plotly, seaborn, plotnine, altair, rpy2; print('Python packages imported successfully')"
+
+      - name: Setup Firefox for Selenium
+        uses: browser-actions/setup-firefox@v1
+
+      - name: Install geckodriver
+        run: |
+          wget -q https://github.com/mozilla/geckodriver/releases/download/v0.34.0/geckodriver-v0.34.0-linux64.tar.gz
+          tar -xzf geckodriver-v0.34.0-linux64.tar.gz
+          sudo mv geckodriver /usr/local/bin/
+          geckodriver --version
+
+      - name: Setup Xvfb (virtual display)
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y xvfb
+
+      - name: Run tests
+        run: |
+          xvfb-run -a python -m pytest tests/ -v
+        env:
+          R_HOME: /opt/R/4.3.3/lib/R
+
+      - name: Build site
+        run: |
+          xvfb-run -a make travis
+        env:
+          R_HOME: /opt/R/4.3.3/lib/R
+
+      - name: Deploy to Netlify (Production)
+        if: github.ref == 'refs/heads/master' && github.event_name == 'push'
+        uses: nwtgck/actions-netlify@v3
+        with:
+          publish-dir: ./web
+          production-deploy: true
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          deploy-message: "Deploy from GitHub Actions - ${{ github.event.head_commit.message }}"
+          enable-pull-request-comment: false
+          enable-commit-comment: false
+        env:
+          NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
+          NETLIFY_SITE_ID: ${{ secrets.NETLIFY_SITE_ID }}
+
+      - name: Deploy to Netlify (Preview)
+        if: github.ref != 'refs/heads/master'
+        uses: nwtgck/actions-netlify@v3
+        with:
+          publish-dir: ./web
+          production-deploy: false
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          deploy-message: "Preview deploy from GitHub Actions"
+          enable-pull-request-comment: true
+          enable-commit-comment: false
+        env:
+          NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
+          NETLIFY_SITE_ID: ${{ secrets.NETLIFY_SITE_ID }}
+
+      - name: Upload build artifacts
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: site-build
+          path: web/
+          retention-days: 7

.travis/authenticate_plotly.py

@@ -1,3 +1,15 @@
-import plotly
-import os
-plotly.tools.set_credentials_file(username=os.environ.get('PLOTLY_USER'), api_key=os.environ.get('PLOTLY_API_KEY'))
+# DEPRECATED: This file is no longer used.
+#
+# With plotly v5+, credentials are not required for local rendering.
+# Plotly authentication is only needed if uploading to Chart Studio.
+# This project renders plots locally to PNG using Kaleido, so no
+# authentication is necessary.
+#
+# Historical code (plotly v4):
+# import plotly
+# import os
+# plotly.tools.set_credentials_file(username=os.environ.get('PLOTLY_USER'), api_key=os.environ.get('PLOTLY_API_KEY'))
+
+import sys
+print("Warning: plotly authentication is no longer needed with plotly v5+", file=sys.stderr)
+print("This script is deprecated and can be removed.", file=sys.stderr)

CLAUDE.md

@@ -0,0 +1,186 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+pythonplot.com is a static website that provides a visual comparison of different Python plotting libraries (pandas, matplotlib, seaborn, plotnine, plotly, altair) and R's ggplot2 for exploratory data analysis. It serves as a "Rosetta Stone" showing how to create the same plots across different libraries.
+
+## Architecture
+
+The site is generated from a Jupyter notebook (Examples.ipynb) that contains tagged code cells. The build process:
+
+1. Executes the notebook using `jupyter nbconvert`
+2. Extracts tagged cells using `render.py`
+3. Generates static HTML using Jinja2 templates
+4. Outputs PNG images and an HTML file to the `web/` directory
+
+### Key Components
+
+- **Examples.ipynb**: Source notebook containing plot examples for each library
+- **render.py**: Core build script that:
+  - Extracts cells tagged with metadata (ex, name:*, package:*)
+  - Extracts base64 PNG images from cell outputs and saves them to web/img/plots/
+  - Parses cell source code and optional markdown comments
+  - Renders the final HTML using Jinja2
+- **templates/t_index.html**: Jinja2 template for the website
+- **web/**: Output directory for generated site
+- **INTRO.md**: Markdown content for the site introduction
+
+### Cell Tagging System
+
+Notebook cells must be tagged with metadata for the render script to process them:
+
+```
+ex                        # Marks cell as an example
+name:scatter-plot         # Maps to names dict in render.py
+package:seaborn           # Maps to packages dict in render.py
+```
+
+Valid plot names and packages are defined in dictionaries at the top of render.py (lines 19-43).
+
+## Development Commands
+
+### Setup Environment
+
+```bash
+make dev_environment    # Installs Python and R dependencies using uv
+make setup             # Alternative command (same as dev_environment)
+```
+
+**Prerequisites:**
+- Python 3.11+ installed on system
+- R 4.0+ installed separately (via Homebrew, apt, or CRAN)
+- uv package manager (auto-installed by make targets if missing)
+
+**Manual Setup:**
+```bash
+# Install uv
+curl -LsSf https://astral.sh/uv/install.sh | sh
+
+# Install Python dependencies
+uv pip install -r requirements.txt
+
+# Install R packages (ggplot2, mgcv)
+./setup_r.sh
+```
+
+### Build Site
+
+```bash
+make qrender           # Quick render from Examples.ipynb without executing
+make render            # Full build: execute notebook, then render (creates timestamped .ipynb)
+make                   # Full build + S3 upload
+```
+
+The render process:
+1. `make render` runs the notebook with `jupyter nbconvert --execute`
+2. Creates a git-commit-stamped copy (Examples.<hash>.ipynb)
+3. Runs `python render.py` to extract cells and generate web/index.html
+4. PNG images are extracted and saved to web/img/plots/ with MD5-based filenames
+
+### Testing
+
+```bash
+make test              # Run pytest tests
+python -m pytest tests/
+```
+
+The test suite (tests/test_plots.py) validates that Examples.ipynb contains all expected plot/package combinations defined in the defined_plots dictionary.
+
+### Local Development
+
+After rendering, serve locally:
+
+```bash
+cd web && python -m http.server
+```
+
+### Image Rendering
+
+All plots are rendered to static PNG images:
+- **Plotly**: Uses Kaleido for local rendering (no authentication required with v5+)
+- **Altair**: Uses native rendering or selenium/geckodriver
+- **R/ggplot2**: Uses rpy2 to interface with system R installation
+
+**Note**: Plotly authentication is no longer needed with plotly v5+. The old authentication code in `.travis/authenticate_plotly.py` is deprecated.
+
+## Adding New Plots
+
+1. Add the plot name to the `names` dictionary in render.py
+2. Add a new cell in Examples.ipynb with code that produces a PNG output
+3. Tag the cell with: `ex`, `name:<plot-name>`, `package:<library-name>`
+4. Keep code lines under ~46 characters to avoid horizontal scrolling in the UI
+5. Optionally add a markdown comment in triple quotes on the first line
+6. Update tests/test_plots.py to include the new plot in defined_plots
+7. Run `make qrender` to test (or `make render` for full rebuild)
+
+## Technical Constraints
+
+- Plot code must generate PNG output in the notebook cell
+- For plotly, images must be generated via their server (requires credentials)
+- R code cells must start with `%%R` magic command
+- Altair cells must start with `%%altair` magic command
+- Code lines should wrap to ~46 characters for proper display
+- All image paths are MD5 hashes of the base64-encoded PNG data
+
+## Dependencies
+
+### Python Environment
+- **Python**: 3.11+ (specified in runtime.txt and pyproject.toml)
+- **Package Manager**: uv (modern, fast alternative to pip)
+- **Jupyter**: For notebook execution
+
+### Plotting Libraries (all latest versions)
+- pandas 2.0+
+- matplotlib 3.7+
+- seaborn 0.13+
+- plotnine 0.13+
+- plotly 5.24+ (with Kaleido for image export)
+- altair 5.0+
+- statsmodels 0.14+
+
+### R Environment
+- **R**: 4.0+ (system installation required)
+- **R Packages**: ggplot2, mgcv
+- **Python-R Bridge**: rpy2 3.5+
+
+### Other Key Dependencies
+- Jinja2 with jinja2-highlight for templating
+- selenium 4.15+ with geckodriver for browser automation
+- markdown for text processing
+
+**Configuration Files:**
+- `pyproject.toml`: Modern Python project metadata and dependencies
+- `requirements.txt`: Pin-free dependency list
+- `setup_r.sh`: R package installation script
+
+## CI/CD
+
+The project uses **GitHub Actions** for continuous integration and deployment (migrated from Travis CI).
+
+### Workflow (.github/workflows/deploy.yml)
+
+**On Every Push:**
+1. Setup R 4.3 using r-lib/setup-r action
+2. Install R packages (ggplot2, mgcv)
+3. Setup Python 3.11
+4. Install uv and Python dependencies
+5. Setup Firefox and geckodriver for selenium
+6. Run pytest tests with xvfb (virtual display)
+7. Execute notebook and render site
+8. Deploy to Netlify:
+   - **master branch**: Production deployment
+   - **Other branches**: Preview deployments
+
+**Required Secrets:**
+- `NETLIFY_AUTH_TOKEN`: Netlify authentication token
+- `NETLIFY_SITE_ID`: Netlify site identifier
+
+**Environment Variables:**
+- `R_HOME`: Set to R installation path (e.g., `/opt/R/4.3.3/lib/R` on GitHub runners)
+
+### Legacy CI Files
+- `.travis.yml`: Old Travis CI config (deprecated)
+- `.travis/`: Old CI scripts (mostly deprecated)
+- `.travis/authenticate_plotly.py`: No longer needed with plotly v5+