fix(ibis): handle JSON result for MySQL

[goldmedal]

MySQL uses JSON to present ARRAY and STRUCT values. However, ibis doesn't handle JSON type correctly. To show the result, we cast the JSON value to text for display purposes.

Summary by CodeRabbit

• New Features

  • Added MySQL database connector support with automatic type conversion for JSON, UUID, and Decimal columns
  • Implemented JSON column casting for improved data compatibility

• Tests

  • Added comprehensive integration tests for JSON query functionality on MySQL
  • Expanded test fixtures for MySQL connector configuration and schema setup

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Introduces MySqlConnector class to handle MySQL-specific type conversions for unsupported PyArrow types (Decimal, UUID, JSON). Adds test infrastructure with MySQL container fixtures and JSON test data, refactors function list path configuration into shared fixtures, and implements integration tests for JSON query handling via the MySQL connector.

Changes

Cohort / File(s)	Summary
MySQL Connector Implementation ibis-server/app/model/connector.py	Introduces MySqlConnector subclass of IbisConnector with _handle_pyarrow_unsupported_type() method to handle Decimal (rounding), UUID (cast to string), and JSON (cast to string) type conversions, and _cast_json_columns() helper method. Wires DataSource.mysql to instantiate this connector in the factory.
Test Fixtures & Configuration ibis-server/tests/routers/v3/connector/mysql/conftest.py	Adds MySQL test container initialization via SQLAlchemy to create json_test schema with JSON columns and sample data. Introduces autouse fixtures set_remote_function_list_path and set_remote_white_function_list_path for path configuration management, with module-level path variables pointing to test resources.
Test Updates ibis-server/tests/routers/v3/connector/mysql/test_functions.py	Refactors to import function_list_path and white_function_list_path from conftest instead of defining locally. Removes autouse fixtures and updates test_function_list to explicitly set paths before making requests.
JSON Query Integration Test ibis-server/tests/routers/v3/connector/mysql/test_query.py	Adds new test_json_query() test with manifest_str fixture to verify MySQL connector handles JSON data correctly, validating correct string conversion and data types in query responses.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

• Type conversion logic for MySQL-specific types (Decimal, UUID, JSON) requires verification of correctness
• Test fixture setup with SQLAlchemy initialization and sample data insertion
• Multiple file coordination across connector implementation and test refactoring
• Integration test assertions on JSON data handling and dtype conversions

Suggested reviewers

• wwwy3y3
• douenergy

Poem

🐰 A MySQL warren emerges from the code,
JSON columns dancing down the database road,
Decimals rounded, UUIDs recast,
Tests ensure each query holds fast! 🔗✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 7.14% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix(ibis): handle JSON result for MySQL' directly and clearly summarizes the main change: handling JSON results from MySQL. It accurately reflects the primary objective of the PR.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (3)

ibis-server/app/model/connector.py (1)

323-353: JSON casting behavior for MySQL looks correct; consider delegating to base handler

The override mirrors the base IbisConnector behavior for Decimal and UUID and adds dt.JSON casting to "string", which should fix pyarrow incompatibility for MySQL JSON columns while preserving existing numeric/UUID handling.

Two small follow‑ups to consider:

• You could call super()._handle_pyarrow_unsupported_type first and then only loop for dt.JSON to avoid duplicating the Decimal/UUID logic and keep MySQL aligned with any future changes to the base handler.
• It may be worth verifying (via a quick schema inspection in tests) that MySQL array/struct values that come back as JSON are indeed typed as dt.JSON by ibis; if some surface as other types (e.g., dt.Array / dt.Struct), they would currently bypass this casting.

Example refactor sketch:

def _handle_pyarrow_unsupported_type(self, ibis_table: Table, **kwargs) -> Table:
    result_table = super()._handle_pyarrow_unsupported_type(ibis_table, **kwargs)
    for name, dtype in result_table.schema().items():
        if isinstance(dtype, dt.JSON):
            result_table = self._cast_json_columns(result_table=result_table, col_name=name)
    return result_table

ibis-server/tests/routers/v3/connector/mysql/test_query.py (2)

9-41: Manifest definition is clear; fixture can be synchronous

The manifest mirrors the json_test schema and MySQL data source cleanly, and the base64+orjson encoding matches existing patterns. Since manifest_str doesn’t perform any async I/O, you can simplify it to a regular synchronous fixture:

@pytest.fixture(scope="module")
def manifest_str():
    return base64.b64encode(orjson.dumps(manifest)).decode("utf-8")

This avoids depending on async‑fixture behavior for something that’s purely in‑memory.

---

44-67: JSON query test correctly validates cast‑to‑string; watch out for key‑order brittleness

The test does a good job of:

• Disabling fallback so it hits the ibis/MySqlConnector path.
• Verifying both JSON columns are surfaced as "string" dtypes.
• Asserting representative row values from json_test.

The strict equality check on the full JSON string (including key order) for object_col could be a bit brittle if MySQL/ibis/serialization ever changes ordering; parsing the value as JSON and asserting on the resulting dict would make the test more robust while still confirming that JSON is returned as a string.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5949b46 and 8fba3c2.

📒 Files selected for processing (4)

• ibis-server/app/model/connector.py (2 hunks)
• ibis-server/tests/routers/v3/connector/mysql/conftest.py (2 hunks)
• ibis-server/tests/routers/v3/connector/mysql/test_functions.py (1 hunks)
• ibis-server/tests/routers/v3/connector/mysql/test_query.py (1 hunks)

🧰 Additional context used

🧬 Code graph analysis (1)

ibis-server/app/model/connector.py (1)

ibis-server/app/model/data_source.py (1)

• DataSource (62-221)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)

• GitHub Check: ci
• GitHub Check: ci

🔇 Additional comments (4)

ibis-server/tests/routers/v3/connector/mysql/conftest.py (2)

4-52: MySQL container + JSON seed setup looks solid

Using MySqlContainer plus a SQLAlchemy engine to create and populate json_test with JSON columns is clear and scoped to the session; the explicit commit() ensures data is ready for the JSON tests. This is a good place to centralize MySQL JSON test data.

---

57-75: Centralizing remote function list configuration in mysql conftest

Defining function_list_path / white_function_list_path here and wiring them via autouse fixtures keeps MySQL tests consistent and avoids duplication. The fixtures correctly set and then reset the config on each test, so global state shouldn’t leak across tests.

ibis-server/app/model/connector.py (1)

101-102: MySQL data source correctly routed to MySqlConnector

Adding the explicit DataSource.mysql branch ensures MySQL goes through the new connector and its custom type handling instead of the generic IbisConnector, which is the right integration point.

ibis-server/tests/routers/v3/connector/mysql/test_functions.py (1)

7-12: Reusing function list paths from mysql conftest is a good consolidation

Importing base_url, function_list_path, and white_function_list_path from the shared mysql conftest keeps this test aligned with the centralized configuration and avoids hard‑coding paths in multiple places. The explicit set/reset logic in test_function_list still makes the intended behavior clear.