Skip to content

Conversation

@tpellissier-msft
Copy link
Collaborator

This pull request refactors the Dataverse SDK's table and column management to consistently use logical names with publisher prefixes (e.g., new_sampleitem, new_code) instead of friendly or schema names. The changes affect both documentation and code, enforcing stricter naming conventions and improving case-insensitive lookups. The update also simplifies metadata caching and usage throughout the SDK.

API and Documentation Updates

  • All table and column operations (create_table, create_columns, get_table_info, list_tables, delete_table, delete_columns) now require logical names with publisher prefixes, and documentation/examples have been updated to reflect this. Friendly names and schema names are no longer accepted. [1] [2] [3] [4]

Codebase Refactoring

  • The SDK's internal logic now uses a unified metadata cache keyed by logical name, and introduces a TableMetadata TypedDict for efficient lookups. [1] [2]
  • Added a helper method _normalize_logical_name to ensure all logical names are treated case-insensitively, preventing errors from inconsistent casing.

Examples and Usage

  • All examples (README.md, quickstart.py, file_upload.py) now use logical names with publisher prefixes for table and column operations, demonstrating the new conventions. [1] [2] [3] [4]

Behavioral Changes

  • list_tables() now returns a list of logical names instead of dictionaries with schema/friendly names.
  • All metadata lookups and mutations are case-insensitive, reducing user error and improving robustness. [1] [2]

Error Handling

  • Improved error messages and exceptions for invalid names or missing publisher prefixes, with more precise documentation about failure modes.

@tpellissier-msft tpellissier-msft marked this pull request as ready for review November 10, 2025 15:28
Copilot AI review requested due to automatic review settings November 10, 2025 15:28
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the Dataverse SDK to enforce consistent naming conventions by normalizing logical names to lowercase and implementing case-insensitive lookups throughout the codebase. The changes introduce a unified entity metadata cache, consolidate helper functions into a shared test utilities module, and update all documentation and examples to reflect the new logical-name-first approach.

  • Introduced _normalize_logical_name() for case-insensitive logical name handling
  • Consolidated entity metadata caching into a unified TableMetadata TypedDict structure
  • Updated all table/column operations to accept logical names with publisher prefixes
  • Refactored test helpers into shared tests/unit/test_helpers.py module

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
tests/unit/test_helpers.py New shared test utilities module providing mock authentication, HTTP clients, and reusable test data
tests/unit/data/test_naming_normalization.py Comprehensive test suite for logical name normalization and case-insensitive caching
tests/unit/data/test_logical_crud.py Refactored to use shared test helpers instead of duplicated mock classes
tests/unit/data/test_enum_optionset_payload.py Updated to import DummyAuth and DummyConfig from shared test_helpers
tests/unit/core/test_http_errors.py Updated to import DummyAuth and DummyHTTPClient from shared test_helpers
src/dataverse_sdk/data/odata.py Core implementation of logical name normalization, unified metadata cache, and case-insensitive operations
src/dataverse_sdk/client.py Updated public API documentation to reflect logical-name-first naming convention
examples/basic/quickstart.py Updated to use lowercase logical names with publisher prefix throughout
examples/advanced/file_upload.py Updated table creation to match new naming conventions
README.md Updated API reference table to document logical-name-first approach

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tpellissier-msft and others added 11 commits November 10, 2025 07:54
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

# Escape single quotes in logical name
logical_escaped = self._escape_odata_quotes(logical_name)
params = {
"$select": "MetadataId,LogicalName,SchemaName,EntitySetName",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we also store metadataid in cache? in general just anything that seems commonly used

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and if we do that we can add a cache check inside this function so we dont need to make a call to get these values everytime

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a follow up item here would be good to get our caching stuff better-organized. Some quick thoughts:

  • I think caching metadataId is a good idea
  • In some cases, getting metadata from the cache makes a lot of sense. But there are some cases where we call this method with consistency-strong because we want to fetch newly created columns or other recently-updated metadata. In those cases we don't want to fetch from cache (not our own SDK cache, and not even the server side cache). So it might be best to have 2 helpers, one for getting metadata info that we don't expect to change (like metadataid, schemaname, etc) and another for force-checking the server metadata and fetching column information etc.
  • If we're making those changes, that's a good opportunity to expose a user-facing method to flush our internal metadata caches (just re-init the dictionaries, probably) in case any of the 'static' metadata like schemaname etc. actually does change.

Probably this is best handled in a separate item.

@tpellissier-msft tpellissier-msft changed the title Use case-insenstivie prefixed logicalName in all operations Use case-insenstive prefixed logicalName in all operations Nov 10, 2025
@tpellissier-msft tpellissier-msft changed the title Use case-insenstive prefixed logicalName in all operations Use case-insensitive prefixed logicalName in all operations Nov 10, 2025
@suyask-msft
Copy link
Collaborator

This is a reminder to update ReadMe or other doc files to account for changes in this PR to various functions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants