Expose custom models via SDK

[gandersteele]

Summary

• Adds Python SDK support for model-based custom entities, enabling programmatic management of LLM-in-the-loop NER model training
• Introduces classes for entity management, version/guideline refinement, and model training workflows
• Wraps existing backend endpoints with no server-side changes required

Changes

New files:

• tonic_textual/classes/model_entity.py - Core classes (ModelEntity, ModelEntityVersion, TrainedModel) with full workflow support
• tonic_textual/services/model_entity.py - Service layer for CRUD operations

Modified:

• tonic_textual/redact_api.py - Added convenience methods on TonicTextual client

Features

Entity Management:

• create_model_entity(), get_model_entity(), list_model_entities(), delete_model_entity()

Test Data & Guidelines Refinement:

• upload_test_data() - Upload files with ground truth spans
• version.wait_for_completion() - Wait for LLM annotation
• version.get_metrics() - Get F1/precision/recall scores
• version.get_suggested_guidelines() - Get LLM-suggested improvements
• entity.create_version() - Create new version with refined guidelines

Training:

• upload_training_data() / upload_training_file()
• create_trained_model(version_id) - Create model with specific guidelines
• model.start_training() / model.wait_for_training()
• model.activate() - Set as active model for entity

Example Usage

from tonic_textual.redact_api import TonicTextual

textual = TonicTextual()

# Create entity and upload test data
entity = textual.create_model_entity(
    name='PRODUCT_CODE',
    guidelines='Identify product codes like SKU-12345.'
)
entity.upload_test_data([
    {'text': 'Order SKU-123 shipped.', 'spans': [{'start': 6, 'end': 13}]}
])

# Refine guidelines based on metrics
version = entity.get_latest_version()
version.wait_for_completion()
print(f'F1: {version.get_metrics().f1_score}')

# Train model
entity.upload_training_data([{'text': '...', 'fileName': 'train.txt'}])
model = entity.create_trained_model(version.id)
model.start_training()
model.wait_for_training()
model.activate()

Test plan

• Tested entity CRUD operations against production API
• Verified test data upload with ground truth saves correctly (files show "Reviewed" status)
• Confirmed guidelines refinement loop works (F1 improved from 0.6 → 0.73 with refined guidelines)
• Validated full training workflow: upload → annotate → train → activate

[Copilot]

Pull request overview

This PR introduces SDK support for managing model-based custom entities in Tonic Textual, enabling programmatic workflows for training custom NER models with LLM-assisted annotation and guidelines refinement.

Changes:

• Added core classes for model entities, versions, and trained models with full lifecycle management
• Implemented service layer for CRUD operations on model-based entities
• Extended the TonicTextual client with convenience methods for entity management

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
tonic_textual/classes/model_entity.py	Core domain classes supporting entity creation, test/training data upload, version management, metrics retrieval, and model training workflows
tonic_textual/services/model_entity.py	Service layer providing CRUD operations for model entities with API endpoint integration
tonic_textual/redact_api.py	Client-level convenience methods delegating to the model entity service

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The get() method creates a new requests.Session for a single HTTP call, which provides no benefit over using the session already managed by the client. This pattern is repeated in list() and creates unnecessary overhead. Consider removing the session context manager unless the client requires it, or reuse a client-managed session.

[Copilot]

The list() method makes N+1 HTTP requests: one to fetch all entities, then one per model-based entity to fetch full details. For large numbers of entities, this will be slow. Consider whether the /api/custom-entities endpoint can return full entity data, or if a dedicated endpoint exists for listing model-based entities with complete information.

[Copilot]

The json module is already imported at the module level (line 8), so this local import is redundant and creates naming confusion. Remove this line and use the module-level json import instead.

Suggested change

	import json as json_module
	endpoint = "training/files" if is_training else "test/files"
	# API expects multipart with 'document' (JSON metadata) and 'file' (content)
	document = json_module.dumps({"fileName": file_name})
	endpoint = "training/files" if is_training else "test/files"
	# API expects multipart with 'document' (JSON metadata) and 'file' (content)
	document = json.dumps({"fileName": file_name})

[Copilot]

Use the module-level json.dumps() instead of json_module.dumps() after removing the redundant import on line 505.

Suggested change

	import json as json_module
	endpoint = "training/files" if is_training else "test/files"
	# API expects multipart with 'document' (JSON metadata) and 'file' (content)
	document = json_module.dumps({"fileName": file_name})
	endpoint = "training/files" if is_training else "test/files"
	# API expects multipart with 'document' (JSON metadata) and 'file' (content)
	document = json.dumps({"fileName": file_name})