Add dataset serialization to support distributed eval runs

[danielnyari-seon]

Important

Add items_to_dict() in DatasetClient for dataset item serialization, with tests for distributed processing support.

• Behavior:
  • Add items_to_dict() method in DatasetClient to serialize dataset items into dictionaries for distributed processing.
  • Example usage provided for distributed systems like AWS Step Functions and Prefect.
• Tests:
  • Add test_items_to_dict_basic() in test_datasets.py to verify basic serialization structure.
  • Add test_items_to_dict_multiple_items() to check handling of multiple items.
  • Add test_items_to_dict_reconstruct_item() to ensure items can be reconstructed from dicts.
  • Add test_items_to_dict_with_run() to verify reconstructed items work with run() context manager.

^{This description was created by for de4c08b. You can customize this summary. It will automatically update as commits are pushed.}

Disclaimer: Experimental PR review

Greptile Overview

Greptile Summary

This PR adds dataset serialization capabilities to the langfuse-python client to support distributed evaluation workflows. The main changes include a new items_to_dict() method in DatasetClient that converts dataset items into serializable dictionaries, and modifications to DatasetItemClient to store a reference to the original DatasetItem Pydantic model. This enables dataset items to be passed between distributed processing systems like AWS Step Functions, Azure Functions, or Prefect tasks where objects need to be JSON-serialized and reconstructed in separate processes.

The implementation stores the original DatasetItem model in a new dataset_item attribute on DatasetItemClient and provides a straightforward serialization method that calls .dict() on each item. The DatasetClient constructor also now accepts an optional langfuse parameter for more flexible client management in distributed scenarios. This change integrates seamlessly with the existing codebase architecture by leveraging the existing Pydantic models and maintaining backward compatibility.

Important Files Changed

Filename	Score	Overview
langfuse/_client/datasets.py	4/5	Adds items_to_dict() method and stores dataset_item reference for serialization support
tests/test_datasets.py	5/5	Comprehensive test coverage for the new serialization functionality with proper validation

Confidence score: 4/5

• This PR is safe to merge with minimal risk as it adds new functionality without breaking existing APIs
• Score reflects well-structured implementation with comprehensive test coverage, but slight concern about imports within test functions rather than at module level
• Pay close attention to the import statements in test functions which violate typical Python import conventions

Sequence Diagram

sequenceDiagram
    participant User
    participant DatasetClient
    participant DatasetItem
    participant DistributedWorker
    participant DatasetItemClient
    participant LangfuseSpan
    participant LangfuseAPI

    User->>DatasetClient: "items_to_dict()"
    DatasetClient->>DatasetItem: "dict()"
    DatasetItem-->>DatasetClient: "serialized_dict"
    DatasetClient-->>User: "List[Dict[str, Any]]"
    
    User->>DistributedWorker: "send serialized items"
    DistributedWorker->>DatasetItem: "DatasetItem(**item_dict)"
    DatasetItem-->>DistributedWorker: "reconstructed_item"
    DistributedWorker->>DatasetItemClient: "DatasetItemClient(item, langfuse)"
    DatasetItemClient-->>DistributedWorker: "client_instance"
    
    DistributedWorker->>DatasetItemClient: "run(run_name=name)"
    DatasetItemClient->>LangfuseSpan: "start_as_current_span()"
    LangfuseSpan-->>DatasetItemClient: "span_context"
    DatasetItemClient->>LangfuseAPI: "create dataset_run_item"
    LangfuseAPI-->>DatasetItemClient: "run_item_created"
    DatasetItemClient-->>DistributedWorker: "yield span"
    
    DistributedWorker->>LangfuseSpan: "update_trace()"
    DistributedWorker->>LangfuseSpan: "score_trace()"
    DistributedWorker->>DatasetItemClient: "flush()"
    DatasetItemClient->>LangfuseAPI: "flush traces and scores"

Loading

Context used:

• Rule from dashboard - Move imports to the top of the module instead of placing them within functions or methods. (source)

[greptile-apps]

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[CLAassistant]

All committers have signed the CLA.