Real time Update Patterns

Real-time Update Patterns

Table of Contents

1. Introduction
2. Bar Updates vs Quote Updates
3. TVBar Structure and Timestamp Precision
4. TVDatafeedQuoteValues Structure
5. Real-time Update Handlers
6. Data Transformation Patterns
7. Performance Considerations
8. Out-of-Order Message Handling

Introduction

This document details the real-time update patterns in PyTradingView, focusing on the distinction between bar updates (candlestick data) and quote updates (tick data). It covers the proper formatting of TVBar objects with UTC millisecond precision, handling partial bars during active trading periods, and the structure of TVDatafeedQuoteValues. The document also addresses timing considerations for update handlers, data transformation from various sources, performance optimization for high-frequency updates, and strategies for maintaining data consistency.

Bar Updates vs Quote Updates

PyTradingView distinguishes between two primary types of real-time updates: bar updates and quote updates. Bar updates represent candlestick data (OHLCV) at specific time intervals, while quote updates provide tick-level market data including bid, ask, last price, and other real-time metrics.

Bar updates are delivered through the subscribeBars method and processed by the onTick callback, which receives TVBar objects. These updates are typically used for charting candlestick patterns and technical analysis. Quote updates are delivered through the subscribeQuotes method and processed by the onRealtimeCallback, which receives TVQuoteData objects. These updates are essential for trading functionality, order execution, and market depth analysis.

TVBar Structure and Timestamp Precision

The TVBar class represents a single candlestick data point with OHLCV values. It requires precise timestamp formatting in UTC milliseconds since the Unix epoch. For daily, weekly, and monthly bars, the timestamp should represent the trading day at 00:00 UTC, not the session start time.

The TVBar structure includes:

• time: Timestamp in UTC milliseconds
• open: Opening price for the period
• high: Highest price during the period
• low: Lowest price during the period
• close: Closing price for the period
• volume: Trading volume (optional)

During active trading periods, partial bars are updated in real-time as new trades occur. The system must handle these partial bars correctly by updating the high, low, and close values while the bar is still forming. When a new bar begins, a complete TVBar object should be pushed to the onTick callback.

classDiagram
class TVBar {
+int time
+float open
+float high
+float low
+float close
+Optional[float] volume
+to_dict() dict
}

Loading

**Diagram sources **

TVDatafeedQuoteValues Structure

The TVDatafeedQuoteValues class contains the current market data for a symbol, used by various TradingView widgets including the Order Ticket, Legend, Watchlist, Details, News, and Depth of Market. All properties are optional but should be populated to support full trading functionality.

Key fields include:

• ch: Price change from the day's open
• chp: Price change percentage
• lp: Last price (most recent trade)
• ask: Current ask price
• bid: Current bid price
• spread: Difference between ask and bid
• open_price: Today's opening price
• high_price: Today's high price
• low_price: Today's low price
• prev_close_price: Previous session's closing price
• volume: Today's trading volume

The TVQuoteData wrapper class contains the status ("ok" or "error"), symbol name, and the quote values, providing a standardized response structure for quote data requests.

classDiagram
class TVDatafeedQuoteValues {
+Optional[float] ch
+Optional[float] chp
+Optional[str] short_name
+Optional[str] exchange
+Optional[str] description
+Optional[float] lp
+Optional[float] ask
+Optional[float] bid
+Optional[float] spread
+Optional[float] open_price
+Optional[float] high_price
+Optional[float] low_price
+Optional[float] prev_close_price
+Optional[float] volume
+Optional[str] original_name
}
class TVQuoteData {
+QuoteStatus s
+str n
+Union[TVDatafeedQuoteValues, dict] v
}
TVQuoteData --> TVDatafeedQuoteValues : "contains"

Loading

**Diagram sources **

Real-time Update Handlers

Real-time updates are processed through callback handlers that are registered when subscribing to data streams. The onTick callback handles bar updates, receiving complete TVBar objects whenever a new bar is available or an existing bar is updated. The onRealtimeCallback handles quote updates, receiving arrays of TVQuoteData objects containing the latest market data.

Timing considerations include:

• Bar updates should be pushed immediately when a new bar is complete or when significant price movements occur in partial bars
• Quote updates should be pushed at appropriate frequencies based on market activity, balancing responsiveness with performance
• Both handlers should process updates efficiently to avoid blocking the event loop
• Error handling should be implemented to manage connectivity issues and data inconsistencies

The callback types are defined in TVCallbacks.py as TVSubscribeBarsCallback (Callable[['TVBar'], None]) and TVQuotesCallback (CallableList['TVQuoteData', None]), ensuring type safety in the implementation.

Data Transformation Patterns

Data from various sources must be transformed into the required TVBar and TVQuoteData formats before being pushed to the update handlers. Common source formats include WebSocket streams, database change data capture (CDC), and message queues.

For WebSocket streams:

• Parse incoming messages to extract price and volume data
• Aggregate tick data into OHLCV bars based on the requested resolution
• Format timestamps in UTC milliseconds
• Construct TVBar objects and push to onTick callback

For database CDC:

• Capture changes to trading data tables
• Transform row-level changes into incremental updates
• Handle both full refreshes and delta updates
• Convert to TVBar or TVQuoteData format as appropriate

For message queues:

• Consume messages from topics/queues
• Deserialize and validate message content
• Transform into standardized TVDatafeed formats
• Push to appropriate callbacks based on message type

The transformation process should include data validation, error handling, and logging to ensure data integrity.

Performance Considerations

High-frequency updates require careful performance optimization to maintain system responsiveness and prevent resource exhaustion. Key considerations include message batching, rate limiting, and memory management.

Message batching can significantly improve performance by reducing the number of callback invocations. Instead of pushing each update individually, accumulate updates and push them in batches at regular intervals or when a batch size threshold is reached.

Rate limiting prevents overwhelming the client with updates by capping the frequency of updates based on resolution and market conditions. For example, limit 1-second bar updates to 10 per second even if more ticks are received.

Memory management strategies include:

• Reusing TVBar and TVQuoteData objects when possible
• Implementing object pooling for frequently created objects
• Properly cleaning up subscriptions to prevent memory leaks
• Monitoring memory usage and implementing garbage collection strategies

The system should also implement backpressure mechanisms to handle situations where data production exceeds consumption capacity.

flowchart TD
A[Data Source] --> B{Rate Limiter}
B --> |Within Limits| C[Message Batch]
B --> |Exceeded| D[Drop/Queue]
C --> E{Batch Size/Time}
E --> |Threshold Met| F[Transform to TVBar/TVQuoteData]
E --> |Threshold Not Met| C
F --> G[Push to Callback]
G --> H[Client Update]

Loading

**Diagram sources **

Out-of-Order Message Handling

Maintaining data consistency in the presence of out-of-order messages is critical for accurate charting and trading decisions. Network latency and distributed system architecture can result in messages arriving out of chronological order.

Strategies for handling out-of-order messages include:

• Timestamp validation to detect and correct out-of-order sequences
• Buffering recent messages to allow for reordering
• Implementing sequence numbers when available from the data source
• Using idempotent updates that can be safely applied multiple times
• Maintaining state to detect and resolve conflicts

When an out-of-order bar update is received, the system should:

1. Validate the timestamp against the current bar timeline
2. If the update belongs to a previous bar, update the historical data
3. If the update belongs to the current bar, update the partial bar values
4. If the update belongs to a future bar, buffer it until the timeline advances

For quote updates, out-of-order messages should be processed based on their timestamps, ensuring that the most recent data is always displayed, regardless of arrival order.