Skip to content

Batch processor: strictness support #1571

@jmacd

Description

@jmacd

In the batch processor we have a bit of logic with the following effect:

When the flush reason is because of size (as opposed to shutdown or timer), AND
when there is a maximum batch size, AND
when there is a minimum batch size, AND
when more than one batch was output, AND
the last batch is less than the minimum size, THEN
withhold the last request, retaining it to accumulate a full batch and resetting the flush timer.

This is a risky operation. It requires a certain degree of correctness in the lower-level otap_df_pdata::otap::batching logic. While it appears that this logic is correct, the consequences of it not being correct are severe, as it will mean acking or nacking incorrect inputs.

Proposing a way to disable this feature, by setting withhold_residual: false (or something like that).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions