Skip to content

Conversation

@refcell
Copy link
Contributor

@refcell refcell commented Jan 8, 2026

Description

Increases the broadcast channel buffer from 20 to 1000 to prevent RecvError::Lagged errors when flashblock processing falls behind.

Problem

Nodes with flashblocks enabled show "WAL containing too many blocks" warnings every ~6 minutes, with flashblocks-canon lagging ~400 blocks behind (#316). The 20-message buffer overflows when processing can't keep up with incoming flashblocks.

Fix

Increase buffer to 1000 messages (~200s of headroom), preventing subscriber lag during temporary processing delays.

Note

Makes progress on #316.

@refcell refcell self-assigned this Jan 8, 2026
@cb-heimdall
Copy link
Collaborator

cb-heimdall commented Jan 8, 2026

🟡 Heimdall Review Status

Requirement Status More Info
Reviews 🟡 0/1
Denominator calculation
Show calculation
1 if user is bot 0
1 if user is external 0
2 if repo is sensitive 0
From .codeflow.yml 1
Additional review requirements
Show calculation
Max 0
0
From CODEOWNERS 0
Global minimum 0
Max 1
1
1 if commit is unverified 1
Sum 2

@haardikk21
Copy link
Collaborator

How do we know it is related to the buffer size? I don't understand how the math adds up here. Our metrics for processing duration dont have spikes which would delay processing significantly enough for a buffer of even size 20 to be full.

@refcell
Copy link
Contributor Author

refcell commented Jan 8, 2026

How do we know it is related to the buffer size? I don't understand how the math adds up here. Our metrics for processing duration dont have spikes which would delay processing significantly enough for a buffer of even size 20 to be full.

You're right -- I overlooked where this was being sent. Closing

@refcell refcell closed this Jan 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants