rocksdb_catchup_task keeps failing during external node sync – statement timeout #156
Description
Path: /infrastructure/nodes/running-a-node
Hi,
I'm trying to run an Abstract node using the external-node setup with Docker, following the official documentation from both Abstract and zkSync:
https://docs.abs.xyz/infrastructure/nodes/introduction
https://docs.zksync.io/zksync-era/tooling/external-node/
https://matter-labs.github.io/zksync-era/core/latest/guides/external-node/
The node initializes and starts syncing, but always ends up crashing with the following error:
javascript
Copier
Modifier
Error: One or more tasks failed: ["Task oneshot_runner failed: Oneshot task state_keeper/rocksdb_catchup_task failed: Failed to catch up RocksDB to Postgres: Failed getting number of logs for L2 block #15615999: Query get_storage_logs_row_count(...) failed: error returned from database: canceling statement due to statement timeout"]
What I’ve already tried:
Increased PostgreSQL statement_timeout to 15min (both via ALTER ROLE and postgresql.conf)
Added index on miniblock_number in the storage_logs table:
sql
Copier
Modifier
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_storage_logs_miniblock ON storage_logs (miniblock_number);
Completely wiped all Docker volumes and restarted the node from scratch.
Let the node run for multiple hours: it progresses through a few blocks, then always crashes on the same type of timeout.
Notes:
PostgreSQL logs show that queries on storage_logs are extremely slow (~20+ seconds).
It seems like RocksDB and Postgres are not staying in sync, despite being freshly initialized.
Questions:
Is there a known issue with this recovery mechanism?
Should we not use the example external-node.yml Docker setup and instead prefer zkstack CLI (e.g. via zkstackup)?
Is there any way to improve database performance during this phase, or skip recovery from genesis?
Thanks in advance for your help and for maintaining this project!