Dataflow Locks #162
Description
- Dataflow parallelism may be superflous
- Improve locks: a flow-wide lock for writers only, a reader-writer lock for matview
- Fork Rocksdb and provide a commit that keeps locks unreleased.
- Use fork to update dataflow while holding DB locks, then release everything together.
Forking Rocksdb:
- Remove the Clear() call from Commit() function in rocksdb/utilities/pessimistic_transaction.cc. Consider using a flag to disable/enable calling clear.
- Expose Clear() so that we can call it manually from Pelton after committing. Consider using destructor or rollback.
- Test via logging / monitoring of locks.
- Open issue/PR?