From e768d78a3512ab6fab5dc4714d960bab13b6a8bc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9=20Carneiro?= Date: Wed, 25 Sep 2024 11:57:01 -0400 Subject: [PATCH 1/3] docs: initial parts of send-tx queue design --- .../0006-send-tx-queue.md | 273 ++++++++++++++++++ 1 file changed, 273 insertions(+) create mode 100644 projects/hathor-wallet-headless/0006-send-tx-queue.md diff --git a/projects/hathor-wallet-headless/0006-send-tx-queue.md b/projects/hathor-wallet-headless/0006-send-tx-queue.md new file mode 100644 index 0000000..ec3f3f5 --- /dev/null +++ b/projects/hathor-wallet-headless/0006-send-tx-queue.md @@ -0,0 +1,273 @@ +- Feature Name: send_tx_queue +- Start Date: 2024-09-20 +- Author: Andre Carneiro + +# Summary +[summary]: #summary + +Make all APIs that send transactions on the network to enqueue a task instead of checking a lock. + +# Motivation +[motivation]: #motivation + +A call to send a transaction on the network acquires the individual wallet send-tx lock, any subsequent call that sends transactions will fail until the lock is released. +This is so that only 1 wallet can choose UTXOs at a time to avoid choosing conflicting UTXOs in different transactions. +The wallet-lib can expose the `PromiseQueue` class so that the headless can add tasks to send transactions on the queue instead of rejecting calls. +The goal being to increase transaction throughput since the caller does not need to waste time retrying until the wallet is free. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +## PromiseQueue + +The wallet-lib implementation of the `PromiseQueue` works by only letting X tasks run concurrently, this can be used as a queue of requests to send transactions. + +To minimize the time on queue we could try to have only the UTXO selection on queue, since the wallet facade does not grant such control granularity we can instead add a task to acquire the send-tx lock. +Once the lock is acquired we can prepare the transaction and release the lock, the next task should start while the current execution proceeds to push the tx. + +## Overview of operations + +Most operations should work similarly to the code below. + +```javascript +// Await to acquire lock +const lock = await taskQueue.add(acquireLockTask); + +try { + // Prepare transaction: each api will use a different method + const sendTx = await wallet.methodX(); + await sendTx.prepareTx(); +} finally { + // Release the lock once the utxo selection is done. + lock.release(); +} + +// Mine and push the transaction on the network. +// If this step fails the UTXOs will be automatically released. +await sendTx.runFromMining(); +``` + +## Improvements to send-tx process + +### Task selection + +The default method os task selection is based on a `PriorityQueue` on which the next task is always the one with higher priority, but the order among tasks of the same priority is not guaranteed. +This means that priority selection is a very important step, we can grant requests that arrive early more priority or add a `priority` argument where the client can give higher/lower priority to a request. + +We can also change the queue implementation to use a FIFO (First In First Out) queue which would make priority not relevant. + +The type of queue used will be configured on a per-instance basis under `sendTxQueueType` with the options `priority` and `fifo`. +Or with the environment variable (when running on docker) `HEADLESS_SEND_TX_QUEUE_TYPE` with the same options. + +### Fire and forget requests + +While usual requests will leave the client connected until the request is finished, so the client will receive the complete transaction when it finishes, the task is also aborted if the client disconnects. + +A "fire and forget" request will add a similar task to the queue but will return 200 to the client with a task id once the task is enqueued. +The task id will be added on the task cache, so the client can poll for the completion status. + +Using the "fire and forget" tasks will unblock the client and not tie the time to send the transaction with the client request timeout. +To change the type of request the client should just add an argument `task` as `true` to the headless. + +### Time to send transactions + +In a scenario where we want to send many transacttions in a short amount of time the queue of requests will automatically improve the throughput by minimizing the time between requests. +Once a transaction is prepared (UTXOs chosen and `Transaction` instance created) the next one will start, instead of the current implementation where we release the lock and wait for the next call. +The client, not knowing when the lock is released may add a long wait time, then the time to make the request itself would make the time to start a new transaction higher. + +## SendTx Task Cache + +The cache will be a simple Map, where we store the task status indexed by the `taskId`. + +### Task status API + +
+ + GET /wallet/tasks/send-tx/:taskId (Get the status of a send-tx task) + +This API will return the task status from an internal cache. + + +##### Parameters + +> | name | type | data type | description | location | +> | ----------- | -------- | --------- | ------------------------ | -------- | +> | taskId | required | string | The id of the task | path | +> | x-wallet-id | required | string | Wallet owner of the task | header | + +##### Responses + +> | http code | content-type | response | +> | --------- | ------------------ | ----------------------------------------------------------------- | +> | `200` | `application/json` | `{"success":true, "code": 3, "status": "Done", "txId": "abc123"}` | +> | `400` | `application/json` | `{"success": false, "message":"Bad Request"}` | +> +> Where the possible status are: +> - Waiting (0) +> - Executing (1) +> - Error (2) +> - Done (3) + +##### Example cURL + +> ```javascript +> curl -X POST -H 'X-Wallet-ID: main' 'http://localhost:8000/wallet/tasks/send-tx/123' +> ``` + +
+ + + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +## Wallet-Lib + +### PromiseQueue + +The `PromiseQueue` implementation needs to be improved to accept another queue implementation (`Queue` in this case). + +## Headless SendTx Queue + +```ts +const sendTxQueue = new PromiseQueue(); +// Concurrency should always be 1. +sendTxQueue.concurrency = 1; + + +/** + * @param {string} taskId + * @param {string} walletId + * @param {AbortSignal} signal - Used to abort the transaction + * @param {string} walletId + */ +async function waitForLock(taskId, walletId, signal, priority) { + // TODO: Create task status as waiting + try { + await sendTxQueue.add(async ({ signal }) => { + // Exit with error on abort + signal?.throwIfAborted(); + + while (!sendTxLock.get(walletId).lock(lockTypes.SEND_TX)) { + await new Promise(resolve => { + setTimeout(resolve, 100); + }); + signal?.throwIfAborted(); + } + // TODO: Move task status to Executing + }, { signal, priority }); + } catch (error) { + // TODO: Move task status to Error + throw error; + } +} +``` + +## Headless SendTx Task Cache + +```ts +type TaskWaiting = { + status: 'Waiting'; + code: 0; +}; + +type TaskExecuting = { + status: 'Executing'; + code: 1; +}; + +type TaskError = { + status: 'Error'; + code: 2; + error: Error; + message: string; + finishedAt: number; +}; + +type TaskDone = { + status: 'Done'; + code: 3; + txId: string; + finishedAt: number; +}; + +type TaskStatus = TaskWaiting | TaskExecuting | TaskError | TaskDone; + +const sendTxTasks = new Map(); +const CACHE_CLEAN_TIMEOUT = 60000; // 1 minute +// Timer is just for safe cleanup when the wallet shuts down. +const timer: ReturnType = null; + +function cleanCache() { + const now = Date.now(); + for (const [taskId, status] in sendTxTasks.entries()) { + const shouldClean = status.finishedAt && (now - status.finishedAt > CACHE_CLEAN_TIMEOUT); + if (shouldClean) { + sendTxTasks.delete(taskId); + } + } + + timer = setTimeout(cleanCache, CACHE_CLEAN_TIMEOUT/2); +} +``` + +The sendTx cache should be created on a "per-wallet" basis. + + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +- Why is this design the best in the space of possible designs? +- What other designs have been considered and what is the rationale for not + choosing them? +- What is the impact of not doing this? + +# Prior art +[prior-art]: #prior-art + +Discuss prior art, both the good and the bad, in relation to this proposal. +A few examples of what this can include are: + +- For protocol, network, algorithms and other changes that directly affect the + code: Does this feature exist in other blockchains and what experience have + their community had? +- For community proposals: Is this done by some other community and what were + their experiences with it? +- For other teams: What lessons can we learn from what other communities have + done here? +- Papers: Are there any published papers or great posts that discuss this? If + you have some relevant papers to refer to, this can serve as a more detailed + theoretical background. + +This section is intended to encourage you as an author to think about the +lessons from other blockchains, provide readers of your RFC with a fuller +picture. If there is no prior art, that is fine - your ideas are interesting to +us whether they are brand new or if it is an adaptation from other blockchains. + +Note that while precedent set by other blockchains is some motivation, it does +not on its own motivate an RFC. Please also take into consideration that Hathor +sometimes intentionally diverges from common blockchain features. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to resolve through the RFC process + before this gets merged? +- What parts of the design do you expect to resolve through the implementation + of this feature before stabilization? +- What related issues do you consider out of scope for this RFC that could be + addressed in the future independently of the solution that comes out of this + RFC? + +# Future possibilities +[future-possibilities]: #future-possibilities + +## Retry tasks + +To improve reliability we can check any errors on the transaction that are not an impediment (for instance timeout when pushing the transaction) and add the request back in the queue to retry. +This can be configured to retry a few times before actuallly confirming the error. From 32e69227387b3c821faa08f9f272c86e87ac4f59 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9=20Carneiro?= Date: Tue, 15 Oct 2024 12:28:20 -0400 Subject: [PATCH 2/3] docs: send-tx queue revision --- .../0006-send-tx-queue.md | 284 ++++++------------ 1 file changed, 89 insertions(+), 195 deletions(-) diff --git a/projects/hathor-wallet-headless/0006-send-tx-queue.md b/projects/hathor-wallet-headless/0006-send-tx-queue.md index ec3f3f5..4343b96 100644 --- a/projects/hathor-wallet-headless/0006-send-tx-queue.md +++ b/projects/hathor-wallet-headless/0006-send-tx-queue.md @@ -1,6 +1,6 @@ - Feature Name: send_tx_queue - Start Date: 2024-09-20 -- Author: Andre Carneiro +- Author: Andre Carneiro # Summary [summary]: #summary @@ -20,64 +20,114 @@ The goal being to increase transaction throughput since the caller does not need ## PromiseQueue -The wallet-lib implementation of the `PromiseQueue` works by only letting X tasks run concurrently, this can be used as a queue of requests to send transactions. +The wallet-lib implementation of the `PromiseQueue` works by only letting a defined number of tasks to run concurrently, this can be used as a queue of requests to send transactions. -To minimize the time on queue we could try to have only the UTXO selection on queue, since the wallet facade does not grant such control granularity we can instead add a task to acquire the send-tx lock. -Once the lock is acquired we can prepare the transaction and release the lock, the next task should start while the current execution proceeds to push the tx. +The task on queue will be a simple send-tx lock acquire loop that resolves when the lock is acquired. +After the task resolves the transaction sending can proceed as usual. -## Overview of operations +Since the wallet-lib `PromiseQueue` works with an underlying `PriorityQueue` we need to change the queue to be able to use a normal `Queue` since a `PriorityQueue` does not guarantee the order of tasks with the same priority. -Most operations should work similarly to the code below. +## Improvements to send-tx process + +In a scenario where we want to send many transactions in a short amount of time the queue of requests will automatically improve the throughput by minimizing the time between requests. +Once a transaction is prepared (UTXOs chosen and `Transaction` instance created) the next one will start, instead of the current implementation where we release the lock and wait for the next call. +The client, not knowing when the lock is released may add a long wait time, then the time to make the request itself would make the time to start a new transaction higher. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +## Wallet-Lib + +### PromiseQueue -```javascript -// Await to acquire lock -const lock = await taskQueue.add(acquireLockTask); +The `PromiseQueue` implementation needs to be improved to accept another queue implementation (`Queue` in this case). + +The queue methods used by `PromiseQueue` are `isEmpty`, `push` and `pop` so we can easily make both `PriorityQueue` and `Queue` use a compatible interface for `PromiseQueue`. +The `AddTaskOptions` will also need to change since `priority` only works for `PriorityQueue`. + +## Headless SendTx Queue -try { - // Prepare transaction: each api will use a different method - const sendTx = await wallet.methodX(); - await sendTx.prepareTx(); -} finally { - // Release the lock once the utxo selection is done. - lock.release(); +```ts +enum QueueClass { + PRIORITY, + FIFO, } -// Mine and push the transaction on the network. -// If this step fails the UTXOs will be automatically released. -await sendTx.runFromMining(); -``` +// We should use the FIFO implementation and concurrency should always be 1. +const sendTxQueue = new PromiseQueue({ + concurrency: 1, + queueClass: QueueClass.FIFO, +}); -## Improvements to send-tx process -### Task selection +// To enqueue a new task -The default method os task selection is based on a `PriorityQueue` on which the next task is always the one with higher priority, but the order among tasks of the same priority is not guaranteed. -This means that priority selection is a very important step, we can grant requests that arrive early more priority or add a `priority` argument where the client can give higher/lower priority to a request. +async function acquireLock(walletId: string, signal: AbortSignal): CallableFunction { + while(!signal.aborted) { + const canStart = lock.get(walletId).lock(lockTypes.SEND_TX); + if (canStart) { + break; + } -We can also change the queue implementation to use a FIFO (First In First Out) queue which would make priority not relevant. + await delay(100); + } -The type of queue used will be configured on a per-instance basis under `sendTxQueueType` with the options `priority` and `fifo`. -Or with the environment variable (when running on docker) `HEADLESS_SEND_TX_QUEUE_TYPE` with the same options. + if (signal.aborted) { + // The task was aborted + throw new Error('Task aborted'); + } +} -### Fire and forget requests +// pseudo logic on the controller -While usual requests will leave the client connected until the request is finished, so the client will receive the complete transaction when it finishes, the task is also aborted if the client disconnects. +function controllerMethod(req, res) { + const abortController = new AbortController(); + req.on('close', () => { + // If a client disconnects, cancel the task + abortController.abort(); + }); -A "fire and forget" request will add a similar task to the queue but will return 200 to the client with a task id once the task is enqueued. -The task id will be added on the task cache, so the client can poll for the completion status. + await sendTxQueue.add( + async ({ signal }) => { return acquireLock(req.walletId, signal); }, + { signal: abortController.signal }, + ); -Using the "fire and forget" tasks will unblock the client and not tie the time to send the transaction with the client request timeout. -To change the type of request the client should just add an argument `task` as `true` to the headless. + try { + // Send transaction normally + ... + } finally { + lock.get(req.walletId).unlock(lockTypes.SEND_TX); + } +} +``` -### Time to send transactions +# Future possibilities +[future-possibilities]: #future-possibilities -In a scenario where we want to send many transacttions in a short amount of time the queue of requests will automatically improve the throughput by minimizing the time between requests. -Once a transaction is prepared (UTXOs chosen and `Transaction` instance created) the next one will start, instead of the current implementation where we release the lock and wait for the next call. -The client, not knowing when the lock is released may add a long wait time, then the time to make the request itself would make the time to start a new transaction higher. +## Retry tasks + +To improve reliability we can check any errors on the transaction that are not an impediment (for instance timeout when mining/pushing the transaction) and add the request back in the queue to retry. +This can be configured to retry a few times before actuallly confirming the error. -## SendTx Task Cache +## Task selection -The cache will be a simple Map, where we store the task status indexed by the `taskId`. +We could optionally configure the queue to use the `PriorityQueue` so the user can define a priority to his request, allowing transactions to go first depending on how important the user deems them. + +The usual requests can also be ordered by using a decreasing priority, starting at $-1$ and decreasing with each new task. +This means that the highest priority task will always be the user defined ones (because they're always positive) then the ones enqueued first. +We can also reset the counter when the queue becomes empty so that we don't decrease the counter forever. + +## Fire and forget requests + +Usual requests will leave the client connected until the request is finished, this way the user can receive the transaction he created. + +A "fire and forget" request will validate the parameters and add a task on the queue to send the transaction. +The response will be immediate and will not wait for the transaction to be completed, i.e. `HTTP 200 { "success": true, "taskId": "foobar" }`. + +The task id will be added on a cache so the client can poll for the result. + +This allows the transaction "time to send" not bounded by the request timeout. +To change the type of request the client should just add an argument `task` as `true` in the request. ### Task status API @@ -115,159 +165,3 @@ This API will return the task status from an internal cache. > ``` - - - -# Reference-level explanation -[reference-level-explanation]: #reference-level-explanation - -## Wallet-Lib - -### PromiseQueue - -The `PromiseQueue` implementation needs to be improved to accept another queue implementation (`Queue` in this case). - -## Headless SendTx Queue - -```ts -const sendTxQueue = new PromiseQueue(); -// Concurrency should always be 1. -sendTxQueue.concurrency = 1; - - -/** - * @param {string} taskId - * @param {string} walletId - * @param {AbortSignal} signal - Used to abort the transaction - * @param {string} walletId - */ -async function waitForLock(taskId, walletId, signal, priority) { - // TODO: Create task status as waiting - try { - await sendTxQueue.add(async ({ signal }) => { - // Exit with error on abort - signal?.throwIfAborted(); - - while (!sendTxLock.get(walletId).lock(lockTypes.SEND_TX)) { - await new Promise(resolve => { - setTimeout(resolve, 100); - }); - signal?.throwIfAborted(); - } - // TODO: Move task status to Executing - }, { signal, priority }); - } catch (error) { - // TODO: Move task status to Error - throw error; - } -} -``` - -## Headless SendTx Task Cache - -```ts -type TaskWaiting = { - status: 'Waiting'; - code: 0; -}; - -type TaskExecuting = { - status: 'Executing'; - code: 1; -}; - -type TaskError = { - status: 'Error'; - code: 2; - error: Error; - message: string; - finishedAt: number; -}; - -type TaskDone = { - status: 'Done'; - code: 3; - txId: string; - finishedAt: number; -}; - -type TaskStatus = TaskWaiting | TaskExecuting | TaskError | TaskDone; - -const sendTxTasks = new Map(); -const CACHE_CLEAN_TIMEOUT = 60000; // 1 minute -// Timer is just for safe cleanup when the wallet shuts down. -const timer: ReturnType = null; - -function cleanCache() { - const now = Date.now(); - for (const [taskId, status] in sendTxTasks.entries()) { - const shouldClean = status.finishedAt && (now - status.finishedAt > CACHE_CLEAN_TIMEOUT); - if (shouldClean) { - sendTxTasks.delete(taskId); - } - } - - timer = setTimeout(cleanCache, CACHE_CLEAN_TIMEOUT/2); -} -``` - -The sendTx cache should be created on a "per-wallet" basis. - - -# Drawbacks -[drawbacks]: #drawbacks - -Why should we *not* do this? - -# Rationale and alternatives -[rationale-and-alternatives]: #rationale-and-alternatives - -- Why is this design the best in the space of possible designs? -- What other designs have been considered and what is the rationale for not - choosing them? -- What is the impact of not doing this? - -# Prior art -[prior-art]: #prior-art - -Discuss prior art, both the good and the bad, in relation to this proposal. -A few examples of what this can include are: - -- For protocol, network, algorithms and other changes that directly affect the - code: Does this feature exist in other blockchains and what experience have - their community had? -- For community proposals: Is this done by some other community and what were - their experiences with it? -- For other teams: What lessons can we learn from what other communities have - done here? -- Papers: Are there any published papers or great posts that discuss this? If - you have some relevant papers to refer to, this can serve as a more detailed - theoretical background. - -This section is intended to encourage you as an author to think about the -lessons from other blockchains, provide readers of your RFC with a fuller -picture. If there is no prior art, that is fine - your ideas are interesting to -us whether they are brand new or if it is an adaptation from other blockchains. - -Note that while precedent set by other blockchains is some motivation, it does -not on its own motivate an RFC. Please also take into consideration that Hathor -sometimes intentionally diverges from common blockchain features. - -# Unresolved questions -[unresolved-questions]: #unresolved-questions - -- What parts of the design do you expect to resolve through the RFC process - before this gets merged? -- What parts of the design do you expect to resolve through the implementation - of this feature before stabilization? -- What related issues do you consider out of scope for this RFC that could be - addressed in the future independently of the solution that comes out of this - RFC? - -# Future possibilities -[future-possibilities]: #future-possibilities - -## Retry tasks - -To improve reliability we can check any errors on the transaction that are not an impediment (for instance timeout when pushing the transaction) and add the request back in the queue to retry. -This can be configured to retry a few times before actuallly confirming the error. From 6591d11783ef6dc7752a59bf08474c2395cff198 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9=20Carneiro?= Date: Tue, 15 Oct 2024 12:34:46 -0400 Subject: [PATCH 3/3] docs: task breakdown --- projects/hathor-wallet-headless/0006-send-tx-queue.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/projects/hathor-wallet-headless/0006-send-tx-queue.md b/projects/hathor-wallet-headless/0006-send-tx-queue.md index 4343b96..c9213e7 100644 --- a/projects/hathor-wallet-headless/0006-send-tx-queue.md +++ b/projects/hathor-wallet-headless/0006-send-tx-queue.md @@ -165,3 +165,8 @@ This API will return the task status from an internal cache. > ``` + +# Task breakdown + +- \[wallet-lib\] add support for FIFO PromiseQueue (1 dev day) +- \[wallet-headless\] create a send-tx queue for all routes (3 dev days)