Skip to content

Conversation

@rafaelcr
Copy link
Collaborator

No description provided.

rafaelcr and others added 30 commits August 15, 2024 08:59
Changes the TS client server code so it now checks if a predicate UUID
already exists and is active in the target Chainhook node. If it is, it
reuses it. If it isn't it removes it and registers again.
- bump time crate, seeing build failure in
https://github.com/Homebrew/homebrew-core/actions/runs/10126855440/job/28023138251
relates to time-rs/time#693
- also update toolchain and run the cargo fmt/clippy fixes

---------

Signed-off-by: Rui Chen <rui@chenrui.dev>
Installs `semantic-release` in Github actions so it determines and tags
the repo with every new `chainhook-cli` release. This also updates the
`Cargo.toml` and `Cargo.lock` files automatically, but it doesn't yet
update `chainhook-sdk` and `chainhook-types-rs` releases automatically.

This also does not upload a new `chainhook-sdk` release to crates.io

Related to #638
Optimizes build times by fixing the use of the cache github action
Optimizes GH action times by running component tests in parallel
…#648)

This PR makes sure we return a `500` status code error to the Stacks
node when chainhook failed to ingest an event correctly, so it can retry
indefinitely until it succeeds.

This prevents block gaps from occurring in the chainhook index.

Fixes #646
…quests (#642)

This PR adds a new (optional) entry to the config .toml file so outgoing
predicate payload HTTP POST requests can have a configurable timeout
that will cancel the request if hit.

```toml
[predicates]
payload_http_request_timeout_ms = 10000
```

This new setting is useful to public-facing chainhook instances that may
run into unresponsive 3rd party servers that may take forever to respond
to a payload request.

Fixes #643
Changes how redis tests are performed so they run with a dockerized
integration environment instead of relying on a local `redis-server`
install controlled via Rust code.

Adds tasks and launch configs to VScode so they are easier to launch and
adjusts the CI to use it as well.
Starts listening to the new `/stackerdb_chunks` Stacks node event and
parses the incoming message. Parsers are taken from the `stacks_codec`
crate, which has been updated with the latest serializers.

For future PRs:
* Start integrating the signer messages into the predicates framework
* Store signer messages in a local SQLite DB for replay
* Expand unit tests to include full Stacks node integration tests

---------

Co-authored-by: Matthew Little <zone117x@gmail.com>
…658)

Upgrades TS client to new breaking change with updated schema treatments
and unit tests.

**Breaking changes**
* Instead of letting users define a predicate `uuid` by hand, it is now
configured automatically by the client so users can only set the `name`
and we can use that as a key to figure out if we had already registered
a predicate before. This makes UUIDs be local to that running event
observer and avoids collisions when pointing multiple observers to one
single chainhook node.
* Upgrades schema names and configuration options to better reflect what
they do

**Other features**
* Adds on-disk predicate persistence for registered predicates: saves to
disk and recalls upon restarts
* Adds a configurable health check to make sure on-disk predicates are
still healthy, and re-registers them if they are interrupted
* Adds unit tests for new behavior
… and `StacksPayload` (#659)

Related to #651 -- we
need these new fields to make the `StackerDB` chunk messages useful.
Expands the TS client to include new signer messages, and adds missing
`signer_bitvec` and `signer_signatures` fields to block metadata.
Fixes the `slot_id` and `slot_version` field sizes so the signer pubkey
is calculated correctly
Creates a local sqlite DB for signer messages (`stacks_signers.sqlite`)
and stores all messages in it so we can later retrieve them via
predicate scans.
## [1.9.0-stacks-signers.1](v1.8.0...v1.9.0-stacks-signers.1) (2024-10-25)

### Features

* add various new nakamoto block fields to `/new_block` ingestion and `StacksPayload` ([#659](#659)) ([f48cda5](f48cda5))
* include recovered signer pubkeys in new block payload ([#662](#662)) ([b5ad4ba](b5ad4ba))
* include signer messages in Stacks predicate payloads ([#656](#656)) ([aee14bc](aee14bc))
* parse `/stackerdb_chunks` Stacks node event ([#653](#653)) ([e44d84a](e44d84a))
* store signer messages in local sqlite database ([#664](#664)) ([d12acd9](d12acd9))
* support tenure-height in block messages ([#661](#661)) ([99e5a61](99e5a61))
* **ts-client:** add persisted predicates and periodic health check ([#658](#658)) ([535226a](535226a))
* **ts-client:** add signer messages to stacks payloads ([#657](#657)) ([ff66bb2](ff66bb2))

### Bug Fixes

* add a configurable max timeout for outgoing predicate payload requests ([#642](#642)) ([6c1dfa9](6c1dfa9)), closes [#643](#643)
* increase stacks event server body size limit to 500MB ([#660](#660)) ([21234c9](21234c9))
* return 500 status code to Stacks node when event ingestion fails ([#648](#648)) ([63c753c](63c753c)), closes [#646](#646)
* signer pubkey calculation ([#665](#665)) ([cc93873](cc93873))
* update SignerMessage deserializing ([#663](#663)) ([71364c1](71364c1))
* validate ts client predicates before registering ([#639](#639)) ([2d08f72](2d08f72))
## [1.9.0-stacks-signers.2](v1.9.0-stacks-signers.1...v1.9.0-stacks-signers.2) (2024-10-25)

### Features

* add mock signer message support ([#669](#669)) ([ad5fd54](ad5fd54))
zone117x and others added 19 commits October 31, 2024 15:24
Workaround for bug in stacks-core event emitter http code. Regression in
stacks-core where http requests must now have their POST body read
before closing the connection.

```
chainhook-1    | {"msg":"POST /drop_mempool_tx","level":"DEBUG","ts":"2024-10-30T10:41:34.030361652Z"}
stacks-node-1  | WARN [1730284894.031170] [testnet/stacks-node/src/event_dispatcher.rs:496] [relayer-http://0.0.0.0:20443/] Event dispatcher: connection or request failed to chainhook:20455 - Custom { kind: Other, error: "Failed to send 8192 bytes: \"Failed to send socket data\"" }, backoff: 158.191s, attempts: 10
```


Chainhook ignored some events by returning a 200 http response and
closing the connection after reading the http request headers, and
ignoring the request body.
## [1.9.0-stacks-signers.6](v1.9.0-stacks-signers.5...v1.9.0-stacks-signers.6) (2024-10-31)

### Bug Fixes

* read stacks-core http event POST payloads for ignored events ([#673](#673)) ([a01470e](a01470e))
propagate error when storing peer_info
## [1.9.0-stacks-signers.7](v1.9.0-stacks-signers.6...v1.9.0-stacks-signers.7) (2024-11-07)

### Bug Fixes

* propagate error when trying to store peer info ([1bc949a](1bc949a))
Also, remove the old code that attempted to re-import a TSV every 32
Stacks blocks

Fixes #677
## [1.9.0-stacks-signers.8](v1.9.0-stacks-signers.7...v1.9.0-stacks-signers.8) (2024-11-12)

### Bug Fixes

* only import stacks tsv if chainstate is empty ([#684](#684)) ([1f9f37e](1f9f37e)), closes [#677](#677)
merge stacks signers into develop
### Description

This prs add the missing `matches_regex` option for the `print_event`
predicate that is missing in the client SDK.

#### Breaking change?

No

### Checklist

- [ ] All tests pass
- [ ] Tests added in this PR (if applicable)
…ion (#675)

Closes #674

* First commit adds an optional callback that clients can use to modify
an inactive predicate before it is re-registered.
* Second commit fixes an issue where the the `setInterval` could result
in overlapping healthcheck code running if something like the callback
(which could be waiting on a postgres connection, for example) take a
long time.
Multiple parents and charms

---------

Co-authored-by: Hugo CAILLARD <911307+hugocaillard@users.noreply.github.com>
chore: clean dependencies
### Description

A recent PR in stacks-core updated the way cargo feature are handled in
clarity and stacks-common
(stacks-network/stacks-core#5902).

Chainhook and Clarinet both need to update and handle this breaking
changes - annoyingly, both repo need the other to do the update as well.

- this PR tgargets [this
branch](stx-labs/clarinet#1725) in clarinet,
and would need to be merged first
- the clarinet PR will be updated once this one in merge to target
chainhook/develop
- ideally, i will open an other PR on this repo to target a commit that
is on clarinet/main


### CI changes

I think chainhook was built with older versions of rust and libclang out
of extra caution.
But I don't think this is necessary, all of our other rust tools
(including the blockchain itself) are built with stable.
Reads a deployed smart contract ABI from `contract_interface` instead of
`contract_abi` in `/new_block` events. We still keep support for the old
property in case someone runs chainhook with an older Stacks node or TSV
file.
@rafaelcr rafaelcr requested a review from csgui June 13, 2025 15:39
@rafaelcr rafaelcr enabled auto-merge June 13, 2025 15:55
@rafaelcr rafaelcr disabled auto-merge June 13, 2025 15:56
@rafaelcr rafaelcr had a problem deploying to Development-mainnet June 13, 2025 16:09 — with GitHub Actions Error
@rafaelcr rafaelcr had a problem deploying to Development-testnet June 13, 2025 16:09 — with GitHub Actions Error
@rafaelcr rafaelcr had a problem deploying to Development-testnet June 13, 2025 16:11 — with GitHub Actions Failure
@rafaelcr rafaelcr had a problem deploying to Development-mainnet June 13, 2025 16:11 — with GitHub Actions Failure
@rafaelcr rafaelcr merged commit a7094f6 into main Jun 13, 2025
18 of 22 checks passed
@hirosystems-release-bot
Copy link

🎉 This PR is included in version 1.9.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants