Skip to content

Releases: ar-io/ar-io-node

Release 63

22 Dec 18:21
@djwhitt djwhitt
r63
71da00b

Choose a tag to compare

View all tags
Release 63 Latest
Latest

This is an optional release focusing on operator tooling and observability improvements. Key additions include a data retrieval testing tool for gateway validation, separate credentials support for legacy S3 chunk sources, and OTEL-Winston integration for distributed trace correlation in logs.

Added

  • CDB64 Extension Support: Accept .cdb64 file extension in addition to .cdb for CDB64 root TX index files

  • Data Retrieval Testing Tool: New CLI tool for testing data item retrieval from a gateway using TX/data item IDs from a CSV file (tools/test-data-retrieval)

    • Sequential mode: streams through file line by line
    • Random mode: O(1) random byte seeking, no file scan required
    • Continuous mode: runs indefinitely until Ctrl+C, writes JSON results to file
    • Configurable concurrency for parallel requests
    • Comprehensive statistics: success/failure rates, response time percentiles (p50/p95/p99), cache hit rates, status codes, bytes transferred

  • Separate Credentials for Legacy S3 Chunk Source: Add ability to configure separate AWS credentials for the legacy S3 chunk data source, enabling access to S3 buckets in different AWS accounts

    • LEGACY_AWS_S3_ACCESS_KEY_ID: AWS access key for legacy S3 bucket
    • LEGACY_AWS_S3_SECRET_ACCESS_KEY: AWS secret key for legacy S3 bucket
    • LEGACY_AWS_S3_REGION: AWS region (required when using separate credentials)
    • LEGACY_AWS_S3_ENDPOINT: Custom endpoint (optional)

  • Docker Compose Environment Variables: Expose additional environment variables in docker-compose.yaml for legacy chunk sources and chunk rebroadcasting

  • OTEL Winston Integration for Trace ID Correlation: Automatic injection of OpenTelemetry trace context (trace_id, span_id, trace_flags) into all Winston log entries, enabling correlation of logs with distributed traces

Fixed

  • Fix missing parentSpan parameter in handleRangeRequest calls for proper OTEL trace hierarchy in range requests

Docker Images

  • ghcr.io/ar-io/ar-io-envoy:4755fa0a2deb258bfaeaa91ba3154f1f7ef41fda
  • ghcr.io/ar-io/ar-io-core:11f7b981c02a2d3ac27ee80dda7b06dff2ad904b
  • ghcr.io/ar-io/ar-io-clickhouse-auto-import:4512361f3d6bdc0d8a44dd83eb796fd88804a384
  • ghcr.io/ar-io/ar-io-litestream:be121fc0ae24a9eb7cdb2b92d01f047039b5f5e8
Assets 2
Loading

Release 62

17 Dec 16:18
@djwhitt djwhitt
r62
86ffd86

Choose a tag to compare

View all tags
Release 62

This is an optional release that introduces the CDB64-based historical root TX index, a new lookup source for resolving data items to their root transactions. The CDB64 format provides fast O(1) lookups from pre-generated index files, enabling efficient root TX resolution for historical data without database queries. Pre-generated CDB64 index files will be made available in the coming weeks.

Added

  • CDB64-Based Historical Root TX Index: New lookup source for root transaction IDs using the compact CDB64 file format
    • Enable by adding 'cdb' to ROOT_TX_LOOKUP_ORDER (e.g., db,cdb,gateways,graphql)
    • Reads from data/cdb64-root-tx-index directory (configurable via CDB64_ROOT_TX_INDEX_DATA_PATH in Docker)
    • Multi-file directory support with runtime file watching (new .cdb files automatically loaded without restart)
    • File watching can be disabled via CDB64_ROOT_TX_INDEX_WATCH=false
    • CLI tools for index generation and data extraction:
      • SQLite to CDB64 export (tools/export-sqlite-to-cdb64)
      • CSV to CDB64 generation with RFC 4180 support (tools/generate-cdb64-root-tx-index)
      • Rust-backed CDB64 generation for high-throughput (tools/generate-cdb64-root-tx-index-rs)
      • CDB64 to CSV export (tools/export-cdb64-root-tx-index)
    • Binary compatibility with Rust cdb64-rs for cross-language interoperability
    • CDB64 file format specification documentation (docs/cdb64-format.md)

Fixed

  • Expose ROOT_TX_LOOKUP_ORDER environment variable in docker-compose.yaml (was missing from previous releases)

Docker Images

Image SHA
ghcr.io/ar-io/ar-io-core b7523578c7e1103d2e50f3a19e39e71d6e466ebd
ghcr.io/ar-io/ar-io-envoy b7523578c7e1103d2e50f3a19e39e71d6e466ebd
ghcr.io/ar-io/ar-io-clickhouse-auto-import b7523578c7e1103d2e50f3a19e39e71d6e466ebd
ghcr.io/ar-io/ar-io-litestream b7523578c7e1103d2e50f3a19e39e71d6e466ebd
ghcr.io/ar-io/ar-io-observer 0380592e095cd7bf645f6d132b97301d2dad8101
Loading

Release 61

10 Dec 16:28
@djwhitt djwhitt
r61
a0a5fd2

Choose a tag to compare

View all tags
Release 61

This release addresses potential memory growth issues observed on some r60 nodes by converting unbounded caches to bounded LRU caches and adding cleanup for stale peer chunk queues. Users currently on r60 experiencing memory issues should upgrade. Users on earlier releases may want to wait until these improvements have been confirmed in production.

Changed

  • Observer Performance Improvements: Updated bundled observer with chunk verification optimizations ported from ar-io-node
    • TX path Merkle proof parsing eliminates 7-10 API calls per chunk by extracting transaction boundaries directly from tx_path
    • Pre-computed offset-to-block mapping narrows binary search range by 97-99% (from ~1.8M to ~26K blocks)

Fixed

  • Memory Leak Prevention: Address potential memory growth vectors identified during OOM investigation on low-memory nodes
    • Add hourly cleanup of stale peerChunkQueues entries for peers no longer in the active peer list (only removes idle queues)
    • Convert blockByHeightPromiseCache and txPromiseCache from unbounded NodeCache to LRUCache with size limits (1000 blocks, 10000 transactions)
    • Convert SQLite dedupe caches (insertDataHashCache, saveDataContentAttributesCache) from unbounded NodeCache to LRUCache (10000 entries each)
    • Add arweave_peer_chunk_queues_size Prometheus gauge for monitoring

Docker Images

Image SHA
ar-io-core b92874f2c9442b399ad27ab2c09c13fead654b35
ar-io-envoy 4755fa0a2deb258bfaeaa91ba3154f1f7ef41fda
ar-io-observer 0380592e095cd7bf645f6d132b97301d2dad8101
ar-io-litestream be121fc0ae24a9eb7cdb2b92d01f047039b5f5e8
ar-io-clickhouse-auto-import 4512361f3d6bdc0d8a44dd83eb796fd88804a384
Loading

Release 60

03 Dec 21:18
@djwhitt djwhitt
r60
1d12290

Choose a tag to compare

View all tags
Release 60

This is a recommended release due to significant chunk retrieval performance improvements. AR.IO peer chunk fetching now uses parallel requests, reducing worst-case latency from ~150 seconds to ~4 seconds. Additional optimizations include tx_path Merkle proof validation to avoid expensive chain binary searches, offset-based chunk cache lookups via symlinks, and a static offset-to-block mapping that reduces block search iterations by ~29%.

Summary

  • AR.IO Peer Chunk Retrieval Optimization: Parallel peer requests, worst-case latency reduced from ~150s to ~4s
  • tx_path Chunk Validation: DB-first lookup with tx_path Merkle proof fallback
  • Chunk Cache by Absolute Offset: Symlink-based O(1) cache lookups by weave offset
  • Block Search Optimization: Static offset-to-block mapping reduces search iterations by ~29%
  • Chunk POST Early Termination: Stop broadcasting after consecutive 4xx failures (~96% reduction in wasted requests)
  • OTEL Nested Bundle Sampling Policies: Targeted tail-sampling to detect nested bundle offset issues
  • Chunk Rebroadcasting: Optional async rebroadcasting of chunks from configured sources

See CHANGELOG.md for full details.

Docker Images

ghcr.io/ar-io/ar-io-core:755f68c6bc8309ab775a5c9ffd81586f75cbab30
ghcr.io/ar-io/ar-io-envoy:4755fa0a2deb258bfaeaa91ba3154f1f7ef41fda
ghcr.io/ar-io/ar-io-clickhouse-auto-import:4512361f3d6bdc0d8a44dd83eb796fd88804a384
ghcr.io/ar-io/ar-io-litestream:be121fc0ae24a9eb7cdb2b92d01f047039b5f5e8
Loading

Release 59

24 Nov 15:15
@djwhitt djwhitt
r59
21a1ebe

Choose a tag to compare

View all tags
Release 59

[Release 59] - 2025-11-24

This is a recommended release due to important fixes for nested bundle data
item offset handling that could cause incorrect data retrieval. The release
fixes offset calculations in both the TurboDynamoDB data source and database
root TX lookups, ensuring correct data is served for deeply nested bundle
items. It also includes fixes for ArNS manifest path encoding and Observer
wallet failure reporting for shared FQDN gateways. New features include a
dry-run mode for testing transaction uploads without posting to the network,
and a monitoring tool for historical DHA chunk nodes.

Added

  • Historical DHA Chunk Nodes Monitoring Tool: New operator utility for
    monitoring response times and availability of Arweave data endpoints
    (tools/monitor-historical-dha-chunk-nodes)

    • Monitors data-N (1-17) and tip-N (1-5) endpoints with configurable ranges
    • Continuous monitoring mode with real-time table output and statistics
    • JSON export with detailed results and metadata
    • Note: This is a special-purpose tool included for reference and potential
      usefulness to operators debugging data retrieval issues

  • Dry-Run Mode for Upload Testing: New ARWEAVE_POST_DRY_RUN environment
    variable enables testing transaction and chunk uploads without posting to the
    Arweave network

    • When enabled, both POST /tx and POST /chunk requests are simulated with
      200 OK responses
    • Works on both port 3000 (Envoy) and port 4000 (direct to Node.js app)
    • Envoy routing is conditional: routes to core for dry-run, to Arweave nodes
      when disabled
    • Perfect for testing apps like ArDrive and large uploads without burning AR
      tokens
    • By default, transactions are validated (signature verification) and chunks
      are validated (merkle proof verification) before returning success
    • Set ARWEAVE_POST_DRY_RUN_SKIP_VALIDATION=true to skip validation for faster
      testing
    • Only the final network broadcast is skipped

Changed

  • When CDP API keys are provided (CDP_API_KEY_ID and CDP_API_KEY_SECRET),
    the gateway now automatically uses the Coinbase facilitator with enhanced
    Onramp integration, overriding the X_402_USDC_FACILITATOR_URL setting

Fixed

  • PostgreSQL SSL Configuration: Fixed inverted SSL flag logic where
    LEGACY_PSQL_SSL_REJECT_UNAUTHORIZED=true (default) was incorrectly disabling
    certificate validation instead of enabling it

    • Now correctly applies strict SSL validation by default
    • Set to false to disable certificate validation for cloud providers with
      self-signed certificates

  • PostgreSQL Connection Timeouts: Added timeout configuration for the legacy
    PostgreSQL chunk metadata source to prevent system hangs

    • Server-side statement_timeout (default: 5s) prevents queries from running
      forever
    • idle_in_transaction_session_timeout (default: 10s) cleans up stuck
      transactions
    • Connection pool settings: max, idle_timeout, connect_timeout,
      max_lifetime
    • All settings configurable via environment variables
      (LEGACY_PSQL_STATEMENT_TIMEOUT_MS, etc.)
    • Graceful Postgres connection cleanup on shutdown
    • Prevents the chunk serving system from becoming completely unresponsive when
      Postgres is slow or unreachable

  • Security Dependency Updates: Fixed 6 security vulnerabilities identified by
    yarn audit

    • Added tar@7.5.2 resolution to fix moderate severity race condition in
      duckdb-async dependency chain
    • Upgraded @cucumber/cucumber, @testcontainers/localstack, testcontainers,
      and rimraf to fix high severity glob CLI command injection vulnerabilities
    • Upgraded viem to ^2.39.3
    • All existing resolutions (secp256k1, elliptic, ws, semver) remain required for
      vulnerabilities in @dha-team/arbundles transitive dependencies

  • ArNS Manifest Path Encoding: Fixed manifest paths with URL-encoded
    characters (e.g., spaces as %20) failing when accessed via ArNS subdomain

    • Direct TX ID access worked because Express auto-decodes req.params
    • ArNS subdomain access failed because req.path is not auto-decoded
    • Now decodes manifest paths in the ArNS middleware for consistent behavior

  • TurboDynamoDB Data Source: Fixed nested bundle data items having incorrect
    rootDataItemOffset values when retrieved from Turbo's DynamoDB

    • The raw data path was overwriting correct absolute offsets cached from the
      rootParentInfo path with incorrect values (offset: 0, dataOffset: payloadDataStart)
    • Now preserves the correct offsets by only caching size and contentType from raw data

  • Database Root TX Offset Lookup: Fixed getRootTxFromData returning incorrect
    offset for nested bundle data items

    • Was returning root_parent_offset (parent bundle offset) instead of
      root_data_item_offset (absolute data item offset)
    • Added fallback calculations for rootDataItemOffset and rootDataOffset when
      pre-computed values are NULL

  • Observer: Updated to 2515e6a - Fixed incorrect wallet failure reporting
    for shared FQDN gateways

    • When multiple wallets share the same FQDN, now correctly identifies which
      specific wallets failed ownership verification
    • Reports non-matching wallets as failed even when gateway passes overall
    • Ensures save-observations contract interactions accurately reflect actual
      ownership failures

  • BUNDLER_URLS environment variable was missing from docker-compose.yaml

  • x402 payment processor now correctly uses Coinbase CDP facilitator when CDP
    credentials are configured

Docker Images

  • ghcr.io/ar-io/ar-io-envoy:4755fa0a2deb258bfaeaa91ba3154f1f7ef41fda
  • ghcr.io/ar-io/ar-io-core:4fc4c14ec6ef220298cfaad313f9448c286feec3
  • ghcr.io/ar-io/ar-io-clickhouse-auto-import:4512361f3d6bdc0d8a44dd83eb796fd88804a384
  • ghcr.io/ar-io/ar-io-litestream:be121fc0ae24a9eb7cdb2b92d01f047039b5f5e8
Loading

Release 58

10 Nov 16:09
@djwhitt djwhitt
r58
9fe8acd

Choose a tag to compare

View all tags
Release 58

Release 58 - 2025-11-10

This is a recommended release due to significant improvements in data retrieval efficiency and payment system reliability. This release introduces a new raw binary chunk data endpoint providing ~40% bandwidth savings, comprehensive rate limit balance management APIs, and intelligent OpenTelemetry tail-based sampling for cost-effective observability. The release also includes critical payment validation fixes and enhanced bundler service discovery for improved client integration.

⚠️ EXPERIMENTAL FEATURES: The rate limiter and x402 payment protocol are experimental features subject to change. API endpoints, parameters, behavior, and configuration options (environment variables) may evolve in future releases as these systems continue to be developed. See docs/x402-and-rate-limiting.md for comprehensive documentation.

Added

  • Raw Binary Chunk Data Endpoint: New /chunk/<offset>/data endpoint returns raw binary chunk data (application/octet-stream) with metadata in response headers instead of base64url-encoded JSON

    • Provides ~40% bandwidth savings compared to the base64url-encoded /chunk/<offset> endpoint
    • Supports both GET and HEAD requests
    • Returns comprehensive metadata in custom headers
    • Supports ETag-based conditional requests (304 Not Modified)
    • Supports Content-Digest header (RFC 9530) for data integrity verification
    • Rate limited at 256 KiB (raw chunk size) vs. 360 KiB for base64url endpoint, resulting in lower per-chunk fees

  • Bundler Service Discovery: The /ar-io/info endpoint now includes a bundlers field for client service discovery

    • Configurable via BUNDLER_URLS environment variable (comma-separated URLs)
    • Defaults to https://turbo.ardrive.io/
    • URLs are validated on startup with descriptive error messages

  • Rate Limit Balance Management API: New REST API endpoints for querying and managing rate limit bucket balances

    • GET /ar-io/rate-limit/ip/:ip - Query IP-based rate limit bucket balance
    • POST /ar-io/rate-limit/ip/:ip - Top up IP-based bucket via x402 payment or admin API key
    • GET /ar-io/rate-limit/resource - Query resource-based bucket balance
    • POST /ar-io/rate-limit/resource - Top up resource-based bucket via x402 payment or admin API key
    • Dual authentication: x402 payment protocol (public) or admin API key (private/testing)
    • 10x capacity multiplier applied to x402 payments

  • OpenTelemetry Collector with Tail-Based Sampling ⚠️ EXPERIMENTAL: New OTEL Collector sidecar in docker-compose deployments implements intelligent tail-based sampling to reduce telemetry costs by 80-95%

    • Five intelligent sampling policies make decisions after traces complete
    • 100% of traces with errors, slow requests, x402 verified payments, and paid rate limit token usage
    • 1% (configurable) of successful, fast, unpaid requests for baseline metrics
    • Support for multiple telemetry backends (Honeycomb, Grafana Cloud, Datadog, New Relic, Elastic APM)
    • Optional deployment via docker-compose profile (docker compose --profile otel up)

Changed

  • Chunk Request Routing: Removed Envoy proxy route for GET /chunk/ requests - all chunk endpoints now route directly to the AR.IO gateway application
  • Transaction-Level Merkle Path Support: The /chunk/<offset> endpoint now includes tx_path in JSON responses when available
  • Observer: Updated to fcd0f36 - Doubled offset observation sample rate to 2% for improved network robustness
  • AR.IO Info Endpoint Structure: The bundlers field in /ar-io/info endpoint response is now nested under services.bundlers
  • Rate Limiter Bucket Keys: Standardized bucket key format across Redis and Memory rate limiter implementations
  • Browser Paywall for Chunk Requests: Optimized payment flow for chunk endpoints to use direct URL payment instead of redirect endpoint

Fixed

  • Payment Processor and x402 Validation: Multiple improvements to payment validation and error handling

    • Prevent infinite redirect loop on payment failure
    • Validate payment type before settlement
    • Prevent silent success when payment settlement cannot grant tokens
    • Validate resource target format before settling payment
    • Validate Host header presence and format before processing payments
    • Use configured capacity multiplier consistently

  • Rate Limiter Configuration: Fixed configuration handling for consistent behavior

    • Ensure consistent resource key normalization across Redis and Memory implementations
    • Use configurable capacity multiplier in rate limit routes

Docker Images

All images are tagged with their respective git commit SHAs:

  • envoy: ghcr.io/ar-io/ar-io-envoy:f7c5fd8f3ad894e7a9e12162941a252b129c04b1
  • core: ghcr.io/ar-io/ar-io-core:2ea9703941aedc28cc15ee8e753764fecfce3a7a
  • clickhouse-auto-import: ghcr.io/ar-io/ar-io-clickhouse-auto-import:4512361f3d6bdc0d8a44dd83eb796fd88804a384
  • observer: ghcr.io/ar-io/ar-io-observer:fcd0f3642d5755c4b6adfdd4fb298d127dbb0f68
  • litestream: ghcr.io/ar-io/ar-io-litestream:be121fc0ae24a9eb7cdb2b92d01f047039b5f5e8

Installation

See the README for installation instructions.

For full details, see the CHANGELOG.

Loading

Release 57

03 Nov 19:49
@djwhitt djwhitt
r57
a4b1324

Choose a tag to compare

View all tags
Release 57

This is a recommended release. This release focuses on improving gateway infrastructure with enhanced CDN compatibility and a new gateway-based offset discovery system. Key improvements include a new root transaction index using HEAD requests to AR.IO gateways, configurable Cache-Control headers for better CDN integration, and numerous bug fixes for proxy support. The release also includes extensive documentation improvements.

Added

  • GatewaysRootTxIndex for Offset Discovery: New root transaction index using HEAD requests to AR.IO gateways for discovering data item offsets
    • Multi-gateway support with priority tiers and automatic fallback (single attempt per gateway to prevent thundering herd)
    • Per-gateway rate limiting with TokenBucket
    • LRU caching for offset results
    • Configuration via GATEWAYS_ROOT_TX_URLS, GATEWAYS_ROOT_TX_REQUEST_TIMEOUT_MS, GATEWAYS_ROOT_TX_RATE_LIMIT_BURST_SIZE, GATEWAYS_ROOT_TX_RATE_LIMIT_TOKENS_PER_INTERVAL, GATEWAYS_ROOT_TX_RATE_LIMIT_INTERVAL, GATEWAYS_ROOT_TX_CACHE_SIZE
  • Configurable Cache-Control Private Directive: CDN compatibility via CACHE_PRIVATE_SIZE_THRESHOLD (default: 100 MB) and CACHE_PRIVATE_CONTENT_TYPES environment variables
    • Adds private directive to Cache-Control headers for content exceeding size threshold or matching content types
    • Ensures rate limiting and x402 payment requirements are enforced even when CDNs are deployed in front of ar-io-node
  • Enhanced Rate Limiting Observability: Client IP now logged separately in rate limit exceeded messages for better debugging and monitoring

Fixed

  • Proxy Support Fixes:
    • Fixed x402 resource URLs to use SANDBOX_PROTOCOL when behind reverse proxies/CDNs
    • Fixed inconsistent IP extraction between rate limiter bucket keys and allowlist checks
  • Chunk Endpoint Performance: Apply rate limits before expensive txResult lookup
    • Reordered operations to check rate limits first, improving performance under high load
  • Cache-Control Content Type Matching: Normalize content types by stripping parameters (e.g., text/html; charset=utf-8 → text/html)
    • Ensures proper Cache-Control header matching for configured content types

Documentation

  • Comprehensive rate limiting documentation cleanup (~200-300 lines of duplication removed)
  • Documented all 4 rate limit metrics (request, IP, chunk, x402 token consumption)
  • Added automated payment workflow testing examples for x402
  • Removed private key export recommendations from x402 testing examples
  • Clarified complete IP extraction fallback order for proxy scenarios
  • Clarified Cloudflare header extraction behavior
  • Removed redundant mentions of x402 requiring rate limiter

Docker Images

  • ar-io-envoy: ghcr.io/ar-io/ar-io-envoy:159d6467108122a3413c5ab45150d334dc9fb78f
  • ar-io-core: ghcr.io/ar-io/ar-io-core:08025d13a5bd1cb244d297a0cf48a5fa89ca8255
  • ar-io-clickhouse-auto-import: ghcr.io/ar-io/ar-io-clickhouse-auto-import:4512361f3d6bdc0d8a44dd83eb796fd88804a384
  • ar-io-litestream: ghcr.io/ar-io/ar-io-litestream:be121fc0ae24a9eb7cdb2b92d01f047039b5f5e8
  • ar-io-observer: ghcr.io/ar-io/ar-io-observer:7384807c660228579b312474090c47ea9b7727ec (unchanged from previous release)
Loading

Release 56

27 Oct 16:16
@djwhitt djwhitt
r56
c670cdc

Choose a tag to compare

View all tags
Release 56

This is a recommended release due to fixes for nested bundle offset calculations.

This release continues the x402 payment protocol expansion from Release 55, extending payment and rate limiting support to the chunk endpoint and adding comprehensive operator documentation. The /ar-io/info endpoint now exposes rate limiter and payment configuration for programmatic gateway discovery. This release also includes important fixes for nested bundle offset calculations that could affect data retrieval, making it a recommended upgrade for all operators.

Docker Images

  • ar-io-core: ghcr.io/ar-io/ar-io-core:96ed3a30f3a3623c487460d04a6fb3cd0f0eae84
  • ar-io-envoy: ghcr.io/ar-io/ar-io-envoy:159d6467108122a3413c5ab45150d334dc9fb78f
  • ar-io-clickhouse-auto-import: ghcr.io/ar-io/ar-io-clickhouse-auto-import:4512361f3d6bdc0d8a44dd83eb796fd88804a384
  • ar-io-litestream: ghcr.io/ar-io/ar-io-litestream:be121fc0ae24a9eb7cdb2b92d01f047039b5f5e8
  • ar-io-observer: ghcr.io/ar-io/ar-io-observer:7384807c660228579b312474090c47ea9b7727ec

Added

  • Chunk Endpoint Payment and Rate Limiting: Added x402 payment and rate limiting support to GET /chunk/:offset endpoint for gateway monetization and traffic control:
    • Uses fixed size assumption (~360 KiB) for predictable pricing without waiting for chunk retrieval
    • Configurable via CHUNK_GET_BASE64_SIZE_BYTES environment variable (default: 368,640 bytes)
    • HEAD requests consume one token (to prevent spam)
    • 304 Not Modified responses consume one token (to prevent spam)
    • Compatible with all existing x402 and rate limiter configuration
  • Configuration Validation: Added startup validation that ensures ENABLE_RATE_LIMITER=true when ENABLE_X_402_USDC_DATA_EGRESS=true. The application will fail to start with a clear error message if x402 is enabled without the rate limiter, since x402 payments require rate limiting to function (402 responses are only sent when rate limits are exceeded)
  • Gateway Info Endpoint: The /ar-io/info endpoint now exposes rate limiter and x402 payment configuration when these features are enabled. This allows clients to programmatically discover gateway capabilities, pricing, and limits. New optional response fields:
    • rateLimiter - Per-resource and per-IP bucket capacities, refill rates, and byte convenience fields (when ENABLE_RATE_LIMITER=true)
    • x402 - Payment network, wallet address, facilitator URL, per-byte pricing with min/max bounds, example costs for common sizes (1KB/1MB/1GB), and capacity multiplier for paid tier (when ENABLE_X_402_USDC_DATA_EGRESS=true)
  • x402 and Rate Limiter Documentation: Added comprehensive operator guide at docs/x402-and-rate-limiting.md covering x402 payment protocol and rate limiting configuration:
    • Configuration via .env files with detailed examples
    • Secrets management using volume mounts for Coinbase Develop Program credentials
    • Complete list of rate limited endpoints including GET /chunk/:offset
    • Token consumption patterns and pricing models for all endpoints
    • Integration with x402 facilitator services
    • Testing and troubleshooting guidance
  • Coinbase Developer Platform Environment Variables: Added environment variables for Coinbase Developer Platform (CDP) integration:
    • CDP_API_KEY_ID - CDP API key identifier
    • CDP_API_KEY_SECRET - CDP API secret key
    • CDP_API_KEY_SECRET_FILE - Load CDP secret from file for improved security

Changed

  • Glossary: Added new "Rate Limiter & x402 Payment Protocol" section consolidating related terms:
    • Facilitator - Payment verification and settlement service
    • Rate Limiter - Traffic control system overview
    • Rate Limiter Token Types - Paid vs regular token pools
    • Token Bucket Algorithm - Rate limiting algorithm details
    • x402 Protocol - HTTP 402 payment protocol definition
  • CDP Environment Variables: Refactored Coinbase Developer Platform API key configuration:
    • Removed X_402_CDP_CLIENT_KEY_FILE (client key is public, doesn't need file-based loading)
    • Split into separate CDP_API_KEY_ID and CDP_API_KEY_SECRET variables
    • Added CDP_API_KEY_SECRET_FILE for secure file-based loading of sensitive API secret

Fixed

  • Docker Compose Configuration: Added ENABLE_DATA_ITEM_ROOT_TX_SEARCH and ENABLE_PASSTHROUGH_WITHOUT_OFFSETS environment variables to docker-compose.yaml, .env.example, and docs/envs.md. These options control offset-aware data source behavior and were previously only defined in src/config.ts, making them unavailable for Docker Compose users to configure via .env files
  • Data Handler Rate Limiting: Fixed rate limiting for non-indexed data by:
    • Removing dataAttributes !== undefined check that prevented rate limiting before data indexing
    • Using data.size (always available) as primary source for content size calculation with fallback to dataAttributes?.size
    • Aligning content size calculation with actual Content-Length header values
    • Ensuring consistent rate limiting across all data endpoints (raw data, manifest, and bundled data)
  • Nested Bundle Data Item Offset Calculation: Fixed multiple offset calculation issues affecting nested bundle data item retrieval:
    • Corrected Turbo DynamoDB dataOffset to use absolute semantics (offset + headerSize) instead of relative semantics, ensuring consistency with bundle parsing and type documentation
    • Fixed rootDataItemOffset calculation to include target item's offset, not just parent dataOffset values
    • Fixed region boundary validation to handle offset=0 correctly (was incorrectly skipping validation due to truthy checks)
    • Added backward compatibility fallback for data without dataOffset attributes
  • 304 Not Modified Pre-Charging: Prevented 304 Not Modified responses from being pre-charged with full content size before checking If-None-Match headers. Now correctly predicts 304 responses and applies minimal token charge without denying legitimate cached requests

Full changelog available at: https://github.com/ar-io/ar-io-node/blob/main/CHANGELOG.md

Loading

Release 55

21 Oct 15:12
@djwhitt djwhitt
r55
7b65fb4

Choose a tag to compare

View all tags
Release 55

This is an optional release focused on x402 payment protocol improvements.

This release represents a major milestone in the gateway x402 payment protocol implementation. The x402 capabilities have evolved from an MVP supporting only limited data endpoints to a full, mostly feature-complete solution. The browser paywall now uses redirect mode to properly handle content-type metadata, and rate limiting has been extended to work correctly across all content delivery paths including manifests, ArNS names, and range requests.

Added

  • Token Consumption Metrics: New rate_limit_tokens_consumed_total Prometheus counter for monitoring rate limiter usage with labels:
    • bucket_type (ip/resource) - Which bucket consumed tokens
    • token_type (paid/regular) - Which token pool was used
    • domain - Domain consuming the tokens
    • Enables monitoring and alerting on token consumption patterns
  • Environment Variables:
    • RATE_LIMITER_TYPE: Configure rate limiter implementation ("memory" for development/testing, "redis" for production)
    • CDP_API_KEY_SECRET_FILE: Load CDP secret API key from file instead of environment variable for improved security (for Coinbase Onramp integration)
    • RATE_LIMITER_ARNS_ALLOWLIST: Comma-separated list of ArNS names that bypass rate limiting and payment verification

Changed

  • Token Consumption Priority: Changed token consumption order to prioritize regular tokens:
    • Regular tokens consumed first, then paid tokens
    • Paid tokens now act as overflow capacity instead of being consumed immediately
    • Paid token balance still provides bypass of per-resource rate limits
    • This change provides better value to paying users as paid tokens last longer
  • Rate Limiting and Payment Architecture: Refactored internal architecture for improved maintainability (no operator-visible behavior changes beyond those listed above)

Fixed

  • X402 Browser Paywall: Implemented redirect mode to fix blob URL content-type handling issues:
    • Browser requests now receive proper redirects after payment verification
    • Resolves content-type metadata loss that occurred with blob URLs
    • Preserves original content metadata in browser delivery
  • Rate Limiting for Manifests and ArNS: Fixed rate limits to correctly apply to manifest-resolved and ArNS resources:
    • Rate limits now apply after manifest resolution to actual content size
    • ArNS resources are now properly rate limited
    • Ensures consistent rate limiting across all content delivery paths
  • Range Request Token Consumption: Fixed rate limiter to charge tokens based on actual bytes served in range requests instead of full content size
  • Rate Limiter Token Tracking: Fixed internal token bucket tracking to properly record consumption in all edge cases
  • Token Consumption for Non-Data Responses: Prevented token consumption for 304 Not Modified and HEAD responses which don't transfer content data

Docker Images

  • envoy: ghcr.io/ar-io/ar-io-envoy:159d6467108122a3413c5ab45150d334dc9fb78f
  • core: ghcr.io/ar-io/ar-io-core:da77a3da2aa0e1f637b5a89189c79af7b3ebcc63
  • clickhouse-auto-import: ghcr.io/ar-io/ar-io-clickhouse-auto-import:4512361f3d6bdc0d8a44dd83eb796fd88804a384
  • observer: ghcr.io/ar-io/ar-io-observer:7384807c660228579b312474090c47ea9b7727ec
  • litestream: ghcr.io/ar-io/ar-io-litestream:be121fc0ae24a9eb7cdb2b92d01f047039b5f5e8
Loading

Release 54

13 Oct 18:47
@djwhitt djwhitt
r54
02bd70a

Choose a tag to compare

View all tags
Release 54

This is a recommended release due to the improvements to chunk observation
and retrieval. The release enhances peer selection for chunk operations,
introduces experimental X402 payment protocol support, and enables offset
observation enforcement by default to strengthen network reliability.

Added

  • X402 Payment Protocol (Experimental): Optional USDC-based payment
    system for accessing rate-limited content. This feature is experimental
    and will be rapidly built out in upcoming releases.
    • Dynamic content-based pricing (default: $0.0000000001/byte = $0.10/GB)
    • USDC payments via Coinbase facilitator on Base network (mainnet and
      testnet supported)
    • Rate limiter integration with 10x capacity multiplier for paid tier
    • Proportional bucket top-off capped to actual price paid
    • HTML paywall for browser clients, JSON responses for API clients
    • Settlement timeout protection (5s default)
    • Configuration via 13 new X_402_* environment variables (see
      .env.example for details)
    • Note: Currently only applies to /<id> and /raw/<id> endpoints
      for non-manifests
  • Expanded default preferred chunk GET node pool from 12 to 22 nodes,
    adding data-13 through data-17 (5 additional data nodes) and tip-1
    through tip-5 (5 tip nodes) for improved redundancy and load
    distribution across the Arweave network.
  • Added GraphQL as third fallback option in ROOT_TX_LOOKUP_ORDER (after
    db and turbo) to prevent lookup failures when Turbo's circuit breaker
    is open, enabling more resilient root transaction discovery.
  • Added deterministic weight-based peer selection for chunk operations
    (both GET and POST). Peers are now sorted by weight in descending order
    and top N selected, ensuring preferred peers (weight 100) are always
    tried first instead of probabilistic selection.
  • Added preferred peer weight preservation for chunk operations to prevent
    weight degradation for operator-configured preferred peers during
    temporary failures. Preferred peers maintain their initial weight (100)
    regardless of success/failure, honoring operator configuration while
    allowing discovered peers to adapt based on performance.

Changed

  • Observer: Enabled offset observation enforcement by default.
    OFFSET_OBSERVATION_ENFORCEMENT_ENABLED now defaults to true instead
    of false. Gateway assessments will fail if offset validation fails,
    strengthening network reliability requirements. Operators can opt-out by
    explicitly setting OFFSET_OBSERVATION_ENFORCEMENT_ENABLED=false.
  • Reduced logging verbosity by moving DNS resolution and sync bucket
    operational logs from debug/info to silly level. DNS resolution messages
    ('Resolving hostname', 'Resolved IPv4/IPv6 addresses') and sync bucket
    updates ('Parsed ETF sync buckets', 'Updated sync buckets') now use
    silly level, while completion and peer selection messages remain at
    debug level for visibility.

Fixed

  • Fixed preferred peer weight preservation to only apply to chunk
    operations (GET/POST) instead of all operation categories. Previously,
    preferred chunk peers maintained constant weight across chain, getChunk,
    and postChunk operations. Now preferred chunk peers can undergo normal
    warming/cooling when used for chain operations, preventing indefinite
    selection of peers that perform poorly for chain operations while still
    maintaining constant weight for chunk operations.
  • Fixed ANS-104 data item header parsing for Ethereum signatures (type 3)
    by using correct 65-byte uncompressed public key length instead of
    20-byte address length. This resolves "Invalid buffer" errors when
    parsing Ethereum-signed data items. Also updated
    MAX_DATA_ITEM_HEADER_SIZE from 6228 to 8257 bytes to account for
    MultiAptos signature type (largest supported), and replaced custom
    signature/owner length methods with getSignatureMeta() from arbundles
    library for consistency.
  • Fixed root TX discovery to use non-blocking rate limiting instead of
    blocking when rate limits are reached. Services now use
    tryRemoveTokens() and skip rate-limited gateways/sources immediately
    rather than waiting indefinitely, preventing request blocking and
    improving responsiveness. Also fixed GraphQL service to return
    dataSize instead of incorrect size field.

Known Issues

  • The x402 browser paywall currently uses blob URLs for content delivery
    after successful payment. This causes issues with content-type handling
    and browser behavior as the blob URL doesn't preserve the original
    content metadata. We plan to fix this in upcoming releases by
    either contributing to the x402 SDK to add a page reload option, or
    implementing a custom paywall template that properly handles redirects
    after payment verification.
Loading