-
Notifications
You must be signed in to change notification settings - Fork 15
feat(apps/ensadmin): better handle projections and outdated snapshots #1251
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
d21549a
7e55416
281a044
add646c
046fd2a
a398f1c
010e11f
877cdf2
7d3367a
c4265aa
9d5b542
83f20a6
523d475
cce1037
21707a1
f90d6ab
c2c8831
b10ef04
55a6a87
467d159
393626e
13209c1
ea42b6c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| --- | ||
| "@ensnode/ensnode-react": minor | ||
| "ensadmin": minor | ||
| --- | ||
|
|
||
| Enhance useIndexingStatus with in-memory snapshot caching and real-time projection updates for smoother, continuously refreshed indexing status between API fetches |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -33,6 +33,7 @@ import { cn } from "@/lib/utils"; | |
| import { BackfillStatus } from "./backfill-status"; | ||
| import { BlockStats } from "./block-refs"; | ||
| import { IndexingStatusLoading } from "./indexing-status-loading"; | ||
| import { ProjectionInfo } from "./projection-info"; | ||
|
|
||
| interface IndexingStatsForOmnichainStatusSnapshotProps< | ||
| OmnichainIndexingStatusSnapshotType extends | ||
|
|
@@ -323,22 +324,32 @@ export function IndexingStatsForSnapshotFollowing({ | |
| */ | ||
| export function IndexingStatsShell({ | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is a general comment with overall feedback. Not about this file or this line necessarily. I continue to experience the core issue described in #1200 Here's a screenshot from my testing of the latest Vercel Preview of this PR.
The backstory of this screenshot is that I am testing by keeping my browser window open to this page and then just waiting for the issue to reproduce itself. https://adminensnode-cbuey61q7-namehash.vercel.app/status?connection=https%3A%2F%2Fapi.alpha.green.ensnode.io%2F What's happening is that the indexing status is loaded into memory and therefore the "Status" page is correctly rendering itself. Then in the background, the requests for indexing status are happening automatically every few seconds..
which then causes this to appear in the UI:
This needs to be completely impossible. We should continue using the existing snapshot that's already cached into memory. It's still perfectly good!
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This shouldn't happen anymore. |
||
| omnichainStatus, | ||
| realtimeProjection, | ||
| children, | ||
| }: PropsWithChildren<{ omnichainStatus?: OmnichainIndexingStatusId }>) { | ||
| }: PropsWithChildren<{ | ||
| omnichainStatus?: OmnichainIndexingStatusId; | ||
| realtimeProjection?: RealtimeIndexingStatusProjection; | ||
| }>) { | ||
| return ( | ||
| <Card className="w-full flex flex-col gap-2"> | ||
| <CardHeader> | ||
| <CardTitle className="flex gap-2 items-center"> | ||
| <span>Indexing Status</span> | ||
| <CardTitle className="flex gap-2 items-center justify-between"> | ||
| <div className="flex gap-2 items-center"> | ||
| <span>Indexing Status</span> | ||
|
|
||
| {omnichainStatus && ( | ||
| <Badge | ||
| className={cn("uppercase text-xs leading-none")} | ||
| title={`Omnichain indexing status: ${formatOmnichainIndexingStatus( | ||
| omnichainStatus, | ||
| )}`} | ||
| > | ||
| {formatOmnichainIndexingStatus(omnichainStatus)} | ||
| </Badge> | ||
| )} | ||
| </div> | ||
|
|
||
| {omnichainStatus && ( | ||
| <Badge | ||
| className={cn("uppercase text-xs leading-none")} | ||
| title={`Omnichain indexing status: ${formatOmnichainIndexingStatus(omnichainStatus)}`} | ||
| > | ||
| {formatOmnichainIndexingStatus(omnichainStatus)} | ||
| </Badge> | ||
| )} | ||
| {realtimeProjection && <ProjectionInfo realtimeProjection={realtimeProjection} />} | ||
| </CardTitle> | ||
| </CardHeader> | ||
|
|
||
|
|
@@ -422,7 +433,10 @@ export function IndexingStatsForRealtimeStatusProjection({ | |
| <section className="flex flex-col gap-6"> | ||
| {maybeIndexingTimeline} | ||
|
|
||
| <IndexingStatsShell omnichainStatus={omnichainStatusSnapshot.omnichainStatus}> | ||
| <IndexingStatsShell | ||
| omnichainStatus={omnichainStatusSnapshot.omnichainStatus} | ||
| realtimeProjection={realtimeProjection} | ||
| > | ||
| {indexingStats} | ||
| </IndexingStatsShell> | ||
| </section> | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| "use client"; | ||
|
|
||
| import { InfoIcon } from "lucide-react"; | ||
|
|
||
| import type { RealtimeIndexingStatusProjection } from "@ensnode/ensnode-sdk"; | ||
|
|
||
| import { RelativeTime } from "@/components/datetime-utils"; | ||
| import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip"; | ||
|
|
||
| interface ProjectionInfoProps { | ||
| realtimeProjection: RealtimeIndexingStatusProjection; | ||
| } | ||
|
|
||
| /** | ||
| * Displays metadata about the current indexing status projection in a tooltip. | ||
| * Shows when the projection was generated, when the snapshot was taken, and worst-case distance. | ||
| */ | ||
| export function ProjectionInfo({ realtimeProjection }: ProjectionInfoProps) { | ||
| const { projectedAt, snapshot, worstCaseDistance } = realtimeProjection; | ||
| const { snapshotTime } = snapshot; | ||
|
|
||
| return ( | ||
| <Tooltip delayDuration={300}> | ||
| <TooltipTrigger asChild> | ||
| <button | ||
| type="button" | ||
| className="inline-flex items-center justify-center rounded-md text-sm font-medium transition-colors hover:bg-accent hover:text-accent-foreground h-8 w-8" | ||
| aria-label="Indexing Status Metadata" | ||
| > | ||
| <InfoIcon className="h-4 w-4" /> | ||
| </button> | ||
| </TooltipTrigger> | ||
| <TooltipContent | ||
| side="right" | ||
| className="bg-gray-50 text-sm text-black shadow-md outline-none w-80 p-4" | ||
| > | ||
| <div className="flex flex-col gap-3"> | ||
| <div className="flex flex-col gap-1"> | ||
| <div className="font-semibold text-xs text-gray-500 uppercase"> | ||
| Worst-Case Distance* | ||
| </div> | ||
| <div className="text-sm"> | ||
| {worstCaseDistance !== null ? `${worstCaseDistance} seconds` : "N/A"} | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there a special reason for this null check? Shouldn't this value always be guaranteed to be defined? |
||
| </div> | ||
| </div> | ||
|
|
||
| <div className="text-xs text-gray-600 leading-relaxed"> | ||
| * as of real-time projection generated{" "} | ||
| <RelativeTime timestamp={projectedAt} includeSeconds conciseFormatting /> from indexing | ||
| status snapshot captured{" "} | ||
| <RelativeTime timestamp={snapshotTime} includeSeconds conciseFormatting />. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| </div> | ||
| </div> | ||
| </TooltipContent> | ||
| </Tooltip> | ||
| ); | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,15 +1,50 @@ | ||
| import { useQuery } from "@tanstack/react-query"; | ||
| import { keepPreviousData, useQuery } from "@tanstack/react-query"; | ||
| import { useMemo } from "react"; | ||
|
|
||
| import type { IndexingStatusRequest, IndexingStatusResponse } from "@ensnode/ensnode-sdk"; | ||
| import { | ||
| createRealtimeIndexingStatusProjection, | ||
| type IndexingStatusRequest, | ||
| type IndexingStatusResponse, | ||
| IndexingStatusResponseCodes, | ||
| } from "@ensnode/ensnode-sdk"; | ||
|
|
||
| import type { QueryParameter, WithSDKConfigParameter } from "../types"; | ||
| import { createIndexingStatusQueryOptions } from "../utils/query"; | ||
| import { useENSNodeSDKConfig } from "./useENSNodeSDKConfig"; | ||
| import { useNow } from "./useNow"; | ||
|
|
||
| interface UseIndexingStatusParameters | ||
| extends IndexingStatusRequest, | ||
| QueryParameter<IndexingStatusResponse> {} | ||
|
|
||
| /** | ||
| * Hook for fetching and tracking indexing status with client-side projection updates. | ||
| * | ||
| * Clients often need frequently updated worst-case distance for their logic, | ||
| * but calling the API every second would be inefficient. Instead, we fetch a | ||
| * snapshot and keep it in memory. We then asynchronously attempt to update it every 10 seconds. | ||
| * | ||
| * From the most recently cached snapshot, this hook instantly generates new projections — | ||
| * entirely in memory. Each projection provides a recalculation of worst-case distance based on: | ||
| * • The current time (when the projection was generated) | ||
| * • The snapshot's absolute timestamps of recorded indexing progress | ||
| * | ||
| * This works reliably because indexing progress is virtually always non-decreasing over | ||
| * time (virtually never goes backward). Clients can safely assume that a snapshot from a | ||
| * few seconds ago is still valid for building new projections. Since snapshots | ||
| * exclusively contain absolute timestamps, we can reuse a snapshot across time to continuously compute updated worst-case projections without additional API calls. | ||
| * | ||
| * **Error Handling:** | ||
| * When the indexing status API returns an error response, this hook continues to display | ||
| * the last successful snapshot while projecting forward in time. This provides a graceful | ||
| * degradation experience - the UI shows slightly stale but still useful data rather than | ||
| * breaking completely. | ||
| * | ||
| * @param parameters - Configuration options | ||
| * @param parameters.config - ENSNode SDK configuration (optional, uses context if not provided) | ||
| * @param parameters.query - TanStack Query options for customizing query behavior (refetchInterval, enabled, etc.) | ||
| * @returns TanStack Query result containing a new indexing status projection based on the current time | ||
| */ | ||
| export function useIndexingStatus( | ||
| parameters: WithSDKConfigParameter & UseIndexingStatusParameters = {}, | ||
| ) { | ||
|
|
@@ -21,9 +56,51 @@ export function useIndexingStatus( | |
| const options = { | ||
| ...queryOptions, | ||
| refetchInterval: 10 * 1000, // 10 seconds - indexing status changes frequently | ||
| placeholderData: keepPreviousData, // Keep showing previous data during refetch and on error | ||
| ...query, | ||
| enabled: query.enabled ?? queryOptions.enabled, | ||
| }; | ||
|
|
||
| return useQuery(options); | ||
| const queryResult = useQuery(options); | ||
|
|
||
| // Extract the current snapshot from the query result. | ||
| // Thanks to placeholderData: keepPreviousData, this will continue showing | ||
| // the last successful snapshot even when subsequent fetches fail. | ||
| // Note: queryFn now throws on error responses, so data will only contain valid responses | ||
|
|
||
| // debug | ||
| const currentSnapshot = | ||
| queryResult.data?.responseCode === IndexingStatusResponseCodes.Ok | ||
| ? queryResult.data.realtimeProjection.snapshot | ||
| : null; | ||
|
|
||
| // / worstCaseDistance is measured in seconds | ||
|
|
||
| // Get current timestamp that updates every second. | ||
| // Each component instance gets its own timestamp | ||
| const projectedAt = useNow(1000); | ||
|
|
||
| // Generate projection from cached snapshot using the synchronized timestamp. | ||
| // useMemo ensures we only create a new projection object when values actually change, | ||
| // maintaining referential equality for unchanged data (prevents unnecessary re-renders). | ||
| const projectedData = useMemo(() => { | ||
| if (!currentSnapshot) return null; | ||
|
|
||
| const realtimeProjection = createRealtimeIndexingStatusProjection(currentSnapshot, projectedAt); | ||
|
|
||
| return { | ||
| // debugging | ||
| responseCode: "ok" as const, | ||
| realtimeProjection, | ||
| } satisfies IndexingStatusResponse; | ||
| }, [currentSnapshot, projectedAt]); | ||
|
|
||
| if (projectedData) { | ||
| return { | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't necessarily define a return type above for I'd like to explore this separately so it doesn't block this PR.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok thanks. I'm not able to share advice on this. Seeking your leadership here. Thanks. |
||
| ...queryResult, | ||
| data: projectedData, | ||
| }; | ||
| } | ||
|
|
||
| return queryResult; | ||
| } | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was going to ditch this and use |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| import { getUnixTime } from "date-fns"; | ||
| import { useEffect, useState } from "react"; | ||
|
|
||
| /** | ||
| * Hook that returns the current Unix timestamp, updated at a specified interval. | ||
| * | ||
| * @param refreshRate - How often to update the timestamp in milliseconds (default: 1000ms) | ||
| * @returns Current Unix timestamp that updates every refreshRate milliseconds | ||
| * | ||
| * @example | ||
| * ```tsx | ||
| * // Updates every second | ||
| * const now = useNow(1000); | ||
| * | ||
| * // Updates every 5 seconds | ||
| * const now = useNow(5000); | ||
| * ``` | ||
| */ | ||
| export function useNow(refreshRate = 1000): number { | ||
| const [now, setNow] = useState(() => getUnixTime(new Date())); | ||
|
|
||
| useEffect(() => { | ||
| const interval = setInterval(() => { | ||
| setNow(getUnixTime(new Date())); | ||
| }, refreshRate); | ||
|
|
||
| return () => clearInterval(interval); | ||
| }, [refreshRate]); | ||
|
|
||
| return now; | ||
| } |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.




There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could use
rookshere I guess withuseIntervalWhenfor cleaner code. Since this component is used elsewhere, I'm not sure if the original implementation was designed without this improvement.