From 461219b552ba13c2855eaf89b071b177ea456e94 Mon Sep 17 00:00:00 2001 From: Milinda Dias Date: Wed, 21 Jan 2026 19:22:29 +0530 Subject: [PATCH 1/9] feat: cache warmer in memory switchover --- docs/concepts/cache-warmer.mdx | 38 ++++++++++++++++++++ docs/router/configuration.mdx | 26 ++++++++------ docs/router/deployment/config-hot-reload.mdx | 6 ++-- 3 files changed, 57 insertions(+), 13 deletions(-) diff --git a/docs/concepts/cache-warmer.mdx b/docs/concepts/cache-warmer.mdx index a35bdb08..c21158e7 100644 --- a/docs/concepts/cache-warmer.mdx +++ b/docs/concepts/cache-warmer.mdx @@ -69,3 +69,41 @@ Users can manually recompute slow queries from the Cosmo Studio. Currently, reco + +## In-Memory Switchover Cache Warming + +The in-memory switchover feature provides automatic cache persistence across router restarts, ensuring zero cold starts when deploying new configurations or updates. + +### How It Works + +The router can restart due to two reasons, either due to a config change or due to a schema change. Due to the structure of the router internals we have two slight variations on how we handle the in-memory switchover cache warming: + +1. **Before Shutdown**: In case of config changes, the router extracts all queries from the current plan cache, preserving the "hot" queries that were actively being used. + +2. **During Startup**: The router with the updated config receives the queries from the previous plan cache and uses them to warm up its plan cache before serving traffic. + +3. **Result**: The updated router starts with a fully warmed cache, eliminating latency spikes that would normally occur during cold starts. + +When using the in memory switchover, the first start will still experience a cold start, as there is no prior populated planner cache. Subsequent restarts will benefit from the in-memory switchover. + +### When can you use the In-Memory Switchover + +The in-memory switchover is a fallback method. Which means if your cosmo account has access to the Cosmo Cloud CDN cache warmer, that will be used instead of the in-memory switchover. However, if this is not the case, the in-memory switchover will be used as the primary cache warming method. + + + The in-memory switchover won't be used as a fallback for sources other than the Cosmo Cloud CDN cache warmer, which is the default source. + + +### Configuration + +The in-memory switchover is **enabled by default when cache warming is enabled**. It automatically activates when: + +* Cache warming is enabled on the router configuration (`cache_warmup.enabled: true`) +* Cosmo CDN cache warmer is not available or disabled for the account +* `in_memory_switchover_fallback` is set to `true` (default) + +```yaml +cache_warmup: + enabled: true + in_memory_switchover_fallback: true # Enabled by default +``` diff --git a/docs/router/configuration.mdx b/docs/router/configuration.mdx index 72cef7eb..fcd33e89 100644 --- a/docs/router/configuration.mdx +++ b/docs/router/configuration.mdx @@ -2173,13 +2173,14 @@ apollo_router_compatibility_flags: ## Cache warmer -| Environment Variable | YAML | Required | Description | Default Value | -| -------------------- | ---------------- | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | -| | enabled | | Enable the cache warmer. | false | -| | workers | | The number of workers for the cache warmup to run in parallel. Higher numbers decrease the time to warm up the cache but increase the load on the system. | 8 | -| | items_per_second | | The number of cache warmup items to process per second. Higher numbers decrease the time to warm up the cache but increase the load on the system. | 50 | -| | timeout | | The timeout for warming up the cache. This can be used to limit the amount of time cache warming will block deploying a new config. The period is specified as a string with a number and a unit, e.g. 10ms, 1s, 1m, 1h. The supported units are 'ms', 's', 'm', 'h'. | 30s | -| | source | | The source of the cache warmup items. Only one can be specified. If empty, the cache warmup source is the Cosmo CDN and it requires a graph to be set. | | +| Environment Variable | YAML | Required | Description | Default Value | +| --------------------------------------------------- | ------------------------------ | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | +| CACHE_WARMUP_ENABLED | enabled | | Enable the cache warmer. | false | +| CACHE_WARMUP_ITEMS_PER_SECOND | items_per_second | | The number of cache warmup items to process per second. Higher numbers decrease the time to warm up the cache but increase the load on the system. | 50 | +| CACHE_WARMUP_TIMEOUT | timeout | | The timeout for warming up the cache. This can be used to limit the amount of time cache warming will block deploying a new config. The period is specified as a string with a number and a unit, e.g. 10ms, 1s, 1m, 1h. The supported units are 'ms', 's', 'm', 'h'. | 30s | +| CACHE_WARMUP_WORKERS | workers | | The number of workers for the cache warmup to run in parallel. Higher numbers decrease the time to warm up the cache but increase the load on the system. | 8 | +| | source | | The source of the cache warmup items. Only one can be specified. If empty, the cache warmup source is the Cosmo CDN and it requires a graph to be set. | | +| CACHE_WARMUP_IN_MEMORY_SWITCHOVER_FALLBACK_ENABLED | in_memory_switchover_fallback | | Enable in-memory switchover fallback that persists the plan cache across router restarts. When enabled, the router will use queries from the previous instance's plan cache to warm up the new cache on restart. Applicable when Cosmo is used but the cache warmer on cosmo is not enabled. | true | ### Example YAML config: @@ -2191,15 +2192,16 @@ cache_warmup: workers: 8 items_per_second: 50 timeout: 30s + in_memory_switchover_fallback: true ``` ### Source The source of the cache warmup items. Only one can be specified. If empty, the cache warmup source is the Cosmo CDN and it requires a graph to be set. -| Environment Variable | YAML | Required | Description | Default Value | -| -------------------- | ---- | ---------------------- | ------------------------------------------------------------ | ------------- | -| | path | | The path to the directory containing the cache warmup items. | | +| Environment Variable | YAML | Required | Description | Default Value | +| -------------------- | -------------- | ---------------------- | ------------------------------------------------------------ | ------------- | +| | filesystem.path | | The path to the directory containing the cache warmup items. | | ### Example YAML config: @@ -2211,6 +2213,8 @@ cache_warmup: workers: 8 items_per_second: 50 timeout: 30s + in_memory_switchover_fallback: true source: - path: "./cache-warmer/operations" + filesystem: + path: "./cache-warmer/operations" ``` diff --git a/docs/router/deployment/config-hot-reload.mdx b/docs/router/deployment/config-hot-reload.mdx index daa467a3..439b878e 100644 --- a/docs/router/deployment/config-hot-reload.mdx +++ b/docs/router/deployment/config-hot-reload.mdx @@ -38,6 +38,8 @@ This will behave similarly to the CDN polling with regard to router lifecycle. WebSocket or SSE (Server-Sent Events) connections are long-lived and will be closed immediately after the active graph instance is shut down. Clients must reconnect to restore the connection. -### Latency Implications +### Latency Implications and Cache Warming -When the configuration is swapped, a new graph is created internally. This implies that the planner cache is invalidated, as a plan is coupled to a specific graph. In high-traffic scenarios, this can increase latency. We are currently working on a solution to mitigate this by warming up the cache with the latest top 100 operations during the initial router start and configuration changes. +When the configuration is swapped, a new graph is created internally. This implies that the planner cache is invalidated, as a plan is coupled to a specific graph. In high-traffic scenarios, this can increase latency. + +To mitigate this issue, the router supports **cache warming**. For more details on cache warming, see the [Cache Warmer documentation](/concepts/cache-warmer). From 57e856ce0dde318a0a305e91f32811be717bc046 Mon Sep 17 00:00:00 2001 From: Milinda Dias <83293842+SkArchon@users.noreply.github.com> Date: Fri, 30 Jan 2026 18:35:42 +0530 Subject: [PATCH 2/9] fix: review comments --- docs/concepts/cache-warmer.mdx | 41 ++++++++++++++++++++++++++++------ 1 file changed, 34 insertions(+), 7 deletions(-) diff --git a/docs/concepts/cache-warmer.mdx b/docs/concepts/cache-warmer.mdx index c21158e7..081fb0b7 100644 --- a/docs/concepts/cache-warmer.mdx +++ b/docs/concepts/cache-warmer.mdx @@ -78,7 +78,7 @@ The in-memory switchover feature provides automatic cache persistence across rou The router can restart due to two reasons, either due to a config change or due to a schema change. Due to the structure of the router internals we have two slight variations on how we handle the in-memory switchover cache warming: -1. **Before Shutdown**: In case of config changes, the router extracts all queries from the current plan cache, preserving the "hot" queries that were actively being used. +1. **Before Shutdown**: In case of config changes (from hot config reloading), the router extracts all queries from the current plan cache, preserving the queries that were in the planner cache before the cache is cleared for shutdown. 2. **During Startup**: The router with the updated config receives the queries from the previous plan cache and uses them to warm up its plan cache before serving traffic. @@ -88,22 +88,49 @@ When using the in memory switchover, the first start will still experience a col ### When can you use the In-Memory Switchover -The in-memory switchover is a fallback method. Which means if your cosmo account has access to the Cosmo Cloud CDN cache warmer, that will be used instead of the in-memory switchover. However, if this is not the case, the in-memory switchover will be used as the primary cache warming method. +The in-memory switchover can be used either as a fallback method or as a primary method for cache warming. This depends on the router configuration, the defaults provided by the router enables it as a fallback to the Cosmo Cloud CDN cache warmer, which is a Cosmo enterprise feature. + +In case the in-memory switchover is used with the Cosmo Cloud CDN cache warmer, whenever either +* Getting the list of operations from the CDN fails +* The request to the CDN succeeded but did not have a list of operations in it + +We will use the fallback and load the list of operations from the in-memory switchover (if any operations exists). - The in-memory switchover won't be used as a fallback for sources other than the Cosmo Cloud CDN cache warmer, which is the default source. + The in-memory switchover cannot be used as a fallback for sources other than the Cosmo Cloud CDN cache warmer. ### Configuration -The in-memory switchover is **enabled by default when cache warming is enabled**. It automatically activates when: +The in-memory switchover can be enabled as a fallback for the CDN cache warmer, in order to do this you need to ensure + +* Cache warming is enabled on the router configuration (`cache_warmup.enabled: true`) +* `source.cdn.enabled` is set to `true` (This is true by default, and does not need to be explicitly specified) +* `in_memory_fallback` is set to `true` (default) + +```yaml +cache_warmup: + enabled: true + in_memory_fallback: true # Enabled by default + + # This is implicitly enabled by default, you do not need to explictly specify the following + source: + cdn: + enabled: true +``` + +However you may want to use the in-memory switchover without the CDN cache warmer, as this is an enterprise feature. In order to do this you need to ensure * Cache warming is enabled on the router configuration (`cache_warmup.enabled: true`) -* Cosmo CDN cache warmer is not available or disabled for the account -* `in_memory_switchover_fallback` is set to `true` (default) +* `source.cdn.enabled` is set to `false` (This needs to be specified explicitly as the default is false) +* `in_memory_fallback` is set to `true` (default) ```yaml cache_warmup: enabled: true - in_memory_switchover_fallback: true # Enabled by default + in_memory_fallback: true # Enabled by default + + source: + cdn: + enabled: false ``` From 2fbd5032ebc2a816a1346cc4816cb0b591134608 Mon Sep 17 00:00:00 2001 From: Milinda Dias <83293842+SkArchon@users.noreply.github.com> Date: Fri, 30 Jan 2026 18:36:35 +0530 Subject: [PATCH 3/9] fix: review comments --- docs/concepts/cache-warmer.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/concepts/cache-warmer.mdx b/docs/concepts/cache-warmer.mdx index 081fb0b7..95048431 100644 --- a/docs/concepts/cache-warmer.mdx +++ b/docs/concepts/cache-warmer.mdx @@ -88,7 +88,7 @@ When using the in memory switchover, the first start will still experience a col ### When can you use the In-Memory Switchover -The in-memory switchover can be used either as a fallback method or as a primary method for cache warming. This depends on the router configuration, the defaults provided by the router enables it as a fallback to the Cosmo Cloud CDN cache warmer, which is a Cosmo enterprise feature. +The in-memory switchover can be used either as a fallback method or as a primary method for cache warming. This depends on the router configuration, the defaults provided by the router enables it as a fallback to the Cosmo Cloud CDN cache warmer, which is a Cosmo enterprise feature that needs to be enabled explicitly in Cosmo. In case the in-memory switchover is used with the Cosmo Cloud CDN cache warmer, whenever either * Getting the list of operations from the CDN fails From c8172dd54bac1f245656871a3f5ac10e4345f175 Mon Sep 17 00:00:00 2001 From: Milinda Dias <83293842+SkArchon@users.noreply.github.com> Date: Fri, 30 Jan 2026 18:42:52 +0530 Subject: [PATCH 4/9] fix: review comments --- docs/router/configuration.mdx | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/router/configuration.mdx b/docs/router/configuration.mdx index fcd33e89..b4eb0470 100644 --- a/docs/router/configuration.mdx +++ b/docs/router/configuration.mdx @@ -2180,7 +2180,7 @@ apollo_router_compatibility_flags: | CACHE_WARMUP_TIMEOUT | timeout | | The timeout for warming up the cache. This can be used to limit the amount of time cache warming will block deploying a new config. The period is specified as a string with a number and a unit, e.g. 10ms, 1s, 1m, 1h. The supported units are 'ms', 's', 'm', 'h'. | 30s | | CACHE_WARMUP_WORKERS | workers | | The number of workers for the cache warmup to run in parallel. Higher numbers decrease the time to warm up the cache but increase the load on the system. | 8 | | | source | | The source of the cache warmup items. Only one can be specified. If empty, the cache warmup source is the Cosmo CDN and it requires a graph to be set. | | -| CACHE_WARMUP_IN_MEMORY_SWITCHOVER_FALLBACK_ENABLED | in_memory_switchover_fallback | | Enable in-memory switchover fallback that persists the plan cache across router restarts. When enabled, the router will use queries from the previous instance's plan cache to warm up the new cache on restart. Applicable when Cosmo is used but the cache warmer on cosmo is not enabled. | true | +| CACHE_WARMUP_IN_MEMORY_SWITCHOVER_ENABLED | in_memory_switchover | | Enable in-memory switchover that persists the plan cache across router restarts. When enabled, the router will use queries from the previous instance's plan cache to warm up the new cache on restart. Can be used as a fallback or as the primary source for warming the CDN. | true | ### Example YAML config: @@ -2192,7 +2192,7 @@ cache_warmup: workers: 8 items_per_second: 50 timeout: 30s - in_memory_switchover_fallback: true + in_memory_switchover: true ``` ### Source @@ -2202,6 +2202,7 @@ The source of the cache warmup items. Only one can be specified. If empty, the c | Environment Variable | YAML | Required | Description | Default Value | | -------------------- | -------------- | ---------------------- | ------------------------------------------------------------ | ------------- | | | filesystem.path | | The path to the directory containing the cache warmup items. | | +| | cdn.enabled | | Whether the CDN is the source for warming the cache. | true | ### Example YAML config: @@ -2215,6 +2216,8 @@ cache_warmup: timeout: 30s in_memory_switchover_fallback: true source: + cdn: + enabled: true filesystem: path: "./cache-warmer/operations" ``` From 46b8bc066befec720522936cce966fc1c375b891 Mon Sep 17 00:00:00 2001 From: Milinda Dias <83293842+SkArchon@users.noreply.github.com> Date: Fri, 30 Jan 2026 18:44:15 +0530 Subject: [PATCH 5/9] fix: review comments --- docs/concepts/cache-warmer.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/concepts/cache-warmer.mdx b/docs/concepts/cache-warmer.mdx index 95048431..c7a66d82 100644 --- a/docs/concepts/cache-warmer.mdx +++ b/docs/concepts/cache-warmer.mdx @@ -113,7 +113,7 @@ cache_warmup: enabled: true in_memory_fallback: true # Enabled by default - # This is implicitly enabled by default, you do not need to explictly specify the following + # This is implicitly enabled by default, you do not need to explicitly specify the following source: cdn: enabled: true From 614e2fa474a0309e63d9660504777bb801c872a6 Mon Sep 17 00:00:00 2001 From: Milinda Dias Date: Sat, 31 Jan 2026 02:32:37 +0530 Subject: [PATCH 6/9] fix: review comments --- docs/router/configuration.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/router/configuration.mdx b/docs/router/configuration.mdx index b4eb0470..4cf2ecab 100644 --- a/docs/router/configuration.mdx +++ b/docs/router/configuration.mdx @@ -2180,7 +2180,7 @@ apollo_router_compatibility_flags: | CACHE_WARMUP_TIMEOUT | timeout | | The timeout for warming up the cache. This can be used to limit the amount of time cache warming will block deploying a new config. The period is specified as a string with a number and a unit, e.g. 10ms, 1s, 1m, 1h. The supported units are 'ms', 's', 'm', 'h'. | 30s | | CACHE_WARMUP_WORKERS | workers | | The number of workers for the cache warmup to run in parallel. Higher numbers decrease the time to warm up the cache but increase the load on the system. | 8 | | | source | | The source of the cache warmup items. Only one can be specified. If empty, the cache warmup source is the Cosmo CDN and it requires a graph to be set. | | -| CACHE_WARMUP_IN_MEMORY_SWITCHOVER_ENABLED | in_memory_switchover | | Enable in-memory switchover that persists the plan cache across router restarts. When enabled, the router will use queries from the previous instance's plan cache to warm up the new cache on restart. Can be used as a fallback or as the primary source for warming the CDN. | true | +| CACHE_WARMUP_IN_MEMORY_FALLBACK | in_memory_fallback | | Enable in-memory switchover that persists the plan cache across router restarts. When enabled, the router will use queries from the previous instance's plan cache to warm up the new cache on restart. Can be used as a fallback or as the primary source for warming the CDN. | true | ### Example YAML config: @@ -2192,7 +2192,7 @@ cache_warmup: workers: 8 items_per_second: 50 timeout: 30s - in_memory_switchover: true + in_memory_fallback: true ``` ### Source @@ -2214,7 +2214,7 @@ cache_warmup: workers: 8 items_per_second: 50 timeout: 30s - in_memory_switchover_fallback: true + in_memory_fallback: true source: cdn: enabled: true From 76edfcd2b7aa2b2e017ddc92c52c875824331a80 Mon Sep 17 00:00:00 2001 From: Milinda Dias Date: Sat, 31 Jan 2026 02:40:01 +0530 Subject: [PATCH 7/9] fix: review comments --- docs/concepts/cache-warmer.mdx | 24 ++++++++++++------------ docs/router/configuration.mdx | 2 +- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/docs/concepts/cache-warmer.mdx b/docs/concepts/cache-warmer.mdx index c7a66d82..0c7b4e48 100644 --- a/docs/concepts/cache-warmer.mdx +++ b/docs/concepts/cache-warmer.mdx @@ -76,7 +76,7 @@ The in-memory switchover feature provides automatic cache persistence across rou ### How It Works -The router can restart due to two reasons, either due to a config change or due to a schema change. Due to the structure of the router internals we have two slight variations on how we handle the in-memory switchover cache warming: +The router can restart for two reasons: either a config change or a schema change. Due to the structure of the router internals, we have two slight variations on how we handle the in-memory switchover cache warming: 1. **Before Shutdown**: In case of config changes (from hot config reloading), the router extracts all queries from the current plan cache, preserving the queries that were in the planner cache before the cache is cleared for shutdown. @@ -84,17 +84,17 @@ The router can restart due to two reasons, either due to a config change or due 3. **Result**: The updated router starts with a fully warmed cache, eliminating latency spikes that would normally occur during cold starts. -When using the in memory switchover, the first start will still experience a cold start, as there is no prior populated planner cache. Subsequent restarts will benefit from the in-memory switchover. +When using the in-memory switchover, the first start will still experience a cold start, as there is no prior populated planner cache. Subsequent restarts will benefit from the in-memory switchover. -### When can you use the In-Memory Switchover +### When to Use the In-Memory Switchover The in-memory switchover can be used either as a fallback method or as a primary method for cache warming. This depends on the router configuration, the defaults provided by the router enables it as a fallback to the Cosmo Cloud CDN cache warmer, which is a Cosmo enterprise feature that needs to be enabled explicitly in Cosmo. -In case the in-memory switchover is used with the Cosmo Cloud CDN cache warmer, whenever either +When the in-memory switchover is used with the Cosmo Cloud CDN cache warmer, the fallback is triggered when either: * Getting the list of operations from the CDN fails -* The request to the CDN succeeded but did not have a list of operations in it +* The request to the CDN succeeds but does not return a list of operations (either no operations are cached or the manifest has not been created yet) -We will use the fallback and load the list of operations from the in-memory switchover (if any operations exists). +In these cases, the router will use the fallback and load the list of operations from the in-memory switchover (if any operations exist). The in-memory switchover cannot be used as a fallback for sources other than the Cosmo Cloud CDN cache warmer. @@ -102,10 +102,10 @@ We will use the fallback and load the list of operations from the in-memory swit ### Configuration -The in-memory switchover can be enabled as a fallback for the CDN cache warmer, in order to do this you need to ensure +The in-memory switchover can be enabled as a fallback for the CDN cache warmer. To do this, ensure: -* Cache warming is enabled on the router configuration (`cache_warmup.enabled: true`) -* `source.cdn.enabled` is set to `true` (This is true by default, and does not need to be explicitly specified) +* Cache warming is enabled in the router configuration (`cache_warmup.enabled: true`) +* `source.cdn.enabled` is set to `true` (this is true by default and does not need to be explicitly specified) * `in_memory_fallback` is set to `true` (default) ```yaml @@ -119,10 +119,10 @@ cache_warmup: enabled: true ``` -However you may want to use the in-memory switchover without the CDN cache warmer, as this is an enterprise feature. In order to do this you need to ensure +However, you may want to use the in-memory switchover without the CDN cache warmer, as the CDN is an enterprise feature. To do this, ensure: -* Cache warming is enabled on the router configuration (`cache_warmup.enabled: true`) -* `source.cdn.enabled` is set to `false` (This needs to be specified explicitly as the default is false) +* Cache warming is enabled in the router configuration (`cache_warmup.enabled: true`) +* `source.cdn.enabled` is set to `false` (this needs to be specified explicitly, as the default is true) * `in_memory_fallback` is set to `true` (default) ```yaml diff --git a/docs/router/configuration.mdx b/docs/router/configuration.mdx index 4cf2ecab..a766beca 100644 --- a/docs/router/configuration.mdx +++ b/docs/router/configuration.mdx @@ -2180,7 +2180,7 @@ apollo_router_compatibility_flags: | CACHE_WARMUP_TIMEOUT | timeout | | The timeout for warming up the cache. This can be used to limit the amount of time cache warming will block deploying a new config. The period is specified as a string with a number and a unit, e.g. 10ms, 1s, 1m, 1h. The supported units are 'ms', 's', 'm', 'h'. | 30s | | CACHE_WARMUP_WORKERS | workers | | The number of workers for the cache warmup to run in parallel. Higher numbers decrease the time to warm up the cache but increase the load on the system. | 8 | | | source | | The source of the cache warmup items. Only one can be specified. If empty, the cache warmup source is the Cosmo CDN and it requires a graph to be set. | | -| CACHE_WARMUP_IN_MEMORY_FALLBACK | in_memory_fallback | | Enable in-memory switchover that persists the plan cache across router restarts. When enabled, the router will use queries from the previous instance's plan cache to warm up the new cache on restart. Can be used as a fallback or as the primary source for warming the CDN. | true | +| CACHE_WARMUP_IN_MEMORY_FALLBACK | in_memory_fallback | | Enable in-memory switchover that persists the plan cache across router restarts. When enabled, the router will use queries from the previous instance's plan cache to warm up the new cache on restart. Can be used as a fallback or as the primary source for cache warming. | true | ### Example YAML config: From 42701a242bd4962e06f6be9631bafe9bee785cb9 Mon Sep 17 00:00:00 2001 From: Milinda Dias Date: Sun, 1 Feb 2026 01:33:54 +0530 Subject: [PATCH 8/9] fix: review comments --- docs/concepts/cache-warmer.mdx | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/concepts/cache-warmer.mdx b/docs/concepts/cache-warmer.mdx index 0c7b4e48..9755b2ac 100644 --- a/docs/concepts/cache-warmer.mdx +++ b/docs/concepts/cache-warmer.mdx @@ -70,39 +70,39 @@ Users can manually recompute slow queries from the Cosmo Studio. Currently, reco -## In-Memory Switchover Cache Warming +## In-Memory Fallback Cache Warming -The in-memory switchover feature provides automatic cache persistence across router restarts, ensuring zero cold starts when deploying new configurations or updates. +The in-memory fallback cache warming feature preserves the planner cache across hot config reloads and schema changes, allowing it to be rewarmed automatically and reducing latency spikes during restarts. ### How It Works -The router can restart for two reasons: either a config change or a schema change. Due to the structure of the router internals, we have two slight variations on how we handle the in-memory switchover cache warming: +After the router has started, the router can be reloaded for two reasons: either a config change or a schema change. Due to the structure of the router internals, we have two slight variations on how we handle the in-memory switchover cache warming: -1. **Before Shutdown**: In case of config changes (from hot config reloading), the router extracts all queries from the current plan cache, preserving the queries that were in the planner cache before the cache is cleared for shutdown. +1. **Before Reload**: In case of config changes (from hot config reloading), the router extracts all queries from the current plan cache, preserving the queries that were in the planner cache before the cache is cleared for reloading. -2. **During Startup**: The router with the updated config receives the queries from the previous plan cache and uses them to warm up its plan cache before serving traffic. +2. **During Reload**: The router with the updated config receives the queries from the previous plan cache that existed before reloading, and uses them to warm up its current plan cache before serving traffic. -3. **Result**: The updated router starts with a fully warmed cache, eliminating latency spikes that would normally occur during cold starts. +3. **Result**: The updated router reloads with a fully warmed cache, eliminating latency spikes that would normally occur during cold starts. -When using the in-memory switchover, the first start will still experience a cold start, as there is no prior populated planner cache. Subsequent restarts will benefit from the in-memory switchover. +When using the in-memory fallback, the first start will still experience a cold start, as there is no prior populated planner cache. *Only subsequent reloads* will benefit from the in-memory fallback. -### When to Use the In-Memory Switchover +### When to Use the In-Memory Fallback -The in-memory switchover can be used either as a fallback method or as a primary method for cache warming. This depends on the router configuration, the defaults provided by the router enables it as a fallback to the Cosmo Cloud CDN cache warmer, which is a Cosmo enterprise feature that needs to be enabled explicitly in Cosmo. +The in-memory fallback can be used either as a fallback method or as a primary method for cache warming. This depends on the router configuration, the defaults provided by the router enables it as a fallback to the Cosmo Cloud CDN cache warmer, which is a feature that needs to be enabled explicitly in Cosmo. -When the in-memory switchover is used with the Cosmo Cloud CDN cache warmer, the fallback is triggered when either: +When the in-memory fallback is used with the Cosmo Cloud CDN cache warmer, the fallback is triggered when either: * Getting the list of operations from the CDN fails * The request to the CDN succeeds but does not return a list of operations (either no operations are cached or the manifest has not been created yet) -In these cases, the router will use the fallback and load the list of operations from the in-memory switchover (if any operations exist). +In these cases, the router will use the fallback and load the list of operations from the in-memory fallback (if any operations exist). - The in-memory switchover cannot be used as a fallback for sources other than the Cosmo Cloud CDN cache warmer. + The in-memory fallback cannot be used as a fallback for sources other than the Cosmo Cloud CDN cache warmer. ### Configuration -The in-memory switchover can be enabled as a fallback for the CDN cache warmer. To do this, ensure: +The in-memory fallback can be enabled as a fallback for the CDN cache warmer. To do this, ensure: * Cache warming is enabled in the router configuration (`cache_warmup.enabled: true`) * `source.cdn.enabled` is set to `true` (this is true by default and does not need to be explicitly specified) @@ -119,7 +119,7 @@ cache_warmup: enabled: true ``` -However, you may want to use the in-memory switchover without the CDN cache warmer, as the CDN is an enterprise feature. To do this, ensure: +To use the in-memory fallback as the primary source, ensure: * Cache warming is enabled in the router configuration (`cache_warmup.enabled: true`) * `source.cdn.enabled` is set to `false` (this needs to be specified explicitly, as the default is true) From 11e6be58884af14be27297050158ac5bfa362ec6 Mon Sep 17 00:00:00 2001 From: Milinda Dias Date: Sun, 1 Feb 2026 01:36:34 +0530 Subject: [PATCH 9/9] fix: review comments --- docs/concepts/cache-warmer.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/concepts/cache-warmer.mdx b/docs/concepts/cache-warmer.mdx index 9755b2ac..cebbdf20 100644 --- a/docs/concepts/cache-warmer.mdx +++ b/docs/concepts/cache-warmer.mdx @@ -88,7 +88,7 @@ When using the in-memory fallback, the first start will still experience a cold ### When to Use the In-Memory Fallback -The in-memory fallback can be used either as a fallback method or as a primary method for cache warming. This depends on the router configuration, the defaults provided by the router enables it as a fallback to the Cosmo Cloud CDN cache warmer, which is a feature that needs to be enabled explicitly in Cosmo. +The in-memory fallback can be used either as a fallback method or as a primary method for cache warming. This depends on the router configuration; the defaults provided by the router enable it as a fallback to the Cosmo Cloud CDN cache warmer, which is a feature that needs to be enabled explicitly in Cosmo. However, it can also be used as the primary method of rewarming the plan cache. When the in-memory fallback is used with the Cosmo Cloud CDN cache warmer, the fallback is triggered when either: * Getting the list of operations from the CDN fails