feat: resubmit overloaded errors

[uinstinct]

Description

Retry overloaded errors from the model provider's server with an exponential backoff.

resolves CON-5040

AI Code Review

• Team members only: AI review runs automatically when PR is opened or marked ready for review
• Team members can also trigger a review by commenting @continue-review

Checklist

• [] I've read the contributing guide
• [] The relevant docs, if any, have been updated or created
• [] The relevant tests, if any, have been updated or created

Screen recording or screenshot

feat.mp4

Tests

[ What tests were added or updated to ensure the changes work as expected? ]

---

Summary by cubic

Add automatic retries with exponential backoff when the model provider is overloaded to reduce failed chats and improve reliability. Addresses CON-5040.

• New Features
  • Retries up to 3 times with a 1s base delay (1s, 2s, 4s).
  • Detects overload via error messages containing “overloaded” or “malformed json”.
  • Cancels the stream before retry; on final failure, shows the error dialog and logs gui_stream_error to PostHog with metadata.

^{Written for commit c391353. Summary will update automatically on new commits.}

[continue]

Keep this PR in a mergeable state →

Learn more

---

All Green is an AI agent that automatically:

✅ Addresses code review comments

✅ Fixes failing CI checks

✅ Resolves merge conflicts

[cubic-dev-ai]

2 issues found across 1 file

Prompt for AI agents (all 2 issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="gui/src/redux/thunks/streamThunkWrapper.tsx">

<violation number="1" location="gui/src/redux/thunks/streamThunkWrapper.tsx:53">
P1: Missing `cancelStream()` call for non-retryable errors. The original code always called `cancelStream()` when any error occurred, but this branch doesn&#39;t.</violation>

<violation number="2" location="gui/src/redux/thunks/streamThunkWrapper.tsx:64">
P1: Missing `return` after handling non-retryable error. Without it, the loop continues after showing the error dialog, causing the operation to retry even for errors that shouldn&#39;t be retried.</violation>
</file>

_{Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR}

[cubic-dev-ai]

P1: Missing cancelStream() call for non-retryable errors. The original code always called cancelStream() when any error occurred, but this branch doesn't.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At gui/src/redux/thunks/streamThunkWrapper.tsx, line 53:

<comment>Missing `cancelStream()` call for non-retryable errors. The original code always called `cancelStream()` when any error occurred, but this branch doesn&#39;t.</comment>

<file context>
@@ -8,43 +8,61 @@ import { ThunkApiType } from &quot;../store&quot;;
+        const delayMs = OVERLOADED_DELAY_MS * 2 ** attempt;
+        await new Promise((resolve) =&gt; setTimeout(resolve, delayMs));
+        await dispatch(cancelStream());
+      } else {
+        dispatch(setDialogMessage(&lt;StreamErrorDialog error={e} /&gt;));
+        dispatch(setShowDialog(true));
</file context>

Suggested change

	} else {
	} else {
	await dispatch(cancelStream());

✅ Addressed in c391353

[cubic-dev-ai]

P1: Missing return after handling non-retryable error. Without it, the loop continues after showing the error dialog, causing the operation to retry even for errors that shouldn't be retried.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At gui/src/redux/thunks/streamThunkWrapper.tsx, line 64:

<comment>Missing `return` after handling non-retryable error. Without it, the loop continues after showing the error dialog, causing the operation to retry even for errors that shouldn&#39;t be retried.</comment>

<file context>
@@ -8,43 +8,61 @@ import { ThunkApiType } from &quot;../store&quot;;
+        };
 
-    posthog.capture(&quot;gui_stream_error&quot;, errorData);
+        posthog.capture(&quot;gui_stream_error&quot;, errorData);
+      }
+    }
</file context>

Suggested change

	posthog.capture("gui_stream_error", errorData);
	posthog.capture("gui_stream_error", errorData);
	return;

✅ Addressed in c391353

[continue]

Review Feedback Addressed

I've reviewed the issues identified by the code review and they are valid. Here are the fixes:

Issue 1: Missing cancelStream() call

Problem: The original code always called cancelStream() when any error occurred, but the non-retryable error branch was missing this call.

Fix: Added await dispatch(cancelStream()); before showing the error dialog.

Issue 2: Missing return statement

Problem: Without a return, the loop would continue after handling a non-retryable error, causing unnecessary retries even for errors that shouldn't be retried.

Fix: Added return; after capturing the PostHog event.

Changes Made

} else {
+  await dispatch(cancelStream());
   dispatch(setDialogMessage(<StreamErrorDialog error={e} />));
   dispatch(setShowDialog(true));

   const errorData = {
     error_type: statusCode ? `HTTP ${statusCode}` : "Unknown",
     error_message: parsedError,
     model_provider: providerName,
     model_title: modelTitle,
   };

   posthog.capture("gui_stream_error", errorData);
+  return;
}

You can apply these fixes to your branch:

git fetch origin pr-9082
git cherry-pick 8b97c16

Or manually apply the changes from commit 8b97c16.