Skip to content

Conversation

@marcius-llmus
Copy link

@marcius-llmus marcius-llmus commented Dec 12, 2025

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue)

New Package?

Did I fill in the tool.llamahub section in the pyproject.toml and provide a detailed README.md for my new integration or package?

  • Yes
  • No

Version Bump?

Did I bump the version in the pyproject.toml file of the package I am updating? (Except for the llama-index-core package)

  • Yes
  • No

Type of Change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Your pull-request will likely not be merged unless it is covered by some form of impactful unit testing.

  • I added new unit tests to cover this change
  • I believe this change is already covered by existing unit tests

How to reproduce the bug:

  • llama-index-llms-google-genai: 0.8.0
import asyncio
import os
import random

from llama_index.core.agent import FunctionAgent
from llama_index.core.tools import FunctionTool
from llama_index.llms.google_genai import GoogleGenAI


async def read_pseudo_file(file_name: str) -> str:
    """Reads a file (pseudo) with a 2-second delay to test parallel execution."""
    print(f" [System] Start reading {file_name}...")
    await asyncio.sleep(2)
    print(f" [System] Finished reading {file_name}.")
    return f"Pseudo-content for {file_name}: {random.randint(1000, 9999)}"

def build_poc_agent() -> FunctionAgent:
    """
    Builds a Proof of Concept FunctionAgent with Math tools and Gemini 3.0.
    """
    tools = [
        FunctionTool.from_defaults(async_fn=read_pseudo_file),
    ]

    llm = GoogleGenAI(
        model="gemini-3-pro-preview",
        api_key=os.getenv("GOOGLE_API_KEY"),
    )

    system_prompt = "You are a specialized math agent. Use the provided tools to calculate answers."

    return FunctionAgent(tools=tools, llm=llm, system_prompt=system_prompt)


async def main():
    agent = build_poc_agent()
    try:
        user_input = "first think about 5 names, then tell a story about then, then select 5 random file names and read in parallel"
        handler = agent.run(user_msg=user_input)
        async for event in handler.stream_events():
            pass

        await handler
        print(handler)
    except KeyboardInterrupt:
        exit(0)


if __name__ == "__main__":
    asyncio.run(main())

the problem:
https://github.com/run-llama/llama_index/pull/20362/files#diff-c079fb67256ff4dae95c19afb0be6dc4b93c1f167bd82bf144c97b942cf2a9a7L200 It was adding a new signature (None) to all deltas. This would make the signatures list mismatch the actual content list, as for text, we count 1 sig for 1 text block (fully merged)


Accordingly to https://ai.google.dev/gemini-api/docs/thought-signatures#non-function-call other parts could also have thought signatures like text.
and here

 parts = response.candidates[0].content.parts
        for part in parts:
            if part.text:
                if part.thought:

We skip empty text blocks. So if a signature come in a empty text block we will skip it and it could reduce llm performance. The problem of checking for if part.text is not None is that after a tool call, the llm will send an empty text by some reason and if its not skipped, it breaks the streaming silently. So I decided to keep the original behavior and skip empty (part.text being "") instead of checking for if is None

@marcius-llmus marcius-llmus marked this pull request as ready for review December 12, 2025 16:44
@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Dec 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant