Gemini 2.5 Flash truncates output when used with LangChain middlewares

[stzonis]

Checked other resources

• This is a bug, not a usage question.
• I added a clear and descriptive title that summarizes this issue.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
• This is not related to the langchain-community package.
• I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Package (Required)

• langchain
• langchain-openai
• langchain-anthropic
• langchain-classic
• langchain-core
• langchain-cli
• langchain-model-profiles
• langchain-tests
• langchain-text-splitters
• langchain-chroma
• langchain-deepseek
• langchain-exa
• langchain-fireworks
• langchain-groq
• langchain-huggingface
• langchain-mistralai
• langchain-nomic
• langchain-ollama
• langchain-perplexity
• langchain-prompty
• langchain-qdrant
• langchain-xai
• Other / not sure / general

Example Code (Python)

from langgraph.graph import StateGraph, START, END
from langchain.agents import create_agent
from langchain.chat_models import init_chat_model
from langchain_core.messages import AnyMessage
from typing_extensions import TypedDict, Annotated
import operator

from langchain.agents.middleware import TodoListMiddleware

class State(TypedDict):
    messages: Annotated[list[AnyMessage], operator.add]

async def agent_node(state: State) -> State:        
    agent = create_agent(
        model=init_chat_model("google_genai:gemini-2.5-flash", max_tokens=16000),
        middleware=[TodoListMiddleware()],  # Issue: truncation occurs with any middleware
    )
    
    result = await agent.ainvoke({"messages": state["messages"]})
    return {"messages": [result["messages"][-1]]}

builder = StateGraph(State)
builder.add_node("agent", agent_node)
builder.add_edge(START, "agent")
builder.add_edge("agent", END)

app = builder.compile()

# Test with a simple request
result = app.invoke({"messages": [{"role": "user", "content": "Tell me a story with 500 words about Paris."}]})
print(result["messages"][-1]["content"])

# Output: Gets only a few sentences instead of 500 words

Error Message and Stack Trace (if applicable)

Description

When using google_genai:gemini-2.5-flash with any LangChain middleware (e.g., TodoListMiddleware, FilesystemMiddleware, SubAgentMiddleware), the agent's response stops after a few words, even with high max_tokens values.

Expected Behavior
The model should generate complete responses when used with middlewares, just as it does without them.

Actual Behavior
Responses are truncated prematurely after a few words. The truncation does not occur when middlewares are disabled or when other models are used.

Workarounds
Remove middlewares when using Gemini 2.5 Flash
Switch to other models (e.g., openai:gpt-4o) which work correctly with middlewares.

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 25.1.0: Mon Oct 20 19:32:41 PDT 2025; root:xnu-12377.41.6~2/RELEASE_ARM64_T6000
Python Version: 3.11.3 (v3.11.3:f3909b8bc8, Apr 4 2023, 20:12:10) [Clang 13.0.0 (clang-1300.0.29.30)]

Package Information

langchain_core: 0.3.80
langchain: 0.3.27
langchain_community: 0.4.1
langsmith: 0.4.43
langchain_classic: 1.0.0
langchain_google_genai: 2.0.10
langchain_text_splitters: 0.3.11
langgraph_sdk: 0.2.9

Optional packages not installed

langserve

Other Dependencies

aiohttp<4.0.0,>=3.8.3: Installed. No version info available.
async-timeout<5.0.0,>=4.0.0;: Installed. No version info available.
claude-agent-sdk>=0.1.0;: Installed. No version info available.
dataclasses-json<0.7.0,>=0.6.7: Installed. No version info available.
filetype: 1.2.0
google-generativeai: 0.8.5
httpx-sse<1.0.0,>=0.4.0: Installed. No version info available.
httpx<1,>=0.23.0: Installed. No version info available.
httpx>=0.25.2: Installed. No version info available.
jsonpatch<2.0.0,>=1.33.0: Installed. No version info available.
langchain-anthropic;: Installed. No version info available.
langchain-aws;: Installed. No version info available.
langchain-azure-ai;: Installed. No version info available.
langchain-classic<2.0.0,>=1.0.0: Installed. No version info available.
langchain-cohere;: Installed. No version info available.
langchain-community;: Installed. No version info available.
langchain-core<1.0.0,>=0.3.72: Installed. No version info available.
langchain-core<2.0.0,>=0.3.75: Installed. No version info available.
langchain-core<2.0.0,>=1.0.0: Installed. No version info available.
langchain-core<2.0.0,>=1.0.1: Installed. No version info available.
langchain-deepseek;: Installed. No version info available.
langchain-fireworks;: Installed. No version info available.
langchain-google-genai;: Installed. No version info available.
langchain-google-vertexai;: Installed. No version info available.
langchain-groq;: Installed. No version info available.
langchain-huggingface;: Installed. No version info available.
langchain-mistralai;: Installed. No version info available.
langchain-ollama;: Installed. No version info available.
langchain-openai;: Installed. No version info available.
langchain-perplexity;: Installed. No version info available.
langchain-text-splitters<1.0.0,>=0.3.9: Installed. No version info available.
langchain-text-splitters<2.0.0,>=1.0.0: Installed. No version info available.
langchain-together;: Installed. No version info available.
langchain-xai;: Installed. No version info available.
langsmith-pyo3>=0.1.0rc2;: Installed. No version info available.
langsmith<1.0.0,>=0.1.125: Installed. No version info available.
langsmith<1.0.0,>=0.1.17: Installed. No version info available.
langsmith<1.0.0,>=0.3.45: Installed. No version info available.
langsmith>=0.1.17: Installed. No version info available.
numpy>=1.26.2;: Installed. No version info available.
numpy>=2.1.0;: Installed. No version info available.
openai-agents>=0.0.3;: Installed. No version info available.
opentelemetry-api>=1.30.0;: Installed. No version info available.
opentelemetry-exporter-otlp-proto-http>=1.30.0;: Installed. No version info available.
opentelemetry-sdk>=1.30.0;: Installed. No version info available.
orjson>=3.10.1: Installed. No version info available.
orjson>=3.9.14;: Installed. No version info available.
packaging<26.0.0,>=23.2.0: Installed. No version info available.
packaging>=23.2: Installed. No version info available.
pydantic: 2.12.4
pydantic-settings<3.0.0,>=2.10.1: Installed. No version info available.
pydantic<3,>=1: Installed. No version info available.
pydantic<3.0.0,>=2.7.4: Installed. No version info available.
pytest>=7.0.0;: Installed. No version info available.
pyyaml<7.0.0,>=5.3.0: Installed. No version info available.
PyYAML<7.0.0,>=5.3.0: Installed. No version info available.
PyYAML>=5.3: Installed. No version info available.
requests-toolbelt>=1.0.0: Installed. No version info available.
requests<3,>=2: Installed. No version info available.
requests<3.0.0,>=2.0.0: Installed. No version info available.
requests<3.0.0,>=2.32.5: Installed. No version info available.
requests>=2.0.0: Installed. No version info available.
rich>=13.9.4;: Installed. No version info available.
SQLAlchemy<3,>=1.4: Installed. No version info available.
sqlalchemy<3.0.0,>=1.4.0: Installed. No version info available.
SQLAlchemy<3.0.0,>=1.4.0: Installed. No version info available.
tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available.
typing-extensions<5.0.0,>=4.7.0: Installed. No version info available.
vcrpy>=7.0.0;: Installed. No version info available.
zstandard>=0.23.0: Installed. No version info available.

[stzonis]

Not a token limit issue: Increasing max_tokens to 16000 doesn't help.
First observed with: create_deep_agent (which uses middlewares by default), then reproduced with create_agent + manual middleware addition.

[keenborder786]

@stzonis I just ran the following code

from langgraph.graph import StateGraph, START, END
from langchain.agents import create_agent
from langchain.chat_models import init_chat_model
from langchain_core.messages import AnyMessage
from typing_extensions import TypedDict, Annotated
import operator

from langchain.agents.middleware import TodoListMiddleware

class State(TypedDict):
    messages: Annotated[list[AnyMessage], operator.add]

async def agent_node(state: State) -> State:
    agent = create_agent(
        model=init_chat_model("google_genai:gemini-2.5-flash", max_tokens=16000),
        middleware=[TodoListMiddleware()],  # Issue: truncation occurs with any middleware
    )

    result = await agent.ainvoke({"messages": state["messages"]})
    return {"messages": [result["messages"][-1]]}

builder = StateGraph(State)
builder.add_node("agent", agent_node)
builder.add_edge(START, "agent")
builder.add_edge("agent", END)

app = builder.compile()
import asyncio
async def main():
# Test with a simple request
    result = await app.ainvoke({"messages": [{"role": "user", "content": "Tell me a story with 500 words about Paris."}]})
    print(result["messages"][-1].content)

if __name__ == "__main__":
    asyncio.run(main())

And got the following output:

[{'type': 'text', 'text': "The morning light in Paris painted the city in hues of soft gold, illuminating the intricate ironwork of balconies and the sleepy cobblestones of narrow streets. Elodie, a young artist with a sketchbook perpetually clutched in her hand, emerged from her tiny apartment in Le Marais, the scent of fresh croissants and strong coffee beckoning her onward. Her destination: a quiet corner overlooking the Seine, where she hoped to capture the city's awakening.\n\nShe walked past boulangeries just opening their doors, the warm glow spilling onto the pavement, and florists arranging vibrant bouquets that seemed to sing with color. The city hummed with a gentle rhythm, a melody of distant church bells and the murmur of early risers. Elodie felt a familiar surge of inspiration, a feeling that Paris, in its quiet moments, revealed its truest self.\n\nReaching her spot near Pont Neuf, she settled onto a worn stone bench, her gaze sweeping across the river. The Eiffel Tower, still partially veiled in a delicate mist, stood sentinel in the distance, a graceful silhouette against the brightening sky. Barges drifted lazily, their reflections shimmering on the water’s surface like fractured dreams. Elodie opened her sketchbook, her charcoal poised.\n\nHours melted away as she sketched, lost in the interplay of light and shadow, the subtle shifts in the clouds, and the ceaseless flow of the river. A street musician began to play a melancholic accordion tune nearby, its notes weaving into the fabric of the morning, adding another layer to the city’s symphony. Tourists, more numerous now, bustled past, their cameras clicking, eager to capture the magic Elodie felt in her soul.\n\nLater, seeking a different perspective, Elodie wandered through the Latin Quarter, its ancient streets whispering tales of philosophers and revolutionaries. She found herself in a small, hidden courtyard, overgrown with ivy and fragrant with jasmine. A weathered fountain trickled softly, its gentle murmur a soothing counterpoint to the distant city bustle. Here, she sketched an old man reading a newspaper on a sun-drenched bench, his face etched with stories, a living testament to Paris's enduring spirit.\n\nAs evening approached, the city transformed once more. Lights began to twinkle along the avenues, and the Eiffel Tower ignited, showering the Parisian sky with a million golden sparks. Elodie, now on a different bridge, watched the spectacle, her heart full. Paris was not just a city of landmarks; it was a living, breathing entity, a muse that constantly offered new perspectives, new inspirations, and an endless wellspring of beauty. She closed her sketchbook, a day's worth of Parisian moments captured within its pages, ready for tomorrow's unfolding story."}]

Everything seems okay to me. Can you post your langsmith trace prehaps?

[stzonis]

Here is the trace. It generates the full story, but displays only the beginning.

https://eu.smith.langchain.com/public/140f00ca-369c-40fc-b133-06557cb1c10e/r

[keenborder786]

In your trace @stzonis I can see that your system prompt is entirely wrong and does not make any sense and does not related to what agent is being asked to do. Can you please update the System Prompt to the task that is more relevant to what you are trying to acheive.

[stzonis]

Here is a new run: https://eu.smith.langchain.com/public/8c9e1392-feed-4549-945a-e8ec85014242/r

with this code:

from langgraph.graph import StateGraph, START, END
from langchain.agents import create_agent
from langchain.chat_models import init_chat_model
from langchain_core.messages import AnyMessage
from typing_extensions import TypedDict, Annotated
import operator

from langchain.agents.middleware import TodoListMiddleware

class State(TypedDict):
    messages: Annotated[list[AnyMessage], operator.add]

async def agent_node(state: State) -> State:
    agent = create_agent(
        model=init_chat_model("google_genai:gemini-2.5-flash", max_tokens=16000),
        middleware=[TodoListMiddleware()],  # Issue: truncation occurs with any middleware
    )

    result = await agent.ainvoke({"messages": state["messages"]})
    return {"messages": [result["messages"][-1]]}

builder = StateGraph(State)
builder.add_node("agent", agent_node)
builder.add_edge(START, "agent")
builder.add_edge("agent", END)

app = builder.compile()
import asyncio
async def main():
# Test with a simple request
    result = await app.ainvoke({"messages": [{"role": "user", "content": "Tell me a story with 500 words about Paris."}]})
    print(result["messages"][-1].content)

if __name__ == "__main__":
    asyncio.run(main())

The system prompt is from TodoListMiddleware.

Still the same display problem:

[fraricci]

Hi all,

I may chime here, I was about to post about the same issue.
I also have the same exact issue with gemini-flash-2.5.
It seems that the output (see image of the trace above) gets split in two parts:
content.0.text contains the first part and content.1 is a string that contains the rest of the output.
The issue is that I guess the following objects (middlewares, but I have the same happening with the Chat UI)
will receive only the first part 'content.0.text` (maybe by default?) and not the other entries of the content.

I noticed this with gemini-flash-2.5, but not with gpt-oss.

Hope this help, thanks!