Working Group Proposal: Agentic Foundation and LangChain4j Next-Gen

[cescoffier]

1. Summary

This working group will explore and define the foundation of the next generation of AI-Infused and Agentic applications built with LangChain4j and Quarkus LangChain4j. The group will investigate core changes to the current programming model, improve developer experience, build stronger interoperability mechanisms (especially around A2A), and define architectural separation of responsibilities (e.g., executor vs planner). The output will serve as the foundation for building resilient, scalable, and extensible AI-infused and agent systems on the Quarkus platform.

2. Motivation

While LangChain4j and Quarkus LangChain4j already provide strong foundations, building real-world agentic applications has revealed several pain points and architectural limitations. These range from ergonomic and modeling issues to missing runtime support for observability, visualization, memory management, and resilience.

This WG aims to:

• streamline the developer experience,
• improve core primitives (invocations, workflows, A2A),
• enable pluggable planning engines (like the built-in workflow constructs, supervisor…),
• strengthen memory and context engineering support,
• and provide around cost, observability, and correctness features

3. Scope

This working group will cover the following areas:

Developer Model Simplification

• Eliminate redundancies in tool/function/agent definitions
• Fix common friction points (e.g., outputName inconsistencies)
• Explore alternatives to "String"-based keys (e.g., typesafe references, enums)

⠀
Invocation Model Enhancements

• Rename AgentInvocation to Invocation to enable invocations of regular methods/tools
• Study retry, rollback (compensation), and durability mechanisms, especially in function calling, and whether these should be managed at the planner level or execution level
• Improve human in the loop and agent conversation support

⠀
A2A (Agent-to-Agent) Improvements

• Automatic exposure of agents as A2A services
• Declarative A2A clients
• Studies how A2A agents can be integrated into applications (what are the patterns?)

⠀
Architectural Refactoring

• Apply Autonomic Reference Architecture: clear separation between operational/execution execution (observe, act, maintain state) and analytical planning (decide, select following invocation)
• Design for pluggable planners (rules, workflows, LLM-based (supervisor))

⠀
LangChain4j Core Improvements

• Revamp the LangChain4j API to support non-blocking, asynchronous processing
• Improve memory management (lifecycle, scoping, resource reuse, short-term/long-term memory) across tool/agent executions
• Explore and improve support for Context Engineering:
  • Better control over prompt construction
  • Enhanced context injection at build/runtime
  • Modular strategies for short-term and long-term memory enrichment

⠀
Developer Experience & Tooling

• Quarkus Dev UI integration:
  • Visualize workflows
  • Trace agent invocations and tool calls
• Build-time validation for LangChain4j constructs (missing tools, bad metadata, etc.)
• Evaluation harnesses and support for repeatable agent test scenarios

⠀
Cost & Observability

• Track and optimize token usage
• Condense tool result JSON
• Estimate invocation cost during planning
• Support tracing (OpenTelemetry), logging, and metric collection (Micrometer/OTEL bridge) - already quite advanced
• Document techniques to control/minimize token usage, like TokenWindowChatMemory instead of MessageWindowChatMemory,
  filter unnecessary MCP tools, select required tools dynamically via AiServices.toolSelector (to be implemented)

⠀
Workflow Integration

• Generalized interface to plug in workflow engines as a planner
• Coordination between agents and workflows (e.g., agent as a workflow step)
  ⠀

4. Out of Scope

This working group does not cover:

• UI/UX or frontend agent interfaces beyond Dev UI integration

⠀

5. Definition of Done

The WG will be considered done when the following milestones are met:

• A simplified and more declarative developer model is available
• The LangChain4j API supports non-blocking execution paths
• At least one alternative planner (e.g., a workflow engine) can be plugged into the agentic runtime
• Dev UI extension is available for invocation/workflow visualization
• A2A support includes declarative clients and automatic exposure
• Context engineering and memory management strategies are documented and implemented
• Build-time validation and token usage observability are implemented
• A reference set of patterns/examples demonstrates how to build production-grade agentic systems using the new model

⠀

6. Points of Contact

• Mario, Dmytro, Georgios, Clement…
• A2A Team (Stefano, Farah)
• Workflow automation team

⠀

7. Communication & Tracking

• This WG is a quarkiverse working group
• Zulip stream: to be announced

[dliubarskyi]

I would also add to "Cost & Observability":

1. Document techniques to control/minimize token usage. For example (from the top of my mind):

• Use TokenWindowChatMemory instead of MessageWindowChatMemory
• Filter unnecessary MCP tools by setting McpToolProvider.filter/filterToolNames
• Select required tools dynamically via AiServices.toolSelector (to be implemented)
• etc

2. Implement AiService.toolSelector to be able to dynamically select tools to be available to the LLM for the current invocation

[cescoffier]

Added! Thanks!

[dliubarskyi]

Also "Condense tool result JSON", I just realised that it is currently full of whitespaces

[cescoffier]

Done.

[sberyozkin]

Thinking how a security identity can be used neatly to impact the flow should be in scope IMHO

[cescoffier]

I'm not sure—it deserves its own WG, as it's a deeper problem than people think. It's not just "I propagate my principal and it's done". Each agent may have different access rights and so on.

Happy to discuss it, BTW!

[sberyozkin]

Hi @cescoffier

It's not just "I propagate my principal and it's done"

+1, I agree, and this is not what I was primarily thinking about.

My thinking is, that a concept of the secure agent should become one of the key differentiators of what LC4J and QLC4J will offer in the agentic space, in addition to the flexibility, ease of use, ease of integration. So that we talk about the agentic AI security from the get go, in contrast to some other systems.

For ex, in one of the Devoxx Be sessions, there was a reference to a signed agent, perhaps it can be about accompanying its model output with the signature of the agent to assert the agent's identity, instead of only propagating the user authentication token. Or there could be some high level API support for the identity specific operations.

I'm not exactly sure what form it should take, beyond of what you mentioned, where we just inject the identity or control access to agents, but I do believe that an Agentic AI that will go to prod must have a good security support and if it can be done in LC4j/QLC4J better than in other systems than it would be good...

In any case, I'd be happy to be involved in this WG's effort, to learn about what you'll all be doing, and if I can spot some opportunity to get some security hook somewhere, I'd suggest it...

A reference set of patterns/examples demonstrates how to build production-grade agentic systems using the new model

+1.

IMHO, at least one of the examples should demonstrate, how an autonomous agentic system can react to some changes in a real user's details (new emails, calendar updates, profile changes) by analyzing a security identity of this user, captured from Google, GitHub, Keycloak or some other provider that allows to use the user authentication tokens to fetch user specific detals (emails, etc). If agreed, then when the WG gets closer to its completion, I can help with creating such an example

Thanks

[edeandrea]

Something around tool invocations as well - guardrails around tool invocations: quarkiverse/quarkus-langchain4j#990

We should implement this in upstream LC4j.

[geoand]

Sure, but this WG is more about breaking new ground

[emmanuelbernard]

LGTM as the envelope for the WG. My intuition is that the definition of done will be reduced after a few explorations but we will see, there is one and that's what matters.

[cescoffier]

Yes, and honestly, I hope so... otherwise it will never be completed.

[cescoffier]

Working group created: WG - Agentic Foundation and LangChain4j Next (view)

[unsignedint]

Hey team, super excited about this work. Our team (mostly @ejstuart and @Wal8800) have been working to build a complex user-facing agentic application.

We would love to be involved to provide some feedback at least, if not contribute where we can.

One concern we have is UI/UX being "out of scope." What we have learned is that, actually, there is quite an impedance mismatch between current state and what we need to provide a beautiful (streaming) UI/UX, human-in-the-loop, etc. A concrete a example is streaming for top-level agents vs tool calls vs sub-agents. The point is the API interface and persistence data models are critical to UX and we should be empathetic to that.

[ejstuart]

To add, some examples of use cases for streaming:

1. Top-level agent streaming - The ongoing text response (between sub agent calls and tool calls) and final text response to the client/UI.
2. Tool call streaming - Real-time updates about what tools are being called ("Searching database...", "Analyzing results...")
3. Sub-agent streaming - When a supervisor delegates to a worker agent, you might want to show that sub-conversation

Top-level agent streaming is paramount to us but ultimately these use cases are worth considering early either way to ensure we establish an architecture that can accommodate them in future iterations.

[geoand]

Unfortunately, all things related to non-blocking / streaming require a vast rearchitecting of LangChain4j, which will probably be deferred for a while

[cescoffier]

Yes, streaming / non-blocking is on the roadmap for LC4J 2.0. But it will require some time to get there.

[scottslewis]

Howdy. For the past few months I've been a contributor to the mcp-java-sdk and member of the relatively new primitive-grouping-wg, a new MCP working group looking at adding a 'grouping' primitive for Tools, Resources, Prompts.

I'm also the primary author of this proposal for adding Groups to the MCP protocol. I've been prototyping the proposed spec enhancement with an enhanced version of the mcp-java-sdk. This repo shows an example of using the mcp-java-sdk Group primitive for dynamically organizing tools in an mcp server. This is a technical proposal that has not yet gone through the full SEP process.

Also, the primitive-grouping-wg has created a discussion for use cases for primitive grouping. From the discussion above it seems as though there might be some use cases for primitive grouping in mcp (e.g. tool organization, 'guardrails' for mcp tools and other primitives, identity and/or role based primitive access, trust categories, etc. With this discussion we are hoping to get input from as much of the dev community as possible, so that the grouping proposals can be grounded in actual server and client-developer-community use cases.

I was pointed to this list by a colleague as a working group that would probably interested in the mcp-java-sdk project technical direction and governance/maintenance, as it's likely that Java mcp servers and client devs will require support in the mcp-java-sdk for (e.g.) the latest mcp protocol specs/features...for interoperability as well as getting access to new/desired features (e.g. grouping). Please let me know if this is the right place and/or redirect. If there is an appropriate slack and/or discord channel to meet working group members online, I would like to know that also. Thanksinadvance.

[cescoffier]

Thanks, let me try to clarify what I said earlier, and also what we would hope to see from the “official” Java SDK.

From the Quarkus side, it’s not that we technically can’t put Reactor on the classpath or call its APIs. We can. The issue is that Quarkus has deliberately not built its reactive story on Reactor, but on Vert.x + Mutiny, with its own non-blocking and reactive abstractions. 

When we tried to integrate Reactor-based APIs in the past, the outcome was mostly confusion:

• You end up with two reactive models in the same app (Mutiny + Reactor), plus conversions between them.
• For something as low-level as the MCP transport, those conversions sit on a very hot path and add overhead.
• It complicates the mental model for Quarkus users, who are already “trained” on Mutiny and Vert.x.

Because of this, Quarkus attempts to minimize Reactor's presence in the core framework; we don’t want MCP support to inadvertently introduce a second, competing reactive model and network stack into applications.

What we’d really like from the Java SDK is something much more unopinionated:

• For async/streaming APIs, use Reactive Streams / JDK Flow interfaces as the contract rather than a concrete implementation like Reactor.
• If the SDK really wants to ship an implementation, a tiny zero-dependency library such as Mutiny Zero (reactive-streams compliant publishers, no external deps, adapters to Flow/Mutiny/etc.) is a possible option. 

That keeps the core neutral, and lets frameworks adapt to it in their own way (Reactor on Spring side, Mutiny/Vert.x on Quarkus side, etc.).

Similarly, for the network stack, I don’t think the SDK should preclude a single HTTP/server/client choice (e.g., Reactor Netty, Servlet, etc.). Different frameworks use very different stacks (Tomcat, Jetty, Netty, Vert.x HTTP, JDK HTTP client…). Having the core SDK define a transport SPI (with maybe a minimal default implementation that has no Spring/Reactor dependency) and then provide Spring/Reactor, Vert.x, etc. bindings as optional modules would make it much easier for us (and others) to adopt it.

This aligns closely with the feedback in the MCP discussion about the Java SDK being overly coupled to a specific framework and the desire for a framework-agnostic SPI that any JVM application can plug into. 

To your question about previous discussions: there have been a few threads in the MCP org around exactly this topic (for example: “Why is the Java SDK not generic?”), and some side conversations in the Quarkus ecosystem that led us to build our own server/client instead of trying to wrap the current SDK. 

So, in short:

• Quarkus can technically use Reactor, but we’ve intentionally avoided it in our reactive core because it conflicts with our existing model.
• We’d strongly prefer an MCP Java SDK whose core is:
• Framework-agnostic
• Based on neutral types (plain Java + Reactive Streams / Flow)
• With transports and specific stacks (Reactor Netty, Spring, Vert.x, etc.) provided as pluggable modules, not as hard requirements.

If the SDK evolves in that direction, I’d be very happy to see Quarkus adopt it.

[scottslewis]

Thanks for the detailed responses. A few follow up comments and questions.

Because of this, Quarkus attempts to minimize Reactor's presence in the core framework; we don’t want MCP support to inadvertently introduce a second, competing reactive model and network stack into applications.

Yeah I understand this.

What we’d really like from the Java SDK is something much more unopinionated:

* For async/streaming APIs, use Reactive Streams / JDK Flow interfaces as the contract rather than a concrete implementation like Reactor.

I'm game on unopinionated...but practically speaking: is there any way that other folks would ever come together behind something like this? e.g. mutiny with converters?

Another 'official sdk' approach could be to break up the sdk and allow both reactor abstractions to support mcp development...but that sounds potentially very complicated to create and maintain (in the official sdk), as well as possibly complicated for sdk users (devs).

FWIW, Years ago I created a grpc-and-protoc-based impl of the osgi remote services/rsa specs, where the protoc code generator (from proto file Service definition)...generated both a java interface contract...using Flowable and Single for the streaming and regular rpc via a protoc plugin and the grpc implementation. This sort of approach (higher level service streaming/async service/tool/etc abstractions could provide a path. Forgive me for thinking out loud.

* If the SDK really wants to ship an implementation, a tiny zero-dependency library such as Mutiny Zero (reactive-streams compliant publishers, no external deps, adapters to Flow/Mutiny/etc.) is a possible option.  

I would like that...but how to get that into the 'official' sdk.

That keeps the core neutral, and lets frameworks adapt to it in their own way (Reactor on Spring side, Mutiny/Vert.x on Quarkus side, etc.).

Again...no problems with that myself.

Similarly, for the network stack, I don’t think the SDK should preclude a single HTTP/server/client choice (e.g., Reactor Netty, Servlet, etc.). Different frameworks use very different stacks (Tomcat, Jetty, Netty, Vert.x HTTP, JDK HTTP client…). Having the core SDK define a transport SPI (with maybe a minimal default implementation that has no Spring/Reactor dependency) and then provide Spring/Reactor, Vert.x, etc. bindings as optional modules would make it much easier for us (and others) to adopt it.

I agree with this also. I know from the mcp discord and new SEP process (mcp's protocol enhancement process) that there are a number of proposals around creating consistent (across languages if possible) transport abstractions, adding such an abstraction to the python impl (probably the most popular language currently for mcp server developers), allowing (e.g.) grpc as a transport by not being spec-bound to JSONMessage.

This aligns closely with the feedback in the MCP discussion about the Java SDK being overly coupled to a specific framework and the desire for a framework-agnostic SPI that any JVM application can plug into. 

Yeah. Again this sounds exactly right to me, but there are a lot of forces here. I would like to find some way to

a) provide an agnostic-as-possible sdk that is useful to java devs, (as well as framework and mcp dev tool makers);
2) Be able to support rapid mcp protocol migration (e.g. tasks are being added almost as we speak...hopefully grouping is next);
3) maintain both mcp client -> server and backward compatibility, as well as full inter-language/runtime interoperability of mcp servers and clients.

To your question about previous discussions: there have been a few threads in the MCP org around exactly this topic (for example: “Why is the Java SDK not generic?”), and some side conversations in the Quarkus ecosystem that led us to build our own server/client instead of trying to wrap the current SDK. 

Thanks, I was not aware of it and read through it. This discussion happened in April though, and here we are in November without much apparent project progress on SPI approach...from my observation of the mcp-java-sdk prs/commits via my fork.

Not a great sign also (IMHO) that I requested being added as a project team member in September...along with opening a number of issues and features...without any response from tzolov or colleagues. To be fair, the org and governance structure are new, of course, and currently being pulled in a number of directions.

So, in short:

* Quarkus can technically use Reactor, but we’ve intentionally avoided it in our reactive core because it conflicts with our existing model.

* We’d strongly prefer an MCP Java SDK whose core is:

* Framework-agnostic

* Based on neutral types (plain Java + Reactive Streams / Flow)

* With transports and specific stacks (Reactor Netty, Spring, Vert.x, etc.) provided as pluggable modules, not as hard requirements.

If the SDK evolves in that direction, I’d be very happy to see Quarkus adopt it.

I definitely agree on all of the above. I would like to find a way to contribute in those directions.

[cescoffier]

Hi @scottslewis,

Thanks for the thoughtful feedback and the detailed follow-up. I really appreciate your engagement on this. I think we are on the same page.

You raise very valid concerns, and I agree with all of them:

1. The need for a framework-agnostic SDK core for mcp‑java‑sdk (and related tooling), one that is based on plain Java + Reactive Streams / Flow, and doesn’t impose a specific reactive stack (e.g., Reactor) or transport. This aligns exactly with what we’ve been advocating. As we noted earlier: “what we’d really like from the Java SDK is something much more unopinionated … use Reactive Streams/JDK Flow rather than a concrete implementation like Reactor.” 

2. Your point about concrete progress (or lack thereof) in the SDK landscape is well-taken. We are seein the same. This is precisely why we decided long ago to build our own server/client stack in Quarkus rather than retrofit the existing SDK, because the coupling to Spring/Reactor created friction and architectural mismatches (especially given Quarkus’s Vert.x + Mutiny reactive core).

Now, the question for me is: what should the next step be?

[scottslewis]

Hi @scottslewis,

Thanks for the thoughtful feedback and the detailed follow-up. I really appreciate your engagement on this. I think we are on the same page.

You raise very valid concerns, and I agree with all of them:

1. The need for a framework-agnostic SDK core for mcp‑java‑sdk (and related tooling), one that is based on plain Java + Reactive Streams / Flow, and doesn’t impose a specific reactive stack (e.g., Reactor) or transport. This aligns exactly with what we’ve been advocating. As we noted earlier: “what we’d really like from the Java SDK is something much more unopinionated … use Reactive Streams/JDK Flow rather than a concrete implementation like Reactor.” 

Absolutely. The good news is that for the current mcp-java-sdk, the core of core (spec package, transport packages) classes do not have many compile time references to reactor classes...and the ones that are there are pretty much the reactor version of Flow.Publisher (interfaces generally). So what I'm saying is, at first blush I don't think creating an stack-agnostic api would be terribly difficult for a mcp-core.

2. Your point about concrete progress (or lack thereof) in the SDK landscape is well-taken. We are seein the same. This is precisely why we decided long ago to build our own server/client stack in Quarkus rather than retrofit the existing SDK, because the coupling to Spring/Reactor created friction and architectural mismatches (especially given Quarkus’s Vert.x + Mutiny reactive core).

Now, the question for me is: what should the next step be?

I have two answers

1. I've just provided a pr to refactor the mcp-java-sdk with an mcp json gen/parse + validation api api that separates the impl (Jackson +) from mcp-core. This is a provider arch for mcp json gen/parse + validation (a default impl is required for spec compliant serialization/deserialization + json schema validation. As a happy side effect, it also provides a nice fix for osgi environments

2. I propose a similiar-to-1 joint effort to make the mcp-sdk (or at least core pieces of it) react/flow api agnostic. I suspect with collaboration from the mutiny (one?) team and/or the reactive team it would not be a gigantic refactoring effort.

If you are game, I would suggest that we have chat.

[cescoffier]

Sure! Definitely interested!