Support for generic gateway

[Bennoo]

What you'd like to see

I would like to be able to set as environment variable a custom gateway URL supporting the OpenAI API scheme.
This env variable + the token can makes me able to use my company AI gateway.

Why you'd like to see it

My company have multiple models behind an internal gateway. I would like to use it.

[mfranzon]

I have a similar use case, can I work on this feature and then see if it matches the requirements?

[Bennoo]

Do you ask me ?
Because if yes, I assume everybody can work on an issue!
I would be happy to help if needed

[mfranzon]

Do you ask me ? Because if yes, I assume everybody can work on an issue! I would be happy to help if needed

It was a question for the docker team just to know if could be interesting having this feature in place. But in any case thank you for being available to help me in this!

[rumpl]

How would you see this work?

You can already re-define the base url to any provider, see here for an example: https://github.com/docker/cagent/blob/main/examples/dmr.yaml#L14

In your case, instead of “dmr” you would put “openai”. Is that enough?

I guess it’s harder to share agents externally with your gateway in there, not sure if you plan on having these agents also be available to others outside of your company

[Bennoo]

You are right, it will not be shareable as is. In my personal use case, I don't look at sharing these agents but it should stay robust and well designed.

In my view, the configuration and the agent definitions should stay separated.
I’m thinking of something similar to the openai CODEX-CLI config file, where you can define model providers explicitly.

Something like this:

model_provider = "Azure"


[model_providers.azure]
name = "Azure"
base_url = "https://YOUR_PROJECT_NAME.openai.azure.com/openai"
env_key = "AZURE_OPENAI_API_KEY"

You’d define each provider’s configuration, then set one as active.
That way, the configuration remains outside the agent definition itself. You can share the agents but the config should be set for each user.

Just thinking out loud here — maybe not the best approach, but it could help keep things cleaner and more modular.

What do you all think?

[mfranzon]

How would you see this work?

You can already re-define the base url to any provider, see here for an example: https://github.com/docker/cagent/blob/main/examples/dmr.yaml#L14

In your case, instead of “dmr” you would put “openai”. Is that enough?

I guess it’s harder to share agents externally with your gateway in there, not sure if you plan on having these agents also be available to others outside of your company

In my case I missed this case in which you can already define "openai". It should be enough for me.
Thank you!

[AbhijithMallya]

You can use it like this also

models:
  devstral:
    provider: openai
    model: Devstral-Small-2507-Q8_0.gguf
    temperature: 0.1
    max_tokens: 5000
    base_url: http://<host_name>:<port>/llamacpp/v1/

The above example is for llamacpp. I have even tried Groq, it works by replacing the URL API Key

[Bennoo]

Like that one!
And then, you refer the model name in the agent definition and the agent is linked to a complete provider definition!

[Bennoo]

How would you see this work?

You can already re-define the base url to any provider, see here for an example: https://github.com/docker/cagent/blob/main/examples/dmr.yaml#L14

In your case, instead of “dmr” you would put “openai”. Is that enough?

I guess it’s harder to share agents externally with your gateway in there, not sure if you plan on having these agents also be available to others outside of your company

Maybe thats enough, you are right.. But can you define new provider?

[Bennoo]

I have an error when trying a simple thing like this:

agents:
  root:
    model: claude
    description: "Simple echo agent that repeats exactly what the user says"
    instruction: |
      You are an echo agent. Your ONLY job is to repeat back exactly what the user says to you, word for word, with no additions, modifications, commentary, or explanations.

models:
  claude:
    provider: anthropic
    model: claude-3-7-sonnet-latest
    temperature: 0.0
    max_tokens: 1000

I have the following error:

[10:5] unknown field "provider"

Do you know why? This example is from the doc files

[AbhijithMallya]

I have an error when trying a simple thing like this:

agents:
root:
model: claude
description: "Simple echo agent that repeats exactly what the user says"
instruction: |
You are an echo agent. Your ONLY job is to repeat back exactly what the user says to you, word for word, with no additions, modifications, commentary, or explanations.

models:
claude:
provider: anthropic
model: claude-3-7-sonnet-latest
temperature: 0.0
max_tokens: 1000
I have the following error:

[10:5] unknown field "provider"
Do you know why? This example is from the doc files

#!/usr/bin/env cagent run
version: "2"

Please add this at the top of the yaml

[rumpl]

Yes, the version is still needed, this won’t be needed any more in the next version, see #747

[Bennoo]

Ok thanks! This is working now.
So, could you confirm such a thing is not possible:

#!/usr/bin/env cagent run
version: "2"

agents:
  root:
    model: custom
    description: "Simple echo agent that repeats exactly what the user says"
    instruction: |
      You are an echo agent. Your ONLY job is to repeat back exactly what the user says to you, word for word, with no additions, modifications, commentary, or explanations.

models:
  custom:
    provider: company-gateway
    model: gpt-5-codex
    base_url: "https://gateway.dev.company.tech/v1"

I need to update the file provider.go to be able to do such a thing?

[rumpl]

Yes, for now if you wanted to have a shortcut form for your provider (for example model: company-gateway/gpt-5-codex) you would need to change the provider.go file, but I'm thinking this could also be in a config somewhere, adding compatible providers usually means just declaring things. We might want to do that for cases like yours

[Bennoo]

That will be awesome for my case, a separate config with custom providers. At least for openai API compatible providers.
Thanks