Using Codex with a custom API gateway

So we all know that OpenAIs codex is, at the time of writing this short note, one of the best coding agents that are currently available. But then one often does not want to use it against ones ChatGPT subscription due to limited credits - and maybe also not against the platform API from OpenAI for various reasons:

• One may want to like to exploit agentic behaviour of local running models and just play with them.
• One may need to guarantee privacy - not sending data to an external cloud service and trace privacy for performance and capabilities.
• One may try to use local hardware to avoid rate limits or exploding bills.
• One may have a local LLM service, like for example at TU Wien, that one may want to use for all of the reasons above.

Note that for consumer hardware it is very unlikely that you will achieve a result that you are really satisfied with in the same way as with the frontier cloud hosted models like GPT 5.5 or GPT 5.6 because today’s strongest coding agents rely not only on model quality but also on very large context windows, strong reasoning capabilities and reliable tool use. When your institution operates the necessary hardware and provides for example GLM 5.2 you may be satisfied with the outcome even for complex tasks though.

• Arbitration and Gateway
• Executing Codex
  • Profile Configuration
  • Model Metadata
  • Launching Codex
• Conclusion
• References

Arbitration and Gateway

For pure local inference using the ollama runtime one may directly use the --oss feature of codex, as has also been explained previously. But especially when you are operating with services distributed over the whole Internet or want to use different frontends for your self-hosted LLMs there is need for arbitration

Previously I have introduced my mini-apigw. There are also solutions like the cloud service OpenRouter or the gateway Aqueduct that has been developed at TU Wien on top of LiteLLM Router. The problem with many backends is - they do not support the same feature set, especially over the new and (still) rapidly evolving Responses API. And since codex is a product by OpenAI it uses nearly every feature from responses. This lead to the client not being able to perform its operation against services as our locally hosted GLM 5.2 - or against my older ollama deployments.

To resolve that problem, especially since I am also playing with more experimental backends as running llama.cpp or colibri, I decided to resolve the problem once and for all: mini-apigw now supports a responses shim that speaks the same language as advanced tools as codex while supporting any legacy backend via the chat completions API or the ollama client. This is a typical case where a simple API gateway helps to resolve compatibility issues - hopefully on the long run. The idea is to never change the application facing API even when the backend gets exchanged or updated - with all features like scalability and accounting being there “for free”. Especially due to the rapidly evolving nature of the Responses API the shim that provides compatibility needs regular updates - but putting this into a central place like a gateway means not having to keep track of such changes for each and every application individually.

In addition the application of an API gateway allows to achieve vendor independence. Even in case one is using cloud services one can easily swap different backends by different vendors without a single change on the application facing API or configuration. Another gain from using a gateway is separation of concerns with respect to API tokens (though it also introduces a central spot where tokens for different backends can be stolen) - every application gets an individual API token, the gateway itself is responsible to authenticate against a backend. Accounting and monitoring allows one to inspect the requests and responses and keep track of usage of the different backends.

Executing Codex

Profile Configuration

Now the interesting part - how do we run codex with a custom backend? Usually one needs to create a new profile, which is done by creating a {PROFILENAME}.config.toml file in the local ~/.codex directory.

Here is an example that may be called example.config.toml:

model_provider = "exampleprovider"
model_context_window = 256144
model = "glm-5.2-744b-preview"
model_reasoning_effort = "medium"
model_catalog_json = "/usr/home/exampleuser/.codex/example-models.json"

[features]
standalone_web_search = false
imagegenext = false
terminal_resize_reflow = false

[model_providers.exampleprovider]
name = "Example gateway"
base_url = "https://example.com/v1"
wire_api = "responses"
env_key = "EXAMPLE_API_KEY"
timeout = 600
requires_openai_auth = false

As one can see the file has multiple sections. In the generic section one selects the model provider and can set model properties that can also be overridden later on. The model_catalog_json though is important. It contains the base system prompt as well as information about context size and compactification for the given models. I personally think its a good idea to create a single JSON file for all models available on ones own gateway.

The [features] section behaves like the config.toml of codex itself.

Then a provider is specified. This has to match the name of the model_provider. Here one defines a name and a base URI. The wire_api always has to be responses, this field exists for legacy reasons. And since one does not want to leak the OpenAI authentication token one can use requires_openai_auth=false. When supplying the env_key a bearer token is added to each requests Authentication header. The token itself is fetched from an environment variable with the name specified in env_key

Model Metadata

The second important file is the model_catalog_json. This contains a JSON list of all known models.

{
  "models" : [
    {
      "slug": "glm-5.2-744b-preview",
      "display_name": "GLM 5.2 744B Preview",
      "description": "GLM 5.2 744B Preview via Example Gateway",
      "provider": "exampleprovider",
      "visibility": "list",
      "supported_in_api": true,
      "priority": 100,

      "default_reasoning_level": "medium",
      "supported_reasoning_levels": [
        {
          "effort": "low",
          "description": "Fast reasoning"
        },
        {
          "effort": "medium",
          "description": "Balanced reasoning"
        },
        {
          "effort": "high",
          "description": "Deeper reasoning"
        }
      ],
      "supports_reasoning_summaries": false,
      "default_reasoning_summary": "none",
      "support_verbosity": true,
      "default_verbosity": "low",

      "shell_type": "shell_command",
      "apply_patch_tool_type": "freeform",
      "web_search_tool_type": "text_and_image",
      "supports_parallel_tool_calls": false,
      "supports_image_detail_original": false,

      "context_window": 262144,
      "max_context_window": 262144,
      "effective_context_window_percent": 75,

      "truncation_policy": {
        "mode": "tokens",
        "limit": 20000
      },

      "experimental_supported_tools": [],
      "input_modalities": ["text"],
      "supports_search_tool": false,

      "base_instructions": "You are Codex, a coding agent. Work in short, precise steps. Use shell and patch tools carefully. Do not repeat yourself. When tool calls are needed, emit valid tool calls only. If you are stuck, summarize the blocker and stop instead of looping."
    },
    {
      "slug": "glm-4.7-flash:q4_K_M",
      "display_name": "GLM 4.7 Flash",
      "description": "GLM 4.7 Flash via Example gateway",
      "provider": "example",
      "visibility": "list",
      "supported_in_api": true,
      "priority": 100,

      "default_reasoning_level": "medium",
      "supported_reasoning_levels": [
        {
          "effort": "low",
          "description": "Fast reasoning"
        },
        {
          "effort": "medium",
          "description": "Balanced reasoning"
        },
        {
          "effort": "high",
          "description": "Deeper reasoning"
        }
      ],

      "supports_reasoning_summaries": false,
      "default_reasoning_summary": "none",
      "support_verbosity": true,
      "default_verbosity": "low",

      "shell_type": "shell_command",
      "apply_patch_tool_type": "freeform",
      "web_search_tool_type": "text_and_image",
      "supports_parallel_tool_calls": false,
      "supports_image_detail_original": false,

      "context_window": 262144,
      "max_context_window": 262144,
      "effective_context_window_percent": 75,

      "truncation_policy": {
        "mode": "tokens",
        "limit": 20000
      },

      "experimental_supported_tools": [],
      "input_modalities": ["text"],
      "supports_search_tool": false,

      "base_instructions": "You are Codex, a coding agent. Work in short, precise steps. Use shell and patch tools carefully. Do not repeat yourself. When tool calls are needed, emit valid tool calls only. If you are stuck, summarize the blocker and stop instead of looping."
    }
  ]
}

The models in the list are addressed via the slug, which is also displayed in the TUI of codex. The provider has to be specified in the example.config.toml that we created earlier on. In addition only supported reasoning levels should be listed, this is model dependent. As one can see I also disabled reasoning_summaries due to buggy implementations on some of my backends. The shell_type and apply_patch_tool_type depend on the training and expectations of the model one is using. I also disabled parallel tool calls and other features that are not widely supported over all APIs to reduce friction in the compatibility layer. In the end I also disabled the search tool, which would have to be implemented in the gateway (maybe this will happen in future). The base_instructions are then used to specify the system prompt to which the complex prompts by codex get appended.

Note that the parameter effective_context_window_percent is not a model parameter - but a hint to codex how large of the context window one should utilize. Whenever the context goes above the specified threshold, compactification should happen - it is very undesirable to fill up the context window of a model completely due to weird behaviour happening for such edge cases. In addition longer running loops may exhaust an already nearly full context window.

If you need ideas on how to phrase the base_instructions you may take a look at ~/.codex/models_cache.json. The one presented in this article is just a placeholder.

Launching Codex

With the files in place one can launch codex - for this example they are:

• ~/.codex/example.config.toml
• ~/.codex/example-models.json

To launch codex with the specified OpenAI compatible API backend:

env EXAMPLE_API_KEY=sk-example codex -p example -m glm-4.7-flash:q4_K_M

That’s it. If one is lucky codex works - though I experienced that especially for smaller models longer running /goal loops yield corrupted context.

Conclusion

With this setup Codex becomes independent of any particular inference backend. Existing applications continue to use the same Responses API while the gateway translates requests to whatever local or remote model is available. This also makes experimenting with new inference engines considerably easier.

References

• Commercial Products
  • OpenAIs codex
  • OpenAIs ChatGPT
  • OpenAIs platform API
• Frontier Models (Commercial):
  • OpenAIs GPT 5.5
  • OpenAIs GPT 5.6
• Frontier Models (Open):
  • zAIs GLM 5.2
• mini-apigw: A Lightweight Gateway for Multi-Model AI Infrastructure
  • Including the PyPi project and GitHub repository
• The ollama runtime, that makes executing models locally very easy
• llama.cpp as LLM runtime
• colibri as runtime for the GLM-5.2 744B MoE system on a 25GB RAM consumer machine
• TU-Wiens Aqueduct gateway
• A short story of using QWen3.6:35b-a3b with codex

This article is tagged:

• Programming
• Artificial Intelligence
• System administration
• Large Language Models
• Machine learning
• Vibe coding
• Case study