Migrate Codex CLI to DeepSeek with LiteLLM Proxy

Introduction

Many developers have built internal automation scripts and code analysis tools based on OpenAI Codex series models, including gpt-3.5-turbo-instruct and the older code-davinci-002. These CLI-based coding assistants deliver reliable code generation capabilities, but teams face two obvious operational pain points.

First, OpenAI’s pricing and service stability introduce cost uncertainty. API expenses can spike quickly under heavy workloads, and service interruptions may block development pipelines. Second, domestic open-source large models such as DeepSeek provide competitive code-generation quality at a lower cost. However, developers cannot directly point existing Codex CLI tools to DeepSeek endpoints.

The core obstacle is protocol incompatibility. Classic Codex relies on the Completion API, while DeepSeek’s public model service exposes Chat Completion interfaces. Request structures, parameter definitions and response formats differ substantially. Rewriting every existing CLI tool to adapt to the new chat API requires massive engineering overhead. LiteLLM solves this problem by acting as a translation proxy. It exposes an OpenAI-compatible /v1/completions endpoint locally, then converts incoming Completion requests into valid Chat Completion calls for DeepSeek. This lets legacy Codex CLI tools run with almost zero source-code modification.

This article walks through the complete migration workflow. It covers fundamental protocol differences, LiteLLM proxy deployment, YAML model configuration, code modification for CLI clients, request-response translation logic, troubleshooting common runtime errors, and production-grade optimization strategies.

1. Core Differences: Completion API vs Chat Completion API

Before configuring the proxy layer, developers must fully understand the two distinct interaction paradigms. The difference goes beyond simple field naming; they represent fundamentally separate design patterns for LLM inference.

1.1 Completion API: The Continuation Model

OpenAI Codex is built around the Completion API, available at the /v1/completions endpoint. This API is designed for text continuation and code autocompletion.

Sample request structure:

{
  "model": "code-davinci-002",
  "prompt": "Write a Python function to calculate Fibonacci numbers\n",
  "temperature": 0.2,
  "max_tokens": 500,
  "stop": ["\n\n"]
}

Key characteristics:

• prompt: A single plain text input. The model treats this string as incomplete content and continues writing from its end.
• No native concept of multi-turn dialogue or role separation. There is no distinction between system, user and assistant messages.
• Generated text returns in the choices[0].text field of the response.

This design perfectly matches code completion and single-shot text generation scenarios. Legacy Codex CLI tools usually construct one long concatenated prompt containing context history and task instructions and submit it via this endpoint.

1.2 Chat Completion API: Structured Dialogue Model

Models including GPT-3.5-turbo, GPT-4 and DeepSeek use /v1/chat/completions. Requests are structured as a message array with role tags.

Sample request structure:

{
  "model": "deepseek-chat",
  "messages": [
    {"role": "system", "content": "You are a helpful coding assistant."},
    {"role": "user", "content": "Write a Python function to calculate Fibonacci numbers."}
  ],
  "temperature": 0.2,
  "max_tokens": 500
}

Key characteristics:

• messages: An array of objects tagged with system, user, or assistant roles. This formalizes multi-turn conversation context.
• The model reads the conversation history and generates an assistant reply responding to the latest user message. The system message sets global behavioral rules.
• Output content resides in choices[0].message.content.

1.3 Root of Migration Conflicts

A raw Completion JSON payload sent directly to DeepSeek’s chat endpoint will return a 400 Bad Request error. The DeepSeek API expects a messages array, not a standalone prompt string.

LiteLLM works as a translation middleware.

• Exposed to the local Codex CLI: It serves the standard OpenAI /v1/completions interface and accepts prompt parameters.
• Internal forwarding: It converts the single prompt string into a properly formatted messages array and sends the request to DeepSeek’s Chat Completion API.
• Response reformatting: It extracts message.content from DeepSeek’s reply and repackages it into the text field required by Completion clients.

The proxy also handles parameter mapping, unsupported argument filtering and stream conversion.

2. Deploy LiteLLM as a Model Proxy Gateway

LiteLLM is a Python library with powerful capabilities to run independently as a proxy server. We will deploy this local service as the unified entry point for all Codex CLI traffic.

2.1 Environment Preparation and LiteLLM Installation

A Python 3.8 or newer environment is required. Using a virtual environment is strongly recommended to avoid dependency conflicts.

# Create and activate virtual environment
python -m venv litellm-env
# Linux / macOS
source litellm-env/bin/activate
# Windows
litellm-env\Scripts\activate

# Install LiteLLM
pip install litellm

After installation, the litellm CLI command becomes available.

2.2 Configure DeepSeek as Backend Model

Prepare your DeepSeek API key and base endpoint. Store credentials in environment variables to avoid hardcoding secrets inside configuration files.

# Set DeepSeek API credentials
export DEEPSEEK_API_KEY="sk-your-deepseek-api-key-here"
export DEEPSEEK_API_BASE="https://api.deepseek.com"

Create a YAML configuration file named model_config.yaml to map a custom model alias to DeepSeek chat.

model_list:
  - model_name: my-deepseek-coder
    litellm_params:
      model: deepseek-chat
      api_key: os.environ/DEEPSEEK_API_KEY
      api_base: os.environ/DEEPSEEK_API_BASE

Configuration explanation:

• model_name: Custom alias. Your CLI will reference this name with --model my-deepseek-coder. This is the only model reference you need to modify in client code.
• litellm_params.model: The internal LiteLLM identifier for DeepSeek chat.
• api_key and api_base: Read values from environment variables.

2.3 Launch LiteLLM Proxy Server

Start the proxy service and load the YAML config. The server simulates the OpenAI API schema.

litellm --config ./model_config.yaml --api_base http://localhost:4000 --drop_params

Parameter breakdown:

• --config: Path pointing to your YAML model definition.
• --api_base http://localhost:4000: The local listening address and port. You can change to any unused port.
• --drop_params: Critical flag. It discards parameters unsupported by DeepSeek’s chat endpoint. Many native Completion parameters will trigger request rejection if forwarded unfiltered.

Once started, you will see log output confirming:
LiteLLM: Proxy server started on http://localhost:4000

This local service acts as an OpenAI-compatible gateway. It accepts both /v1/completions and /v1/chat/completions requests on port 4000.

3. Modify Codex CLI Client to Target Local Proxy

Original Python SDK code calling OpenAI Completion might look like this:

import openai

openai.api_base = "https://api.openai.com/v1/"

response = openai.Completion.create(
    model="code-davinci-002",
    prompt="Write a quicksort function",
    temperature=0.2,
    max_tokens=500
)
print(response.choices[0].text)

Only two changes are required to redirect traffic to LiteLLM proxy:

import openai

# 1. Point api_base to local LiteLLM proxy
openai.api_base = "http://localhost:4000/v1/"

# 2. Use custom alias defined in model_config.yaml
response = openai.Completion.create(
    model="my-deepseek-coder",
    prompt="Write a quicksort function",
    temperature=0.2,
    max_tokens=500
)
print(response.choices[0].text)

The rest of your source code remains unchanged. openai.Completion.create will send a valid Completion request to http://localhost:4000/v1/completions. LiteLLM translates and forwards it to DeepSeek automatically.

For enterprise developers running multiple model proxies and routing complex LLM traffic, 4sapi, an API gateway, can assist with unified access management and traffic governance alongside local proxy deployments.

4. Core Translation Logic and Parameter Mapping

Understanding how LiteLLM converts requests helps diagnose unexpected behavior after migration.

4.1 Prompt to Messages Conversion

LiteLLM’s default behavior maps the raw Completion prompt string into the user message content in the chat request. Developers can optionally inject static system instructions inside YAML configuration.

Original Completion request:

{
  "prompt": "Write a quicksort function in Python",
  "temperature":0.2
}

Converted Chat Completion payload sent to DeepSeek:

{
  "model":"deepseek-chat",
  "messages":[
    {"role":"user","content":"Write a quicksort function in Python"}
  ],
  "temperature":0.2
}

Important caveat: If your legacy prompt depends heavily on Codex’s native continuation behavior and relies on special token sequences, the translated prompt may lose subtle context semantics. Complex use cases require careful prompt tuning.

4.2 Parameter Mapping & Unsupported Arguments

Parameters with direct cross-model compatibility:

• max_tokens
• temperature
• top_p
• stream (streaming output works for DeepSeek chat)

Parameters that require filtering:
best_of, logprobs, suffix are exclusive to OpenAI Completion. If these pass to DeepSeek, they will cause 400 errors. The --drop_params flag removes unsupported keys automatically.

4.3 Reverse Response Format Translation

DeepSeek returns content inside choices[0].message.content. LiteLLM repackages this value into choices[0].text so legacy Codex CLI parsers work with zero edits.

DeepSeek raw response:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Here is the Python quicksort implementation..."
      },
      "finish_reason": "stop"
    }
  ]
}

Transformed Completion response returned to CLI:

{
  "choices": [
    {
      "text": "Here is the Python quicksort implementation...",
      "finish_reason": "stop"
    }
  ]
}

The client application reads .text just like it did with original Codex responses.

5. Practical Debugging & Common Troubleshooting

Even with correct configuration, migration frequently hits predictable runtime failures. This section lists typical error patterns and diagnostic steps.

5.1 Invalid model name / Model not in config

Symptom: LiteLLM rejects requests and logs invalid model identifier.

• Verify model_name in YAML matches exactly the model string passed by CLI. Case sensitivity applies.
• Inspect LiteLLM startup logs to confirm your YAML file loaded successfully.
• Validate YAML syntax; YAML is strict about indentation.

5.2 401 Authentication Error or Missing API Key

Symptom: DeepSeek backend returns authentication failure.

• Check environment variables DEEPSEEK_API_KEY and DEEPSEEK_API_BASE.
• Confirm the API key has available quota and permission access to deepseek-chat.

5.3 400 Bad Request returned by DeepSeek

Symptom: LiteLLM forwards request but DeepSeek rejects payload.

• Start LiteLLM with --debug flag to print raw outbound request body.
• Inspect whether unsupported parameters remain in the request. Always use --drop_params.
• Confirm DeepSeek base URL is correct; do not append /v1/chat/completions manually.

5.4 Output quality or speed differs from original Codex

Model behavior differences are expected. DeepSeek-chat and Codex are distinct models with different training distributions.

• Tune temperature values. Chat models often respond differently to temperature settings than older completion models.
• Inject a fixed system prompt inside the LiteLLM YAML config to enforce coding style and constraints.

model_list:
  - model_name: my-deepseek-coder
    litellm_params:
      model: deepseek-chat
      api_key: os.environ/DEEPSEEK_API_KEY
      api_base: os.environ/DEEPSEEK_API_BASE
    model_info:
      system_prompt: "You are an expert Python programmer. Always write concise, efficient, and well-commented code."

• Enable streaming if your CLI supports it. LiteLLM will relay stream chunks from DeepSeek.

6. Advanced Configuration & Production Hardening

After basic validation, add production safeguards to improve reliability.

6.1 Multi-model Fallback and Load Balancing

The YAML file can define multiple backends, including multiple DeepSeek endpoints across regions, or mix OpenAI and Claude models. When one backend fails, LiteLLM automatically fails over to the next model entry with identical model_name. This improves pipeline availability.

6.2 Rate Limiting and Request Throttling

LiteLLM proxy supports global and per-model rate limits to prevent accidental API quota exhaustion.

litellm --config ./model_config.yaml --api_base http://localhost:4000 --drop_params --rpm 100

--rpm 100 restricts requests to 100 requests per minute.

6.3 Secret Authentication for Proxy Endpoint

Add an API key requirement to secure your local proxy service so only authorized clients can access it.

litellm --config ./model_config.yaml --api_base http://localhost:4000 --drop_params --master-key sk-proxy-secret

Clients must include the key inside the Authorization header to call the proxy.

6.4 Logging and Observability

For long-running production workloads, structured logging is essential. LiteLLM can write logs to files and integrate with Langfuse for usage tracking, cost analytics and trace inspection. Detailed request tracing helps pinpoint protocol conversion defects.

7. Migration Evaluation and Best Practices

Performance & Cost Assessment

DeepSeek’s API pricing is generally cheaper than OpenAI Codex. The proxy layer introduces minor network forwarding latency, usually tens of milliseconds. For code generation tasks, this overhead is often acceptable.

Practical Recommendations

1. Start small. Test with simple scripts first. Validate output quality, code style and failure modes before migrating all production CLI workloads.
2. Use a dedicated system prompt. This stabilizes coding output and aligns DeepSeek behavior closer to your original Codex expectations.
3. Keep monitoring enabled. Track request volume, failure rate and latency metrics for the proxy and backend model service.
4. Expect prompt rework. Even with protocol translation, model behavior differs. Prepare prompt adjustments to compensate for model-specific biases.

Conclusion

LiteLLM removes the largest barrier to migrating legacy Codex CLI tools from OpenAI Completion APIs to DeepSeek chat models. It acts as a translation proxy, bridging the structural gap between the older single-prompt Completion interface and modern message-based Chat Completion schema.

The workflow only requires minimal client-side edits. Developers retain all existing tooling, automation pipelines and prompt logic while switching the underlying LLM provider. This pattern is broadly reusable for any legacy Completion workload migrating to chat-native open models.

While LiteLLM handles most translation work, teams must still test output quality, tune prompts and implement production safeguards such as fallback routing, rate limits and audit logging. The approach balances low migration cost with the economic benefits of switching to domestic open code models.

International access: https://4sapi.com
Domestic access: https://4sapi.cn