Step 3

Create an LLM Proxy

⏱ 7 min

Overview

An LLM Proxy is the core object in AI Gateway. It is the OpenAI-compatible endpoint your applications call instead of calling LLM providers directly. In this step you’ll configure the proxy and connect it to an LLM provider via model-based routing.

Step 1 — Navigate to LLM Proxies

Go to omni.mulesoft.com and log in
In the lower-left corner, confirm you are in the business group assigned to you
In the left navigation, click LLM Proxies
Click Add LLM Proxy

Step 2 — Configure Gateway tab

Fill in the following fields:

Field	Value	Notes
Proxy Name	`<your-name>-proxy`	Use a unique name to identify your proxy
Description	`LLM Proxy for Connect AI 2026 workshop`	Free text
Base path	`/my-proxy`	Must start with `/` — this becomes part of your endpoint URL
Environment	`Sandbox`	Select the Sandbox environment
Omni Gateway	Select your gateway from the list	Click the radio button next to your assigned gateway
Port	`8081`	Keep the default value
Consumer endpoint	`<your-omni-gateway-base-url>/my-proxy`	Join your Omni Gateway base URL with the base path — e.g. `https://attendee-gateway-abc123.cloudhub.io/my-proxy`

Click Continue.

Step 3 — Configure Routing Strategy tab

Under Routing strategy, select Model-based

Configure the route:

Field	Value	Notes
Target → Provider	`OpenAI`	Select OpenAI as the LLM provider
Model	`GPT-4o Mini`	Select GPT-4o Mini from the model dropdown
Authentication → Destination URL	(provided by your workshop instructor)	LLM Proxy Endpoint
API Key	(provided by your workshop instructor)	LLM Proxy Token

Click Add LLM Proxy

Your LLM Proxy is now deployed. You should see it listed under LLM Proxies with status Active.

Note: The Destination URL and API Key point to a shared workshop proxy that routes to OpenAI on your behalf — you do not need your own OpenAI API key for this workshop.

What’s next

Your LLM Proxy is live. The next step adds governance policies to control how it’s consumed.

Continue to Apply Policies →