Skip to content

README.md

Latest commit

 

History

History
154 lines (105 loc) · 5.25 KB

README.md

File metadata and controls

154 lines (105 loc) · 5.25 KB
Raw

Run SimpleL7Proxy

Run the proxy locally or in Azure, connect one backend, and verify that traffic reaches it.

TL;DR

  1. Clone the repository and choose a local or Azure setup.
  2. Connect an LLM endpoint, APIM instance, or the included LLM simulator.
  3. Check readiness, send a request, and confirm the proxy response headers.

Expected outcome: /readiness returns 200 OK, and a proxied response identifies the selected backend in the BackendHost header.

1. Clone the Repository

Run all remaining commands from the repository root unless a step says otherwise.

git clone https://github.com/microsoft/SimpleL7Proxy.git
cd SimpleL7Proxy

2. Choose Where to Run

Use the local path for development and the Container Apps path for an Azure deployment.

Run Locally

Use .NET 10 or Docker.

Run in Azure Container Apps

Use the repository deployment workflow.

Azure Container Apps deployment

Note

When running a container with Port=8000, publish the same container port: -p 8000:8000. See Deploy to Azure Container Apps for image and ingress configuration.

3. Connect a Backend

Configure one backend before checking readiness.

LLM Endpoint

Use an Azure OpenAI or Azure AI Foundry endpoint.

API Management

Use an APIM gateway with the priority-and-retry policy.

LLM Simulator

Run the included simulator without a real model endpoint.

Warning

/liveness can succeed before a backend is eligible. Do not send test traffic until /readiness returns 200 OK.

4. Verify the Proxy

Check health first, then send a request through the proxy.

curl -i http://localhost:8000/liveness
curl -i http://localhost:8000/readiness
curl -i http://localhost:8000/your/path

A successful proxied response normally includes these proxy-generated headers:

Header Meaning
BackendHost Backend that handled the request
Request-Queue-Duration Time spent in the priority queue
Request-Process-Duration Time spent processing after dequeue
Total-Latency Total time from enqueue to response
Attempts Backend attempts in the current dispatch cycle
Lifetime-Attempts Backend attempts across requeue cycles

Exhausted-host error responses use a different set of diagnostic headers, including x-Request-Queue-Duration and x-Total-Latency. See Headers and Status Codes for the response-specific contract.

Use CompanionApp

Use CompanionApp for an interactive request and telemetry view.

cd src/CompanionApp
dotnet run

Open the URL printed by CompanionApp, then configure the server as http://localhost:8000 or use the HTTPS URL assigned to your Container App.

CompanionApp request

If Event Hub is configured, connect it in CompanionApp to inspect proxy activity. Select a thumbnail to open the full-size image.

Event Hub request view Event Hub monitor view Event Hub insights view
Event Hub enqueue view Event Hub attempt view Event Hub final response view

Verification Checklist

  • /liveness returns 200 OK.
  • /readiness returns 200 OK.
  • A test request returns the expected backend response.
  • BackendHost identifies the selected backend on a successful proxied response.
  • CompanionApp or eventslog.json shows the request when telemetry is configured.

Next Steps

Task Document
Run from source or Docker Run Locally
Configure Azure App Configuration Configure Azure App Configuration
Deploy to Azure Container Apps Deploy to Azure Container Apps
Test without a model endpoint LLM Simulator
Review runtime settings Environment Variables
Contribute to the project Development and Contributing
Diagnose startup or request failures Troubleshooting

For exact health behavior and a focused verification sequence, see Verify SimpleL7Proxy.