AI API routing guide: sticky egress, concurrency and timeouts

The point of a route for AI API calls isn't which one is fastest — it's which one keeps every request landing on the same egress. A browser tab that drops just needs a refresh; a timeout inside a script can force a whole batch to rerun, and you may pay for the same tokens twice. Below, sticky egress, concurrency and retry timeouts are taken one at a time, followed by a troubleshooting sequence you can follow as written.

How AI API calls differ from browser use

The same subscription that opens pages in a browser may not carry a script. The difference isn't bandwidth — it's the shape of the request and the cost of failure.

• Connection lifetime differs. A page load in the browser is dozens of short requests, each finishing quickly. A streaming API response keeps one connection open for tens of seconds or even minutes, and any jitter along the way shows up directly in the output.
• Failure cost differs. In the browser you retry once. A failed batch job has to be rerun, which burns quota and budget, and resubmitting can produce duplicate results.
• Egress sensitivity differs. Many API services run risk checks on the account plus the egress IP. An egress address that keeps changing within a single day tends to trigger step-up verification, temporary rate limits, and in bad cases an outright refusal.
• The concurrency model differs. A browser opens a handful of tabs; a script can open dozens of connections at once, and the pressure lands on the client connection pool and the egress NAT session table.

Put those four together and the order becomes clear: secure the egress first, then talk about route type, and tune timeout values last.

Sticky egress: why it matters more than raw speed

Sticky egress means requests from one account keep leaving through the same IP, or the same small pool of IPs, over time. It answers the question "does the server recognise you", not "how fast is it".

What egress drift actually breaks

• When sessions or tokens are bound to an IP, reconnecting onto a new IP invalidates them and forces re-authentication.
• Endpoints with an IP allowlist return 403 the moment the egress address changes.
• Risk engines that see one account signing in from several regions in a short window raise the verification level.
• While debugging, the egress IP in the logs is different every time, so it's hard to tell a route problem from a code problem.

A shared egress is not the same as a bad route

Plenty of people blame route quality for slowdowns at peak hours, when the real cause is a shared egress: many users sit behind one IP, the target service sees a burst of requests from that address and starts rate limiting. Check the error type — if you get 429s or connection resets while bandwidth tests look normal, the problem is almost certainly on the egress side.

How to verify the egress is actually sticky

Find an echo endpoint that returns the source IP of the request, send requests back to back with the proxy on, and log each result; disconnect, reconnect and run another round; then test again a few hours later. Only three matching rounds count as sticky.

export ECHO_URL="your echo endpoint"

# Send repeated requests and watch for egress IP drift
for i in 1 2 3 4 5; do
  curl -s --max-time 10 "$ECHO_URL"
  printf '\n'
  sleep 2
done

Matching concurrency to route type

The concurrency ceiling is set by four segments at once: the local connection pool, the multiplexing settings in your proxy client, route bandwidth plus the egress NAT session table, and the account-level rate limit at the target service. Saturate any one of them and you see timeouts or refused connections; looking at only one segment leads to the wrong conclusion.

How the three route types differ

Pick by workload shape, not by price. Streaming and batch jobs go on the private line first; direct is enough for ad-hoc debugging; relay is how you keep the two apart — debugging on one route, production on another.

Two things to check at the protocol layer

Shadowsocks, VMess, Trojan and VLESS run over TCP; Hysteria2 and TUIC are QUIC-based and run over UDP. For API calls, the protocol choice affects two things:

• Clock validation. VMess depends on the client and server agreeing on time; a large system clock skew fails the handshake outright. It looks like a dead route, but one time sync usually fixes it.
• UDP availability. Hysteria2 and TUIC hold up better on lossy paths, but they need a network that doesn't block UDP. Some office and public networks do block it, and then you have to fall back to a TCP-based protocol.

Timeouts and retries: the minimum client-side setup

Timeouts aren't one number. Split a request into four phases: connection setup, TLS handshake, time to first byte, and the idle gap between chunks during streaming. Set them as one lump and you can't tell which phase failed when you debug.

• Connect timeout should be short — if it won't connect, switch routes instead of hanging.
• First-byte timeout covers TLS plus the wait from sending the request to the first byte coming back, so it can be looser than the connect timeout.
• Idle read timeout applies to streaming: it measures only the gap between two chunks, not the total duration.
• Total duration should not be set for streaming requests, or your own client will cut off long answers.

Retries need three conditions: the request is retryable, the error is retryable, and the count is capped. Connection failures, read timeouts, 429 and 5xx are worth retrying; 400, 401 and 403 are parameter or auth errors that behave the same however many times you retry — they just amplify the problem.

import random
import time

import httpx

API_URL = "https://api.example.com/v1/chat"

# Set the four timeouts separately: connect 5s, idle read 60s
timeout = httpx.Timeout(connect=5.0, read=60.0, write=10.0, pool=5.0)
# The pool cap sets your local concurrency ceiling
limits = httpx.Limits(max_connections=16, max_keepalive_connections=8)

with httpx.Client(timeout=timeout, limits=limits) as client:
    for attempt in range(4):
        try:
            with client.stream("POST", API_URL, json={"stream": True}) as resp:
                if resp.status_code in (429, 500, 502, 503, 504):
                    raise httpx.HTTPStatusError(
                        "retryable", request=resp.request, response=resp
                    )
                resp.raise_for_status()
                for line in resp.iter_lines():
                    handle(line)  # Handle each line as it arrives; don't collect into a list first
            break
        except (httpx.ConnectError, httpx.ReadTimeout, httpx.HTTPStatusError):
            if attempt == 3:
                raise
            # Exponential backoff with jitter so jobs don't retry in lockstep
            time.sleep(min(2 ** attempt, 8) + random.uniform(0, 0.5))

How DNS and routing rules work together

A DNS leak means the domain is resolved by your local DNS, so you get the node nearest to you rather than the node nearest to your egress. For most APIs this matters little, since the service uses a single global entry point. But for services that return different results by region, the resolution result decides which region your request hits — and auth can fail along with it.

Split routing beats global mode

Pin the API domains to one route group and send everything else through the default group. Three benefits: stable egress, the ability to switch just that group while debugging, and one failing route that doesn't take other jobs down with it. Write rules by domain suffix rather than by IP — API entry IPs change, domain names don't.

Check order

1. Resolve the target domain with the proxy on and record the result.
2. Turn the proxy off, resolve again, and compare whether both results land in the same region.
3. Confirm the final egress IP with an echo endpoint instead of trusting the resolution result alone.
4. If the resolution result and the egress region disagree, check the client's DNS settings and the priority of your routing rules.

A checklist for choosing a route

The previous sections compressed into a list you can run through when picking a route — more useful than staring at a spec sheet. The four figures below are conditions you can confirm on the subscription side, not speed-test results.

• 120+ countries and regions to egress from
• 150+ routes, splittable by domain
• Unlimited devices — the bottleneck sits at the egress
• 7 days no-questions-asked refund, enough for a full observation run

• ✅ Is the egress IP sticky, and does it stay the same after a reconnect?
• ✅ Can you choose the egress region, and are there several routes to switch between?
• ✅ Does it support long-lived connections and streaming, and does the client buffer?
• ✅ Is per-account concurrency capped, and is the cap local or on the egress side?
• ✅ Does it support per-domain routing, and can those rules be adjusted on their own?
• ✅ Does billing match your call volume: monthly plans reset on the activation date, data packs last until used and never expire.
• ❌ Looking only at headline bandwidth and ignoring egress stability.
• ❌ Running debugging, production and batch jobs through the same egress.
• ❌ Judging a route by a single request.

Troubleshooting: from egress IP to concurrency

When something breaks, don't start by changing parameters. Work through the order below; each step gives you a clear verdict.

1. Confirm the proxy is actually on. Request the echo endpoint and check whether the egress IP is in the target region. If the IP hasn't changed, nothing further is worth checking.
2. Confirm DNS. Compare resolution results inside and outside the proxy to see whether a leak is happening.
3. Confirm system time. VMess and similar protocols rely on clock validation; too much skew fails the handshake.
4. Confirm TLS. Certificate errors, SNI mismatches and interception by middleboxes all show up as connection resets.
5. Confirm streaming output. Look at time to first byte and whether chunks arrive line by line to judge buffering.
6. Confirm concurrency. Ramp up from low concurrency and record the distribution of error types rather than just the average.
7. Confirm with a route switch. Reproduce on another route: if it reproduces, the problem is the client or the code; if not, it's the route.

A week of observation and the conclusion

A single request proves nothing. Cross-border routes vary by time of day — weekday daytime, evening peak and the small hours are three different states. Log the same set of requests for a week, five fields each time:

• Time and route name
• Egress IP
• Time to first byte
• HTTP status code or error type
• Whether a retry was triggered

After a week, read the distribution, not the average: which time window the errors cluster in, which route, which error type. If 429s cluster at evening peak, the egress is shared; if connect timeouts cluster on one route, that route's entry point is congested; if TLS errors appear sporadically, check the client and any middleboxes first.