Data Science Wire

Regional inference now available on AI Gateway

Vercel Blog - AI5d4 min read

AI Gateway now supports regional inference . Set inferenceRegion on a request to pin it to the US or EU. Every model provider that supports the selected region handles it the same way. Inference runs there, and any data the provider keeps is stored there. AI Gateway supports two pinned regions, plus global routing: Region Where inference runs us A US data center eu An EU data center global Any region If no model provider can serve it, the request fails rather than running somewhere else. Every response reports the region that served it, so you can confirm where each request ran. Here's a reque

Read the full story at Vercel Blog - AI

More in MLOps / LLMOps