AI Universe · Model access
Model access zoom 2
How does the app reach a model? 4 concept buckets. Each one is a job you can learn once; underneath it, who does that job on each platform.
Concept
Inference API
The request–response door to a model: messages in, text or tool calls out.
chat-completions shapestreamingtool calling
Physical: who does this job, where
| Open source | OpenAI-compatible servers (vLLM, Ollama) |
|---|---|
| Azure | Foundry Models / Azure OpenAI |
| AWS | Bedrock |
| GCP | Vertex AI |
| Vendors | OpenAI, Anthropic, Groq APIs |
Concept
Gateway
One governed door for every model call: keys, limits, logging, screening.
allow-listrate and token limitskey managementfailover
Physical: who does this job, where
| Open source | LiteLLM |
|---|---|
| Azure | API Management AI gateway |
| AWS | Bedrock + API Gateway |
| GCP | Apigee |
| Vendors | Portkey, Kong, Cloudflare AI Gateway |
Concept
Routing and model choice
Deciding which model answers which request.
rules firsta router modelcheapest model that passes
Physical: who does this job, where
| Open source | RouteLLM, your own rules |
|---|---|
| Azure | Foundry model router |
| AWS | Bedrock intelligent prompt routing |
| GCP | — |
| Vendors | OpenRouter, Martian |
Concept
Caching
Not paying twice for the same work.
prompt (prefix) cachingsemantic caching
Physical: who does this job, where
| Open source | Redis, GPTCache |
|---|---|
| Azure | APIM semantic cache |
| AWS | Bedrock prompt caching |
| GCP | Vertex context caching |
| Vendors | provider prompt caching |