Azure API Management: Building a Scalable API Gateway

Azure API Management (APIM) is Microsoft's fully managed service for publishing, securing, transforming, and monitoring APIs at enterprise scale. It acts as the critical gateway between your backend services and consumers — mobile apps, partner integrations, developer portals, and microservices.

APIM Architecture

flowchart LR
    subgraph Consumers["API Consumers"]
        MOB[Mobile Apps]
        WEB[Web Clients]
        PARTNER[Partner\nIntegrations]
        INTERNAL[Internal\nMicroservices]
    end

    subgraph APIM["Azure API Management"]
        GW[API Gateway\nPolicies / Routing]
        PORTAL[Developer Portal\nDocs / Testing]
        MGMT[Management API\nProvisioning]

        subgraph Policies["Policy Engine (inbound → backend → outbound)"]
            AUTH[JWT Validation\nOAuth 2.0 / API Key]
            RATE[Rate Limiting\nQuota Enforcement]
            CACHE[Response Caching]
            TRANSFORM[Request/Response\nTransformation]
        end
    end

    subgraph Backends["Backend Services"]
        FUNC[Azure Functions]
        ACA[Container Apps]
        APPSVC[App Service]
        LEGACY[Legacy / On-Prem\nvia VPN]
    end

    Consumers --> GW
    GW --> Policies
    Policies --> Backends
    GW --> PORTAL

    style Consumers fill:#dbeafe,stroke:#3b82f6,color:#1e3a8a
    style APIM fill:#ede9fe,stroke:#8b5cf6,color:#4c1d95
    style Backends fill:#d1fae5,stroke:#059669,color:#065f46

Tier Comparison

Step 1: Create an APIM Instance

az group create --name rg-apim-demo --location eastus

# Developer tier for testing (takes 30–40 min to provision)
az apim create \
  --resource-group rg-apim-demo \
  --name mycompany-apim \
  --publisher-name "My Company" \
  --publisher-email admin@mycompany.com \
  --sku-name Developer \
  --location eastus

# Check provisioning state
az apim show \
  --resource-group rg-apim-demo \
  --name mycompany-apim \
  --query "provisioningState"

Step 2: Import an API

# Import from OpenAPI spec (Swagger URL)
az apim api import \
  --resource-group rg-apim-demo \
  --service-name mycompany-apim \
  --api-id orders-api \
  --path /orders \
  --specification-format OpenApi \
  --specification-url "https://petstore.swagger.io/v2/swagger.json" \
  --display-name "Orders API" \
  --api-type http

Step 3: Policy Pipeline

sequenceDiagram
    participant C as API Consumer
    participant GW as APIM Gateway
    participant BE as Backend Service

    C->>GW: Request + Authorization header
    Note over GW: INBOUND POLICIES
    GW->>GW: Validate JWT (Azure AD)
    GW->>GW: Check rate limit (100/min per subscription)
    GW->>GW: Check cache — hit?
    alt Cache hit
        GW-->>C: 200 Cached response
    else Cache miss
        GW->>BE: Forwarded request (transformed)
        Note over GW: BACKEND POLICIES
        BE-->>GW: Response
        Note over GW: OUTBOUND POLICIES
        GW->>GW: Store in cache (5 min)
        GW->>GW: Remove internal headers
        GW-->>C: 200 Response
    end

Full Inbound + Outbound Policy

<policies>
  <inbound>
    <base />

    <!-- 1. Validate Azure AD JWT -->
    <validate-jwt header-name="Authorization" failed-validation-httpcode="401">
      <openid-config url="https://login.microsoftonline.com/{tenant-id}/v2.0/.well-known/openid-configuration" />
      <required-claims>
        <claim name="aud" match="all">
          <value>{api-client-id}</value>
        </claim>
      </required-claims>
    </validate-jwt>

    <!-- 2. Rate limit: 100 calls per minute per subscription -->
    <rate-limit-by-key
      calls="100"
      renewal-period="60"
      counter-key="@(context.Subscription.Id)" />

    <!-- 3. CORS for web clients -->
    <cors allow-credentials="true">
      <allowed-origins>
        <origin>https://myapp.azurewebsites.net</origin>
      </allowed-origins>
      <allowed-methods>
        <method>GET</method>
        <method>POST</method>
        <method>PUT</method>
        <method>DELETE</method>
      </allowed-methods>
      <allowed-headers>
        <header>Authorization</header>
        <header>Content-Type</header>
      </allowed-headers>
    </cors>

    <!-- 4. Check response cache -->
    <cache-lookup vary-by-developer="false" vary-by-developer-groups="false">
      <vary-by-query-parameter>page</vary-by-query-parameter>
      <vary-by-query-parameter>pageSize</vary-by-query-parameter>
    </cache-lookup>
  </inbound>

  <backend>
    <base />
    <!-- Retry on 5xx with exponential backoff -->
    <retry condition="@(context.Response.StatusCode >= 500)" count="3" interval="2" max-interval="10" delta="2">
      <forward-request />
    </retry>
  </backend>

  <outbound>
    <base />
    <!-- Cache successful responses for 5 minutes -->
    <cache-store duration="300" />
    <!-- Strip internal headers before sending to consumer -->
    <set-header name="X-Powered-By" exists-action="delete" />
    <set-header name="X-AspNet-Version" exists-action="delete" />
  </outbound>

  <on-error>
    <base />
    <!-- Standardise error response format -->
    <set-body>@{
      return new JObject(
        new JProperty("error", context.LastError.Message),
        new JProperty("requestId", context.RequestId)
      ).ToString();
    }</set-body>
  </on-error>
</policies>

Step 4: API Versioning

flowchart TD
    CONSUMER[API Consumer] --> APIM[APIM Gateway]
    APIM --> V1["/v1/orders\n(legacy — maintained)"]
    APIM --> V2["/v2/orders\n(current — new features)"]
    APIM --> V3["/v3/orders\n(preview — limited access)"]

    V1 --> BE1[Orders Service v1\nAzure Functions]
    V2 --> BE2[Orders Service v2\nContainer Apps]
    V3 --> BE3[Orders Service v3\nContainer Apps Preview]

    style V2 fill:#d1fae5,stroke:#059669,color:#065f46
    style V3 fill:#fef3c7,stroke:#f59e0b,color:#78350f
    style V1 fill:#fee2e2,stroke:#ef4444,color:#7f1d1d

# Create versioned API set
az apim api versionset create \
  --resource-group rg-apim-demo \
  --service-name mycompany-apim \
  --display-name "Orders API Versions" \
  --scheme Segment \
  --version-header-name "api-version"

# Create v2 of the API
az apim api create \
  --resource-group rg-apim-demo \
  --service-name mycompany-apim \
  --api-id orders-api-v2 \
  --path /orders \
  --display-name "Orders API v2" \
  --api-version v2

Step 5: Named Values (Secrets Management)

Store reusable configuration as Named Values — never hardcode backend URLs or API keys in policies:

# Store a backend URL as a named value
az apim nv create \
  --resource-group rg-apim-demo \
  --service-name mycompany-apim \
  --named-value-id orders-backend-url \
  --display-name "Orders Backend URL" \
  --value "https://orders-api-prod.azurewebsites.net" \
  --secret false

# Reference in policy: {{orders-backend-url}}

<!-- Reference named value in backend policy -->
<set-backend-service base-url="{{orders-backend-url}}" />

For secrets (API keys, tokens):

# Store as a Key Vault reference
az apim nv create \
  --resource-group rg-apim-demo \
  --service-name mycompany-apim \
  --named-value-id backend-api-key \
  --display-name "Backend API Key" \
  --key-vault-secret-identifier "https://myvault.vault.azure.net/secrets/BackendApiKey" \
  --secret true

Step 6: Monitoring

# Enable Application Insights integration
az apim logger create \
  --resource-group rg-apim-demo \
  --service-name mycompany-apim \
  --logger-id apim-ai-logger \
  --logger-type applicationInsights \
  --connection-string "InstrumentationKey=xxx;IngestionEndpoint=https://eastus-0.in.applicationinsights.azure.com/"

# Diagnostic settings on the API
az apim api diagnostic create \
  --resource-group rg-apim-demo \
  --service-name mycompany-apim \
  --api-id orders-api \
  --logger-id apim-ai-logger \
  --sampling-percentage 100

Key KQL queries for APIM:

// Request volume and error rate by API operation
ApiManagementGatewayLogs
| where TimeGenerated > ago(1h)
| summarize
    Total = count(),
    Errors = countif(ResponseCode >= 400),
    AvgDuration = avg(TotalTime)
  by OperationId
| extend ErrorRate = Errors * 100.0 / Total
| order by Total desc

// Top consumers by request volume
ApiManagementGatewayLogs
| where TimeGenerated > ago(24h)
| summarize RequestCount = count() by SubscriptionId
| order by RequestCount desc
| take 10

Best Practices

Use Named Values for all configuration. Never hardcode backend URLs, API keys, or environment-specific values in policy XML — Named Values make policies environment-independent.

Version from day one. It is far harder to add versioning to an existing API than to build it in from the start. Use URL path versioning (/v1/, /v2/) for simplicity.

Apply rate limiting per subscription, not per IP. IP-based limiting breaks behind NAT. Subscription-based limiting is fair and auditable.

Use retry policies for resilience. Wrap backend calls in <retry> with exponential backoff — transient errors are common in distributed systems.

Enable caching for read-heavy APIs. Even a 60-second cache on list endpoints dramatically reduces backend load at peak traffic.

Troubleshooting

Key Takeaways

• ✅ APIM decouples consumers from backends — backend changes never break consumer contracts
• ✅ Policy XML handles auth, rate limiting, caching, and transformation at the gateway layer
• ✅ Named Values keep policies clean and environment-agnostic
• ✅ Version from day one — URL path versioning is the most universally compatible approach
• ✅ Application Insights integration gives you full visibility into API usage and latency

Additional Resources

• Azure API Management documentation
• Policy reference
• API versioning guide
• Developer portal customisation
• APIM Bicep reference

What policies have been most valuable in your APIM implementation? Share your patterns below.