**Z notes on API architecture - companion to [APIM notes](id:56fabaf6-e8aa-45d0-a1c1-89f247f0a93f)**

# 1\. What is API Architecture?

API architecture is the set of rules, patterns, and structural decisions that govern how APIs are designed, exposed, consumed, and maintained across a system. It sits above individual API implementation - it's about **how APIs fit together** as a platform.

In Microlise's context: the APIOps pipeline, the APIM gateway layer, OpenShift clusters, and the OpenAPI specs are all artefacts **of** an architectural decision. Understanding the architecture behind them makes the pipeline choices make sense.

# 2\. Architectural Styles

Different styles define how clients and servers communicate. These are not mutually exclusive - a platform can expose multiple styles simultaneously (e.g. REST externally, gRPC internally).

## 2.1 REST (Representational State Transfer)

The dominant style for public and partner APIs. Key constraints:

  - **Stateless**: Each request must contain all the context needed to fulfil it. No session state is stored server-side between calls.

  - **Resource-oriented**: APIs are modelled around nouns (resources), not verbs (actions).
    
      - Good: `GET /vehicles/{id}`
      - Bad: `POST /getVehicle`

  - **Uniform interface**: Standard HTTP verbs carry semantic meaning:
    
    | Verb   | Meaning                     |
    | ------ | --------------------------- |
    | GET    | Read a resource             |
    | POST   | Create a resource           |
    | PUT    | Replace a resource entirely |
    | PATCH  | Partially update a resource |
    | DELETE | Remove a resource           |
    

  - **Layered system**: Clients don't know if they're talking to the real backend or a gateway/proxy/cache. This is exactly what Microlise's APIM layer provides.

  - **Cacheable**: Responses should declare whether they can be cached, enabling CDN and client-side optimisation.

OpenAPI Specification (OAS/Swagger) is the standard way to **describe** a REST API. The `spec` referred to throughout the APIOps pipeline is this document.

## 2.2 GraphQL

A query language for APIs developed by Meta. Instead of fixed endpoints, clients send a query describing exactly what data they need.

  - Single endpoint: `POST /graphql`
  - Client drives the shape of the response - no over-fetching or under-fetching.
  - Good for: complex, interconnected data models; front-end teams who iterate quickly.
  - Trade-off: harder to cache, more complex server-side resolver logic, linting/governance is less mature than OAS.

Not currently the Microlise APIM pattern but worth understanding as a contrast.

## 2.3 gRPC (Google Remote Procedure Call)

Uses Protocol Buffers (protobuf) as the interface definition language and HTTP/2 as transport.

  - Strongly typed contracts defined in `.proto` files.
  - Extremely high performance - binary serialisation, multiplexed streams.
  - Ideal for: internal service-to-service calls, microservices, high-throughput scenarios.
  - Trade-off: not human-readable, harder to test with standard tooling (curl, Postman), less browser-friendly.

Think of gRPC as what might live **behind** an API gateway - internal communication between microservices - while REST/OAS faces outward toward customers.

## 2.4 AsyncAPI / Event-Driven APIs

Not all APIs are request-response. Event-driven APIs use messaging patterns:

  - **Webhooks**: Server POSTs to a client-registered URL when an event occurs.
  - **WebSockets**: Persistent bi-directional connection between client and server.
  - **Server-Sent Events (SSE)**: One-way stream from server to client.
  - **Message queues** (Kafka, RabbitMQ, Azure Service Bus): Decoupled async messaging.

AsyncAPI is the OAS equivalent for event-driven interfaces - a specification format for documenting these contracts.

# 3\. API Gateway Pattern

This is the core of what APIM implements. An API gateway sits as an intermediary between consumers (customers, internal teams) and backend services.

``` mermaid
flowchart TD
    EC[External Consumer]
    GW["API Gateway / APIM
    ────────────────────
    Auth · Rate Limiting
    Transforms · Routing
    Logging · Caching"]
    SA["Service A\n(OpenShift)"]
    SB["Service B\n(OpenShift)"]
    SC["Service C\n(OpenShift)"]

    EC --> GW
    GW --> SA
    GW --> SB
    GW --> SC
```

## 3.1 What the gateway does

| Concern                     | What it means                                                             |
| --------------------------- | ------------------------------------------------------------------------- |
| **Authentication**          | Verifies who the caller is (OAuth2 tokens, API keys, mutual TLS)          |
| **Authorisation**           | Determines what the caller is allowed to do (scopes, claims)              |
| **Rate limiting**           | Caps requests per second/minute/hour per consumer or globally             |
| **Throttling**              | Gracefully slows or queues excess requests rather than rejecting them     |
| **Request transformation**  | Rewrites headers, payloads, or paths before forwarding to backends        |
| \*Response transformation\* | Strips internal fields, reformats responses for the consumer contract     |
| **Routing**                 | Directs traffic to the correct backend based on path, headers, or content |
| **Load balancing**          | Distributes traffic across backend instances                              |
| **Caching**                 | Stores responses to reduce backend load for idempotent requests           |
| **Observability**           | Centralises access logs, metrics, and tracing across all APIs             |

In Azure APIM specifically, these concerns are implemented as **policies** - XML-based declarative rules that run at gateway level.

## 3.2 Reverse Proxy vs API Gateway

The Microlise notes reference a Reverse Proxy pipeline alongside APIM. These are related but distinct:

| Aspect             | Reverse Proxy                  | API Gateway                                    |
| ------------------ | ------------------------------ | ---------------------------------------------- |
| Primary purpose    | Routing and TLS termination    | Full API lifecycle management                  |
| Protocol awareness | Layer 4/7 (TCP/HTTP)           | Layer 7, API-aware (understands REST, OAS)     |
| Policy engine      | Minimal (Nginx/HAProxy config) | Rich (auth, transforms, quotas, subscriptions) |
| Developer portal   | No                             | Yes - consumer-facing API catalogue            |
| Examples           | Nginx, HAProxy, Traefik        | Azure APIM, Kong, AWS API Gateway              |

A common pattern (and likely what Microlise uses) is: Reverse proxy handles ingress and TLS termination → traffic forwarded to APIM for policy enforcement → APIM routes to OpenShift services.

# 4\. API Design Principles

## 4.1 Contract-First Design

Define the OpenAPI spec **before** writing implementation code. The spec is the source of truth.

Benefits:

  - Frontend/consumer teams can mock and build against the spec immediately.
  - Linting pipelines (like the one in APIOps gated build) can enforce governance before any code ships.
  - Breaking change detection is automated.

The APIOps pipeline enforces this: the spec is committed to source control, linted, reviewed by the API Governance Council, and only then does the publish pipeline sync it to APIM environments.

## 4.2 Versioning Strategies

APIs evolve. Versioning prevents changes from breaking existing consumers.

| Strategy              | Example                               | Trade-offs                                          |
| --------------------- | ------------------------------------- | --------------------------------------------------- |
| **URI versioning**    | `/v1/vehicles`, `/v2/vehicles`        | Explicit, cacheable, easy to route. Pollutes paths. |
| **Header versioning** | `Accept: application/vnd.api.v2+json` | Clean URIs. Harder to test, less cache-friendly.    |
| **Query param**       | `/vehicles?version=2`                 | Simple but considered poor practice for REST.       |

URI versioning is the most common and is what Azure APIM handles well via routing rules.

## 4.3 Breaking vs Non-Breaking Changes

Knowing what constitutes a breaking change is critical for API governance (i.e. why the API GC reviews spec PRs).

| **Non-breaking (additive)**             | **Breaking**                                      |
| --------------------------------------- | ------------------------------------------------- |
| Adding a new optional field to response | Removing or renaming a field                      |
| Adding a new endpoint                   | Changing a field's type                           |
| Adding a new optional query parameter   | Making an optional parameter required             |
| New enum value (with caution)           | Changing HTTP status codes for existing scenarios |
|                                         | Changing authentication schemes                   |

## 4.4 Resource Naming Conventions

  - Use **nouns**, not verbs: `/journeys` not `/getJourneys`
  - Use **plural** for collections: `/vehicles` not `/vehicle`
  - Use **kebab-case** for multi-word: `/driver-events` not `/driverEvents`
  - Nest to show ownership, but limit depth: `/vehicles/{id}/journeys` is fine; `/vehicles/{id}/journeys/{jid}/events/{eid}/metadata` is not.
  - Never expose internal implementation details in paths (`/{internalDatabaseId}` leaks schema).

## 4.5 HTTP Status Codes

Correct status codes are part of the API contract. Misuse breaks consumers who rely on them.

| Code | Meaning               | When to use                                        |
| ---- | --------------------- | -------------------------------------------------- |
| 200  | OK                    | Successful GET, PUT, PATCH                         |
| 201  | Created               | Successful POST that created a resource            |
| 204  | No Content            | Successful DELETE or action with no response body  |
| 400  | Bad Request           | Client sent malformed/invalid data                 |
| 401  | Unauthorized          | Not authenticated (no or invalid token)            |
| 403  | Forbidden             | Authenticated but not authorised for this resource |
| 404  | Not Found             | Resource does not exist                            |
| 409  | Conflict              | State conflict (duplicate, version mismatch)       |
| 422  | Unprocessable Entity  | Semantically invalid (e.g. invalid date range)     |
| 429  | Too Many Requests     | Rate limit exceeded                                |
| 500  | Internal Server Error | Unhandled server-side failure                      |
| 503  | Service Unavailable   | Downstream dependency down, circuit breaker open   |

# 5\. API Security Architecture

## 5.1 Authentication Patterns

| Pattern            | How it works                                                       | Typical use                                |
| ------------------ | ------------------------------------------------------------------ | ------------------------------------------ |
| **API Keys**       | Static key passed in header (`x-api-key`) or query param           | Simple, internal/partner APIs              |
| **OAuth 2.0**      | Token-based; client obtains a bearer token from auth server        | Public APIs, delegated access              |
| **OpenID Connect** | OAuth 2.0 + identity layer (ID tokens, user info endpoint)         | APIs that need to know **who** the user is |
| **Mutual TLS**     | Both client and server present certificates                        | High-security service-to-service           |
| **JWT**            | Signed token carrying claims; verified without calling auth server | Stateless auth at gateway level            |

Azure APIM supports all of these via policies. A common pattern: APIM validates the JWT at the gateway before the request ever reaches an OpenShift pod.

## 5.2 OAuth 2.0 Grant Types

| Grant type                    | Use case                                                     |
| ----------------------------- | ------------------------------------------------------------ |
| **Client Credentials**        | Machine-to-machine (no user involved). Most common for APIs. |
| **Authorization Code**        | User-facing apps; user logs in and delegates access          |
| **Authorization Code + PKCE** | Same as above but for SPAs/mobile (no client secret)         |
| **Implicit** (deprecated)     | Was used for SPAs - replaced by Auth Code + PKCE             |

## 5.3 Zero Trust at the API Layer

Zero Trust means: **never trust, always verify** - even internal services must authenticate.

Principles applied to APIs:

  - Every service-to-service call requires a valid token (no implicit trust on the internal network).
  - Tokens have minimum required scopes (principle of least privilege).
  - mTLS between internal services adds a second layer even if a token is compromised.
  - All traffic - internal and external - goes through the gateway and is logged.

# 6\. API Lifecycle Management

This maps directly to the APIOps workflow in the APIM notes.

``` mermaid
flowchart LR
    Design[Design]
    Develop[Develop]
    Test[Test]
    Publish[Publish]
    Monitor[Monitor]
    Retire[Retire]

    Design --> Develop --> Test --> Publish --> Monitor --> Retire

    Design -.-> D1["OAS Spec\nContract First"]
    Develop -.-> D2["C# Project\nTemplates\nOpenShift"]
    Test -.-> D3["Gated Build\nPipeline\nLinting + API GC"]
    Publish -.-> D4["APIM Publish\nPipeline\nDev → Cert → Prod"]
    Monitor -.-> D5["Analytics\nDashboards\nAPIM Portal"]
    Retire -.-> D6["Deprecation\nNotices\nVersion Sunset"]
```

## 6.1 API Governance

The API Governance Council (API GC) referenced in the notes is the enforcement body for architectural standards. Common governance concerns:

  - **Linting**: Automated rules against the OAS spec. The APIOps pipeline uses scripts from `ApiManagement.Pipeline.AgentScripts` to enforce this.
  - **Review gates**: No spec change merges without GC approval - prevents inconsistent or insecure APIs reaching production.
  - **Naming standards**: Enforced in the spec review (see §4.4).
  - **Breaking change policy**: Defines how long old versions must be supported before retirement.
  - **Security policy**: All APIs must use approved auth methods; no unauthenticated endpoints in production.

## 6.2 APIOps (GitOps for APIs)

APIOps applies GitOps principles to API management: the APIM configuration is stored as code in a Git repository and the pipeline is the only mechanism that changes APIM state.

Key properties:

  - **Declarative**: The `Microlise.APIOps` repo describes the desired state of all APIs in APIM.
  - **Versioned**: Every change is a PR - full audit trail.
  - **Automated**: The publish pipeline does the two-way sync; no manual APIM portal edits.
  - **Environment promotion**: Changes flow Dev -\> Cert -\> Prod, with a manual approval gate before Prod.

This is analogous to how Terraform or Helm work for infrastructure - the repo **is** the truth.

# 7\. API Observability

An often-overlooked architectural concern. APIs you can't observe are APIs you can't operate.

## 7.1 The Three Pillars

| Pillar      | What it captures                                            | Tooling examples                   |
| ----------- | ----------------------------------------------------------- | ---------------------------------- |
| **Logs**    | Discrete events: requests, responses, errors, auth failures | Azure Monitor, ELK, Splunk         |
| **Metrics** | Aggregated numbers over time: latency, error rate, RPS      | Prometheus, Azure Metrics, Grafana |
| **Traces**  | End-to-end request journey across services                  | Jaeger, Zipkin, Azure App Insights |

## 7.2 Key API Metrics to Track

  - **Latency**: p50, p95, p99 - not just average. Averages hide outliers.
  - **Error rate**: 5xx rate (server errors) and 4xx rate (client errors) separately.
  - **Throughput**: Requests per second - used to set rate limits and plan capacity.
  - **Availability**: Uptime percentage. SLAs are usually defined here (99.9% = \~8.7h downtime/year).
  - **Quota consumption**: How much of a consumer's rate limit are they using?

Azure APIM exposes all of these natively and can emit them to Azure Monitor.

## 7.3 Correlation IDs

Every request should carry a unique `correlation-id` (or `x-request-id`) header. The gateway generates one if the client doesn't provide it and forwards it to all downstream services. This makes it possible to trace a single user request across multiple microservice logs.

``` mermaid
flowchart LR
    C[Client]
    APIM["APIM\ngenerates correlation-id: abc-123"]
    SA["Service A logs\nabc-123 · vehicle lookup"]
    SB["Service B logs\nabc-123 · journey history"]

    C --> APIM
    APIM --> SA
    SA --> SB
```

# 8\. Microservices & API Design

The OpenShift deployment model in Microlise's stack implies microservices. API architecture must account for how services communicate internally vs. externally.

## 8.1 Internal vs External APIs

| Aspect          | Internal (East-West)                    | External (North-South)                     |
| --------------- | --------------------------------------- | ------------------------------------------ |
| Consumers       | Other microservices                     | Customers, partners, third parties         |
| Protocol        | gRPC, internal REST, message queues     | REST over HTTPS via APIM                   |
| Auth            | mTLS, service accounts, internal tokens | OAuth2, API keys managed by APIM           |
| Discoverability | Service mesh / internal DNS             | Developer portal in APIM                   |
| Governance      | Team-level conventions                  | API GC, formal versioning, SLA commitments |

## 8.2 API Aggregation / BFF Pattern

Backend for Frontend (BFF): a dedicated API layer tailored to a specific consumer (e.g. a mobile app, a portal). Instead of the consumer calling 5 microservices, a BFF aggregates them into a single call.

``` mermaid
flowchart LR
    MA[Mobile App]
    BFF[BFF: Mobile API]
    VS[Vehicle Service]
    JS[Journey Service]
    DS[Driver Service]

    MA --> BFF
    BFF --> VS
    BFF --> JS
    BFF --> DS
```

APIM policies can implement lightweight aggregation, but for complex cases a dedicated BFF service is cleaner.

## 8.3 Service Mesh (Complementary to API Gateway)

A service mesh (e.g. Istio, Linkerd) manages **internal** service-to-service communication within OpenShift/Kubernetes:

  - mTLS between pods automatically.
  - Traffic policies (retries, circuit breaking) at the network level.
  - Observability (traces, metrics) without code changes.

The API Gateway handles North-South (external) traffic; the service mesh handles East-West (internal). They are complementary, not competing.

# 9\. API Design Patterns

## 9.1 Pagination

Never return unbounded collections. Standard patterns:

  - **Offset/limit**: `GET /journeys?offset=0&limit=50`. Simple but inefficient at high offsets.
  - **Cursor-based**: `GET /journeys?cursor=eyJpZCI6MTAwfQ==`. Efficient for large datasets; the cursor encodes the last seen position.
  - **Page-based**: `GET /journeys?page=3&pageSize=50`. User-friendly but shares offset's inefficiency.

Response should include metadata:

``` json
{
  "data": [...],
  "pagination": {
    "total": 1420,
    "limit": 50,
    "nextCursor": "eyJpZCI6MTUwfQ=="
  }
}
```

## 9.2 Filtering, Sorting, and Field Selection

  - Filtering: `GET /vehicles?status=active&driverType=HGV`
  - Sorting: `GET /journeys?sort=-startedAt` (prefix `-` for descending)
  - Field selection (sparse fieldsets): `GET /vehicles?fields=id,registration,status` - reduces payload size.

## 9.3 Idempotency

A request is idempotent if making it multiple times produces the same result as making it once. Crucial for retry logic.

| Method | Idempotent? | Safe (no side effects)? |
| ------ | ----------- | ----------------------- |
| GET    | Yes         | Yes                     |
| PUT    | Yes         | No                      |
| DELETE | Yes         | No                      |
| POST   | No          | No                      |
| PATCH  | No\*        | No                      |

\*PATCH can be designed to be idempotent but isn't by definition.

For non-idempotent operations (POST), use an `Idempotency-Key` header. The server stores the result keyed to that value; duplicate requests return the cached result rather than processing again.

## 9.4 HATEOAS

Hypermedia as the Engine of Application State - responses include links to related actions:

``` json
{
  "id": "v-123",
  "registration": "AB12 CDE",
  "_links": {
    "self": { "href": "/vehicles/v-123" },
    "journeys": { "href": "/vehicles/v-123/journeys" },
    "driver": { "href": "/drivers/d-456" }
  }
}
```

Rarely implemented fully in practice but worth understanding as the most complete expression of REST.

# 10\. OpenAPI Specification Deep Dive

Since OAS is central to the APIOps pipeline, understanding its structure is practical knowledge.

``` yaml
openapi: "3.1.0"
info:
  title: Vehicle Service API
  version: "2.0.0"
  description: Manages vehicle records for the Microlise platform.

servers:
  - url: https://api.microlise.com/v2
    description: Production

paths:
  /vehicles/{vehicleId}:
    get:
      summary: Get a vehicle by ID
      operationId: getVehicleById          # Unique identifier used in code gen
      tags: [Vehicles]
      parameters:
        - name: vehicleId
          in: path
          required: true
          schema:
            type: string
            format: uuid
      responses:
        "200":
          description: Vehicle found
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Vehicle"
        "404":
          $ref: "#/components/responses/NotFound"

components:
  schemas:
    Vehicle:
      type: object
      required: [id, registration]
      properties:
        id:
          type: string
          format: uuid
        registration:
          type: string
          example: "AB12 CDE"
        status:
          type: string
          enum: [active, inactive, maintenance]

  responses:
    NotFound:
      description: Resource not found
      content:
        application/json:
          schema:
            $ref: "#/components/schemas/ProblemDetails"

  securitySchemes:
    oauth2:
      type: oauth2
      flows:
        clientCredentials:
          tokenUrl: https://auth.microlise.com/oauth2/token
          scopes:
            vehicles:read: Read vehicle data
            vehicles:write: Create and update vehicles

security:
  - oauth2: [vehicles:read]
```

Key OAS concepts:

  - `operationId`: Used by code generators and APIM to reference operations in policies.
  - `$ref`: DRY principle - define schemas and responses once, reuse everywhere.
  - `components`: The library section - schemas, parameters, responses, security schemes.
  - `tags`: Grouping for the developer portal - consumers see organised API docs.
  - `security`: Applied globally here; can be overridden per-operation.

# 11\. Connecting the Dots: Microlise Architecture Map

Mapping the APIM notes to the architectural concepts above:

| APIM Note Item                           | Architectural Concept                              |
| ---------------------------------------- | -------------------------------------------------- |
| Separate layer between customers & APIs  | API Gateway Pattern (§3)                           |
| OpenAPI spec / swagger build             | Contract-First Design (§4.1), OAS (§10)            |
| APIOps pipeline, spec in git             | APIOps / GitOps for APIs (§6.2)                    |
| API Governance Council review            | API Governance (§6.1), Breaking Changes (§4.3)     |
| Linting scripts (`AgentScripts`)         | Automated governance enforcement                   |
| Gated + main pipeline                    | CI/CD gates for quality and security               |
| Two-way sync, publish pipeline           | Declarative state management (APIOps)              |
| Dev -\> Cert -\> Prod with approval gate | Environment promotion pattern (§6.2)               |
| Gen cluster vs Prod cluster              | Environment isolation, blast radius reduction      |
| Reverse proxy pipeline                   | Reverse Proxy vs API Gateway (§3.2)                |
| Quay registry, OpenShift containers      | Microservices deployment, East-West traffic (§8.1) |
| Swagger/OAS spec as PR artefact          | Spec-as-code, version-controlled contracts         |

# 12\. Further Reading & Reference

## Recommended (from APIM notes)

  - Quick Start Kubernetes - Nigel Poulton ([OpenShift vs K8s](https://microliseuk.sharepoint.com/sites/StorageCompute/ContainerPlatformUsers/SitePages/How-is-OpenShift-different-from-Kubernetes.aspx))

## Additional Architecture Resources

  - [OpenAPI Specification 3.1.0 (official)](https://spec.openapis.org/oas/v3.1.0)
  - [AsyncAPI Documentation](https://www.asyncapi.com/docs)
  - [gRPC Official Docs](https://grpc.io/docs/)
  - [Azure API Management Docs](https://learn.microsoft.com/en-us/azure/api-management/)
  - [Microservices.io - API Gateway Pattern](https://microservices.io/patterns/apigateway.html)
  - [OAuth 2.0 (oauth.net)](https://oauth.net/2/)
  - [Swagger / OAS Reference](https://swagger.io/specification/)

## Key Terms Glossary

| Term            | Definition                                                                   |
| --------------- | ---------------------------------------------------------------------------- |
| OAS / Swagger   | OpenAPI Specification - a standard format for describing REST APIs           |
| APIOps          | Applying GitOps principles to API management (spec-as-code, pipeline-driven) |
| APIM            | API Management - the platform/layer that governs API lifecycle               |
| Gateway         | Intermediary that enforces policy (auth, rate limiting, routing) for APIs    |
| Spec            | Short for specification - the OAS JSON/YAML document describing an API       |
| Idempotency     | Property where repeating a request has the same effect as making it once     |
| mTLS            | Mutual TLS - both parties in a connection authenticate with certificates     |
| BFF             | Backend for Frontend - an API tailored to a specific consumer's needs        |
| Service Mesh    | Infrastructure layer managing internal service-to-service communication      |
| Breaking Change | An API change that requires existing consumers to update their integration   |