:PROPERTIES:
:ID:       49b195c8-e116-40ca-86e8-62c65dbb5a4f
:END:
#+title: API Architecture
#+filetags: :technical:notes:api:
#+category: Career Concepts

*Z notes on API architecture - companion to [[id:56fabaf6-e8aa-45d0-a1c1-89f247f0a93f][APIM notes]]*

* 1. What is API Architecture?

API architecture is the set of rules, patterns, and structural decisions that govern how APIs are designed, exposed, consumed, and maintained across a system. It sits above individual API implementation - it's about *how APIs fit together* as a platform.

In Microlise's context: the APIOps pipeline, the APIM gateway layer, OpenShift clusters, and the OpenAPI specs are all artefacts *of* an architectural decision. Understanding the architecture behind them makes the pipeline choices make sense.

* 2. Architectural Styles

Different styles define how clients and servers communicate. These are not mutually exclusive - a platform can expose multiple styles simultaneously (e.g. REST externally, gRPC internally).

** 2.1 REST (Representational State Transfer)

The dominant style for public and partner APIs. Key constraints:

- *Stateless*: Each request must contain all the context needed to fulfil it. No session state is stored server-side between calls.
- *Resource-oriented*: APIs are modelled around nouns (resources), not verbs (actions).
  - Good: ~GET /vehicles/{id}~
  - Bad: ~POST /getVehicle~
- *Uniform interface*: Standard HTTP verbs carry semantic meaning:
  | Verb   | Meaning                        |
  |--------+--------------------------------|
  | GET    | Read a resource                |
  | POST   | Create a resource              |
  | PUT    | Replace a resource entirely    |
  | PATCH  | Partially update a resource    |
  | DELETE | Remove a resource              |
- *Layered system*: Clients don't know if they're talking to the real backend or a gateway/proxy/cache. This is exactly what Microlise's APIM layer provides.
- *Cacheable*: Responses should declare whether they can be cached, enabling CDN and client-side optimisation.

OpenAPI Specification (OAS/Swagger) is the standard way to *describe* a REST API. The ~spec~ referred to throughout the APIOps pipeline is this document.

** 2.2 GraphQL

A query language for APIs developed by Meta. Instead of fixed endpoints, clients send a query describing exactly what data they need.

- Single endpoint: ~POST /graphql~
- Client drives the shape of the response - no over-fetching or under-fetching.
- Good for: complex, interconnected data models; front-end teams who iterate quickly.
- Trade-off: harder to cache, more complex server-side resolver logic, linting/governance is less mature than OAS.

Not currently the Microlise APIM pattern but worth understanding as a contrast.

** 2.3 gRPC (Google Remote Procedure Call)

Uses Protocol Buffers (protobuf) as the interface definition language and HTTP/2 as transport.

- Strongly typed contracts defined in ~.proto~ files.
- Extremely high performance - binary serialisation, multiplexed streams.
- Ideal for: internal service-to-service calls, microservices, high-throughput scenarios.
- Trade-off: not human-readable, harder to test with standard tooling (curl, Postman), less browser-friendly.

Think of gRPC as what might live *behind* an API gateway - internal communication between microservices - while REST/OAS faces outward toward customers.

** 2.4 AsyncAPI / Event-Driven APIs

Not all APIs are request-response. Event-driven APIs use messaging patterns:

- *Webhooks*: Server POSTs to a client-registered URL when an event occurs.
- *WebSockets*: Persistent bi-directional connection between client and server.
- *Server-Sent Events (SSE)*: One-way stream from server to client.
- *Message queues* (Kafka, RabbitMQ, Azure Service Bus): Decoupled async messaging.

AsyncAPI is the OAS equivalent for event-driven interfaces - a specification format for documenting these contracts.

* 3. API Gateway Pattern

This is the core of what APIM implements. An API gateway sits as an intermediary between consumers (customers, internal teams) and backend services.

#+begin_src mermaid
flowchart TD
    EC[External Consumer]
    GW["API Gateway / APIM
    ────────────────────
    Auth · Rate Limiting
    Transforms · Routing
    Logging · Caching"]
    SA["Service A\n(OpenShift)"]
    SB["Service B\n(OpenShift)"]
    SC["Service C\n(OpenShift)"]

    EC --> GW
    GW --> SA
    GW --> SB
    GW --> SC
#+end_src

** 3.1 What the gateway does

| Concern                  | What it means                                                                 |
|--------------------------+-------------------------------------------------------------------------------|
| *Authentication*         | Verifies who the caller is (OAuth2 tokens, API keys, mutual TLS)              |
| *Authorisation*          | Determines what the caller is allowed to do (scopes, claims)                  |
| *Rate limiting*          | Caps requests per second/minute/hour per consumer or globally                 |
| *Throttling*             | Gracefully slows or queues excess requests rather than rejecting them         |
| *Request transformation* | Rewrites headers, payloads, or paths before forwarding to backends            |
| *Response transformation*| Strips internal fields, reformats responses for the consumer contract         |
| *Routing*                | Directs traffic to the correct backend based on path, headers, or content     |
| *Load balancing*         | Distributes traffic across backend instances                                  |
| *Caching*                | Stores responses to reduce backend load for idempotent requests               |
| *Observability*          | Centralises access logs, metrics, and tracing across all APIs                 |

In Azure APIM specifically, these concerns are implemented as *policies* - XML-based declarative rules that run at gateway level.

** 3.2 Reverse Proxy vs API Gateway

The Microlise notes reference a Reverse Proxy pipeline alongside APIM. These are related but distinct:

| Aspect            | Reverse Proxy                             | API Gateway                                      |
|-------------------+-------------------------------------------+--------------------------------------------------|
| Primary purpose   | Routing and TLS termination               | Full API lifecycle management                    |
| Protocol awareness| Layer 4/7 (TCP/HTTP)                      | Layer 7, API-aware (understands REST, OAS)       |
| Policy engine     | Minimal (Nginx/HAProxy config)            | Rich (auth, transforms, quotas, subscriptions)   |
| Developer portal  | No                                        | Yes - consumer-facing API catalogue              |
| Examples          | Nginx, HAProxy, Traefik                   | Azure APIM, Kong, AWS API Gateway                |

A common pattern (and likely what Microlise uses) is: Reverse proxy handles ingress and TLS termination → traffic forwarded to APIM for policy enforcement → APIM routes to OpenShift services.

* 4. API Design Principles

** 4.1 Contract-First Design

Define the OpenAPI spec *before* writing implementation code. The spec is the source of truth.

Benefits:
- Frontend/consumer teams can mock and build against the spec immediately.
- Linting pipelines (like the one in APIOps gated build) can enforce governance before any code ships.
- Breaking change detection is automated.

The APIOps pipeline enforces this: the spec is committed to source control, linted, reviewed by the API Governance Council, and only then does the publish pipeline sync it to APIM environments.

** 4.2 Versioning Strategies

APIs evolve. Versioning prevents changes from breaking existing consumers.

| Strategy            | Example                               | Trade-offs                                          |
|---------------------+---------------------------------------+-----------------------------------------------------|
| *URI versioning*    | ~/v1/vehicles~, ~/v2/vehicles~        | Explicit, cacheable, easy to route. Pollutes paths. |
| *Header versioning* | ~Accept: application/vnd.api.v2+json~ | Clean URIs. Harder to test, less cache-friendly.    |
| *Query param*       | ~/vehicles?version=2~                 | Simple but considered poor practice for REST.       |

URI versioning is the most common and is what Azure APIM handles well via routing rules.

** 4.3 Breaking vs Non-Breaking Changes

Knowing what constitutes a breaking change is critical for API governance (i.e. why the API GC reviews spec PRs).

| *Non-breaking (additive)*              | *Breaking*                                       |
|----------------------------------------+--------------------------------------------------|
| Adding a new optional field to response | Removing or renaming a field                    |
| Adding a new endpoint                  | Changing a field's type                          |
| Adding a new optional query parameter  | Making an optional parameter required            |
| New enum value (with caution)          | Changing HTTP status codes for existing scenarios|
|                                        | Changing authentication schemes                  |

** 4.4 Resource Naming Conventions

- Use *nouns*, not verbs: ~/journeys~ not ~/getJourneys~
- Use *plural* for collections: ~/vehicles~ not ~/vehicle~
- Use *kebab-case* for multi-word: ~/driver-events~ not ~/driverEvents~
- Nest to show ownership, but limit depth: ~/vehicles/{id}/journeys~ is fine; ~/vehicles/{id}/journeys/{jid}/events/{eid}/metadata~ is not.
- Never expose internal implementation details in paths (~/{internalDatabaseId}~ leaks schema).

** 4.5 HTTP Status Codes

Correct status codes are part of the API contract. Misuse breaks consumers who rely on them.

| Code | Meaning                       | When to use                                        |
|------+-------------------------------+----------------------------------------------------|
| 200  | OK                            | Successful GET, PUT, PATCH                         |
| 201  | Created                       | Successful POST that created a resource            |
| 204  | No Content                    | Successful DELETE or action with no response body  |
| 400  | Bad Request                   | Client sent malformed/invalid data                 |
| 401  | Unauthorized                  | Not authenticated (no or invalid token)            |
| 403  | Forbidden                     | Authenticated but not authorised for this resource |
| 404  | Not Found                     | Resource does not exist                            |
| 409  | Conflict                      | State conflict (duplicate, version mismatch)       |
| 422  | Unprocessable Entity          | Semantically invalid (e.g. invalid date range)     |
| 429  | Too Many Requests             | Rate limit exceeded                                |
| 500  | Internal Server Error         | Unhandled server-side failure                      |
| 503  | Service Unavailable           | Downstream dependency down, circuit breaker open   |

* 5. API Security Architecture

** 5.1 Authentication Patterns

| Pattern          | How it works                                               | Typical use                              |
|------------------+------------------------------------------------------------+------------------------------------------|
| *API Keys*       | Static key passed in header (~x-api-key~) or query param   | Simple, internal/partner APIs            |
| *OAuth 2.0*      | Token-based; client obtains a bearer token from auth server| Public APIs, delegated access            |
| *OpenID Connect* | OAuth 2.0 + identity layer (ID tokens, user info endpoint) | APIs that need to know *who* the user is |
| *Mutual TLS*     | Both client and server present certificates                | High-security service-to-service         |
| *JWT*            | Signed token carrying claims; verified without calling auth server | Stateless auth at gateway level |

Azure APIM supports all of these via policies. A common pattern: APIM validates the JWT at the gateway before the request ever reaches an OpenShift pod.

** 5.2 OAuth 2.0 Grant Types

| Grant type              | Use case                                                       |
|-------------------------+----------------------------------------------------------------|
| *Client Credentials*    | Machine-to-machine (no user involved). Most common for APIs.   |
| *Authorization Code*    | User-facing apps; user logs in and delegates access            |
| *Authorization Code + PKCE* | Same as above but for SPAs/mobile (no client secret)      |
| *Implicit* (deprecated) | Was used for SPAs - replaced by Auth Code + PKCE               |

** 5.3 Zero Trust at the API Layer

Zero Trust means: *never trust, always verify* - even internal services must authenticate.

Principles applied to APIs:
- Every service-to-service call requires a valid token (no implicit trust on the internal network).
- Tokens have minimum required scopes (principle of least privilege).
- mTLS between internal services adds a second layer even if a token is compromised.
- All traffic - internal and external - goes through the gateway and is logged.

* 6. API Lifecycle Management

This maps directly to the APIOps workflow in the APIM notes.

#+begin_src mermaid
flowchart LR
    Design[Design]
    Develop[Develop]
    Test[Test]
    Publish[Publish]
    Monitor[Monitor]
    Retire[Retire]

    Design --> Develop --> Test --> Publish --> Monitor --> Retire

    Design -.-> D1["OAS Spec\nContract First"]
    Develop -.-> D2["C# Project\nTemplates\nOpenShift"]
    Test -.-> D3["Gated Build\nPipeline\nLinting + API GC"]
    Publish -.-> D4["APIM Publish\nPipeline\nDev → Cert → Prod"]
    Monitor -.-> D5["Analytics\nDashboards\nAPIM Portal"]
    Retire -.-> D6["Deprecation\nNotices\nVersion Sunset"]
#+end_src

** 6.1 API Governance

The API Governance Council (API GC) referenced in the notes is the enforcement body for architectural standards. Common governance concerns:

- *Linting*: Automated rules against the OAS spec. The APIOps pipeline uses scripts from ~ApiManagement.Pipeline.AgentScripts~ to enforce this.
- *Review gates*: No spec change merges without GC approval - prevents inconsistent or insecure APIs reaching production.
- *Naming standards*: Enforced in the spec review (see §4.4).
- *Breaking change policy*: Defines how long old versions must be supported before retirement.
- *Security policy*: All APIs must use approved auth methods; no unauthenticated endpoints in production.

** 6.2 APIOps (GitOps for APIs)

APIOps applies GitOps principles to API management: the APIM configuration is stored as code in a Git repository and the pipeline is the only mechanism that changes APIM state.

Key properties:
- *Declarative*: The ~Microlise.APIOps~ repo describes the desired state of all APIs in APIM.
- *Versioned*: Every change is a PR - full audit trail.
- *Automated*: The publish pipeline does the two-way sync; no manual APIM portal edits.
- *Environment promotion*: Changes flow Dev  ->  Cert  ->  Prod, with a manual approval gate before Prod.

This is analogous to how Terraform or Helm work for infrastructure - the repo *is* the truth.

* 7. API Observability

An often-overlooked architectural concern. APIs you can't observe are APIs you can't operate.

** 7.1 The Three Pillars

| Pillar    | What it captures                                            | Tooling examples                   |
|-----------+-------------------------------------------------------------+------------------------------------|
| *Logs*    | Discrete events: requests, responses, errors, auth failures | Azure Monitor, ELK, Splunk         |
| *Metrics* | Aggregated numbers over time: latency, error rate, RPS      | Prometheus, Azure Metrics, Grafana |
| *Traces*  | End-to-end request journey across services                  | Jaeger, Zipkin, Azure App Insights |

** 7.2 Key API Metrics to Track

- *Latency*: p50, p95, p99 - not just average. Averages hide outliers.
- *Error rate*: 5xx rate (server errors) and 4xx rate (client errors) separately.
- *Throughput*: Requests per second - used to set rate limits and plan capacity.
- *Availability*: Uptime percentage. SLAs are usually defined here (99.9% = ~8.7h downtime/year).
- *Quota consumption*: How much of a consumer's rate limit are they using?

Azure APIM exposes all of these natively and can emit them to Azure Monitor.

** 7.3 Correlation IDs

Every request should carry a unique ~correlation-id~ (or ~x-request-id~) header. The gateway generates one if the client doesn't provide it and forwards it to all downstream services. This makes it possible to trace a single user request across multiple microservice logs.

#+begin_src mermaid
flowchart LR
    C[Client]
    APIM["APIM\ngenerates correlation-id: abc-123"]
    SA["Service A logs\nabc-123 · vehicle lookup"]
    SB["Service B logs\nabc-123 · journey history"]

    C --> APIM
    APIM --> SA
    SA --> SB
#+end_src

* 8. Microservices & API Design

The OpenShift deployment model in Microlise's stack implies microservices. API architecture must account for how services communicate internally vs. externally.

** 8.1 Internal vs External APIs

| Aspect         | Internal (East-West)                       | External (North-South)                       |
|----------------+--------------------------------------------+----------------------------------------------|
| Consumers      | Other microservices                        | Customers, partners, third parties           |
| Protocol       | gRPC, internal REST, message queues        | REST over HTTPS via APIM                     |
| Auth           | mTLS, service accounts, internal tokens    | OAuth2, API keys managed by APIM             |
| Discoverability| Service mesh / internal DNS                | Developer portal in APIM                     |
| Governance     | Team-level conventions                     | API GC, formal versioning, SLA commitments   |

** 8.2 API Aggregation / BFF Pattern

Backend for Frontend (BFF): a dedicated API layer tailored to a specific consumer (e.g. a mobile app, a portal). Instead of the consumer calling 5 microservices, a BFF aggregates them into a single call.

#+begin_src mermaid
flowchart LR
    MA[Mobile App]
    BFF[BFF: Mobile API]
    VS[Vehicle Service]
    JS[Journey Service]
    DS[Driver Service]

    MA --> BFF
    BFF --> VS
    BFF --> JS
    BFF --> DS
#+end_src

APIM policies can implement lightweight aggregation, but for complex cases a dedicated BFF service is cleaner.

** 8.3 Service Mesh (Complementary to API Gateway)

A service mesh (e.g. Istio, Linkerd) manages *internal* service-to-service communication within OpenShift/Kubernetes:

- mTLS between pods automatically.
- Traffic policies (retries, circuit breaking) at the network level.
- Observability (traces, metrics) without code changes.

The API Gateway handles North-South (external) traffic; the service mesh handles East-West (internal). They are complementary, not competing.

* 9. API Design Patterns

** 9.1 Pagination

Never return unbounded collections. Standard patterns:

- *Offset/limit*: ~GET /journeys?offset=0&limit=50~. Simple but inefficient at high offsets.
- *Cursor-based*: ~GET /journeys?cursor=eyJpZCI6MTAwfQ==~. Efficient for large datasets; the cursor encodes the last seen position.
- *Page-based*: ~GET /journeys?page=3&pageSize=50~. User-friendly but shares offset's inefficiency.

Response should include metadata:
#+BEGIN_SRC json
{
  "data": [...],
  "pagination": {
    "total": 1420,
    "limit": 50,
    "nextCursor": "eyJpZCI6MTUwfQ=="
  }
}
#+END_SRC

** 9.2 Filtering, Sorting, and Field Selection

- Filtering: ~GET /vehicles?status=active&driverType=HGV~
- Sorting: ~GET /journeys?sort=-startedAt~ (prefix ~-~ for descending)
- Field selection (sparse fieldsets): ~GET /vehicles?fields=id,registration,status~ - reduces payload size.

** 9.3 Idempotency

A request is idempotent if making it multiple times produces the same result as making it once. Crucial for retry logic.

| Method | Idempotent? | Safe (no side effects)? |
|--------+-------------+-------------------------|
| GET    | Yes         | Yes                     |
| PUT    | Yes         | No                      |
| DELETE | Yes         | No                      |
| POST   | No          | No                      |
| PATCH  | No*         | No                      |

*PATCH can be designed to be idempotent but isn't by definition.

For non-idempotent operations (POST), use an ~Idempotency-Key~ header. The server stores the result keyed to that value; duplicate requests return the cached result rather than processing again.

** 9.4 HATEOAS

Hypermedia as the Engine of Application State - responses include links to related actions:

#+BEGIN_SRC json
{
  "id": "v-123",
  "registration": "AB12 CDE",
  "_links": {
    "self": { "href": "/vehicles/v-123" },
    "journeys": { "href": "/vehicles/v-123/journeys" },
    "driver": { "href": "/drivers/d-456" }
  }
}
#+END_SRC

Rarely implemented fully in practice but worth understanding as the most complete expression of REST.

* 10. OpenAPI Specification Deep Dive

Since OAS is central to the APIOps pipeline, understanding its structure is practical knowledge.

#+BEGIN_SRC yaml
openapi: "3.1.0"
info:
  title: Vehicle Service API
  version: "2.0.0"
  description: Manages vehicle records for the Microlise platform.

servers:
  - url: https://api.microlise.com/v2
    description: Production

paths:
  /vehicles/{vehicleId}:
    get:
      summary: Get a vehicle by ID
      operationId: getVehicleById          # Unique identifier used in code gen
      tags: [Vehicles]
      parameters:
        - name: vehicleId
          in: path
          required: true
          schema:
            type: string
            format: uuid
      responses:
        "200":
          description: Vehicle found
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Vehicle"
        "404":
          $ref: "#/components/responses/NotFound"

components:
  schemas:
    Vehicle:
      type: object
      required: [id, registration]
      properties:
        id:
          type: string
          format: uuid
        registration:
          type: string
          example: "AB12 CDE"
        status:
          type: string
          enum: [active, inactive, maintenance]

  responses:
    NotFound:
      description: Resource not found
      content:
        application/json:
          schema:
            $ref: "#/components/schemas/ProblemDetails"

  securitySchemes:
    oauth2:
      type: oauth2
      flows:
        clientCredentials:
          tokenUrl: https://auth.microlise.com/oauth2/token
          scopes:
            vehicles:read: Read vehicle data
            vehicles:write: Create and update vehicles

security:
  - oauth2: [vehicles:read]
#+END_SRC

Key OAS concepts:
- ~operationId~: Used by code generators and APIM to reference operations in policies.
- ~$ref~: DRY principle - define schemas and responses once, reuse everywhere.
- ~components~: The library section - schemas, parameters, responses, security schemes.
- ~tags~: Grouping for the developer portal - consumers see organised API docs.
- ~security~: Applied globally here; can be overridden per-operation.

* 11. Connecting the Dots: Microlise Architecture Map

Mapping the APIM notes to the architectural concepts above:

| APIM Note Item                             | Architectural Concept                                 |
|--------------------------------------------+-------------------------------------------------------|
| Separate layer between customers & APIs    | API Gateway Pattern (§3)                           |
| OpenAPI spec / swagger build               | Contract-First Design (§4.1), OAS (§10)         |
| APIOps pipeline, spec in git               | APIOps / GitOps for APIs (§6.2)                    |
| API Governance Council review              | API Governance (§6.1), Breaking Changes (§4.3)  |
| Linting scripts (~AgentScripts~)           | Automated governance enforcement                      |
| Gated + main pipeline                      | CI/CD gates for quality and security                  |
| Two-way sync, publish pipeline             | Declarative state management (APIOps)                 |
| Dev  ->  Cert  ->  Prod with approval gate | Environment promotion pattern (§6.2)               |
| Gen cluster vs Prod cluster                | Environment isolation, blast radius reduction         |
| Reverse proxy pipeline                     | Reverse Proxy vs API Gateway (§3.2)                |
| Quay registry, OpenShift containers        | Microservices deployment, East-West traffic (§8.1) |
| Swagger/OAS spec as PR artefact            | Spec-as-code, version-controlled contracts            |

* 12. Further Reading & Reference

** Recommended (from APIM notes)

- Quick Start Kubernetes - Nigel Poulton ([[https://microliseuk.sharepoint.com/sites/StorageCompute/ContainerPlatformUsers/SitePages/How-is-OpenShift-different-from-Kubernetes.aspx][OpenShift vs K8s]])

** Additional Architecture Resources

- [[https://spec.openapis.org/oas/v3.1.0][OpenAPI Specification 3.1.0 (official)]]
- [[https://www.asyncapi.com/docs][AsyncAPI Documentation]]
- [[https://grpc.io/docs/][gRPC Official Docs]]
- [[https://learn.microsoft.com/en-us/azure/api-management/][Azure API Management Docs]]
- [[https://microservices.io/patterns/apigateway.html][Microservices.io - API Gateway Pattern]]
- [[https://oauth.net/2/][OAuth 2.0 (oauth.net)]]
- [[https://swagger.io/specification/][Swagger / OAS Reference]]

** Key Terms Glossary

| Term           | Definition                                                                   |
|----------------+------------------------------------------------------------------------------|
| OAS / Swagger  | OpenAPI Specification - a standard format for describing REST APIs           |
| APIOps         | Applying GitOps principles to API management (spec-as-code, pipeline-driven) |
| APIM           | API Management - the platform/layer that governs API lifecycle               |
| Gateway        | Intermediary that enforces policy (auth, rate limiting, routing) for APIs    |
| Spec           | Short for specification - the OAS JSON/YAML document describing an API       |
| Idempotency    | Property where repeating a request has the same effect as making it once     |
| mTLS           | Mutual TLS - both parties in a connection authenticate with certificates     |
| BFF            | Backend for Frontend - an API tailored to a specific consumer's needs        |
| Service Mesh   | Infrastructure layer managing internal service-to-service communication      |
| Breaking Change| An API change that requires existing consumers to update their integration   |