zaine/org_roam
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
86ccc39daeb52480c129317b64bfeaf20aefb48f
org_roam/Career Concepts/20260121173318-api_architecture.org
Zaine Arch 86ccc39dae Updating log output
2026-05-06 22:54:52 +01:00

26 KiB
Raw Blame History

API Architecture

Z notes on API architecture - companion to APIM notes

1. What is API Architecture?

API architecture is the set of rules, patterns, and structural decisions that govern how APIs are designed, exposed, consumed, and maintained across a system. It sits above individual API implementation - it's about how APIs fit together as a platform.

In Microlise's context: the APIOps pipeline, the APIM gateway layer, OpenShift clusters, and the OpenAPI specs are all artefacts of an architectural decision. Understanding the architecture behind them makes the pipeline choices make sense.

2. Architectural Styles

Different styles define how clients and servers communicate. These are not mutually exclusive - a platform can expose multiple styles simultaneously (e.g. REST externally, gRPC internally).

2.1 REST (Representational State Transfer)

The dominant style for public and partner APIs. Key constraints:

  • Stateless: Each request must contain all the context needed to fulfil it. No session state is stored server-side between calls.

  • Resource-oriented: APIs are modelled around nouns (resources), not verbs (actions).

    • Good: GET /vehicles/{id}
    • Bad: POST /getVehicle

  • Uniform interface: Standard HTTP verbs carry semantic meaning:

    Verb Meaning
    GET Read a resource
    POST Create a resource
    PUT Replace a resource entirely
    PATCH Partially update a resource
    DELETE Remove a resource
  • Layered system: Clients don't know if they're talking to the real backend or a gateway/proxy/cache. This is exactly what Microlise's APIM layer provides.
  • Cacheable: Responses should declare whether they can be cached, enabling CDN and client-side optimisation.

OpenAPI Specification (OAS/Swagger) is the standard way to describe a REST API. The spec referred to throughout the APIOps pipeline is this document.

2.2 GraphQL

A query language for APIs developed by Meta. Instead of fixed endpoints, clients send a query describing exactly what data they need.

  • Single endpoint: POST /graphql
  • Client drives the shape of the response - no over-fetching or under-fetching.
  • Good for: complex, interconnected data models; front-end teams who iterate quickly.
  • Trade-off: harder to cache, more complex server-side resolver logic, linting/governance is less mature than OAS.

Not currently the Microlise APIM pattern but worth understanding as a contrast.

2.3 gRPC (Google Remote Procedure Call)

Uses Protocol Buffers (protobuf) as the interface definition language and HTTP/2 as transport.

  • Strongly typed contracts defined in .proto files.
  • Extremely high performance - binary serialisation, multiplexed streams.
  • Ideal for: internal service-to-service calls, microservices, high-throughput scenarios.
  • Trade-off: not human-readable, harder to test with standard tooling (curl, Postman), less browser-friendly.

Think of gRPC as what might live behind an API gateway - internal communication between microservices - while REST/OAS faces outward toward customers.

2.4 AsyncAPI / Event-Driven APIs

Not all APIs are request-response. Event-driven APIs use messaging patterns:

  • Webhooks: Server POSTs to a client-registered URL when an event occurs.
  • WebSockets: Persistent bi-directional connection between client and server.
  • Server-Sent Events (SSE): One-way stream from server to client.
  • Message queues (Kafka, RabbitMQ, Azure Service Bus): Decoupled async messaging.

AsyncAPI is the OAS equivalent for event-driven interfaces - a specification format for documenting these contracts.

3. API Gateway Pattern

This is the core of what APIM implements. An API gateway sits as an intermediary between consumers (customers, internal teams) and backend services.

flowchart TD
    EC[External Consumer]
    GW["API Gateway / APIM
    ────────────────────
    Auth · Rate Limiting
    Transforms · Routing
    Logging · Caching"]
    SA["Service A\n(OpenShift)"]
    SB["Service B\n(OpenShift)"]
    SC["Service C\n(OpenShift)"]

    EC --> GW
    GW --> SA
    GW --> SB
    GW --> SC

3.1 What the gateway does

Concern What it means
Authentication Verifies who the caller is (OAuth2 tokens, API keys, mutual TLS)
Authorisation Determines what the caller is allowed to do (scopes, claims)
Rate limiting Caps requests per second/minute/hour per consumer or globally
Throttling Gracefully slows or queues excess requests rather than rejecting them
Request transformation Rewrites headers, payloads, or paths before forwarding to backends
Response transformation Strips internal fields, reformats responses for the consumer contract
Routing Directs traffic to the correct backend based on path, headers, or content
Load balancing Distributes traffic across backend instances
Caching Stores responses to reduce backend load for idempotent requests
Observability Centralises access logs, metrics, and tracing across all APIs

In Azure APIM specifically, these concerns are implemented as policies - XML-based declarative rules that run at gateway level.

3.2 Reverse Proxy vs API Gateway

The Microlise notes reference a Reverse Proxy pipeline alongside APIM. These are related but distinct:

Aspect Reverse Proxy API Gateway
Primary purpose Routing and TLS termination Full API lifecycle management
Protocol awareness Layer 4/7 (TCP/HTTP) Layer 7, API-aware (understands REST, OAS)
Policy engine Minimal (Nginx/HAProxy config) Rich (auth, transforms, quotas, subscriptions)
Developer portal No Yes - consumer-facing API catalogue
Examples Nginx, HAProxy, Traefik Azure APIM, Kong, AWS API Gateway

A common pattern (and likely what Microlise uses) is: Reverse proxy handles ingress and TLS termination → traffic forwarded to APIM for policy enforcement → APIM routes to OpenShift services.

4. API Design Principles

4.1 Contract-First Design

Define the OpenAPI spec before writing implementation code. The spec is the source of truth.

Benefits:

  • Frontend/consumer teams can mock and build against the spec immediately.
  • Linting pipelines (like the one in APIOps gated build) can enforce governance before any code ships.
  • Breaking change detection is automated.

The APIOps pipeline enforces this: the spec is committed to source control, linted, reviewed by the API Governance Council, and only then does the publish pipeline sync it to APIM environments.

4.2 Versioning Strategies

APIs evolve. Versioning prevents changes from breaking existing consumers.

Strategy Example Trade-offs
URI versioning /v1/vehicles, /v2/vehicles Explicit, cacheable, easy to route. Pollutes paths.
Header versioning Accept: application/vnd.api.v2+json Clean URIs. Harder to test, less cache-friendly.
Query param /vehicles?version=2 Simple but considered poor practice for REST.

URI versioning is the most common and is what Azure APIM handles well via routing rules.

4.3 Breaking vs Non-Breaking Changes

Knowing what constitutes a breaking change is critical for API governance (i.e. why the API GC reviews spec PRs).

Non-breaking (additive) Breaking
Adding a new optional field to response Removing or renaming a field
Adding a new endpoint Changing a field's type
Adding a new optional query parameter Making an optional parameter required
New enum value (with caution) Changing HTTP status codes for existing scenarios
Changing authentication schemes

4.4 Resource Naming Conventions

  • Use nouns, not verbs: /journeys not /getJourneys
  • Use plural for collections: /vehicles not /vehicle
  • Use kebab-case for multi-word: /driver-events not /driverEvents
  • Nest to show ownership, but limit depth: /vehicles/{id}/journeys is fine; /vehicles/{id}/journeys/{jid}/events/{eid}/metadata is not.
  • Never expose internal implementation details in paths (/{internalDatabaseId} leaks schema).

4.5 HTTP Status Codes

Correct status codes are part of the API contract. Misuse breaks consumers who rely on them.

Code Meaning When to use
200 OK Successful GET, PUT, PATCH
201 Created Successful POST that created a resource
204 No Content Successful DELETE or action with no response body
400 Bad Request Client sent malformed/invalid data
401 Unauthorized Not authenticated (no or invalid token)
403 Forbidden Authenticated but not authorised for this resource
404 Not Found Resource does not exist
409 Conflict State conflict (duplicate, version mismatch)
422 Unprocessable Entity Semantically invalid (e.g. invalid date range)
429 Too Many Requests Rate limit exceeded
500 Internal Server Error Unhandled server-side failure
503 Service Unavailable Downstream dependency down, circuit breaker open

5. API Security Architecture

5.1 Authentication Patterns

Pattern How it works Typical use
API Keys Static key passed in header (x-api-key) or query param Simple, internal/partner APIs
OAuth 2.0 Token-based; client obtains a bearer token from auth server Public APIs, delegated access
OpenID Connect OAuth 2.0 + identity layer (ID tokens, user info endpoint) APIs that need to know who the user is
Mutual TLS Both client and server present certificates High-security service-to-service
JWT Signed token carrying claims; verified without calling auth server Stateless auth at gateway level

Azure APIM supports all of these via policies. A common pattern: APIM validates the JWT at the gateway before the request ever reaches an OpenShift pod.

5.2 OAuth 2.0 Grant Types

Grant type Use case
Client Credentials Machine-to-machine (no user involved). Most common for APIs.
Authorization Code User-facing apps; user logs in and delegates access
Authorization Code + PKCE Same as above but for SPAs/mobile (no client secret)
Implicit (deprecated) Was used for SPAs - replaced by Auth Code + PKCE

5.3 Zero Trust at the API Layer

Zero Trust means: never trust, always verify - even internal services must authenticate.

Principles applied to APIs:

  • Every service-to-service call requires a valid token (no implicit trust on the internal network).
  • Tokens have minimum required scopes (principle of least privilege).
  • mTLS between internal services adds a second layer even if a token is compromised.
  • All traffic - internal and external - goes through the gateway and is logged.

6. API Lifecycle Management

This maps directly to the APIOps workflow in the APIM notes.

flowchart LR
    Design[Design]
    Develop[Develop]
    Test[Test]
    Publish[Publish]
    Monitor[Monitor]
    Retire[Retire]

    Design --> Develop --> Test --> Publish --> Monitor --> Retire

    Design -.-> D1["OAS Spec\nContract First"]
    Develop -.-> D2["C# Project\nTemplates\nOpenShift"]
    Test -.-> D3["Gated Build\nPipeline\nLinting + API GC"]
    Publish -.-> D4["APIM Publish\nPipeline\nDev → Cert → Prod"]
    Monitor -.-> D5["Analytics\nDashboards\nAPIM Portal"]
    Retire -.-> D6["Deprecation\nNotices\nVersion Sunset"]

6.1 API Governance

The API Governance Council (API GC) referenced in the notes is the enforcement body for architectural standards. Common governance concerns:

  • Linting: Automated rules against the OAS spec. The APIOps pipeline uses scripts from ApiManagement.Pipeline.AgentScripts to enforce this.
  • Review gates: No spec change merges without GC approval - prevents inconsistent or insecure APIs reaching production.
  • Naming standards: Enforced in the spec review (see §4.4).
  • Breaking change policy: Defines how long old versions must be supported before retirement.
  • Security policy: All APIs must use approved auth methods; no unauthenticated endpoints in production.

6.2 APIOps (GitOps for APIs)

APIOps applies GitOps principles to API management: the APIM configuration is stored as code in a Git repository and the pipeline is the only mechanism that changes APIM state.

Key properties:

  • Declarative: The Microlise.APIOps repo describes the desired state of all APIs in APIM.
  • Versioned: Every change is a PR - full audit trail.
  • Automated: The publish pipeline does the two-way sync; no manual APIM portal edits.
  • Environment promotion: Changes flow Dev -> Cert -> Prod, with a manual approval gate before Prod.

This is analogous to how Terraform or Helm work for infrastructure - the repo is the truth.

7. API Observability

An often-overlooked architectural concern. APIs you can't observe are APIs you can't operate.

7.1 The Three Pillars

Pillar What it captures Tooling examples
Logs Discrete events: requests, responses, errors, auth failures Azure Monitor, ELK, Splunk
Metrics Aggregated numbers over time: latency, error rate, RPS Prometheus, Azure Metrics, Grafana
Traces End-to-end request journey across services Jaeger, Zipkin, Azure App Insights

7.2 Key API Metrics to Track

  • Latency: p50, p95, p99 - not just average. Averages hide outliers.
  • Error rate: 5xx rate (server errors) and 4xx rate (client errors) separately.
  • Throughput: Requests per second - used to set rate limits and plan capacity.
  • Availability: Uptime percentage. SLAs are usually defined here (99.9% = ~8.7h downtime/year).
  • Quota consumption: How much of a consumer's rate limit are they using?

Azure APIM exposes all of these natively and can emit them to Azure Monitor.

7.3 Correlation IDs

Every request should carry a unique correlation-id (or x-request-id) header. The gateway generates one if the client doesn't provide it and forwards it to all downstream services. This makes it possible to trace a single user request across multiple microservice logs.

flowchart LR
    C[Client]
    APIM["APIM\ngenerates correlation-id: abc-123"]
    SA["Service A logs\nabc-123 · vehicle lookup"]
    SB["Service B logs\nabc-123 · journey history"]

    C --> APIM
    APIM --> SA
    SA --> SB

8. Microservices & API Design

The OpenShift deployment model in Microlise's stack implies microservices. API architecture must account for how services communicate internally vs. externally.

8.1 Internal vs External APIs

Aspect Internal (East-West) External (North-South)
Consumers Other microservices Customers, partners, third parties
Protocol gRPC, internal REST, message queues REST over HTTPS via APIM
Auth mTLS, service accounts, internal tokens OAuth2, API keys managed by APIM
Discoverability Service mesh / internal DNS Developer portal in APIM
Governance Team-level conventions API GC, formal versioning, SLA commitments

8.2 API Aggregation / BFF Pattern

Backend for Frontend (BFF): a dedicated API layer tailored to a specific consumer (e.g. a mobile app, a portal). Instead of the consumer calling 5 microservices, a BFF aggregates them into a single call.

flowchart LR
    MA[Mobile App]
    BFF[BFF: Mobile API]
    VS[Vehicle Service]
    JS[Journey Service]
    DS[Driver Service]

    MA --> BFF
    BFF --> VS
    BFF --> JS
    BFF --> DS

APIM policies can implement lightweight aggregation, but for complex cases a dedicated BFF service is cleaner.

8.3 Service Mesh (Complementary to API Gateway)

A service mesh (e.g. Istio, Linkerd) manages internal service-to-service communication within OpenShift/Kubernetes:

  • mTLS between pods automatically.
  • Traffic policies (retries, circuit breaking) at the network level.
  • Observability (traces, metrics) without code changes.

The API Gateway handles North-South (external) traffic; the service mesh handles East-West (internal). They are complementary, not competing.

9. API Design Patterns

9.1 Pagination

Never return unbounded collections. Standard patterns:

  • Offset/limit: GET /journeys?offset=0&limit=50. Simple but inefficient at high offsets.
  • Cursor-based: GET /journeys?cursor=eyJpZCI6MTAwfQ==. Efficient for large datasets; the cursor encodes the last seen position.
  • Page-based: GET /journeys?page=3&pageSize=50. User-friendly but shares offset's inefficiency.

Response should include metadata:

{
  "data": [...],
  "pagination": {
    "total": 1420,
    "limit": 50,
    "nextCursor": "eyJpZCI6MTUwfQ=="
  }
}

9.2 Filtering, Sorting, and Field Selection

  • Filtering: GET /vehicles?status=active&driverType=HGV
  • Sorting: GET /journeys?sort=-startedAt (prefix - for descending)
  • Field selection (sparse fieldsets): GET /vehicles?fields=id,registration,status - reduces payload size.

9.3 Idempotency

A request is idempotent if making it multiple times produces the same result as making it once. Crucial for retry logic.

Method Idempotent? Safe (no side effects)?
GET Yes Yes
PUT Yes No
DELETE Yes No
POST No No
PATCH No* No

*PATCH can be designed to be idempotent but isn't by definition.

For non-idempotent operations (POST), use an Idempotency-Key header. The server stores the result keyed to that value; duplicate requests return the cached result rather than processing again.

9.4 HATEOAS

Hypermedia as the Engine of Application State - responses include links to related actions:

{
  "id": "v-123",
  "registration": "AB12 CDE",
  "_links": {
    "self": { "href": "/vehicles/v-123" },
    "journeys": { "href": "/vehicles/v-123/journeys" },
    "driver": { "href": "/drivers/d-456" }
  }
}

Rarely implemented fully in practice but worth understanding as the most complete expression of REST.

10. OpenAPI Specification Deep Dive

Since OAS is central to the APIOps pipeline, understanding its structure is practical knowledge.

openapi: "3.1.0"
info:
  title: Vehicle Service API
  version: "2.0.0"
  description: Manages vehicle records for the Microlise platform.

servers:
  - url: https://api.microlise.com/v2
    description: Production

paths:
  /vehicles/{vehicleId}:
    get:
      summary: Get a vehicle by ID
      operationId: getVehicleById          # Unique identifier used in code gen
      tags: [Vehicles]
      parameters:
        - name: vehicleId
          in: path
          required: true
          schema:
            type: string
            format: uuid
      responses:
        "200":
          description: Vehicle found
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Vehicle"
        "404":
          $ref: "#/components/responses/NotFound"

components:
  schemas:
    Vehicle:
      type: object
      required: [id, registration]
      properties:
        id:
          type: string
          format: uuid
        registration:
          type: string
          example: "AB12 CDE"
        status:
          type: string
          enum: [active, inactive, maintenance]

  responses:
    NotFound:
      description: Resource not found
      content:
        application/json:
          schema:
            $ref: "#/components/schemas/ProblemDetails"

  securitySchemes:
    oauth2:
      type: oauth2
      flows:
        clientCredentials:
          tokenUrl: https://auth.microlise.com/oauth2/token
          scopes:
            vehicles:read: Read vehicle data
            vehicles:write: Create and update vehicles

security:
  - oauth2: [vehicles:read]

Key OAS concepts:

  • operationId: Used by code generators and APIM to reference operations in policies.
  • $ref: DRY principle - define schemas and responses once, reuse everywhere.
  • components: The library section - schemas, parameters, responses, security schemes.
  • tags: Grouping for the developer portal - consumers see organised API docs.
  • security: Applied globally here; can be overridden per-operation.

11. Connecting the Dots: Microlise Architecture Map

Mapping the APIM notes to the architectural concepts above:

APIM Note Item Architectural Concept
Separate layer between customers & APIs API Gateway Pattern (§3)
OpenAPI spec / swagger build Contract-First Design (§4.1), OAS (§10)
APIOps pipeline, spec in git APIOps / GitOps for APIs (§6.2)
API Governance Council review API Governance (§6.1), Breaking Changes (§4.3)
Linting scripts (AgentScripts) Automated governance enforcement
Gated + main pipeline CI/CD gates for quality and security
Two-way sync, publish pipeline Declarative state management (APIOps)
Dev -> Cert -> Prod with approval gate Environment promotion pattern (§6.2)
Gen cluster vs Prod cluster Environment isolation, blast radius reduction
Reverse proxy pipeline Reverse Proxy vs API Gateway (§3.2)
Quay registry, OpenShift containers Microservices deployment, East-West traffic (§8.1)
Swagger/OAS spec as PR artefact Spec-as-code, version-controlled contracts

12. Further Reading & Reference

Recommended (from APIM notes)

Additional Architecture Resources

Key Terms Glossary

Term Definition
OAS / Swagger OpenAPI Specification - a standard format for describing REST APIs
APIOps Applying GitOps principles to API management (spec-as-code, pipeline-driven)
APIM API Management - the platform/layer that governs API lifecycle
Gateway Intermediary that enforces policy (auth, rate limiting, routing) for APIs
Spec Short for specification - the OAS JSON/YAML document describing an API
Idempotency Property where repeating a request has the same effect as making it once
mTLS Mutual TLS - both parties in a connection authenticate with certificates
BFF Backend for Frontend - an API tailored to a specific consumer's needs
Service Mesh Infrastructure layer managing internal service-to-service communication
Breaking Change An API change that requires existing consumers to update their integration
Reference in New Issue View Git Blame Copy Permalink