vault/Career/20260410122754-esp_known_issues.md

Overview

This file documents all known issues, risks, and blockers identified in the handover material. Understanding these early will help you avoid surprises.

Issue 1 — Database Schema Inconsistency

What the Problem Is

Existing customer databases have drifted from the official ESP schema. The dacpac built by Team Ludo reflects what the schema should be, but what's actually in customer databases doesn't match.

Impact

• The dacpac cannot be used against any customer database until that database is manually aligned
• Deploying a dacpac against a mismatched DB risks data corruption or failures

What Needs to Happen

For each customer database:

1. A DBA or developer inspects the actual DB schema
2. Identifies differences from the dacpac's expected schema
3. Writes and applies manual scripts to bring the DB into line
4. Verifies a trial dacpac deployment succeeds against a copy
5. Marks the manifest dacpacReady: true

Mitigations in the Pipeline

• The pipeline must check the dacpacReady flag before touching the database
• If false, fail immediately with a clear error message
• See ESP — Database & Dacpac for implementation detail

Issue 2 — Build Artifact Is a Flat DLL Dump

What the Problem Is

The ESP Main Build pipeline produces a single large flat folder containing all DLLs for all applications mixed together. There is no clear separation like:

/artifacts/DLService/DLService.dll
/artifacts/EmailService/EmailService.dll

Instead it's more like:

/artifacts/DLService.dll
/artifacts/SomeSharedLibrary.dll
/artifacts/EmailService.dll
/artifacts/AnotherThing.dll
... (hundreds of files)

Impact

• The PREDEPLOY stage can't simply copy a folder to a service's install path
• Scripts will need to know which DLLs belong to which application
• This mapping does not yet exist

What Needs to Happen

• Investigate the build artifact structure in detail
• Create a mapping: application → list of DLLs/files it needs
• This mapping may need to be stored in the scripts, the manifest, or a separate config file
• Ideally, feed back to the build team to separate the artifacts properly

Issue 3 — No Test Environment

What the Problem Is

There is no dedicated sandbox environment to run test deployments against. Any deployment the team runs is against a real customer environment.

Impact

• Mistakes affect real customers
• You cannot safely iterate and test the pipeline without risk
• Developing and debugging deployment logic is much harder

What Needs to Happen

• Set up a dedicated ESP test environment (raised as TBC in handover)
• This may require ESS to provision a server, or Microlise to build one
• Until then, exercise extreme caution with any deployment run

Issue 4 — Licensing Prevents Local Execution

What the Problem Is

ESP cannot be run on developer machines due to licensing restrictions.

Impact

• You cannot test your deployment scripts against a local ESP instance
• Debugging requires deploying to a real (or test) server
• Makes the development feedback loop longer

Mitigation

• The three-tier architecture helps here: Tier 3 functions can be unit tested in isolation without a real ESP environment
• Use Pester (PowerShell testing framework) for unit tests
• End-to-end testing requires access to a real environment

Issue 5 — Cycle Checks Require Citrix UI Navigation

What the Problem Is

Post-deployment validation ("cycle checks") involves navigating through the Citrix application UI to verify functionality. This is:

• Extensive and time-consuming
• Very hard to automate (requires UI automation tooling like Selenium or Tosca)
• Dependent on Citrix being accessible from the test runner

Short-Term Mitigation

Implement cycle checks as a manual prompt in the POSTDEPLOY stage — the pipeline pauses and waits for a human to confirm the checks passed.

Long-Term Options

• Investigate API-level checks if ESP exposes any (bypasses UI entirely)
• UI automation tools (e.g. Ranorex, Tosca, Selenium with Citrix plugin)
• Simplified smoke tests that don't require full UI navigation

Issue 6 — Citrix Upgrade Mechanism Unknown

What the Problem Is

The exact technical mechanism for upgrading a Citrix-delivered application is not yet confirmed. The working assumption is that it involves:

• Copying files to the Citrix server
• Handling any locked executable (the running app may lock its own .exe)

What Needs to Happen

• Confirm with ESS or Citrix documentation how Citrix app upgrades work
• Determine if there's a Citrix-native method (e.g. App Layering, PVS updates) vs. a simple file swap
• Understand how to handle sessions that lock the executable

Issue 7 — Test/Prod on Same Server

What the Problem Is

Some customers may have their TEST and PROD instances on the same physical server, differentiated only by install path or port.

Impact

A misconfigured manifest or script bug could cause a PROD deployment when a TEST deployment was intended.

Mitigations

• The manifest must clearly separate TEST and PROD config even if they share a server
• Consider an explicit confirmation prompt when deploying to PROD
• AzDO approval gates for PROD stages (not yet designed)
• The VALIDATE stage should catch obvious misconfigurations

Risk Register Summary

#	Issue	Severity	Status	Mitigation
1	DB schema inconsistency	High	Unresolved	dacpacReady flag, manual DB fix first
2	Flat DLL artifact	Medium	Unresolved	Needs DLL-to-app mapping
3	No test environment	High	TBC	Treat all runs as live
4	No local execution	Medium	Inherent	Unit test Tier 3 with Pester
5	Cycle checks need UI	Medium	Manual short-term	Automate later; manual prompt now
6	Citrix upgrade mechanism unknown	Medium	TBC	Investigate with ESS
7	Test/Prod on same server	Medium	Design needed	Manifest separation, approval gates