128 lines
5.8 KiB
Org Mode
Executable File
128 lines
5.8 KiB
Org Mode
Executable File
:PROPERTIES:
|
|
:ID: 46add22d-e562-4e3a-a301-d4aea2552952
|
|
:END:
|
|
#+DATE: 2026-04-10
|
|
#+filetags: :database:sql:ess:esp:microlise:
|
|
#+STARTUP: showall
|
|
#+title: ESP — Database & Dacpac
|
|
|
|
* The ESP Database
|
|
:PROPERTIES:
|
|
:RELATED: [[id:a6c345df-8db9-4538-b87e-0e72e2414905][ESP Deployment Pipeline — Index]]
|
|
:END:
|
|
|
|
ESP uses a single SQL Server database called *EspBroker*. This is deployed and upgraded as part of the overall ESP deployment process.
|
|
|
|
* What Is a Dacpac?
|
|
|
|
A *dacpac* (Data-tier Application Package) is a packaged representation of a SQL Server database *schema*. It is a file with the extension ~.dacpac~.
|
|
|
|
Rather than writing migration scripts that say "ALTER TABLE, ADD COLUMN..." manually, you instead describe the *desired end state* of the database, and the dacpac deployment tool (~sqlpackage~) calculates the delta and applies it automatically.
|
|
|
|
** How Dacpac Deployment Works
|
|
#+BEGIN_SRC
|
|
Your dacpac file (desired schema)
|
|
│
|
|
▼
|
|
sqlpackage.exe
|
|
│
|
|
├── Connects to target database
|
|
├── Compares desired schema to actual schema
|
|
├── Generates a diff
|
|
└── Applies the diff (ALTER TABLE, CREATE INDEX, etc.)
|
|
#+END_SRC
|
|
|
|
** Advantages Over Raw SQL Scripts
|
|
| Dacpac | Raw SQL Scripts |
|
|
|-----------------------------------------+-------------------------------------------|
|
|
| Declarative — describe what you want | Imperative — describe each change step |
|
|
| Tool calculates the diff automatically | You must track and apply changes manually |
|
|
| Idempotent — safe to run multiple times | Can fail if run twice without care |
|
|
| Schema is versioned as code | Scripts can get out of sync |
|
|
|
|
** Relevant PowerShell / CLI
|
|
#+BEGIN_SRC powershell
|
|
# Deploy a dacpac using sqlpackage
|
|
& "C:\Program Files\Microsoft SQL Server\160\DAC\bin\sqlpackage.exe" `
|
|
/Action:Publish `
|
|
/SourceFile:"EspBroker.dacpac" `
|
|
/TargetServerName:"ROMAC-DEV-DB01" `
|
|
/TargetDatabaseName:"EspBroker"
|
|
#+END_SRC
|
|
|
|
* Team Ludo's Dacpac
|
|
|
|
Team Ludo built a dacpac solution for EspBroker. This is already in the repo and represents the *target schema* that all customer databases should eventually conform to.
|
|
|
|
* The Critical Problem — Schema Inconsistency
|
|
|
|
** What the Problem Is
|
|
Existing customer databases have drifted from the official schema over time. This drift likely happened due to:
|
|
- Manual hotfixes applied directly to production databases
|
|
- Different versions of ESP deployed to different customers at different times
|
|
- No enforced schema management historically
|
|
|
|
This means the dacpac's expected schema does not match what is actually in customer databases.
|
|
|
|
** Consequence
|
|
If you attempt to deploy the dacpac against an inconsistent database, ~sqlpackage~ will either:
|
|
- Fail with errors (best case — nothing is changed)
|
|
- Apply incorrect changes that corrupt data (worst case)
|
|
|
|
** What Needs to Happen
|
|
Before the dacpac can be used for a customer:
|
|
1. A database expert must *manually inspect* the customer's database
|
|
2. Identify all schema differences between actual and expected state
|
|
3. Write and apply *manual SQL scripts* to bring the DB into alignment
|
|
4. Verify the dacpac can then deploy cleanly (ideally against a copy)
|
|
5. Only then mark the customer as ~dacpacReady: true~ in the manifest
|
|
|
|
** Pipeline Safeguard
|
|
The deployment pipeline *must* check the ~dacpacReady~ flag before attempting database deployment, and fail clearly if it is ~false~:
|
|
|
|
#+BEGIN_SRC powershell
|
|
function Invoke-DatabaseDeployment {
|
|
param($Manifest)
|
|
|
|
if (-not $Manifest.Database.DacpacReady) {
|
|
Write-Error "Database for $($Manifest.Customer) $($Manifest.Environment) is not dacpac-ready. " +
|
|
"Manual schema alignment is required before deployment."
|
|
throw "Database not ready for dacpac deployment."
|
|
}
|
|
|
|
# Proceed with dacpac deployment...
|
|
Deploy-Dacpac -DacpacPath $artifactPath -Server $Manifest.Servers.DbServer -Database "EspBroker"
|
|
}
|
|
#+END_SRC
|
|
|
|
* Database Deployment in the Deployment Stage
|
|
|
|
Database deployment sits inside the *DEPLOY stage*. See [[id:cd3cd02f-c9b3-4455-8ce5-b2a5aa458fed][ESP — Deployment Stages]] for full stage detail. The database is typically deployed before services are started up to ensure the schema is ready for the new application code.
|
|
|
|
Suggested order within the DEPLOY stage:
|
|
1. Stop Windows Services
|
|
2. Deploy database dacpac (if dacpacReady)
|
|
3. Handle Citrix sessions
|
|
4. Swap application files
|
|
5. Start Windows Services back up
|
|
|
|
* Rollback Considerations
|
|
|
|
If the dacpac deploys successfully but a subsequent step fails, rolling back the database is non-trivial. Dacpac does not natively support rollback — options are:
|
|
- Restore from backup (requires a backup to have been taken immediately before)
|
|
- Write a counter-dacpac that reverts the schema (complex, error-prone)
|
|
|
|
This is one of the reasons the *rollback plan is still TBC*. See [[id:82c3d447-d6d3-498e-9f66-aee60c752462][ESP — Open Questions & TBC Items]].
|
|
|
|
A *database backup must be taken before any deployment* — this should be a mandatory step in the PREDEPLOY stage.
|
|
|
|
* SQL Server Concepts Relevant Here
|
|
|
|
| Concept | Relevance |
|
|
|----------------+--------------------------------------------------------------|
|
|
| Schema | The structure of tables, columns, indexes, constraints, etc. |
|
|
| ~sqlpackage~ | Microsoft CLI tool that applies dacpac files |
|
|
| ~BACPAC~ | Like dacpac but includes data — useful for backup/restore |
|
|
| SQL Agent Jobs | Scheduled SQL jobs that may need to be handled during deploy |
|
|
| Linked Servers | DB connections to other servers — may be part of ESP's setup |
|