zaine/org_web
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

fixing gitignore

Browse Source
This commit is contained in:
Zaine Arch
2025-12-28 21:19:27 +00:00
parent fb120bc50a
commit 8f234a746d
221 changed files with 8064 additions and 11740 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
blogs/2025/11-november/02-11-week-review.org Normal file
View File

@@ -0,0 +1,22 @@
#+TITLE: [02-11-2025] - Weekly Review
#+OPTIONS: num:nil
#+DATE: <2025-11-02 Sun>
#+filetags: :review:
#+COMMENTS: t
#+SLUG: 02-11-25-week-review
[Was a mid week file, then moved it to a review file]
* Mid Week Review
- Mostly work related updates. Working from home on monday 10th (for trial, so we get used to it and connecting to the VPN)
- Subsequently we will be able to work from home on mondays and fridays for the following weeks (beginning either 17th or 24th?)
- Still working on powerbi (not a fan)
- Upskilling needs to take more this week and the next
- Probation meeting on thursday next week [06/11]
- It's getting darker earlier and more windy
* TODOS
** DONE
Write the retrospective notes
** DONE
More competencies

20
blogs/2025/11-november/02-11-week-review.org~ Normal file
View File

@@ -0,0 +1,20 @@
#+TITLE: [02-11-2025] - Weekly Review
#+OPTIONS: num:nil
#+DATE: <2025-11-02 Sun>
#+filetags: :review:
[Was a mid week file, then moved it to a review file]
* Mid Week Review
- Mostly work related updates. Working from home on monday 10th (for trial, so we get used to it and connecting to the VPN)
- Subsequently we will be able to work from home on mondays and fridays for the following weeks (beginning either 17th or 24th?)
- Still working on powerbi (not a fan)
- Upskilling needs to take more this week and the next
- Probation meeting on thursday next week [06/11]
- It's getting darker earlier and more windy
* TODOS
** DONE
Write the retrospective notes
** DONE
More competencies

44
blogs/2025/11-november/09-11-week-review.org Normal file
View File

@@ -0,0 +1,44 @@
#+TITLE: [09-11-2025] - Weekly Review
#+OPTIONS: num:nil
#+DATE:<2025-11-09 Sun 20:08>
#+filetags: :review:
#+COMMENTS: t
#+SLUG: 09-11-25-week-review
* Weekly review
- Week 5 work finished!
- Working from home on monday 10th.
- Maybe do weekly work retrospectives? Using the start stop continue model.
- Probation meeting went quite well Alhamdulillah. Keep doing what you're doing was the message.
- Should start to record successes, either here or on post-it notes. Helps with imposter syndrome.
- Led a meeting for the first time (07/11). Was with Lloyd and Oli from team monopoly (SecDevOps). They understood the problem statement clearly and everything was explained properly to them.
- Bowling Kata was reduced to 29 minutes.
Some thoughts about the nginx reverse proxying architecture. This week I transferred all the docker services to a subdomain.
[[../../../assets/sketchnotes/09-11-2025-nginx-architecture.svg]]
Weekly retrospective using the start stop continue model:
[[../../../assets/sketchnotes/09-11-2025-retrospectives.svg]]
Here is the current state of the competencies (20.5%):
[[../../../assets/images/competencies/competencies-09-11-25.png]]
During the meeting, we were told to achieve 60% by the end of January.
And here is the dev plan:
[[../../../assets/images/competencies/dev-plan-09-11-25.png]]
* Next week:
- Competencies to do:
- Aware of the Benefits of Normalisation
- Aware of PowerShell, Python, Bash and Pester
- Aware of Lean, Agile & extreme programming Principles
- Understands Server Operating Systems (might be tough)
- Understanding of TCP/IP

42
blogs/2025/11-november/09-11-week-review.org~ Normal file
View File

@@ -0,0 +1,42 @@
#+TITLE: [09-11-2025] - Weekly Review
#+OPTIONS: num:nil
#+DATE:<2025-11-09 Sun 20:08>
#+filetags: :review:
* Weekly review
- Week 5 work finished!
- Working from home on monday 10th.
- Maybe do weekly work retrospectives? Using the start stop continue model.
- Probation meeting went quite well Alhamdulillah. Keep doing what you're doing was the message.
- Should start to record successes, either here or on post-it notes. Helps with imposter syndrome.
- Led a meeting for the first time (07/11). Was with Lloyd and Oli from team monopoly (SecDevOps). They understood the problem statement clearly and everything was explained properly to them.
- Bowling Kata was reduced to 29 minutes.
Some thoughts about the nginx reverse proxying architecture. This week I transferred all the docker services to a subdomain.
[[../../../assets/sketchnotes/09-11-2025-nginx-architecture.svg]]
Weekly retrospective using the start stop continue model:
[[../../../assets/sketchnotes/09-11-2025-retrospectives.svg]]
Here is the current state of the competencies (20.5%):
[[../../../assets/images/competencies/competencies-09-11-25.png]]
During the meeting, we were told to achieve 60% by the end of January.
And here is the dev plan:
[[../../../assets/images/competencies/dev-plan-09-11-25.png]]
* Next week:
- Competencies to do:
- Aware of the Benefits of Normalisation
- Aware of PowerShell, Python, Bash and Pester
- Aware of Lean, Agile & extreme programming Principles
- Understands Server Operating Systems (might be tough)
- Understanding of TCP/IP

90
blogs/2025/11-november/16-11-week-review.org Normal file
View File

@@ -0,0 +1,90 @@
#+TITLE: [16-11-2025] - Weekly Review
#+OPTIONS: num:nil
#+DATE:<2025-11-10 Mon 17:44>
#+filetags: :review:
#+WIP:
#+COMMENTS: t
#+SLUG: 16-11-25-week-review
* Previous Week List:
- Competencies to do:
- +Aware of the Benefits of Normalisation+
- Aware of PowerShell, Python, Bash and Pester
- +Aware of Lean, Agile & extreme programming Principles+
- Understands Server Operating Systems (might be tough)
- Understanding of TCP/IP
- +Understands responsive design+
- +Understands Network design+
* Weekly review:
- <2025-11-10 Mon>:
- Got assigned to create a database that can help the whole company. Value added work! The database will have system names linked to the team names. In other words, which teams are working under which systems? Also it will have the repo names and which teams own which repos.
- <2025-11-11 Tue>
- Worked on getting the DB for the system-team-repo. Met with Jez and he said to get the IPT names and use them instead. [reminder to attatch the links for this].
- Created a powershell script to load all the data into the SQL server (local)
- <2025-11-12 Wed>
- Met with the whole team monopoly (+ others). -> First time being in a group huddle
- Was interesting to see how [[https://www.yworks.com/][yworks]] is used
- <2025-11-13 Thu>
- Angular training. Was taught about the new Angular 19 changes and how to use them. Things like the the ~@~ decorator was discussed.
- <2025-11-14 Fri>
- Training on datamarts and airflow. Learnt a lot about how they can be used. I wrote a few slack messages explaining to the team what it is, and also got clarification from Filipo. Also sent a request to create a new repository for this project.
Was able to complete 4/5 competencies this week. The reason for not completing them is that Friday was used mainly for meetings and other admin tasks. In hindsight, I /could/ have completed them, as there were times when I chose to not do value added work.
[[../../../assets/sketchnotes/16-11-2025-retrospectives.svg]]
Daily retrospectives:
[[../../../assets/images/reviews/16-11-25-daily-retros.jpg]]
One thing that really helped this week is ~guac~. I was able to use this whilst at work, very handy when wanting to refer to some notes/files that is only located at home.
[[../../../assets/images/reviews/15-11-2025-guac.png]]
* Other
** Airflow/Datamart
[[../../../posts/career/airflow.org][Airflow Post]]
*** Diagram
*A work colleague has written the following and made this diagram*
[[../../../assets/images/diagrams/airflow.png]]
1. AZDO repo for our datamart to go in. Clone the data mart example template
2. Datamart, can use one big one that has different tables for everything that we need.
3. Local airflow that calls api and gets data. continuation tokens can be used. there is a good example
4. Upload everything to microlise servers where it is run automatically
5. Power bi, change data sources to be references to our sql. Can have all tables in power bi. Use filters
*** Summary that I wrote in slack:
- https://azdo.microlise.com/MicroliseCollection/Microlise/_git/DataMart.SQLServer: very useful link explaining what datamarts are, ELT (extract, load, transform) orchestration and how to set this up
- https://azdo.microlise.com/MicroliseCollection/Microlise/_git/DataMart.DataEngineering.InsightReadiness: an example of what a datamart repo looks like. in the template repo, the two important files are tsqlt and DataMartTemplate (in this instance its called InsightReadiness )
- tsqlt coverage should be above 60%
- https://microliseuk.sharepoint.com/sites/msteams_bb7636: This link is the CoP (community of practice) for the airflow. You'll find useful information regarding what apache airflow is and how to set things up.
- https://microliseuk.sharepoint.com/sites/msteams_bb7636/SitePages/Getting-started-with-Airflow.aspx: This link tells you how to set airflow up locally. Once you're done, this will be published to an airflow scheduler platform. Keep in mind that when you want to publish this, document that it succeeds in the PR.
- Just a summary from Filippo:
- A DataMart it's simply a database that is made ad-hoc to contain a certain subset of data (Specific tables with specific columns).
- Once that is created Airflow is used to put data in that database. So it extract data from other sources (APIS, databases) and loads it into the datamart we have created.
- Once you get Airflow locally working and a new Datamart (deployed locally) you can then write a dag to load data into the Mart.
- http://airmastqa01.mms.local:8080/home: this link shows all the DAGs that are currently running in a testing environment, and when they are scheduled. Login using MSUP (edited)
- http://airmastprd01.mms.local:8080/home: This link is the prod version
- https://azdo.microlise.com/MicroliseCollection/Microlise/_git/DevDocs?path=/Airflow/HowTo/PowerBiReportRefresh.md&version=GBmain&_a=preview: Might prove to be useful when we do the powerbi refresh thing...
* Next week:
- Understands Server Operating Systems (might be tough)
- Understanding of TCP/IP
- Aware of PowerShell, Python, Bash and Pester

87
blogs/2025/11-november/16-11-week-review.org~ Normal file
View File

@@ -0,0 +1,87 @@
#+TITLE: [16-11-2025] - Weekly Review
#+OPTIONS: num:nil
#+DATE:<2025-11-10 Mon 17:44>
#+filetags: :review:
#+WIP:
* Previous Week List:
- Competencies to do:
- +Aware of the Benefits of Normalisation+
- Aware of PowerShell, Python, Bash and Pester
- +Aware of Lean, Agile & extreme programming Principles+
- Understands Server Operating Systems (might be tough)
- Understanding of TCP/IP
- +Understands responsive design+
- +Understands Network design+
* Weekly review:
- <2025-11-10 Mon>:
- Got assigned to create a database that can help the whole company. Value added work! The database will have system names linked to the team names. In other words, which teams are working under which systems? Also it will have the repo names and which teams own which repos.
- <2025-11-11 Tue>
- Worked on getting the DB for the system-team-repo. Met with Jez and he said to get the IPT names and use them instead. [reminder to attatch the links for this].
- Created a powershell script to load all the data into the SQL server (local)
- <2025-11-12 Wed>
- Met with the whole team monopoly (+ others). -> First time being in a group huddle
- Was interesting to see how [[https://www.yworks.com/][yworks]] is used
- <2025-11-13 Thu>
- Angular training. Was taught about the new Angular 19 changes and how to use them. Things like the the ~@~ decorator was discussed.
- <2025-11-14 Fri>
- Training on datamarts and airflow. Learnt a lot about how they can be used. I wrote a few slack messages explaining to the team what it is, and also got clarification from Filipo. Also sent a request to create a new repository for this project.
Was able to complete 4/5 competencies this week. The reason for not completing them is that Friday was used mainly for meetings and other admin tasks. In hindsight, I /could/ have completed them, as there were times when I chose to not do value added work.
[[../../../assets/sketchnotes/16-11-2025-retrospectives.svg]]
Daily retrospectives:
[[../../../assets/images/reviews/16-11-25-daily-retros.jpg]]
One thing that really helped this week is ~guac~. I was able to use this whilst at work, very handy when wanting to refer to some notes/files that is only located at home.
[[../../../assets/images/reviews/15-11-2025-guac.png]]
* Other
** Airflow/Datamart
[[../../../posts/career/airflow.org][Airflow Post]]
*** Diagram
*A work colleague has written the following and made this diagram*
[[../../../assets/images/diagrams/airflow.png]]
1. AZDO repo for our datamart to go in. Clone the data mart example template
2. Datamart, can use one big one that has different tables for everything that we need.
3. Local airflow that calls api and gets data. continuation tokens can be used. there is a good example
4. Upload everything to microlise servers where it is run automatically
5. Power bi, change data sources to be references to our sql. Can have all tables in power bi. Use filters
*** Summary that I wrote in slack:
- https://azdo.microlise.com/MicroliseCollection/Microlise/_git/DataMart.SQLServer: very useful link explaining what datamarts are, ELT (extract, load, transform) orchestration and how to set this up
- https://azdo.microlise.com/MicroliseCollection/Microlise/_git/DataMart.DataEngineering.InsightReadiness: an example of what a datamart repo looks like. in the template repo, the two important files are tsqlt and DataMartTemplate (in this instance its called InsightReadiness )
- tsqlt coverage should be above 60%
- https://microliseuk.sharepoint.com/sites/msteams_bb7636: This link is the CoP (community of practice) for the airflow. You'll find useful information regarding what apache airflow is and how to set things up.
- https://microliseuk.sharepoint.com/sites/msteams_bb7636/SitePages/Getting-started-with-Airflow.aspx: This link tells you how to set airflow up locally. Once you're done, this will be published to an airflow scheduler platform. Keep in mind that when you want to publish this, document that it succeeds in the PR.
- Just a summary from Filippo:
- A DataMart it's simply a database that is made ad-hoc to contain a certain subset of data (Specific tables with specific columns).
- Once that is created Airflow is used to put data in that database. So it extract data from other sources (APIS, databases) and loads it into the datamart we have created.
- Once you get Airflow locally working and a new Datamart (deployed locally) you can then write a dag to load data into the Mart.
- http://airmastqa01.mms.local:8080/home: this link shows all the DAGs that are currently running in a testing environment, and when they are scheduled. Login using MSUP (edited)
- http://airmastprd01.mms.local:8080/home: This link is the prod version
- https://azdo.microlise.com/MicroliseCollection/Microlise/_git/DevDocs?path=/Airflow/HowTo/PowerBiReportRefresh.md&version=GBmain&_a=preview: Might prove to be useful when we do the powerbi refresh thing...
* Next week:
- Understands Server Operating Systems (might be tough)
- Understanding of TCP/IP
- Aware of PowerShell, Python, Bash and Pester

78
blogs/2025/11-november/23-11-week-review.org Normal file
View File

@@ -0,0 +1,78 @@
#+TITLE: [23-11-2025] - Weekly Review
#+OPTIONS: num:nil
#+DATE: <2025-11-17 Mon 18:05>
#+filetags: :review:
#+WIP:
#+COMMENTS: t
#+SLUG: 23-11-25-week-review
* Previous Week List:
- Competencies to do:
- +Aware of PowerShell, Python, Bash and Pester+
- Understands Server Operating Systems (might be tough)
- Understanding of TCP/IP
There wasn't much time to do competencies this week because the new sprint started. We were told that this project should ideally be finished within the next two weeks so that we can start working on some other things (Angular related; Kevin did say that it would be related to creating a system that shows who makes the commits, how often and the relationship between committers).
* Weekly Review
- <2025-11-17 Mon>:
- Created a new repository for the team using the #help-azdo channel.
- Set up this repo by renaming references etc.
- Played around with datamarts, got the team and system tables working, although haven't yet written the ~tSQLt~ tests for them yet.
- <2025-11-18 Tue>
- Demo-ed the STR (system team repo) database.
- Started to create the tables for the Mean time to restore and lead time for change.
- <2025-11-19 Wed>
- Finished off creating the data tables for MTTR and LTTC
- <2025-11-20 Thu>
- Started creating Airflows for the repositories table
- <2025-11-21 Fri>
- Data science training, Research Operations, Kafka, Airflow
** Daily retrospectives:
[[../../../assets/images/reviews/23-11-25-daily-retros.jpg]]
** Weekly retrospectives
[[../../../assets/sketchnotes/23-11-2025-retrospectives.svg]]
* Other
** Books
As I finished another book and also spent some time adding more services, I wanted to collate a list of books (of a wide genre) to read. One I came across is [[https://theinvisiblementor.com/ikigai-the-japanese-secret-to-a-long-and-happy-life-summary/][Ikigai]].
** reader.zainezq.com
Something interesting I found when I wanted to take notes on the book called /The Toyota Way/; this is a service that lets you read books as well as convert a ~pdf~ file to ~epub~.
[[../../../assets/images/reviews/23-11-2025-reader.png]]
** booksvnc.zainezq.com
In order to add books, edit metadata and start the server for the ~reader.zainezq.com~
[[../../../assets/images/reviews/23-11-2025-vnc.png]]
** netdata.zainezq.com
One thing I wanted to have information for is:
#+BEGIN_SRC bash
uname -a
hostnamectl
cat /etc/os-release
lscpu
free -h
lsblk -f
df -h
ip a
#+END_SRC
[[../../../assets/images/reviews/23-11-2025-netdata.png]]
* Next Week
- Understands Server Operating Systems (might be tough)
- Understanding of TCP/IP

76
blogs/2025/11-november/23-11-week-review.org~ Normal file
View File

@@ -0,0 +1,76 @@
#+TITLE: [23-11-2025] - Weekly Review
#+OPTIONS: num:nil
#+DATE: <2025-11-17 Mon 18:05>
#+filetags: :review:
#+WIP:
* Previous Week List:
- Competencies to do:
- +Aware of PowerShell, Python, Bash and Pester+
- Understands Server Operating Systems (might be tough)
- Understanding of TCP/IP
There wasn't much time to do competencies this week because the new sprint started. We were told that this project should ideally be finished within the next two weeks so that we can start working on some other things (Angular related; Kevin did say that it would be related to creating a system that shows who makes the commits, how often and the relationship between committers).
* Weekly Review
- <2025-11-17 Mon>:
- Created a new repository for the team using the #help-azdo channel.
- Set up this repo by renaming references etc.
- Played around with datamarts, got the team and system tables working, although haven't yet written the ~tSQLt~ tests for them yet.
- <2025-11-18 Tue>
- Demo-ed the STR (system team repo) database.
- Started to create the tables for the Mean time to restore and lead time for change.
- <2025-11-19 Wed>
- Finished off creating the data tables for MTTR and LTTC
- <2025-11-20 Thu>
- Started creating Airflows for the repositories table
- <2025-11-21 Fri>
- Data science training, Research Operations, Kafka, Airflow
** Daily retrospectives:
[[../../../assets/images/reviews/23-11-25-daily-retros.jpg]]
** Weekly retrospectives
[[../../../assets/sketchnotes/23-11-2025-retrospectives.svg]]
* Other
** Books
As I finished another book and also spent some time adding more services, I wanted to collate a list of books (of a wide genre) to read. One I came across is [[https://theinvisiblementor.com/ikigai-the-japanese-secret-to-a-long-and-happy-life-summary/][Ikigai]].
** reader.zainezq.com
Something interesting I found when I wanted to take notes on the book called /The Toyota Way/; this is a service that lets you read books as well as convert a ~pdf~ file to ~epub~.
[[../../../assets/images/reviews/23-11-2025-reader.png]]
** booksvnc.zainezq.com
In order to add books, edit metadata and start the server for the ~reader.zainezq.com~
[[../../../assets/images/reviews/23-11-2025-vnc.png]]
** netdata.zainezq.com
One thing I wanted to have information for is:
#+BEGIN_SRC bash
uname -a
hostnamectl
cat /etc/os-release
lscpu
free -h
lsblk -f
df -h
ip a
#+END_SRC
[[../../../assets/images/reviews/23-11-2025-netdata.png]]
* Next Week
- Understands Server Operating Systems (might be tough)
- Understanding of TCP/IP

167
blogs/2025/11-november/30-11-week-review.org Normal file
View File

@@ -0,0 +1,167 @@
#+TITLE: [30-11-2025] - Weekly Review
#+OPTIONS: num:nil
#+DATE: <2025-11-30 Sun 17:09>
#+filetags: :review:
#+WIP:
#+COMMENTS: t
#+SLUG: 30-11-25-week-review
* Previous Week List:
- Competencies to do:
- +Aware of relational databases+
- +Aware of TLS and SSL+
- +Aware of good error handling and logging+
- Understanding of TCP/IP
- Understands Server Operating Systems (might be tough)
Been extremely busy with this sprint, getting the work delivered in a timely manner; there wasn't as much time to focus on competencies.
* Weekly Review
- <2025-11-24 Mon>:
- Got the news the company is reducing staff. See this [[https://www.thebusinessdesk.com/eastmidlands/news/2104597-microlise-issues-profit-warning-after-weaker-than-expected-trading][article]]
- Got the DAG for PR's to work
- <2025-11-25 Tue>
- Worked mainly on threads and commits DAGs
- <2025-11-26 Wed>
- We had a meeting with Kevin on the scoring system for the redundancy thing
- Finished creating and testing all the DAG's
- <2025-11-27 Thu>
- Refinement meeting
- Started creating base case tests for SQL
- Made a script to automate the creation of testing files
- <2025-11-28 Fri>
- Finished base case tests
- Training with Lloyd on SRE and Techops
** Daily retrospectives:
[[../../../assets/images/reviews/30-11-25-daily-retros.jpg]]
** Weekly retrospectives
[[../../../assets/sketchnotes/30-11-2025-retrospectives.svg]]
* Other
** Powershell script:
Here is the powershell script that generates the files for testing:
#+BEGIN_SRC ps1
# Define the base folders
$securityFolder = "C:\Users\zqayyum\Source\Repos\DataMart.Academy.DoraMetrics\tsqlt\Security"
$scriptsFolder = "C:\Users\zqayyum\Source\Repos\DataMart.Academy.DoraMetrics\tsqlt"
# Create base folders if they don't exist
if (!(Test-Path $securityFolder)) { New-Item -ItemType Directory -Path $securityFolder }
if (!(Test-Path $scriptsFolder)) { New-Item -ItemType Directory -Path $scriptsFolder }
# Define main file names (schemas)
$fileNames = @(
"testMergeStagingDataCommits.sql",
"testMergeStagingDataEnvironments.sql",
"testMergeStagingDataPullRequests.sql",
"testMergeStagingDataReleases.sql",
"testMergeStagingDataRepositories.sql",
"testMergeStagingDataThreads.sql"
)
# Define stored procedure test files
$storedProcFiles = @(
"Test changed rows are logged.sql",
"Test changed rows are not logged as inserted.sql",
"Test changed rows are updated.sql",
"Test errors are logged.sql",
"Test import date on changed rows is updated.sql",
"Test new rows are inserted.sql",
"Test new rows are not logged as updated.sql"
)
foreach ($name in $fileNames) {
# Create .sql file under Security
$filePath = Join-Path $securityFolder $name
$schemaName = [System.IO.Path]::GetFileNameWithoutExtension($name)
# Content for schema file
$schemaContent = @"
CREATE SCHEMA [$schemaName]
AUTHORIZATION [dbo];
GO
EXECUTE sp_addextendedproperty
@name = N'tSQLt.TestClass',
@value = 1,
@level0type = N'SCHEMA',
@level0name = N'$schemaName';
GO
"@
Set-Content -Path $filePath -Value $schemaContent
# Create schema folder under Scripts
$schemaFolderPath = Join-Path $scriptsFolder $schemaName
if (!(Test-Path $schemaFolderPath)) {
New-Item -ItemType Directory -Path $schemaFolderPath | Out-Null
}
# Create Stored Procedures subfolder
$storedProcFolderPath = Join-Path $schemaFolderPath "Stored Procedures"
if (!(Test-Path $storedProcFolderPath)) {
New-Item -ItemType Directory -Path $storedProcFolderPath | Out-Null
}
# Create stored procedure files with dynamic content
foreach ($procFile in $storedProcFiles) {
$procFilePath = Join-Path $storedProcFolderPath $procFile
$procName = [System.IO.Path]::GetFileNameWithoutExtension($procFile)
$procContent = @"
CREATE PROCEDURE $schemaName.[$procName]
AS
BEGIN
SET NOCOUNT ON;
-- Assemble
DECLARE @RowsUpdated INT;
EXEC tsqlt.FakeTable @SchemaName = 'staging', @TableName = 'tbl_TemplateTable';
EXEC tsqlt.FakeTable @SchemaName = 'data', @TableName = 'tbl_TemplateTable';
EXEC tsqlt.FakeTable @SchemaName = 'logging', @TableName = 'tbl_ExecutionCounts';
INSERT INTO data.tbl_TemplateTable(UnitDescription, TrunionDisplacementInMM)
VALUES('UT7000', '9');
INSERT INTO staging.tbl_TemplateTable(UnitDescription, TrunionDisplacementInMM)
VALUES('UT7000', '6');
-- Act
EXEC staging.pr_MergeStagingData;
-- Assert
SELECT @RowsUpdated = RowsUpdated FROM logging.tbl_ExecutionCounts;
EXEC tsqlt.AssertEquals
@Actual = @RowsUpdated,
@Expected = 1;
END
"@
Set-Content -Path $procFilePath -Value $procContent
}
}
Write-Host "All files and folders created successfully with dynamic procedure content"
#+END_SRC
** Finances
I decided to spin up a docker container to manage finances. For the moment there is no way to connect to a Bank without using an intermediary (privacy concerns).
[[../../../assets/images/reviews/30-11-2025-acutal.png]]
* Next Week
- Understands Server Operating Systems (might be tough)
- Understanding of TCP/IP

165
blogs/2025/11-november/30-11-week-review.org~ Normal file
View File

@@ -0,0 +1,165 @@
#+TITLE: [30-11-2025] - Weekly Review
#+OPTIONS: num:nil
#+DATE: <2025-11-30 Sun 17:09>
#+filetags: :review:
#+WIP:
* Previous Week List:
- Competencies to do:
- +Aware of relational databases+
- +Aware of TLS and SSL+
- +Aware of good error handling and logging+
- Understanding of TCP/IP
- Understands Server Operating Systems (might be tough)
Been extremely busy with this sprint, getting the work delivered in a timely manner; there wasn't as much time to focus on competencies.
* Weekly Review
- <2025-11-24 Mon>:
- Got the news the company is reducing staff. See this [[https://www.thebusinessdesk.com/eastmidlands/news/2104597-microlise-issues-profit-warning-after-weaker-than-expected-trading][article]]
- Got the DAG for PR's to work
- <2025-11-25 Tue>
- Worked mainly on threads and commits DAGs
- <2025-11-26 Wed>
- We had a meeting with Kevin on the scoring system for the redundancy thing
- Finished creating and testing all the DAG's
- <2025-11-27 Thu>
- Refinement meeting
- Started creating base case tests for SQL
- Made a script to automate the creation of testing files
- <2025-11-28 Fri>
- Finished base case tests
- Training with Lloyd on SRE and Techops
** Daily retrospectives:
[[../../../assets/images/reviews/30-11-25-daily-retros.jpg]]
** Weekly retrospectives
[[../../../assets/sketchnotes/30-11-2025-retrospectives.svg]]
* Other
** Powershell script:
Here is the powershell script that generates the files for testing:
#+BEGIN_SRC ps1
# Define the base folders
$securityFolder = "C:\Users\zqayyum\Source\Repos\DataMart.Academy.DoraMetrics\tsqlt\Security"
$scriptsFolder = "C:\Users\zqayyum\Source\Repos\DataMart.Academy.DoraMetrics\tsqlt"
# Create base folders if they don't exist
if (!(Test-Path $securityFolder)) { New-Item -ItemType Directory -Path $securityFolder }
if (!(Test-Path $scriptsFolder)) { New-Item -ItemType Directory -Path $scriptsFolder }
# Define main file names (schemas)
$fileNames = @(
"testMergeStagingDataCommits.sql",
"testMergeStagingDataEnvironments.sql",
"testMergeStagingDataPullRequests.sql",
"testMergeStagingDataReleases.sql",
"testMergeStagingDataRepositories.sql",
"testMergeStagingDataThreads.sql"
)
# Define stored procedure test files
$storedProcFiles = @(
"Test changed rows are logged.sql",
"Test changed rows are not logged as inserted.sql",
"Test changed rows are updated.sql",
"Test errors are logged.sql",
"Test import date on changed rows is updated.sql",
"Test new rows are inserted.sql",
"Test new rows are not logged as updated.sql"
)
foreach ($name in $fileNames) {
# Create .sql file under Security
$filePath = Join-Path $securityFolder $name
$schemaName = [System.IO.Path]::GetFileNameWithoutExtension($name)
# Content for schema file
$schemaContent = @"
CREATE SCHEMA [$schemaName]
AUTHORIZATION [dbo];
GO
EXECUTE sp_addextendedproperty
@name = N'tSQLt.TestClass',
@value = 1,
@level0type = N'SCHEMA',
@level0name = N'$schemaName';
GO
"@
Set-Content -Path $filePath -Value $schemaContent
# Create schema folder under Scripts
$schemaFolderPath = Join-Path $scriptsFolder $schemaName
if (!(Test-Path $schemaFolderPath)) {
New-Item -ItemType Directory -Path $schemaFolderPath | Out-Null
}
# Create Stored Procedures subfolder
$storedProcFolderPath = Join-Path $schemaFolderPath "Stored Procedures"
if (!(Test-Path $storedProcFolderPath)) {
New-Item -ItemType Directory -Path $storedProcFolderPath | Out-Null
}
# Create stored procedure files with dynamic content
foreach ($procFile in $storedProcFiles) {
$procFilePath = Join-Path $storedProcFolderPath $procFile
$procName = [System.IO.Path]::GetFileNameWithoutExtension($procFile)
$procContent = @"
CREATE PROCEDURE $schemaName.[$procName]
AS
BEGIN
SET NOCOUNT ON;
-- Assemble
DECLARE @RowsUpdated INT;
EXEC tsqlt.FakeTable @SchemaName = 'staging', @TableName = 'tbl_TemplateTable';
EXEC tsqlt.FakeTable @SchemaName = 'data', @TableName = 'tbl_TemplateTable';
EXEC tsqlt.FakeTable @SchemaName = 'logging', @TableName = 'tbl_ExecutionCounts';
INSERT INTO data.tbl_TemplateTable(UnitDescription, TrunionDisplacementInMM)
VALUES('UT7000', '9');
INSERT INTO staging.tbl_TemplateTable(UnitDescription, TrunionDisplacementInMM)
VALUES('UT7000', '6');
-- Act
EXEC staging.pr_MergeStagingData;
-- Assert
SELECT @RowsUpdated = RowsUpdated FROM logging.tbl_ExecutionCounts;
EXEC tsqlt.AssertEquals
@Actual = @RowsUpdated,
@Expected = 1;
END
"@
Set-Content -Path $procFilePath -Value $procContent
}
}
Write-Host "All files and folders created successfully with dynamic procedure content"
#+END_SRC
** Finances
I decided to spin up a docker container to manage finances. For the moment there is no way to connect to a Bank without using an intermediary (privacy concerns).
[[../../../assets/images/reviews/30-11-2025-acutal.png]]
* Next Week
- Understands Server Operating Systems (might be tough)
- Understanding of TCP/IP