13
Some checks failed
Build Quartz Notes / build (push) Failing after 20s

This commit is contained in:
2026-07-13 09:16:09 +01:00
parent 9645ee23b0
commit 129ce1442b
336 changed files with 3812 additions and 579 deletions

0
Career/Database - SQL/Airflow Dags.md Normal file → Executable file
View File

0
Career/Database - SQL/Airflow Tasks.md Normal file → Executable file
View File

0
Career/Database - SQL/Airflow.md Normal file → Executable file
View File

View File

Before

Width:  |  Height:  |  Size: 447 KiB

After

Width:  |  Height:  |  Size: 447 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 545 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 114 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

0
Career/Database - SQL/Database MOC.md Normal file → Executable file
View File

View File

0
Career/Database - SQL/Postgres.md Normal file → Executable file
View File

View File

@@ -0,0 +1,179 @@
---
note type:
- database
- sql
- note
date: 2026-07-03
done:
link: https://bertwagner.com/posts/introduction-to-sql-server-execution-plans/
---
## What is an Execution Plan?
- SQL Server is a **declarative language**:
- You specify _what_ data you want, not _how_ to retrieve it.
- An **execution plan** shows:
- How SQL Server decides to retrieve the data.
- The sequence of operations (joins, scans, seeks, etc.).
### Key Idea
- The **Query Optimizer**:
- Evaluates multiple possible ways to execute a query.
- Chooses a plan that is **“good enough”**, not necessarily the absolute fastest.
- Reason:
- Evaluating every possible plan would be too slow and impractical.
### When Problems Occur
- Poor performance happens when:
- The optimizer picks an inefficient plan.
- Causes may include:
- Inaccurate statistics
- Incorrect assumptions about data distribution
- Execution plans help diagnose these issues.
---
## Types of Execution Plan Views
### 1. Text-Based Execution Plan
To see an execution plan for your query, you can run `SET SHOWPLAN_ALL ON`. This will provide a text-based tree representation of the plan:
`SET SHOWPLAN_ALL ON GO --Query
`GO SET SHOWPLAN_ALL OFF GO`
![[Pasted image 20260703150027.png]]
### 2. Graphical Execution Plan (Preferred)
- Generated via:
- **“Display Estimated Execution Plan”** in SSMS
- Features:
- Visual flow of query execution
- Easier to interpret than text-based plans
![[Pasted image 20260703150047.png]]
#### XML View
- Access via:
- Right-click → “Show Execution Plan XML”
- Contains:
- Detailed metadata and properties not always visible graphically
---
## Estimated vs Actual Execution Plans
### Estimated Execution Plan
- Created **before execution**
- Based on:
- Table statistics
- Metadata
- Does **NOT include runtime data**
---
### Actual Execution Plan
- Created **after execution**
- Includes:
- Actual number of rows processed
- Number of executions
- Runtime metrics
### Key Difference
|Feature|Estimated Plan|Actual Plan|
|---|---|---|
|When generated|Before execution|After execution|
|Uses real data|No|Yes|
|Structure|Same|Same|
|Runtime metrics|No|Yes|
**Important Insight:**
- The plan structure is usually identical.
- Only the **runtime statistics differ**.
---
## Live Query Statistics (SQL Server 2016+)
### What It Does
- Shows execution progress in **real time**
- Combines:
- Estimated plan + live runtime data
### Benefits
- Helps identify **bottlenecks quickly**
- Useful for:
- Long-running queries
- Beginners learning execution plans
### Example Insight
- If a **Clustered Index Scan** takes most time:
- Focus optimisation efforts there.
---
## Historical Execution Plans
### 1. Plan Cache
- SQL Server stores plans for reuse.
- Access via DMV:
- `sys.dm_exec_query_plan`
#### Key Notes
- Contains:
- Cached (previously executed) query plans
- Limitations:
- Plans may be removed from cache
- Typically shows **estimated plans only**
---
### 2. Query Store
- Stores historical query performance data
- Requirements:
- Must be enabled on the database
#### Benefits
- Allows:
- Retrieval of past execution plans
- Query performance analysis over time
---
## Why Execution Plans Matter
- Provide visibility into:
- How SQL Server retrieves data
- Essential for:
- **Performance tuning**
- Identifying inefficient operations
- Diagnosing slow queries
---
## Summary
- Execution plans reveal **how SQL Server executes queries**.
- The optimizer selects a **“good enough” plan**, not always optimal.
- Main viewing options:
- Text plan
- Graphical plan (most useful)
- XML plan
- Types of plans:
- **Estimated** → Pre-execution
- **Actual** → Includes runtime data
- Advanced tools:
- **Live Query Statistics** → real-time insights
- **Plan Cache & Query Store** → historical analysis

7
Career/Database - SQL/SQL Joins.md Normal file → Executable file
View File

@@ -3,9 +3,8 @@ note type:
- database
- theory
date: 2026-06-03
done:
done: true
---
# The SQL JOIN Clause
The JOIN clause is used to combine rows from two or more tables, based on a related column between them.
@@ -17,4 +16,6 @@ Here are the different types of JOINs in SQL:
- RIGHT (OUTER) JOIN: Returns all rows from the right table, and only the matched rows from the left table
- FULL (OUTER) JOIN: Returns all rows when there is a match in either the left or right table
![[Pasted image 20260603221345.png]]
![[Pasted image 20260603221345.png]]
![[Pasted image 20260619152151.png]]

View File

@@ -4,7 +4,7 @@ note type:
- database
- theory
date: 2026-06-03
done:
done: true
link: https://app.pluralsight.com/ilx/video-courses/sql-joins-constraints-normalization-subqueries/course-overview
---
# Common Aggregate Functions
@@ -771,130 +771,199 @@ Movies_Promotions
- `*` = many
# SQL INNER JOINs (Level 4 Notes)
# SQL INNER JOIN — Explained with an Example
## Overview
## Example Tables
- **INNER JOIN** is used to combine rows from two (or more) tables based on a related column.
- It returns **only the matching records** between the tables (the overlapping part).
- Common use case: retrieving related data in a **single query instead of multiple queries**.
## Key Concepts
### 1. Problem Without JOIN
- To get reviews and corresponding movie titles:
1. Query reviews: SELECT review, movie_id FROM Reviews;
2. Use returned `movie_id`s to query movies: SELECT title FROM Movies WHERE id IN (1, 3, 4);
- This requires **multiple queries**, which is inefficient.
### 2. INNER JOIN Syntax
### Movies
```
SELECT *
FROM Movies
INNER JOIN Reviews
ON Movies.id = Reviews.movie_id;
id | title
---|-------------
1 | Don Juan
2 | The Lost World
3 | Peter Pan
4 | Robin Hood
```
#### Explanation:
- `INNER JOIN Reviews`: specifies the table to join.
- `ON Movies.id = Reviews.movie_id`: defines how rows relate:
- `Movies.id` = primary key
- `Reviews.movie_id` = foreign key
### 3. Result Behaviour
- Only rows with **matching keys in both tables** are returned.
- Examples:
- Movies without reviews → **excluded**
- Reviews without movies → **excluded**
- A movie with multiple reviews → appears **multiple times**
#### Example:
- Movie **"Don Juan"** with 3 reviews → appears 3 times.
- Movie **"Peter Pan"** with no reviews → does not appear.
### 4. Order of Tables
Both queries return the same result:
`FROM Movies INNER JOIN Reviews`
or
`FROM Reviews INNER JOIN Movies`
- Because INNER JOIN returns only **matching data**, order doesn't change the result.
### 5. Selecting Specific Columns
Instead of retrieving all columns (`SELECT *`), specify only what you need:
### Reviews
```
SELECT Movies.title, Reviews.review
FROM Movies
INNER JOIN Reviews
ON Movies.id = Reviews.movie_id;
id | movie_id | body
---|----------|-------------------
1 | 1 | Great movie
2 | 1 | Loved it
3 | 1 | Classic
4 | 2 | Not bad
5 | NULL | Random review
```
- Important: Prefix columns with table names to avoid ambiguity.
## What INNER JOIN Does
## 6. INNER JOIN Across Multiple Tables
Returns **only rows where both tables match**.
You can join more than two tables in a single query.
- Ignores:
- Movies with no reviews
- Reviews with no valid movie
### Example: Get movie title and genres
## Basic Query
```
SELECT Movies.title, Genres.name
FROM Movies
INNER JOIN Movies_Genres
  ON Movies.id = Movies_Genres.movie_id
INNER JOIN Genres
  ON Movies_Genres.genre_id = Genres.id
WHERE Movies.title = 'Peter Pan';
SELECT m.title, r.body
FROM Movies m
INNER JOIN Reviews r
ON m.id = r.movie_id;
```
### How it works:
1. Join **Movies → Movies_Genres** via `movie_id`
2. Join **Movies_Genres → Genres** via `genre_id`
3. Filter for `"Peter Pan"`
## Visual Understanding
- Think of INNER JOIN like the **intersection of two circles (Venn diagram)**:
- Left circle = Movies
- Right circle = Reviews
- Result = only the overlapping middle
## Key Takeaways
- Use **INNER JOIN** to retrieve related data from multiple tables in one query.
- It only returns **matching rows**.
- Always use the `ON` clause to define relationships.
- Specify columns for cleaner results.
- Multiple joins can be chained for more complex relationships.
## Quick Example Summary
## Result (conceptually)
```
-- Basic join
SELECT Movies.title, Reviews.review
FROM Movies
INNER JOIN Reviews
ON Movies.id = Reviews.movie_id;
-- Multi-table join
SELECT Movies.title, Genres.name
FROM Movies
INNER JOIN Movies_Genres
  ON Movies.id = Movies_Genres.movie_id
INNER JOIN Genres
  ON Movies_Genres.genre_id = Genres.id;
Don Juan | Great movie
Don Juan | Loved it
Don Juan | Classic
The Lost World | Not bad
```
## Key Observations
- Only matching data is returned
- `Don Juan` appears 3 times (3 reviews)
- `Peter Pan` is missing (no reviews)
- `Random review` is missing (`movie_id = NULL`)
## Why Use INNER JOIN
Instead of doing this:
```
-- Query 1
SELECT movie_id, body FROM Reviews;
-- Query 2
SELECT title FROM Movies WHERE id IN (1, 2);
```
You can do it in one query:
```
SELECT m.title, r.body
FROM Movies m
INNER JOIN Reviews r
ON m.id = r.movie_id;
```
Cleaner and more efficient.
## Table Order
These are equivalent:
```
FROM Movies m
INNER JOIN Reviews r
ON m.id = r.movie_id;
FROM Reviews r
INNER JOIN Movies m
ON m.id = r.movie_id;
```
Because INNER JOIN only keeps matches.
## Selecting Specific Columns
Avoid `SELECT *`:
```
SELECT m.title, r.body
FROM Movies m
INNER JOIN Reviews r
ON m.id = r.movie_id;
```
This keeps output clean and avoids ambiguity.
## INNER JOIN with Multiple Tables
### Example: Add Genres
#### Additional Tables
```
Movies_Genres
movie_id | genre_id
---------|----------
1 | 10
1 | 11
3 | 12
Genres
id | name
---|---------
10 | Drama
11 | Romance
12 | Fantasy
```
### Query
```
SELECT m.title, g.name
FROM Movies m
INNER JOIN Movies_Genres mg
  ON m.id = mg.movie_id
INNER JOIN Genres g
  ON mg.genre_id = g.id;
```
### Result (conceptually)
```
Don Juan | Drama
Don Juan | Romance
Peter Pan | Fantasy
```
## Mental Model
Think of INNER JOIN as:
- “Give me only rows that exist in **both tables**
## Behaviour Summary
| Situation | Included? |
| --------------------- | --------- |
| Movie with reviews | Yes |
| Movie with no reviews | No |
| Review with no movie | No |
| | |
## One-Line Summary
INNER JOIN = **only matching rows between tables**
# SQL Aliases (Columns & Tables)
## Column Aliases
@@ -982,123 +1051,250 @@ INNER JOIN Genres g ON mg.genre_id = g.id;
# SQL Outer Joins (LEFT & RIGHT)
## Overview
# SQL Outer Joins
Outer joins allow you to combine rows from two tables even when there is no match in one of them. They help ensure that you dont lose data from one side of the relationship.
## Example Tables
### Movies
```
id | title
---|-------------
1 | Don Juan
2 | The Lost World
3 | Peter Pan
4 | Robin Hood
```
### Reviews
```
id | movie_id | body
---|----------|-------------------
1 | 1 | Great movie
2 | 1 | Loved it
3 | 1 | Classic
4 | 2 | Not bad
5 | NULL | Random review
```
## LEFT OUTER JOIN
### Purpose
### What it means
- Returns **all records from the left table** (`Movies`)
- Returns **matching records from the right table** (`Reviews`)
- If no match exists, the right-side columns are filled with `NULL`
“Show me **all movies**, and any reviews they have.”
### Syntax
```
SELECT *
FROM Movies
LEFT OUTER JOIN Reviews
ON Movies.id = Reviews.movie_id;
```
### Key Observations
- Every movie appears in the result
- Movies with multiple reviews appear multiple times
- Movies with **no reviews still appear**, with `NULL` values for review columns
### Example Output Insight
- `Don Juan` appears **3 times** (3 reviews)
- `Peter Pan` appears **once** with no review (NULL values)
## Refining the LEFT JOIN
### Improvements
1. Use table aliases
2. Select only relevant columns
3. Order results
### Example Query
### Query
```
SELECT m.title, r.body
FROM Movies m
LEFT OUTER JOIN Reviews r
ON m.id = r.movie_id
ORDER BY r.id;
LEFT JOIN Reviews r
ON m.id = r.movie_id;
```
### Result Behaviour
### Result (conceptually)
- Movies without reviews (e.g. `Peter Pan`) appear **last** due to ordering by `review id`
```
Don Juan | Great movie
Don Juan | Loved it
Don Juan | Classic
The Lost World | Not bad
Peter Pan | NULL
Robin Hood | NULL
```
### Key idea
- Every movie is shown
- If a movie has no reviews → `NULL`
- If a movie has many reviews → it repeats
Example:
- `Don Juan` appears 3 times
- `Peter Pan` appears once with `NULL`
### How to remember
> LEFT JOIN = **“Keep everything on the left (Movies)”**
## RIGHT OUTER JOIN
### Purpose
### What it means
- Returns **all records from the right table** (`Reviews`)
- Returns **matching records from the left table** (`Movies`)
- If no match exists, the left-side columns are filled with `NULL`
“Show me **all reviews**, even if they dont belong to a movie.”
### Syntax
```
SELECT *
FROM Movies
RIGHT OUTER JOIN Reviews
ON Movies.id = Reviews.movie_id;
```
### Scenario Highlight
- Some `movie_id` values in `Reviews` are set to `NULL`
- These reviews do not link to any movie
### Key Observations
- All reviews are included
- Reviews without a corresponding movie show `NULL` in movie fields
## Refining the RIGHT JOIN
### Example Query
### Query
```
SELECT m.title, r.body
FROM Movies m
RIGHT OUTER JOIN Reviews r
ON m.id = r.movie_id
ORDER BY r.id;
RIGHT JOIN Reviews r
ON m.id = r.movie_id;
```
### Result Behaviour
### Result (conceptually)
- All reviews listed
- Reviews with no associated movie have `NULL` titles
```
Don Juan | Great movie
Don Juan | Loved it
Don Juan | Classic
The Lost World | Not bad
NULL | Random review
```
## LEFT vs RIGHT JOIN Summary
### Key idea
|Join Type|Includes All From|Missing Matches Show As|
- Every review is shown
- If a review has no movie → `NULL` in movie column
Example:
- The `Random review` has no movie → title is `NULL`
### How to remember
> RIGHT JOIN = **“Keep everything on the right (Reviews)”**
## Side-by-Side
| Situation | LEFT JOIN | RIGHT JOIN |
| --------------------- | ------------------- | ------------------ |
| Movie with no reviews | shown (NULL review) | not shown |
| Review with no movie | not shown | shown (NULL movie) |
## Quick Mental Model
- LEFT JOIN → “I care about **Movies**
- RIGHT JOIN → “I care about **Reviews**
- `NULL` = missing relationship
## One-Line Summary
- LEFT JOIN → keep all rows from **first table**
- RIGHT JOIN → keep all rows from **second table**
# SQL Subqueries (Level 5)
## Example Tables
**Movies**
|id|title|sales|duration|
|---|---|---|---|
|1|Batman|50000|120|
|2|Superman|30000|110|
|3|Iron Man|25000|105|
|4|Robin Hood|45000|140|
**Promotions**
|id|movie_id|category|
|---|---|---|
|LEFT OUTER JOIN|Left table|NULLs in right columns|
|RIGHT OUTER JOIN|Right table|NULLs in left columns|
|1|1|non-cash|
|2|2|cash|
|3|4|non-cash|
## Key Takeaways
---
- Use **LEFT JOIN** when you care about all records from the first (left) table
- Use **RIGHT JOIN** when you care about all records from the second (right) table
- `NULL` values indicate missing relationships
- Ordering can affect where unmatched rows appear in results
## What is a Subquery?
- A **subquery** is a query nested inside another query.
- The inner query runs first, and its result is used by the outer query.
- Outer joins ensure you dont lose unmatched data
- LEFT JOIN = “Show everything from the left”
- RIGHT JOIN = “Show everything from the right”
- Useful for identifying missing relationships and incomplete data
## Example 1: Using `IN` with a Subquery
### Goal
Find the **sum of sales** for movies with a **non-cash promotion**.
```
SELECT SUM(sales)
FROM Movies
WHERE id IN (
    SELECT movie_id
    FROM Promotions
    WHERE category = 'non-cash'
);
```
- Inner query returns: `1, 4`
- Outer query sums: `50000 + 45000 = 95000`
## Alternative: Using a Join
```
SELECT SUM(sales)
FROM Movies
INNER JOIN Promotions
  ON Movies.id = Promotions.movie_id
WHERE category = 'non-cash';
```
- Same result, different approach
- Join is typically more performant
## Subquery Variants
```
SELECT *
FROM Movies
WHERE id NOT IN (
    SELECT movie_id
    FROM Promotions
);
```
- Returns movies with **no promotions**
## Example 2: Aggregate Subquery
### Goal
Find movies with **above-average duration**
```
SELECT *
FROM Movies
WHERE duration > (
    SELECT AVG(duration)
    FROM Movies
);
```
- Average duration = `(120 + 110 + 105 + 140) / 4 = 118.75`
- Result: only **Robin Hood (140)**
## Key Takeaway
- Subqueries break problems into steps using intermediate results
- Use them when aggregates or filtering logic cant be expressed directly in `WHERE`