vault/Career/Big (O) - Time and Space Complexity.md

---
note type:
  - theory
  - note
date: 2026-06-03
done: true
---
# Big (O) - Time and Space Complexity

## Intro

Time Complexity: Describes the amount of time necessary to execute an algorithm

Space Complexity: Describes the amount of memory or space utilized by an algorithm/program

Both - asymptotically

## Technical Definition of Big O

is a mathematical notation that describes the limiting behaviour of a function when the arguments tend towards a particular value or infinity. Why do we need it? it helps us understand how the performance of an algorithm changes as the size of the input grows, providing a simple way to compare and analyse different algorithms' efficiency.

Improvement in time complexity is often more important as memory is cheap and readily available

In Big O, there are six major types of complexities (time and space):

  - Constant: O(1)

  - Linear time: O(n)

  - Logarithmic time: O(n log n)

  - Quadratic time: O(n<sup>2</sup>)

  - Exponential time: O(2<sup>n</sup>)

  - Factorial time: O(n\!)

## Big O - linear example

Suppose we are given a problem where we have a list of \`N\` numbers of unknown length. We are asked to use code to find and return "True" if the number 2 is in the list and "False" otherwise. Our solution could be to go through every position in the list and check if the number at that position is equal to 2.

\[3, 10, 2, 7\]

``` python
for number in list:
   if number == 2:
      return True
   else:
      continue
return False
```

This would take N time, we need to check every number in the list once, making this solution O(N) - linear time. This looks at the worst case scenarion, if 2 was at the start of the list we know it would take a constant time, however if it's at the end then it would take N time.

![[Big-O-Notation-3130482830.png]]

In the graph above focus on the tail end of the graphs because Big O is concerned with "as the input size grows what happens to the speed of the operations".

|            |              |                                                                            |                                    |                      |
| ---------- | ------------ | -------------------------------------------------------------------------- | ---------------------------------- | -------------------- |
| Complexity | Name         | Description                                                                | Common Use Cases                   | Performance at Scale |
| O(1)       | Constant     | Runtime unaffected by input size                                           | Hash tables, array access          | Excellent            |
| O(log n)   | Logarithmic  | Runtime increases slowly (typically halved at each step)                   | Binary search, balanced trees      | Very good            |
| O(n)       | Linear       | Runtime scales linearly (proportional)                                     | Linear search, array traversal     | Good                 |
| O(n log n) | Linearithmic | Between linear and quadratic (often seen in divide and conquer algorithms) | Efficient sorting algorithms       | Fair                 |
| O(n²)      | Quadratic    | Runtime squares with input size                                            | Nested loops, simple sorting       | Poor                 |
| O(2ⁿ)      | Exponential  | Runtime doubles with each input                                            | Recursive solutions, combinatorics | Very poor            |
| O(n\!)     | Factorial    | Runtime grows by factorial (extremely slow)                                | Permutations, traveling salesman   | Terrible             |

## Summation of complexities:

When you have multiple operations in an algorithm that each have a linear time complexity O(n), and these operations are sequential (not nested), the overall time complexity of the algorithm remains linear, O(n).

Here's how it works:

1.  Summing Linear Operations: If your algorithm involves several separate linear operations, such as:

	• First iterating over an array of n elements,
	• Then, in a separate loop, iterating over the same or another array of n elements,
	• And perhaps another loop doing the same,
	each operation has a complexity of O(n). If you sum these, the resulting complexity for these sequential operations is O(n) + O(n) + O(n), and so on.

2. Simplification: According to Big O notation rules, when you add complexities of the same order, the overall complexity is dominated by the term that grows fastest as n increases. For linear operations, O(n) + O(n) + O(n) simplifies to O(n) because the growth rate in terms of the largest input size n doesn't change-it remains linear.

Linear time complexity, denoted as O(n), means that the time required to complete the execution of an algorithm increases linearly with the increase in the size of the input data. In essence, if you double the size of the input, you double the time it takes to process it.

Suppose you have a task to sum the number 5, n times. The number of operations (in this case, additions) you perform directly corresponds to n. For instance:

  - If n = 1, you perform the operation 1 time: 5.
  - If n = 2, you perform the operation 2 times: 5+5.
  - If n = 3, you perform the operation 3 times: 5+5+5.
  - And so on…

In each of these cases, the number of addition operations you perform is exactly equal to n. The computational cost grows directly with n, which is the very definition of linear time complexity. Here's a breakdown:

  - When n = 1, the number of operations is 1.
  - When n = 10, the number of operations is 10.
  - When n = 100, the number of operations is 100.
  - When n = 1000, the number of operations is 1000.

In general, the total time taken for this task can be described as a function T(n) = n, where T(n) represents the total time or total number of operations, and n is the number of times you need to add 5. This function is a straight line when plotted against n, hence it is classified under linear time complexity O(n).