org_roam/20250804201706-big_o_complexity.org

big-o-complexity

• Big (O) - Time and Space Complexity
  • Intro
  • Technical Definition of Big O
  • Big O - linear example
  • Summation of complexities:

Big (O) - Time and Space Complexity

Intro

Time Complexity: Describes the amount of time necessary to execute an algorithm

Space Complexity: Describes the amount of memory or space utilized by an algorithm/program

Both - asymptotically

Technical Definition of Big O

is a mathematical notation that describes the limiting behaviour of a function when the arguments tend towards a particular value or infinity. Why do we need it? it helps us understand how the performance of an algorithm changes as the size of the input grows, providing a simple way to compare and analyse different algorithms' efficiency.

Improvement in time complexity is often more important as memory is cheap and readily available

In Big O, there are six major types of complexities (time and space):

• Constant: O(1)
• Linear time: O(n)
• Logarithmic time: O(n log n)
• Quadratic time: O(n^2)
• Exponential time: O(2^n)
• Factorial time: O(n!)

Big O - linear example

Suppose we are given a problem where we have a list of `N` numbers of unknown length. We are asked to use code to find and return "True" if the number 2 is in the list and "False" otherwise. Our solution could be to go through every position in the list and check if the number at that position is equal to 2.

[3, 10, 2, 7]

  for number in list:
     if number == 2:
        return True
     else:
        continue
  return False

This would take N time, we need to check every number in the list once, making this solution O(N) - linear time. This looks at the worst case scenarion, if 2 was at the start of the list we know it would take a constant time, however if it's at the end then it would take N time.

In the graph above focus on the tail end of the graphs because Big O is concerned with "as the input size grows what happens to the speed of the operations".

————————–———————————-———————————————————-

Complexity

Name

Description

Common Use Cases

Performance at Scale

————————–———————————-———————————————————-

O(1)

Constant

Runtime unaffected by input size

Hash tables, array access

Excellent

————————–———————————-———————————————————-

O(log n)	Logarithmic	Runtime increases slowly	Binary search, balanced trees	Very good
		(typically halved at each step)

————————–———————————-———————————————————-

O(n)	Linear	Runtime scales linearly	Linear search, array traversal	Good
		(proportional)

————————–———————————-———————————————————-

O(n log n)	Linearithmic	Between linear and quadratic	Efficient sorting algorithms	Fair
		(often seen in divide and conquer
		algorithms)

————————–———————————-———————————————————-

O(n²)

Quadratic

Runtime squares with input size

Nested loops, simple sorting

Poor

————————–———————————-———————————————————-

O(2ⁿ)

Exponential

Runtime doubles with each input

Recursive solutions, combinatorics

Very poor

————————–———————————-———————————————————-

O(n!)	Factorial	Runtime grows by factorial	Permutations, traveling salesman	Terrible
		(extremely slow)

————————–———————————-———————————————————-

Summation of complexities:

When you have multiple operations in an algorithm that each have a linear time complexity O(n), and these operations are sequential (not nested), the overall time complexity of the algorithm remains linear, O(n).

Here's how it works:

1. Summing Linear Operations: If your algorithm involves several separate linear operations, such as:

• First iterating over an array of n elements, • Then, in a separate loop, iterating over the same or another array of n elements, • And perhaps another loop doing the same, each operation has a complexity of O(n). If you sum these, the resulting complexity for these sequential operations is O(n) + O(n) + O(n), and so on.

1. Simplification: According to Big O notation rules, when you add complexities of the same order, the overall complexity is dominated by the term that grows fastest as n increases. For linear operations, O(n) + O(n) + O(n) simplifies to O(n) because the growth rate in terms of the largest input size n doesn't change-it remains linear.

Linear time complexity, denoted as O(n), means that the time required to complete the execution of an algorithm increases linearly with the increase in the size of the input data. In essence, if you double the size of the input, you double the time it takes to process it.

Suppose you have a task to sum the number 5, n times. The number of operations (in this case, additions) you perform directly corresponds to n. For instance:

• If n = 1, you perform the operation 1 time: 5.
• If n = 2, you perform the operation 2 times: 5+5.
• If n = 3, you perform the operation 3 times: 5+5+5.
• And so on…

In each of these cases, the number of addition operations you perform is exactly equal to n. The computational cost grows directly with n, which is the very definition of linear time complexity. Here's a breakdown:

• When n = 1, the number of operations is 1.
• When n = 10, the number of operations is 10.
• When n = 100, the number of operations is 100.
• When n = 1000, the number of operations is 1000.

In general, the total time taken for this task can be described as a function T(n) = n, where T(n) represents the total time or total number of operations, and n is the number of times you need to add 5. This function is a straight line when plotted against n, hence it is classified under linear time complexity O(n).