Example. Time complexity of searching and sorting

[comp1110-2F]
  • Liam O'Connor
  • Linear search through an array of \(n\) elements: If the target happens to be the first element, the algorithm stops immediately; this is the best case and takes constant time, \(\mathrm {O}(1)\). In the worst case the element is last or not present, so the algorithm must check all \(n\) elements. The worst-case running time is therefore \(\mathrm {O}(n)\).
  • Binary search on a sorted array. Each comparison eliminates half of the remaining elements. After one step at most \(\frac {n}{2}\) elements remain, then \(\frac {n}{4}\), then \(\frac {n}{8}\), and so on. After about \(\log _2 n\) steps only one element remains. Thus binary search runs in \(\mathrm {O}(\log n)\) time in the worst case.
  • Selection sort repeatedly scans the remaining part of the array to find the smallest element. Even if the array is already sorted, it still performs the same number of comparisons, roughly \(\frac {n(n-1)}{2}\). Its running time therefore grows like \(n^2\), so it is \(\mathrm {O}(n^2)\) in both the best and worst cases.
  • Insertion sort behaves differently. If the array is already sorted, each new element is compared once and left in place, so the running time grows linearly: \(\mathrm {O}(n)\). But if the array is in reverse order, each insertion shifts many elements and the total work grows like \(n^2\). Thus the best case is \(\mathrm {O}(n)\) and the worst case is \(\mathrm {O}(n^2)\).
  • Bubble sort with early exit is similar. If the list is already sorted, it finishes after one pass, which is \(\mathrm {O}(n)\). In the worst case — where one small element appears out of order towards the end of the list — it performs many passes and becomes \(\mathrm {O}(n^2)\).
  • Merge sort works differently. It divides the list in half, sorts each half, and then merges them. The list can be halved only about \(\log _2 n\) times, and each level of this process requires work proportional to \(n\) to perform the merges. The total work therefore grows like \(n \log n\). Unlike some previous sorts, this running time does not depend on the initial order of the input: both the best and worst cases are \(\mathrm {O}(n \log n)\).

Context

    Algorithmic Complexity

    [comp1110-2A]
    • Liam O'Connor

    Algorithmic complexity is about how the running time (or space, see below) of an algorithm grows as the size of its input grows. If an input has size \(n\), we ask: what happens to the running time when \(n\) becomes large? Does it grow proportionally to \(n\)? To \(n^2\)? Or much more slowly?

    Definition. Big O Notation

    [comp1110-22]
    • Liam O'Connor

    A function \(f(x)\) is in the set \(\mathrm {O}(g(x))\) if and only if there exists constants \(c\) and \(n_0\) such that for any \(n\geq n_0\), \(f(n)\leq c \cdot g(n)\).

    It's a common convention that when we say a function \(f(x)\) is \(\mathrm {O}(g(x))\), we mean that \(g(x)\) provides the tighest possible upper bound (as described above) for \(f(x)\).

    Big O for algorithms

    [comp1110-2B]
    • Liam O'Connor

    By setting our function \(f(n)\) to be the running time taken by our algorithm on an input of size \(n\), we can use Big O Notation to describe the time complexity of our algorithms. This means that \(g(n)\) is an upper bound on how quickly \(f(n)\) can grow (up to a constant factor). For example, if an algorithm performs about \(3n + 5\) steps, then its running time grows proportionally to \(n\), so we say it is \(\mathrm {O}(n)\). As \(n\) becomes large, the constant \(5\) and coefficient \(3\) become less significant.

    Ignoring constant factors

    [comp1110-2C]
    • Liam O'Connor

    The Big O Notation ignores constant factors, and this is usually the correct approach. An algorithm that takes \(3n\) steps and one that takes \(100n\) steps are both linear (\(\mathrm {O}(n)\)). The second is slower, but if the input size doubles, both running times double. Big-O focuses on this growth behaviour, because for large inputs the growth rate matters far more than constant multipliers. A quadratic algorithm in \(\mathrm {O}(n^2)\) will eventually be far slower than any linear algorithm, even if the linear one has a large constant factor.

    Complexity cases

    [comp1110-2D]
    • Liam O'Connor

    Often the running time of an algorithm depends on the arrangement of the input (such as in insertion sort). For this reason we talk about best-case, average-case worst-case complexity. Worst-case complexity is usually the most useful, because it gives a guarantee that the algorithm will never take longer than that bound.

    Example. Time complexity of searching and sorting

    [comp1110-2F]
    • Liam O'Connor
    • Linear search through an array of \(n\) elements: If the target happens to be the first element, the algorithm stops immediately; this is the best case and takes constant time, \(\mathrm {O}(1)\). In the worst case the element is last or not present, so the algorithm must check all \(n\) elements. The worst-case running time is therefore \(\mathrm {O}(n)\).
    • Binary search on a sorted array. Each comparison eliminates half of the remaining elements. After one step at most \(\frac {n}{2}\) elements remain, then \(\frac {n}{4}\), then \(\frac {n}{8}\), and so on. After about \(\log _2 n\) steps only one element remains. Thus binary search runs in \(\mathrm {O}(\log n)\) time in the worst case.
    • Selection sort repeatedly scans the remaining part of the array to find the smallest element. Even if the array is already sorted, it still performs the same number of comparisons, roughly \(\frac {n(n-1)}{2}\). Its running time therefore grows like \(n^2\), so it is \(\mathrm {O}(n^2)\) in both the best and worst cases.
    • Insertion sort behaves differently. If the array is already sorted, each new element is compared once and left in place, so the running time grows linearly: \(\mathrm {O}(n)\). But if the array is in reverse order, each insertion shifts many elements and the total work grows like \(n^2\). Thus the best case is \(\mathrm {O}(n)\) and the worst case is \(\mathrm {O}(n^2)\).
    • Bubble sort with early exit is similar. If the list is already sorted, it finishes after one pass, which is \(\mathrm {O}(n)\). In the worst case — where one small element appears out of order towards the end of the list — it performs many passes and becomes \(\mathrm {O}(n^2)\).
    • Merge sort works differently. It divides the list in half, sorts each half, and then merges them. The list can be halved only about \(\log _2 n\) times, and each level of this process requires work proportional to \(n\) to perform the merges. The total work therefore grows like \(n \log n\). Unlike some previous sorts, this running time does not depend on the initial order of the input: both the best and worst cases are \(\mathrm {O}(n \log n)\).

    Space complexity

    [comp1110-2E]
    • Liam O'Connor

    We can also analyse how much memory an algorithm uses. This is called space complexity. Algorithms like insertion sort and selection sort operate directly on the array and use only a few extra variables, so their space complexity is \(\mathrm {O}(1)\). Merge sort, however, needs an additional temporary array during the merge step, so it requires \(\mathrm {O}(n)\) extra space.

    Time complexity for BSTs

    [comp1110-2Y]
    • Liam O'Connor

    The shape of a binary search tree has a big effect on how quickly (in terms of time complexity) we can look up or insert a student. Each operation involves following a path from the root down to a leaf, so the number of steps is roughly equal to the height of the tree. In a well-balanced tree, where the left and right branches of each node have about the same number of students, the height is approximately \(\log _2 n\) for \(n\) students. This means that each comparison eliminates roughly half of the remaining nodes, and lookups and insertions are fast, taking \(\mathrm {O}(\log n)\) steps. If the tree is unbalanced, for example if students are inserted in alphabetical order, the tree becomes like a long chain, with each new student added as the right child of the previous node. In this case, the height grows to \(n\), and lookups or insertions may require visiting every node, which is \(\mathrm {O}(n)\) time. Random insertion orders typically produce moderately well-balanced shape, so average lookup time is often close to \(\mathrm {O}(\log n)\) but the worst case could still be linear.

    Analysing the Run-Time Complexity of a Function

    [comp1110-7R]
    • Fabian Muehlboeck

    The basic idea here is that each expression and statement here has a cost. Many standard expressions and statements have a constant cost - things like assignments, basic calculations with numbers, comparisons with ==, and so on. Some things might be more expensive than others, but the key question you should always ask first when asked about the time complexity of a function is: "in terms of what?". That is, what is our "n" (and potentially other variables)?

    When we talk about the complexity of our searching and sorting functions, we always talk about the time complexity being in terms of the size of the given array. Oftentimes, it is quite clear what "n" should be, at other times, you can get very different results depending on your choice of parameter, so always keep in mind that it is important to be aware of what "n" is.

    Once you have chosen some set of parameters you care about (i.e. at least one "n", but potentially more variables), you essentially want to look for places in your code where that size influences how much work you do. At the very least, this means looking for loops where the count of iterations depends on your parameters. You also have to watch out for function calls that use values depending on the parameters (say you pass your array to a helper function). In that case, you will have to check how that parameter influences the complexity of that function. You will therefore end up with a number of loops controlled by your parameters, and a number of other operations whose cost is not constant because they are influenced by your parameters.

    Once you have your non-constant-cost operations and loops, you proceed as follows:

    1. look at your innermost loops (those that have no other loops controlled by your parameters nested within them). Sum up the non-constant costs of any operations within them. If there are no such operations, the cost of the code within the loop is constant itself, i.e. \(O(1)\). Summing up complexity costs is essentially an operation looking for the maximum, i.e. \(O(n) + O(n) + O(log_2 n) = O(n)\), but \(O(n) + O(m) = O(n + m)\) if you do not have any specific information that one of n or m is always larger than the other.
    2. Now multiply the cost of the inner operations of the loop with how many times the loop will execute depending on your parameters (that is, a loop in binary search typically executes \(O(log_2 n)\) times, while a regular linear loop through an array or list executes \(O(n)\) times. If your inner operations have cost \(O(n)\), and your loop iterates \(O(n)\) times, you now have a cost of \(O(n * n) = O(n^2)\) for the whole loop).
    3. Collapse these inner loops to treat them as atomic operations of non-constant cost, and proceed again from step 1, until no loops are left. The complexity of your function is the sum (max) of the complexity all remaining operations.

    Example. Analysing the Run-Time Complexity of a Function

    [comp1110-7S]
    • Fabian Muehlboeck

    Based on our simple rules, we now work an example of analysing the complexity of the following function in terms of the size of the given array:

    int determineComplexity(int[] numbers) { int n=numbers.length; int result=0; for (int i=0; i<n; i++) { for (int j=i+1; j<n; j++) { if (numbers[i]==numbers[j]) { result++; } } } for (int i=0; i<n; i++) { for (int j=Math.max(i-1,0); j<Math.min(i+2,n); j++) { result = result + numbers[i][j]; } } return result; }

    This function contains two pairs of nested loops, and each loop mentions the variable n, which conveniently is the size of the given array. We'll start by looking at the first innermost loop:

    for (int j=i+1; j<n; j++) { if (numbers[i]==numbers[j]) { result++; } }

    Array accesses and increasing the value of result are constant operations, so is checking a boolean in an if-condition. That is, the body of the loop has time complexity \(O(1)\). The loop itself starts from some value between 0 and n and goes up to n, without any constant limit as to how far away from n the start might be, and as such executes \(O(n)\) times. Therefore, the overall time complexity of this loop is \(O(n * 1) = O(n)\).

    We can now look at it's parent loop, having collapsed its inner loop:

    for (int i=0; i<n; i++) { //O(n)-cost operation here }

    This loop clearly executes n times, each with an \(O(n)\)-cost operation inside. As such, the total time complexity of this outer loop is \(O(n * n) = O(n^2)\).

    Moving on to the second inner loop:

    for (int j=Math.max(i-1,0); j<Math.min(i+2,n); j++) { result = result + numbers[i][j]; }

    This one is more interesting. The inside of the loop again has a constant cost. But how often does the loop iterate? We start from the maximum of 0 and i-1, and go up to the minimum of i+2 and n. Let's consider the cases:

    • If i-1 is less than 0, we start from 0. Then:
      • If i+2 is less than n, we go up to i+2. Since i-1 is less than 0, i is at most 0, and therefore i+2 is at most 2. This means that the loop does at most 2 iterations, which is a constant.
      • Otherwise, n is less than or equal to i+2, which we already established is at most 2, and we again end up with at most 2, and therefore a constant number of, iterations.
    • If i-1 is greater than or equal to 0, we start from i-1. Then:
      • If i+2 is less than n, we go up to i+2. Since i-1 is 3 less than i+2, the loop does 3 iterations, which is a constant.
      • Otherwise, n is less than or equal to i+2, which we already established is 3 more than the starting point. We again end up with at most 3, and therefore a constant number of, iterations.

    A constant number of iterations over a constant-cost operation means the total time complexity for the inner loop here is \(O(1 * 1) = O(1)\). Now for the parent loop:

    for (int i=0; i<n; i++) { //O(1)-cost operation here }

    This loop clearly executes n times, each with an \(O(1)\)-cost operation inside. As such, the total time complexity of this outer loop is \(O(n * 1) = O(n)\). Having collapsed all our loops, we have:

    int determineComplexity(int[] numbers) { int n=numbers.length; int result=0; //O(n)-cost operation here //O(1)-cost operation here return result; }

    The time complexity of this function is therefore: \(O(n + 1) = O(n)\), i.e. linear.

    Backlinks

      Analysing the Run-Time Complexity of a Function

      [comp1110-7R]
      • Fabian Muehlboeck

      The basic idea here is that each expression and statement here has a cost. Many standard expressions and statements have a constant cost - things like assignments, basic calculations with numbers, comparisons with ==, and so on. Some things might be more expensive than others, but the key question you should always ask first when asked about the time complexity of a function is: "in terms of what?". That is, what is our "n" (and potentially other variables)?

      When we talk about the complexity of our searching and sorting functions, we always talk about the time complexity being in terms of the size of the given array. Oftentimes, it is quite clear what "n" should be, at other times, you can get very different results depending on your choice of parameter, so always keep in mind that it is important to be aware of what "n" is.

      Once you have chosen some set of parameters you care about (i.e. at least one "n", but potentially more variables), you essentially want to look for places in your code where that size influences how much work you do. At the very least, this means looking for loops where the count of iterations depends on your parameters. You also have to watch out for function calls that use values depending on the parameters (say you pass your array to a helper function). In that case, you will have to check how that parameter influences the complexity of that function. You will therefore end up with a number of loops controlled by your parameters, and a number of other operations whose cost is not constant because they are influenced by your parameters.

      Once you have your non-constant-cost operations and loops, you proceed as follows:

      1. look at your innermost loops (those that have no other loops controlled by your parameters nested within them). Sum up the non-constant costs of any operations within them. If there are no such operations, the cost of the code within the loop is constant itself, i.e. \(O(1)\). Summing up complexity costs is essentially an operation looking for the maximum, i.e. \(O(n) + O(n) + O(log_2 n) = O(n)\), but \(O(n) + O(m) = O(n + m)\) if you do not have any specific information that one of n or m is always larger than the other.
      2. Now multiply the cost of the inner operations of the loop with how many times the loop will execute depending on your parameters (that is, a loop in binary search typically executes \(O(log_2 n)\) times, while a regular linear loop through an array or list executes \(O(n)\) times. If your inner operations have cost \(O(n)\), and your loop iterates \(O(n)\) times, you now have a cost of \(O(n * n) = O(n^2)\) for the whole loop).
      3. Collapse these inner loops to treat them as atomic operations of non-constant cost, and proceed again from step 1, until no loops are left. The complexity of your function is the sum (max) of the complexity all remaining operations.

      Related

        Example. Bubble sort

        [comp1110-1Z]
        • August 11, 2026
        • Liam O'Connor

        Bubble sort swaps a lot of elements, but the key idea is that once it finds the maximum element in the unsorted part of the array, it keeps "bubbling" it up further and further, until we reach the end of the unsorted part, where a sorted array of elements larger than the new element is waiting. In that sense, it works like selection sort (either one of them could also just work in the other direction), but with more elements being moved around. We include the "early exit" trick mentioned by Definition [comp1110-1X]:

        void bubbleSort(int[] arr) { int i = arr.length - 1; while(i > 0) { // INV: arr[i..arr.length-1] is sorted, and // for every j in [0..i], arr[j] <= every element // in arr[i+1..arr.length-1] int j = 0; boolean swapped = false; while(j < i) { // INV: for every k < j, arr[k] < arr[j] // and if swapped == false, then arr[0..j-1] is sorted if(arr[j] > arr[j + 1]) { int swap = arr[j]; arr[j] = arr[j + 1]; arr[j+1] = swap; swapped = true; } j = j + 1; } // for every k < i, arr[k] < arr[i], and if // swapped == false, then arr[0..i-1] is sorted if(!swapped) { // at this point, both arr[0..i-1] and arr[i..arr.length-1] // are sorted, and we know that every element in // arr[0..i-1] is less than or equal to every element in // arr[i..arr.length-1]. hence, the whole array is sorted i = 0; } i = i - 1; } // arr[0..arr.length-1] is sorted }

        Definition. Quadratic sorting methods

        [comp1110-1X]
        • August 11, 2026
        • Liam O'Connor, Fabian Muehlboeck

        We demonstrated two sorting algorithms in the lecture:

        • Selection sort: We divide the array into an "unsorted zone" and a "sorted zone". The sorted zone begins empty at the start of the array, and the unsorted zone begins by occupying the whole array. Each loop iteration, we look through the unsorted zone, find the minimum, and swap it with the first element of the unsorted zone (right next to the sorted zone). This means that the sorted zone can be expanded by one element. Repeat until the unsorted zone is of size zero.
        • Bubble sort: Here, the "sorted zone" grows from the end of the array rather than the beginning. Loop through the unsorted zone comparing adjacent elements. If they are not in order, swap them. This will result in the highest element reaching the top. This means we can increase the size of the sorted zone. Repeat until all sorted.

          If we go through the unsorted zone without doing any swaps, we can immediately stop — the array is sorted. This is called an early exit trick.

        I briefly mentioned another one at the start of the workshop:

        • Insertion sort: The most popular of these three, and somewhat similar to selection sort, in that the sorted zone begins empty and grows from the left. Instead of finding the minimum, however, we take the first element from the unsorted zone and remember it (call it X) and shift everything greater than X in the sorted zone to the right one spot. Then we place X in the slot left over.

        These algorithms are called "quadratic" because they take a quadratic amount of key operations (in terms of the size of the array), in this case comparisons and moves/swaps to perform.

        While insertion sort is generally the most popular of the three, each of them have their own advantages, depending on your particular situation. On arrays:

        • Selection sort minimizes the number of times you have to move an array element.
        • Bubble sort only swaps adjacent elements and has an easy way to exit early on sorted arrays.
        • Insertion sort can minimize the number of comparisons that you do (these can become expensive on more complex data types).

        Which of these you actually get depends on the particulars of your implementation, and minor variations (say starting from the other direction) can make a huge difference for certain scenarios.

        Example. Adding specs to contains

        [comp1110-1A]
        • Liam O'Connor, Fabian Muehlboeck

        We can take the contains function from Example [comp1110-16] and annotate it with a specification. It holds no assumptions about the input parameters, so the function has no nontrivial preconditions. Its postcondition, however, captures the notion of correctness for this function:

        /** Returns if an integer is contained in an integer array @param haystack is an integer array @param needle is the integer we want to search for @return true iff there exists some j such that haystack[j] == needle */ boolean contains(int[] haystack, int needle) { int i = 0; while(i < haystack.length) { // i <= haystack.length && forall 0 <= j < i, haystack[j] != needle if(needle == haystack[i]) { return true; } i = i + 1; } // i == haystack.length // forall 0 <= j < haystack.length, haystack[j] != needle return false; }

        Note how the loop invariant, allows us to reason that our postcondition (the @return clause above) is satisfied by our function.

        Example. Binary Search

        [comp1110-1J]
        • Liam O'Connor

        If we add a precondition to the spec of contains, that the input array is sorted, we can use a trick similar to our solution algorithm for our guessing game, which as we saw, can reduce the amount of iterations of the program from linear in the size of the input to logarithmic.

        The idea is that we consistently pick the middle index from the range of possible indices. If the value at this index is our target, we have found what we're looking for, and we're done. If the value at this index is larger than our target, that means our desired index is to the left, and reduces the range of possible indices by half. If it is smaller, then our desired index is to the other half, on the right of the selected midpoint.

        This narrows the range of possible indices, which permits us to repeat this technique until we reach single elements.

        /** Searches for a needle in a sorted haystack @param haystack a sorted (in ascending order) array of integers @param needle an integer to be searched for @return true iff there exists i, haystack[i] == needle */ boolean binarySearch(int[] haystack, int needle) { int lower = 0; int upper = haystack.length - 1; while(lower <= upper) { // INV: if there exists i, haystack[i] == needle, // then lower <= i <= upper int middle = (lower + upper) / 2; if(needle == haystack[middle]) { return true; } else if(needle > haystack[middle]) { lower = middle + 1; } else { upper = middle - 1; } } return false; }

        Example. Merge Sort

        [comp1110-29]
        • Liam O'Connor, Fabian Muehlboeck

        Merge sort is both an efficient sorting algorithm (although it is not in-place) and a classic example of Divide and Conquer techniques. The algorithm works by dividing the input array in half, recursively sorting each half, and then merging the results. We can write an efficient merge function that combines two sorted lists into a single combined sorted lists. Note that we just loop through both arrays once, so this merge function is more efficient than simply appending the two arrays and then sorting them by other means.

        /** * Returns the sorted combination of two * sorted arrays * @param left a sorted array of numbers * @param right a sorted array of numbers * @return a sorted array that contains * exactly the elements of left and right */ int[] merge(int[] left, int[] right) { int leftPos = 0; int rightPos = 0; int[] result = new int[left.length + right.length]; while(leftPos < left.length && rightPos < right.length) { int leftElement = left[leftPos]; int rightElement = right[rightPos]; if(leftElement < rightElement) { result[leftPos + rightPos] = leftElement; leftPos = leftPos + 1; } else { result[leftPos + rightPos] = rightElement; rightPos = rightPos + 1; } } while(leftPos < left.length) { result[leftPos + rightPos] = left[leftPos]; leftPos = leftPos + 1; } while(rightPos < right.length) { result[leftPos + rightPos] = right[rightPos]; rightPos = rightPos + 1; } return result; }

        This function consistutes the "combine" step in the divide and conquer strategy. Now all that is left is the "dividing" and "conquering":

        /** * Returns a sorted version of the part of the * given array between lower and upper * @param arr an array of integers, PRE: not null * @param lower the lower index of the part of the * array we want to sort, PRE: lower <= upper, * PRE: 0 <= lower < arr.length * @param upper the upper index of the part of the * array we want to sort, PRE: lower <= upper, * PRE: 0 <= upper < arr.length * @return a sorted version of the numbers in the * arr between lower and uppper */ int[] mergeSort(int[] arr, int lower, int upper) { if(arr.length==0) { return arr; } if(upper == lower) { return new int[]{arr[lower]}; } int middle = (lower + upper) / 2; int[] left = mergeSort(arr, lower, middle); int[] right = mergeSort(arr, middle+1, upper); return merge(left, right); }

        Note: This functions efficiency can be slightly improved by only allocating one output array and passing it into each function, however this obscures the functionality of the algorithm.

        bars magnifying-glass caret-down plus minus arrow-right xmark