Example. Binary Search

[comp1110-1J]
  • Liam O'Connor

If we add a precondition to the spec of contains, that the input array is sorted, we can use a trick similar to our solution algorithm for our guessing game, which as we saw, can reduce the amount of iterations of the program from linear in the size of the input to logarithmic.

The idea is that we consistently pick the middle index from the range of possible indices. If the value at this index is our target, we have found what we're looking for, and we're done. If the value at this index is larger than our target, that means our desired index is to the left, and reduces the range of possible indices by half. If it is smaller, then our desired index is to the other half, on the right of the selected midpoint.

This narrows the range of possible indices, which permits us to repeat this technique until we reach single elements.

/** Searches for a needle in a sorted haystack @param haystack a sorted (in ascending order) array of integers @param needle an integer to be searched for @return true iff there exists i, haystack[i] == needle */ boolean binarySearch(int[] haystack, int needle) { int lower = 0; int upper = haystack.length - 1; while(lower <= upper) { // INV: if there exists i, haystack[i] == needle, // then lower <= i <= upper int middle = (lower + upper) / 2; if(needle == haystack[middle]) { return true; } else if(needle > haystack[middle]) { lower = middle + 1; } else { upper = middle - 1; } } return false; }

Context

    Binary Search

    [comp1110-77]
    • Fabian Muehlboeck

    Example. Guessing Game

    [comp1110-13]
    • Liam O'Connor, Fabian Muehlboeck

    In the lecture, we created a guessing game, choosing a "random" number based on a fair die roll.

    int generateNumber() { return 34; } void main() { int secret = generateNumber(); boolean correctGuess = false; while(!correctGuess) { String guess = IO.readln("Guess a number! "); int guessedNumber = Integer.parseInt(guess); if(guessedNumber == secret) { IO.println("You win!"); correctGuess = true; } else if(secret < guessedNumber) { IO.println("Smaller!"); } else { IO.println("Bigger!"); } } }

    Note. Bounding our Guesses

    [comp1110-14]
    • Liam O'Connor, Fabian Muehlboeck

    If we know that the range of possible answers in Example [comp1110-13] is \(0\) to \(100\), then we can algorithmically devise a procedure that will always guess the right answer:

    1. We know that the number is between \(0\) and \(100\). Guess \(50\). We get the report Smaller!.
    2. We know that the number is between \(0\) and \(49\). Guess \(25\). We get the report Bigger!.
    3. We know that the number is between \(26\) and \(49\). Guess \(37\). We get the report Smaller!.
    4. We know that the number is between \(26\) and \(36\). Guess \(31\). We get the report Bigger!.
    5. We know that the number is between \(32\) and \(36\). Guess \(34\). We succeed.

    Following this procedure of always bisecting the possible range means that we will always get the answer in at most \(\log _2 100 \simeq 7\) guesses. In general, this bisecting means that for a range \(0\) to \(n\), we will get the answer in at most \(\log _2 n\) guesses.

    Example. Binary Search

    [comp1110-1J]
    • Liam O'Connor

    If we add a precondition to the spec of contains, that the input array is sorted, we can use a trick similar to our solution algorithm for our guessing game, which as we saw, can reduce the amount of iterations of the program from linear in the size of the input to logarithmic.

    The idea is that we consistently pick the middle index from the range of possible indices. If the value at this index is our target, we have found what we're looking for, and we're done. If the value at this index is larger than our target, that means our desired index is to the left, and reduces the range of possible indices by half. If it is smaller, then our desired index is to the other half, on the right of the selected midpoint.

    This narrows the range of possible indices, which permits us to repeat this technique until we reach single elements.

    /** Searches for a needle in a sorted haystack @param haystack a sorted (in ascending order) array of integers @param needle an integer to be searched for @return true iff there exists i, haystack[i] == needle */ boolean binarySearch(int[] haystack, int needle) { int lower = 0; int upper = haystack.length - 1; while(lower <= upper) { // INV: if there exists i, haystack[i] == needle, // then lower <= i <= upper int middle = (lower + upper) / 2; if(needle == haystack[middle]) { return true; } else if(needle > haystack[middle]) { lower = middle + 1; } else { upper = middle - 1; } } return false; }

    Recursion and Loop Invariants

    [comp1110-7N]
    • Fabian Muehlboeck

    Consider the iterative version of binary search, and specifically its loop invariant:

    Example. Binary Search

    [comp1110-1J]
    • Liam O'Connor

    If we add a precondition to the spec of contains, that the input array is sorted, we can use a trick similar to our solution algorithm for our guessing game, which as we saw, can reduce the amount of iterations of the program from linear in the size of the input to logarithmic.

    The idea is that we consistently pick the middle index from the range of possible indices. If the value at this index is our target, we have found what we're looking for, and we're done. If the value at this index is larger than our target, that means our desired index is to the left, and reduces the range of possible indices by half. If it is smaller, then our desired index is to the other half, on the right of the selected midpoint.

    This narrows the range of possible indices, which permits us to repeat this technique until we reach single elements.

    /** Searches for a needle in a sorted haystack @param haystack a sorted (in ascending order) array of integers @param needle an integer to be searched for @return true iff there exists i, haystack[i] == needle */ boolean binarySearch(int[] haystack, int needle) { int lower = 0; int upper = haystack.length - 1; while(lower <= upper) { // INV: if there exists i, haystack[i] == needle, // then lower <= i <= upper int middle = (lower + upper) / 2; if(needle == haystack[middle]) { return true; } else if(needle > haystack[middle]) { lower = middle + 1; } else { upper = middle - 1; } } return false; }

    Contrast this with Recursive binary search. Notice how the loop invariant is essentially encoded in the preconditions for lower and upper. The line of reasoning is the same - the loop invariant must be true at the end of each loop, just as the preconditions of the recursive function call have to be true at each such recursive function call. The precondition must be true at the first call of the recursive function, just as the loop invariant has to be true when entering the loop.

    In exchange, we can assume that the loop invariant is true at the start of the loop and after it, just as we can assume the precondition holds at the start of the function, and (typically) in the base cases.

    Backlinks

      Definition. Binary Search Trees

      [comp1110-2V]
      • Liam O'Connor, Fabian Muehlboeck

      While we can replace linear search with binary search on sorted arrays to improve how many comparisons we have to do, this is not so straightforward with linked lists, even if sorted. The structure of the linked list requires linear traversal, rather than the random access required for binary search.

      Instead, we will store sorted data differently so that finding elements can be much more efficient. Each node of a binary tree contains some data (in our case, an integer) and two branches:

      //BST // - null (for an empty tree) // - a BST object (for a tree node) /** * INVARIANT: the values stored in all descendants * on the left are <= v * the values stored in all descendants * on the right are >= v */ class BST { BST(int v, BST l, BST r) { value = v; left = l; right = r; } int value; BST left; BST right; }
      • value is the integer at this node.
      • left is a tree of integers in the tree that are less than or equal to the value in the current node (Null if there are none).
      • right is a tree of integers in the tree that are greater than or equal to the value in the current node (Null if there are none).

      The BST has a Data Invariant to ensure the conditions on the values stored in the left and right children.

      Can we do better than O(log n) for lookups?

      [comp1110-4J]
      • Liam O'Connor

      Binary Search Trees and sorted arrays give us \(\mathrm {O}(\log n)\) lookup, but this requires the data to be sorted, and sorting requires a total order on keys. Even when a total order exists, \(\mathrm {O}(\log n)\) starts to hurt for very large \(n\). Furthermore, for some key types, such as composite objects with no natural ordering, defining a total order may be awkward or arbitrary.

      The goal is \(\mathrm {O}(1)\) (average-case) lookup, insertion, and deletion, regardless of whether the keys have a natural ordering. This is what hash tables deliver.

      Example. Recursive binary search

      [comp1110-27]
      • Liam O'Connor, Fabian Muehlboeck

      A recursive version of our iterative Binary Search. Note that we had to add additional parameters lower and upper to give us enough flexibility to use this function recursively. Such generalisation is often required when working with recursive functions. Where needed, you can define another function without those parameters that simply calls the main one with suitable initial values.

      /** Searches for a needle in a sorted haystack @param haystack a sorted (in ascending order) array of integers @param needle an integer to be searched for @param lower an integer expressing the lower bound of where needle could be in haystack; PRE: if exists i, haystack[i] == needle, then i >= lower @param upper an integer expressing the upper bound of where needle could be in haystack; PRE: if exists i, haystack[i] == needle, then i <= upper @return true iff there exists i, haystack[i] == needle */ boolean binarySearch(int[] haystack, int needle, int lower, int upper) { if(lower>upper) { return false; } int middle = (lower + upper) / 2; if(haystack[middle] == needle) { return true; } if(needle > haystack[middle]) { return binarySearch(haystack, needle, middle + 1, upper); } else { return binarySearch(haystack, needle, lower, middle - 1); } }

      Try it out in the Java visualiser.

      Example. Time complexity of searching and sorting

      [comp1110-2F]
      • Liam O'Connor
      • Linear search through an array of \(n\) elements: If the target happens to be the first element, the algorithm stops immediately; this is the best case and takes constant time, \(\mathrm {O}(1)\). In the worst case the element is last or not present, so the algorithm must check all \(n\) elements. The worst-case running time is therefore \(\mathrm {O}(n)\).
      • Binary search on a sorted array. Each comparison eliminates half of the remaining elements. After one step at most \(\frac {n}{2}\) elements remain, then \(\frac {n}{4}\), then \(\frac {n}{8}\), and so on. After about \(\log _2 n\) steps only one element remains. Thus binary search runs in \(\mathrm {O}(\log n)\) time in the worst case.
      • Selection sort repeatedly scans the remaining part of the array to find the smallest element. Even if the array is already sorted, it still performs the same number of comparisons, roughly \(\frac {n(n-1)}{2}\). Its running time therefore grows like \(n^2\), so it is \(\mathrm {O}(n^2)\) in both the best and worst cases.
      • Insertion sort behaves differently. If the array is already sorted, each new element is compared once and left in place, so the running time grows linearly: \(\mathrm {O}(n)\). But if the array is in reverse order, each insertion shifts many elements and the total work grows like \(n^2\). Thus the best case is \(\mathrm {O}(n)\) and the worst case is \(\mathrm {O}(n^2)\).
      • Bubble sort with early exit is similar. If the list is already sorted, it finishes after one pass, which is \(\mathrm {O}(n)\). In the worst case — where one small element appears out of order towards the end of the list — it performs many passes and becomes \(\mathrm {O}(n^2)\).
      • Merge sort works differently. It divides the list in half, sorts each half, and then merges them. The list can be halved only about \(\log _2 n\) times, and each level of this process requires work proportional to \(n\) to perform the merges. The total work therefore grows like \(n \log n\). Unlike some previous sorts, this running time does not depend on the initial order of the input: both the best and worst cases are \(\mathrm {O}(n \log n)\).

      Related

        Example. Adding specs to contains

        [comp1110-1A]
        • Liam O'Connor, Fabian Muehlboeck

        We can take the contains function from Example [comp1110-16] and annotate it with a specification. It holds no assumptions about the input parameters, so the function has no nontrivial preconditions. Its postcondition, however, captures the notion of correctness for this function:

        /** Returns if an integer is contained in an integer array @param haystack is an integer array @param needle is the integer we want to search for @return true iff there exists some j such that haystack[j] == needle */ boolean contains(int[] haystack, int needle) { int i = 0; while(i < haystack.length) { // i <= haystack.length && forall 0 <= j < i, haystack[j] != needle if(needle == haystack[i]) { return true; } i = i + 1; } // i == haystack.length // forall 0 <= j < haystack.length, haystack[j] != needle return false; }

        Note how the loop invariant, allows us to reason that our postcondition (the @return clause above) is satisfied by our function.

        Note. Bounding our Guesses

        [comp1110-14]
        • Liam O'Connor, Fabian Muehlboeck

        If we know that the range of possible answers in Example [comp1110-13] is \(0\) to \(100\), then we can algorithmically devise a procedure that will always guess the right answer:

        1. We know that the number is between \(0\) and \(100\). Guess \(50\). We get the report Smaller!.
        2. We know that the number is between \(0\) and \(49\). Guess \(25\). We get the report Bigger!.
        3. We know that the number is between \(26\) and \(49\). Guess \(37\). We get the report Smaller!.
        4. We know that the number is between \(26\) and \(36\). Guess \(31\). We get the report Bigger!.
        5. We know that the number is between \(32\) and \(36\). Guess \(34\). We succeed.

        Following this procedure of always bisecting the possible range means that we will always get the answer in at most \(\log _2 100 \simeq 7\) guesses. In general, this bisecting means that for a range \(0\) to \(n\), we will get the answer in at most \(\log _2 n\) guesses.

        Example. Guessing Game

        [comp1110-13]
        • Liam O'Connor, Fabian Muehlboeck

        In the lecture, we created a guessing game, choosing a "random" number based on a fair die roll.

        int generateNumber() { return 34; } void main() { int secret = generateNumber(); boolean correctGuess = false; while(!correctGuess) { String guess = IO.readln("Guess a number! "); int guessedNumber = Integer.parseInt(guess); if(guessedNumber == secret) { IO.println("You win!"); correctGuess = true; } else if(secret < guessedNumber) { IO.println("Smaller!"); } else { IO.println("Bigger!"); } } }
        bars magnifying-glass caret-down plus minus arrow-right xmark