Definition. Amortised complexity and array doubling

[comp1110-43]
  • Liam O'Connor, Fabian Muehlboeck

When an array-backed collection is full, we cannot simply extend it, because arrays have a fixed length. Instead, we create a new array of double the current size and copy all elements across. This makes any individual add that triggers a resize cost \(\mathrm {O}(n)\). However, such expensive calls are rare: after doubling, the next \(\frac {n}{2}\) insertions are cheap (\(\mathrm {O}(1)\) each).

Amortised analysis spreads the cost of an expensive operation across the many cheap operations that preceded it. If we charge each element a small constant at the time it is first inserted to "pre-pay" for its eventual copy, we find that each insertion costs \(\mathrm {O}(1)\) on average over any sequence of \(n\) insertions, even though individual insertions may cost \(\mathrm {O}(n)\).

To see that this is the case here, consider the overall cost of what we have done so far after \(n + 1\) insertions. In the worst case, we just had to double the size of the array for the last insertion, meaning that the array now has size \(2n\). That last doubling involved \(n\) copies. The doubling before that involved \(\frac {n}{2}\) copies. The one before that involved \(\frac {n}{4}\) copies. And so on. All of those copy operations sum up to less than \(2n\) copies. This means that per insertion (of the \(n + 1\) insertions we did so far), we made less than 2 copies (a constant), plus the constant amount of work for the insertion. The combination of those costs is still constant.

Context

    Designing and Testing an Interface

    [comp1110-8G]
    • Fabian Muehlboeck

    Example. The GuestList interface (String version)

    [comp1110-3X]
    • Liam O'Connor, Fabian Muehlboeck

    Our GuestList interface will represent a set of names. The interface exposes three operations:

    public interface GuestList { // record that a guest is coming void add(String name); // record that a guest is no longer coming void remove(String name); // query whether a guest is coming boolean isComing(String name); }

    The semantics are set-like: a guest is either coming or not; adding the same guest twice has the same effect as adding them once. The interface says nothing about how this is implemented — that is left to implementing classes.

    Definition. Unit testing with JUnit

    [comp1110-40]

      A unit test checks a single, small piece of a program in isolation from the rest of the system. Unit tests contrast with integration tests, which test how multiple components work together.

      JUnit is the standard unit-testing framework for Java. A JUnit test is an ordinary Java method annotated with @Test. Inside the method, special JUnit methods are used similarly to the assertions were used previously in this course:

      • assertTrue(expr) — fails if expr is false
      • assertFalse(expr) — fails if expr is true
      • assertEquals(expected, actual) — fails if the two values differ
      • Many more, see the JUnit documentation for a full list.

      If any assertion fails, JUnit reports the test as failed and shows which assertion was violated.

      Definition. Test-Driven Development

      [comp1110-41]
      • Liam O'Connor

      Test-driven development (TDD) is a practice where tests are written before the implementation. The workflow is:

      1. Write a test for the behaviour you want
      2. Watch it fail (since there is no implementation yet)
      3. Write the minimum implementation to make it pass
      4. Repeat

      This encourages thinking carefully about what a piece of code should do before thinking about how it should work. It also ensures that every piece of behaviour has a test from the start.

      Definition. Black-box and white-box testing

      [comp1110-3Z]

        When writing tests we can take two perspectives on the code under test:

        • In black-box testing, tests are written against the interface (or specification) only, without looking at the implementation. This is the preferred approach for unit-testing interfaces, because the tests remain valid across all implementations.
        • In white-box testing, tests are written with knowledge of the implementation internals, targeting specific branches, edge cases, or internal states.

        Example. JUnit tests for GuestList

        [comp1110-3Y]
        • Liam O'Connor

        The following unit tests exercise the GuestList interface in a black-box fashion. To use JUnit in IntelliJ, create a new Java file and include the following imports at the top. You can run all tests in the file by pressing the green play button next to the class name, or run a single test by pressing the play button next to that test method.

        import org.junit.jupiter.api.Test; import static org.junit.jupiter.api.Assertions.*; public class GuestListTests { //Unit Tests - test the parts of your program separately //Black-Box tests - on the interface @Test void test() { GuestList gl = new GuestArrayUniqueList(); assertFalse(gl.isComing("Fabian")); gl.add("Fabian"); assertTrue(gl.isComing("Fabian")); assertFalse(gl.isComing("Nicholas")); gl.add("Nicholas"); assertTrue(gl.isComing("Fabian")); assertTrue(gl.isComing("Nicholas")); gl.remove("Fabian"); assertFalse(gl.isComing("Fabian")); assertTrue(gl.isComing("Nicholas")); gl.add("Fabian"); assertTrue(gl.isComing("Fabian")); assertFalse(gl.isComing("Ann")); gl.remove("Ann"); assertFalse(gl.isComing("Ann")); gl.add("Nicholas"); assertTrue(gl.isComing("Nicholas")); gl.remove("Nicholas"); assertFalse(gl.isComing("Nicholas")); } @Test void removeFromEmpty() { GuestList gl = new GuestArrayList(); gl.remove("Fabian"); assertFalse(gl.isComing("Fabian")); } }

        Notice that the variable gl is declared as GuestList, not as a specific implementation class. This ensures the test targets the interface contract rather than any implementation detail, respecting abstraction boundaries. Each @Test method is independent: JUnit creates a fresh instance of the test class for each one, so the two tests do not interfere with each other.

        Example. Lazy array implementation of GuestList

        [comp1110-42]
        • Liam O'Connor, Fabian Muehlboeck

        The first implementation of the GuestList interface stores names in a plain String[] array. It is called lazy because add does not check for duplicates before inserting. Duplicates are only resolved later, at remove time, by nulling out all occurrences.

        The lazy approach is simple but wastes space: the array may accumulate many nulls over time. When the array is filled up, we double its size (see Definition [comp1110-43]).

        public class GuestArrayList implements GuestList { String[] data = new String[]{null, null}; int count = 0; @Override public void add(String name) { if(count == data.length) { String[] old = data; data = new String[count * 2]; int i=0; while(i < count) { data[i] = old[i]; i = i + 1; } } data[count] = name; count = count + 1; } @Override public void remove(String name) { int i = 0; while(i < count) { if(data[i] != null && data[i].equals(name)) { data[i]=null; } i = i +1; } } @Override public boolean isComing(String name) { int i = 0; while(i < count) { if(Objects.equals(data[i], name)) { return true; } i = i + 1; } return false; } }

        Definition. Amortised complexity and array doubling

        [comp1110-43]
        • Liam O'Connor, Fabian Muehlboeck

        When an array-backed collection is full, we cannot simply extend it, because arrays have a fixed length. Instead, we create a new array of double the current size and copy all elements across. This makes any individual add that triggers a resize cost \(\mathrm {O}(n)\). However, such expensive calls are rare: after doubling, the next \(\frac {n}{2}\) insertions are cheap (\(\mathrm {O}(1)\) each).

        Amortised analysis spreads the cost of an expensive operation across the many cheap operations that preceded it. If we charge each element a small constant at the time it is first inserted to "pre-pay" for its eventual copy, we find that each insertion costs \(\mathrm {O}(1)\) on average over any sequence of \(n\) insertions, even though individual insertions may cost \(\mathrm {O}(n)\).

        To see that this is the case here, consider the overall cost of what we have done so far after \(n + 1\) insertions. In the worst case, we just had to double the size of the array for the last insertion, meaning that the array now has size \(2n\). That last doubling involved \(n\) copies. The doubling before that involved \(\frac {n}{2}\) copies. The one before that involved \(\frac {n}{4}\) copies. And so on. All of those copy operations sum up to less than \(2n\) copies. This means that per insertion (of the \(n + 1\) insertions we did so far), we made less than 2 copies (a constant), plus the constant amount of work for the insertion. The combination of those costs is still constant.

        Example. Strict array implementation of GuestList

        [comp1110-45]
        • Liam O'Connor, Fabian Muehlboeck

        This implementation of GuestList is called strict (as opposed to the lazy implementation) because it enforces uniqueness eagerly: add calls isComing first and returns immediately if the guest is already present.

        This implementation maintains the following data invariant: all non-null entries in data from 0 to count-1 are distinct. Because we are in the same package as GuestArrayList, we have access to its package-private fields and know about how its methods work. We also know that enforcing uniqueness changes nothing about the externally expected behaviour of the class. As such, we can make our new class a subtype of GuestArrayList, and use inheritance to reuse existing code, changing only what needs to be changed:

        public class GuestArrayUniqueList extends GuestArrayList { // Invariant: all non-null values in data[0..count-1] are unique @Override public void add(String name) { if(isComing(name)) { return; } super.add(name); } public void remove(String name) { int i = 0; while(i<count) { if(data[i] != null && data[i].equals(name)) { data[i]=null; return; } i = i +1; } } }

        The add method is overridden to the isComing check first. However, after this check, the code of the method is the same as before, and we can just call the superclass' implementation of that method instead of copying it into here.

        Because the invariant guarantees at most one copy exists, remove can stop after finding the first match. Contrast with the lazy implementation, where remove must scan the entire array to null out all copies. However, because this change to the code of remove is deep within the code of that method, we have to rewrite it fully, rather than calling the superclass' implementation.

        Warning. Encapsulation and defensive copying

        [comp1110-47]
        • Liam O'Connor

        Encapsulation means that the internal state of an object can only be changed through its defined methods. This is broken if a method returns a direct reference to an internal array or other mutable object, because the caller can then modify it from outside.

        The fix is to return instead a defensive copy: a fresh object with the same contents. The caller is then free to do whatever they like with the copy without affecting the internal state of the object.

        Example. Encapsulation danger when returning internal arrays

        [comp1110-46]
        • Liam O'Connor, Fabian Muehlboeck

        Suppose we extend the GuestList interface with a method coming() that returns the current list of guests. This seems straightforward, but a naive implementation illustrates a common encapsulation hazard: returning a direct reference to the internal array.

        // DANGEROUS — exposes internal state public String[] coming() { return this.data; }

        A caller who receives this reference can write list.coming()[0] = "Intruder" and corrupt the guest list without going through add or remove, potentially breaking the data invariant. The correct fix is to return a defensive copy:

        // SAFE — caller gets their own array public String[] coming() { String[] copy = new String[this.count]; int i = 0; while (i < this.count) { copy[i] = this.data[i]; i = i + 1; } return copy; }

        Backlinks

          Definition. Collision resolution: chaining vs linear probing

          [comp1110-4T]
          • Liam O'Connor

          When two keys map to the same bucket in a hash table — a collision — a collision resolution strategy is needed. There are many approaches, but here are two:

          • Chaining: each bucket holds a linked list (or other collection) of all entries that hash to that index. Lookup scans the list linearly. With a good hash function and a sufficiently large array, lists stay short and lookup is \(\mathrm {O}(1)\) amortised.
          • Linear probing: the backing array stores entries directly (no linked lists). On a collision, the table scans forward (wrapping around) until it finds an empty slot. Lookup similarly scans forward until it finds the key or an empty slot. Linear probing has better cache performance than chaining but is sensitive to clustering: long runs of occupied slots that slow down both insertion and lookup. Once the hash table becomes too full, the array size is typically doubled and all elements re-hashed and copied to the new array, similarly to the trick used for ArrayList.

          Java's HashMap uses chaining (with a tree fallback for very long chains). Linear probing is used in some high-performance open-addressing maps.

          Definition. HashMap and HashSet in Java

          [comp1110-4V]
          • Liam O'Connor

          java.util.HashMap<K, V> is the standard library's general-purpose hash table. It implements the Map<K, V> interface from Java collections with \(\mathrm {O}(1)\) amortised put, get, and containsKey, handles resizing automatically, and uses chaining internally.

          java.util.HashSet<E> is a hash table storing only keys (no associated values), implementing Set<E>. It is backed by a HashMap internally.

          Both require that keys correctly implement equals and hashCode.

          Example. Lazy array implementation of GuestList

          [comp1110-42]
          • Liam O'Connor, Fabian Muehlboeck

          The first implementation of the GuestList interface stores names in a plain String[] array. It is called lazy because add does not check for duplicates before inserting. Duplicates are only resolved later, at remove time, by nulling out all occurrences.

          The lazy approach is simple but wastes space: the array may accumulate many nulls over time. When the array is filled up, we double its size (see Definition [comp1110-43]).

          public class GuestArrayList implements GuestList { String[] data = new String[]{null, null}; int count = 0; @Override public void add(String name) { if(count == data.length) { String[] old = data; data = new String[count * 2]; int i=0; while(i < count) { data[i] = old[i]; i = i + 1; } } data[count] = name; count = count + 1; } @Override public void remove(String name) { int i = 0; while(i < count) { if(data[i] != null && data[i].equals(name)) { data[i]=null; } i = i +1; } } @Override public boolean isComing(String name) { int i = 0; while(i < count) { if(Objects.equals(data[i], name)) { return true; } i = i + 1; } return false; } }

          The Java collections API

          [comp1110-4E]
          • Liam O'Connor, Fabian Muehlboeck

          Java provides a rich set of built-in data structures in the java.util package, collectively known as the collections API. Now that we have built GuestList from scratch, we can see how the same ideas appear in the standard library.

          The GuestList interface is essentially a set, and Java provides a Set<T> interface for exactly this purpose. The TreeSet<T> implementation of Set<T> stores elements in sorted order using a binary tree structure:

          import java.util.Set; import java.util.TreeSet; // ... Set<String> guests = new TreeSet<String>(); guests.add("Fabian"); guests.contains("Fabian"); // true guests.remove("Fabian");

          The collections API also provides:

          In practice, you would use these standard library types rather than writing your own. The exercise of building GuestList was to understand what is happening under the hood.

          Related

            Definition. Arrays

            [comp1110-15]
            • Liam O'Connor

            Arrays are a primary way to store linear, ordered collections of data. The elements of an array must all have the same type. An array type is written by appending [] to the type of the array elements. For example, the type of an array of int values is written as int[], whereas the type of an array of String values is written String[]. We create an array with the new keyword, like so:

            int[] arr = new int[50];

            The above code makes space in memory (called allocating) for a sequence of 50 int values, and stores the location of that space in memory in the newly-created variable arr.

            By default, all ints in our array will be set to zero. If we wish to initialise our array with other values, we can include them in the new invocation like so:

            // size no longer needed as javac will count the elements int[] arr = new int[]{10,100,-2,8,0};

            We can access the contents of an array by index. The ith element of the array a (starting counting at zero) can be retreived using the notation a[i], for example:

            int first = arr[0]; IO.println(first + " is the first (index 0) element!");

            We can also update elements of an array using this notation:

            arr[0] = first + 1; IO.println("The first element is now: " + arr[0]); // prints 11

            To get the length of an array a, we can write a.length:

            int len = arr.length; IO.println("The array arr has " + len + " elements.");

            Warning. 

            • Liam O'Connor

            For Strings, we write str.length() (i.e. calling a method, with the parentheses), but for arrays, we just write arr.length.

            bars magnifying-glass caret-down plus minus arrow-right xmark