Consider the usual algorithm for multiplying two n-by-n matrices a and b to produce the product c.
for( i = 0; i < n; i++)
for(j = 0; j < n; j++){
c[i][j] = 0;
for( k = 0; k < n; k++)
c[i][j] += a[i][k] * b[k][j];
}
The table that follows shows how many times each statement is executed.
| for( i = 0; i < n; i++) | n times - once for each value of i |
| for(j = 0; j < n; j++){ | n2 times - once for each value of i and j |
| c[i][j] = 0; | n2 times |
| for( k = 0; k < n; k++) | n3 times |
| c[i][j] += a[i][k] * b[k][j]; | n3 times |
| } |
We conclude that the number of steps for the matrix multiplication is O(n3), and say that the algorithm takes O(n3) time. This is not a precise time measure - 8 multiplications, after all, would take more physical time than 8 additions, and the speed of the processor would make a difference, and so on. But we can say that an O(n3) algorithm is better than an O(n4) algorithm - at least for large problem sizes.
The next table will help us think about the practical effects of complexity. Assume that our computer can do one step (whatever a step is) per millisecond.
| Algorithm | Time complexity | Max problem size 1 second |
Max problem size 1 minute |
Max problem size 1 hour |
| A1 | n | 1000 | 60000 | 3600000 |
| A2 | 10n | 100 | 6000 | 360000 |
| A3 | n log2n | 140 | 4893 | 2.0*105 |
| A4 | n2 | 31 | 244 | 1897 |
| A5 | 2n2 | 22 | 173 | 1341 |
| A6 | n3 | 10 | 39 | 153 |
| A7 | 10n3 | 4 | 18 | 71 |
| A8 | 2n | 9 | 15 | 21 |
| A9 | 10n | 3 | 4 | 6 |
To see where we get line A4, for example, note that if we can do 1000 steps, then we can solve a problem of size SQRT(1000) = 31.62. But since problem sizes come in integers only, this means that we can finish a problem of size 31, but not 32, in 1000 steps.
To see the effect of complexity another way, suppose we buy a new computer that is ten times faster. Compare the maximum problem size we can do in one time period before and after the speedup.
| Algorithm | Time complexity | Max problem size before speedup |
Max problem size after speedup |
| A1 | n | s1 | 10*s1 |
| A2 | 10n | s2 | 10*s2 |
| A3 | n log2n | s3 | ~10*s3 for large s3 |
| A4 | n2 | s4 | 3.16*s4 |
| A5 | 2n2 | s5 | 3.16*s5 |
| A6 | n3 | s6 | 2.15*s6 |
| A7 | 10n3 | s7 | 2.15*s7 |
| A8 | 2n | s8 | s8+3.3 |
| A9 | 10n | s9 | s9+1 |
Let's see where the A6 line comes from. Before the speedup, we could do k steps, and the maximum problem size we could do was the cube root of k, so s6 = k1/3. After the speedup, we can do 10k steps, so the maximum problem size is (10k)1/3 = 101/3*k1/3 = 2.15*s6. Be sure to compare with the A7 line, noting the effect of a constant factor.
It should now be a little clearer what we mean when we say that quicksort is better than bubblesort. The problem size is the number of elements to be sorted. Bubblesort takes time O(n2), and quicksort takes time O(n log n). Now bubblesort is in fact faster for small problem sizes because the O-notation conceals constant factors. But for large problem sizes, quicksort will be faster.