Efficiency of Algorithms

Background
Analysis
Measurement
- Time
- Space

Algorithms must be analyzed to determine resource usage
Efficiency is analogous to engineering productivity for a repeating process
Measured in complexity of time and space (resources)

Background

Importance of efficiency with respect to time was first emphasized by Ada Lovelace in 1843 regarding the babbage machine:

In almost every computation a great variety of arrangements for the succession of the process is possible, and variouis considerations must influence the selections amongst them for the purpose of a calculating engine. One essential object is to chose that arrangement which shall tend to reduce to a minimum the time necessary for completing the calculation.

Analysis

The most commonly used notation to describe resource consumption (complexity) is using Donald Knuth's Big O notation.

Notation	Name	Example
O(1)	Constant	Finding the median in a sorted list of measurements
O(log n)	Logarithmic	Finding an item in an sorted array with a binary search
O(n)	Linear	Finding an item in an unsorted array
O(n log n)	Linearithmic	Performing a fast sorting algorithm
O(n²)	Quadratic	Multiplying two n digit numbers by a simple algorithm
O(n²), c > 1	Exponential	Finding the optimal solution to the traveling salesman problem

Measurement

Expression as a function of the size of the input n, with the main measurements being:

Time: How long does the algorithm take to complete?
Space: How much working memory is required?

Other measurements for devices whose power needs to be factored in, such as for battery operated devices or super computers:

Direct power: How much power is needed to compute the algorithm?
Indirect power: How much power is needed for cooling, lighting, etc?

Less commonly used:

Transmission size: How much data must be transferred for operation?
External space: Space needed physically (such as on disk)
Response time: Relevant in real time applications
Total cost: Cost of hardware dedication

Time

Uses time complexity analysis
Measured in CPU time usage
Detailed estimates needed to compare performance
Difficult to measure in parallel processing

Space

Concerned with the usage of memory resources, typically the memory or storage needed to hold:

Code for the algorithm
Working space
Input data
Output data
Processing data

2.8 KiB Raw Blame History