diff --git a/notes/efficiency-of-algorithms.org b/notes/efficiency-of-algorithms.org new file mode 100644 index 0000000..34f75f7 --- /dev/null +++ b/notes/efficiency-of-algorithms.org @@ -0,0 +1,66 @@ +#+TITLE: Efficiency of Algorithms +#+AUTHOR: Christopher James Hayward + +#+HUGO_BASE_DIR: ~/.local/source/website +#+HUGO_SECTION: notes + ++ Algorithms must be analyzed to determine resource usage ++ Efficiency is analogous to engineering productivity for a repeating process ++ Measured in complexity of time and space (resources) + +* Background + +Importance of efficiency with respect to time was first emphasized by *Ada Lovelace* in 1843 regarding the babbage machine: + +#+begin_quote +In almost every computation a great variety of arrangements for the succession of the process is possible, and variouis considerations must influence the selections amongst them for the purpose of a calculating engine. One essential object is to chose that arrangement which shall tend to reduce to a minimum the time necessary for completing the calculation. +#+end_quote + +* Analysis + +The most commonly used notation to describe resource consumption (complexity) is using *Donald Knuth's* =Big O= notation. + +| Notation | Name | Example | +|--------------+--------------+----------------------------------------------------------------| +| O(1) | Constant | Finding the median in a sorted list of measurements | +| O(log n) | Logarithmic | Finding an item in an sorted array with a binary search | +| O(n) | Linear | Finding an item in an unsorted array | +| O(n log n) | Linearithmic | Performing a fast sorting algorithm | +| O(n²) | Quadratic | Multiplying two n digit numbers by a simple algorithm | +| O(n²), c > 1 | Exponential | Finding the optimal solution to the traveling salesman problem | + +* Measurement + +Expression as a function of the size of the input ~n~, with the main measurements being: + ++ Time :: How long does the algorithm take to complete? ++ Space :: How much working memory is required? + +Other measurements for devices whose power needs to be factored in, such as for battery operated devices or super computers: + ++ Direct power :: How much power is needed to compute the algorithm? ++ Indirect power :: How much power is needed for cooling, lighting, etc? + +Less commonly used: + ++ Transmission size :: How much data must be transferred for operation? ++ External space :: Space needed physically (such as on disk) ++ Response time :: Relevant in real time applications ++ Total cost :: Cost of hardware dedication + +** Time + ++ Uses time complexity analysis ++ Measured in CPU time usage ++ Detailed estimates needed to compare performance ++ Difficult to measure in parallel processing + +** Space + +Concerned with the usage of memory resources, typically the memory or storage needed to hold: + ++ Code for the algorithm ++ Working space ++ Input data ++ Output data ++ Processing data