Add note

notes/efficiency-of-algorithms.org

@@ -0,0 +1,66 @@
+#+TITLE: Efficiency of Algorithms
+#+AUTHOR: Christopher James Hayward
+
+#+HUGO_BASE_DIR: ~/.local/source/website
+#+HUGO_SECTION: notes
+
++ Algorithms must be analyzed to determine resource usage
++ Efficiency is analogous to engineering productivity for a repeating process
++ Measured in complexity of time and space (resources)
+
+* Background
+
+Importance of efficiency with respect to time was first emphasized by *Ada Lovelace* in 1843 regarding the babbage machine:
+
+#+begin_quote
+In almost every computation a great variety of arrangements for the succession of the process is possible, and variouis considerations must influence the selections amongst them for the purpose of a calculating engine. One essential object is to chose that arrangement which shall tend to reduce to a minimum the time necessary for completing the calculation.
+#+end_quote
+
+* Analysis
+
+The most commonly used notation to describe resource consumption (complexity) is using *Donald Knuth's* =Big O= notation.
+
+| Notation     | Name         | Example                                                        |
+|--------------+--------------+----------------------------------------------------------------|
+| O(1)         | Constant     | Finding the median in a sorted list of measurements            |
+| O(log n)     | Logarithmic  | Finding an item in an sorted array with a binary search        |
+| O(n)         | Linear       | Finding an item in an unsorted array                           |
+| O(n log n)   | Linearithmic | Performing a fast sorting algorithm                            |
+| O(n²)        | Quadratic    | Multiplying two n digit numbers by a simple algorithm          |
+| O(n²), c > 1 | Exponential  | Finding the optimal solution to the traveling salesman problem |
+
+* Measurement
+
+Expression as a function of the size of the input ~n~, with the main measurements being:
+
++ Time :: How long does the algorithm take to complete?
++ Space :: How much working memory is required?
+
+Other measurements for devices whose power needs to be factored in, such as for battery operated devices or super computers:
+
++ Direct power :: How much power is needed to compute the algorithm?
++ Indirect power :: How much power is needed for cooling, lighting, etc?
+
+Less commonly used:
+
++ Transmission size :: How much data must be transferred for operation?
++ External space :: Space needed physically (such as on disk)
++ Response time :: Relevant in real time applications
++ Total cost :: Cost of hardware dedication
+
+** Time
+
++ Uses time complexity analysis
++ Measured in CPU time usage
++ Detailed estimates needed to compare performance
++ Difficult to measure in parallel processing
+
+** Space
+
+Concerned with the usage of memory resources, typically the memory or storage needed to hold:
+
++ Code for the algorithm
++ Working space
++ Input data
++ Output data
++ Processing data