(アルゴリズムの設計方法)

http://www.sw.it.aoyama.ac.jp/2018/DA/lecture13.html

© 2009-18 Martin J. Dürst 青山学院大学

- Remaining schedule
- Leftovers/summary of last lecture
- Algorithm design strategies
- Examples of 'difficult' problems

- December 13: 13th lecture (algorithm design strategies)
- December 20: 14th lecture (NP-completenes, reducibility)
- January 10: 15th lecture (approximation algorithms)
- January 22-29: Term Final Exam

- Coverage:
- Complete contents of lecture and handouts
- No need to be able to write Ruby code, but need to be able to
understand it, and to
**write your own pseudocode** - Type of problems:
- Similar to problems in Discrete Mathematics I or Computer Practice I
- Past exams: 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017
- How to view example solutions:
- Press [表示
(S)] button or [S] key. To revert, press [非表示 (P)]
button or press [P] key.

Sometimes, more than one key press is needed to start switching.

Some images and example solutions are missing. - Important points:
- Read problems carefully (distinguish between calculation, proof, explanation,...)
- Be able to explain concepts in
**your own**words

**Combine**and**apply**knowledge from different lectures- Write
**clearly** - Answers can be in Japanese or English

- Dynamic programming is an
*algorithm design strategy* - Dynamic programming is suited for problems where the overall (optimal) solution can be obtained from solutions for subproblems, but the subproblems overlap
- The time complexity of dynamic programming depends on the structure of the actual problem
- Dynamic programming is closely related to memoization

- Simple/simplistic algorithms
- Brute force algorithms
- Greedy algorithms

Find overall optimal solution by selecting locally optimal solutions - Divide and conquer

Divide problem into nonoverlapping subproblems - Dynamic programming

Find overall optimal solution from solutions for overlapping subproblems

- A knapsack with capacity
`c`(weight or volume) and `n`items`s`_{1},...,`s`(each with a weight/volume, and in some variations a value)_{n}- Goal: Find the best way to pack the knapsack

Depending on the details of the problem, the best algorithm design strategy is different

- All items are the same; how many items fit in?

- Pack as many items as possible

- Use as much capacity as possible (integer version)

- Maximise value

Example: Capacity `c` = 20kg, weight per item: 3.5kg

'Algorithm': Divide the capacity by the weight per item, round down

Answer for example: 5 items

Design strategy: simplistic 'algorithm'

- Sometimes too simple to be called 'algorithm'
- Examples:
- Select the third-smallest element from a sorted array
- Obtain the surface of a rectangle from the length of its sides
- Closed formula of a number sequence

- Often forgotten because computers are now so fast

Example: Capacity `c` = 20kg, weight of items: 8kg, 2kg, 4kg, 7kg,
2kg, 1kg, 5kg, 12kg

Algorithm: Sort items by increasing weight, pack starting with lightest item

Answer for example: 5 items (e.g. 1kg, 2kg, 2kg, 4kg, 5kg)

Design strategy: Greedy algorithm

Time complexity: `O`(`n` log `n`)

- Develop solution step-by-step
- Consider only locally optimal solutions
- Optimal substructure is a precondition, but the structure is different than for dynamic programming
- Time complexities of
`O`(`n`) and`O`(`n`log`n`) are frequent - Examples: Calculating change, many planning/scheduling problems, some algorithms on graphs,...

Example: Capacity `c` = 20kg, weight of items
(`w`_{s1},,...,`w`_{sn}):
8kg, 2kg, 4kg, 7kg, 2kg, 1kg, 5kg, 12kg

Algorithm: Consider subproblems with capacity `c'` ≦ `c`
using only the first `k` items `s`_{1},...,
`s`` _{k}` (

Design strategy: Dynamic programming

Recursive problem description:

max_weight(0, ...) = max_weight(x, {}) = 0

max_weight(`c`_{1}, {`s`_{1},...,
`s`` _{k}`}) =
max(max_weight(

k↓ c→ |
0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 |

0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

1 (8kg) | ×0 | ×0 | ×0 | ×0 | ×0 | ×0 | ×0 | ×0 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 |

2 (2kg) | ×0 | ×0 | ○2 | ○2 | ○2 | ○2 | ○2 | ○2 | ×8 | ×8 | ○10 | ○10 | ○10 | ○10 | ○10 | ○10 | ○10 | ○10 | ○10 | ○10 | ○10 |

3 (4kg) | ×0 | ×0 | ×2 | ×2 | ○4 | ○4 | ○6 | ○6 | ×8 | ×8 | ×10 | ×10 | ○12 | ○12 | ○14 | ○14 | ○14 | ○14 | ○14 | ○14 | ○14 |

4 (7kg) | ×0 | ×0 | ×2 | ×2 | ×4 | ×4 | ×6 | ○7 | ×8 | ○9 | ×10 | ○11 | ×12 | ○13 | ×14 | ○15 | ○15 | ○17 | ○17 | ○19 | ○19 |

5 (2kg) | ×0 | ×0 | ×2 | ×2 | ×4 | ×4 | ×6 | ×7 | ×8 | ×9 | ×10 | ×11 | ×12 | ×13 | ×14 | ×15 | ○16 | ×17 | ×17 | ×19 | ×19 |

6 (1kg) | ×0 | ○1 | ×2 | ○3 | ×4 | ○5 | ×6 | ×7 | ×8 | ×9 | ×10 | ×11 | ×12 | ×13 | ×14 | ×15 | ×16 | ×17 | ○18 | ×19 | ○20 |

7 (5kg) | ×0 | ×1 | ×2 | ×3 | ×4 | ×5 | ×6 | ×7 | ×8 | ×9 | ×10 | ×11 | ×12 | ×13 | ×14 | ×15 | ×16 | ×17 | ×18 | ×19 | ×20 |

8 (12kg) | ×0 | ×1 | ×2 | ×3 | ×4 | ×5 | ×6 | ×7 | ×8 | ×9 | ×10 | ×11 | ×12 | ×13 | ×14 | ×15 | ×16 | ×17 | ×18 | ×19 | ×20 |

Table rows: Items to add (or not) to knapsack

Table columns: Total capacity

How to read off solution:

- Start at bottom right corner.
- If there is a ×, do not include the item on this row, and move up one row.
- If there is a ○:

- Do include the item on this row
- Move up one row
- Move left
`w`rows, where`w`is the weight of the item included

Answer for example: 1kg, 7kg, 4kg, 8kg (total: 20kg; other solutions possible)

Time complexity: `O`(`c``n`)

Example: Capacity `c` = 20kg, weight of items: 8kg, 500¥; 2kg,
2000¥; 4kg, 235¥; 7kg, 700¥; 2kg, 400¥; 1kg, 100¥; 5kg, 450¥; 12kg,
650¥

Algorithm: Try all possible solutions

Design strategy: Brute force

Implementation: Dknapsack.rb (of various algorithms, only brute force finds optimal solution)

- Try all solution possibilities (combinations, permutations,...)
- Choose the best solution
- May be very slow
- May be improved by shortcuts (e.g. compare with best solution found up to now)
- For more examples, see next lecture

- All items are the same; how many items fit in?

Solution: Divide capacity by item weight

Design strategy: Simplistic 'algorithm' - Pack as many items as possible

Solution: Start with lightest items

Design strategy: Greedy algorithm - Use as much capacity as possible (integer version)

Solution: Consider subproblems with capacity`c'`≦`c`and items`s`_{1},...,`s`(_{k}`k`≦`n`)

Design Strategy: Dynamic programming - Maximize value

Design Strategy: Brute force

- Useful when developing algorithms
- Consider design strategies one-by-one
- For some problems, some strategies can be excluded quickly
- Depending on the details of the problem, the best strategy may be different
- For the same problem and the same strategy, there may be several algorithms

- Be able to distinguish between "simple" and "difficult" problems
- Expand your view, let's look at the algorithm universe

`n`binary variables- A logical formula using these variables
- The formula is the conjunction of the disjunction (of negation)
- All disjunctions use exactly 3 terms
- Problem: Find values for each variable so that the overall formula becomes true
- Problem variant: Decide whether a solution is possible or not
- Example (' indicates negation):

(`x`_{1}∨`x`_{2}∨`x`_{4}) ∧ (`x`_{1}'∨`x`_{3}∨`x`_{4}') ∧ (`x`_{2}∨`x`_{3}'∨`x`_{4}) ∧ (`x`_{1}'∨`x`_{2}'∨`x`_{3}') - Number of possible answers: 2
^{n} - Time to check all possible answers:
`O`(`n`2^{n}) - Currently, no faster algorithm is known
- Currently, there is no proof that there is no faster algorithm

- Graph with
`n`vertices - If two vertices are connected by an edge, there is a conflict
- Problem: Find the largest independent set (i.e. subset of vertices without conflicts)
- Problem variant: Decide whether there is an independent set of size
≧
`k` - Number of possible answers: 2
^{n} - Time to check all possible answers:
`O`(`n`2^{n}) - Currently, no faster algorithm is known
- Currently, there is no proof that there is no faster algorithm

`n`towns- Distances (or time or cost) between each pair of cities
- Problem: Find the shortest (fastest/cheapest) tour that visits all cities exactly once
- Problem variant: Decide whether there is a tour of size (duration/cost)
≦
`k` - Number of possible answers:
`n`! - Time to check all possible answers:
`O`(`n`!) - Currently, no faster algorithm is known
- Currently, there is no proof that there is no faster algorithm

(no need to submit)

- Review this lecture
- Review Dknapsack.rb
- Find commonalities of 3-SAT, independent set, and traveling salesman problems
- Prepare for term final exam

- NP-completeness
- NP-完全性
- reducibility
- 帰着可能性
- approximation algorithms
- 近似アルゴリズム
- brute force
- 総当たり方、腕力法、虱潰し
- greedy algorithm
- 貪欲アルゴリズム
- knapsack problem
- ナップサック問題
- capacity
- 容量
- closed formula
- 「閉じた式」
- number sequence
- 数列
- independent set
- 独立集合
- conflict
- 競合
- traveling salesman
- 巡回セールスマン