(アルゴリズムの設計方法)

http://www.sw.it.aoyama.ac.jp/2019/DA/lecture13.html

© 2009-20 Martin J. Dürst 青山学院大学

- Remaining schedule
- Leftovers/summary of last lecture
- Algorithm design strategies
- Examples of 'difficult' problems

- January 9 (today): 13th lecture (algorithm design strategies)
- January 16: 14th lecture (NP-completenes, reducibility;
**makeup class**)

**Important: Contents of makeup class is part of final exam!** - January 23: 15th lecture (approximation algorithms)
- January 30, 09:30-10:55: Term Final Exam

- Dynamic programming is an
*algorithm design strategy* - Dynamic programming is suited for problems where the overall (optimal) solution can be obtained from solutions for subproblems, but the subproblems overlap
- The time complexity of dynamic programming depends on the structure of the actual problem
- Dynamic programming is closely related to memoization

- Simple/simplistic algorithms
- Brute force algorithms
- Greedy algorithms

Find overall optimal solution by selecting locally optimal solutions - Divide and conquer

Divide problem into nonoverlapping subproblems - Dynamic programming

Find overall optimal solution from solutions for overlapping subproblems

- A knapsack with capacity
`c`(weight or volume) and `n`items`s`_{1},...,`s`(each with a weight/volume, and in some variations a value)_{n}- Goal: Find the best way to pack the knapsack

Depending on the details of the problem, the best algorithm design strategy is different

- All items are the same; how many items fit in?

- Pack as many items as possible

- Use as much capacity as possible (integer version)

- Maximise value

Example: Capacity `c` = 20kg, weight per item: 3.5kg

'Algorithm': Divide the capacity by the weight per item, round down

Answer for example: 5 items

Design strategy: simplistic 'algorithm'

- Sometimes too simple to be called 'algorithm'
- Examples:
- Select the third-smallest element from a sorted array
- Obtain the surface of a rectangle from the length of its sides
- Closed formula of a number sequence

- Often forgotten because computers are so fast now

Example: Capacity `c` = 20kg, weight of items: 8kg, 2kg, 4kg, 7kg,
2kg, 1kg, 5kg, 12kg

Algorithm: Sort items by increasing weight, pack starting with lightest item

Answer for example: 5 items (e.g. 1kg, 2kg, 2kg, 4kg, 5kg)

Design strategy: Greedy algorithm

Time complexity: `O`(`n` log
`n`)

- Develop solution step-by-step
- Consider only locally optimal solutions
- Optimal substructure is a precondition, but the structure is different than for dynamic programming
- Time complexities of
`O`(`n`) and`O`(`n`log`n`) are frequent - Examples: Calculating change, many planning/scheduling problems, some algorithms on graphs (e.g. minimum spanning tree,...),...

Example: Capacity `c` = 20kg, weight of items
(`w`_{s1},...,`w`_{sn}):
8kg, 2kg, 4kg, 7kg, 2kg, 1kg, 5kg, 12kg

Algorithm: Consider subproblems with capacity `c'` ≦ `c`
using only the first `k` items `s`_{1},...,
`s`` _{k}` (

Design strategy: Dynamic programming

Recursive problem description:

max_weight(0, ...) = 0 = max_weight(x, {})

max_weight(`c`_{1}, {`s`_{1},...,
`s`` _{k}`}) =
max(max_weight(

max_weight(

k↓ c→ |
0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 |

0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

1 (8kg) | 0 | ×0 | ×0 | ×0 | ×0 | ×0 | ×0 | ×0 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 | ○8 |

2 (2kg) | 0 | ×0 | ○2 | ○2 | ○2 | ○2 | ○2 | ○2 | ×8 | ×8 | ○10 | ○10 | ○10 | ○10 | ○10 | ○10 | ○10 | ○10 | ○10 | ○10 | ○10 |

3 (4kg) | 0 | ×0 | ×2 | ×2 | ○4 | ○4 | ○6 | ○6 | ×8 | ×8 | ×10 | ×10 | ○12 | ○12 | ○14 | ○14 | ○14 | ○14 | ○14 | ○14 | ○14 |

4 (7kg) | 0 | ×0 | ×2 | ×2 | ×4 | ×4 | ×6 | ○7 | ×8 | ○9 | ×10 | ○11 | ×12 | ○13 | ×14 | ○15 | ○15 | ○17 | ○17 | ○19 | ○19 |

5 (2kg) | 0 | ×0 | ×2 | ×2 | ×4 | ×4 | ×6 | ×7 | ×8 | ×9 | ×10 | ×11 | ×12 | ×13 | ×14 | ×15 | ○16 | ×17 | ×17 | ×19 | ×19 |

6 (1kg) | 0 | ○1 | ×2 | ○3 | ×4 | ○5 | ×6 | ×7 | ×8 | ×9 | ×10 | ×11 | ×12 | ×13 | ×14 | ×15 | ×16 | ×17 | ○18 | ×19 | ○20 |

7 (5kg) | 0 | ×1 | ×2 | ×3 | ×4 | ×5 | ×6 | ×7 | ×8 | ×9 | ×10 | ×11 | ×12 | ×13 | ×14 | ×15 | ×16 | ×17 | ×18 | ×19 | ×20 |

8 (12kg) | 0 | ×1 | ×2 | ×3 | ×4 | ×5 | ×6 | ×7 | ×8 | ×9 | ×10 | ×11 | ×12 | ×13 | ×14 | ×15 | ×16 | ×17 | ×18 | ×19 | ×20 |

Table rows: Items to add (or not) to knapsack

Table columns: Total capacity

Table entries: ×: Item in this row is not used; ○: Item in this row is used; number: max weight possible

How to find the solution: See next slide

- Start at bottom right corner of the table
- If there is a ×, do not include the item on this row, and move up one row.
- If there is a ○:

- Do include the item on this row
- Move left
`w`columns, where`w`is the weight of the item included - Move up one row

- Repeat until you arrive at a row or column that is 0

Answer for example: 1kg, 7kg, 4kg, 8kg (total: 20kg; other solutions possible)

Time complexity: `O`(`c``n`)

Example: Capacity `c` = 20kg, weight and value of items: 8kg,
500¥; 2kg, 2000¥; 4kg, 235¥; 7kg, 700¥; 2kg, 400¥; 1kg, 100¥; 5kg, 450¥;
12kg, 650¥

Algorithm: Try all possible solutions

Design strategy: Brute force

Implementation: Dknapsack.rb (of various algorithms, only brute force finds optimal solution if weights and values can be non-integers)

- Try all solution possibilities (combinations, permutations,...)
- Choose the best solution
- May be very slow
- May be improved by shortcuts (e.g. compare with best solution found up to now)
- For more examples, see next lecture

- All items are the same; how many items fit in?

Solution: Divide capacity by item weight

Design strategy: Simplistic 'algorithm' - Pack as many items as possible

Solution: Start with lightest items

Design strategy: Greedy algorithm - Use as much capacity as possible (integer version)

Solution: Consider subproblems with capacity`c'`≦`c`and items`s`_{1},...,`s`(_{k}`k`≦`n`)

Design Strategy: Dynamic programming - Maximize value

Design Strategy: Brute force

- Useful when developing algorithms
- Consider design strategies one-by-one
- For some problems, some strategies can be excluded quickly
- Depending on the details of the problem, the best strategy may be different
- For the same problem and the same strategy, there may be several algorithms

- Be able to distinguish between 'simple' and 'difficult' problems
- Expand your view, look at the algorithm universe

`n`binary variables- A logical formula using these variables
- The formula is a conjunction of disjunctions (of negations)
- All disjunctions use exactly 3 terms
- Problem: Find values for each variable so that the overall formula becomes true
- Problem variant: Decide whether a solution is possible or not
- Example (' indicates negation):

(`x`_{1}∨`x`_{2}∨`x`_{4}) ∧ (`x`_{1}'∨`x`_{3}∨`x`_{4}') ∧ (`x`_{2}∨`x`_{3}'∨`x`_{4}) ∧ (`x`_{1}'∨`x`_{2}'∨`x`_{3}') - Number of possible answers: 2
^{n} - Time to check all possible answers:
`O`(`n`2^{n}) - Currently, no faster general algorithm is known
- Currently, there is no proof that there is no faster general algorithm

- Graph with
`n`vertices - If two vertices are connected by an edge, there is a conflict
- Problem: Find the largest independent set (i.e. subset of vertices without conflicts)
- Problem variant: Decide whether there is an independent set of size
≧
`k` - Number of possible answers: 2
^{n} - Time to check all possible answers:
`O`(`n`2^{n}) - Currently, no faster general algorithm is known
- Currently, there is no proof that there is no faster general algorithm

`n`towns- Distances (or time or cost) between each pair of towns
- Problem: Find the shortest (fastest/cheapest) tour that visits all cities exactly once
- Problem variant: Decide whether there is a tour of size (duration/cost)
≦
`k` - Number of possible answers:
`n`! - Time to check all possible answers:
`O`(`n`!) - Currently, no faster general algorithm is known
- Currently, there is no proof that there is no faster general algorithm

(no need to submit)

- Review this lecture
- Review Dknapsack.rb
- Find commonalities of 3-SAT, independent set, and traveling salesman problems
- Prepare for term final exam

- NP-completeness
- NP-完全性
- reducibility
- 帰着可能性
- approximation algorithms
- 近似アルゴリズム
- brute force
- 総当たり方、腕力法、虱潰し
- greedy algorithm
- 貪欲アルゴリズム
- knapsack problem
- ナップサック問題
- capacity
- 容量
- closed formula
- 「閉じた式」
- number sequence
- 数列
- minimum spanning tree (problem)
- 最小全域気 (問題)
- independent set
- 独立集合
- conflict
- 競合
- traveling salesman
- 巡回セールスマン