# Algorithm Design Strategies

(アルゴリズムの設計方法)

## Data Structures and Algorithms

### 13th lecture, December 10, 2015

http://www.sw.it.aoyama.ac.jp/2015/DA/lecture13.html

### Martin J. Dürst

© 2009-15 Martin J. Dürst 青山学院大学

# Today's Schedule

• Remaining schedule
• Leftovers/summary of last lecture
• Algorithm design strategies
• The algorithm universe

# Remaining Schedule

• December 17: 14th lecture (NP-completenes, reducibility)
• January 7: 15th lecture (approximation algorithms)
• January 14: Monday lectures!
• January 21: Make-up Classes
• January 26 - February 2: Term Final Exam Period

# Term Final Exam: Coverage

Contents of all lectures/handouts/programs
プログラムそのものは書く必要がないが、プログラムを理解する、そして自分の議事コードを書ける必要はある
No need to be able to write Ruby code, but need to be able to understand it, and to write your own pseudocode

Similar to problems in Discrete Mathematics I or Computer Practice I

# Term Final Exam: Past Exams

2008年度2009年度2010年度2011年度2012年度2013年度2014年度

Opera 12.17 (Windows/Mac/Linux)、
Google Chrome + Style Chooser 又は
Firefox + Context Style Switcher を使用
「表示」→「スタイル」→「solutions」で表示
Use one of the above browsers (with add-ons) and switch to 'solution' style to check solutions

# Term Final Exam: Important

• 問題をよく読む (計算、証明、説明などの区別)
Read problems carefully (calculation vs. proof vs. explanation,...)
• 概念の定義を自分の言葉でおさえる
Be able to define and explain terms in your own words
• 綺麗な字で書く
Write legibly
• Answers can be in Japanese or in English

# Algorithm Design Strategies/Methods

• Simple/simplistic algorithms
• Brute force algorithms
• Greedy algorithms
Find overall optimal solution by selecting locally optimal solutions
• Divide and conquer
Divide problem into nonoverlapping subproblems
• Dynamic programming
Find overall optimal solution from solutions for overlapping subproblems

# The Knapsack Problem

• A knapsack with capacity c (weight or volume) and
• n items s1,..., sn (each with a weight/volume, and in some variations a value)
• Goal: Find the best way to pack the knapsack

Depending on the details of the problem, the best algorithm design strategy is different

# Variations of the Knapsack Problem

1. All items are the same; how many items fit in?
2. Pack as many items as possible
3. Use as much capacity as possible (integer version)
4. Maximise value

# All Items are the Same

Example: Capacity c = 20kg, weight per item: 3.5㎏

'Algorithm': Divide the capacity by the weight per item, round down

Design strategy: simple "algorithm"

# Simplistic Algorithm

• Sometimes too simple to be called 'algorithm'
• Examples:
• Select the third-smallest element from a sorted array
• Obtain the surface of a rectangle from the length of its sides
• Closed formula of a number sequence
• Often forgotten because computers are now so fast

# Pack as Many Items as Possible

Example: Capacity c = 20kg, weight of items: 8kg, 2kg, 4kg, 7kg, 2kg, 1kg, 5kg, 12kg

Algorithm: Sort items by increasing weight (1kg, 2kg, 2kg, 4kg, 5kg, 7kg, 8kg, 12kg), pack starting from lightest item

Answer for example: 5 items (e.g. 1kg, 2kg, 2kg, 4kg, 5kg)

Design strategy: Greedy algorithm

# Greedy Algorithm

• Develop solution step-by-step
• Consider only locally optimal solutions
• Optimal substructure is a precondition, but the structure is different than for dynamic programming
• Time complexities of O(n) and O(n log n) are frequent
• Examples: Calculating change

# Use as Much Capacity as Possible (Integer Version)

Example: Capacity c = 20kg, weight of items: 8kg, 2kg, 4kg, 7kg, 2kg, 1kg, 5kg, 12kg

Algorithm: Consider subproblems with capacity c'c and items s1,..., sk (kn) (complexity: O(cn))

Answer for example: 8kg, 2kg, 4kg, 1kg, 5kg (total: 20kg; other solutions possible)

Design strategy: Dynamic programming

# Value Maximization

Example: 8kg, 500¥; 2kg, 2000¥; 4kg, 235¥; 7kg, 700¥; 2kg, 400¥; 1kg, 1¥; 5kg, 450¥; 12kg, 650¥

Algorithm: Try all possible solutions

Design strategy: Brute force

Implementation: Dknapsack.rb (of various algorithms, only brute force finds optimal solution)

# Variations of the Knapsack Problem (Summary)

1. All items are the same; how many items fit in?
Solution: Divide capacity by item weight
Design strategy: Simplistic 'algorithm'
2. Pack as many items as possible
Design strategy: Greedy algorithm
3. Use as much capacity as possible (integer version)
Solution: Consider subproblems with capacity c'c and items s1,..., sk (kn)
Design Strategy: Dynamic programming
4. Maximise value
Design Strategy: Brute force

# Algorithm Design Strategies

• Useful when developing algorithms
• Consider design strategies one-by-one
• For some problems, some strategies can be excluded quickly
• Depending on the details of the problem, the best strategy may be different
• For the same problem and the same strategy, there may be several algorithms

# Goal for Remaining Time

• Be able to distinguish between "simple" and "difficult" problems
• Expand your view, let's look at the algorithm universe

# Example Problem 1: 3-SAT

• n binary variables
• A logical formula using these variables
• The formula is the conjunction of the disjunction (of negation)
• All disjunctions use exactly 3 terms
• Problem: Find values for each variable so that the overall formula becomes true
• Problem variant: Decide whether a solution is possible or not
• Example (' indicates negation):
(x1x2x4) ∧ (x1'∨x3x4') ∧ (x2x3'∨x4) ∧ (x1'∨x3'∨x4')
• Number of possible answers: 2n
• Time to check all possible answers: O(n2n)
• Currently, no faster algorithm is known
• Currently, there is no proof that there is no faster algorithm

# Example Problem 2: Independent Set

• Graph with n vertices
• If two vertices are connected by an edge, there is a conflict
• Problem: Find the largest independent set (i.e. subset of vertices without conflicts)
• Problem variant: Decide whether there is an independent set of size ≧k
• Number of possible answers: 2n
• Time to check all possible answers: O(n2n)
• Currently, no faster algorithm is known
• Currently, there is no proof that there is no faster algorithm

# Example Problem 3: Traveling Salesman

• n cities
• Distances (or time or cost) between each pair of cities
• Problem: Find the shortest (fastest/cheapest) tour that visits all cities exactly once
• Problem variant: Decide whether there is a tour of size (duration/cost) ≦k
• Number of possible answers: n!
• Time to check all possible answers: O(n!)
• Currently, no faster algorithm is known
• Currently, there is no proof that there is no faster algorithm

# Homework

(no need to submit)

• Review this lecture
• Find commonalities of 3-SAT, independent set, and traveling salesman problems
• Plan preparation for term final exam

# Glossary

NP-completeness
NP-完全性
reducibility

approximation algorithms

brute force

greedy algorithm

knapsack problem
ナップサック問題
capacity

closed formula
「閉じた式」
number sequence

independent set

conflict

traveling salesman