# Approximation Algorithms

(近似アルゴリズム)

## Data Structures and Algorithms

### 15th lecture, January 23, 2020

http://www.sw.it.aoyama.ac.jp/2019/DA/lecture15.html

### Martin J. Dürst © 2009-20 Martin J. Dürst 青山学院大学

# Today's Schedule

• Summary of last lecture
• How to deal with NP problems
• Approximation algorithms
• Schedule from now on
• Student Survey

# Final Exam

January 30, 09:30-10:55

Any questions?

# Summary of Last Lecture

• There are problems that cannot (yet?) be solved in polynomial time
• Most of these problems are NP-complete (or NP-hard)
• There may be some algorithms that are not in P, but not NP-complete
(an example is graph isomorphism)
• It is important to recognize difficult (NP-complete) problems early
• There are problems that are even more difficult than NP-complete problems

# The Importance of Dealing with NP Problems

• There are many NP-complete and NP-hard problems (and some even more difficult problems)
• Most such problems have many practical applications
• No efficient algorithms are known
• A practical approach is needed

# Strategies for Dealing with NP Problems

• Limit or change problem
• Concentrate on actual data
• Design and use an approximation algorithm

There are some general solutions, but mostly, each problem has to be addressed separately.

# Changing NP Problems

Example: Traveling salesman

• General problem: Arbitrary cost structure
• Simplification: Costs are planar distances on a plane
• Variant: Costs satisfy triangle inequality (AB + BCAC)
(there is a polynomial algorithm that solves this variant within 1.5 times of the optimal solution)

# Deal with Actual Data

Example: 3-SAT

Problems can be classified into three main categories:

• The number of terms is high relative to the number of variables
→ It may be easy to show that there is no solution
• The number of terms is low relative to the number of variables
→ It may be easy to find a solution
• The number of terms is medium relative to the number of variables
→ This is the really difficult case

With many careful optimizations and tricks, practical usages are possible.

Competition: http://www.satcompetition.org

# Approximation Algorithms

• Even if finding a perfect solution is impossible, it may still be desirable to find a close-to-optimal solution
• An algorithm producing an approximate (close to optimal) solution is called an approximation algorithm
• For many approximation algorithms, there may be a guarantee of how much from the optimum the solution can differ
• There are problem-specific approximation algorithms
• There are also general approximation algorithms (algorithm design strategies)
• Hill climbing
• Simulated annealing
• Genetic algorithms
• ...
• Some of these algorithms are used e.g. in machine learning,...
• It may be hard to get even good approximations in polynomial time for many problems (Unique Games Conjecture)
• Quantum computers may contribute to solve such problems quickly in the future

# Problem-specifc Approximation Algorithms

Using m identical machines, finish (as quickly as possible) a number n of tasks that each take time ti to complete

• Algorithm 1:
• Assign the next task to a machine as soon as the previous task finishes
• The overall time is guaranteed to be ≦ 2 times the optimal solution time
• Algorithm 2:
• Assign the next task to a machine as soon as the previous task finishes
• The overall time is guaranteed to be ≦ 1.5 times the optimal solution time

# Hill Climbing

• Produce solutions close to the current solution,
and select the best one among these
• Repeat until no improvement is possible anymore
• Problem: Easily possible to get stuck in a local optimum

# Simulated Annealing

• Origin of name:
Cristal production by carefully lowering temperature
• Randomly change the solution to produce new solution candidates
• Always keep new solutions that are better
• Keep some of the not so good solutions, too
• Repeat but slowly reduce the amount of random change (corresponds to reducing the temperature)
• Stop after a certain number of steps or when there is no further improvement
• Use the overall best solution as the output of the algorithm
• Speedup possible in hardware, see Fujitsu's DAU (Digital Annealer Unit)
• Problems:
• Tuning is necessary for each problem (speed of lowering temperature, ...)
• Not possible to combine partial solutions

# Genetic Algorithms

• Using concepts from evolution theory
• Solution details are interpreted as genetic information
• To get from one generation to the next:
• Combine information from two (or more) different solutions to get a new solution (corresponds to sexual reproduction)
• Move information pieces around inside the new solution (corresponds to crossover)
• Modify information randomly (corresponds to mutation)
• In each generation, produce a lot of new solutions,
and delete (most of) the less optimal ones (corresponds to natural selection)
• Stop after a certain number of generations or when there is no further improvement
• Use the overall best solution as the output of the algorithm
• Problems:
• How to combine solutions
• Parameters choice (number of generations, number of solutions per generation,...)
• Time needed
• Evolution seems to favor 'reasonably adapted' over 'extremely well adapted' individuals/gene pool

# Summary

Approaches to deal with "intractable problems" that cannot be solved perfectly:

• Change or limit the problem
• Concentrate on actual data
• Design and use an approximation algorithm
• Find an approximate solution:
• Problem-specific algorithms
• Hill climbing
• Simulated annealing
• Genetic algorithms, ...

# Schedule from Now On

• January 30, 09:30-10:55: Term Final Exam
• Spring term of junior year (3rd year): Language Theory and Compilers
• Senior year (4th year): Bachelor Research

# Student Survey

(leftover from last lecture)

(授業改善のための学生アンケート)

WEB Survey

お願い: 自由記述に必ず良かった点、問題点を具体的に書きましょう

(悪い例: 発音が分かりにくい; 良い例: さ行が濁っているかどうか分かりにくい)

# Glossary

approximation algorithm

quantum computer

planar distance

triangle inequality

conjecture

hill climbing

simulated annealing

genetic algorithm

ロード・バランシング
machine learning

local optimum

cristal

evolution theory

genetic information

generation

sexual reproduction

crossover

mutation

natural selection