NP-Completeness, Reducibility

(NP-完全性、帰着可能性)

Data Structures and Algorithms

14th lecture, January 12, 2023

https://www.sw.it.aoyama.ac.jp/2022/DA/lecture14.html

Martin J. Dürst

Today's Schedule

Remaining Schedule
Leftovers and summary of last lecture
NP problems
Reduction
NP-completeness
Student survey

Remaining Schedule

January 12: this lecture
January 19: 15th lecture, Approximation Algorithms
January 26, 09:30-10:55 (85 min): Term Final Exam

Questions about Final Exam

Summary of Last Lecture

Commonalities for many algorithms
⇒ Algorithm design strategies
Use design strategies to generate ideas for new algorithms
Main design strategies:
- Simple/simplistic algorithms
- Brute force
- Greedy algorithms
- Divide and conquer
- Dynamic programming
The appropriate strategy depends on the structure of the problem and its subproblems
Depending on details of the problem, the best design strategy can change
(Example: knapsack problem)

Today's Goal

Be able to distinguish between 'simple' and 'difficult' problems
Expand your view, look at the algorithm universe

Example Problems from Last Lecture

Example problem 1: 3-SAT
Example problem 2: Idependent Set
Example problem 3: Traveling Salesman

Homework: Find Commonalities in Problems

Wide applicability in practice
No known algorithm that is significantly faster than brute force
Brute force takes exponential time
If n increases (example: n=100), it may become impossible to solve the problem
Currently, there is no proof that there is no faster algorithm
There are many problems with the same properties

Problem Shape

Function problems
- There is only one answer
- Correctness (or not) is easy to check
- Examples: Sorting, matrix multiplication, Fibonacci function,...
Optimization problems
- Find the greatest, fastest, shortest,..., or
  as big as possible, as short as possible,...
- Optimality may not be easy to check
Decision problems
- Answer is only yes or no

Decision Problem

Problem where the result can only be true or false
(function problem that returns a boolean variable)
It is possible to convert other problems to decision problems
Optimization problems: Find the largest solution → Is there a solution larger than k?
A decision problem cannot be more difficult than the original problem
(but it may be easier in some cases)
For theory, decision problems are easier to handle

Polynomial Problems and Exponential Problems

Polynomial problems (Ω(n^c)) are called tractable
Examples: Ω(n²), Ω(n³)
Exponential problems (Ω(cⁿ)) are called intractable
Examples: Ω(1.1ⁿ), Ω(2ⁿ)
It is very important to recognize exponential problems early

(the time complexity of a problem is the lowest time complexity among the algorithms that solve the problem, or the lowest theoretical time complexity)

Properties of Polynomial Time

(or why is polynomial time so special)

Problems with large exponents (e.g. Ω(n⁵)) are very rare
Addition and multiplication of polynomials results in polynomials (ring of polynomials)
Polynomial time is not significantly influenced by choice of machine model
Example: Parallel processing with m computers will at best lead to a speedup by a factor of m
It is difficult to find a criterion (e.g. >Ω(n^2.731)??) to subdivide polynomial problems

The set of problems that can be solved in polynomial time is denoted as P

Definition of NP Problems

The variants of problems 1-3 from last lecture are all NP problems
Defining properties of NP problems:
1. If the answer is true, and there is some evidence, then this can be verified quickly (i.e. in polynomial time)
  (however, if the answer is false, verification may take longer)
  (evidence: a solution to 3-SAT, a short path in the Traveling Salesman problem,...)
2. If we can use as many computers as we want, we can find the answer in polynomial time
Why "NP"?: Nondeterministic Polynomial Time
By definition, NP problems are decision problems
Problems that are at least NP are called NP-hard
- May not be decision problems
- May be more difficult than NP

P vs. NP

The set of NP problems is denoted as NP
The set of problems solvable in polynomial time is denoted as P
Problems in P may become NP with only minor changes
Example: Shortest path (P) and longest path (NP) between two vertices in a graph
There are many NP problems
P ⊆ NP (by definition)
P = NP ? P ≠ NP ?
This is the most famous and most important unsolved problem in Computer Science
Clay Mathematics Institute Millenium Problems Rules

Comparing Problems in NP

Need to get order into NP problems
Use comparison of problems
Use reduction to compare problems

An Example of Reduction

Solving a 3-SAT problem by converting (reducing) it to an independent set problem

Create a triangle (graph) for each term of the 3-SAT formula
Label each vertext of a triangle with a variable (or its negation) from the term
Add edges to the graph connecting vertices with the same variable, but opposite negation state
Solve the independent set problem for the graph
Use the variables of the selected vertices (with their negation state) as the solution of the 3-SAT problem
Example (' indicates negation):
(x₁∨x₂∨x₄) ∧ (x₁'∨x₃∨x₄') ∧ (x₂∨x₃'∨x₄) ∧ (x₁'∨x₂'∨x₃')

Overview of Reduction

Idea: Solve a problem A by converting it to a different problem B
(prove that the problem can be solved)
Input for A ⇒ [conversion] ⇒ input for B ⇒ algorithm for B ⇒ output for B ⇒ [back-conversion] ⇒ output for A
Conversion in both directions has to be possible in polynomial time
If A can be reduced (is reducible) to B (in polynomial time), we write A ≤_P B
This mean that (within polynomial conversion), A is simpler (or of the same difficulty as) B

NP-Completeness

An NP problem to which all other NP problems can be reduced is called NP-complete
This means that an NP-complete problem is more difficult (or of the same difficulty) as any other NP problem
Almost all NP problems for which there is no known polynomial algorithm can be shown to be NP-complete
If P ≠ NP, then there are also some problems that are not in P, but also not NP-complete
(but not many such problems are known; main examples are graph isomorphism and prime factoring)

Computational Complexity Theory

Goal: Classification of computational problems
Criteria for classification:
- Time complexity
- Memory complexity (e.g. class PSPACE)
- Shape and size of circuit (or hardware in general)
- Parallelization
- Randomness
- Oracles
- Quantum computing

Reference: Complexity Zoo

Post's Correspondence Problem

(Emil Post, 1946)

Given: A language using two or more characters, and two sequences of words of equal length: W = [w₁, w₂, ... w_n], V = [v₁, v₂, ... v_n]
Problem: Is it possible to choose a (non-empty) sequence of corresponding words from W and V to form the same sentence?
(words may be repeated; spaces between words are ignored)
The problem can be visualized as a card game
Example: W = [AA, B], V = [A, ABA]
Solution: AA-B-AA = A-ABA-A (1-2-1)
Example without solution: W = [CCD, CD, C], V = [CC, DDC, D]
(reason: independent of word choice, the last character of the overall string will always be different)
The above two examples are easy to solve
It can be proven that there exists no general algorithm for this problem

Summary

There are many problems that we (currently) cannot solve in polynomial time
Many of these problems are NP-complete (or NP-hard)
It is important to discover such problems early
There are also problems that are even more difficult to solve than NP-complete problems

Student Survey

(授業改善のための学生アンケート)

WEB Survey

お願い: 自由記述に必ず良かった点、問題点を具体的に書きましょう

(悪い例: 発音が分かりにくい; 良い例: さ行が濁っているかどうか分かりにくい)

Glossary

polynomial problem: 多項式問題
tractable problem: 手に負える問題
exponential problem: 指数的問題
intractable problem: 手に負えない問題
ring of polynomials: 多項式環
nondeterministic polynomial time: 非決定性多項式時間
function problem: 関数問題
optimization problem: 最適化問題
decision problem: 決定問題
NP-hard: NP 困難
reduction: 帰着
reducibility: 帰着可能性
NP-completeness: NP 完全性
graph isomorphism: グラフ同型
Computational Complexity Theory: 計算複雑性理論
Quantum computing: 量子計算
Post's correspondence problem: ポストの対応問題
approximation algorithms: 近似アルゴリズム