More PPs and notes etc

Author: AustinTSchaffer

OMSCS/Courses/GA/07.3 - LP - Geometry.md

@@ -21,9 +21,9 @@ The Optimum of LP is achieved at a vertex of the feasible region except for 2 ca
 	- $3x+2y \ge 6$
 	- $x,y \ge 0$
 - The objective function has nothing to do with it. There are no points which satisfy all constraints.
-
-![[Pasted image 20260401145018.png]]
-
+	
+	![[Pasted image 20260401145018.png]]
+	
 ### Unbounded
 - This means that the feasible region is arbitrarily large.
 - Consider $\max \space x+y$

OMSCS/Courses/GA/08.2 - NP - Knapsack.md

@@ -4,3 +4,53 @@ tags:
   - Algorithms
 ---
 # 08.2 - NP - Knapsack
+- NP-Completeness
+- All of NP can be reduced to SAT
+- SAT can be reduced to 3SAT
+- 3SAT can be reduced to IS
+- IS can be reduced to Clique and Vertex Cover
+- **3SAT can be reduced to Subset-Sum**
+- **Subset-Sum can be reduced to Knapsack**
+
+Recall that Knapsack's DP solution requires $O(nB)$ time, which is not polynomial in the size of the input. It would need to be $O(n^p \cdot log(B))$ instead of $O(n^p \cdot B)$.
+
+## Subset Sum Problem
+- **Input:** positive integers: $a_1,...,a_n$ and $t$
+- **Output:** subset $S$ of $\{1,...,n\}$ where
+	- $\sum_{i \in S} a_i = t$ (if such a sum exists)
+	- No otherwise
+- You can use Dynamic Programming to find an $O(nt)$ time algorithm, this is not a P-time algorithm, since it's order $O(n^p \cdot t)$ not $O(n^p \cdot log(t))$.
+
+### Theorem: Subset-Sum is NP-Complete
+- It's in NP, because we can validate the solution on $O(n \space log \space t)$ time.
+- We can reduce 3SAT to Subset-Sum.
+- Input to subset-sum: $2n+2m+1$ numbers.
+	- $v_i,v_i'$ will correspond to $x_i,\overline{x_i}$
+	- $S_j,\overline{S_j}$ will correspond to $C_j,\overline{C_j}$
+	- $t$
+	- All are $\le n+m$ digits long in base 10
+	- $t \approx 10^{n+m}$
+- How to set the variables? Carefully.
+
+![[Pasted image 20260407142432.png]]
+![[Pasted image 20260407142908.png]]
+
+### Correctness
+![[Pasted image 20260407143023.png]]
+
+![[Pasted image 20260407143055.png]]
+
+![[Pasted image 20260407143137.png]]
+
+
+### Example
+![[Pasted image 20260407142655.png]]
+
+![[Pasted image 20260407142728.png]]
+
+![[Pasted image 20260407142801.png]]
+
+
+## Ungraded Homework
+Prove that Knapsack is NP-Complete: [[8.99.1 - Knapsack is NP-Complete (TODO)]]
+	

OMSCS/Courses/GA/Practice Problems/7.11 - Dual to the Example (TODO).md

@@ -0,0 +1,27 @@
+---
+tags:
+  - OMSCS
+  - Algorithms
+  - Practice
+---
+# 7.11 - Dual to the Example
+
+Write the dual to the following linear program.
+
+- $\max x+y$
+- $2x+y \le 3$
+- $x+3y \le 5$
+- $x,y \ge 0$
+
+Find the optimal solutions to both primal and dual LPs.
+
+**Dual LP**
+- $\min \space 3y_1 + 5y_2$
+- s.t.
+	- $y_1,y_2 \ge 0$
+	- $2y_1 + y_2 \ge 1$
+	- $y_1 + 3y_2 \ge 1$
+
+This one got a little ugly, but it's the same process from [[7.12 - Prove Optimality of Point]].
+
+![[Scanned_20260408-1637.jpg]]

OMSCS/Courses/GA/Practice Problems/7.11 - Dual to the Example.md

@@ -0,0 +1,89 @@
+---
+tags:
+  - OMSCS
+  - Algorithms
+  - Practice
+---
+# 7.12 - Prove Optimality of Point
+
+For the linear program
+
+- $\max x_1-2x_3$
+- $x_1-x_2 \le 1$
+- $2x_2 - x_3 \le 1$
+- $x_1,x_2,x_3 \ge 0$
+
+prove that the solution $(x_1, x_2, x_3) = (3/2, 1/2, 0)$ is optimal.
+
+## Work
+Okay, so there's 5 half-spaces being defined here. Each half-space is defined by a plane. Each pair of planes defines a line. Each trio of planes define a point. Since there are 5 spaces, there are $5 \choose 3$ points.
+
+$${5 \choose 3} = \frac{5!}{(3!)(2!)}=\frac{(5)(4)}{(2)(1)}=10$$
+
+What are all of these points? How can we find them?
+
+We can write all of the constraint inequalities as equalities.
+
+1. $x_1=0$
+2. $x_2=0$
+3. $x_3=0$
+4. $x_1-x_2=1$
+5. $2x_2-x_3=1$
+
+Now we can perform the 5 nCr 3 "joins" of these equations to find the vertices formed by each equation. If there is no solution to a triplet of equations, then there does not exist a singular point from the 3 equations, meaning they do not form a vertex of the feasible region. If they do form a vertex, we need to check whether that point lies on the boundary of the feasible region. This basically means we need to check to make sure it doesn't violate any of the constraints. The most common constraint violation will likely be $x_1,x_2,x_3 \ge 0$. Then we can evaluate the objective function at each of the feasible points, and take the max over those points.
+
+Here's that work. There's apparently only 4 vertices in the feasible region.
+
+| Equations | X            | Feasible?            | Obj: $x_1-2x_3$ |
+| --------- | ------------ | -------------------- | --------------- |
+| 1,2,3     | (0,0,0)      | Yes                  | 0               |
+| 1,2,4     | No solution  | n/a                  | n/a             |
+| 1,2,5     | (0,0,-1)     | no ($x_3 \not\ge 0$) | n/a             |
+| 1,3,4     | (0,-1,0)     | no ($x_2 \not\ge 0$) | n/a             |
+| 1,3,5     | (0,1/2,0)    | yes                  | 0               |
+| 1,4,5     | (0,-1,...)   | no ($x_2 \not\ge 0$) | n/a             |
+| 2,3,4     | (1,0,0)      | yes                  | 1               |
+| 2,3,5     | No solution. | n/a                  | n/a             |
+| 2,4,5     | (1,0,-1)     | no ($x_3 \not\ge 0$) | n/a             |
+| 3,4,5     | (3/2,1/2,0)  | yes                  | **1.5**         |
+
+Among the vertices of the feasible region, $(3/2,1/2,0)$ maximizes the objective function, with value $3/2$. Looking at the geometric view, this LP appears to be potentially unbounded.
+
+We need to make sure that the Dual LP is also feasible in order to show that the Primal LP is bounded. Minimizing the objective function for the Dual LP also gives us an upper bound on the maximum value for the Primal LP's objective function.
+
+**Primal**
+- $\max \space x_1+0x_2-2x_3$
+- s.t. $1x_1-1x_2+0x_3 \le 1$
+- s.t. $0x_1 + 2x_2 - 1x_3 \le 1$
+- s.t. $x_1,x_2,x_3 \ge 0$
+
+**Dual**
+- $\min\space y_1 + y_2$
+- s.t. $1y_1 + 0y_2 \ge 1$
+- s.t. $-1y_1 + 2y_2 \ge 0$
+- s.t. $0y_1 - 1y_2 \ge -2$
+- s.t. $y_1,y_2 \ge 0$
+
+We now need to check all of the points of intersection of the corresponding equalities, like we did with the primal. In this case, we have 2 unknowns, so we need to take these 5 equations and choose 2, resulting in ${5 \choose 2} = 10$ possible feasible region vertices.
+
+1. $y_1=0$
+2. $y_2=0$
+3. $y_1=1$
+4. $-y_1+2y_2 = 0$
+5. $-y_2=-2$
+
+| Equations | Y           | Feasible?                       | Obj: $y_1+y_2$ |
+| --------- | ----------- | ------------------------------- | -------------- |
+| 1,2       | (0,0)       | No ($y_1 \not\ge 1$)            | n/a            |
+| 1,3       | No solution | n/a                             | n/a            |
+| 1,4       | (0,...)     | No ($y_1 \not\ge 1$)            | n/a            |
+| 1,5       | (0,...)     | No ($y_1 \not\ge 1$)            | n/a            |
+| 2,3       | (1,0)       | No ($-y_1+2y_2 = -1 \not\ge 0$) | n/a            |
+| 2,4       | (0,0)       | No ($y_1 \not\ge 1$)            | n/a            |
+| 2,5       | No solution | n/a                             | n/a            |
+| 3,4       | (1, 1/2)    | Yes                             | **3/2**        |
+| 3,5       | (1,2)       | Yes                             | 3              |
+| 4,5       | (4,2)       | Yes                             | 6              |
+Here we find that the feasible region of the Dual LP has 3 vertices, of which $(1, 1/2)$ minimizes the objective function with value 3/2. Therefore, we've found an upper bound on the possible value for the Primal LP's objective function.
+
+Point $(3/2,1/2,0)$ maximizes the Primal LP's objective function, with value $3/2$, which is equal to the minimum value of the Dual LP's objective function.

OMSCS/Courses/GA/Practice Problems/7.12 - Prove Optimality of Point (TODO).md

@@ -0,0 +1,49 @@
+---
+tags:
+  - OMSCS
+  - Algorithms
+  - Practice
+---
+# 7.2 - Duckwheat
+> Duckwheat is produced in Kansas and Mexico and consumed in New York and California. Kansas produces 15 shnupells of duckwheat and Mexico 8. Meanwhile, New York consumes 10 shnupells and California 13. The transportation costs per shnupell are $4 from Mexico to New York, $1 from Mexico to California, $2 from Kansas to New York, and $3 and from Kansas to California.
+> 
+> Write a linear program that decides the amounts of duckwheat (in shnupells and fractions of a shnupell) to be transported from each producer to each consumer, so as to minimize the overall transportation cost.
+
+They couldn't have worded this dumber.
+
+## Work
+- Variables
+	- $kc$ - Units from Kansas to California
+	- $mc$ - Units from Mexico to California
+	- $ky$ - Units from Kansas to New (Y)ork
+	- $my$ - Units from Mexico to New (Y)ork
+- Objective:
+	- $\min \space 4my+1mc+2ky+3kc$ (as stated)
+	- $\max \space -4my-1mc-2ky-3kc$ (normalized)
+- Constraints
+	- $kc + ky = 15$
+		- $kc+ky \le 15$
+		- $kc+ky \ge 15 \implies -ky-ky \le 15$
+	- $mc+my=8$
+		- $mc + my \le 8$
+		- $-mc-my \le 8$
+	- $ky+my=10$
+		- $ky+my \le 10$
+		- $-ky-my \le 10$
+	- $kc+mc=13$
+		- $kc+mc \le 13$
+		- $-kc-mc \le 13$
+	- $kc,mc,ky,my \ge 0$
+
+## Normal Form
+- **Objective:** $\max \space -4my-mc-2ky-3kc$
+- **Constraints:**
+	- $kc+ky \le 15$
+	- $-ky-ky \le 15$
+	- $mc + my \le 8$
+	- $-mc-my \le 8$
+	- $ky+my \le 10$
+	- $-ky-my \le 10$
+	- $kc+mc \le 13$
+	- $-kc-mc \le 13$
+	- $kc,mc,ky,my \ge 0$

OMSCS/Courses/GA/Practice Problems/7.12 - Prove Optimality of Point.md

@@ -0,0 +1,31 @@
+---
+tags:
+  - OMSCS
+  - Algorithms
+  - Practice
+---
+# 7.7 - Necessary and Sufficient Conditions
+Find necessary and sufficient conditions on the reals $a$ and $b$ under which the linear program
+
+- $\max \space x+y$
+- $ax+by \le 1$
+- $x,y \ge 0$
+
+Is (a) infeasible, (b) unbounded, and (c) has a unique optimal solution.
+
+## Infeasible
+We need to find $a$ and $b$ such that the feasible region is empty. This never occurs. $(x=0,y=0)$ always satisfies the constraints, regardless of the values of $a$ and $b$.
+
+## Unbounded
+The LP becomes unbounded whenever $a \le 0$ or $b \le 0$.
+
+- When $a$ is 0, the half-plane $ax+by \le 1$ is defined by a line which is parallel with the x axis.
+- When $b$ is 0, the half-plane $ax+by \le 1$ is defined by a line which is parallel with the y axis.
+- When $b \gt 0$ and $a \lt 0$, the half-plane $ax+by \le 1$ is defined by a line with a positive slope, and the feasible region contains all points below that line. (0,0) is always contained within that region.
+- When $a \gt 0$ and $b \lt 0$, the half-plane $ax+by \le 1$ is defined by a line with a positive slope, and the feasible region contains all points above that line. (0,0) is always contained within that region.
+- When both $a,b < 0$, the half-plane $ax+by \le 1$ is defined by a line with negative slope, and the feasible region contains all points above that line. (0,0) is always contained within that region.
+
+On the corollary, if both $a,b > 0$, then the half-plane $ax+by \le 1$ is defined by a line with negative slope, and the feasible region contains all points below that line. (0,0) is always contained within that region. The difference with this case is that the half plane is also bounded by the x and y axes, constraining the region to a triangular shape.
+
+## Unique Optimal Solution
+This LP has a unique optimal solution whenever $a,b>0$ and $a \ne b$. This causes the feasible region to be bounded, forming a right triangle where the hypotenuse is not parallel with $x+y$. This causes the objective function to become maximized at one of the two points.