python NumericalMethods.py

Evaluates/Finds the least squared solutions of Ax=b for x with 3 different methods. A is an m x n matrix, x and b are 1 x n column vectors. Uses A and b from the svd-data.csv dataset.

1. Singular Value Decomposition (A=U.S.V^t) regardless of A's shape or rank

x=sum(i=0, r){ (u_i^t.b / s_i).v_i } where r is the effective rank of A

2. Normal Equations without inverse of A^t.A

A^t.A.x = A^t.b

3. Normal Equations with inverse of A^t.A

x = (A^t.A)^-1.A^t.b

Finds the 2-norm of the residual vector for each.

Each method results in a different value for x. The value of x used to produce b is

x = [1, 1, 1, 1, 1, 1, 1]^t

The condition number of A^t.A is very high which mean any operations that used A^t.A are ill-conditioned, thus, inaccurate. This is why the solutions of x vary so much.