# HW2 (matrices) Solutions
#
# 1. Fit a cubic to the data
x = c(10, 11, 12, 13, 14, 15)
y = c(18, 16, 17, 19, 21, 30)
plot(x,y)
#by viewing yhat=yh = b0+b1x+b2x^2+b3x^3 as the product of the matrix X=[1, x,
#x^2, x^3] with the column vector b=[b0, b1, b2, b3], i.e. yh = Xb, so X'yh=X'Xb
#(X' = X transposed, = t(X)), so b= (X'X)^-1 X yh. Assume that a best estimate of
# b is obtained from this expression for b when yh is replaced by y.
# In the end, use lines(x,yh) to show a reasonable fit
# 2. Using the design matrix X produced in problem 1 above
# a) Compute the hat operator matrix H = X(X^TX)^-1X^T which maps y to yh
# b) H is a projection operator, so it leaves components of y that are in the
# range space of X as is, and it takes to zero components of y that are
# orthogonal to this range space, i.e. components that lie in the error space.
# Use eigen(H) to look at the eigenvalues of H and explain why they are
# consistent with a 4-dimensional model space and a 2-dimensional error space
# (note that y is a 6-dimensional data vector). Projection operators are
# matrices for which the eigenvalues are all 0's or 1's.
# c) verify, via HX, that the columns of X are eigenvectors of H with eigenvalues of 1.
# d) The operator I-H projects y onto the error space vector e.
# Show that e=(I-H)y is an eigenvector of H with an eigen value of 0
# (you can approximate numbers less than 1e-6 as zero).