Test 1 Review

• There exists a $\xi$ in $[a,b]$ with $f(\xi) = 0$.

ANS: TRUE, IVT

• There exists a $\xi$ in $[a,b]$ with $f'(\xi) = 0$.

ANS: FALSE, smells like mean value theorem, but isn't

• There exists a $\xi$ in $[a,b]$ with $f'(\xi) \cdot (b-a) = f(b) - f(c)$.

ANS: FALSE, though if we assumed $f'(x)$ exists, then this is the mean value theorem.

What are the assumptions on $f$ used in your proof?

ANS: Did this in class assuming $C^2$, and using the following to solve for $f'(x)$:

We needed a very large value of $k$. What if we tried this over a smaller interval, say $0 \leq \xi \leq 1/2$, instead? How big would $k$ need to be then.

We used $f^{(k)}(x) = \pm 1 / (k (1+x)^k)$.

ANS: We plug in and need to bound:

Taking $x=1/2$, we can bound this with

Solving with the computer we get 13, as $2^{-53} = 1.11\dots \cdot 10^{-16}$ and:

f(k) = 1/(k+1)/factorial(k+1) * (1/2)^(k+1)
xs = 1:15
ys = map(f, xs)
[xs ys]

15×2 Array{Float64,2}:
  1.0  0.0625                
  2.0  0.006944444444444444  
  3.0  0.0006510416666666666 
  4.0  5.208333333333334e-5  
  5.0  3.616898148148148e-6  
  6.0  2.2144274376417232e-7 
  7.0  1.2110150049603175e-8 
  8.0  5.980321012149716e-10 
  9.0  2.691144455467372e-11 
 10.0  1.1120431634162695e-12
 11.0  4.2473870824926955e-14
 12.0  1.5079480766246258e-15
 13.0  5.0008482132959525e-17
 14.0  1.5558194441365185e-18
 15.0  4.558064777743707e-20 

Chapter 2

Some sample problems

• If $1 = 1.00 \cdot 10^2$. What is $\epsilon$?

ANS: 1.01 - 1.00 = .01 = 10^(-2). (This is a different epsilon than in the book, which is half of this.)

• What is $3.14 \cdot 10^0 - 3.15 \cdot 10^0$?

ANS: 1.00 10^(-2). (The 0's come from needing to pad things out.)

• What is $4.00 \cdot 10^0$ times $3.00 \cdot 10^1$?

ANS: 12 * 10^(0+1) = 1.20 * 10^2

• What is $\delta$ (where $fl(x \cdot y) = (x\cdot y)\cdot (1\cdot\delta)$) when computing $1.23 \cdot 10^4$ times $4.32 \cdot 10^1$?

ANS: Here x*y = 531360.0 = 5.31360 * 10^5 And fl(x*y) = 5.31 * 10^5. The value of $\delta$ then would satisfy: $1 + \delta = 5.31 / 5.31360$, or $-0.00068$.

• How many total numbers are representable in this form ($0$ is not)?

ANS: we have $pp$ can be 00, 01, 10, or 11 and m is only one of 4 values and $\pm$ one of 2, so the answer is $2\cdot 4\cdot 4$.

• What is $\epsilon$?

ANS =1+ = 1.01 * 2^0, so 1+ - 1 is $2^{-2}$. Again, the book's epsilon is 1/2 this amount.

• what is $1.11 \cdot 2^1 - 1.00 \cdot 2^0$?

ANS: 1.11 * 2^1 - 0.10 * 2^1 = 1.01 * 2

• Convert the number $-1.01 \cdot 2^{-2}$ to decimal.

ANS:

-(1 + 0*(1/2) + 1 *(1/4)) * 1/2^2

-0.3125

• Let $x=1.11 \cdot 2^0$ and $y=1.11 \cdot 2^1.$ Find $\delta$ in

ANS: 1.11 = 1 + 1/2 + 1/4, 1.11*2 = (1 + 1/2 + 1/4)*2, so

The exact different is $-7/4$. The rounded difference is (0.11 - 1.11)*2 = 2 = 8/4. The delta value then solve 7/4 = 8/4(1+delta), or delta=-1/8. This is less than or equal $2^{-2}/2$

ANS: We have the sign bit is 0, so the number is postive. The significand is $1 + 1/4 + 1/16 = 1.3125$. The exponent is 10100 - 01111 is just $(16 +4) - (8 + 4 + 2 + 1) = 5$. The significand is $1 + 1/4 + 1/16$. So all told, we get:

2^5 * ( 1 + 1/4 + 1/16)

42.0

ANS: Horners method is:

x = 2
((1*x + 2)*x + 3)*x + 1

23

Can you think of why the direct approach might cause issues for some values of $x$ in that range?

ANS: Near $x=0$ we have subtraction of like-sized quantities. This is a possible source of error. The new expression doesn't involve that. It has issues near $x=0$ that are addressed by using expm.

• $\log(x) - \log(y)$

ANS: Use $\log(x/y)$ to avoid issues if $x$ and $y$ are close

• $x^{-3} (\sin(x) - x)$

ANS: Using taylor, we see if $xS$ is close to $0$ that this is just $x^{-3}\cdot(-x^3/3!) = 1/6$.

• $\sin(x) - \tan(x)$

  .

ANS: The issue is near $0$, we have the difference in tangent line expressions is $x - x^3/3! + \mathcal{O}(x^5)$ and $x - x^3/3 + \mathcal{O}(x^5)$ so near $0$ we could use $-x^3/2$.

ANS: We had the theorem in class that said if $2^{-q} \leq 1 - y/x$ than at most $q$ binary bits are lost. So we have to solve for $x$ which makes $2^{-2} = 1 - 1/\sqrt{x^2 + 1}$. Being lazy, we have

using SymPy
u  = symbols("u")
solve(1 - 1/sqrt(u^2 + 1) - 1/4, u)

What value of $k$ will ensure that the error over $[0, 1/4]$ is no more than $10^{-3}$?

ANS: The error term is the last term:

That is, floating point multiplication is not associative. You can verify by testing (0.1 * 0.2)*0.3 and 0.1 * (0.2 * 0.3).

ANS: Suppose $x$, $y$ and $z$ are floating point numbers. Then $fl(xy) = xy(1+\delta_1)$ and $fl(yz) = yx (1 + \delta_2)$ where both delta's are small but need not be the same. So the left hand side is:

That is, if you computed the difference quotient, $(f(x+h)-f(x))/h$ in floating would you expect smaller and smaller values of $h$ would converge. Why?

ANS: NO! We get $fl(f(x+h))$ is basically $f(x+h)(1+\delta)$ and $fl(f(x)) = f(x)(1+\delta_2)$. So all told, the difference in the top is

ANS: using the derivative, $[\log(y(x))]' = y'(x)/y(x)$ we get from a first order taylor expansion:

But $y'(x) \approx (y(x+h) - y(x))/h$, so the above becomes:

Chapter 3 – solving f(x) = 0

Some sample problems

• Let $f(x) = x^2 - 2$. Starting with $a_0, b_0 = 1, 2$, find $a_4,b_4$.

ANS: This is for the bisection method:

We can see that

a0, b0 = 1//1, 2//1
c0 = (a0 + b0)/2 ## f(c0) >0

a1,b1 = a0, c0
c1 =  (a1 + b1)/2 ## f(c1) < 0

a2, b2 = c1, b1
c2 =  (a2 + b2)/2 ## f(c1) < 0

a3, b3 = c2, b2
c3 =  (a3 + b3)/2 ## f(c1) > 0

a4, b4 = a3, c3
c4 =  (a3 + b3)/2

a4,b4, c4,  abs(sqrt(2) - c4) <= 1/2^5*(b0 - a0)

(11//8, 23//16, 23//16, true)

• Let $e_n$ be $c_n - c$. The order of convergence of $c_n$ is $q$ provided

Using the bound above, what is the obvious guess for the order of convergence?

ANS: We have $(b_n - a_n) / (b_{n-1} - a_{n-1}) = 1/2$, so we expect $c_{n+1}/c_n$ to be around 1/2 too. That is linear convergence.

• Explain why the bisection method is no help in finding the zeros of

ANS: the function doesn't cross $0$, so we can't find $a_0$, $b_0$.

• In floating point, the computation of the midpoint via $(a+b)/2$ is

discouraged and using $a + (b-a)/2$ is suggested. Why?

ANS: the errors follow $fl(a+b) = (a+b)(1+\delta)$, so if $a$ and $b$ are big, then the error $fl(a+b) - (a+b) = (a+b)(1+\delta)$ is bigger. This is most dramatic with overflow.

• Mathematically if $a < b$, it is always the case that there exists a $c = (a+b)/2$ and $a < c < b$. Is this also always the case in floating point? Can you think of an example of when it wouldn't be?

ANS. No. If $a=1^+$ and $b=1$, then we can't fit a value in between.

• To compute $\pi$ as a solution to $\sin(x) = 0$, one might use the bisection method with $a_0, b_0 = 3,4$. Were you to do so, how many steps would it take to find an error of no more than $10^{-16}$?

ANS: We saw that we need to solve for $n$ with

This gave:

Or in this case

p = 16
ceil(p * log(10)/log(2) - 1)

53.0

• A simple zero for a function $f(x)$ is one where $f'(x) \neq 0$. Some algorithms have different convergence properties for functions with only simple zeros as compared to those with non-simple zeros. Would the bisection algorithm have a difference?

ANS: Well, yes and no. The error bound doesn't depend on $f'(x)$ so the answer is no. However, when a zero is not simple, the function may not cross the $x$ axis. That can be an issue.

• If you answered yes above, you could still be right, even though you'd be wrong mathematically (Why? look at the bound on the error and the assumptions on $f$.). This is because for functions with non simple zeros, you can have a lot of numeric issues creep in. The book gives an example of the function lie $f(x) = (x-1)^5$. Explain what is going on with this graph near $x=1$:

using Plots
f(x) = x^5 - 5x^4 +10x^3 -10x^2 + 5x -1
plot(f, 0.999, 1.001)

• For $f(x) = x^2 - 2$, and $x_0 = 1$ and $x_1 = 1.5$ compute 3 steps

of a) the bisection method, b) Newton's method, c) the secant method

ANS: The floating point computation has a lot of error, as we aren't doing it exactly (or efficiently). Roughly we have

Suppose $x<1$ but close to $1$. When $f(x)$ is near $0$, the error term – which may be positive or negative – can push the floating point value above the $x$ axis even though the mathematical part, $f(x)<0$.

• We have $f(x) = \sin(x)$ has $[3,4]$ as a bracketing interval. Give a bound on the error $c_n - r$ after 10 steps of the bisection method.

ANS: Well, $n=10$ has $c_{10} = 1/2 (b_{10}-a_{10}) = 1/2 \cdot 2^{-10}(b_0-a_0)=2^{-11}$.

• We have $f(x) = x^2 - s$ has a solution $\sqrt{s}$, $s > 0$. Compute $1/2 \cdot f''(\xi)/f'(x_0)$ for $x_0 = s$. Compute the error, $e_1$.

ANS: We have $f'(x) = 2s$, $f''(x) = 2$, so $1/2\cdot 2/(2s) = 1/(2s)$. $e_1 = 1/(2s)*e_0^2 = 1/(2s)*(s-\sqrt(s))^2$.

• For $f(x) = \sin(x)$, find an interval $[-\delta, \delta]$ for which the

newton iterates will converge quadratically to $0$.

ANS: We need to solve $\delta \cdot C(\delta) < 1$, where $C(\delta) = 1/2 \cdot max(|\sin(x)) / min(|\cos(y)|)$. We note:

This uses $\cos(x) > 1 - x^2/2$ near $0$. Solving then

We get $\delta^2 + \delta - 2$ which is solved with $\delta =1$. So any $0 < \delta < 1$, should work.

• Newton's method is applied to the function $f(x) = \log(x) - s$ to find $e^s$. If $x_n < e^s$ show $x_{n+1} < e^s$. If $e_0 > 0$ yet $x_1 > 0$, show $e_1 < 0$. (mirror the proof of one of the theorems)

ANS: This is $f'>0$ and $f''< 0$ so we have

so $e_1$, $e_2$, $\dots$ are all negative meaning, $e_i < r$ for $i \geq 1$. Since $f'>0$, and $f(x_i) < 0$ when $i \geq 1$, we must have $f(x_i)$ are increasing for $i\geq 1$.