Scribd Logo
Upload

Welcome to Scribd!

  • 8 views

    Sec 4-7

    Uploaded by

    Florin
    The document describes the least squares approximation method. It seeks to find a polynomial approximation p(x) that minimizes the average error over an interval by minimizing the integral of the squared difference between the function f(x) and the approximation p(x). This results in a linear system of equations whose solution gives the coefficients of p(x) that provide the best least squares approximation of f(x). The optimal approximation is expanded in terms of Legendre polynomials.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Sec 4-7

    Uploaded by

    Florin
    8 views15 pages
    The document describes the least squares approximation method. It seeks to find a polynomial approximation p(x) that minimizes the average error over an interval by minimizing the integral of the squared difference between the function f(x) and the approximation p(x). This results in a linear system of equations whose solution gives the coefficients of p(x) that provide the best least squares approximation of f(x). The optimal approximation is expanded in terms of Legendre polynomials.

    Original Description:

    Aerth

    Original Title

    sec_4-7

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document describes the least squares approximation method. It seeks to find a polynomial approximation p(x) that minimizes the average error over an interval by minimizing the integral of the squared difference between the function f(x) and the approximation p(x). This results in a linear system of equations whose solution gives the coefficients of p(x) that provide the best least squares approximation of f(x). The optimal approximation is expanded in terms of Legendre polynomials.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    8 views15 pages

    Sec 4-7

    Uploaded by

    Florin
    The document describes the least squares approximation method. It seeks to find a polynomial approximation p(x) that minimizes the average error over an interval by minimizing the integral of the squared difference between the function f(x) and the approximation p(x). This results in a linear system of equations whose solution gives the coefficients of p(x) that provide the best least squares approximation of f(x). The optimal approximation is expanded in terms of Legendre polynomials.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 15

    LEAST SQUARES APPROXIMATION

    Another approach to approximating a function f (x)


    on an interval a ≤ x ≤ b is to seek an approximation
    p(x) with a small ‘average error’ over the interval of
    approximation. A convenient definition of the average
    error of the approximation is given by
    " Z b #1
    1 2
    E(p; f ) ≡ [f (x) − p(x)]2 dx (1)
    b−a a
    This is also called the root-mean-square-error (de-
    noted subsequently by RMSE ) in the approximation
    of f (x) by p(x). Note first that choosing p(x) to
    minimize E(p; f ) is equivalent to minimizing
    Z b
    [f (x) − p(x)]2 dx
    a
    thus dispensing with the square root and multiplying
    fraction (although the minimums are generally differ-
    ent). The minimizing of (1) is called the least squares
    approximation problem.
    Example. Let f (x) = ex, let p(x) = α0 + α1x, α0,
    α1 unknown. Approximate f (x) over [−1, 1]. Choose
    α0, α1 to minimize
    Z 1
    g(α0, α1) ≡ [ex − α0 − α1x]2 dx (2)
    −1
    Z 1 ( 2x 2 2 2 x
    )
    e + α0 + α1x − 2α0e
    g(α0, α1) = x dx
    −1 −2α1xe + 2α0α1x
    Integrating,

    g(α0, α1) = c1α20 + c2α21 + c3α0α1 + c4α0 + c5α1 + c6


    with constants {c1, . . . , c6}, e.g.
    ³ ´
    c1 = 2, 1
    c6 = e − e−1 /2.
    g is a quadratic polynomial in the two variables α0,
    α1. To find its minimum, solve the system
    ∂g ∂g
    = 0, =0
    ∂α0 ∂α1
    It is simpler to return to (2) to differentiate, obtaining
    Z 1
    2 [ex − α0 − α1x] (−1) dx = 0
    −1
    Z 1
    2 [ex − α0 − α1x] (−x) dx = 0
    −1
    This simplifies to
    Z 1
    2α0 = exdx = e − e−1
    −1
    Z 1
    2
    α1 = xexdx = 2e−1
    3 −1

    e − e−1 .
    α0 = = 1.1752
    2
    .
    α1 = 3e−1 = 1.1036
    Using these values for α0 and α1, we denote the re-
    sulting linear approximation by

    1(x) = α0 + α1x
    It is called the best linear approximation to ex in the
    sense of least squares. For the error,
    .
    max |ex − 1(x)| = 0.439
    −1≤x≤1
    Errors in linear approximations of ex:

    Approximation Max Error RMSE


    Taylor t1(x) 0.718 0.246
    Least squares 1(x) 0.439 0.162
    Chebyshev c1(x) 0.372 0.184
    Minimax m1(x) 0.279 0.190

    x
    y=e

    y=l (x) 1
    1

    x
    -1 1

    The linear least squares approximation to ex


    THE GENERAL CASE

    Approximate f (x) on [a, b], and let n ≥ 0. Seek p(x)


    to minimize the RMSE. Write

    p(x) = α0 + α1x + · · · + αnxn

    Z1 " #2
    f (x) − α0 − α1x
    g(α0, α1, . . . , αn) ≡ dx
    − · · · − αnxn
    −1
    Find coefficients α0, α1, . . . , αn to minimize this in-
    tegral. The integral g(α0, α1, . . . , αn) is a quadratic
    polynomial in the n + 1 variables α0, α1, . . . , αn.

    To minimize g(α0, α1, . . . , αn), invoke the conditions


    ∂g
    = 0, i = 0, 1, . . . , n
    ∂αi
    This yields a set of n+1 equations that must be satis-
    fied by a minimizing set α0, α1, . . . , αn for g. Manip-
    ulating this set of conditions leads to a simultaneous
    linear system.
    To better understand the form of the linear system,
    consider the special case of [a, b] = [0, 1]. Differenti-
    ating g with respect to each αi, we obtain
    Z 1
    2 [ex − α0 − · · · − αnxn] (−1) dx = 0
    −1
    Z 1
    2 [ex − α0 − · · · − αnxn] (−x) dx = 0
    −1
    ..
    Z 1
    2 [ex − α0 − · · · − αnxn] (−xn) dx = 0
    −1
    Then the linear system is
    n
    X Z 1
    αj
    = xif (x) dx, i = 0, 1, . . . , n
    j=0 i + j + 1 0

    We will study the solution of simultaneous linear sys-


    tems in Chapter 6. There we will see that this linear
    system is ‘ill-conditioned’ and is difficult to solve ac-
    curately, even for moderately sized values of n such as
    n = 5. As a consequence, this is not a good approach
    to solving for a minimizer of g(α0, α1, . . . , αn).
    LEGENDRE POLYNOMIALS

    Define the Legendre polynomials as follows.

    P0(x) = 1

    1 dn h³ 2 ´ni
    Pn(x) = n
    · n x −1 , n = 1, 2, . . .
    n!2 dx
    For example,

    P1(x) = x
    1³ 2 ´
    P2(x) = 3x − 1
    2
    1³ 3 ´
    P3(x) = 5x − 3x
    2
    1³ 4 2
    ´
    P4(x) = 35x − 30x + 3
    8
    The Legendre polynomials have many special proper-
    ties, and they are widely used in numerical analysis
    and applied mathematics.
    y

    x
    -1 1

    P (x)
    1
    P (x)
    2
    P (x)
    3
    P (x)
    4

    -1

    Legendre polynomials of degrees 1, 2, 3, 4


    PROPERTIES

    Introduce the special notation


    Z b
    (f, g) = f (x)g(x) dx
    a
    for general functions f (x) and g(x).

    • Degree and normalization:

    deg Pn = n, Pn(1) = 1, n≥0

    • Triple recursion relation: For n ≥ 1,


    2n + 1 n
    Pn+1(x) = xPn(x) − Pn−1(x)
    n+1 n+1

    • Orthogonality and size:



    ³ ´ 
    0,
     i 6= j
    Pi, Pj = 2

     , i=j
    2j + 1
    • Zeroes:
    All zeroes of Pn(x) are located in [−1, 1] ;
    all zeroes are simple roots of Pn(x)

    • Basis: Every polynomial p(x) of degree ≤ n can


    be written in the form
    n
    X
    p(x) = β j Pj (x)
    j=0
    with the choice of β 0, β 1, . . . , β n uniquely deter-
    mined from p(x):
    ³ ´
    p, Pj
    βj = ³ ´, j = 0, 1, . . . , n
    Pj , Pj
    FINDING THE LEAST SQUARES
    APPROXIMATION

    We solve the least squares approximation problem on


    only the interval [−1, 1]. Approximation problems on
    other intervals [a, b] can be accomplished using a lin-
    ear change of variable.

    We seek to find a polynomial p(x) of degree n that


    minimizes
    Z b
    [f (x) − p(x)]2 dx
    a
    This is equivalent to minimizing

    (f − p, f − p) (3)
    We begin by writing p(x) in the form
    n
    X
    p(x) = β j Pj (x)
    j=0
    n
    X
    p(x) = β j Pj (x)
    j=0
    Substitute into (3), obtaining
    ge (β 0
    , β 1, . . . , β n) ≡ (f − p, f − p)

    n
    X n
    X
    = f − β j Pj , f − β iPi
    j=0 i=0
    Expand this into the following:
    n (f, P )2
    X j
    ge = (f, f ) −
    j=0 (Pj , Pj )
    n ³ " #2
    X ´ (f, Pj )
    + Pj , Pj β j −
    j=0 (Pj , Pj )
    Looking at this carefully, we see that it is smallest
    when
    (f, Pj )
    βj = , j = 0, 1, . . . , n
    (Pj , Pj )
    Looking at this carefully, we see that it is smallest
    when
    (f, Pj )
    βj = , j = 0, 1, . . . , n
    (Pj , Pj )
    The minimum for this choice of coefficients is
    n (f, P )2
    X j
    ge = (f, f ) −
    j=0 (Pj , Pj )
    We call
    n (f, P )
    X j
    n(x) = Pj (x) (4)
    j=0 (Pj , Pj )
    the least squares approximation of degree n to f (x)
    on [−1, 1].

    If β n = 0, then its actual degree is less than n.


    Example. Approximate f (x) = ex on [−1, 1]. We use
    (4) with n = 3:
    3
    X (f, Pj )
    3(x) = β j Pj (x), βj = (5)
    j=0 (Pj , Pj )
    The coefficients {β 0, β 1, β 2, β 3} are as follows.

    j 0 1 2 3
    βj 2.35040 0.73576 0.14313 0.02013

    Using (5) and simplifying,


    2 3
    3(x) = .996294 + .997955x + .536722x + .176139x

    The error in various cubic approximations:

    Approximation Max Error RMSE


    Taylor t3(x) .0516 .0145
    Least squares 3(x) .0112 .00334
    Chebyshev c3(x) .00666 .00384
    Minimax m3(x) .00553 .00388
    y

    0.0112

    x
    -1 1

    -0.00460

    Error in the cubic least squares approximation to ex

    You might also like