SLIDE 01: Convexity and unconstrained optimization

Welcome to our first session on nonlinear programming. Today we connect three things: the shape of a problem, the information provided by derivatives, and the steps taken by an optimization algorithm. Look at the two surfaces. The convex bowl has a simple global structure. The nonconvex surface has a different geometry, so a numerical stopping point needs more interpretation. We will calculate small examples by hand, then reproduce those calculations in Python. Keep a notebook nearby. At the practice slides, pause the narration and try the calculation before moving to the answer.

SLIDE 02: Learning goals

By the end of this session, you should be able to explain three ideas. First, decide whether a set or function is convex, using a definition or a valid test. Second, calculate gradient descent and Newton steps, rather than treating the solver as a black box. Third, check what a computed result means. A small gradient tells us about stationarity; by itself, it does not always identify a minimum. We will combine the objective value, the gradient, and curvature information to make a defensible conclusion.

SLIDE 03: The three-hour session

This is a three-hour learning session, including practice and a break. Spend about sixty minutes on convexity and the first exercises. Then take a ten-minute break. The next seventy-five minutes cover unconstrained optimization, including hand calculations. Reserve the final thirty-five minutes for the Python lab, independent practice, and closing questions. The audio itself is shorter than the full session. Pause to draw a figure, calculate an update, or run a notebook cell whenever prompted. Learning the method requires doing those steps yourself.

SLIDE 04: The NLP problem

Nonlinear programming includes a nonlinear objective or nonlinear constraint. Our main focus today is unconstrained minimization: choosing continuous variables to make a scalar objective as small as possible, with no additional equality or inequality constraints. We still respect the function's domain. For example, a logarithm requires a positive argument. Convexity is a separate issue from how many variables we have. A one-variable function can be nonconvex, and a function with many variables can be convex. The surfaces illustrate geometry, not a distinction between small and large problems.

SLIDE 05: Convex sets

A set is convex when the entire straight segment between any two points in the set remains inside the set. The expression for z describes a point on that segment. Theta ranges from zero to one, so we take a weighted average of the endpoints. In the shaded disk, the amber segment stays inside. However, checking just this one segment does not prove the definition for every pair. General results help: half-spaces and balls are convex, and an intersection of convex sets is convex. We will use that intersection rule in the next exercise.

SLIDE 06: A nonconvex feasible set

Here the feasible set is a ring. The squared distance from the origin must be at least one and at most four. Equivalently, the radius is between one and two. Both endpoints shown have radius one point five, so both are feasible. Their midpoint is the origin, whose squared distance is zero. That violates the lower bound. The straight segment passes through the hole, so the set is not convex. Notice the logic: one valid counterexample is enough to disprove convexity, even though many other segments may remain feasible.

SLIDE 07: Set exercise

Try these two sets before proceeding. The first is the shaded triangle: both coordinates are nonnegative, and their sum is at most four. The second contains points whose squared distance from the origin is at least one. For each set, either explain why every segment remains feasible, or find two feasible endpoints whose midpoint is infeasible. Do not answer only from the appearance of a drawing. For the triangle, think about intersections of half-spaces. For the second set, try points on opposite sides of the unit circle. Pause here and write your reasoning.

SLIDE 08: Set exercise: solution

The first set is convex because it is the intersection of three half-spaces. Each of its inequalities is affine. The second set is not convex. Choose the points minus one, zero, and one, zero. Each has squared distance one, so both satisfy the constraint. Their midpoint is zero, zero, which has squared distance zero and violates the constraint. Be careful with the inequality sign. Squared distance less than or equal to one describes a convex disk. Squared distance greater than or equal to one describes the exterior, and the midpoint counterexample applies.

SLIDE 09: A convex function and its chord

Now move from convex sets to convex functions. Choose two points on a function's graph and connect them with a straight chord. For a convex function, the graph is on or below that chord between the endpoints. The teal curve shows the function, and the amber line shows the chord. The input halfway between the endpoints has a function value below the corresponding height on the chord. This must hold for every pair of inputs and every weight between zero and one. The function's domain must itself be convex, so those intermediate inputs are available.

SLIDE 10: The convexity inequality

Read the inequality in two stages. On the left, first average the inputs using theta and one minus theta, then evaluate the function. On the right, first evaluate the function at each endpoint, then average those two values using the same weights. Convexity says that the first result is no larger than the second. These operations are different: the function of an average is generally not the average of the function values. In the figure, the teal midpoint value represents the left side, while the higher amber point represents the right side.

SLIDE 11: Example: f(x)=x²

Let us substitute actual numbers. The function is x squared, and the endpoints are minus one and three. With equal weights, the midpoint input is one. Evaluating the function there gives one. At the endpoints, the function values are one and nine. Their average is five. Therefore, the midpoint value, one, is below the average endpoint value, five. The vertical dashed segment shows that difference. This example illustrates the inequality. A proof that x squared is convex everywhere comes from its second derivative, which is the positive constant two.

SLIDE 12: A counterexample for nonconvexity

Consider q of x, equal to the square of the quantity x squared minus one. At minus one and plus one, the expression inside the outer square is zero, so both function values are zero. Their midpoint is zero, but q of zero is one. The convexity inequality would require one to be less than or equal to zero. That is false. Therefore, this function is nonconvex on the real line. It is smooth and nonnegative, but neither of those properties implies convexity. The two wells will also help us understand numerical algorithms later.

SLIDE 13: Convex, strictly convex, and strongly convex

Convex, strictly convex, and strongly convex are different statements. Convexity allows the graph to touch a chord between distinct endpoints. Strict convexity requires a strict inequality at every interior point of such a chord. Strong convexity adds a uniform lower curvature bound. For a twice continuously differentiable function on an open convex domain, the Hessian must be bounded below by m times the identity, for one fixed positive number m. Compare the two graphs. X squared has curvature two everywhere. X to the fourth is strictly convex, but its curvature is zero at the origin, so it is not strongly convex on the whole real line.

SLIDE 14: The Hessian test

The Hessian criterion has assumptions that we must keep. Suppose the function is twice continuously differentiable on an open convex domain. Then it is convex exactly when its Hessian is positive semidefinite at every point in that domain. The words every point matter. In the double-well example, the second derivative is positive near the two minima but negative near the middle. The shaded region highlights that negative curvature. Testing the Hessian only at a favorable point would miss it. A local curvature check and a global convexity claim are not interchangeable.

SLIDE 15: Reading a symmetric Hessian

For a symmetric Hessian, positive semidefinite means that d transpose H d is nonnegative for every direction d. This is a statement about directional curvature, not about whether every matrix entry is nonnegative. Equivalently, all eigenvalues must be nonnegative. The saddle surface illustrates what mixed eigenvalue signs mean: the surface rises in one direction and falls in another. For the displayed function, x squared minus y squared, the Hessian has diagonal entries two and minus two. Its eigenvalues have opposite signs, so this Hessian is indefinite, not positive semidefinite.

SLIDE 16: Example: a convex quadratic

This quadratic will be our main example. The objective is u minus one, squared, plus four times v plus two, squared. Differentiating twice gives a diagonal Hessian with entries two and eight. The matrix is constant, and both eigenvalues are positive. Consequently, this function is strongly convex everywhere on the two-dimensional real space. In the surface plot, the horizontal coordinates are shifted errors: u minus one and v plus two. The bottom of the bowl corresponds to zero error in both coordinates. The larger weight in the v term creates stronger curvature in that direction.

SLIDE 17: A global certificate from the gradient

A differentiable convex function lies above each of its tangent planes. That is the first inequality on the slide. The right side is the linear approximation at x, evaluated at another point y. Now suppose the gradient at x star is zero. The linear correction disappears, leaving f of y greater than or equal to f of x star for every point in the domain. This certifies a global minimum. Strict convexity additionally gives at most one minimizer. It does not, by itself, guarantee that a minimum is attained; we will separate those ideas shortly.

SLIDE 18: Why local minima are global in a convex problem

Here is the geometric reason a local minimum of a convex problem must be global. Suppose x star is a local minimum, but another feasible point y has a lower objective. Move a small positive fraction of the way from x star toward y. Convexity of the feasible set keeps that new point feasible. Convexity of the objective puts its value below the weighted average of the endpoint values, which is lower than the value at x star. We can make this move arbitrarily small. That contradicts local minimality. The proof needs both a convex objective and a convex feasible set.

SLIDE 19: Existence and uniqueness are separate

Uniqueness and existence answer different questions. Strict convexity says there cannot be two distinct minimizers, if a minimizer exists. It does not promise that a minimum is attained. Consider the exponential function on the real line. It is strictly convex and always positive. As x goes farther left, the value approaches zero, but no finite x makes it exactly zero. Thus zero is the infimum, or greatest lower bound, but there is no minimizer on the real line. Before claiming a unique solution, first establish that a solution exists.

SLIDE 20: Convex optimization problems

A convex minimization problem combines a convex objective with a convex feasible set. One standard way to describe that set uses convex inequality functions constrained to be less than or equal to zero, together with affine equalities. The direction of the inequality matters. A convex function constrained to be greater than or equal to zero does not generally produce a convex set. Linear programming is a special case of convex optimization. Integer restrictions in a mixed-integer model generally remove that convex feasible-set structure. For maximization, the corresponding favorable case uses a concave objective over a convex feasible set.

SLIDE 21: Classification exercise

Classify these three minimization problems on the entire real line. For each one, answer separately: is the objective convex, is the minimum attained, and is the minimizer unique? The first objective is x to the fourth. The second is minus x squared. The third is the double-well function. You may use derivatives, the chord test, or the fact that a square is nonnegative. Remember that having a stationary point does not prove that a minimum exists. Pause the narration and write three short conclusions before opening the solution.

SLIDE 22: Classification exercise: solution

For the first problem, x to the fourth is strictly convex and nonnegative. It equals zero only at the origin, so there is one global minimizer. For the second problem, minus x squared is concave and becomes arbitrarily negative as the magnitude of x grows. There is no global minimum. The stationary point at zero is a maximum. For the third problem, the midpoint test already showed nonconvexity. Nevertheless, the objective is a square, so its lower bound is zero, attained at both minus one and plus one. Nonconvexity does not prevent global minima from existing. This completes the first block; take a ten-minute break before continuing.

SLIDE 23: Derivatives and local change

We now turn from problem geometry to numerical steps. The gradient collects first partial derivatives. For a small displacement p, the first-order change in the objective is approximately the inner product of the gradient and p. The Hessian supplies the second-order correction: one half times p transpose H p. In the figure, the straight line is the first-order model near the current point. It agrees with the function's value and slope there, but becomes less accurate farther away. For a quadratic function, the second-order Taylor model is exact; for a general nonlinear function, it is an approximation.

SLIDE 24: Stationary points and curvature

Separate necessary conditions from sufficient conditions. At an interior local minimum of a differentiable unconstrained function, the gradient must be zero. If the function is twice continuously differentiable, its Hessian there must also be positive semidefinite. Those are necessary conditions. A stronger statement gives sufficiency: at a stationary point, a positive-definite Hessian guarantees a strict local minimum. This is a local result unless additional information, such as convexity over the whole domain, gives a global conclusion. A positive-semidefinite Hessian with zero eigenvalues may be inconclusive, as we will see.

SLIDE 25: A stationary saddle point

The function on this slide is exactly x squared minus y squared. At the origin, both first derivatives vanish, so the origin is stationary. Along the horizontal slice where y is zero, the function becomes x squared, which is positive away from the origin. Along the slice where x is zero, the function becomes minus y squared, which is negative away from the origin. Thus arbitrarily close to the origin, we can find both higher and lower values. The point is a saddle. The negative sign in the second slice comes directly from the original function's minus y squared term.

SLIDE 26: A zero Hessian can be inconclusive

At the origin, x to the fourth and minus x to the fourth have the same first two derivative values: both are zero. Yet their behavior is opposite. The first function is nonnegative and equals zero at the origin, giving a strict minimum. The second is nonpositive and equals zero there, giving a strict maximum. Therefore, a zero Hessian does not classify a stationary point. It means the second-order test has not settled the question. We must inspect higher-order behavior or use a direct argument about nearby function values.

SLIDE 27: The shared quadratic example

Write down the main example carefully before computing. Our variable order is u, then v. The gradient's first component is twice u minus one, and the second component is eight times v plus two. The Hessian is diagonal, with two and eight on the diagonal and zeros elsewhere. We start at u equals three and v equals zero. We will use exactly this same objective and initial point for gradient descent and Newton, so differences in the paths come from the methods, rather than from changing the problem.

SLIDE 28: The exact solution as a reference

We can solve the main example exactly and use that solution as a reference. Set the two gradient components to zero. Twice u minus one equals zero gives u equals one. Eight times v plus two equals zero gives v equals minus two. Substitution into the objective gives zero. There is also an independent check: both terms in the objective are nonnegative squares, so no value below zero is possible. Both terms vanish only at this point. That establishes the unique global minimum without relying on a numerical solver's output.

SLIDE 29: The negative gradient direction

At the starting point, three, zero, the gradient is four, sixteen. The negative gradient direction is therefore minus four, minus sixteen. The gradient points toward increasing objective values; reversing it gives a local descent direction. The arrows in the contour plot are scaled for visibility, so their drawn lengths are not full algorithmic steps. Also notice that the gradient is perpendicular to the local contour when both coordinate axes use the same scale. Choosing a direction is only the first decision. We still need to choose how far to move along it.

SLIDE 30: Why −g is a descent direction

To check a direction p, compute the directional derivative: g transpose p. If it is negative, sufficiently small positive steps reduce the objective. With p equal to minus g, this quantity is minus the squared norm of the gradient. At our starting point, the calculation is minus sixteen minus two hundred fifty-six, which equals minus two hundred seventy-two. The curve on the right shows the objective along this direction as the step size changes. It initially falls, but eventually rises. A descent direction does not mean that every possible positive step length is safe.

SLIDE 31: The iterative update

An iterative update has two separate ingredients. Start with the current vector, then add a positive step size times the chosen direction. The subscript k labels the iteration; it is not an exponent. In gradient descent, the direction is the negative gradient at the current point. In a full Newton step, the direction comes from a linear system and the step size is one. Today we will examine fixed step sizes and selected full Newton steps. Next session we will learn how to choose step sizes automatically instead of selecting them in advance.

SLIDE 32: Gradient descent: first step

Let us calculate the first gradient descent step slowly. At three, zero, the objective is four plus sixteen, giving twenty. The gradient is four, sixteen. Multiply this vector by the step size, zero point one, to get zero point four, one point six. Subtract it from the current point. The new point is two point six, minus one point six. Now evaluate the objective again. The u error is one point six, while the v error is positive zero point four. Squaring and weighting gives two point five six plus zero point six four, or three point two. The new objective is much lower than twenty.

SLIDE 33: Gradient descent: second step

For the second step, recompute the gradient at the new point. Do not reuse the original gradient. At two point six, minus one point six, the gradient is three point two, three point two. Multiplying by zero point one gives zero point three two in each coordinate. Subtracting gives two point two eight, minus one point nine two. The new errors are one point two eight and zero point zero eight. Their contributions to the objective are one point six three eight four and zero point zero two five six. Adding gives one point six six four. The value has decreased again, but the gradient is not yet zero.

SLIDE 34: The gradient descent path

The path explains the sequence of numbers we just calculated. With step size zero point one, the v error contracts rapidly, while the u error contracts more slowly. The first move is therefore steep in the plot, and later moves become mostly horizontal as the method approaches one, minus two. For this strongly convex quadratic, we know the limiting point is the unique global minimum. In a general nonconvex problem, a decreasing sequence of objective values alone would not justify that classification. We still inspect the final gradient, curvature, and termination reason.

SLIDE 35: Step size changes the behavior

Changing only the step size can change the outcome completely. For this particular quadratic, convergence from every starting point requires a positive step size below zero point two five. At exactly zero point two five, the v error changes sign but does not shrink. The objective approaches a positive level instead of zero for our starting point. At zero point three, a nonzero v error grows in magnitude and the method diverges. These numbers are specific to the curvature of this example. We will derive them next, rather than treating one quarter as a universal rule.

SLIDE 36: Derivation of the step-size interval

Define the errors as u minus one and v plus two. Substituting the gradient update into those definitions gives simple recurrences. The next u error is one minus twice alpha, multiplied by the current u error. The next v error is one minus eight alpha, multiplied by the current v error. Each geometric sequence approaches zero from every initial value when the absolute value of its multiplier is below one. The first condition gives alpha between zero and one. The second gives alpha between zero and one quarter. Their intersection is the required interval. The endpoints are excluded because an absolute multiplier of one does not shrink the error.

SLIDE 37: Stopping and checking a result

A useful stopping rule should report why the algorithm stopped. In the lab, we check the gradient norm, limit the number of iterations, and guard against nonfinite or excessively large values. A small change in x is not enough by itself: an extremely small step size can make the algorithm barely move even when the gradient is large. Record the final point, objective value, gradient norm, and stopping reason. For nonconvex examples, also inspect curvature. The decreasing curve is useful evidence of progress, but it does not replace these checks or a mathematical classification.

SLIDE 38: Practice: a larger gradient step

Now calculate one step yourself with a larger step size, zero point two. Keep the starting point at three, zero, and use the same starting gradient, four, sixteen. First multiply the gradient by the step size. Then subtract component by component. Finally, substitute your new coordinates into the original objective. Pay special attention to the quantity v plus two before squaring. Compare the new coordinates with the exact minimum, one, minus two. Did a coordinate pass to the other side of its optimal value? Pause here and complete the arithmetic before continuing.

SLIDE 39: Practice solution: α=0.2

Multiplying the gradient by zero point two gives zero point eight, three point two. Subtracting from three, zero, produces two point two, minus three point two. The new errors are positive one point two and negative one point two. Their squared contributions are one point four four and five point seven six, totaling seven point two. The v coordinate has crossed minus two, but the objective has still fallen from twenty. Crossing the minimizer in one coordinate does not necessarily mean failure. Also notice that this larger step gives less improvement on the first iteration than the earlier step size of zero point one.

SLIDE 40: The Newton quadratic model

Newton's method uses a quadratic model of the objective around the current point. The model contains the current value, a linear term from the gradient, and a quadratic term from the Hessian. Differentiate the model with respect to the displacement p. Setting that derivative to zero gives g plus H p equals zero. Thus we solve H p equals minus g. When the Hessian is positive definite, this stationary point minimizes the quadratic model. Otherwise, that interpretation can fail. In the plot, the quadratic model approximates a nonquadratic function locally; its minimizer provides a proposed step, not an automatic guarantee about the full function.

SLIDE 41: Newton direction at the initial point

For our main quadratic, the Newton system is especially simple. At the starting point, the Hessian has diagonal entries two and eight, and the gradient is four, sixteen. The first equation is twice the u component of p equals minus four. The second is eight times the v component equals minus sixteen. Both components of p are minus two. Compare that with the negative gradient, minus four, minus sixteen: Newton accounts for different curvature in the two coordinates. In code, solve the linear system directly rather than explicitly forming the matrix inverse.

SLIDE 42: Newton update and verification

Add the Newton direction, minus two, minus two, to the starting point, three, zero. The result is one, minus two. Evaluating the objective gives zero, and evaluating the gradient gives zero in both components. This is exactly the global minimizer we derived independently. The contour plot shows one straight move to the center. The one-step result is not a general promise of Newton's method. Here it works because the objective is a positive-definite quadratic, whose second-order model is exact everywhere, and we take the full step.

SLIDE 43: When the one-step result applies

We can show the one-step property algebraically. Suppose the objective is one half x transpose H x, plus c transpose x, plus a constant, with H constant and positive definite. Its gradient is H x plus c. The Newton equation is H p equal to minus H x minus c. Rearranging gives H times the quantity x plus p equal to minus c. Therefore, the updated point is exactly the unique stationary point. For a nonquadratic objective, higher-order terms remain outside the local model, so one step generally does not solve the original problem.

SLIDE 44: Newton on a nonquadratic function

Consider r of x equal to x to the fourth plus x squared. Its first derivative is four x cubed plus two x, and its second derivative is twelve x squared plus two. At x equals one, these are six and fourteen. The Newton direction is minus six divided by fourteen, or minus three sevenths. Adding that to one gives four sevenths, approximately zero point five seven one four. The objective falls from two to about zero point four three three one five three. This is an improvement, but the minimum is still at zero. Even strong convexity does not make a nonquadratic function a one-step Newton problem.

SLIDE 45: Newton can move uphill in a nonconvex region

Now see what can go wrong with an unmodified Newton step. For the double-well function at x equals zero point two, the first derivative is minus zero point seven six eight, while the second derivative is minus three point five two. The Newton direction is minus the first derivative divided by the second derivative, giving approximately minus zero point two one eight one eight two. The new point is minus one over fifty-five. The objective rises from zero point nine two one six to about zero point nine nine nine three three nine. The product of gradient and direction is positive, so the direction itself is uphill locally. This is more than a step-size problem.

SLIDE 46: A direction needs a descent check

Before taking a step, inspect the inner product of the gradient and the proposed direction. A negative value indicates local descent. If the Hessian is positive definite and the gradient is nonzero, the Newton direction has a negative inner product with the gradient. If the Hessian lacks that positive-definite structure, raw Newton may produce an ascent direction or an unusable system. Practical methods can modify the curvature model or use a trust region. A line search along an ascent direction is not a substitute for fixing the direction when the method requires descent. We will develop safeguards in the next session.

SLIDE 47: Scaling and elongated contours

Scaling can make a problem much easier or harder for a fixed-step method. This example uses a weight of one hundred in the second squared error, rather than the weight of four in our main example. The Hessian eigenvalues are two and two hundred, so their ratio is one hundred. The contours are long and narrow. One step size must accommodate the steep direction, which can make progress in the flatter direction slow. Define the second scaled coordinate as ten times the second error. In the new coordinates, the objective is a sum of equally weighted squares. When reporting a solution, convert back to the original variables.

SLIDE 48: Gradient descent and Newton

Gradient descent uses the gradient to choose a direction and needs a suitable step size. Its individual steps are simple, but scaling and curvature can strongly affect convergence. Newton uses the Hessian as well and solves a linear system. It can make much more effective moves on suitable problems, but each iteration costs more, and nonconvex curvature requires care. Neither method should be judged only by whether it returned a point. Both need stopping criteria and interpretation. Our examples show why knowing the geometry helps us understand what the algorithm actually did.

SLIDE 49: Python lab: workflow

Open the guided Python notebook now. The lab follows the same numerical examples as the slides, so you can compare outputs directly with your hand calculations. We will define the objective and derivatives, trace gradient descent, change the step size, verify a Newton step, and investigate nonconvex starting points. Run the cells in order. Before each result appears, make a prediction. The executed reference is available for comparison, but try the calculations yourself first. If your answer differs, check variable order, derivative formulas, and whether the gradient was recomputed at the current iterate.

SLIDE 50: Lab 1: functions and derivative checks

Start with the functions in the first notebook section. Represent x as a NumPy array in the order u, v. The objective returns one scalar. The gradient returns an array with two components. The Hessian returns a symmetric two-by-two array. At three, zero, verify the expected values: objective twenty, gradient four, sixteen, and Hessian diagonal two, eight. Next run the central finite-difference check. It approximates derivatives by evaluating nearby points. Small discrepancies are normal because of truncation and floating-point error, but a large discrepancy suggests a formula or indexing mistake. Check more than one point.

SLIDE 51: Lab 2: a gradient descent trace

Run the first two gradient descent updates and compare the objective sequence with twenty, three point two, and one point six six four. Then use the trace function to continue the iterations. It stores the current point, objective, and gradient norm before deciding whether to stop. This makes the trace easier to interpret, because iteration zero is the original starting point. If the first values do not match, inspect the gradient function and the subtraction in the update. Once the arithmetic agrees, examine the path and the decreasing objective. Report the stopping reason as well as the final coordinates.

SLIDE 52: Lab 3: step-size experiments

Run the five step sizes listed on this slide using the same initial point and stopping settings. The three values below one quarter should converge, although their paths and speeds differ. Zero point two four is close to the boundary, so the v coordinate oscillates while its error shrinks. At zero point two five, the error oscillates without shrinking. At zero point three, it grows. The implementation may stop a growing run with a guard rather than allowing overflow. That guard is not convergence. Relate each observed behavior to the multiplier one minus eight alpha, and compare the traces instead of reading only a final status.

SLIDE 53: Lab 4: Newton and a reference solver

For the manual Newton calculation in Python, solve the Hessian system with NumPy's linear algebra solver. At the original starting point, the direction should be minus two, minus two, and the updated point should be one, minus two. Verify both the objective and the gradient there. The optional SciPy comparison uses the trust-exact method with analytic derivatives. That solver uses a trust-region strategy, so its iteration count need not equal the single full Newton step we calculated by hand. Compare the final result and tolerances, while recognizing that the algorithms do not necessarily take the same path.

SLIDE 54: Lab 5: nonconvex starting points

Now use gradient descent on the double-well function with step size zero point zero five. Compare the five starting points on the slide. The negative starting points approach minus one, and the positive starting points approach plus one in these runs. Starting exactly at zero is different: the gradient is already zero, so the stopping test can trigger immediately. However, the second derivative there is minus four, identifying a local maximum. At plus or minus one, the second derivative is eight and the objective is zero. Since the objective is nonnegative everywhere, those two minima are global. This is why gradient-based termination alone is not a classification.

SLIDE 55: Independent practice

Try the three independent practice questions before reading the next slides. In question A, the cross term couples x and y, so the Hessian is no longer diagonal. Compute the mixed derivatives carefully. In question B, differentiate the quartic objective and take exactly one full Newton step from two. Then substitute the updated point into the original function. In question C, provide two functions that share zero first and second derivatives at the origin but have opposite classifications there. Pause here and write the steps. If time is limited, complete A now and use B and C as follow-up practice.

SLIDE 56: Practice A: coupled quadratic solution

For question A, the gradient components are two x plus two y minus four, and two x plus six y minus eight. The Hessian has rows two, two, and two, six. Its characteristic polynomial is lambda squared minus eight lambda plus eight, giving eigenvalues four plus or minus two times the square root of two. Both are positive. Setting the gradient to zero gives x plus y equals two, and x plus three y equals four. Subtract the equations to get y equals one, then x equals one. The objective value is one. Because the Hessian is constant and positive definite, this is the unique global minimum.

SLIDE 57: Practice B and C: worked answers

For question B, at x equals two, the first derivative of x to the fourth plus x squared is thirty-six, and the second derivative is fifty. The Newton direction is minus thirty-six divided by fifty, or minus zero point seven two. The new point is one point two eight. Substitution gives approximately four point three two two seven five five, down from twenty. It is still not the exact minimum at zero. For question C, use x to the fourth and minus x to the fourth. Their first two derivatives vanish at zero, but one has a strict minimum and the other a strict maximum.

SLIDE 58: Closing questions

Before finishing, answer these three questions in your own words. First, does a positive Hessian at a single point establish global convexity? Second, what happens with step size zero point two five in our quadratic example from three, zero? Third, why can Newton increase the double-well objective when starting at zero point two? Give a reason, not just yes or no. For the second question, use the error recurrence. For the third, inspect the sign of gradient times direction. Pause the narration and write your answers before proceeding.

SLIDE 59: Closing answers

For the first question, no. The Hessian test for convexity requires positive semidefiniteness throughout the open convex domain, not just at one point. For the second, the v-error multiplier is minus one. The error changes sign but retains its magnitude, so the iterates do not approach the minimizer from the specified starting point. The objective tends to sixteen rather than zero. For the third, the second derivative at zero point two is negative. Raw Newton produces a direction with positive gradient-times-direction, and direct evaluation confirms an objective increase. These answers connect the geometry to the observed numerical behavior.

SLIDE 60: Next session: choosing the step size

Keep four questions in mind whenever you solve an optimization problem. What is its geometry? What do the derivatives tell us? How does the algorithm choose its direction and step size? And what evidence supports the final answer? Today we connected these questions through convexity, gradient descent, Newton steps, and numerical checks. Next session we will choose step sizes using backtracking and the Armijo condition, then introduce BFGS and practical stopping rules. Save your notebook, including the runs that oscillated or moved uphill. Those examples give us concrete reasons for the safeguards we study next.