Math 164 – Optimization – Spring 2019

Course overview

The Goal of the class is to become familiar
with the theory and algorithms for linear and non-linear optimization of
multivariable functions
. This involves learning some of the
frequently used algorithms for optimization, proving in some cases
their convergence and understanding some of the theoretical
and practical issues surrounding their use. Additionally the course
will give opportunities to implement and learn the algorithms
using python. We will also review some linear algebra, and
learn some interesting theory related to duality, linear programs
and convex functions.

Text book: An Introduction to Optimization by Chong and Zak. Additionally some notes
may be provided from time to time.

General information

Instructor: Sylvester Eriksson-Bique,
6164 Math Sciences Building.

Office Hours: Monday at 12 pm, Wednesday at 4pm and Thursday at time TBD in PIC lab.

Lecture times: MWF 10-10:50am, GEOLOGY 3656


  • Dominic Yang, discussion section Tuesday 10-10:50 in PIC lab.

HW: Available on CCLE.

Exams: One in class midterm: Monday May 6th. Three-hour final: Thursday June 13th 8am-11am Location TBD, and available on
myUCLA from 11/26 onwards, and will be announced.

Bring student ID to both midterm and the final.
No books allowed. You can bring a calculator. Some of the more complicated formulas, if needed, will be provided.


The course will consist of

  • A midterm worth 25 %
  • A final exam: 35 %
  • Participation in discussion sections: 15 %
  • Roughly weekly Homework: 25 %

Extra credit opportunities:

  • Optimization challenge. Up to 1-5 %, depending on the quality of the work.
  • Activity on Piazza can give 1-2 extra credit depending on level of participation.

Course grade assignment

  • There will be no curve for the class, unless there is a need to curve scores upwards.
  • You should expect an A or higher if you have 94 % or above, a B or higher if you have 84 %,
    and a C if you receive 74 % or higher. However, these limits may be slightly adjusted up or down as needed.

PIC Lab, Python and discussions

We will have weekly discussions, or PIC lab sessions led by the TA. These will involve a worksheet and will use
Python. The TA will prepare a worksheet for these sessions, and you
will get credit for attendance at the end if you worked on the worksheet sincerely. You may, and are encouraged,
to work in groups. If you have no experience with python, try and find someone who has. The TA should
also help you with some basics. I will do my best to review
some basics, although this is not a python class and I am not an expert on this aspect. The goal
of these assignments is to employ some of the methods in practice, and through trial and error discover
what works.

I will also host an office hour in the PIC lab to discuss implementation and to test algorithms and
do demonstrations.

HW policies

The HW will be due usually the following Friday, and there will be about 8 assignments. You can
collaborate, but should submit your OWN work. Copied work will lead to a warning/discussion,
and if further repeated to no credit and possible further action.

Optimization challenge

There will be a list of optimization challenges. You can work in a group or alone,
and can get 1-5 % for implementing an algorithm to credibly solve the optimization problem
and to write a short ~5 page report on it describing your approach, choices, trials and errors. This report
is due at the end of class. I will grade them based on the quality of your work. 1 % is given to
a report that is complete, but lacks in substance and the implementation is generally unsatisfactory.
2-3 % will be the most common grades, and are for generally good solutions that give a result and show
that the student/group mastered most of the topic. 4-5 % is for projects with exceptional merit, with 5 %
awarded to only the very best of the projects, and 4 % to a few very good ones.

A group may be at most three people. If you work in a group, you must all contribute.
If you choose to work in a group, you must indicate
in your project file who did what part. Your descriptions should be specific (this person wrote this
section and coded this part, etc.). Broad descriptions may lead to a lower grade as I can not determine
for sure who did what. If I suspect that a person may have contributed significantly less, I will discuss
this with that person and inform them of my conclusion.

During the last two weeks of class you can also present your work to me in person. This way I can give you
feedback and you can increase the chances of getting a higher grade. I will also cap the extra credit
at 3 % if there is no discussion with me. If the project is done in a group, all members should be present
and I would like to see all members discuss the project, and may ask each individually about their contribution.


  • University policy requires participation in the final.
  • Only exceptions can excuse from midterm, in which case grade based on replacement exam.
  • HW will always be announced on CCLE and in class.
  • No late HW, the lowest HW dropped
  • HW due online usually by Friday following the posting. Calculation steps
    must be detailed, such as in the examples in our book.
  • HW returned in class or online on CCLE.
  • Consists of programming, T/F questions, explanations and analysis problems.
  • Each graded problem on HW graded for completeness, correctness and level of understanding.
  • Some HW may not be graded or is just graded for completeness.
  • Bring ID to exam. No notes or books allowed.
  • There will be no make up exams.
  • Participation in discussion sections is necessary for full credit.
    You receive full credit for attending 8/9 of all the sessions.


Tentative course schedule: Tentative

Lecture Date Section Topic
1 Monday April 1st Sections I 1,4,5 Overview of class, intro and basics of calculus and real analysis.
2 Wednesday April 3rd Part I 2,3,4 Linear algebra review
3 Friday April 5th Part I 4,5, II 6 Taylor’s series, gradients and hessians, optimality contraints and feasible directions.
4 Monday April 8th Part II 7 One dimensional search methods I
5 Wednesday April 10th Part II 7 One dimensional search methods II
6 Friday April 12th Part II 7 One dimensional search methods III
7 Monday April 15th Part II 8 Gradient descent I
8 Wednesday April 17th Part II 8 Gradient descent II
9 Friday April 19th Part II 9 Newtons method I
10 Monday April 22nd Part II 9 Newtons method II
11 Wednesday April 24th Part II 9 Gauss-Newton method
12 Friday April 26th Part II 10 Conjugate direction method I
13 Monday April 29th Part II 10 Conjugate direction method II
14 Wednesday May 1st Part II 11 Quasinewton methods
15 Friday May 3rd Part II 11 Continuation of prior, left TBD for flexibility.
16 Monday May 6th Up to Part II 11 Midterm: Nonlinear optimization and line search methods
17 Wedneasday May 8th Part II 12, Part III 16 Solving Linear equations I
18 Friday May 10th Part II 12, Part III 16 Solving Linear equations II
19 Monday May 13th Part III 15 Linear Programming I
20 Wed May 15th Part III 15 Linear Programming II
21 Friday May 17th Part III 16 Simplex method I
22 Monday May 20th Part III 16 Simplex method II
23 Wednesday May 22nd Part III 16 Simplex method III
24 Friday May 24th Part III 17 Duality I
25 Memorial day May 27th No lecture Happy memorial day!
26 Wednesday May 29th Part III 22 Convex Optimization I
27 Friday May 31st Part IV 22 Convex Optimization II
28 Monday June 3rd Part IV 20 Non-linear constrained optimizations
29 Wednesday June 5th Part IV 21 KKT conditions
30 Friday June 7th TBD Additional topics such as Machine learning/review
FINAL Thur Dec 13th Cumulative 8am-11am Location TBD

Notice about academic integrity

From the office of the Dean of Students:

“With its status as a world-class research institution, it is critical that the University uphold the highest
standards of integrity both inside and outside the classroom. As a student and member of the UCLA
community, you are expected to demonstrate integrity in all of your academic endeavors. Accordingly,
when accusations of academic dishonesty occur, The Office of the Dean of Students is charged with
investigating and adjudicating suspected violations. Academic dishonesty includes, but is not limited
to, cheating, fabrication, plagiarism, multiple submissions or facilitating academic misconduct.”

Students are expected to be aware of the University policy on academic integrity in the UCLA Student
Conduct Code:

Please note the sections on (1) cheating, (2) plagiarism, and (3) unauthorized study aids.
Violation of course policy involving plagiarism, cheating, or possession of course materials during
exams will be referred to the Dean of Students, who will be encouraged to take strong action. Do not
cheat! The penalties can be very harsh. Do not believe it if you hear that “everyone does it.” You
generally do not hear about the punishments because they are kept confidential. If you are found
responsible by the Dean of Students for violating course policy, cheating on any course materials, or
giving or receiving unauthorized help, a zero will be assigned for the entire assignment. No exceptions
will be made! Past examples of penalties also include loss of an entire term of credit and suspension for
several terms. If you plan to apply to graduate or professional school, such a negative mark on your
record may be a major obstacle to admission.

No cell phones are allowed during exams. They must be left in your bag and turned off, or submitted to
the designated TA/proctor. Students may not use a cell phone as a clock to keep time, nor as a
calculator. No hats are allowed in the testing room; they must be left in your bag.

Notice about sexual harassment, discrimination, and assault

Title IX prohibits gender discrimination, including sexual harassment, domestic and dating violence,
sexual assault, and stalking. Students who have experienced sexual harassment or sexual violence can
receive confidential support and advocacy from a CARE advocate:

The CARE Advocacy Office for Sexual and Gender-Based Violence
1st Floor, Wooden Center West
(310) 206-2465

You can also report sexual violence or sexual harassment directly to the University’s Title IX

Kathleen Salvaty
2241 Murphy Hall
(310) 206-3417

CAE brochure and Counceling

My goal is to foster an inclusive, supportive and encouraging enviroment for you to learn. Many of us suffer from various degrees
of learning difficulties, exam stress etc, or difficulties in our private lives. Please be aware that the university has many resources to help with challenges you might face
and that I as an instructor will do my best to assist and accomodate different learners. Please do not be afraid to reach out to a councelor,
me or your TA for contact information to various services. Also, please review the attached links for more resources.

%d bloggers like this: