Machine learning studies algorithms for building data-driven models that can make predictions about data and novel observations. Recent developments in machine learning approaches based on deep neural networks, also known as deep learning, have led to performance break-throughs and catalyzed research in many closely related fields, including computer vision, natural language processing, speech recognition and robotics. This course aims to not only cover the fundamentals of deep learning, but also give a grasp of contemporary research. The course will start with a brief overview of machine learning and numerical optimization. Then, the basic techniques and modern approaches in designing, training and visualizing feedforward neural network architectures and convolutional neural networks will be introduced. Convolutional neural network-based methods for spatial localization of visual entities in images will be covered. Recurrent neural network architectures, and their applications in language and image understanding will be discussed. Recent advances in deep generative models will follow. Finally, deep reinforcement learning will be covered.
Prerequisite(s)
Corequisite(s)
Special Requisite(s)
Instructor(s)
Assist. Prof. Dr. Fatma Patlar Akbulut
Course Assistant(s)
Schedule
Office Hour(s)
Teaching Methods and Techniques
Lecture, Discussion, LAB, Assignments and Project.
Principle Sources
I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016. http://www.deeplearningbook.org.
K. P. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012.
C. M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
Other Sources
Course Schedules
Week
Contents
Learning Methods
1. Week
Introduction, Overview of machine learning, linear classifers, loss functions
Oral presentation, Laboratory
2. Week
Optimization, Stochastic gradient descent and contemporary variants, back-propagation
Oral presentation, Laboratory
3. Week
Feedforward networks and training, Activation functions, initialization, regularization, batch normalization, model selection, ensembles
Oral presentation, Laboratory
4. Week
Feedforward networks and training, Activation functions, initialization, regularization, batch normalization, model selection, ensembles
Deep learning for spatial localization, Transposed convolution, e_cient pooling, object detection, semantic segmentation
Oral presentation, Laboratory
8. Week
Recurrent neural networks, Recurrent neural networks (RNN), long-short term memory (LSTM), language
models, machine translation, image captioning, video processing, visual question answering, video processing, learning from descriptions, attention
Oral presentation, Laboratory
9. Week
Recurrent neural networks, Recurrent neural networks (RNN), long-short term memory (LSTM), language
models, machine translation, image captioning, video processing, visual question answering, video processing, learning from descriptions, attention
Oral presentation, Laboratory
10. Week
Recurrent neural networks, Recurrent neural networks (RNN), long-short term memory (LSTM), language
models, machine translation, image captioning, video processing, visual question answering, video processing, learning from descriptions, attention
Oral presentation, Laboratory
11. Week
Deep generative models, Auto-encoders, variational auto-encoders, generative adversarial networks, auto-regressive models, generative image models, unsupervised and self-supervised representation learning
Oral presentation, Laboratory
12. Week
Deep generative models, Auto-encoders, variational auto-encoders, generative adversarial networks, auto-regressive models, generative image models, unsupervised and self-supervised representation learning
Oral presentation, Laboratory
13. Week
Deep reinforcement learning, Policy gradient methods, Q-Learning,Project presentations
Oral presentation, Laboratory
14. Week
Deep reinforcement learning, Policy gradient methods, Q-Learning,Project presentations
Oral presentation, Laboratory
15. Week
16. Week
17. Week
Assessments
Evaluation tools
Quantity
Weight(%)
Midterm(s)
1
30
Homework / Term Projects / Presentations
3
15
Project(s)
1
15
Final Exam
6
40
Program Outcomes
PO-1
Adequate knowledge in mathematics, science and engineering subjects pertaining to the relevant discipline; ability to use theoretical and applied information in these areas to model and solve engineering problems.
PO-2
Ability to identify, formulate, and solve complex engineering problems; ability to select and apply proper analysis and modelling methods for this purpose.
PO-3
Ability to design a complex system, process, device or product under realistic constraints and conditions, in such a way so as to meet the desired result; ability to apply modern design methods for this purpose. (Realistic constraints and conditions may include factors such as economic and environmental issues, sustainability, manufacturability, ethics, health, safety issues, and social and political issues according to the nature of the design.)
PO-4
Ability to devise, select, and use modern techniques and tools needed for engineering practice; ability to employ information technologies effectively.
PO-5
Ability to design and conduct experiments, gather data, analyse and interpret results for investigating engineering problems.
PO-6
Ability to work efficiently in intra-disciplinary and multi-disciplinary teams; ability to work individually.
PO-7
Ability to communicate effectively, both orally and in writing; knowledge of a minimum of one foreign language.
PO-8
Recognition of the need for lifelong learning; ability to access information, to follow developments in science and technology, and to continue to educate him/herself.
PO-9
Awareness of professional and ethical responsibility.
PO-10
Information about business life practices such as project management, risk management, and change management; awareness of entrepreneurship, innovation, and sustainable development.
PO-11
Knowledge about contemporary issues and the global and societal effects of engineering practices on health, environment, and safety; awareness of the legal consequences of engineering solutions.
Learning Outcomes
LO-1
To understand the fundamentals of deep learning.
LO-2
To know the main techniques in deep learning and the main research in this field.
LO-3
Be able to design and implement deep neural network systems.
LO-4
Be able to identify new application requirements in the field of computer vision.
LO-5
Be able to identify reasonable work goals and estimate the resources required to achieve the objectives.
LO-6
Be able to structure and prepare scientific and technical documentation describing project activities.
LO-7
Be able to autonomously extend the knowledge acquired during the study course by reading and understanding scientific and technical documentation.
