Recently, deep learning techniques have advanced to the point that they can achieve state-of-the-art performance on a wide variety of tasks, such as image classification, speech recognition, and natural language processing. This course will give you practical experience with:
• with both feedforward and recurrent neural networks,
• Convolutional neural networks,
• Recurrent neural networks,
• Attention and Memory,
• Autoencoders and Autoregressive Models,
• Generative Adversarial Networks,
• Variational Autoencoders
• regularization, dropout, and normalization to improve generalization.
• Gradient descent and backpropagation of loss functions
Prerequisite(s)
Corequisite(s)
Special Requisite(s)
Instructor(s)
Assist. Prof. Dr. Fatma Patlar Akbulut
Course Assistant(s)
Schedule
Office Hour(s)
Teaching Methods and Techniques
Lecture, Discussion, LAB, Assignments and Project.
Principle Sources
I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016. http://www.deeplearningbook.org.
K. P. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012.
C. M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
Other Sources
Course Schedules
Week
Contents
Learning Methods
1. Week
Introduction to Deep Learning
Python Basics
2. Week
Machine Learning Overview
Linear Regression: -Estimation, Errors, Analysis-, Polynomial Regression, Image Classification using a Linear Classifier, Cross Validation
3. Week
Multi-Layer Perceptrons
Designing a perceptron, multi-layer perceptron, activation functions, optimization algorithms, Feedforward Network Training by Backpropagation
Building CNN architecture, Transfer Learning with CNN, Semantic segmentation, Object Detection, Instance Segmentation
6. Week
Understanding and Visualizing CNNs
Visualizing the activations and first-layer weights, Feature representations with invariance and covariance, Embedding the codes with t-SEN, Visualize class activation maps, DeepDream, Neural Style Transfer
7. Week
Recurrent Neural Networks
Building RNN architecture, Building a LSTM/GRU model to train time series data,Creating Unigram, Bigram and Trigram Language Models, Build Image Caption Generator with CNN & LSTM, Text Summarization Using a Seq2Seq Model
8. Week
MIDTERM
Oral presentation, Laboratory
9. Week
Attention, Transformers, and Memory
End-to-end machine translation with LSTM and attention, Multi-signal modeling, Pretraining during transformers, Vision Transformers
Semantic Similarity, Semantic Clustering, Language Models, Self-supervised learning in NLP, Self-supervised learning in vision –Inpainting -Jigsaw puzzles
14. Week
Project Presentations
Project Presentations
15. Week
16. Week
17. Week
Assessments
Evaluation tools
Quantity
Weight(%)
Midterm(s)
1
25
Homework / Term Projects / Presentations
1
10
Project(s)
1
30
Program Outcomes
PO-1
Adequate knowledge in mathematics, science and engineering subjects pertaining to the relevant discipline; ability to use theoretical and applied information in these areas to model and solve engineering problems.
PO-2
Ability to identify, formulate, and solve complex engineering problems; ability to select and apply proper analysis and modelling methods for this purpose.
PO-3
Ability to design a complex system, process, device or product under realistic constraints and conditions, in such a way so as to meet the desired result; ability to apply modern design methods for this purpose. (Realistic constraints and conditions may include factors such as economic and environmental issues, sustainability, manufacturability, ethics, health, safety issues, and social and political issues according to the nature of the design.)
PO-4
Ability to devise, select, and use modern techniques and tools needed for engineering practice; ability to employ information technologies effectively.
PO-5
Ability to design and conduct experiments, gather data, analyse and interpret results for investigating engineering problems.
PO-6
Ability to work efficiently in intra-disciplinary and multi-disciplinary teams; ability to work individually.
PO-7
Ability to communicate effectively, both orally and in writing; knowledge of a minimum of one foreign language.
PO-8
Recognition of the need for lifelong learning; ability to access information, to follow developments in science and technology, and to continue to educate him/herself.
PO-9
Awareness of professional and ethical responsibility.
PO-10
Information about business life practices such as project management, risk management, and change management; awareness of entrepreneurship, innovation, and sustainable development.
PO-11
Knowledge about contemporary issues and the global and societal effects of engineering practices on health, environment, and safety; awareness of the legal consequences of engineering solutions.
Learning Outcomes
LO-1
Identify the deep learning algorithms which are more appropriate for various types of learning tasks in various domains.
LO-2
Develop an appreciation for what is involved in learning models from data
LO-3
Analyze and evaluate the design and implementation of deep learning methods.
LO-4
Understand the theory behind deep learning methods such as Convolutional Neural Networks, and Recurrent Neural Networks, Autoencoders, and Deep Genarative Networks
LO-5
Evaluate the performance of different deep learning models (e.g., with respect to the bias-variance trade-off, overfitting and underfitting, estimation of test error).
LO-6
Implement deep learning algorithms and solve real-world problems.
