COMP7905A - Reverse engineering and malware analysis

Semester 2, 2022-23

Professor
Frankie F.K. Li
Teaching assistants
Ka Kin Ken Ma
Po Shing Ken Wong
Syllabus This course provides students a foundational knowledge about reverse engineering and malware analysis, through the study of various cases and hand-on analysis of malware samples.  It covers fundamental concepts in malware investigations so as to equip the students with enough background knowledge in handling malicious software attacks.  Various malware incidents will be covered, such as cases in Ransomware, banking-Trojan, state-sponsored and APT attacks, cases in Stuxnet and malicious software attacks on Industrial Control System and IoT devices.  With the experience of studying these cases and analyzing selected samples, the students will be able to understand the global cyber security landscape and its future impact.  Hands-on exercises and in-depth discussion will be provided to enable students to acquire the required knowledge and skill set for defending and protecting an enterprise network environment.

Students should have programming/development skills (Assembly, C, C++, Python) and knowledge in Operating System and computer network.
Introduction by Professor Cybersecurity has become the top priority for any organizations in protecting their digital assets or online activities.  Modern sophisticated adversaries can easily find vulnerabilities and make exploits to launch attacks for financial gains or as a way to achieve specific objectives.  Cases of denial-of-service attacks to business servers, leakage of business and customer private information, advance malware attacks, fraudulent and malicious websites are found very common in the daily news.  This module provides you with basic knowledge in malware analysis processes and their complexities as well as illustrate on how to build an analytical capability best suited enterprises environment.  The tools and techniques presented in this course are intended for students to analyze selected samples in virtualization test-bed, so they can think like a hacker to understand the Tactic, Technique and Procedures (TTP) of malware authors.
Learning Outcomes
Course Learning Outcomes Relevant Programme Learning Outcomes
CLO1. Be able to understand the cyber security challenges raised from malicious software attacks PLO.1, 4, 6, 7, 8, 9, 14, 15
CLO2. Be able to analyze the security risks, threats and potential vulnerabilities on enterprise networks environment PLO.3, 5, 6, 7, 8, 9, 10, 11, 12, 13
CLO3. Be able to carry out independent analysis of modern malware samples using behavioral, code analysis and memory forensic techniques PLO2, 3, 4, 5, 6, 7, 8, 9, 11
CLO4. Be able to apply the learned techniques to protect, reduce the security risks and avoid malicious software attacks on computer systems or networks PLO.3, 5, 6, 7, 8, 9, 16
CLO5. Be able to research independently and use learned skills and tools to investigate malicious software attacks and implement or update a cyber protection plan PLO.1, 2, 3, 14, 15, 16
View Programme Learning Outcomes
Pre-requisites Students should have programming/development skills (Assembly, C, C++, Python) and knowledge in Operating System and computer network.
Compatibility Mutually exclusive with: COMP7804 E-commerce security cases and technologies
Topics covered
Course Content No. of Hours Course Learning Outcomes
1. Modern malware 3 CLO1, CLO2
2. OS internals 4 CLO1, CLO2, CLO4
3. Sandbox virtualization and behavioral analysis 5 CLO2, CLO3, CLO4, CLO5
4. Reverse engineering and code analysis of binaries (PE file format and ELF file format) 9 CLO2, CLO3, CLO4, CLO5
5. Identify malware through memory forensics 3 CLO1, CLO2, CLO3, CLO4, CLO5
6. Cyber threat intelligence and IOCs 3 CLO1, CLO2, CLO5
7. Incident response 3 CLO2, CLO5
 
Assessment
Description Type Weighting * Tentative Assessment Period /
Examination Period ^
Course Learning Outcomes
Individual projects (traditional malware identification and investigation) Continuous Assessment 15% - CLO2, CLO3, CL04
Individual projects (advance malware investigation investigation) Continuous Assessment 30% - CLO2, CLO3, CL04
Group project (identification of malicious functions of sample malware) Continuous Assessment 15% - CLO2, CLO3, CL04
Written exam covers all taught content in the course Written Examination 40% 8 - 23 May 2023 CL01, CL05
* The weighting of coursework and examination marks is subject to approval
^ The exact examination date uses to be released when all enrolments are confirmed after add/drop period by the Examinations Office.  Students are obliged to follow the examination schedule.  Students should NOT enrol in the course if they are not certain that they will be in Hong Kong during the examination period.  Absent from examination may result in failure in the course. There is no supplementary examination for all MSc curriculums in the Faculty of Engineering.
Course materials Recommended readings:
  • Practical Malware Analysis: The Hands-On Guide to Dissection Malicious Software by Michael Sikorski and Andrew Honig, Mar 2012
  • Windows Malware Analysis Essentials, by Victor Marak, August 2015.
  • Learning Malware Analysis: Explore the concepts, tools, and techniques to analysze and investigate Windows malware by Monnappa K A, Jun 2018
Session dates
Date Time Venue Remark
Session 1 17 Jan 2023 (Tue) 7:00pm - 10:00pm LE-4 Face-to-face
Session 2 31 Jan 2023 (Tue) 7:00pm - 10:00pm LE-4 Face-to-face
Session 3 7 Feb 2023 (Tue) 7:00pm - 10:00pm LE-4 Face-to-face
Session 4 14 Feb 2023 (Tue) 7:00pm - 10:00pm LE-4 Face-to-face
Session 5 21 Feb 2023 (Tue) 7:00pm - 10:00pm LE-4 Face-to-face
Session 6 28 Feb 2023 (Tue) 7:00pm - 10:00pm LE-4 Face-to-face
Session 7 14 Mar 2023 (Tue) 7:00pm - 10:00pm LE-4 Face-to-face
Session 8 21 Mar 2023 (Tue) 7:00pm - 10:00pm LE-4 Face-to-face
Session 9 28 Mar 2023 (Tue) 7:00pm - 10:00pm LE-4 Face-to-face
Session 10 4 Apr 2023 (Tue) 7:00pm - 10:00pm LE-4 Face-to-face
LE - Library Extension Building
Add/drop 16 January, 2023 - 4 February, 2023
Maximum class size 148
Back