Mustakimur Khandaker

Aloha

I am a systems and software security researcher who approaches security from the perspective of how things actually break under the hood. Besides that, my work spans threat telemetry engines, kernel hardening, malware analysis, IAM for AI agents, and AI red-teaming.

Currently, I work as a Staff Security Research Engineer at Samsung Research America, focusing on security architecture and low-level threat modeling for the Knox Zero Trust. Besides that, my work spans threat telemetry engines, kernel hardening, malware analysis, IAM for AI agent, and AI red teaming.

Before returning to industry, I was an Assistant Professor of Cybersecurity at the University of Georgia, leading a systems security research lab and teaching low-level security. I earned my Ph.D. from Florida State University under Dr. Zhi Wang, specializing in compiler-enforced software defenses, binary hardening, and enclave security—publishing across top venues like USENIX Security, ASPLOS, and IEEE EuroS&P (Best Paper Award).

I started out as an Android software engineer at Samsung R&D Institute Bangladesh after graduating from CUET in 2012. That early experience sparked my passion for system internals, binary analysis, and building low-level defenses. Today, along with developing open-source security tools and blogging about emerging threats, I share my travels, gardening, and book recommendations in my diary.

Contact Info

• Email: mustakcsecuet@gmail.com
• Office: Samsung Research America
• Mail: 665 Clyde Ave, Mountain View, CA 94043

Publications

• COIN attacks: on the insecurity of enclave untrusted interfaces in SGX.
  Mustakimur Rahman Khandaker, Yueqiang Cheng, Zhi Wang, Tao Wei.
  Proceedings of the 25th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’20), Lausanne, Switzerland, March 2020 (18%).
  [Paper] [Presentation] [Source]

• Origin-sensitive Control Flow Integrity.
  Mustakimur Rahman Khandaker, Wenqing Liu, Abu Naser, Zhi Wang, Jie Yang.
  Proceedings of the 28th USENIX Security Symposium (USENIX Security’19), Santa Clara, CA, USA, August 2019 (16%).
  [Paper] [Presentation] [Source]

• Adaptive Call-site Sensitive Control Flow Integrity.
  Mustakimur Rahman Khandaker, Abu Naser, Wenqing Liu, Zhi Wang, Yajin Zhou, Yueqiang Cheng.
  Proceedings of the 4th IEEE European Symposium on Security and Privacy (EuroS&P’19), Stockholm, Sweden, Jun 2019 (20%).
  [Paper] [Presentation] [Source] [Best Paper Award]

• Understanding the Challenges in Detecting Vulnerabilities of Rust Applications.
  Diane B. Stephens, Kawkab Aldoshan, Mustakimur Rahman Khandaker.
  IEEE Secure Development Conference (SecDev’24), Pittsburgh, PA, October 2024 (31%).
  [Paper]

• RustLIVE: Reducing the Learning Barriers of Rust Through Visualization.
  Diane B. Stephens, Kyu Hyung Lee, Mustakimur Rahman Khandaker.
  Frontiers in Education 2024 (FIE’24), Washington DC, USA, October 2024.
  [Paper]

• Secure In-cache Execution.
  Yue Chen, Mustakimur Rahman Khandaker, Zhi Wang.
  Proceedings of the 20th International Symposium on Research in Attacks, Intrusions, and Defenses (RAID’17), Atlanta, Georgia, September 2017. [Paper]

• Pinpointing Vulnerabilities.
  Yue Chen, Mustakimur Rahman Khandaker, Zhi Wang.
  Proceedings of the 12th ACM Asia Conference on Computer and Communications Security (AsiaCCS’17), Abu Dhabi, United Arab Emirates, April 2017. [Paper]

• Location-based early disaster warning and evacuation system on mobile phones using OpenStreetMap.
  Khandaker Mustakimur Rahman, Tauhidul Alam, Mahfuzulhoq Chowdhury.
  IEEE 2012 Conference on Open Systems (ICOS ’12), Kuala Lumpur, 21-24 Oct. 2012. [Paper]

Professional Career

Samsung Research America

• Knox Zero Trust: Telemetry Engine Architecture, MITRE ATT&CK Threat Mapping, On-Device Threat Detection, Incident Response Playbooks, Code Review
• OS & Kernel Hardening: Knox Ultra Subsystem Hardening, Attack Surface Reduction, RKP & ARM MTE Integration, Real-Time Exploit Detection
• AI Agent Security & IAM: Multi-Agent Orchestration Architecture, SELinux IPC Reference Monitor, Non-Human Identity & Credential Management
• AI Red Teaming: Multi-Agent Cascading Exploits, Cross-App Data Exfiltration, Indirect Prompt Injection (Calendar, SMS, Web), Adversarial Tooling

University of Georgia & Florida State University

• Control Flow Integrity: Origin-Sensitive CFI, Adaptive Call-Site Sensitive CFI, Context-Sensitive Policies, Compiler-Enforced Defense
• Enclave & Platform Security: Intel SGX Interface Vulnerabilities (COIN Attacks), LLVM/Clang Dynamic Analysis, Hardware-Enforced Isolation
• Memory-Safe Languages: Rust Vulnerability Analysis, RustLIVE Visualization Tool, Safe Systems Tooling, Open-Source Crate Development
• Binary Analysis & Exploitation: Automated Vulnerability Pinpointing, Fuzzing (AFL++, LibFuzzer, Syzkaller), Dynamic Binary Instrumentation, Vulnerability Disclosures (CVEs)

Samsung Research and Development Institute Bangladesh

• Android Graphics & Frameworks: Architecture Draw/Sketch/Paint, Canvas UI & Layer Management, Multi-Touch Scaling Engine, Native Undo/Redo Pipeline
• Samsung PIMS Applications: Calendar, Clock, and Calculator Feature Implementation, Stylus & SNote Integration, 500+ Subsystem Bug Triages & Fixes

Teaching Experience

• Cyber Security (CSCI 4250/6250)
• Secure Programming (CSCI 8245)
• Computer Networks (CSCI 6760)
• Object Oriented Programming (COP 3330)

Volunteer Service

• Program Committee:
  • Annual Computer Security Applications Conference (ACSAC), 2022.
  • The International Workshop on Security, Privacy, and Trust for Emergency Events, 2020.
• Journal Reviewer:
  • IEEE Transactions on Computers (TC-CS)
  • IEEE Transactions on Dependable and Secure Computing (TDSC-CS)

Honors & Awards

• Best Paper Award (2019) — IEEE European Symposium on Security and Privacy.
• Career Development Influencer (2024) — Career Center @ University of Georgia.
• Graduate Research Assistant Award (2019) — Computer Science @ Florida State University.
• Graduate Teaching Assistant Award (2018) — Computer Science @ Florida State University.
• Competitive Programming (2017-2019) — Top, ACM @ FSU.