Mustakimur Khandaker

Aloha

I am a systems and software security researcher who approaches security from the perspective of how things actually break under the hood, not just to find vulnerabilities, but to build practical solutions for them.

Currently, I work as a Staff Security Research Engineer at Samsung Research America, focusing on security architecture and low-level threat modeling for the Knox Zero Trust. Besides that, my work spans threat telemetry engines, kernel hardening, malware analysis, IAM for AI agent, and AI red teaming.

Before returning to industry, I was an Assistant Professor of Cybersecurity at the University of Georgia, leading a systems security research lab and teaching low-level security. I earned my Ph.D. from Florida State University under Dr. Zhi Wang, specializing in compiler-enforced software defenses, binary hardening, and enclave security—publishing across top venues like USENIX Security, ASPLOS, and IEEE EuroS&P (Best Paper Award).

I started out as an Android software engineer at Samsung R&D Institute Bangladesh after graduating from CUET in 2012. That early experience sparked my passion for system internals, binary analysis, and building low-level defenses. Today, along with developing open-source security tools and blogging about emerging threats, I share my travels, gardening, and book recommendations in my diary.

Contact Info

• Email: mustakcsecuet@gmail.com
• Office: Samsung Research America
• Mail: 665 Clyde Ave, Mountain View, CA 94043

Publications

• COIN attacks: on the insecurity of enclave untrusted interfaces in SGX.
  Mustakimur Rahman Khandaker, Yueqiang Cheng, Zhi Wang, Tao Wei.
  Proceedings of the 25th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’20), Lausanne, Switzerland, March 2020 (18%).
  [Paper] [Presentation] [Source]

• Origin-sensitive Control Flow Integrity.
  Mustakimur Rahman Khandaker, Wenqing Liu, Abu Naser, Zhi Wang, Jie Yang.
  Proceedings of the 28th USENIX Security Symposium (USENIX Security’19), Santa Clara, CA, USA, August 2019 (16%).
  [Paper] [Presentation] [Source]

• Adaptive Call-site Sensitive Control Flow Integrity.
  Mustakimur Rahman Khandaker, Abu Naser, Wenqing Liu, Zhi Wang, Yajin Zhou, Yueqiang Cheng.
  Proceedings of the 4th IEEE European Symposium on Security and Privacy (EuroS&P’19), Stockholm, Sweden, Jun 2019 (20%).
  [Paper] [Presentation] [Source] [Best Paper Award]

• Understanding the Challenges in Detecting Vulnerabilities of Rust Applications.
  Diane B. Stephens, Kawkab Aldoshan, Mustakimur Rahman Khandaker.
  IEEE Secure Development Conference (SecDev’24), Pittsburgh, PA, October 2024 (31%).
  [Paper]

• RustLIVE: Reducing the Learning Barriers of Rust Through Visualization.
  Diane B. Stephens, Kyu Hyung Lee, Mustakimur Rahman Khandaker.
  Frontiers in Education 2024 (FIE’24), Washington DC, USA, October 2024.
  [Paper]

• Secure In-cache Execution.
  Yue Chen, Mustakimur Rahman Khandaker, Zhi Wang.
  Proceedings of the 20th International Symposium on Research in Attacks, Intrusions, and Defenses (RAID’17), Atlanta, Georgia, September 2017. [Paper]

• Pinpointing Vulnerabilities.
  Yue Chen, Mustakimur Rahman Khandaker, Zhi Wang.
  Proceedings of the 12th ACM Asia Conference on Computer and Communications Security (AsiaCCS’17), Abu Dhabi, United Arab Emirates, April 2017. [Paper]

• Location-based early disaster warning and evacuation system on mobile phones using OpenStreetMap.
  Khandaker Mustakimur Rahman, Tauhidul Alam, Mahfuzulhoq Chowdhury.
  IEEE 2012 Conference on Open Systems (ICOS ’12), Kuala Lumpur, 21-24 Oct. 2012. [Paper]

Professional Career

Samsung Research America [2025 - Now]

• Knox Zero Trust: Telemetry Engine Architecture, MITRE ATT&CK Threat Mapping, On-Device Threat Detection, Incident Response Playbooks, Code Review.
• OS & Kernel Hardening: Subsystem Hardening, Attack Surface Reduction, RKP Enhancement.
• AI Agent Security & IAM: Multi-Agent Orchestration Architecture, Intent Monitor, Non-Human Identity & Credential Management.
• AI Red Teaming: Multi-Agent Cascading Exploits, Cross-App Data Exfiltration, Indirect Prompt Injection.

University of Georgia [2020 - 2025] & Florida State University [2015 - 2020]

• Control Flow Integrity: Origin-Sensitive CFI, Call-Site Sensitive CFI, Constrain-sensitive CFI, Context-Sensitive Policies on Cross-DSO Application.
• Enclave & Platform Security: SGX Enclave Security, Embedded OS Security, Hybrid Program Analysis.
• Memory-Safe Languages: Rust Vulnerability, Semantics Visualizer, Crate Development.
• Binary Analysis & Exploitation: Vulnerability Pinpointing, Fuzzing (AFL++, LibFuzzer, Syzkaller), Exploit Generation, Binary Instrumentation.

Samsung Research and Development Institute Bangladesh [2013 - 2015]

• Artecture Draw/Sketch/Paint: Canvas UI & Layer Management, Multi-Touch Scaling Engine, Native Undo/Redo Pipeline.
• Samsung PIMS Applications: Calendar, Clock, and Calculator Feature Development, Stylus Integration, 500+ Bug Triages & Fixes.

Teaching Experience [2018 - 2025]

• Cyber Security (CSCI 4250/6250).
• Secure Programming (CSCI 8245).
• Computer Networks (CSCI 6760).
• Object Oriented Programming (COP 3330).

Volunteer Service [2011 - Now]

• Program Committee:
  • Annual Computer Security Applications Conference (ACSAC).
  • The International Workshop on Security, Privacy, and Trust for Emergency Events.
  • IEEE Security & Privacy Conference Student PC.
• Journal Reviewer:
  • IEEE Transactions on Computers (TC-CS)
  • IEEE Transactions on Dependable and Secure Computing (TDSC-CS)
• Social Impact:
  • Los Altos High School Grade 12 AVID.

Honors & Awards

• Best Paper Award (2019) — IEEE European Symposium on Security and Privacy.
• Career Development Influencer (2024) — Career Center @ University of Georgia.
• Graduate Research Assistant Award (2019) — Computer Science @ Florida State University.
• Graduate Teaching Assistant Award (2018) — Computer Science @ Florida State University.
• Competitive Programming (2017-2019) — Top, ACM @ FSU.