AI Security – Backdoors and Poisoned Data

In this interview at Black Hat Asia 2024, we spoke with Adrian Wood and Mary Walker, security engineers from Dropbox, about the critical issues surrounding AI security, backdoors, and malware.

Adrian and Mary explained that many users rely on pre-existing machine learning (ML) models from public repositories rather than creating their own. This introduces vulnerabilities similar to those found in open-source software. Using in-house data requires careful handling to avoid bias and unintended consequences, while third-party models can be compromised.

They emphasized that downloading and running models from the internet can introduce malware. Attackers can backdoor models to alter their functions or insert malicious code, posing significant threats, especially in sensitive industries.

Adrian and Mary also stressed the importance of understanding the ML environment, ensuring proper logging, and having incident response plans in place. Companies should prepare by conducting tabletop exercises and securing their supply chains.

For more educational information on machine learning: https://gist.github.com/5stars217/236…

Dropbox, Security Engineer – Adrian Wood, aka threlfall, currently works for Dropbox on their red team. He has worked as a red team consultant for WHITEHACK, a company he founded, and later as a lead engineer for an offensive security research team at a US bank. His research recently has been in supply chain attacks on CI/CD and ML systems, which includes maintaining the offsec ml playbook and has presented on these topics at DEFCON 30, 31, the DEFCON AI village, Cackalackycon and more.

Dropbox, Security Engineer – Mary Walker, aka mairebear, currently works for Dropbox on their threat intelligence team; she splits her time at work between research (primarily focused on ML) and building tooling to help her team move faster. She’s previously worked at a major online retailer on their malware analysis and forensics team, a US bank on their red team, and an energy company in their SOC. Her background is primarily in DFIR and malware analysis, with a keen interest in production environments.

Recorded 18th April 2024, 4.30pm, BlackHat Asia 2024, Singapore
#BHAsia #mysecuritytv #blackhat