Security Unlocked

Share

Identity Threats, Tokens, and Tacos

Ep. 20

Every day there are literally billions of authentications across Microsoft – whether it’s someone checking their email, logging onto their Xbox, or hopping into a Teams call – and while there are tools like Multi-Factor Authentication in place to ensure the person behind the keyboard is the actual owner of the account, cyber-criminals can still manipulate systems. Catching one of these instances should be like catching the smallest needle in the largest haystack, but with the algorithms put into place by the Identity Security team at Microsoft, that haystack becomes much smaller, and that needle, much larger.


On today’s episode, hosts Nic Fillingham and Natalia Godyla invite back Maria Puertos Calvo, the Lead Data Scientist in Identity Security and Protection at Microsoft, to talk with us about how her team monitors such a massive scale of authentications on any given day. They also look deeper into Maria’s background and find out what got her into the field of security analytics and A.I. in the first place, and how her past in academia helped that trajectory.  


In this Episode You Will Learn:

• How the Identity Security team uses AI to authenticate billions of logins across Microsoft

• Why Fingerprints are fallible security tools

• How machine learning infrastructure has changed over the past couple of decades at Microsoft


Some Questions that We Ask:

• Is the sheer scale of authentications throughout Microsoft a dream come true or a nightmare for a data analyst?

• Do today’s threat-detection models share common threads with the threat-detection of previous decades?

• How does someone become Microsoft’s Lead Data Scientist for Identity Security and Protection?


Resources:


#IdentityJobs at Microsoft:

https://careers.microsoft.com/us/en/search-results?keywords=%23identityjobs


Maria’s First Appearance on Security Unlocked, Tackling Identity Threats with A.I.:

https://aka.ms/SecurityUnlockedEp08


Maria’s Linkedin:

https://www.linkedin.com/in/mariapuertas/


Nic’s LinkedIn:

https://www.linkedin.com/in/nicfill/


Natalia’s LinkedIn:

https://www.linkedin.com/in/nataliagodyla/


Microsoft Security Blog:

https://www.microsoft.com/security/blog/


Transcript

[Full transcript can be found at https://aka.ms/SecurityUnlockedEp20]


Nic Fillingham:

Hello, and welcome to Security Unlocked, a new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft security engineering and operations teams. I'm Nic Fillingham.


Natalia Godyla:

And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft security, deep dive into the newest threat intel, research, and data science.


Nic Fillingham:

And profile some of the fascinating people working on Artificial Intelligence in Microsoft security.


Natalia Godyla:

And now, let's unlock the pod.


Nic Fillingham:

Hello, Natalia. Welcome to episode 20 of Security Unlocked. This is, uh, an interesting episode. People may notice that your voice is absent from the... This interview that we had with Maria Puertos Calvo. How, how you doing? You okay? You feeling better?


Natalia Godyla:

I am, thank you. I'm feeling much better, though I am bummed I missed this conversation with Maria. I had so much fun talking with her in episode eight about tackling identity threats with AI. I'm sure this was equally as good. So, give me the scoop. What did you and Maria talk about?


Nic Fillingham:

It was a great conversation. So, you know, this is our 20th episode, which is kind of crazy, of Security Unlocked, and we get... We're getting some great feedback from listeners. Please, send us more, we want to hear your thoughts on the... On the podcast. But there've been a number of episodes where people contact us afterwards on Twitter or an email and say, "Hey, that guest was amazing," you know, "I wanna hear more." And Maria was, was definitely one of those guests who we got feedback that they'd love for us to invite them back and learn more about their story. So, Maria is on the podcast today to tell us about her journey into security and then her path to Microsoft. I won't give much away, but I will say that, if you're studying and you're considering a path into cyber security, or you're considering a path into data science, I think you're gonna really enjoy Maria's story, how she sort of walks through her academia and then her time into Microsoft. We talk about koalas and we talk about the perfect taco.


Natalia Godyla:

Yeah, to pair with the guac which she covered the first time around. Now tacos. I feel like we're building a meal here. I'm kind of digging the idea of a Security Unlocked recipe book. I, I think we need some kind of mocktail or cocktail to pair with this.


Nic Fillingham:

Yeah, I do think two recipes might not be enough to qualify for a recipe book.


Natalia Godyla:

Yeah, I mean, I'm feeling ambitious. I think... I think we could get more recipes, fill out a book. But with that, I, I cannot wait to hear Maria's episode. So, on with the pod?


Nic Fillingham:

On with the pod.


Nic Fillingham:

Maria Puertos Calvo, welcome back to the Security Unlocked podcast. How are you doing?


Maria Puertos Calvo:

Hi, I'm doing great, Nic. Thank you so much for having me back. I am super flattered you guys, like, invited me for the second time.


Nic Fillingham:

Yeah, well, thank you very much for coming back. The episode that we, we, we first met you on the podcast was episode eight which we called Tackling Identity Threats With AI, which was a really, really popular episode. We got great feedback from listeners and we thought, uh, let's, let's bring you back and hear a bit more about your, your own story, about how you got into security, how you got into identity, how you got into AI. And then sort of how you found your way to Microsoft.


Nic Fillingham:

But since we last spoke, I want to get the timeline right. Did you have twins in that period of time or had the twins already happened when we spoke to you in episode eight?


Maria Puertos Calvo:

(laughs) No, the twins had already happened. They-


Nic Fillingham:

Got it.


Maria Puertos Calvo:

I think it's been a few months. But they're, they are nine, nine months old now. Yeah.


Nic Fillingham:

Nine months old. And, and the other interesting thing is you're now in Spain.


Maria Puertos Calvo:

Yes.


Nic Fillingham:

When we spoke to you last, you were in the Redmond area or is that right?


Maria Puertos Calvo:

Yes, yes. The... Last time when we, we spoke, I, I was in Seattle. But I was about to make this, like, big trip across the world to come to Spain and, and the reason was, actually, you know, that the twins hadn't met my family. I am originally from Spain, and, and my whole family is, is here. And, you know, because of COVID and everything that happened, they weren't able to travel to the US to see us when they were born. So, my husband and I decided to just, like, you know, do a trip and take them. And, and we're staying here for a few months now.


Nic Fillingham:

That's awesome. I've been to Madrid and I've been to... I think I've only been to Madrid actually. Where, where... Are you in that area? What part of Spain are you in?


Maria Puertos Calvo:

Yes, yes. I'm in Madrid. I'm in Madrid. I, I'm from Madrid.


Nic Fillingham:

Aw- awesome. Beautiful city. I love it. So, obviously, we met you in episode eight, but if you could give us, uh, a little sort of mini reintroduction to who you are, what's your job at Microsoft, what does your... What does your day-to-day look like, that'd be great.


Maria Puertos Calvo:

Yeah. So, I am the lead data scientist in identity secure and protection, identity security team who... We are in charge of making sure that all of the users who use, uh, Microsoft identity services, either Azure Active Directory or Microsoft account, are safe and protected from malicious, you know, uh, cyber criminals. So, so, my team builds the algorithms and detections that are then put into, uh, protections. Like, for example, we build machine learning for risk based authentication. So, if we... If our models think an authentication is, is probably compromised, then maybe that authentication is challenged with MFA or blocked depending on the configuration of the tenet, et cetera.


Maria Puertos Calvo:

So, my team's day-to-day activities are, you know, uh, uh, building new detections using new data sets across Microsoft. We have so much data between, you know, logs and APIs and interactions b- between all of our customers with Microsoft systems. Uh, so, so, we analyze the data and, and we build models, uh, apply AI machine learning to detect those bad activities in the ecosystem. It could be, you know, an account compromised a sign-in that looks suspicious, but also fraud. Let's say, like, somebody, uh, creates millions of spammy email addresses with Microsoft account, for example to do bad things to the ecosystem, we're also in charge of detecting that.


Nic Fillingham:

Got it. So, every time I log in, or every time I authenticate with either my Azure Active Directory account for work or my personal Microsoft account, that authentication, uh, event flows through a set of systems and potentially a set of models that your team owns. And then if they're... And if that authentication is sort of deemed legitimate, I'm on my way to the service that I'm accessing. And if it's deemed not legitimate, it can go for a challenge through MFA or it'll be blocked? Did, did I get that right?


Maria Puertos Calvo:

You got that absolutely right.


Nic Fillingham:

So, that means... And I think we might've talked about this on the last podcast, but I still... I... As a long-term employee of Microsoft, I still get floored by the, the sheer scale of all this. So, there's... I mean, there's hundreds of millions of Microsoft account users, because that's the consumer service. So, that's gonna be everything from X-Box and Hotmail and Outlook.com and using the Bing website. So, that's, that's literally in the hundreds of millions realm. Is it... Is it a billion or is it... Is it just hundreds of millions?


Maria Puertos Calvo:

It depends on how you count them. Uh, if it's per day, it's hundreds of millions, per month I think it's close to a billion. Yes, for... Of users. But the number of authentications overall is much higher, 'cause, you know, the users are authenticating in s- in s- many cases, many, many times a day. A lot of what we evaluate is not only, like, your username and password authentications, there's also the, you know, the model authe- authentication particles that have your tokens cash in the application and those come back for request for access. So, the... We evaluate those as well.


Maria Puertos Calvo:

So, it's, uh... It's actually tens of billions of authentications a day for both the Microsoft account system and the Azure Active Directory system. Azure Active Directory is also a... Really big, uh, it's almost... It's, it's getting really close to Microsoft account in terms of monthly, monthly active users. And actually, this year, with, you know, COVID, and everybody, you know, the... All the schools, uh, going remote and so many people going to work from home, we have seen a huge increase in, in, in monthly active users for Azure Active Directory as well.


Nic Fillingham:

And do you treat those two systems separately? Uh, or, or are they essentially the same? It's the same anomaly detection and it's the same sort of models that you'd use to score and determine if a... If an authentication attempt is, is, uh, is legitimate or, or otherwise?


Maria Puertos Calvo:

It's, like, theoretically the same. You know, like, we, we use the same methodology. But then there are different... The, the two systems are different. They live in different places with different architectures. The data that is logged i- is different. So, these, these were initially not, you know... I- identity only, uh, took care of those two systems, like, a few years ago, before they w- used to be owned by different teams. So, the architecture underneath is still different. So, we still have to build different models and maintain them differently and, you know, uh, uh, tune them differently. So, so it is more work, but, uh, the, the theory and the idea, their... How we built them is, is very similar.


Nic Fillingham:

Are there some sort of trends that have, you know, appeared, having these two massive, massive systems sort of running in parallel but with the same sort of approach? What kind of behaviors or what kind of anomalies do you see detected in one versus the other? Do they sort of function sort of s- similar? Like, similar enough? Or do you see some sort of very different anomalies that appear in one system and, and not another.


Maria Puertos Calvo:

They're, interestingly, pretty different. Uh, when we see attack spikes and things like that, they don't always reflect one or the other. I think the, the motivation of the people that attack enterprises and organizations, it's, it's definitely from the, the hackers that are attacking consumer accounts. I think they're, you know, they're so in the black market separately, and they're priced separately, you know, and, and differently. And I think they're, they're generally used for different purposes. We see sometimes spikes in correlation, but, but not that much.


Nic Fillingham:

Before we sort of, uh, jump in to, to your personal story into security, into Microsoft, into, into data science, is the... You know, these... Talking about these sheer numbers, talking about the hundreds of millions of, of authentications, I think you said, like, tens of billions that are happening every day. Is that a dream for a data scientist to just have such a massive volume of data and signals at your fingertips that you can use to go and build models, train models, refine models? Is that, you know... Is this adage of more signal equals better, does that apply? Or at some point do you now have challenges of too much signal and you're now working on a different set of problems?


Maria Puertos Calvo:

That's a great question. It is an absolute dream and it's also a nightmare. (laughs) So, yeah. It is... It... And I'll tell you why for both, right? Like, a... It is a great dream. Like, obviously, you bet... The, the sheer scale of the data, the, you know, the, the fact... There are a lot of things that are easier, because sometimes when you're working with data and statistics, you have to do a lot of things to estimate if,


Maria Puertos Calvo:

... it's like the things that you're competing are statistically significant, right? Like, do I have enough data to approach that this sample, it's going to be, uh, reflection of reality, and things like that. With the amount of data that we have, with the amount of users that we have, it's the, we don't have that, we, we don't really have that problem, right? Like we are able to observe, you know, the whole rollout without having to, to figure out if what we're seeing, you know, it's similar to the whole world or not.


Maria Puertos Calvo:

So that's really cool. Also, because we're, you know, have so many users, then we also have, you know, we're a big focus for attackers. So, so we can see everything, you know, that happens in, in, in the cybersecurity world and like the adversary wall, we can find it in, in our data. And, and that is really interesting. Right. It's, it's really cool.


Nic Fillingham:

That sounds fascinating. But let, let, let's table that for a second. 'Cause I'd love to sort of go back in time and I'd love to learn about your journey into security, into sort of computer science, into tech, where did it all start? So you grew up in Madrid, is that right?


Maria Puertos Calvo:

Yes. I grew up in Madrid and when I was finishing high school and I was trying to figure out like, why do I do, I just decided to study telecommunication engineering, it's what's called a Spain, but it's ev- you know, the, the equivalent who asked degrees electrical engineering. Because I was actually, you know, really, really interested in math and science and physics. They were like my favorite subjects in high school. I was pretty, really good at it actually.


Maria Puertos Calvo:

And, but at the same time, I was like, well, this, you know, an engineering degree sounds like something that I could apply all of this to. And the one that seems like the coolest and the future and like I, I, is electrical engineering. Like I, at that time, computer science was also kind of like my second choice, but I knew that in electrical engineering, I could also learn a lot of computer science.


Maria Puertos Calvo:

It w- it has like a curriculum that includes a lot of computer science, but also you learn about communication theory and, you know, things like how do cell phones work? And how does television work? And you can learn about computer vision and image processing and all, all kinds of signal processing. I just found it fascinating.


Maria Puertos Calvo:

So, so I, I started that in college and then when I finished college, it was 2010. So it was right in the middle of the great recession, which actually hits Spain really, really, really badly when it came to the, the labor market, the unemployment back then, I think it was something like 25%-


Nic Fillingham:

Wow.


Maria Puertos Calvo:

... and people who were getting out of school, even in engineering degrees, which were traditionally degrees that would have, you know, great opportunities. They were not really getting good jobs. People, only consulting firms were hiring them, um, and, and really paying really, really little money. It was actually pretty kind of a shame. So I said, what, what, what should I do? And I, I had been a good student during college, so, and I had a professor that, you know, he, that I had done my kind of thesis with him and his research group.


Maria Puertos Calvo:

And he said, "Hey, why didn't you just like, continue studying? Like, you can actually go for your PhD and, because you have really good grades, I'm sure you can just get it full of finance. You can get a scholarship that will like finance, you know, four years of PhD. And you know, that way you don't have to pay for your studies, but also you kind of like, you're like a researcher and you have, uh, like money to live." And I was like, well, that sounds like a really good plan.


Nic Fillingham:

Sounds good.


Maria Puertos Calvo:

Like I actually, yeah. So, so I could do in that. And, and I, you know, then my master said, this masters say, wasn't computer science, but it was very pick and choose, right? Like, like you could pick your branch and what classes you took. And so the master's was the first half of the PhD was basically getting all your PhD qualifying courses, which also are equivalent to, to doing your masters.


Maria Puertos Calvo:

So I picked kind of like the artificial intelligence type branch, which had a lot of, you know, classes on machine learning and learn a lot of things that are apply that are user apply machine learning, it's like, uh, natural language processing and speech and speaker recognition and biometrics and computer vision. Basically, all kinds of fields of artificial intelligence, where, where in the courses that I took. And, and I really, really fou- found it fascinating. There wasn't, you know, a data science degree back then, like now everybody has a data science degree, but this is like 10 years ago. Uh, at least, you know, in Spain, there wasn't a data science degree.


Maria Puertos Calvo:

But this is like the closest thing, uh, that, and that was my first contact with, uh, you know, artificial intelligence and machine learning. And I, I loved it. And, and then I did my masters thesis on, uh, kind of like, uh, biometrics in, in terms of applying statistical models to forensic fingerprints to, to understand if a person can be falsely, let's say, accused of a crime because their fingerprint brand only matches a fingerprint that is found in a crime scene.


Maria Puertos Calvo:

So kind of try to figure out like, how likely is that. Because there have been people in the past that having wrongly convicted, uh, because of their fingerprints have been found in a crime scene. And then after the fact they have found the right person and then, you know, like, uh, it's not a very scientific method, what is followed right now. So that, that was a really cool thing too, that then I never did anything related to that in my life, but, but it was a very cool thing to study when I was in, in school.


Nic Fillingham:

Well, that, that's fair. I've, I've got some questions about that. That's fascinating. So how did you even stumble upon that as a, as a, as a, as a research focus? Was there a, a particular case you might've read in the, in the news or something like, I, I think I've never heard of people being falsely accused or convicted through having the same fingerprints, I guess, unless you're an identical twin.


Maria Puertos Calvo:

Mm-hmm (affirmative). (laughs) Actually, I can tell you because I have identical twins, but also that, because I studied a lot of our fingerprints is that identical twins do not have the same fingerprints.


Nic Fillingham:

Wow.


Maria Puertos Calvo:

Uh, because fingerprints are formed when you're in the womb. So they're not, they're not like a genetic thing. They happen kind of like, as a random pattern when, when your body is forming in the womb, and they happen, they're different. Uh, so, so humans have unique fingerprints and that's true, but the problem with the, the finger frame recognition is that, it's very partial, and is very imperfect because the, the late latent, it's called the latent fingerprint, the one that is found in a crime scene is then recovered, you know, using like some powder, and it's kind of like, you, you just found some, you know, sweaty thing and a surface, and then you have to lift that from there. Right.


Maria Puertos Calvo:

And, and that has imperfections in, and it only, it's not going to be like a full fingerprint. You're going to have a partial fingerprint. And then, then you, basically, the way the matching works is using this like little poin- points and, and bifurcations of the riches that exist in your fingerprint. And, and then, you know, looking at the, the location and direction of those, then they're matched with other fingerprints to understand if they're the same one or not. But the, because you don't have the full picture, it is possible that you make a mistake.


Maria Puertos Calvo:

The one case that it's been kind of really, really famous actually happened with the Madrid bombings that happened in 2004, where, you know, they, they blew up, uh, some trains and, and a couple of hundred people died. Then they, they actually found a fingerprint in one of the, I don't remember, like in the crime scene and it actually match in the FBI fingerprint database. It matched the fingerprint of a lawyer from Portland, Oregon, I believe it's what it was. And then he was initially, you know, uh, I don't know if you ended up being convicted, but, but you know, it wasn't-


Nic Fillingham:

He was a suspect.


Maria Puertos Calvo:

... it was a really famous case. Yes. I think he was initially convicted. And then, but then he was not after they found the right person and they, they actually found that yeah, both fingerprints, like the, the guy whose fingerprint it really was. And these other guys, they, their fingerprints both match the crime scene fingerprint, but that's only because it was only a piece of it. Right. You, you don't put your finger, like, you don't roll it left to right. Like when you arrive at the airport, right. That they make you roll your finger, and lay have the whole thing it's, you're maybe just, you know, the, the, the criminal fingerprint is, is very small.


Nic Fillingham:

Was that a big part of the, the research was trying to understand how much of a fingerprint is necessary for a sort of statistically relevant or sort of accurate determination that it belongs to, to the, to the right person?


Maria Puertos Calvo:

Yeah. So the results of the research they'd have some outcome around, like, depending on how many of those points that are used for identification, which are called minutia, depending on how, how many of those are available, it changes the probability of a random match with a random person, basically. So the more points you have, the less likely it is that will happen.


Nic Fillingham:

The one thing, like, as, as we're talking about this, that I sort of half remember from maybe being a kid, I don't know, growing up in Australia is don't koalas have fingerprints that are the same as humans. Did I make that up? Do you know anything about this?


Maria Puertos Calvo:

(laughs) I'm sure, I have no idea. (laughs) I have never heard such a thing.


Nic Fillingham:

I have a-


Maria Puertos Calvo:

Now I wanna know.


Nic Fillingham:

...I'm gonna have to look this up.


Maria Puertos Calvo:

Yeah.


Nic Fillingham:

I have a feeling that koa- koalas, (laughs) have fingerprints that are either very close to or indistinguishable from, from humans. I'm gonna look this one up.


Maria Puertos Calvo:

I wonder if like a koala could ever be wrongly convicted of a crime.


Nic Fillingham:

Right, right. So like, if I want to go rob a bank in Australia, all I need to do is like, bring a koala with me and leave the koala in the bank after I've successfully exited the bank with all the gold bars in my backpack. And then the police would show up and they arrest the koala and they'd get the fingerprints and they go, well, it must be the koala.


Maria Puertos Calvo:

Exactly.


Nic Fillingham:

This is a foolproof plan.


Maria Puertos Calvo:

(laughs)


Nic Fillingham:

I'm glad I discussed this with you on the podcast. Thank you, Marie, for validating my poses.


Maria Puertos Calvo:

Now, now you can't publish this.


Nic Fillingham:

Oh, we talked about fingerprints. Oh, crumbs you're right. Yeah. Okay. All right. We have to edit this out of the, (laughs) out of there quick.


Maria Puertos Calvo:

(laughs)


Nic Fillingham:

Um, okay. I didn't realize we had talked so much about fingerprints. That's my fault, but I found that fascinating. Thank you. So what happens next? Do you then go to Microsoft? Do you come straight out of your education at university in Madrid, straight to Microsoft?


Maria Puertos Calvo:

Kind of and no. So what happens next is that while I, I finished the master's part of this PhD, and at this time I'm actually dating my now husband, and he's an American, uh, working in Washington D.C. as an electrical engineer. So I, you know, I finished my master's and my, I say, why, why do I kind of wanna go be in the US uh, so I can be with him. And, you know, I have the space, the scholarship they'll actually lets me go do research abroad and you know, like kind of pays for it. So


Maria Puertos Calvo:

Find, um, another research group in the University of Maryland, College Park, which is really, really close to, to DC. And, and I go there to do research for, uh, six months. So, I spent six months there also doing research. Uh, also using, uh, machine learning for, for a different around iris recognition. And, you know, the six months went by and I was like, "Well, I want to stay a little longer," like, "I, you know, I really like living here," and I extended that, like, another six months. I... And at that point, you know, I wasn't really allowed to do that with my scholarship, so I just asked my professor to, you know, finance me for that time. And, and, uh, and at that time, I decided, like, you know, I, I actually don't think I wanna, like, pursue this whole PHD thing.


Maria Puertos Calvo:

So, so I stayed six more months working for him, and then I decided I, I, I'm not a really big fan of academia. I went into research in, in grad school in Spain mostly because there weren't other opportunities. I was super, you know, glad I did 'cause I, I love all the research and the knowledge that I gained with all... You know, with my master's where I learned everything about Artificial Intelligence. But at this point, I really, really wanted to go into industry. Uh, so I applied to a lot of jobs in a lot of different companies. You know, figuring out, like, my background is in biometrics and machine learning. Things like that. Data science is not a word that had ever come to my mind that I was or could be, but I was more, like, interested in, like, you know, maybe software roles related to companies that did things that I had a similar background in.


Maria Puertos Calvo:

For like a few months, I was looking in... I, I didn't even get calls. And I had no work experience other than, you know, I had been through college and grad school. So, I had... You know, and, and I was from Spain and from a Spanish university, and there was really nothing in my resume that was, like, oh, this is like the person we need to call. So, nobody called me. (laughs) And, and then one day, uh, I, I received a LinkedIn message from a Microsoft recruiter. And she says, "Hey, I have... I'm interested in talking to you about, uh, well, Microsoft." So I said, "Oh, my God. That sounds amazing." So, she calls me and we talk about it, and she's like, "Yeah, there's like this team at Microsoft that is like run mostly by data scientists and what they do is they help prevent fraud, abuse, and compromise for a lot of Microsoft online services."


Maria Puertos Calvo:

So, they, they basically use data and machine learning to do things like stopping spam for Outlook.com, doing, like, family safety like finding, like, things on the web that, that should be, like, not for children. They were also doing, like, phishing detection on the browser. Um, like phishing URL detection on the browser and a co- compromise detection for Microsoft Account. And so I was like, "Sure, that sounds amazing." You know? "I would love to be in the process." And I was actually lying because I did not want to move to Seattle. (laughs) Like, at that time, I was so hopeful that I will find a job at, you know, somewhere in DC on the east coast, which is like closer to Spain and where, where we lived in. But at the same time, you know, Microsoft calls and you don't say no mostly when nobody else is calling you.


Maria Puertos Calvo:

Um, so, so I said, "Sure, let's, you know, I, uh... The, the least I can do is, like, see how the interview goes." So, I did the phone screen and then I... They, they flew me to Seattle and I had seven interviews and a lunch inter- and a lunch kind of casual interview. So, it was like an eight hour interview. It was from 9:00 to 5:00. And, you know, everything sounded great, the role sounded great. Um, the, the team were... The things that they were doing sounded super interesting. And, to my surprise, the next day when I'm at the airport waiting for my flight to, to go back to DC, the recruiter calls me and says, "Hey, you, you know, you passed the interview and we're gonna make you an offer. You'll have an offer in the... In the mail tomorrow." I was like, "Oh, my God." (laughs) "What?" Like, I could not... This... It's crazy to me that this was, like, only seven years ago, it... But yeah.


Nic Fillingham:

Oh, this is seven... So, this was 2014, 2013?


Maria Puertos Calvo:

Uh, actually, when I did the interview, it was... It was more, more... It was longer. It was 2012.


Nic Fillingham:

2012. Got it.


Maria Puertos Calvo:

And then I... And then starting my Microsoft in 2013.


Nic Fillingham:

Got it.


Maria Puertos Calvo:

I started as a... I think at that time, they called us analysts. But it was funny because the, the team was very proud on the, the fact that they were one of the first teams doing, like, real data science at Microsoft. But there were too many teams at Microsoft calling themselves, and basically only doing, like, analytics and dashboards and things like that. So, because of that, the team that I was in was really proud, and they didn't want to call themselves data scientists, so they... I don't know. We called ourselves, like, analysts PMs, and then we were from that to decision scientists, uh, which I never understood the, the name. (laughs) Uh, but yeah. So, that's how I started.


Nic Fillingham:

Okay, so, so that first role was in... I heard you say Outlook.com. So, were you in the sort of consumer email pipeline team? Is that sort of where that, that sat?


Maria Puertos Calvo:

Yeah. Yeah, so, uh, the team was actually called safety platform. It doesn't exist anymore, but it was a team that provided the abuse, fraud, and, and, like, malicious detections for other teams that were... At the time, it was called the Windows live division.


Nic Fillingham:

Yes.


Maria Puertos Calvo:

So, all the... All the teams that were part of that division, they were like the browser, right? Like, Internet Explorer, Hotmail, which was after named Outlook.com. And Microsoft Account, which is the consumer ecosystem, we're all part of that. And our team, basically, helped them with detections and machine learning for their, their abusers and fraudsters and, and, you know, hackers that, that could affect their customers. So, my first role was actually in the spam team, anti-spam team. I was on outbound, outbound spam detection. So, uh, we will build models to detect when users who send spam from Outlook.com accounts out so we could stop that mail basically.


Nic Fillingham:

And I'd loved to know, like, the models that you were building and training and refining then to detect outbound spam, and then the kinds of sort of machine learning technology that you're, you're playing today. Is there any similarity? Or are they just worlds apart? I mean, we are talking seven years and, you know, seven years in technology may as well be, like, a century. But, you know, is there common threads, is there common learnings from back there, or is everything just changed?


Maria Puertos Calvo:

Yes, both. Like, there, there are, obviously, common threads. You know, the world has evolved, but what really has evolved is the, the, the underlying infrastructure and tools available for people to deploy machine learning models. Like, back then, we... The production machine learning models that were running either in, like, authentication systems, either in off- you know, offline in the background after the fact, or, or even for the... For the mail. The Microsoft developers have to go and, like, code the actual... Let's say that you use, like, I don't know, logistic regression, which is a very typical, easy, uh, machine learning algorithm, right? They had to, like, code that. They had to, you know... There wasn't like a... Like, library that they could call that they would say, "Okay, apply logistic regression to, to this data with these parameters.


Maria Puertos Calvo:

Back then, it was, like... People had to code their own machine learning algorithms from, like, the math that backs them, right? So, that was actually... Make things so much, you know, harder. They... There weren't, like, the tools to actually, like, do, like, data manipulation, visualization, modeling, tuning, the way that we have so many things today. So, that, you know, made things kind of hard. Nothing was... Nothing was, like, easy to use for the data scientists. It... There was a lot of work around, you know, how do you... Like, manual labor. It was like, "Okay, I'm gonna, like, run the model with these parameters, and then, like, you know, b- based on the results, you would change that and tweak it a little bit.


Maria Puertos Calvo:

Today, you have programs that do that for you. And, and then show you all the results in, like, a super cool graph that tells you, uh, you know, like, this is the exact parameters you need to use for maximizing this one, uh, you know, output. Like, if you want to maximize accuracy or precision or recall. That, that is just, like, so much easier.


Nic Fillingham:

That sounds really fascinating. So, Maria, you now... You now run a team. And I, I would love to sort of get your thoughts on what makes a great data scientist and, and what do you look for when you're hiring into, into your team or into sort of your, your broader organization under, uh, under identity. What perspectives and experience and skills are you trying to sort of add in and how do you find it?


Maria Puertos Calvo:

Oh, what a great question. Uh, something that I'm actually... That's... The, the answer of that is something I'm refining every day. The, you know, the more, uh, experience I get and the more people I hire. I, I feel like it's always a learning process. It's like, what works and what doesn't. You know, I try to be open-minded and not try to hire everybody to be like me. So, that's... I'm trying to learn from all the people that I hire that are good. Like, what are their, you know... What's, like, special about them that I should try to look in other people that I hire. But I would say, like, some common threads, I think, it's like... Really good communication skills.


Maria Puertos Calvo:

Like, o- obviously the basics of, you know, being... Having s- a strong background in statistical modeling and machine learning is key. Uh, but many people these days have that. The, the main knowledge is really important in our team because when you apply data science to cyber security, there are a lot of things that make the job really hard. One of them is the, the data is... What... It's called really imbalanced because there are mostly, most of the interactions with, with the system, most of the data represents good activities, and the bad activities are very few and hard to find. They're like maybe less than 1%. So, that makes it harder in general to, to, to get those detections.


Maria Puertos Calvo:

And the other problem is that you're in an adversarial environment, which means, you know, you're not detecting, you know, a crosswalk in, in a road. Like, it's a typical problem of, of computer vision these days. A crosswalk's gonna be a crosswalk today or tomorrow, but if I detect an attacker in the data today and then we enforce... We do something to stop that attacker or to... Or to get them detected, then the next day they might do things differently because they're going to adapt to what you're doing. So, you need to build machine learning models or detections that are robust enough that use, use what we call features or, or that look at data that it's not going to be easy... Easily gameable.


Maria Puertos Calvo:

And, and it's really easy to just say, "Oh, you know, there's an attack coming from, I don't know, like, pick a country, like, China. Let's just, like, make China more important in our algorithm." But, like, maybe tomorrow that same attacker just fakes IP addresses


Maria Puertos Calvo:

Addresses in, in a bot that, that is not in China. It's in, I don't know, in Spain. So, so, you just have to, you know, really get deep into, like, what it means to do data science in our own domain and, and, and gain that knowledge. So, that knowledge, for me, is, is important but it's also something that, that you can gain in the job. But then things like the ability to adapt and, and then also the ability to communicate with all their stakeholders what the data's actually telling us. Because it's, you know... You, you need to be able to tell a story with the data. You need to be able to present the data in a way that other people can understand it, or present the results of your research in, in a way that other people can understand it and really, uh, kind of buy your ideas or, or what you wanna express. And I think that that is really important as well.


Nic Fillingham:

I sort of wanted to touch on what role... Is there a place in data science for people that, that don't have a sort of traditional or an orthodox or a linear path into the field? Can you come from a different discipline? Can you come from sort of an informal education or background? Can you be self-taught? Can you come from a completely different industry? What, what sort of flexibility exists or should there exist for adding in sort of different perspectives and, and sort of diversity in, in this particular space of machine learning?


Maria Puertos Calvo:

Yes. There are... Actually, because it's such a new discipline, when I started at Microsoft, none of us started our degrees or our careers thinking that we wanted to go into data science. And my team had people who had, you know, degrees in economics, degrees in psychology, degrees in engineering, and then they had arrived to data science through, through different ways. I think data science is really like a fancy way of saying statistics. It's like big data statistics, right? It's like how do we, uh, model a lot of data to, like, tell us to do predictions, or, or tell us like what, how the data is distributed, or, or how different data based on different data points looks more like it's this category or this other category. So, it's all really, like, from the field of statistics.


Maria Puertos Calvo:

And statistics is used in any type of research, right? Like, when you... When people in medicine are doing studies or any other kind of social sciences are doing studies, they're using a lot of that, and, and they're more and more using, like, concepts that are really related to what we use in, in data science. So, in that sense, it's, it's really possible to come to a lot of different fields. Generally, the, the people who do really well as data scientists are people who have like a PhD and have then this type of, you know, researching i- but it doesn't really matter what field. I actually know that there, there are some companies out there that their job is to, like, get people that come out of PhD's programs, but they don't have like a... Like a very, you know, like you said, like a linear path to data science, and then, they kind of, like, do like a one year training thing to, like, make them data scientists, because they do have, like, the... All the background in terms of, like, the statistics and the knowledge of the algorithms and everything, but they... Maybe they're, they've been really academic and they're not... They don't maybe know programming or, or things that are more related to the tech or, or they're just don't know how to handle the data that is big.


Maria Puertos Calvo:

So, they get them ready for... To work in the industry, but the dat- you know, I've met a lot of them in, in, in, in my career, uh, people who have gone through these kind of programs, and some of them are PhDs in physics or any other field. So, that's pretty common. In the self-taught role, it's also very possible. I think people who, uh, maybe started as, like, software engineers, for example, and then there's so much content out there that is even free if you really wanna learn data science and machine learning. You can, you know, go from anything from Coursera to YouTube, uh, things that are free, things that are paid, but that you can actually gain great knowledge from people who are the best in the world at teaching this stuff. So, definitely possible to do it that way as well.


Nic Fillingham:

Awesome. Before we let you go, we talked about the perfect guacamole recipe last time because you had that in your Twitter profile.


Maria Puertos Calvo:

Mm-hmm (affirmative). (laughs)


Nic Fillingham:

Do you recall that? I'm not making this up, right? (laughs)


Maria Puertos Calvo:

I do. No. (laughs)


Nic Fillingham:

All right. So, w- so we had the perfect guacamole recipe. I wondered what was your perfect... I- is it like... I wanted to ask about tacos, like, what your thoughts were on tacos, but I, I don't wanna be rote. I don't wanna be, uh, too cliché. So, maybe is there another sort of food that you love that you would like to leave us with, your sort of perfect recipe?


Maria Puertos Calvo:

(laughs) That's really funny. I, I actually had tacos for lunch today. That is, uh... Yeah. (laughs)


Nic Fillingham:

You did? What... Tell me about it. What did you have?


Maria Puertos Calvo:

I didn't make them, though. I, I went out to eat them. Uh-


Nic Fillingham:

Were they awesome? Did you love them?


Maria Puertos Calvo:

They were really good, yeah. So, I think it's-


Nic Fillingham:

All right. Tell us about those tacos.


Maria Puertos Calvo:

Tacos is one of my favorite foods. But I actually have a taco recipe that I make that it's... I find it really good and really easy. So, it's shrimp tacos.


Nic Fillingham:

Okay. All right.


Maria Puertos Calvo:

So, it's, it's super easy. You just, like, marinate your shrimp in, like, a mix of lime, Chipotle... You know those, like, Chipotle chilis that come in a can and with, like, adobo sauce?


Nic Fillingham:

Yeah, the l- it's got like a little... It's like a half can. And in-


Maria Puertos Calvo:

Yeah, and it's, like, really dark, the sauce, and-


Nic Fillingham:

Really dark I think. And in my house, you open the can and you end up only using about a third of it and you go, "I'm gonna use this later," and then you put it in the fridge.


Maria Puertos Calvo:

Yes, and it's like-


Nic Fillingham:

And then it... And then you find it, like, six months later and it's evolved and it's semi-sentient. But I know exactly what you're talking about.


Maria Puertos Calvo:

Exactly. So that... You, you put, like, some of those... That, like, very smokey sauce that comes in that can or, or you can chop up some of the chili in there as well. And then lime and honey. And that's it. You marinate your shrimp in that and then you just, like, cook them in a pan. And then you put that in a tortilla, you know, like corn preferably. But you can use, you know, flour if that's your choice. Uh, and then you make your taco with the... That shrimp, and then you put, like... You, you pickle some sliced red onions very lightly with some lime juice and some salt, maybe for like 10 minutes. You put that on... You know, on your shrimp, and then you can put some shredded cabbage and some avocado, and ready to go. Delicious shrimp tacos for a week night.


Nic Fillingham:

Fascinating. I'm gonna try this recipe.


Maria Puertos Calvo:

Okay.


Nic Fillingham:

Sounds awesome.


Maria Puertos Calvo:

Let me know.


Nic Fillingham:

Maria, thank you again so much for your time. This has been fantastic having you back. The last question, I think it's super quick, are you hiring at the moment, and if so, where can folks go to learn about how they may end up potentially being on your team or, or being in your group somewhere?


Maria Puertos Calvo:

Yes, I am actually. Our team is doubling in size. I am hiring data scientists in Atlanta and in Dublin right now. So, we're gonna be, you know, a very, uh, worldly team, uh, 'cause I'm based in Seattle. So, if you go to Microsoft jobs and search in hashtag identity jobs, I think, uh, all my jobs should be listed there. Um, looking for, you know, data scientists, as I said, to work on fraud and, and cyber security and it's a... It's a great team. Hopefully, yeah, if you're... If that's something you're into, please, apply.


Nic Fillingham:

Awesome. We will put the link in the show notes. Thank you so much for your time. It's been a great conversation.


Maria Puertos Calvo:

Always a pleasure, Nic. Thank you so much.


Natalia Godyla:

Well, we had a great time unlocking insights into security, from research to Artificial Intelligence. Keep an eye out for our next episode.


Nic Fillingham:

And don't forget to tweet us @msftsecurity or email us at securityunlocked@microsoft.com with topics you'd like to hear on a future episode. Until then, stay safe.


Natalia Godyla:

Stay secure.

More Episodes

5/5/2021

Ready or Not, Here A.I. Come!

Ep. 26
Remember the goodoledays when wespent youthfulhours playing hide and seek with our friends in the park?Wellit turns out that game of hide and seek isn’t just for humans anymore.Researchers have begunputting A.I. to the test by having it play this favorite childhood gameover and overandhavingthe softwareoptimize its strategiesthrough automated reinforcement training.In today’s episode,hosts Nic Fillingham and Natalia Godyla speak with Christian Seifert and Joshua Neil about their blog postGamifying machine learning for stronger security and AI models,and how Microsoft is releasing this new open-sourcedcode to help it learn and grow.In This Episode, You Will Learn:What is Microsoft’sCyberBattleSim?What reinforcement learning is and how it is used in training A.I.How theOpenAIGym allowed for AI to be trained and rewarded for learningSome Questions We Ask:Is an A.I. threat actor science fiction or an incoming reality?What are the next steps in training the A.I.?WhowastheCyberBattleSimcreated for?ResourcesOpenAIHide and Seek:OpenAIPlays Hide and Seek…and BreaksTheGame! 🤖Joshua and Christian’sblog post:Gamifying Machine Learning for Stronger Security and AI ModelsChristian Seifert’sLinkedIn:https://www.linkedin.com/in/christian-seifert-phd-6080b51/Joshua Neil’sLinkedIn:https://www.linkedin.com/in/josh-neil/NicFillingham’sLinkedIn:https://www.linkedin.com/in/nicfill/NataliaGodyla’sLinkedIn:https://www.linkedin.com/in/nataliagodyla/Microsoft Security Blog: https://www.microsoft.com/security/blog/Transcript[Full transcript at https://aka.ms/securityunlockedep26]Nic Filingham:Hello and welcome to Security Unlocked! A new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft Security Engineering and Operations Teams. I'm Nic Filingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft Security. Deep dive into the newest threat intel, research, and data science.Nic Filingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security.Natalia Godyla:And now, let's unlock the pod.Nic Filingham:Hello, Natalia! Hello, listeners! Welcome to episode 26 of Security Unlocked. Natalia, how are you?Natalia Godyla:Thank you, Nic. And welcome to all our listeners for another episode of Security Unlocked. Today, we are chatting about gamifying machine learning, super cool, and we are joined by Christian Seifert and Joshua Neil who will share their research on building CyberBattleSim, which investigates how autonomous agents operate in a simulated enterprise environment by using high-level obstruction of computer networks and cyber-security concepts. I sounded very legit, but I did just read that directly from the blog. Nic Filingham:I was very impressed.Natalia Godyla:(laughs)Nic Filingham:If you had not said that you read that from the blog, I would've been like, "Wow". I would to like to subscribe to a newsletter. Natalia Godyla:(laughs)Nic Filingham:But this is a great conversation with, with Christian and Joshua. We talked about what is reinforcement learning. Sort of as a concept and how does that gonna apply to security. Josh and Christian also walked us through sort of why this project was created and it's really to try and get ahead of a future where, you know, malicious actors have access to some level of automated, autonomous tooling. Uh, and so, this is a new project to sort of see what a future might look like when there all these autonomous agents out there doing bad stuff in the cyber world.Natalia Godyla:And there are predecessors to this work, at least in other domains. So, they used a toolkit, a Python-based Open AI Gym interface to build this research project but there have been other applications in the past. OpenAI is, uh, well-known for a hide-and-seek. There is a video on YouTube that shows how the AI learned over time different ways to obstruct the agent and the simulated environment. Things like, blocking them off using some pieces of the wall or jumping over the wall.Nic Filingham:The only thing we should point out is that this CyberBattleSim is an open source project. It's up on GitHub and attained very much want researchers, and really anyone who's interested in this space to go and download it, go and run it, play around with it, and help make it better. And if you have feedback, let us know. There is contact information, uh, through the GitHub page but you can also contact us at Security Unlocked at Microsoft dot com and we can make sure you, uh, get in contact with the team. And with that, on with the pod?Natalia Godyla:On with the pod!Nic Filingham:Welcome to Security Unlocked, new guest, Christian Seifert. Thanks for joining us and welcome returning guest, Josh Neil, back to the podcast. Both of you, welcome. Thanks for being on Security Unlocked.Christian Seifert:Thanks for having us!Joshua Neil:And thanks, Nic.Nic Filingham:Christian, I think as a, as a new guest on the podcast, could we get a little introduction for our listeners? Tell us about, uh, what you do at Microsoft. Tell us about what a day to day look like for you.Christian Seifert:Sure, so I'm a, uh, research lead on the Security and Compliance team. So our overall research team supports a broad range of enterprising consumer products and services in the security space. My team in particular is focused on protecting users from a social engineering attack. So, uh, think of, like, fishing mails for instance. So we're supporting Microsoft Defender for Office and, um, Microsoft Edge browser.Nic Filingham:Got it, and Josh, folks are obviously familiar with you from previous episodes but a, a quick re-intro would be great. Joshua Neil:Thanks. I currently lead the Data Science team supporting Microsoft threat experts, which is our managed hunting service, as well as helping general res... cyber security research for the team.Nic Filingham:Fantastic, uh, again, thank you both for your time. So, today in the podcast, we're gonna talk about a blog post that came out earlier in this month, on April 8, called Gamifying Machine Learning for Stronger Security in AI Models, where you talk about a new project that has sort of just gone live called CyberBattleSim. First off, congratulations on maybe the coolest name? For, uh, sort of a security research project? So, like, I think, you know, just hats off there. I don't who came up with the name but, but great job on that. Second of all, you know, Christian if, if I could start with you, could you give us a sort of an introduction or an overview what is CyberBattleSim and what is discussed in this blog post?Christian Seifert:As I... before talking about the, the simulator, uh, the... let me, let me kind of take a step back and first talk about what we tried to accomplish here and, and why. So, if you think about the security space and, and machine learning in particular, a large portion of machine-learning systems utilized supervised, uh, classifiers. And here, essentially, what we have is, is kinda a labeled data set. So, uh, for example, a set of mails that we label as fish and good. And then, we extract, uh, threat-relevant features. Think of, like, maybe particular words in the body, or header values we believe that are well-suited to differentiate bad mails from good mails. And then our classifiers able to generalize and able to classify new mails that come in. Christian Seifert:There's a few, uh, aspects to consider here. So, first of all, the classifier generalizes based on the data that we present to it. So, it's not able to identify completely unknown mails. Christian Seifert:Second, is that usually a supervised classification approach is, is biased because we are programming, essentially, the, the classifier and what it, uh, should do. And we're utilizing domain expertise, red teaming to kind of figure out what our threat-relevant features, and so there's bias in that. Christian Seifert:And third, a classifier of who has needs to have the data in order to make an appropriate classification. So, if I have classifier that classifies fish mail based on the, the content of the mail but there is the threat-relevant features are in the header, then that classifier needs to have those values as well in order to make that classification. And so, my point is these classifiers are not well-suited to uncover the unknown unknowns. Anything that it has not seen, kinda new type of attack, it is really blind to it. It generalizes over data that, that we present to it. Christian Seifert:And so, what we try to do is to build a system that is able to uncover unknown attacks with the ultimate goal then to, of course, develop autonomous defensive component to defend against those attacks. So, that gives it a little bit of context on why we're pursuing this effort. And this was inspired by reinforcement learning research and the broader research community, mostly that is currently applied kinda in the gaming context. Christian Seifert:So OpenAI actually came out with a neat video a couple of years ago called Hide and Seek. Uh, that video is available on YouTube. I certainly encourage listeners to check it out, but basically it was a game of laser tag where you had a kinda, uh, a red team and a blue team, uh, play the game of laser tag against each other. And at first they, of course, randomly kind of shoot in the air and run around and there is really no order to the chaos. But eventually, that system learned that, “Hey, if a red team member shoots a blue team member, there's a reward.” and the blue team member also learned while running away from the red team member is, is probably a good thing to do. Christian Seifert:And so, OpenAI kinda, uh, established the system and had the blue team and the red team play against each other, and eventually what that led to is really neat strategies that you and I probably wouldn't have come up with. 'Cause what the AI system does, it explores the entire possible actions base and as result comes up with some unexpected strategies. So for instance, uh, there was a blue team member that kinda hid in a room and then a red team guy figured, “Hey, if I jump on a block then I can surf in that environment and get into the room and shoot the blue team member”. So that was a little bit an inspiration because we wanted to also uncover these unknown Christian Seifert:Unknownst in the security context.Nic Filingham:Got it. That's great context. Thank you Christian. I think I have seen that video, is that the one where one of the many unexpected outcomes was the, like, one of the, the, blue or red team players, like, managed to sort of, like, pick up walls and used them as shields and then create ramps to get into, like, hidden parts of the map? Uh, uh, am I thinking about the right video? Christian Seifert:Yes, that's the right video. Nic Filingham:Got it. So the whole idea was that that was an experiment in, in understanding how finding the unknown unknowns, using this game, sort of, this lazar tag, sort of, gaming space. Is, is that accurate?Christian Seifert:That's right, and so, they utilized reinforcement learning in order to train those agent. Another example is, uh, DeepMind's AlphaGo Zero, playing the game of Go, and, and here, again, kind of, two players, two AI systems that play against each other, and, over time, really develop new strategies on how to play the game of Go that, you know, humans players have, have not come up with. Christian Seifert:And it, eventually, lead to a system that achieved superhuman performance and able to beat the champion, Lisa Dole, and I think that was back in 2017. So, really inspiring work, both by OpenAI and DeepMind.Nic Filingham:Got it. I wonder, Josh, is there anything you'd like to- before we, sort of, jump into the content of the blog and, and CyberBattleSim, is there anything you'd like to add from your perspective to, to the context that Christian set us up on? Joshua Neil:Yeah. Thanks, Nic. I, I mean, I think we were really excited about this because... I think we all think this is a natural evolution of, of our adversaries, so, so, currently, our adversaries, the more sophisticated ones, are primarily using humans to attack our enterprises and, that means they're slow and they can make mistakes and they don't learn from the large amount of data that's there in terms of how to do attacks better, because they're humans.Joshua Neil:But I think it's natural, and we just see this, uh, everywhere and, all of technology is that people are bringing in, you know, methods to learn from the data and make decisions automatically, and it's- so it's a natural evolution to say that attackers will be writing code to create autonomous attack capabilities that learn while they're in the enterprise, that piece of software that's launched against the enterprise as an attack, will observe its environment and make decisions on the fly, automatically, from code. Joshua Neil:As a result, that's a frightening proposition because, I think the speed at which these attacks will proceed will be a lot, you know, a lot more quick, but also, being able to use the data to learn effective techniques that get around defenses, you know, we just see data science and machine learning and artificial intelligence doing this all over the place and it's very effective that the ability to consume a large amount of data and make decisions on it, that's what machine learning is all about. And so, we at Microsoft are interested in exploring this ourselves because we feel like the threat is coming and, well, let's get ahead of it, right? Let's go experiment with automated learning methods for attacks and, and obviously, in the end, for defense that, by implementing attack methods that learn, we then can implement defensive methods that will, that will preempt what the real adversaries are doing, eventually, against our customers.Joshua Neil:So, I think that's, sort of, a philosophical thing. And then, uh, I love the OpenAI Hide-and-Seek example because, you know, the analogy is; Imagine that instead of, they're in a room with, um, walls and, and stuff, they're on a computer network, and the computer network has machines, it has applications, it has email accounts, it has users, it's got a cloud applications, but, in the end, you know, an attacker is moving through an environment, getting blocked in various ways by defenses, learning about those blockings and detections and things and finding gaps that they can move through in, in very similar ways. So, I just, sort of, drawing that analogy back, Hide-and-Seek, it is what we're trying to do in cyber defense, you know, is, is Hide-and-Seek. And so the, I think the analogy is very strong.Nic Filingham:Josh, I just wanna quickly clarify on something that, that you said there. So, it sounds like what you're saying is that, while, sort of, automated AI-based attacking, attackers or attacking agents maybe aren't quite prevalent yet, they're, they're coming, and so, a big part of this work is about prepping for that and getting ahead of it. Is, is, is that correct?Joshua Neil:That's correct. I, I'm not aware of sophisticated attack machinery that's being launched against our enter- our customers yet. I haven't seen it, maybe others have. I think it's a natural thing, it's coming, and we better be ready.Christian Seifert:I mean, we , we see some of it already, uh, in terms of adversarial machine learning, where, uh, our machine learning systems are getting attacked, where, maybe the input is manipulated in a way that leads to a misclassification. Most of that is, is currently more, being explored in the research community.Natalia Godyla:How did you apply reinforcement learning? How did you build BattleSim? In the blog you described mapping, some of the core concepts of reinforcement learning to CyberBattleSIm, such as the environment, that action space, the observation space and the reward. Can you talk us through how you translated that to security?Christian Seifert:Yeah. So, so first let, let me talk about reinforcement learning to make sure, uh, listeners understand, kinda, how that works. So, as I mentioned, uh, earlier in the supervised case, we feed a label data set to a learner, uh, and then it able to generalize, and we reinforcement learning works very differently where, you have an agent that sits within an environment, and the agent is, essentially, able to generate the data itself by exploring that environment.Christian Seifert:So, think of an agent in a computer network, that agent could, first of all, scan the network to, maybe, uncover notes and then they're, maybe, uh, actions around interacting with the notes that it uncovers. And based on those interactions, the agent will, uh, receive a reward. That reward actually may be delayed by, like, there could be many, many steps that the agent has to take before the reward, uh, manifests itself. And so, that's, kinda, how the agent learns, it's, e- able to interact in that environment and then able to receive a reward. And so that's, kinda, what, uh, stands, uh, within the core of the, the CyberBAttleSim, because William Bloom, who is the, the brains behind the simulation, has created an environment that is compatible with, uh, common, uh, reinforcement learning tool sets, namely, the OpenAI Gym, that allows you to train agents in that environment.Christian Seifert:And so, the CyberBattleSim represents a simple computer network. So, think of a set of computer nodes, uh, the, the nodes represent a computer, um... Windows, Mac OS, sequel server, and then every node exposes a set of vulnerabilities that the agent could potentially exploit. And so, then, as, kind of, the agent is dropped into that environment, the agent needs to, first, uncover those nodes, so there's a set of actions that allows to explore the state space. Overall, the environment has a, a limited observability, as the agent gets dropped into the environment, you're not necessarily, uh, giving that agent the entire network topology, uh, the agent first needs to uncover that by exploring the network, exploiting nodes, from those nodes, further explore the network and, essentially, laterally move across the network to achieve a goal that we give it to receive that final reward, that allows the agent to learn.Natalia Godyla:And, if I understand correctly, many of the variables were predetermined, such as, the network topology and the vulnerabilities, and, in addition, you tested different environments with different set variables, so how did you determine the different environments that you would test and, within that particular environment, what factors were predetermined, and what those predetermined factors would be.Christian Seifert:So we, we determined that based on the domaine expertise that exists Christian Seifert:... is within the team, so we have, uh, security researchers that are on a Red Team that kind of do that on a day-to-day basis to penetration tests environments. And so, those folks provided input on how to structure that environment, what nodes should be represented, what vulnerabilities should be exposed, what actions the agent is able to take in- in terms of interacting and exploring that, uh, network. So our Red Team experts provided that information. Nic Filingham:I wonder, Christian, if you could confirm for me. So there are elements here in CyberBattleSim that are fixed and predetermined. What elements are not? And so, I guess my question here is if I am someone interfacing with the CyberBattleSim, what changes every time? How would you sorta define the game component in terms of what am I gonna have to try and do differently every time? Christian Seifert:So the- the CyberBattleSim is this parametrized, where you can start it up in a way that the network essentially stays constant over time. So you're able to train an agent. And so, the network size is- is something that is dynamic, that you can, uh, specify upon startup. And then also kinda the node composition, as well as ... So whether ... how many Windows 10 machines you have versus [inaudible 00:19:15] servers, as well as the type of vulnerabilities that are associated with each of those nodes. Nic Filingham:Got it. So every time you- you establish the simulation, it creates those parameters and sort of locks them for the duration of the simulation. But you don't know ... The agent doesn't know in advance what they will d- they will be. The agent has to go through those processes of discovery and reinforcement learning. Christian Seifert:Absolutely. And- and one- one tricky part within reinforcement learning is- is generalizability, right? When you train an agent on Network A, it may be able to learn how to outperform a Red Team member. But if you then change the network topology, the agent may completely flail and not able to perform very well at all and needs to kind of re- retrain again. And that- that's a common problem within the- the re- reinforcement learning research community. Natalia Godyla:In the blog you also noted a few opportunities for improvement, such as building a more realistic model of the simulation. The simplistic model served its purpose, but as you're opening the project to the broader community, it seems l- that you're endeavoring to partner with the other researchers to create a more realistic environment. Have you given some early thought as to how to potentially make the simulation more real over time? Christian Seifert:Absolutely. There is a long list of- of things that we, uh, need to think about. I mean, uh, network size is- is one component. Being able to simulate a- a regular user in that network environment, dynamic aspects of the network environment, where a node essentially is added to the network and then disappears from the network. Uh, all those components are currently not captured in the simulation as it stands today. And the regular user component is an important one because what you can imagine is if we have an attacker that is able to exploit the network and then you have a defender agent within that network as well, if there is no user component, you can very easily secure that network by essentially turning off all the nodes. Christian Seifert:So in- a defender agent needs to also optimize, uh, to keep the productivity of the users that are existing on the network high, which is currently not- not incorporated in- in the simulation. Nic Filingham:Oh, that's w- that's amazing. So there could be, you know, sort of a future iteration, sort of a n- network or environment productivity, like, score or- or even a dial, and you have to sort of keep it above a particular threshold while you are also thwarting the advances of the- of the agent. Christian Seifert:Absolutely. And I mean, that is, I think, a common trade off in the security space, right? There are certain security m-, uh, measures that- that make a network much more secure. Think of like two-factor authentication. But it does u- add some user friction, right? And so, today we're- we're walking that balance, but I'm hoping that there may be new strategies, not just on the attacker's side, but also on the defender's side, that we can uncover that is able to provide higher level of security while keeping productivity high. Nic Filingham:I think you- you- you have covered this, but I- I'd like to ask it again, just to sort of be crystal clear for our audience. So who is the CyberBattleSim for? Is it for Red Teams? Is it for Blue Teams? Is it for students that are, you know, learning about this space? Could you walk us through some of the types of, you know, people and- and roles that are gonna use CyberBattleSim?Christian Seifert:I mean, I think that the CyberBattleSim today is- is quite simplistic. It is a simulated environment. It is not ... It'-s it's modeled after a real world network, but it is far from being a real world network. So it's, uh, simplistic. It's simulated, which gives us some advantages in terms of, uh, scalability and that learning environment. And so at this point in time, I would say, uh the simulation is really geared towards, uh, the research community. There's a lot of research being done in reinforcement learning. A lot of research is focused on games. Because if you think about a game, that is just another simulated environment. And what we're intending to do here with- with some of the open source releases is really put the spotlight on the security problem. And we're hoping that the- the reinforcement learning researchers and the research community at large will pay more attention to this problem in the security domain. Nic Filingham:It's currently sort of more targeted, as you say, as- as researchers, as sort of a research tool. For it to be something that Red Teams and Blue Teams might want to look at adopting, is that somewhere on a road map. For example, if- if you had the ability to move it out of the simulation and into sort of a- a- a VM space or virtual space or perhaps add the ability for users to recreate their own network topology, is that somewhere on your- your wishlist? Christian Seifert:Absolutely. I think there's certainly the goal to eventually have these, uh, autonomous defensive agent deployed in real world environments. And so in order to get to that, simulation needs to become more and more realistic in order to achieve that. Joshua Neil:There's a lot of work to be done there. 'Cause reinforcement learning on graphs, big networks, i- is computationally e- expensive. And just a lot of raw research, mathematics and computing that needs to be done to get to that real- real world setting. And security research. And in incorporating the knowledge of these constraints and goals and rewards and things that ... T- that takes a lot of domain research and getting- getting the- the security situation realistic. So it's hard. Christian Seifert:In the simulation today, it provides the environment and ability for us to train a Red Team agent. So an agent that attacks the environment. Today, the defender is very simplistic, modeled probabilistically around cleaning up machines that have been exploited. So as kinda the next point on the wishlist is really getting to a point where we have the Red Team agent play against a Blue Team agent and kinda play back and forth and see kinda how that influences the dynamic of the game. Natalia Godyla:So Christian, you noted one of the advantages of the abstraction was that it wasn't directly applicable to the real world. And because it wasn't approved as a safeguard against nefarious actors who might use CyberBattleSim for the wrong reason. As you're thinking about the future of the project, how do you plan to mitigate this challenge as you drive towards more realism in the simulation? Christian Seifert:That is certainly a- a- a risk of this sort of research. I think we are still at the early stages, so I think that risk is- is really nonexistent as it stands right now. But I think it can become a risk as the simulation becomes more sophisticated and realistic. Now, we at Microsoft have the responsible AI effort that is being led at the corporate level that looks at, you know, safety, reliability, transparency, accountability, e- et cetera, as kind of principles that we need to incorporate into our AI systems. And we, early on, engaged the proper committees to help us shape the- the solution in a responsible fashion. And so at this point in time, there weren't really any concerns, but, uh, as the simulation evolves and becomes more realistic, I very much expect that we, Christian Seifert:... be, uh, need to employ particular safeguards to prevent abuse. Nic Filingham:And so without giving away the battle plan here, wh- what are some other avenues that are being, uh, explored here as part of this trying to get ahead of this eventual point in the future, where there are automated agents out there in the wild? Joshua Neil:This is the- the core effort that we're making, and it's hard enough. I'll also say I think it's important for security folks like us, especially Microsoft, to try hard things and to try to break new ground and innovation to protect our customers and really the world. And if we only focus on short-term product enhancements, the adversaries will continue to take advantage of our customers' enterprises, and we really do need to be taking these kind of risks. May not work. It's too ... It's really, really hard. And t- and doing and in- in purposefully endeavoring to- to- to tackle really hard problems is- is necessary to get to the next level of innovation that we have to get to. Christian Seifert:And let me add to that. Like, we have a lot of capabilities and expertise at Microsoft. But in the security space, there are many, many challenges. And so I don't think we can do it alone. Um, and so we also need to kinda put a spotlight on the problem and encourage the broader community to help solve these problems with us. And so there's a variety of efforts that we have pursued over the last, uh, couple of years to do exactly that. So, about two years ago we published a [inaudible 00:28:52] data science competition, where we provided a dataset to the broader community, with a problem around, uh, malware classification and machine risk identification and basically asked the community, "Hey, solve this problem." And there was, you know, prize money associated with it. But I really liked that approach because we have ... Again, we have a lot of d- expertise on the team, but we're also a little bit biased, right, in- in terms of kinda the type of people that we have, uh, and the expertise that we have. Christian Seifert:If you present a problem to the broader research community, you'll get a very different approaches on how people solve the problems. Most likely from com- kind of domains that are not security-related. Other example is an RFP. So we funded, uh, several research projects last year. I think it was, uh, $450,000 worth of research projects where, again, we kind of laid out, "Here are some problems that are of interest that we wanna put the spotlight on, and then support the- the research community p- to pursue research in that area." Nic Filingham:So what kind of ... You know, you talk about it being, uh, an area that we all sort of collectively have to contribute to and sort of get b- behind. Folks listening to the podcast right now, going and reading the blog. Would you like everyone to go and- and- and spin up CyberBattleSim and- and give it a shot, and then once they have ... Tell us about the- the types of work or feedback you'd like to see. So it's up on GitHub. What kind of contributions or- or feedback here are you looking for from- from the community? Christian Seifert:I mean, I'd really love to have, uh, reinforcement learning researchers that have done research in this space work with the CyberBattleSim. Kinda going back to the problem that I mentioned earlier, where how can we build agents that are generalizable in a way that they're able to operate on different network topology, different network configuration, I think is an- an- an exciting area, uh, that I'd love to see, uh, the research community tackle. Second portion is- is really enhancing the simulation. I mentioned a whole slew of features that I think would be beneficial to make it more realistic, and then also kinda tackle the problem of- of negatively impacting potential productivities of- of users that operate on that network. So enhancing the- the simulation itself is another aspect. Nic Filingham:Josh, anything you wanted to add to that? Joshua Neil:Yeah, I mean, I- I think those were the- the major audiences we're hoping for feedback from. But a- al- also like Christian said, if a psychologist comes and looks at this and has an idea, send us an email or something. You know, that multidisciplinary advantage we get from putting this out in the open means we're anticipating surprises. And we want those. We want that diversity of thought and approach. A physicist, "You know, this looks like a black hole and here's the m- ..." Who knows? You know, but that's- that's the kind of-Nic Filingham:Everything's a black hole to a physicist- Joshua Neil:(laughs) Yeah. Nic Filingham:... so that's, uh ... Joshua Neil:So, you know, I think that diversity of thinking is what we really solicit. Just take a look, yeah. Anybody listening. Download it. Play with it. Send us an email. We're doing this so that we get your- your ideas and thinking, for us and for the whole community. Because I think we- we also believe that good security, uh, next generation security is developed by everybody, not just Microsoft. And that there is a- there is a good reason to uplift all of humanity's capability to protect themselves, for Microsoft but for everybody, you know? Natalia Godyla:So Christian, what are the baseline results? How long does it take an agent to get to the desired outcome? Christian Seifert:So the s- simulation is designed in a way that also allows humans to play the game. So we had one of our Red Teamers to actually play the game and it took that person about 50 operations to compromise the entire network. Now when we take a- a random agent that kinda uninformed takes random actions on the network, it takes about 500 steps. So that's kind of the- the lower baseline for an agent. And then we trained, uh, a Deep Q, uh, reinforcement learning agent, and it was able to accomplish, uh, the human baseline after about 50, uh, training iterations. Again, network is quite simple. I wouldn't expect that to hold, uh, as kinda the- the simulation scales and becomes more complex, but that was, uh, certainly an encouraging first result. Joshua Neil:And I think the- the significant thing there is, even if the computer is- takes more steps than the human, well, we can make computers run fast, right? We can do millions of iterations way faster than a- than a human and they're cheaper than humans, et cetera. It's automation. Nic Filingham:Is there a point at which the automated agent gets too good, or- or is there sort of a ... What would actually be the definition of almost a failure in this experiment, to some degree? Joshua Neil:I think one- one is to- to sort of interpret your question as it could be overfed. That is, if it's too good, it's too specific and not generalized. And as soon as you throw some different set of constraints or network at it, it fails. So I think that's a- that's a real metric of the performances. Okay, it- it learned on this situation, but how well does it do on the next one? Nic Filingham:Is there anything else, uh, either of you would like to add before we wrap up here? I feel like I've covered a lot of ground. I'm gonna go download CyberBattleSim and- and try and work out how to execute it. But a- anything you'd like to add, Christian? Christian Seifert:No, not from me. It was, uh, great talking to you.Natalia Godyla:Well, thank you Josh and Christian, for joining us on the show today. It was a pleasure. Christian Seifert:Oh, thanks so much. Joshua Neil:Yeah, thanks so much. Lots of fun. Natalia Godyla:Well, we had a great time unlocking insights into security, from research to artificial intelligence. Keep an eye out for our next episode. Nic Filingham:And don't forget to tweet us at MSFTSecurity, or email us at securityunlocked@microsoft.com, with topics you'd like to hear on a future episode. Until then, stay safe. Natalia Godyla:Stay secure.
4/28/2021

Knowing Your Enemy: Anticipating Attackers’ Next Moves

Ep. 25
Anyonewho’severwatched boxing knows that great reflexes can be the difference between achampionshipbeltand a black eye.The flexing ofan opponent’s shoulder, the pivot of theirhip-a good boxer will know enoughnot only topredictand avoidthe incoming upper-cut, but willknow how to turn the attack back on theiropponent.Microsoft’s newestcapabilities in Defender puts cyber attackers in the ring and predicts theirnext attacks as the fight is happening.On today’s episode,hosts Nic Fillingham and Natalia Godyla speak with ColeSodja, Melissa Turcotte, and Justin Carroll(and maybe even a secret, fourth guest!)abouttheirblogposton Microsoft’s Security blogabout the new capabilities of using an A.I.to see the attacker’s next move.In This Episode, You Will Learn:• What kind of data is needed for this level of threat detection and prevention?• The crucial nature of probabilistic graphical modeling in this process• The synergistic relationship between the automated capabilities and the human analystSome Questions We Ask:• What kind of modeling is used and why?• What does the feedback loop between program and analyst look like?• What are the steps taken to identify these attacks?Resources:Justin, Melissa’s, and Cole’s blog post:https://www.microsoft.com/security/blog/2021/04/01/automating-threat-actor-tracking-understanding-attacker-behavior-for-intelligence-and-contextual-alerting/Justin Carroll’s LinkedIn:https://www.linkedin.com/in/justin-carroll-20616574/Melissa Turcotte’s LinkedIn:https://www.linkedin.com/in/mturcotte/ColeSodja’sLinkedIn:https://www.linkedin.com/in/cole-sodja-a255361b/Joshua Neil’s LinkedIn:https://www.linkedin.com/in/josh-neil/NicFillingham’sLinkedIn:https://www.linkedin.com/in/nicfill/NataliaGodyla’sLinkedIn:https://www.linkedin.com/in/nataliagodyla/Transcript[Full transcript at https://aka.ms/SecurityUnlockedEp25]Nic Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft Security engineering and operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories for Microsoft Security, deep dive into the newest threat intel, research and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security.Natalia Godyla:And now, let's unlock the pod. Welcome, everyone, to another episode of Security Unlocked, and hello, Nic, how's it going?Nic Fillingham:It's going well, good to see you on the other side of this Teams call. Although, you and I were in person not 24 hours ago. You were here in Seattle, we were filming some more episodes of the Security Show. I don't think we've really given listeners of the podcast a full, meaty introduction to the Security Show, have we? Do you wanna let listeners know what they can find?Natalia Godyla:We play games and hang out with experts in the industry and we've done everything from building robots with folks, to building blocks, to painting our nails. You can find the Security Show on our YouTube channel, so, YouTube.com/MicrosoftSecurity or you can go to aka.ms/securityshow. We talk with Chris Wysopal, the CTO and co-founder of Veracode on modern secure software development, and Dave Kennedy, who comes to talk to us about SecOps and everything you need for a survival kit in SecOps, so come come check them out.Nic Fillingham:Bad news is you, you have to deal with, uh, Natalia and I on another, uh, media format. But before you go there, make sure you listen to today's episode of Security Unlocked. We have a couple of returning guests. We have Cole and Justin, who have been on before, as well as Josh Neil, who comes on in the, in the last few minutes. And new guest, Melissa. They're all from the Microsoft 365 Defender research team, and they all co-authored a blog from April 1st called Automating Threat Actor Tracking, Understanding Attacker Behavior for Intelligence and Contextual Alerting, which is exactly what it is but I think it buries the lead. Natalia, you had a great TL;DR, what did they do?Natalia Godyla:The team used statistics to predict the threat actor group and the next stage in the attack and really early in the attack, so that we could identify the attack and inform customers so that they could stop it. I think what's really incredible here is, not only the ability to predict that information, but to just do it so early in kill chain. Nic Fillingham:Within two minutes after an attack begin, using this model, Microsoft threat experts were able to send a notification to the customer to let them know an attack was underway. The customer was able to do, you know, the necessary things to get that attack shut down. We'd love, as always, your feedback. Send us emails, securityunlocked@microsoft.com. Hit us up on the Twitters. On with the pod. Natalia Godyla:On with the pod. Nic Fillingham:Well, welcome back to the Security Unlocked podcast, Cole and Justin, and welcome to the Security Unlocked podcast, Melissa. Thanks for joining us today. We have three wonderful guests, with maybe a, a fourth special guest appearing at the end. And today we're gonna be talking about a blog post appearing on the Security blog from April the 1st, called Automating Threat Actor Tracking, Understanding Attacker Behavior for Intelligence and Contextual Alerting. All of the authors from that blog are here with us. Cole, if I could start with you, if you could sort of reintroduce yourself to the audience, give us a little bit, uh, about your role, what you do at Microsoft, and then perhaps hand off to one of your colleagues for the next intro.Cole Sodja:Sure. Will do, thank you. So, hi, I'm Cole. I work in the Microsoft 356 Defender group. I'm a statistician. Primarily my responsibilities are driving, kind of, research and innovation in general, with supporting threat analytics, threat hunting, threat research in general. Yeah, been doing that for about three years now, and I love it, and I that's a little bit about myself, I'll hand it over to Melissa. Melissa Turcotte:All right. My name's Melissa, I work with Cole, so in the same group, Microsoft 365 Defender. I'm also a statistician by background. I've been in the cyber domain for about probably seven years now. I was working for Department of Energy research laboratory in their cyber research group for five years, and I joined Microsoft a year ago. I like all sorts of problems related to cyber. My expertise probably would be in anomaly detection, but anything related to cyber, and there's data in a problem, I like to be involved.Nic Fillingham:And Justin.Justin Carroll:Hey. I also work in the Microsoft 365 Defender team, doing threat intelligence. My main focus is uncovering new threats and actor groups and understanding what they're doing, different modifications to how they're conducting their attacks, and the outcomes of those attacks, and then figuring out the most effective ways to either, communicate that out to customers or action on detection capabilities to stop them from succeeding.Nic Fillingham:Listeners of the podcast will note that you have a super sweet ninja turtles tattoo, is that correct? Justin Carroll:This is accurate, this is definitely accurate. Nic Fillingham:And, and we may or may not have a super secret fourth guest on this episode, who may join us towards the end, who you would, you would know from an very early episode of the podcast, but perhaps we'll keep them secret until the very end. Thank you all for joining us, thank you for your time. Again, we're referring to a, a blog post that, that all of you authored from April 1st. This is a, quite a complex, and, and sort of technical blog post, which I know a lot of our audience will love. Nic Fillingham:I got a little lost in the math, but I, I absolutely was enthralled by what you all have undertaken here. Cole, if I could start with you, can you give us, give us an overview of what's covered in this blog post, and sort of what this project was, how you tackled it, and what we're gonna talk about, uh, on this episode today.Cole Sodja:Yeah. So if I step back, being someone kind of still fairly new in learning, uh, to cyber security, uh, I approached things pretty much with just using data, right? Doing data driven imprints, as I'd say. And through my research, what I started to, um, kinda ask myself is, can we kinda get ahead of cyber security attacks, you know, from a post-breach perspective? Once we see an adversary in a network, can we start to make some predictions, basically, on what they're likely gonna do? Who is the adversary, or is it human operated, is it an automated script, for example. And then if we recognize the adversary, kinda recognize their tactics, their techniques, their procedures, can we say, okay, we're, we're likely gonna see they're gonna ransom this enterprise, for example.Cole Sodja:So I tried to look at it as more of a data mining exercise initially, it's like, can I recognize these type of patterns, and then how predictive are these patterns that we're seeing in terms of what likely is gonna occur. Or put it another way, what type of threat is this, essentially, to the enterprise? So, so that's kinda the background, the motivation. Now, when I started this project, back with Justin and then with Melissa, it started really as let's look for particular, uh, threat actors that we're aware of, that we recognize, that we know about, and see, like, can we start, from a data perspective, classifying okay, is it this group, is it that group, and what does this group tend to do? Cole Sodja:And one of the challenges in that is, is sparsity. Basically, we don't have a lot of labels sitting around out there saying, it's threat actor group A, B, C, D, and so on. We have handfuls of those. Some of these actors, they don't tend to do attacks very frequently, right? They're extremely sparse. So, so one challenge of this, and one the motivation is, how can we actually partner with threat intelligence, for example, and our threat hunters, to try and essentially encode or extract some of their information to help us build models, to help us reason over the uncertainty, essentially. Cole Sodja:And when we say probabilistic modeling, that's what we mean. It's how do we actually quantify this uncertainty, both in what we believe about the actors, or the adversaries in general, as well as what they're gonna do, right, once they've breached your network. So that's kinda how it started, and what this blog's really about is kinda giving a walk-through, essentially, of what we did initially with this research. It started with, and Justin will talk about this in a moment, it started with looking at few, select threat actors that are very serious. Cole Sodja:We started to understand their behaviors more and more and we thought it was a good opportunity, initially, to try and build a model to, again, understand what they're doing, track what they're doing, because they do change their tactics over time, as well as just see if we could get ahead of them. Can we actually notify a customer in advance, before, uh, for example, their organization's ransomed? So, so that's one part of the blog that we'll discuss, and I'll hand it over to my good friend Justin to take it from here.Justin Carroll:So, like, one of the, the main challenges that we kinda face in the intelligence sphere is understanding the particulars of an actor and when they are present in an environment. A lot of times, you'll see the intelligence is really focused on a very particular indicator such as, like, a known IP address that's malicious, or a single behavior. But it's kinda difficult to frequently pivot them out to understand when a suspected attacker is in an environment. A lot of that is due because they don't always do the exact same behaviors when they are compromising... Organization or device. There will be some variation and it basically requires manual enrichment a lot of the times of devices to try and understand the specifics of the attacks and what Justin Carroll:... the final outcomes o- wh- out of that attack, so this opportunity presented one to work with data scientists to, like, really supercharge our efforts so that we could kinda come in understanding a much bigger picture and knowing, essentially, what behaviors that we saw occur and then which ones we might suspect. A lot of times with these human operated ransomware ones, the time to alert, to notify of the expected outcome is often fairly short, in particular with, uh, one of the ones that we worked on to kinda test this method out. We had seen very short instances from time to compromise to ransom, so, um, this was to try and see if we could have a, a highly confident method of enriching that intelligence, um, and then working with other teams to get those alerts out.Natalia Godyla:If I could jump in here for a moment. So, at the beginning of your description, you noted that typically you'd use manual enrichment. Can you talk a little bit about that? So prior to this probabilistic model, how did you go through that manual enrichment process to try to, uh, predict what threat actors they were or determine what stage of an attack it was?Justin Carroll:It would be something along the lines of, let's say, you had intelligence from either a partner team or open source intelligence that says, you know, "These threat actors are using this IP address as part of their attack," and then looking for the presence of that and then finding out what actually occurred on those devices to understand the entirety of the attack, or looking more generically and saying, like, "Okay, we know these attackers like to use a particular behavior as part of their credential theft," and then so looking for all sorts of instances of that credential theft and then kinda continuing to pivot down into one that is leading to the behavior that y- you're looking for. One of the difficulties that you'll see in particular with this and other actors is, like, they will use multiple shared open source tools and payloads. Um, many of them aren't even malware, they're clean tools with legitimate purposes, so it can make it difficult to try and suss out the ones from malicious versus administrative use, so you have to look for that combination of different behaviors to indicate something malicious is afoot.Nic Fillingham:Justin, if I look at the blog, I think it might be the first chapter here, there's a MITRE ATT&CK framework diagram, Figure One, and it, uh, outlines sort of the steps taken here for how this model was able to, with high confidence, identify the, the actor and, uh, send an alert to the customer who was able to shut it down. I wonder if you could sort of, could you walk us through this, these sort of six steps as an example of, of how this work, how this worked in, in sort of real life?Justin Carroll:Yeah. I can walk through basically from a model's perspective, essentially, how it works. Timing, that's more a function of, like, how the attack, uh, typically progresses with this actor. Technically speaking, what the model's really doing is it's encoding each behavior we have, in this case, each MITRE technique in particular in terms of what's the confidence that once we see, for example, initial access follow... Under, let's say, RDP brute force, followed by lateral tool transfer with subset of tools recognized, that particular sequence right there, that's where the model would be like, "Okay, the probability that it's this particular threat actor group conditional on those two things occurring in sequence will be X," and that sequence could occur in a matter of minutes or even days and weeks, dependent on the actor, of course, we're talking about. Justin Carroll:With the, the actor we're showing in this graph, this actor typically will penetrate a network through RDP brute force, but then w- sometimes the, they won't immediately transfer their tools. They might wait a day or two, or sometimes they'll, they'll do it very fast, like, once they basically compromise a log-in then, uh, they'll, they'll go to that machine, there might be some, um, discovery related commands before they transfer or they might just transfer their tools and then that will be the attack box, basically, in which they stage their attack, and then they'll do some additional things.Justin Carroll:So at each step, basically, or each stage of the attack, as we like to call it, the model is basically gonna then update its probabilities and say, "Okay, based on all the information I've seen up to this stage, the probability that it's this actor is P and now, conditional that it's this actor with probability P, the probability that we'll now see, for example, defense evasion and this 'tack will be Q," or, or we could even go further in the attack stage to say, "Now, given all this, what's the probability that we'll see, for example, ransomware or inhibit system recovery in the coming hour? Or in the coming, you know, X time?" Justin Carroll:So the model's able to do that, but it's primarily conditional on the stages it's observed up to a point in time, not so much in terms of the time it takes for the actors to do X.Natalia Godyla:So, in this blog and in our discussion today, we're gearing up to talk about probabilistic graphical modeling as a way to address the challenge that, Cole and Justin, you've set up for us today, and, and for any of our listeners who'd like to follow along in the blog, the blog is titled "Automating threat actor tracking: Understanding attacker behavior for intelligence and contextual alerting" and you can find it on the Microsoft Security blog. I'd love to dive into the probabilistic graphical modeling and perhaps start with a definition of what that means. So, M- Melissa, could you give us an overview of this approach?Melissa Turcotte:Yeah. We have this problem which what they are essentially saying is, we have a collection of things which... I'm a statistician so I often call them variables, but, you know, features, if you will, if that's m- more easy for you to understand, but we, th- these TTPs, th- right. The sets of things that the actors are doing, and we have a collection of them. And given some collection of these, we wanna make a statement about whether or not it's ransomware or whether it's not a specific threat actor, or a group of actors. Right? And this is, this is, like, a perfect, um, example of where probability can help you make these decision, and one thing I'd like to stress is that no one of these features gives you enough information about whether or not it's this actor or this, this group of actors, or it's ransomware, you know, whatever your variable interest is.Melissa Turcotte:It really is the collection of these together that, you know, kind of in Justin's mind, as an analyst, he's, he's making these connections in his head, and I wanna be able to replicate that in some sense, I wanna take into account his knowledge and kind of his decision making process, combined with the data that I have, to make these probabilistic statements about what I think is happening. And graphical models are really great here, probabilistic graphical models in particular, as they kind of provide this joint probability distribution over all these features, and the variable of interest, in this case, is kind of, maybe is it this actor, but not necessarily. I mainly wanna know something about any one of these other features. I may already know it's this actor, and I may wanna be like, "Wh- what are the common things I see this actor do?"Melissa Turcotte:So, so graphical models really shine in this case where you have this collection of things that you are observing, and you kind of want to ask questions about any subset of them. Given some observations of others, and so th- this is a really great tool to use in this setting, and it's also quite interpretable. So if you kind of look, if you're looking at the blog and you see this Figure Two, which is a toy example, but y- you kind of, as a human, you can look at that and you can kind of understand that, "Okay, so I'm seeing transfer tools and lateral movement are related." Um, and you can kind of understand sort of wh- what the relationships the model is making. Um, and so that kind of provides this extra, you know, benefit of this in that, yeah, I can talk an analyst through what this kind of is showing and then i- it's quite interpretable for them even if they don't understand the underlying maths, and that's kind of something we really wanna strive for. Um, you shouldn't have to understand the underlying maths to kind of understand the decisions that are being made.Melissa Turcotte:It's really attractive in this sense, and then the Bayesian networks, why I really like it is kind of, the Bayesian paradigm is... So you, you have, you know, statistics, generally, or data science, you have some data and you're kind of, you know, making inference given the set of data to make statements about things of interest. So the data tells you something about your beliefs and the state of the world, but you have your own subjective beliefs about wh- what you think could and could not happen. The, the Bayesian paradigm kind of combines those two things, so it's, you have your beliefs and then you have what the data is telling you, a- and your ultimate kind of predictions are based on the combination of those things. And generally, the, the way it works is the more data you have, the data will always win through.Melissa Turcotte:So this problem, bringing it back to attacker prediction, is a case where we don't have a lot of data, right? We don't... Companies get attacked... Or we say, companies get attacked all the time but not at the scale at which we collect the underlying data, so like, you know, we have, you know, you as a user are performing actions, logging into computers you use... You know, this shows up in the data thousands of times a day, whereas an attack happens kind of, like, on a monthly scale, so c- the scales of attacks to the data we're getting is just really small, and then when you go into attacks that kind of we've labeled as being attributed to a threat actor, I mean, that's even way smaller. So it's, it's kind of a small data problem, uh, in terms of the number of labels you have.Melissa Turcotte:But what we do have is this analysts who have spent years tracking these people and have their kind of, you know, beliefs about what they do and how they changed over time. And so we Melissa Turcotte:Wanna capture that. We definitely want to include the evidence we see and the data, but we wanna capture that really rich knowledge that we get from the analysts. And so kind of that's where the Bayesian network part becomes attractive because it, it provides a very principled way to, to capture the analysts' expertise, combine that information with the data we're seeing to make these ultimate predictions.Natalia Godyla:For our audience, could you really quickly describe a Bayesian network?Melissa Turcotte:So, a Bayesian network is a way of building a model for a collection of variables whereby the idea is that you have different variables which are related to each other. It, it, it kind of helps draw out or show what those relationships are so, like, in the graph, you know, if there's an arrow from impact... Or from transfer tools to impact that's saying if I see transfer tools, that has a direct impact... I'm gonna use the word impact twice here. Has a direct impact on whether or not I'm going to see impact. So, so it's kind of the way the variables relate to each other and the way the probabilities change according to those relationships. And so a Bayesian network encodes all this information. Nic Fillingham:If I can take another swing at that one... Thank you, Melissa. I'm wondering what were some of the other, uh, techniques that you either considered for this approach? Like, did you experiment with other methods and then ultimately chose Bayesian?Melissa Turcotte:Yes, um, in fact, uh, so the initial kind of... The perhaps most obvious thing to do is to c- to think of decision trees, right? You s- you're, you're, you're seeing, you know, these things over time. Okay, I saw, um, what was the first one? Initial access with this... You don't go as broad as initial access, but I saw initial access using this, you know, minor technique. And so you can kind of think, like, you, you, you have a tree that's kind of... Okay, I saw this, I didn't see this, but I saw this and I didn't see this, so now I think it's this actor. But kind of where this is preferable is the fact that, as Paul says, we don't want to see the whole attack happen before we make a statement about what we think it is. And Bayesian networks work really well in, in the absence of some observed variables. Cole Sodja:Yeah, I'll just quickly chime in. I agree with Melissa. So, I did experiments, for example, with several models including decision trees. Even, um, different forms of Bayesian decision trees like BART for example. And in addition to what Melissa is saying where, for example, predicting the probability that it's threat actor conditioned on certain variables we saw, uh, we might also, as Melissa pointed out, want to say, okay, let's predict, for example, that this threat actor is going to do impact or a certain form of impact. And with decision trees, that means basically you're building multiple decision trees to do that. You can't just build one decision tree... Well, let's put it this way. You can't easily build one decision tree to have multiple target variables. That's something you get for free with the Bayesian network. Another thing I'll say in addition to what, um... To marginalization is the Bayesian network is more general. So, it could actually handle kind of a broader graphical structure. The decision tree is a specific graph. Cole Sodja:So, it kind of already inhibits you, if you will, to learning a certain structure over the data. Whereas the Bayesian nets, they could give you a little more general structure. We could also build these models that are time dependent, what are called dynamic Bayesian networks. That's something much harder to do with tree models. So, it's just a more flexible model as well as I would say. In my experiments, the Bayesian network did perform better on average than the set of decision trees I considered.Nic Fillingham:I'd like to better understand the relationship between this model and folks like Justin. So, is Justin, as a very experienced threat analyst, is Justin helping you define labels and helping you sort of build some of the initial... I'm, gonna get the taxonomy wrong here, so please correct me. But the initial sort of properties of the model? Or is, is Justin, as an analyst, interpreting what you sort of think you have in the model? How, how do I understand the relationship between the analyst and, and how they're providing their expertise into, into this model?Melissa Turcotte:All three.Nic Fillingham:Oh, great. (laughs)Melissa Turcotte:All three things you said is actually correct. So, so hopefully we, we've explained it somewhat well. So, yes. The first stage, right Justin? The analysts are providing us our label data. So, yes. That's the first thing. And then they also help us kind of, you know, you have the raw data, but that's kind of... There's so much data processing that goes... That, that happens before it's kind of... This data's kind of in this tabular forms that's like, yes, we... You know, these are the features we are tracking, so think of your TTPs, the different notes in your graph. Getting the data into that, kind of that schema, the threat analysts help with. So, you know, help define what, what these tactics, techniques, and procedures are that we should track. Like you said, you, you can't be super broad. Lateral movement doesn't really have a lot of meaning, um, to kind of like the different ways in which someone can do lateral movement and how granular w- you want to go. Melissa Turcotte:So, we discuss with the analysts all the time to kind of build up, you know, the ontology, if you will. And then, you know, as a first stage, like I said, it's a small data sample, so we're like... Justin helps inform what the model thinks about in a probabilistic sense. So, you... One thing I might ask him, I, I would be like... If I saw net... you know I'm borrowing from our toy example, but if I saw a network scanning modify system process, transfer tools, but didn't see any of the others, do you think it would be this actor X? Or do you think it would be ransomware? And he would be like, hmm, I would probably 60% certain. I can take that information and encode that directly so that, in the absence of any data, the model would return 60%. It would... If I didn't see any data, it would return what Justin believed was the probability in the presence of a certain number of variables. Melissa Turcotte:And then we kind of see data and we update our beliefs over time based on that. And then, also, after we've kind of trained these things, I go back to Justin and say does this make sense to you? So, he, he's kind of involved in all three, the whole process.Nic Fillingham:Melissa, I think you're telling me you've built a virtual Justin. Melissa Turcotte:We... That, that is what we are literally trying to do. And back it up... And, you know, and back it up with data as well. I'd, I'd like to like... You know, I'm a firm believer that everyone has their subjective beliefs, Justin has beliefs as well. Oftentimes, I can prove analysts wrong. Be like, they think something, I'm like, well, the data is telling me something else. So, we need to figure out, you know, that discrepancy. But, yes. We are essentially trying to build virtual Jus- uh, Justins. Although, like, th- there... I don't think there's any stage upon which we won't need the analysts to constantly feed back in with the new information they have. Nic Fillingham:Got it. And then can it come full circle? Justin, how do you as an analyst, how do you get smarter and better at what you do by what this model is, is telling you? What's the feedback loop look like here for you?Justin Carroll:It's one of those where, basically, using the model kind of super-charged my abilities where, instead of having to look at this very granular kind of like ad hoc, oh, this may be interesting, now I have the instances already serviced to me, and I have a good understanding of what success rate through the kill chain the attacker was able to get. And maybe figure out which ones that I needed to enrich more to understand was there data that we can add into the model because they've done something different that we need to capture and then look for opportunities in that way. So, really, it's basically... It made it where, give or take, sometimes it would take anywhere from 10 to 20 minutes sometimes to try and figure out, like, is this who I think it is? And like, what have they done? What are their goals? To just looking at the result from the model. And within usually seconds, being like, yeah, that looks exactly right. That's... It's confirmed, I think that's spot on. Natalia Godyla:So, Justin, was there something that was the most surprising in working with this model? Something that the model taught you either about threat actors or any details about the features? Justin Carroll:One of the things was kind of reexamining My confidence levels on different parts of the attack. Um, where Melissa was stating, for instance, you know, the data suggesting this and the models coming to this conclusion, uh, you know, thinking that it's this probability, and there would be times where I'd have to kind of reevaluate and think, like, hmm, I might've been missing something or overestimating the prevalence of a particular thing and saying it's related to such. Like, uh, I can tend to get very biased based on my narrow scope of the attacks that I'm looking at and think that it's related to this thing, but the model was able to provide a lot of clarity to some of the behaviors that maybe I didn't think were as confident a signal or extremely confident signal and I wasn't giving them the appropriate weight. That's one of the advantages of using it to understand what the attacker's doing, is I let it do much of the leg work once everything's kind of coded in. And then occasionally, like if we found opportunities where it was like, hmm, this still isn't quite right, then it could be tuned as a c- um, as necessary. Justin Carroll:I think that was probably one of the biggest ones of kind of trying to work through and actually spell out, like, my own thinking processes when I'm evaluating the data. It was something that you just kind of do without thinking, where you're constantly, as an intelligence analyst, looking at data and making conclusions on that data. But you're not usually saying, like, okay, I saw this so I'm gonna give it a 60% probability that it's this. And like, you're, you're just kind of sometimes it's either gut intuition or working on it that way. But actually having the model encode and return back what it was understanding made a, a pretty big impact in trying to understand how my own decision processes work and basically how best to kind of think Justin Carroll:About these different, wide array of attacks that we're constantly investigating.Nic Fillingham:The types of indicators that you're building this model on, again please correct me on my taxonomy here, but you're not looking for, you know, NFO files or like ASCII art or, you know, the actual threat actors name being sort of hidden somewhere in the jpeg that they drop as a, as a for the LOLs, like, they're... You're not looking for a sort of a literal signature of these threat actor groups, you're, you're, what you're, what you're doing is you're, you're seeing the actions that have been taken and without any other way of attributing them to an individual group, you're piecing them together. Nic Fillingham:And as you, as you get more actions and you piece them together based on the, the labels that you get from people like Justin, you're able to, to ultimately have a high probability that it's this threat group actor and they're doing this thing and they're likely to do this thing next. Have I got that right? You're, they're... In no way shape or form are you actually finding a secret text file that has the name, you know, the, the, the handles for all the hackers who are doing it for the LOLs.Cole Sodja:So let me just quickly jump in, you pretty much nailed it. I'll say this, so, we wanted to do both actually, right, because we don't want to restrain the model if it's, if core's gonna add predictive power, so like you said, we're not actually searching, grepping for example, for a threat actor name and some file or image, certainly not that level. But, for example, some of the actors, maybe they have common infrastructure, maybe they use particular types of tools in their attack typically, right? Like, maybe there's a SHA-1 out there they've used a lot in their attack, or, or recurring IP addresses they use as part of brute forcing. Cole Sodja:Those are there, but those are very specific and if you just relied on those, like Melissa was saying, either one or a few of those, you're not gonna generalize. You'll probably miss that attacker, right? But we certainly don't want to exclude it from the model because, um, if we happen to see that, the model will, uh, come back with a different type of probability, right? It'd be like, okay. Now the model might be more confident early, rather than waiting to see how the rest of the kill chain progresses. On the more general side, we probably won't go to the MITRE categories, 'cause they're a little too general, right? But if we go to some of the sub techniques, we don't actually have to look at the particular types of executables, or tools, or IPs used. Cole Sodja:Sometimes just the timing and sequencing is enough actually, to narrow down to, maybe not a particular threat actor, but a group of actors or, more generally, we can say with high competence, you know, this is a human adversary. They're taking this amount of time to do discovery commands, they're, they're doing lateral these type of ways. And the model could recognize that, even without knowing the particular commands, it's just seeing the more general techniques involved, right? So we do a bit of both, actually. We tend to want to rely more on, kind of, the general attacks or indicators as you're saying, that's right. But, we certainly don't want to throw away specifics that are reuse because we could get ahead of the attack much earlier too. So it's a bit of both at the end of the day.Melissa Turcotte:So yes, Nic, if, if, if you have an evil bit, look for the evil bit. You don't need data science for that. Nic Fillingham:(laughs)Natalia Godyla:And how is this model being used today, meaning is this a model that's being used by our internal security team to protect Microsoft and its customers, is it being used by a Microsoft threat experts group or is this actually embedded in some of our solutions today, and our customers are feeling that benefit? And what is the future intent of the model?Justin Carroll:One of those... So, there are multiple uses that are in place for the model. So one of the big things for me, so in my own selfish interest, it's intelligence, it's one of the easiest ways that I can keep tabs on the attacker and continually build new profiles and understand, basically, reports out, this is what they're doing, this is how they're doing it, this is how active they are. Like, are we seeing, you know, large volumes of their attack, are they taking a break, that kinda stuff. Then, the Microsoft threat experts are using it as a signal to help understand attacks early on in the kill chain so that they can get those notifications out ideally before the ransom, which can be quite difficult a lot of the times depending on the adversary and how quickly they seek to ransom. A lot of times there isn't a great deal of time.Cole Sodja:Yeah, there's other products, for example, M365D. So, um, there are plans, uh, it requires some engineering, ultimately, because this is a big product, um, huge customer base and so on. But there are already plans in motion to take what we've built already, as part of this framework, and integrate that into that product. There's other products as well, both from a threat intelligence perspective, and possibly kind of from SOC alerting perspective as well, that I'm in active discussions with other products across Microsoft to do the POC, make sure it works with their data, make sure they're comfortable and then work with their engineering team to at least get that in the plan. Those are ongoing discussion but M365D does have, kinda, I'll say, in their planning cycle, to get this in the product. Nic Fillingham:I wonder if this might be a good time to bring our secret special guest on microphone, Josh, if you're there, I think I might ask, uh, might wonder if you could jump in on this one. I think you've understated the power of what you've built here. From everything that you've just explained, you know, within a couple of minutes of a threat actor getting initial access to have a high probability index to be able to contact the customer and say, here's who we think is inside your network, here's what we think they're gonna do next, so they can shut it down. This is the next level, right? And, and Josh, when we interviewed you on episode three, you were hinting at this, if I'm not mistaken. Is this, is this sort of what you guys have been working on?Joshua Neil:Yeah, I'm so proud that we, that we took it from concept to realized value for the customers and, and at this point we've had that impact with your customers in stopping human operations. And, and so it's really exciting and, and it's, it's on the journey but, you know, if I extract an overall theme from this, it's consistent with that podcast that we had before because I was sort of complaining about AI. And I was sort of complaining about what we see in some of the, in some of the branding and marketing that, that folks do in, in cyber security. And I think this team and, and the work they've done exemplifies the right applications of data driven methods. Joshua Neil:There is no magical, artificial intelligence today. What there is is, and this is a, an experience that all of us on the data science team have had over the, over the past few years, and really for me about 20 years, is we can use data and some mathematics and some computing to begin to automate and accelerate what the humans are doing. And so, by sitting very closely with, and working very hard with the human experts like Justin, we're explicitly encoding their knowledge into models. So that's one thing is that the data science we're doing is to automate some of the stuff they're doing today. But the intention is not to solve the world, not to give our customers a license to solve security, we're, we're not gonna be able to do that. What we are able to do is uplift the sophistication of our customers operations. Joshua Neil:So, you know, what Justin sort of reflected on, uh, he's able to do a more interesting job, a more sophisticated job, because we're taking the data and his knowledge and encoding it and accelerating and automating some of the stuff that he's having to do manually now. And that's where the real nuts and bolts, you know, and the real rubber meets the road here, is that there's no magic gun that's gonna blow away all the adversaries with, with AI. What there is is hard work between data scientists and threat expertise to uplift their capabilities and accelerate their effectiveness in the face of the adversary. And that's what I would like to get across to the, to the listeners, is that by hard work and careful and close collaboration between data science and threat expertise, that's how we really make progress in this space.Nic Fillingham:Thank you so much Josh. And I just wanted to quickly clarify, from a previous comment from Cole, so this model is in use now, correct? Folks like Justin, Microsoft threat analysts, they are using this model now to make the model better, and to be able to get that additional information and those confidence levels in, in, in doing their analyst work. And so Microsoft threat expert customers are directly benefiting from this work, as of today. That's correct, is it?Joshua Neil:That's correct. We've sent targeted attack notifications to customers based on this model.Nic Fillingham:You've all been very, very, generous. Natalia Godyla:Thank you for that. And, and thank you to the whole team here for joining us on the show today. Melissa Turcotte:Absolutely.Cole Sodja:My pleasure.Joshua Neil:It was a lot of fun as always. And, and thank you, Nic and Natalia for this.Natalia Godyla:Well, we had a great time unlocking insights into security, from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us at MSFTSecurity or email us at securityunlocked@microsoft.com with topics you'd like to hear on future episode. Until then, stay safe...Natalia Godyla:Stay secure.
4/21/2021

Below the OS: UEFI Scanning in Defender

Ep. 24
All of us have seen– or at least, are familiar with – the antics of Tom and Jerry or Road Runner and Wile E. Coyote. In each one the coyote or the cat set up these elaborate plans to sabotage their foe, but time and time again, the nimble mouse and the speedy birdareable tooutsmart their attackers.In our thirdepisode discussing Ensuring Firmware Security,hosts Nic Fillingham and Natalia Godylaspeak withShweta JhaandGowtham Reddyabout developing thetoolsthat allow for them to stay one step ahead ofcybercriminals in the cat & mouse game that is cyber security.In this Episode You Will Learn:• Thenewcapabilities within MicrosoftDefenderto scan theUnified Extensible Firmware Interface (UEFI)• How theLoJaxattack compromised UEFI firmware • How UEFI scanning emerged as a capabilitySome Questions that We Ask:• Has UEFI scanning always been possible?• What types of signals is UEFI scanning searching for?• What are the ways bad actors may adjust to avoid UEFI scanning?Resources:Shweta Jha’sLinkedIn:https://www.linkedin.com/in/jhashweta/Gowtham Reddy’sLinkedIn:https://www.linkedin.com/in/gowtham-animi/Defender Blog Post:https://www.microsoft.com/security/blog/2020/06/17/uefi-scanner-brings-microsoft-defender-atp-protection-to-a-new-level/NicFillingham’sLinkedIn:https://www.linkedin.com/in/nicfill/NataliaGodyla’sLinkedIn:https://www.linkedin.com/in/nataliagodyla/Transcript[Full transcript can be found at https://aka.ms/SecurityUnlockedEp24]Nic Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft's security, engineering, and operations teams. I'm Nic Fillingham-Natalia Godyla:And I'm Natalia Godyla. In each episode we'll discuss the latest stories from Microsoft Security, deep dive into the newest threat intel, research and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security.Natalia Godyla:And now, let's unlock the pod.Natalia Godyla:Hello Nic. Welcome to Episode 24. How's it going with you today?Nic Fillingham:Going well, thank you, Natalia. Yes, uh, welcome to you and welcome to listeners to Episode 24 of Security Unlocked. On today's podcast, we speak with Shweta Jha and Gowtham Reddy from the Microsoft Defender for Endpoint engineering team about capabilities in MDE to scan down into the UEFI layer. Now this is the third of three conversations we have that started back in Episode 11 with Nazmus Sakib where we talked about secure core PCs and, and firmware integrity. Then in Episode 14 we spoke with Peter Waxman about the Pluton processor and some of the new work that's happening there to imbed more security tech into sorta silicon onto the actual CPU die itself. And today we're sort of rounding that conversation out with Shweta and Gowtham to talk about how Microsoft Defender for Endpoint can now scan down or can scan down into the UEFI layer. You're gonna hear a bunch jargon, a bunch of technical terms like, I guess, UEFI. That's, we, we could start there.Natalia Godyla:Yes. And UEFI is the Unified Extensible Firmware Interface, so it is the software interface that lies between an operating system and firmware, and is an evolution of BIOS. And we'll also talk about MosaicRegressor which, for those of you that don't know, is the second ever UEFI rootkit which was discovered in 2020, but was used in an attack against NGOs in 2019. And, and for me, the timeline is shocking, second ever in the past year. Normally we hear about the continuous increase of a certain type of attack over the years, and here we're just at the second ever.Nic Fillingham:Yeah. It's a real interesting part of the conversation where we talk about the history of BIOS attacks, firmware attacks, UEFI attacks, and to learn that this has been sort of traditionally a pretty challenging area for attackers to, to breech and compromise. But, you know, Shweta and Gowtham have been, you know, very much ahead of the curve and, and being ahead of, of attackers in, in being able to develop these new capabilities to, from the operating system, scan down to the UEFI layer and look for malware, look for compromise. And it's a, it's a fascinating conversation. Again, it's sort of a completion of three episodes starting with Episode 11 and 14. So if you haven't listened to those, I recommend you add them to the queue. But I guess on with the pod.Natalia Godyla:On with the pod.Nic Fillingham:Welcome to the Security Unlocked podcast. Shweta Jha and Gowtham Reddy, welcome both of you. Thanks for being here.Gowtham Reddy:Thank you.Shweta Jha:Thank you so much for having us. We're so very excited.Nic Fillingham:I'm very excited, too. Now this is gonna be the third conversation in a sort of a mini series that we're running here on the podcast. We started with Nazmus Sakib who introduced us to the idea of secure core PCs and we talked about some of the challenges of firmware integrity and keeping firmware safe. Then we spoke Peter Waxman in another episode to learn about Pluton, the history of, of that technology and sort of what's coming for the Pluton processor. And today we're actually gonna talk about some new capabilities, or newish as of 2020, in Defender to scan down into the UEFI layer. Before we jump into to all that, let's just do some introductions for the audience. Shweta if we could start with you. Who are you? What is your role? What do you do day-to-day at Microsoft? Tell us, what you like the audience to know about you?Shweta Jha:Absolutely. Thank you, Nic. My name Shweta Jha. I am a program manager with Microsoft Defender for Endpoints, and I've been building security solutions features and products, and I'm super excited about it because security is the need today for our, uh, customers. And a few of the features that I built with my team were part of anti-tampering. Investment that we did, EDR block as part of be able to blocking and containment. And then we are gonna talk a lot about UEFI scanner. So pretty much around building security solution and features in this team and helping our customers.Nic Fillingham:Fantastic. And, and Gowtham, welcome to the podcast. If you could also introduce yourself. Uh, tell us about your role. What does your day-to-day look like?Gowtham Reddy:Hi. This is Gowtham Reddy. I'm an engineering manager in Microsoft Defender, uh, Endpoint. So before engineering manager, so I was working as an engineer in the same team for last six years. So I work on, uh, many of the rootkit technologies, the Defender, uh, has and, uh, the remediation technologies to remediate many of the malwares that are present on the system. I have been where I working on this fantastic team, developing like durable protection features that were, and compliment the ever changing malware fields.Nic Fillingham:That's great. So, again, welcome to both of you. Thanks for your time. One of the things we do on the, uh, Security Unlocked podcast here is we, we don't necessarily cover the latest announcements. We, we sort of look back over the last sort of three to six months for interesting sort of technology, interesting advancements in, in security technology, and we bring experts on to, to talk about these new features and capabilities after them sort of being in the wild. Today we're talking about the UEFI scanning capabilities that are in Microsoft Defender, and there's a blog post that, that both of you helped author back in, in June of 2020, which feels like a decade ago, but I guess it's more like six or seven months. So I wondered if one of you might be able to just walk us through. What was that announcement made in that blog post? What was sort of the news? And then I think maybe if the other one or maybe just following, I'll, I'll leave it to how we, how we split this up. But what was announced back in June? And sort of what's happened since then? How have those new capabilities sort of rolled out and what are we seeing with customers actually using them?Shweta Jha:So I, I guess I can get us started, and then I'll hand it over to Gowtham definitely to talk more on the technical details and the, the attacks that we see in the wild, and that's why we kind of built this UEFI scanner. So as you understand, this is a journey, right? We built a layered defense in that security solutions. And when we build any security solution, we need to make sure that we take a holistic approach. So if you look at the operating level of security solutions, we've been getting pretty great at operating level security solutions. And it's not only Microsoft. If you see other security providers as well, they have been doing great, too.Shweta Jha:So what does that mean? It means that because the operating system level security solution is really great, it does making difficult for attackers to not get detected at that level. It's a constant battle, so they have been looking into other means where they can go into the system undetected, and that's where if you look at the data, you would find that in recent past the attacks across hardware and firmware level has been on the rise. So we built UEFI scanner keeping in mind that we should be able to detect those type of attacks, because those type of attacks are not only very dangerous, but often time they are not detected. They persist even if you reboot the system. So the nature of these type of attacks is very dangerous, and keeping that in mind, we decided to build UEFI scanner.Gowtham Reddy:So I can add like why we did, uh, build the UEFI scanner. So because of the operating system security features that Microsoft is constantly working on, the bad guys are trying to go in, down and down in the layered architecture. And so some of the traits of the ia64 went onto the BIOS, tampering the BIOS and, uh, tampering the MBR, the master board required and, uh, VBR based bootkits. So Defender has evolved into that space of counting the MBR and, uh, detecting the bootkits and the boot time. Gowtham Reddy:So as a logical evolution the bad guys are, uh, from the stage of Colonel to the MBR, MBR to the UEFI. So we were anticipating that this kind of evolution is quite possible and the UEFI implants were not very far. So that's the time we found the first UEFI implant called LoJax. So that was a triggering point when we completely committed to ourselves to expand our root kit technology, to detect any kind of rotates presence in the UV. So that was our core idea of expanding or rotating to the layer much below the operating system. So there were some challenges Natalia Godyla:If you don't mind me jumping in, I had a question around that. So...Gowtham Reddy:Mm-hmm (affirmative)Natalia Godyla:... the way you're framing it is that when we started to notice the threat landscape moved to this layer, we decided to invest in this type of technology. What about the technology itself? Had there always been this opportunity to tackle UEFI scanning, or is there something new that we're leveraging in order to solve this problem Now that might not have been around beforehand? Gowtham Reddy:That's a good question. So there was always a chance to exploit the UEFI, but it's about the timing of the attackers to get at target this space because the rest of the platform and ecosystem is getting more and more secure. So the UEFI is not new. So it was there a decade ago, but the implants are new because of the advances in the operating system. Nic Fillingham:So Gowtham, tell us about the LoJax attack that happened. Was it the first or it was one of the first detected compromises of the UEFI firmware? Can you tell us some more about, about that? If folks aren't familiar with it like me? Gowtham Reddy:Mm-hmm (affirmative).So that definitely some theoretical researcher driven, before the LoJax, but the LoJax is a fast known exploitation instance where we know we found it in the wild. It is quite possible even before that a UEFI implant demonstrated in many of the black hat conferences, but those are theoretical in nature. So the research had access to the device and they demonstrated it. But LoJax's is the one where from operating system level. So a particular malware, I would say it as a root kit, which has tried to intrude from kernel mode to the UEFI, and they have installed a UEFI driver. So if we consider the operating system as a drivers, even the firmware itself had some drivers. So they were able to install a driver which actually in turn drops the another kernel mode driver, advanced operating system boots up. it's about the boot sequence. Gowtham Reddy:So first the firmware starts running and it initializes all the system, and then it invokes operating system. So in the LoJax's case, after the firmware is completed, it has already dropped the kernel driver on the operating system, if it is not present. So that means by the end of the firmware sequence, so we have a presence of a kernel driver. And when that kernel driver starts, that is a user mode, malware starts, kicks in. So this keeps repeating even after you were re-install the wares, even if you change the hard disc, the same pattern will be fought. So that's how the LoJax's type work. Nic Fillingham:And I wonder, do we know, what was the breakthrough that made LoJax possible? UEFI has been around for a while. UEFI for probably predates LoJax. And obviously before UEFI, there was sort of the more standard sort of BIOS that probably most folks are familiar with. Can we talk a little bit more about how LoJax came about and sort of what maybe changed or what the breakthrough was on the attacker side? Gowtham Reddy:I would say that there were a couple of open source read-write drivers, which has a capability to access the firmware, using a special interface called SPI. SPI is a something called serial peripheral interest. So using the serial peripheral interface, any kernel driver can instruct the platform hardware layer to read and write any content in the flash. So I think like many of the security industry knows a driver called a read drive, everything, they call it as RWE. So this is the driver using which anybody can read any offset, any device memory, and write. I think this is, the prevalence of this kind of open source tools might be help attackers to develop this kind of ecosystem of all the sequence of the malware, the root kits. Shweta Jha:In addition to what Gowtham said, definitely the work that researchers were doing in this space, it always starts with researcher trying to do something and then attackers trying to find other means. So here are the things. Attackers usually do exploit things that are not done in a right way. So in this case, for example, if there are certain configuration that you need to, or your partner needs to make sure that those are in place, for example, rewrite where you are not providing writing access, just the reading access, and so on. Shweta Jha:So typically in all these type of attacks would see that misconfigured devices are exploited the most, and that misconfiguration happens at the time when the devices are getting built. So that is another factor why these attacks are very successful, because there are misconfigured devices, because while building the devices, somebody messed to configure it and right way. And if you look at the journey, that's where you have a secure core PC, which is designed be secured, making sure that the things that are needed to protect the computer against these types of attacks that are there out from the first day. Natalia Godyla:So my question is about the application of this new technology. So I really appreciate you walking through that attacker workflow. So what type of signals is UEFI scanning, looking for? What is it using to enrich the context of the existing end point data?Gowtham Reddy:That's a very good question. So basically the level of details that UEFI scanner can get is enormous. So this is the area where like the defender has a content scanning. So, uh, we have, uh, extended our content scanning to every file that is present inside the firmware. So this help the defender research to write any kind of content scanning signatures to detect any bad content. So that means in this case, if research knows any implant, so we have a capability to scan the 600 million devices to know if any of our customers have impacted with the specified malicious file. Gowtham Reddy:And this is just one part of our UEFI scanner. And the other part of it is detecting any anomalous behavior inside the firmware. For example, in many of the supply chain attacks like Solarigate, it's quite possible that some of the OEMs channels were compromised and the deliver the firmware updates with the malicious modules in it. Gowtham Reddy:So in this case, our UEFI scanner collects all the metadata about the new for- firmware update and we run heavy amount models in our cloud. And that will tell us if there is an unknown anomaly that exists in this particular firmware update. Instead of a known malware implant so that the UEFI scanner has the two capabilities. One is detecting a known malicious implant, and the other one is anomalous from where presence of a fax. So in this case, we act both ways. Nic Fillingham:What does an anomaly look like in this context? Gowtham Reddy:Anomalies look like, for example, if you have a firmware is a, firmware is a file system, like a typical drive. A presence of an driver file, probably a hedge P driver file or an unsigned driver file. On a Dell OEM is constrained to the anomaly. Because we have trained the model of all the known Dell firmwares with them, a ML model. So any new image with the unexpected file, it will be immediately flagged. Nic Fillingham:And why is ML the sort of approach you've taken here versus sort of heuristics? I would have thought that there's a pretty limited set of content. They could make up sort of firmware and firmware instructions. Obviously, I don't know anything about this space, so I'll caveat that there, but, um, could you talk about why ML versus heuristics versus something else?Gowtham Reddy:In the days of, uh, BIOS, so you are a expectation was right. The bast consists of a series of micro code, Gowtham Reddy:... which is, uh, very limited. And, uh, in the context of UEFI, you have a full file system, uh, which has, like, uh, thousands of files; individual files. And, uh, this causes... Uh, creates, uh, basically a huge amount of, uh, the vectors space, which to scan or to collect the metadata. Gowtham Reddy:So it's not just simple collection of mecra- microcodes. It contains the drivers, it contains the services, it contains a lot of other things. It's a file system like NTFS.Nic Fillingham:Got it. So because UEFI is, as you say, a file system as opposed to... What was BIOS? BIOS was not a file system? BIOS was, uh, sort of a discreet, sort of, low level executable?Gowtham Reddy:Yeah, i- i- it is just a sequence of, uh, microcode instructions that will be run on the firmware. So basically, i- it has a s- uh, fi- se- set of microcodes. Nic Fillingham:So the machine learning models that you reference, w- where are they running? Are some of them running locally? Are they all running in the Cloud? Is it a mixture of the two?Gowtham Reddy:They're all running in the Cloud for now. So we have MDATP Cloud services where we run all this clo- uh, demo models. So our models are really very effective. So recently, we got in, uh, so- so, uh, the UEFI alert by, uh, mal- model. Apparently, it's a kind of, um, true positive because, um, there was a Microsoft engineer who was working on a hardware space.Gowtham Reddy:So he take, uh, firmware. And he kept a developer driver and he flashed on his own device. And, uh, our UEFI scanner immediately caught it and we... the security administrator got an alert and there was an investigation happen. So we are pretty ready to catch any kind of such things now.Natalia Godyla:So we all know it's a cat and mouse game with the threat actors. So what is the team anticipating in terms of how the actors will adjust their processes to evade this new UEFI scanning technology?Gowtham Reddy:That's a good question. We're trying to validate something in a- a lower level of trust, the lower level of ring other than the kernel. So definitely, there is a chance that attacker can modify the firmware presence. Uh, he can spoof the content when defender tries to scan. So this is quite, uh, possible. But we are already working on mitigating that kind of an attacks. Nic Fillingham:So now that this feature, these capabilities, have been live in the product for, uh, I guess over six months at this point, w- what have you learnt? What have you seen in the telemetry? What have you seen in the types of attacks and, I guess, even sort of false positives that have- have come through from- from this new, uh, capability?Gowtham Reddy:Uh, that's a very good question. So we learnt a lot of things. The UEFI file system has never scanned before. So we got some false positives on the content that we scan but we immediately fine-tuned our signatures.Gowtham Reddy:Back in... Six months before, when we published a blog, we only know the first UEFI known implant called LoJax but often we share... There was a second implant called Public. That's called MosaicRegressor and our UEFI scanner has well detected the MosaicRegressor implant. Uh, the- the telemetry count was small. So we did, uh, able to detect the mi- MosaicRegressor.Nic Fillingham:So in this first six months, as well as the LoJax campaign, uh, what's the taxonomy here? How do we f- refer to it?Gowtham Reddy:Uh, we can consider... W- we are, uh, tracking them as an UEFI implant malware or UEFI rootkit. So this is the category we are looking at. So right now, we have, uh, LoJax and we have a MosaicRegressor as, uh, two big families in this space.Nic Fillingham:Big families. Got it. Shweta Jha:Yeah, about MosaicRegressor, I wanted to add a little bit more just to complement what, uh, Gowtham mentioned, how powerful this tool is. And how powerful this particular feature is. So if you read through the MosaicRegressor, uh, breach, it was a nationwide targeted attack.Shweta Jha:This was targeted for diplomats. And this attack, as Gowtham described, first they would insert one module. Uh, that one module would get undetected and then that module would try to do other stuff, like try to, uh, get in touch with command and control and get another, uh, module and so on.Shweta Jha:So the entire c- chain is so very interesting. And I'm glad that we built this feature and we were able to detect it because it's so powerful. Most of the security solution, they're not able to detect because they don't have this, uh, such great capabilities.Shweta Jha:But look at the way this attack was carried. It was pretty much targeted, pretty much nationwide for a few countries, originated from one country. So the sophistication level in the nature itself speaks for it and I'm glad that we, as in our product, we have this capability which can even, you know, unknown, first seen, it can detect those type of attacks as well.Natalia Godyla:In the process of developing this new technology, where were there false starts? What techniques did you try but didn't work to solve this problem?Shweta Jha:Little bit on the journey, right? We have been working on it. Um, so Gowtham explained about how we have rootkit, bootkit level and then we went to the UEFI site and we had to be extremely careful because it's, like, uh, it has a high integrity and high severity of going wrong.Shweta Jha:So we had to be very careful making sure that the running system is not damaged and at this point, I'll hand it over to Gowtham because he can explain, in detail, each and every pieces that we took into consideration to making sure that our customers' device remain intact. So go ahead Gowtham.Gowtham Reddy:Yeah. Thanks Shweta. So, uh, we have indeed explored, uh, many mechanisms like accessing the PCI space from the operating system itself, which we didn't continue to proceed because of some of the pushback from the kernel team to update the haul.Gowtham Reddy:So actually, uh, to accessing any peripheral device from the PCI bus, there are a couple of complications because the peripherals have, uh, specific implementation of Reads and Writes, the bus Reads and Writes. So, uh, the approach we took was, uh, using the SBI interface, which is pretty much, kind of, an, uh, universal interface which is developed by Motorola by a long time ago.Gowtham Reddy:So luckily, what worked in our favor was most of the Intel p- s- uh, chipsets, they support the SBI based access. So they support the SBI, uh, using which we can use the memory map mechanisms to access the PCI space.Gowtham Reddy:So basically, here, what happened was instead of directly using the hardware primitives, we used, uh, software primitives because the chipsets are well supporting the SBI interface. So that's how we landed in our approach. Nic Fillingham:I wanted to circle back to the use of machine learning here in- in solving this problem. How big are the signal sets that you're getting to train the model? How big is the model?Nic Fillingham:Is the model that you use here, to detect anomalies in the firmware layer, is it as sophisticated and large as something as, like, looking for malware on endpoints? Or are we talking, like, a much sort of smaller more, sort of, n- nuance. No, that's not the right word. Sort of a smaller bespoke model?Gowtham Reddy:Uh, I can take that question. So u- usually, uh, in the endpoint when- when applying the malware, um, in machine learning models, we heavily focus on the individual file properties, like file headers, file footers and some file p- properties and so on. Gowtham Reddy:But UEFI case, we built a brand new machine learning model based on the properties of the UEFI image itself. So thanks to David, from our MDATP team. So he come up with a model where... which takes input signals as specific to the UEFI firmware image.Gowtham Reddy:To give some examples, each firmware drive has a lot of GUIDs, called firmware GUIDs. And then they have some properties called, uh, file types and properties. Every property that we took was specific to the firmware. So they are not generic to the specific malware files that we see regular malware detections. So these are highly tailored to the signals from the UEFI firmware image. Nic Fillingham:And were you able to reuse some of the anomaly detection Nic Fillingham:Algorithms are purchased from other parts of the defender engineering org, or did you have to sort of build a brand new model and a brand new way to detect anomalies? Shweta Jha:Yeah. So, we definitely used our existing infrastructure. So, as you know? Uh, we have a massive backend system where we get tons of signals and we run tons and tons of AI and ML model to detect the anomalies and to detect the new trends and so on. So, as Gordon was talking, for this particular UV, AI and ML model, even though where we had to tweak it to make sure that we capture the inputs that are UV specific, the models were used, the pipeline to collect the data that were used and the channel where we surface it to our customers. So, if you look at the end to end story, the way we do things are we detect, we remediate, and we also notify to our SecOps that, "Hey, these are the things that happened in your environment." And that goes in the form of alerts or incidents and so on. So, we used exactly same infrastructure, same pipeline, but specific to UV. Natalia Godyla:So, I know a little earlier in this episode, we talked about the learnings after being in market. What about the impact to SecOps teams? Do we have any early numbers to talk through about what this has raised for our customers? Shweta Jha:That's a great question. We do see here and there, though the number is not pretty high on the implant, but we do see in numbers there, like, as Gordon mentioned about a mosaic regression. We did find that and there are few others also. But I think the most important aspect of this unique feature is that, just a little bit forget about this feature and see that today's world, today, there is no UV scanner, the security admins or SecOps, they, they don't know what is happening at this level. They have tons of device in their organization. And these devices are at this level is completely black box for them, because they don't know whether it is configured well. They don't know if there are implants there. They don't know if there are vulnerabilities that could be exploited. Shweta Jha:So, there's the power of this UV scanner. One is, you know, so we, we built a solution keeping in mind that we will not only detect, we will bring these, these things where they don't have visibility today to understand what is going on. So, the focus area, and then the objective that we have is to detect the implant, either using the heuristic detection or the AI, ML but also read through each and every configuration that are happening at this level and the vulnerabilities that exist at this level and bring that to the, SecOps attention, so that when they look at it, they can take appropriate action to remediate it. So, that's the next step. And that is the work right now, we are currently doing. We do not have, in the form of report, we do see it in our data and we want to make sure that these are available to our SecOps. But just to tell you, there are tons and tons of misconfigured device out there. And it's, it's a little tricky.Gowtham Reddy:To add more about the misconfiguration. So, it's about like the PC settings, like a UV, the BIOS read-write or whatever the settings we'll use to see in when we go to the BIOS in the past. So, the UV must be configured well to support the secure boot, to use the TPM and to use any of the hardware provided features, it must be configured well. If it is misconfigured, you won't get any protection. So, if you have a helmet in your backseat, when you are driving, it won't help you. So, you had to keep it on your head. Shweta Jha:(laughs). That's a great analogy. Nic Fillingham:That leads us to, what is the guidance here for Sec admins and security teams out there? How do they enable this functionality? Is it on by default in, in certain places? What do we need to do to make sure that, that customers are getting the full protection from this capability? Shweta Jha:So, uh, this, this feature is enabled by default on all the devices. Um, we made sure that this is available. And the great news is that it is not only, you know, Windows 10, it is available for servers, download as well. So, that's the power that we have in our solution. Ultimately, if you look at what is the future that are gonna look like, secure core PC is the future we should be heading towards. But because enterprises and customers are not there yet, uh, we have UV scanner to compliment it. The other thing, if we have to talk about the futuristic roadmap, right now, we built the scanner for UV, but there are other network devices like network adapter and things like that. There is a scope to extend these types of capability to those devices as well, because those, there is a possibility to get those devices exploited too. So, that's something we are considering to work through. Nic Fillingham:Got it. So, just to confirm there, so, this new capability is on by default in any device that is being protected by the defender service. Is, is it, is it as simple as that or is there sort of more to it?Shweta Jha:Yes. Any device which is having defender antivirus running.Natalia Godyla:Thank you for that. That was super helpful. And thank you both for joining us on the show today. Shweta Jha:Thank you, Natalia. It was pleasure to be here and talking with our customers. Thank you so much for hosting us. Gowtham Reddy:Thank you Natalia and Nick for hosting us. So, it's been wonderful time talking to you about UV scanner. Thank you so much. Nic Fillingham:Thank you both for your time. Thanks for bringing great innovation to the security space. Shweta Jha:Absolutely. It's a constant journey and we're on it. Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode. Nic Fillingham:And don't forget to tweet us @msftsecurity or email us @securityunlocked@microsoft.com with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure.