Security Unlocked

Share

Re: Tracking Attacker Email Infrastructure

Ep. 19

If you use email, there is a good chance you’re familiar with email scams. Who hasn’t gotten a shady chain letter or suspicious offer in their inbox? Cybercriminals have been using email to spread malware for decades and today’s methods are more sophisticated than ever. In order to stop these attacks from ever hitting our inboxes in the first place, threat analysts have to always be one step ahead of these cybercriminals, deploying advanced and ever-evolving tactics to stop them. 

  

On today’s podcast, hosts Nic Fillingham and Natalia Godyla are joined by Elif Kaya, a Threat Analyst at Microsoft. Elif speaks with us about attacker email infrastructure. We learn what it is, how it’s used, and how her team is combating it. She explains how the intelligence her team gathers is helping to predict how a domain is going to be used, even before any malicious email campaigns begin. It’s a fascinating conversation that dives deep into Elif’s research and her unique perspective on combating cybercrime. 


In This Episode, You Will Learn:  

• The meaning of the terms “RandomU” and “StrangeU” 

• The research and techniques used when gathering intelligence on attacker email structure 

• How sophisticated malware campaigns evade machine learning, phish filters, and other automated technology 

• The history behind service infrastructure, the Netcurs takedown, Agent Tesla, Diamond Fox, Dridox, and more 


Some Questions We Ask:

• What is attacker email infrastructure and how is it used by cybercriminals? 

• How does gaining intelligence on email infrastructures help us improve protection against malware campaigns? 

• What is the difference between “attacker-owned infrastructure” and “compromised infrastructure”? 

• Why wasn’t machine learning or unsupervised learning a technique used when gathering intelligence on attacker email campaigns? 

• What should organizations do to protect themselves? What solutions should they have in place? 

  

Resources:


What tracking an attacker email infrastructure tells us about persistent cybercriminal operations: 

https://www.microsoft.com/security/blog/2021/02/01/what-tracking-an-attacker-email-infrastructure-tells-us-about-persistent-cybercriminal-operations/ 


Elif Kaya:

https://www.linkedin.com/in/elifcyber/ 


Nic’s LinkedIn:  

https://www.linkedin.com/in/nicfill/   


Natalia’s LinkedIn:  

https://www.linkedin.com/in/nataliagodyla/   


Microsoft Security Blog:  

https://www.microsoft.com/security/blog/


Transcript

[Full transcript can be found at https://aka.ms/SecurityUnlockedEp19]

Nic Fillingham:

Hello, and welcome to Security Unlocked, a new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft security engineering and operations teams. I'm Nic Fillingham.


Natalia Godyla:

And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft security, deep dive into the newest threat intel, research, and data science.


Nic Fillingham:

And profile some of the fascinating people working on artificial intelligence in Microsoft security.


Natalia Godyla:

And now, let's unlock the pod.


Nic Fillingham:

Hello, Natalia. Welcome to episode 19 of Security Unlocked. How are you?


Natalia Godyla:

I'm doing great. I'm excited to highlight another woman in our series for Woman's History month, so this'll be number two. And I'm excited to talk about email infrastructures.


Nic Fillingham:

Yes, I am too. Email, we use it every day. We probably use it more than we, we want. We love it. We can't live without it. What's your first memory of email? What was your first email address?


Natalia Godyla:

I was an AOL-er. First email was glassesgirl2002@AOL.com. I'm super proud of that one.


Nic Fillingham:

What's the reference to 2002?


Natalia Godyla:

I'm pretty sure that's when I got my first pair of glasses (laughs).


Nic Fillingham:

Ah. And you-


Natalia Godyla:

I was very excited. I threw a cupcake party.


Nic Fillingham:

Oh, wow.


Natalia Godyla:

(laughs)


Nic Fillingham:

So I'm, I'm pretty old. It was sort, sort of the mid 90s, and I remember like, hitting websites where it asked for an email address, and I'm like, what is an email address?


Natalia Godyla:

(laughs)


Nic Fillingham:

I probably used the internet the best part of, you know, six months before someone explained it to me. And I worked out how to get a Hotmail address, which is called Hotmail because it was actually based on the, the acronym H-T-M-L, and they just put a couple other letters in there to expand it out to say Hotmail. And I remember being, thinking like I was the bees knees, because I was nicf12@hotmail.com.


Natalia Godyla:

(laughs)


Nic Fillingham:

We should have asked our guest Elif Kaya, who you're about to hear from, about her first email address, but we didn't. Instead, we talked about a blog that she helped co-author, uh, that was published beginning of February called, "What Tracking and Attacker email infrastructure tells us about persistent cyber criminal operations." It's a fascinating conversation, and Elif walks us through all of the research that she did here where we learn about attacker email infrastructure and how it's used and created and managed.


Nic Fillingham:

There's a bunch of acronyms you're going to hear. The first one, DGA, domain generation algorithm. You're going to hear StrangeU and RandomU, which are sort of collections of these automatically created domains. And if you sort of want to learn a bit more about them, it's obviously in the blog post as well.


Natalia Godyla:

Yes, and in addition to that, you'll hear reference to Dridex. So, as the RandomU and StrangeU infrastructure was emerging, it was parallel to the disruption of the Netcurs botnet, and those same malware operators who were running the botnet were also using malware like Dridex. And Dridex is a type of malware that utilizes macros to deliver the malware. And with that, on with the pod.


Nic Fillingham:

On with the pod.


Nic Fillingham:

Elif Kaya, welcome to the Security Unlocked podcast. Thank you for joining us.


Elif Kaya:

It's great to be here. Thanks for having me.


Nic Fillingham:

Now, you were part of the. uh, team that authored a blog post on February 1st, 2021. The blog post is "What tracking and attacker email infrastructure tells us about persistent cyber criminal operations." Loved this blog post. I've had so many questions over the years about how these malware campaigns work. What's happening behind the scenes? Where are all the, the infrastructure elements? How are they used? And this blog helped answer so much and sort of joined dots.


Nic Fillingham:

If you are listening to the podcast here and you're not sure what we're talking about, head to the Microsoft security blog. It is a post from Feb 1st. But Elif, could you sort of give us an overview? What was discussed in this blog post? What was sort of the key take away? What was the research that you conducted?


Elif Kaya:

Sure. So uh, I'm part of a, a email research and threat intelligence team, uh, that supports the defender product suite at Microsoft, and what we primarily focus on is tracking email campaigns and email trends over a long period of time and documenting those. So, this blog post kind of came along series of documentation, which we started to bubble up these trends in infrastructure, which is one of my focus areas, starting back in March and running uh, all through the end of the year, where a large series of disparate email campaigns, kind of stretching from very commodity malware that is available for like 15, 20 dollars, to things associated with big name actors, and et cetera, were being delivered with very similar characteristics, despite on the surface the malware being very different, the outcomes being very different, or the cost of the malware targets being very different.


Elif Kaya:

And so, we were able to see within each of these individual campaigns that the infrastructure supporting the email delivery was a consistent theme. So, it starts with when these domains that were used as email addresses to send these from, uh, started being registered to the current day and kind of what campaigns they helped facilitate, when they were registered, and et cetera. So, when people usually talk about infrastructure that supports malware, a lot of the terms get used overlapping. So, when people refer to infrastructure, they generally are referring to the see to addresses, call back addresses that the attacker that owns the malware owns.


Elif Kaya:

But what we've been seeing much more frequently, and what we wanted to explain with the blog post, is that in really concrete ways like you said with actual examples, is that the malware and cyber crime infrastructure is very modular. And so, when we say infrastructure we could mean who's sending the emails from their servers, who's hosting the email addresses, who's posting the phish kits, who's hosting the delivery pages that deliver the malware, and who's writing the malware. And then later, who's delivering the ransomware.


Elif Kaya:

And so these could, in any particular campaign or any particular incident that a sock is looking at, be entirely different people. And so, the reason we wanted to do this blog and detail kind of what we did here and go through each of the cam- malware campaigns that was delivered, was to kind of show like, if you're only focusing on each malware campaign, the next one's going to be right cued up and use all the same infrastructure to deliver maybe something maybe more evasive that, that you'll have to get on top of.


Elif Kaya:

And so, by doing this tracking you can kind of up level it once more, and instead of spending all you time trying to evade one particular malware strain that's going through constant development, you could put a higher focus at stopping kind of the delivery itself, which, we actually detail through the blog, was very consistent over nine months or so, but had a lot less attention focused on it.


Elif Kaya:

So, some of the cases that we discuss in the blog are cases like Makop, which was used very heavily, and in especially South Korea, all throughout April and all throughout the spring, and is still pretty prevalent in terms of direct delivery ransomware in that region. It's usually delivered through other means, but what we saw and what we theorized is that whenever the standard delivery mechanisms for those malware are interrupted, they'll kind of sample other infrastructure delivery providers, which is what we describe as StrangeU and RandomU in the blog.


Elif Kaya:

We use the term StrangeU and RandomU to differentiate two sets of DGA, or domain generation algorithm domain structures that we saw. StrangeU always uses the word strange. Not always, but nearly about 95% of the time. And Random U, couldn't find a better name, but it's just a standard random DGA algorithm, where it's just a bunch of letters and characters. We don't really have a fancy name to give it, but we were able to kind of coalesce around what that was internally, and track the domains as they were registered there. And then, shortly after they would be registered, they would start sending mail from those domains.


Nic Fillingham:

Elif, were you and the team surprised by how much interconnected overlap, agility, and sharing, for one of a better term, they were across these different groups and campaigns and techniques? Were you expecting to see lots of disconnected siloed activities, techniques, groups, et cetera, et cetera? Or were you expecting this amount of overlap, which we'll get to when we sort of explain the, the stuff in the blog?


Elif Kaya:

So, I think it was less that it was a bit of a surprise, and more that we don't often get a pristine example like this. Frequently, when we look at the connected infrastructure, they don't use domains necessarily. They'll use the botnet itself and IP addresses for delivery or other things. So, when we came across this one, we do normally handle and really do a deep dive in individual incidents and cases, so this was a little bit more of a unique example of like, hey, there's really clear patterns here. What can we learn by tracking it over a long period of time, in ways that other metrics are a little harder to track?


Elif Kaya:

But yeah, I, I would say that in general, most email campaigns and phishing campaigns, malware campaigns that you kind of run across, they are gonna have these threads of interconnectivity. They're just going to be at different levels. So, whether that's going to be a level that is kind of more visible for uh, blue teams like the email addresses, the domains themselves, or whether that's going to be something more femoral like IP addresses and hosting providers, or whether that's going to be something that's proxy even more so, like a cluster of compromised domains, similar to, to, you know, what Emotet uses, uh, or use to use, collected in a botnet that has a different way of clustering itself.


Elif Kaya:

And so for these, we were able to just kind of have something that bubbled to the top and made it easy to connect the dots, as well as other items in the header in the malware that we were able to identify. But I think through tracking this, we were able to kind of reaffirm and make a good piece of public example for blue teams that this is a very common method. This is a very common modular technique,


Elif Kaya:

... And it's very simple for attackers to stand this kind of thing up and offer their services to other places. And that's part of why we reference the Necurs botnet as well. Dridex makes a big appearance in the StrangeU and RandomU deliveries, especially later on in our tracking of them, and Dridex is also a prominent, um, delivery from a lot of other of these types of delivery botnets that have happened in the future, whether that's CutWail or, uh, Necurs or other, um, botnets like that. So it, it's very common but it's sometimes very hard to kind of keying in on all of the distinct components of it and evaluate like, is it worth it in this instance to key in on it, um, when our main goal is like, what is the most effective thing we can do to stop the deliveries?


Natalia Godyla:

I'd love to talk a little bit about the history that was described in the blog for the service infrastructure. So from what I understand, the Necurs takedown created a gap in the market where StrangeU and RandomU were able to step in and provide that in- necessary infrastructure. So why was that the replacement? Was there any connection there? And as a second part to that question, what does the evolution of these infrastructures look like? How are they accessible to operators that want to leverage them?


Elif Kaya:

Right. So in this one I can delve a little more into kind of just intuition and, and doing that, because my full-time role is not specifically to, you know, track all the, all the delivery botnets there are and active. The reason that we made the connection to Necurs wasn't because there was an actual connection in terms of affirming this is filling the same role that it was, or this is filling a hole. Because we don't have necessarily a clear picture of every delivery botnet there is. Because the timeframe was very close and because we were able to see shortly after, uh, StrangeU and RandomU started delivering, they initially only had pickup from commodity malware that we could find. So very cheap malware for the first few months of their delivery, such as Makop. Uh, we saw some Agent Tesla, we saw some Diamond Fox.


Elif Kaya:

But as it progressed on, it started picking up the bigger names like Dridex and doing larger campaigns that were more impactful as well. And so by the time that Necurs had ended, we had also seen them doing a lot of those bigger name malwares as well. And so the reason why we tried to make that comparison was largely to show that something very simple and kind of perhaps much less sophisticated and lasting for a lot less length of time as Necurs in the environment can get customers quickly. And so while we didn't do a deep dive into any of the amount of like, how is it being advertised, how are they getting the customers, what we wanted to show is that regardless of what methods they're using to get the customers, they're able to get-


Elif Kaya:

Basically the, the amount of research that was done for Necurs was much more in depth than the amount of research that was necessarily done here. And it was also done from a different angle, that angle was much more operator focused and our angle was much more, what was delivered, what was the impact, what were the trends between all of the different mails? And so we're mostly trying to just position it as, this fulfilled a similar, uh, outcome and got a lot of coverage of something that was very big, lasted for a very long time, many years, and something where somebody just started registering some domains, setting up some mail servers, was able to kind of get off the ground and running in just a few months for relatively low cost.


Nic Fillingham:

So El, if we normally start with an introduction or, or I, I got so excited about this topic that I jumped straight into my first question and I didn't give you an opportunity to introduce yourself. And I wondered, could you do that for us? I know you're, I believe you're a threat analyst or a threat hunter, is that correct?


Elif Kaya:

Yeah, so I'm currently a threat analyst, and you've actually had other people, I think, from my team on here already before. But yeah, I, I'm a threat analyst at Microsoft. I've been on this particular team for about a year now, specifically focusing in email threats, web threats, and I do have especially some focus in infrastructure tracking and domain, uh, generation algorithms in general and trying to make sure that our emails and campaigns that we're tracking are properly scoped and that we're able to kind of extract as many TTPs as we can from them.


Elif Kaya:

And so the role of our team and the role of myself in particular on the team is, when we do these individualized campaigns we look for the IOCs and things like that in it. We scope it, but what we're really looking for is, um, the trends of what's happening so that we can kind of try and pinpoint and escalate to the other teams internally the most impactful changes we could make to the product, or the most impactful changes we could recommend that customers do, if it's something that we don't have a product for or we don't have a protection for, in order to protect against the campaign. And so in this particular instance with this infrastructure, our goal here was to kind of really reiterate to customers that despite all this complexity, the spaghetti-like nature of this, at the end of the day all these different campaigns used kind of a lot of the same both delivery to deliver the email, but the Word documents that they delivered were also very similar.


Elif Kaya:

There, there were a lot of configurations that can be made on the endpoint to kind of really nullify a lot of these campaigns despite what we were able to see and some really evasive techniques that they were developing, the malware operators, over the time.


Nic Fillingham:

Yeah, I, I wonder if you could talk a little bit about how the research was actually conducted. A lot of these domains were not hosted by Microsoft infrastructure, as I, as I understand it. I think you sort of cover that a little bit in the blog. So how do you as a, in, you know, in your role, how do you go about conducting this research? Are you setting up honey pots to try and, uh, receive some of these, these emails and just sort of be a part of the campaign, and then you, you conduct your analysis from there? What, how do you go about, uh, performing this research?


Elif Kaya:

So the bulk of the research I think is performed with various, like some of it is honey pots and some of it's that. A lot of the research that is covered in the blog after we, uh, analyze the malware campaigns, which is a service we offer through, um, MTE, which I think there have been people from MTE that have come on as well, as well as analysis that we do, again, based on, uh, the malware samples that we receive and the email samples that we receive from reports, from externally as well as from open source intelligence. A lot of the domain research here, though, is actually done from, uh, open information. So any domain registrations that there are, the registration fingerprint, as I like to call it, which is all the metadata related to the registration, is publicly available. And so we collect a lot of that information and search it internally.


Elif Kaya:

And this is always something that I like to advise and encourage blue teams at any particular organization, you know, if they have a little bit of extra funding, to try and invest in as well. Because it's definitely, even though it's free and publicly available, you're generally gonna have to get a subscription or set up some kind of collection order to query the "who is" databases and the passive DNS databases that you'll need in order to do some of these pivots. But it kind of starts with finding the malware campaigns and then finding the emails, and then pivoting up towards everything else we can do. And once you have kind of a net of what you're looking for, sender domains and et cetera, you can then kind of go backwards and say, "Okay, now show, show me all the malware campaigns that we have investigated that, that have these components to them. Show me all the phishing campaigns that have these components to them."


Elif Kaya:

And so it's kind of going up and then going back down, but all clustered around that registration data and that domain data. Uh, because whether an attacker decides to use IP addresses or whether they decide to do domains, there's usually always some component of their campaign that they have to use attacker-owned infrastructure for, if that makes sense. We see a lot and it's very common for attackers to u- use compromised infrastructure, so WordPress sites, things like that, to host a lot of their architecture. But especially for things like C2s for mail delivery and other things, they're gonna want some resilient infrastructure that they'll own themselves. And so at what point in the chain they decide to do that is usually an opportunity for us to be able to see if there's any OPSEC errors on their part, and also see if they've conducted other campaigns with that same infrastructure. Yeah, and so differate- differentiating between attacker-owned infrastructure and compromised infrastructure is an additional critical component.


Natalia Godyla:

Now I'm trying to decide which question to go forward with. Can you describe the distinction between those two?


Elif Kaya:

Right. So attacker-owned infrastructure would be something the attacker sets up themselves. So they have to think of the, and populate the data in the domain address and the registration and the tenant themselves. So this encompasses both when attackers use free trial subscriptions for cloud services, it's whenever they go log into Namecheap and they register their own domains, as well as when they have dedicated IP hosting or bulk group hosting as well that they have decided like, "For this portion of my campaign," whether that's command and control, whether that's delivery or et cetera, "I need to make sure that I'm in control of this." We have seen examples where compromised infrastructure, which is the reverse of that where especially small businesses, parked domains, and other insecure WordPress sites, sites that have other types of vulnerabilities, will be compromised and used to, again, do any, any component of that kill chain, whether that's sending mails, hosting the malware, and will be used to do those things as well.


Elif Kaya:

So compromised infrastructure is when the attacker will utilize someone else. The benefit for attackers is it's definitely a lot harder for defenders to identify or take action against that, especially because they don't know how long it'll be compromised for, if it'll ever not be compromised, if the attacker's only leasing access to the compromised domain through a, a kind of, uh, cyber crime as a service provider or not. It becomes harder for the defenders to defend against and detect, because it has less points of contact and familiarity with other compromised domain. If somebody compromises a blog about kittens and a blog about race cars, it's gonna be pretty hard for a lot of things to pick up exactly what's similar about them, because some


Elif Kaya:

... other human worlds apart has made the whole blog but if one attacker has-


Nic Fillingham:

Probably Natalia Godyla


Elif Kaya:

... made five to 15 different sites in a day. (laughs) Yeah, it's a, it's going to have a lot more in common. But the downside of compromised domains for attackers is a, they often have to lease them from the people that initially compromise them and c, those compromised domains could become uncompromised, they have to now maintain access to something they didn't make. And we did also see that with OMO Tech, over the summer when it had come back after being quiet for very long, and people had replaced their payloads on compromised sites with, uh, I think chips with CAATs, something like that. We're back to CAATs.


Nic Fillingham:

You're speaking our language here, like we're, we're, we're on the edge of our seat, you said CAAT like twice in like a minute.


Natalia Godyla:

(laughs)


Elif Kaya:

But when an attacker comprises a lot of their infrastructure on compromised infrastructure, other attackers could compromise it, defenders could compromise it, anyone can kind of... They have to now protect it, whereas if they made it from nowhere and no one owned it, except for them, it's kind of a lot easier for them to just hang out. Because then the kind of only person that's looking out for them a lot of the time, is if somebody is connecting the dots on the infrastructure or the hosting providers, like I think the ones that we cover here is like, IronNet, Namecheap, et cetera, if they're looking out for somebody hosting on their, their infrastructure. But if somebody is just sitting there, they're just being quiet, they're just sending mail, nobody's going to notice that they're compromised probably. Whereas if you're a small business owner and your site ends up on a block list, you're going to go start asking questions, you're going to start trying to get that fixed or take your site down.


Nic Fillingham:

Elif, I'd love to come back to what you talked about with the way that you conducted this research and you, you, you said that getting subscriptions to Huawei Services and DNS records, this is all public record. But there is still some tools required to pass through that information and, and create the pivots. We were talking offline, before we started recording, I'll paraphrase here and please correct me, that you didn't utilize really machine learning as a tool to discover this techniques. Is that, is that correct? Can you talk more about what techniques you did use and didn't use and why something like machine learning or unsupervised learning was not either necessary in this space or wasn't necessary to discover these techniques?


Elif Kaya:

Yeah, I mean, I could talk to the, the techniques that I used and well, I can't say explicitly like why machine learning would or would not be helpful here because I'm not an expert on machine learning. I think in the different campaigns that I've worked on in my career in security, whether it's this one or before I came to Microsoft, I did some more independent research on a large set of Chrome extensions that were also connected by various, uh, commonalities to get those taken down. A lot of this research that can be pretty impactful and pretty widespread doesn't require ML in order to parse and to navigate. And I think part of the reason that ML is a bit unsuited for this at the moment, is because there hasn't been as much manually focused research. And there's been a lot of research done by independent researchers and people in the security community but I have seen a lot less focus in terms of data from tech companies in doing and making publicly available some of this infrastructure surrounded research.


Elif Kaya:

And so what I mean by that is that a lot of security companies focus a lot on the actor name. They focus a lot on the reverse engineering of the malware and those are critical components. In part because that's what the products that they're sometimes selling is AV Surfaces and things like that and that's the point in time that they are protecting against the threat. But when it comes to the infrastructure, companies that would be the most positioned to protect against that threat or have products to protect against that threat, aren't necessarily doing the manual body of research currently necessary I think, in order to guide ML to kind of identify this work. And so right now to say, " Oh, would this be something that ML would be suited to step in?"


Elif Kaya:

And I think that it could in the future be suited to step in slightly but I also think that the way that this works, is currently operating at a level that actually does benefit from, from manual analysis at this time. In part, because it, it doesn't actually take tools that are generally above or beyond what is in a lot of analyst tool set with basic scripting and things like that. Because right now there has been such a non focus from security companies and blue teams, I think on infrastructure and infrastructure commonalities and the way that these campaigns are so modular that, for lack of a better word, there's not a lot of sophistication in it. Most of the sophistication we see in these campaigns are designed to evade automated technology. They're designed to evade ML. They're designed to evade phish filters. They're not really designed to evade humans looking at them, because I think you and me looking at those strange new domains, like you can look at a cluster of them and be like, "These aren't real sites, they're not real."


Natalia Godyla:

(laughs)


Nic Fillingham:

Yeah. I'm not, I'm not going to visit a website called, I'm gonna pick one up here like, eninaquilio.u... Maybe I would actually, that, that looks really cool. (laughs) Okay, gonesa.usastethkent, it's got like no vowels, like he replies strange secure world.


Elif Kaya:

And so we don't actually see a lot of, I guess, advancement in that space from attackers. A lot of the advancement is there in different parts that aren't necessarily bubbled up, but it's happening in the malware itself, in order to evade AV in order to not get alerts that fire on them. It's not necessarily happening to use something other than a macro or send from something other than an obvious phishing email or if obvious phishing source. And a lot of times, uh, one part that's one of my favorite part is these, these registrations frequently use the, .us domain. Many top level domains actually prohibit different parts of obfuscation for the registration record. And so when you register a domain, obviously the attacker kind of doesn't want to use real data, it's not the real name. But they'll use like memes and other things in the registration information, because it's fake data but then you can go and pivot and find where they've used the same meme before. And so-


Nic Fillingham:

Look for old domains registered by Rick Astley.


Natalia Godyla:

(laughs)


Elif Kaya:

Yeah, I think there was one-


Nic Fillingham:

You might be too young for that, me and my friend-


Elif Kaya:

There was, there was one that I think was used, I forget for which one of these malware campaigns where a lot of the registrations were actually happening under a registered email, that was something like, hiIhateantiviruspleaseleavemealone@gamer.com or something (laughs)or like, youcan'ttakethisdown.com. And I was like-


Nic Fillingham:

Try me.


Natalia Godyla:

Challenge accepted.


Nic Fillingham:

It's like a big red, a big red arrow pointing at them.


Elif Kaya:

What is happening in the infrastructure space for a lot of these things is happening pretty rapidly, it's happening at pretty low costs. And it's also happening and looking a lot different and is in a way a lot less glamorous, than a lot of the reverse engineering that is necessarily done but it's very critical. Or the more nation state tracking that is, uh, very popular when or companies are selling threat intelligence products to customers. But when it comes to like security, kind of in a sock, a lot of put is going to get through the doors, regular phishing emails.


Natalia Godyla:

So if the campaigns are targeting the automation that's built in, like you said, the phishing filters, what should organizations be doing to protect themselves? What solution should they have in place, processes?


Elif Kaya:

So some of the big things that I remember from these particular campaigns, um, is if you are rolling any kind of mail protection service or mail service in general, please periodically check your allow lists. The allow lists will frequently have entire IP ranges, entire domain ranges and so even domains like these ones that are very randomized and they're strange and you've never received an email before in your life. Sometimes the configurations of your allow lists for emails can completely cause the mails to bypass other filters. So definitely whether you're running Microsoft for your mail protection or not, please periodically check your allow lists and your filters and kind of have a good understanding of like, do I have any instances where phishing or malware would bypass other protections? Have I set that up? So that's one thing that I think does cut down a lot on some of these, making it to inboxes.


Elif Kaya:

And other as we... And part of the reason why we highlighted at each of the malware campaigns involved here is, uh, the suite of... I always forget the acronym, ASR rules, advanced security rules or configurations that Microsoft offers for office in particular for macro executions and malicious office executions, routinely outside of this blog and other, it's still office word documents, it's still Office Excel documents, it's still macro buttons. And so re-evaluating your controls there and your protections there, especially looking at some of the automatic configurations that we have available now to just turn on, that is going to help there a lot as well. I think are the two biggest like controls that I would recommend people for these kind of items, is checking kind of your allow lists pretty periodically and what your filtering policies are. And checking your, specifically, if you are using Office 365 internally, whether you have configurations set up to not necessarily even just restrict but there are more granular configurations now that you can set up to specifically restrict DLL and other execution from office macros as well.


Nic Fillingham:

Elif, in the section of the blog where it talks about the dry decks campaigns big and small June to July and beyond. It reads here, that it feels like you uncovered a section of sort of experimentation and testing of sort of new techniques. There's references to Shakespeare, there's something I've never heard of called, VBA stomping. Can you talk a little bit about what kinds of experimentation and creativity that you stumbled upon as part of this research? First of all, and what is VBA stomping?


Elif Kaya:

Uh, so VBA stomping, I think we might've actually met VBA purging in the blog. I'm trying to remember


Elif Kaya:

...whether, I think it might've been VBA purging, but surprisingly VBA stomping and purging are separate, but they fulfill the same kind of function, which is to try and make that macro, that like spicy button that everybody wants to press a little harder for malware detection engines to detect. So VBA stomping and purging both operate a little bit differently, but their main goal is to kind of obfuscate the initial VBA code from the actual amount malicious code in general. So that when antivirus engines try and examine it, they're going to see all that Shakespeare text and they're not going to see the malware. And as for the Shakespeare text, (Laughs) it's actually still on virus total. I think if people go and check for any of the files that reach out to the bethermium.com and DFIR, the blog did a great writeup called I believe "Tried X toward dominance" which actually covers in their sandbox what happened after they ran this doc. Which was eventually moved to a PowerShell empire attempts within their sandbox.


Elif Kaya:

But yeah, as far as I can tell from the Shakespeare use for this, it's actually not the first time that poetry (Laughs) and kind of Shakespeare has been used to obfuscate malware. There have been other rats in the past that have used this. Uh, we couldn't find any similarity like this, this was not those. But oddly enough, there is occasionally every now and then poetry or Shakespeare, other things that is used as obfuscation techniques to kind of pat out documents. And in this case, what we actually found is every iteration of the word document that we could find, had all of the functions and pretty much all the code within the document was replaced by different random lines.


Elif Kaya:

So there wasn't actually any contiguous lines within it. So if you looked at two docs, one might have some lines from Hamlet, one might have some lines from some other kind of literature document as well. But I imagine that it was more so just additional stuff to make it. If you're looking for a function in this document, it's gonna look different in this one. If I had to guess, I would say it's probably something similar to an actual defensive technique that we, we being, I guess, myself-


Nic Fillingham:

(Laughs)


Elif Kaya:

...had a few talks on conferences before called I believe Polyverse the company, um, coined the term, but Poly scripting where you use each iteration of something is gonna have a different function name and a different code. But it's all internally, um, it's all going to, the interpreter is going to still interpret it, even though it's random text from externally. In order to help protect against in the case of polyverse and polyscripting, protect WordPress sites from easy exploit. But in the case of the Shakespeare document, probably to prevent against easy YARA rules and things to detect their code, don't click the spicy button. (Laughs)


Nic Fillingham:

Elif. What do we know about these domains that have all been identified? The StrangeYou, the RandomYou, are, they still active? Have they been shut down? Do they get sent back to the DNS registrar? What's the process? What does it look like?


Elif Kaya:

So we have made sure that at least on our end, and turn to our products, that these domains and any new iterations of them, of these particular strains that we identify are blocked, as well as the malware we cover in the report. Those are within our products. As for the domains, because they're not hosted on Microsoft infrastructure, we kind of report them and that's, that's about as much as we can do in terms of their activity. I have no doubt that the operators behind this, will probably just create a new strain, but is also not necessarily set in stone, that the operators behind RandomYou and StrangeYou are the same operators. It could be that they are just operating in a similar kind of space and time to fulfill similar functions.


Elif Kaya:

There was a few campaigns where they both sent the same campaign, which lends a bit of credence to them potentially being at least similarly operated, but nothing concrete. So it is very highly likely that, that they'll just continue to operate under new strains. Uh, and probably the next strain that they'll have will either be more of these, uh, or they'll create a new one. And by a new one, I mean, instead of the word strange, maybe they'll use the word. I don't know, doc.


Nic Fillingham:

How about cat?


Elif Kaya:

Could be cat.


Nic Fillingham:

Or has that been exhausted.


Elif Kaya:

It could be cat. We haven't exhausted the number of cat domains that there could be.


Nic Fillingham:

So it sounds like, uh, you know, one of the things you said in the blog, and I think you mentioned it earlier that paying attention to infrastructure can actually allow uh, Defenders, SOCs, Blue teams to get ahead of a new campaign if a campaign is leveraging existing infrastructure. And so is that the takeaway from this blog post for those folks listening to the podcast right now and reading the blog, is your one sentence takeaway here, like pay attention to infrastructure? Don't forget about the infrastructure? Is that, is that sort of what you'd like folks to come away with?


Elif Kaya:

Yeah, I absolutely. And that's kind of my secret wish with the blog and my secret wish with most of the work that I do, is that it'll make Defenders and Blue teams focus less on the glamor and less on the kind of actor attribution and more on what is working right now. What do I need in my environment? What do I not need environment, my environment? And one of the key points I'll hone in on in order to kind of demonstrate that is these .us domains .us is a, a t- top level domain frequently used, uh, maliciously, but it's also frequently used for reasonable good purposes. What some of our tracking internally does and tracking that I've done before I went to Microsoft, is that attackers have trends of top level domains that they prefer to use from month to month. Certain malware strains, like using some top level domains, other, over others for a variety of reasons.


Elif Kaya:

But if you are running SOC and you were running Blue team get kind of creative about how you can take different steps to either monitor track or block infrastructure that is unnecessary to your organization. Not to impede or cause any kind of interference from productivity, but to kind of keep an eye on attacks and trends that you don't know about yet. For example, .su domains or .icu domains, uh, you might not have almost any benign presence for that in your environment. And so you might want to create custom alerts or custom rules to say like, "Hey, if I see this, maybe this could be the next malware campaign that Microsoft or somebody else hasn't written about but I'm a target of." And so kind of get creative about that, uh, especially if you have those kinds of capabilities within your network to filter on a mail comes in or mail comes out.


Natalia Godyla:

So just stepping away from the block for a minute, what about yourself personally speaking, what are you most passionate about in your work right now? What are you looking to achieve? What is your big goal I guess?


Elif Kaya:

So for myself and the reason that I, I'm still kind of in this field and at Microsoft doing the job that I'm doing right now is, I, I would really like to use these kinds of examples to bubble up what Blue teams that have less funding that are less glamorous and individual people can use in order to find threats. So I really want to try and shift the focus away from big groups or big actors or attribution and more towards what I consider the end goal for security. For me, which is how can I stop people from getting impacted. And so for myself and my own passions and interests insecurity outside of just what I do for work, I'm very focused in web security and browser security, I think there is a big gap that a lot of people focused as well as consumer security.


Elif Kaya:

A lot of these issues that we consistently pop up over and over again, kind of happen in part because of a lack of focus in consumer security. And by consumer, I kind of mean individual non corporations or small corporations. And so kind of the lack of focus in that and leaves a lot of people with the knowledge, but without the tools and resources easily available in order to kind of set themselves up for success. That's kind of a state of compromised websites that are used for botnets and et cetera. Right now, as well as, you know, privacy and security issues that individual users face in their regular day-to-day life with browser extensions and et cetera, where a lot of times browser extension research and browser research in general might get deprioritized due to its focus on individual consumer privacy versus things like malware, which focus a lot of the time on enterprise.


Elif Kaya:

But at least from my perspective, I'm very passionate about malvertising and, and the ways the advertising and web security and email security kind of coalesce around using a lot of the success that they have on individual people in order to leverage those attacks against bigger corporations later. That's where I like to focus a lot of my energy and research.


Nic Fillingham:

Uh, Elif Kaya, thank you so much for your time and thank you for, uh, contributing this great blog posts and helping us wrap our heads around email infrastructure.


Elif Kaya:

Thanks for having me.


Natalia Godyla:

Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.


Elif Kaya:

And don't forget to tweet us at msftsecurity or email us at securityunlocked@microsoft.com with topics you'd like to hear on a future episode. Until then stay safe.


Natalia Godyla:

Stay secure.

More Episodes

5/5/2021

Ready or Not, Here A.I. Come!

Ep. 26
Remember the goodoledays when wespent youthfulhours playing hide and seek with our friends in the park?Wellit turns out that game of hide and seek isn’t just for humans anymore.Researchers have begunputting A.I. to the test by having it play this favorite childhood gameover and overandhavingthe softwareoptimize its strategiesthrough automated reinforcement training.In today’s episode,hosts Nic Fillingham and Natalia Godyla speak with Christian Seifert and Joshua Neil about their blog postGamifying machine learning for stronger security and AI models,and how Microsoft is releasing this new open-sourcedcode to help it learn and grow.In This Episode, You Will Learn:What is Microsoft’sCyberBattleSim?What reinforcement learning is and how it is used in training A.I.How theOpenAIGym allowed for AI to be trained and rewarded for learningSome Questions We Ask:Is an A.I. threat actor science fiction or an incoming reality?What are the next steps in training the A.I.?WhowastheCyberBattleSimcreated for?ResourcesOpenAIHide and Seek:OpenAIPlays Hide and Seek…and BreaksTheGame! 🤖Joshua and Christian’sblog post:Gamifying Machine Learning for Stronger Security and AI ModelsChristian Seifert’sLinkedIn:https://www.linkedin.com/in/christian-seifert-phd-6080b51/Joshua Neil’sLinkedIn:https://www.linkedin.com/in/josh-neil/NicFillingham’sLinkedIn:https://www.linkedin.com/in/nicfill/NataliaGodyla’sLinkedIn:https://www.linkedin.com/in/nataliagodyla/Microsoft Security Blog: https://www.microsoft.com/security/blog/Transcript[Full transcript at https://aka.ms/securityunlockedep26]Nic Filingham:Hello and welcome to Security Unlocked! A new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft Security Engineering and Operations Teams. I'm Nic Filingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft Security. Deep dive into the newest threat intel, research, and data science.Nic Filingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security.Natalia Godyla:And now, let's unlock the pod.Nic Filingham:Hello, Natalia! Hello, listeners! Welcome to episode 26 of Security Unlocked. Natalia, how are you?Natalia Godyla:Thank you, Nic. And welcome to all our listeners for another episode of Security Unlocked. Today, we are chatting about gamifying machine learning, super cool, and we are joined by Christian Seifert and Joshua Neil who will share their research on building CyberBattleSim, which investigates how autonomous agents operate in a simulated enterprise environment by using high-level obstruction of computer networks and cyber-security concepts. I sounded very legit, but I did just read that directly from the blog. Nic Filingham:I was very impressed.Natalia Godyla:(laughs)Nic Filingham:If you had not said that you read that from the blog, I would've been like, "Wow". I would to like to subscribe to a newsletter. Natalia Godyla:(laughs)Nic Filingham:But this is a great conversation with, with Christian and Joshua. We talked about what is reinforcement learning. Sort of as a concept and how does that gonna apply to security. Josh and Christian also walked us through sort of why this project was created and it's really to try and get ahead of a future where, you know, malicious actors have access to some level of automated, autonomous tooling. Uh, and so, this is a new project to sort of see what a future might look like when there all these autonomous agents out there doing bad stuff in the cyber world.Natalia Godyla:And there are predecessors to this work, at least in other domains. So, they used a toolkit, a Python-based Open AI Gym interface to build this research project but there have been other applications in the past. OpenAI is, uh, well-known for a hide-and-seek. There is a video on YouTube that shows how the AI learned over time different ways to obstruct the agent and the simulated environment. Things like, blocking them off using some pieces of the wall or jumping over the wall.Nic Filingham:The only thing we should point out is that this CyberBattleSim is an open source project. It's up on GitHub and attained very much want researchers, and really anyone who's interested in this space to go and download it, go and run it, play around with it, and help make it better. And if you have feedback, let us know. There is contact information, uh, through the GitHub page but you can also contact us at Security Unlocked at Microsoft dot com and we can make sure you, uh, get in contact with the team. And with that, on with the pod?Natalia Godyla:On with the pod!Nic Filingham:Welcome to Security Unlocked, new guest, Christian Seifert. Thanks for joining us and welcome returning guest, Josh Neil, back to the podcast. Both of you, welcome. Thanks for being on Security Unlocked.Christian Seifert:Thanks for having us!Joshua Neil:And thanks, Nic.Nic Filingham:Christian, I think as a, as a new guest on the podcast, could we get a little introduction for our listeners? Tell us about, uh, what you do at Microsoft. Tell us about what a day to day look like for you.Christian Seifert:Sure, so I'm a, uh, research lead on the Security and Compliance team. So our overall research team supports a broad range of enterprising consumer products and services in the security space. My team in particular is focused on protecting users from a social engineering attack. So, uh, think of, like, fishing mails for instance. So we're supporting Microsoft Defender for Office and, um, Microsoft Edge browser.Nic Filingham:Got it, and Josh, folks are obviously familiar with you from previous episodes but a, a quick re-intro would be great. Joshua Neil:Thanks. I currently lead the Data Science team supporting Microsoft threat experts, which is our managed hunting service, as well as helping general res... cyber security research for the team.Nic Filingham:Fantastic, uh, again, thank you both for your time. So, today in the podcast, we're gonna talk about a blog post that came out earlier in this month, on April 8, called Gamifying Machine Learning for Stronger Security in AI Models, where you talk about a new project that has sort of just gone live called CyberBattleSim. First off, congratulations on maybe the coolest name? For, uh, sort of a security research project? So, like, I think, you know, just hats off there. I don't who came up with the name but, but great job on that. Second of all, you know, Christian if, if I could start with you, could you give us a sort of an introduction or an overview what is CyberBattleSim and what is discussed in this blog post?Christian Seifert:As I... before talking about the, the simulator, uh, the... let me, let me kind of take a step back and first talk about what we tried to accomplish here and, and why. So, if you think about the security space and, and machine learning in particular, a large portion of machine-learning systems utilized supervised, uh, classifiers. And here, essentially, what we have is, is kinda a labeled data set. So, uh, for example, a set of mails that we label as fish and good. And then, we extract, uh, threat-relevant features. Think of, like, maybe particular words in the body, or header values we believe that are well-suited to differentiate bad mails from good mails. And then our classifiers able to generalize and able to classify new mails that come in. Christian Seifert:There's a few, uh, aspects to consider here. So, first of all, the classifier generalizes based on the data that we present to it. So, it's not able to identify completely unknown mails. Christian Seifert:Second, is that usually a supervised classification approach is, is biased because we are programming, essentially, the, the classifier and what it, uh, should do. And we're utilizing domain expertise, red teaming to kind of figure out what our threat-relevant features, and so there's bias in that. Christian Seifert:And third, a classifier of who has needs to have the data in order to make an appropriate classification. So, if I have classifier that classifies fish mail based on the, the content of the mail but there is the threat-relevant features are in the header, then that classifier needs to have those values as well in order to make that classification. And so, my point is these classifiers are not well-suited to uncover the unknown unknowns. Anything that it has not seen, kinda new type of attack, it is really blind to it. It generalizes over data that, that we present to it. Christian Seifert:And so, what we try to do is to build a system that is able to uncover unknown attacks with the ultimate goal then to, of course, develop autonomous defensive component to defend against those attacks. So, that gives it a little bit of context on why we're pursuing this effort. And this was inspired by reinforcement learning research and the broader research community, mostly that is currently applied kinda in the gaming context. Christian Seifert:So OpenAI actually came out with a neat video a couple of years ago called Hide and Seek. Uh, that video is available on YouTube. I certainly encourage listeners to check it out, but basically it was a game of laser tag where you had a kinda, uh, a red team and a blue team, uh, play the game of laser tag against each other. And at first they, of course, randomly kind of shoot in the air and run around and there is really no order to the chaos. But eventually, that system learned that, “Hey, if a red team member shoots a blue team member, there's a reward.” and the blue team member also learned while running away from the red team member is, is probably a good thing to do. Christian Seifert:And so, OpenAI kinda, uh, established the system and had the blue team and the red team play against each other, and eventually what that led to is really neat strategies that you and I probably wouldn't have come up with. 'Cause what the AI system does, it explores the entire possible actions base and as result comes up with some unexpected strategies. So for instance, uh, there was a blue team member that kinda hid in a room and then a red team guy figured, “Hey, if I jump on a block then I can surf in that environment and get into the room and shoot the blue team member”. So that was a little bit an inspiration because we wanted to also uncover these unknown Christian Seifert:Unknownst in the security context.Nic Filingham:Got it. That's great context. Thank you Christian. I think I have seen that video, is that the one where one of the many unexpected outcomes was the, like, one of the, the, blue or red team players, like, managed to sort of, like, pick up walls and used them as shields and then create ramps to get into, like, hidden parts of the map? Uh, uh, am I thinking about the right video? Christian Seifert:Yes, that's the right video. Nic Filingham:Got it. So the whole idea was that that was an experiment in, in understanding how finding the unknown unknowns, using this game, sort of, this lazar tag, sort of, gaming space. Is, is that accurate?Christian Seifert:That's right, and so, they utilized reinforcement learning in order to train those agent. Another example is, uh, DeepMind's AlphaGo Zero, playing the game of Go, and, and here, again, kind of, two players, two AI systems that play against each other, and, over time, really develop new strategies on how to play the game of Go that, you know, humans players have, have not come up with. Christian Seifert:And it, eventually, lead to a system that achieved superhuman performance and able to beat the champion, Lisa Dole, and I think that was back in 2017. So, really inspiring work, both by OpenAI and DeepMind.Nic Filingham:Got it. I wonder, Josh, is there anything you'd like to- before we, sort of, jump into the content of the blog and, and CyberBattleSim, is there anything you'd like to add from your perspective to, to the context that Christian set us up on? Joshua Neil:Yeah. Thanks, Nic. I, I mean, I think we were really excited about this because... I think we all think this is a natural evolution of, of our adversaries, so, so, currently, our adversaries, the more sophisticated ones, are primarily using humans to attack our enterprises and, that means they're slow and they can make mistakes and they don't learn from the large amount of data that's there in terms of how to do attacks better, because they're humans.Joshua Neil:But I think it's natural, and we just see this, uh, everywhere and, all of technology is that people are bringing in, you know, methods to learn from the data and make decisions automatically, and it's- so it's a natural evolution to say that attackers will be writing code to create autonomous attack capabilities that learn while they're in the enterprise, that piece of software that's launched against the enterprise as an attack, will observe its environment and make decisions on the fly, automatically, from code. Joshua Neil:As a result, that's a frightening proposition because, I think the speed at which these attacks will proceed will be a lot, you know, a lot more quick, but also, being able to use the data to learn effective techniques that get around defenses, you know, we just see data science and machine learning and artificial intelligence doing this all over the place and it's very effective that the ability to consume a large amount of data and make decisions on it, that's what machine learning is all about. And so, we at Microsoft are interested in exploring this ourselves because we feel like the threat is coming and, well, let's get ahead of it, right? Let's go experiment with automated learning methods for attacks and, and obviously, in the end, for defense that, by implementing attack methods that learn, we then can implement defensive methods that will, that will preempt what the real adversaries are doing, eventually, against our customers.Joshua Neil:So, I think that's, sort of, a philosophical thing. And then, uh, I love the OpenAI Hide-and-Seek example because, you know, the analogy is; Imagine that instead of, they're in a room with, um, walls and, and stuff, they're on a computer network, and the computer network has machines, it has applications, it has email accounts, it has users, it's got a cloud applications, but, in the end, you know, an attacker is moving through an environment, getting blocked in various ways by defenses, learning about those blockings and detections and things and finding gaps that they can move through in, in very similar ways. So, I just, sort of, drawing that analogy back, Hide-and-Seek, it is what we're trying to do in cyber defense, you know, is, is Hide-and-Seek. And so the, I think the analogy is very strong.Nic Filingham:Josh, I just wanna quickly clarify on something that, that you said there. So, it sounds like what you're saying is that, while, sort of, automated AI-based attacking, attackers or attacking agents maybe aren't quite prevalent yet, they're, they're coming, and so, a big part of this work is about prepping for that and getting ahead of it. Is, is, is that correct?Joshua Neil:That's correct. I, I'm not aware of sophisticated attack machinery that's being launched against our enter- our customers yet. I haven't seen it, maybe others have. I think it's a natural thing, it's coming, and we better be ready.Christian Seifert:I mean, we , we see some of it already, uh, in terms of adversarial machine learning, where, uh, our machine learning systems are getting attacked, where, maybe the input is manipulated in a way that leads to a misclassification. Most of that is, is currently more, being explored in the research community.Natalia Godyla:How did you apply reinforcement learning? How did you build BattleSim? In the blog you described mapping, some of the core concepts of reinforcement learning to CyberBattleSIm, such as the environment, that action space, the observation space and the reward. Can you talk us through how you translated that to security?Christian Seifert:Yeah. So, so first let, let me talk about reinforcement learning to make sure, uh, listeners understand, kinda, how that works. So, as I mentioned, uh, earlier in the supervised case, we feed a label data set to a learner, uh, and then it able to generalize, and we reinforcement learning works very differently where, you have an agent that sits within an environment, and the agent is, essentially, able to generate the data itself by exploring that environment.Christian Seifert:So, think of an agent in a computer network, that agent could, first of all, scan the network to, maybe, uncover notes and then they're, maybe, uh, actions around interacting with the notes that it uncovers. And based on those interactions, the agent will, uh, receive a reward. That reward actually may be delayed by, like, there could be many, many steps that the agent has to take before the reward, uh, manifests itself. And so, that's, kinda, how the agent learns, it's, e- able to interact in that environment and then able to receive a reward. And so that's, kinda, what, uh, stands, uh, within the core of the, the CyberBAttleSim, because William Bloom, who is the, the brains behind the simulation, has created an environment that is compatible with, uh, common, uh, reinforcement learning tool sets, namely, the OpenAI Gym, that allows you to train agents in that environment.Christian Seifert:And so, the CyberBattleSim represents a simple computer network. So, think of a set of computer nodes, uh, the, the nodes represent a computer, um... Windows, Mac OS, sequel server, and then every node exposes a set of vulnerabilities that the agent could potentially exploit. And so, then, as, kind of, the agent is dropped into that environment, the agent needs to, first, uncover those nodes, so there's a set of actions that allows to explore the state space. Overall, the environment has a, a limited observability, as the agent gets dropped into the environment, you're not necessarily, uh, giving that agent the entire network topology, uh, the agent first needs to uncover that by exploring the network, exploiting nodes, from those nodes, further explore the network and, essentially, laterally move across the network to achieve a goal that we give it to receive that final reward, that allows the agent to learn.Natalia Godyla:And, if I understand correctly, many of the variables were predetermined, such as, the network topology and the vulnerabilities, and, in addition, you tested different environments with different set variables, so how did you determine the different environments that you would test and, within that particular environment, what factors were predetermined, and what those predetermined factors would be.Christian Seifert:So we, we determined that based on the domaine expertise that exists Christian Seifert:... is within the team, so we have, uh, security researchers that are on a Red Team that kind of do that on a day-to-day basis to penetration tests environments. And so, those folks provided input on how to structure that environment, what nodes should be represented, what vulnerabilities should be exposed, what actions the agent is able to take in- in terms of interacting and exploring that, uh, network. So our Red Team experts provided that information. Nic Filingham:I wonder, Christian, if you could confirm for me. So there are elements here in CyberBattleSim that are fixed and predetermined. What elements are not? And so, I guess my question here is if I am someone interfacing with the CyberBattleSim, what changes every time? How would you sorta define the game component in terms of what am I gonna have to try and do differently every time? Christian Seifert:So the- the CyberBattleSim is this parametrized, where you can start it up in a way that the network essentially stays constant over time. So you're able to train an agent. And so, the network size is- is something that is dynamic, that you can, uh, specify upon startup. And then also kinda the node composition, as well as ... So whether ... how many Windows 10 machines you have versus [inaudible 00:19:15] servers, as well as the type of vulnerabilities that are associated with each of those nodes. Nic Filingham:Got it. So every time you- you establish the simulation, it creates those parameters and sort of locks them for the duration of the simulation. But you don't know ... The agent doesn't know in advance what they will d- they will be. The agent has to go through those processes of discovery and reinforcement learning. Christian Seifert:Absolutely. And- and one- one tricky part within reinforcement learning is- is generalizability, right? When you train an agent on Network A, it may be able to learn how to outperform a Red Team member. But if you then change the network topology, the agent may completely flail and not able to perform very well at all and needs to kind of re- retrain again. And that- that's a common problem within the- the re- reinforcement learning research community. Natalia Godyla:In the blog you also noted a few opportunities for improvement, such as building a more realistic model of the simulation. The simplistic model served its purpose, but as you're opening the project to the broader community, it seems l- that you're endeavoring to partner with the other researchers to create a more realistic environment. Have you given some early thought as to how to potentially make the simulation more real over time? Christian Seifert:Absolutely. There is a long list of- of things that we, uh, need to think about. I mean, uh, network size is- is one component. Being able to simulate a- a regular user in that network environment, dynamic aspects of the network environment, where a node essentially is added to the network and then disappears from the network. Uh, all those components are currently not captured in the simulation as it stands today. And the regular user component is an important one because what you can imagine is if we have an attacker that is able to exploit the network and then you have a defender agent within that network as well, if there is no user component, you can very easily secure that network by essentially turning off all the nodes. Christian Seifert:So in- a defender agent needs to also optimize, uh, to keep the productivity of the users that are existing on the network high, which is currently not- not incorporated in- in the simulation. Nic Filingham:Oh, that's w- that's amazing. So there could be, you know, sort of a future iteration, sort of a n- network or environment productivity, like, score or- or even a dial, and you have to sort of keep it above a particular threshold while you are also thwarting the advances of the- of the agent. Christian Seifert:Absolutely. And I mean, that is, I think, a common trade off in the security space, right? There are certain security m-, uh, measures that- that make a network much more secure. Think of like two-factor authentication. But it does u- add some user friction, right? And so, today we're- we're walking that balance, but I'm hoping that there may be new strategies, not just on the attacker's side, but also on the defender's side, that we can uncover that is able to provide higher level of security while keeping productivity high. Nic Filingham:I think you- you- you have covered this, but I- I'd like to ask it again, just to sort of be crystal clear for our audience. So who is the CyberBattleSim for? Is it for Red Teams? Is it for Blue Teams? Is it for students that are, you know, learning about this space? Could you walk us through some of the types of, you know, people and- and roles that are gonna use CyberBattleSim?Christian Seifert:I mean, I think that the CyberBattleSim today is- is quite simplistic. It is a simulated environment. It is not ... It'-s it's modeled after a real world network, but it is far from being a real world network. So it's, uh, simplistic. It's simulated, which gives us some advantages in terms of, uh, scalability and that learning environment. And so at this point in time, I would say, uh the simulation is really geared towards, uh, the research community. There's a lot of research being done in reinforcement learning. A lot of research is focused on games. Because if you think about a game, that is just another simulated environment. And what we're intending to do here with- with some of the open source releases is really put the spotlight on the security problem. And we're hoping that the- the reinforcement learning researchers and the research community at large will pay more attention to this problem in the security domain. Nic Filingham:It's currently sort of more targeted, as you say, as- as researchers, as sort of a research tool. For it to be something that Red Teams and Blue Teams might want to look at adopting, is that somewhere on a road map. For example, if- if you had the ability to move it out of the simulation and into sort of a- a- a VM space or virtual space or perhaps add the ability for users to recreate their own network topology, is that somewhere on your- your wishlist? Christian Seifert:Absolutely. I think there's certainly the goal to eventually have these, uh, autonomous defensive agent deployed in real world environments. And so in order to get to that, simulation needs to become more and more realistic in order to achieve that. Joshua Neil:There's a lot of work to be done there. 'Cause reinforcement learning on graphs, big networks, i- is computationally e- expensive. And just a lot of raw research, mathematics and computing that needs to be done to get to that real- real world setting. And security research. And in incorporating the knowledge of these constraints and goals and rewards and things that ... T- that takes a lot of domain research and getting- getting the- the security situation realistic. So it's hard. Christian Seifert:In the simulation today, it provides the environment and ability for us to train a Red Team agent. So an agent that attacks the environment. Today, the defender is very simplistic, modeled probabilistically around cleaning up machines that have been exploited. So as kinda the next point on the wishlist is really getting to a point where we have the Red Team agent play against a Blue Team agent and kinda play back and forth and see kinda how that influences the dynamic of the game. Natalia Godyla:So Christian, you noted one of the advantages of the abstraction was that it wasn't directly applicable to the real world. And because it wasn't approved as a safeguard against nefarious actors who might use CyberBattleSim for the wrong reason. As you're thinking about the future of the project, how do you plan to mitigate this challenge as you drive towards more realism in the simulation? Christian Seifert:That is certainly a- a- a risk of this sort of research. I think we are still at the early stages, so I think that risk is- is really nonexistent as it stands right now. But I think it can become a risk as the simulation becomes more sophisticated and realistic. Now, we at Microsoft have the responsible AI effort that is being led at the corporate level that looks at, you know, safety, reliability, transparency, accountability, e- et cetera, as kind of principles that we need to incorporate into our AI systems. And we, early on, engaged the proper committees to help us shape the- the solution in a responsible fashion. And so at this point in time, there weren't really any concerns, but, uh, as the simulation evolves and becomes more realistic, I very much expect that we, Christian Seifert:... be, uh, need to employ particular safeguards to prevent abuse. Nic Filingham:And so without giving away the battle plan here, wh- what are some other avenues that are being, uh, explored here as part of this trying to get ahead of this eventual point in the future, where there are automated agents out there in the wild? Joshua Neil:This is the- the core effort that we're making, and it's hard enough. I'll also say I think it's important for security folks like us, especially Microsoft, to try hard things and to try to break new ground and innovation to protect our customers and really the world. And if we only focus on short-term product enhancements, the adversaries will continue to take advantage of our customers' enterprises, and we really do need to be taking these kind of risks. May not work. It's too ... It's really, really hard. And t- and doing and in- in purposefully endeavoring to- to- to tackle really hard problems is- is necessary to get to the next level of innovation that we have to get to. Christian Seifert:And let me add to that. Like, we have a lot of capabilities and expertise at Microsoft. But in the security space, there are many, many challenges. And so I don't think we can do it alone. Um, and so we also need to kinda put a spotlight on the problem and encourage the broader community to help solve these problems with us. And so there's a variety of efforts that we have pursued over the last, uh, couple of years to do exactly that. So, about two years ago we published a [inaudible 00:28:52] data science competition, where we provided a dataset to the broader community, with a problem around, uh, malware classification and machine risk identification and basically asked the community, "Hey, solve this problem." And there was, you know, prize money associated with it. But I really liked that approach because we have ... Again, we have a lot of d- expertise on the team, but we're also a little bit biased, right, in- in terms of kinda the type of people that we have, uh, and the expertise that we have. Christian Seifert:If you present a problem to the broader research community, you'll get a very different approaches on how people solve the problems. Most likely from com- kind of domains that are not security-related. Other example is an RFP. So we funded, uh, several research projects last year. I think it was, uh, $450,000 worth of research projects where, again, we kind of laid out, "Here are some problems that are of interest that we wanna put the spotlight on, and then support the- the research community p- to pursue research in that area." Nic Filingham:So what kind of ... You know, you talk about it being, uh, an area that we all sort of collectively have to contribute to and sort of get b- behind. Folks listening to the podcast right now, going and reading the blog. Would you like everyone to go and- and- and spin up CyberBattleSim and- and give it a shot, and then once they have ... Tell us about the- the types of work or feedback you'd like to see. So it's up on GitHub. What kind of contributions or- or feedback here are you looking for from- from the community? Christian Seifert:I mean, I'd really love to have, uh, reinforcement learning researchers that have done research in this space work with the CyberBattleSim. Kinda going back to the problem that I mentioned earlier, where how can we build agents that are generalizable in a way that they're able to operate on different network topology, different network configuration, I think is an- an- an exciting area, uh, that I'd love to see, uh, the research community tackle. Second portion is- is really enhancing the simulation. I mentioned a whole slew of features that I think would be beneficial to make it more realistic, and then also kinda tackle the problem of- of negatively impacting potential productivities of- of users that operate on that network. So enhancing the- the simulation itself is another aspect. Nic Filingham:Josh, anything you wanted to add to that? Joshua Neil:Yeah, I mean, I- I think those were the- the major audiences we're hoping for feedback from. But a- al- also like Christian said, if a psychologist comes and looks at this and has an idea, send us an email or something. You know, that multidisciplinary advantage we get from putting this out in the open means we're anticipating surprises. And we want those. We want that diversity of thought and approach. A physicist, "You know, this looks like a black hole and here's the m- ..." Who knows? You know, but that's- that's the kind of-Nic Filingham:Everything's a black hole to a physicist- Joshua Neil:(laughs) Yeah. Nic Filingham:... so that's, uh ... Joshua Neil:So, you know, I think that diversity of thinking is what we really solicit. Just take a look, yeah. Anybody listening. Download it. Play with it. Send us an email. We're doing this so that we get your- your ideas and thinking, for us and for the whole community. Because I think we- we also believe that good security, uh, next generation security is developed by everybody, not just Microsoft. And that there is a- there is a good reason to uplift all of humanity's capability to protect themselves, for Microsoft but for everybody, you know? Natalia Godyla:So Christian, what are the baseline results? How long does it take an agent to get to the desired outcome? Christian Seifert:So the s- simulation is designed in a way that also allows humans to play the game. So we had one of our Red Teamers to actually play the game and it took that person about 50 operations to compromise the entire network. Now when we take a- a random agent that kinda uninformed takes random actions on the network, it takes about 500 steps. So that's kind of the- the lower baseline for an agent. And then we trained, uh, a Deep Q, uh, reinforcement learning agent, and it was able to accomplish, uh, the human baseline after about 50, uh, training iterations. Again, network is quite simple. I wouldn't expect that to hold, uh, as kinda the- the simulation scales and becomes more complex, but that was, uh, certainly an encouraging first result. Joshua Neil:And I think the- the significant thing there is, even if the computer is- takes more steps than the human, well, we can make computers run fast, right? We can do millions of iterations way faster than a- than a human and they're cheaper than humans, et cetera. It's automation. Nic Filingham:Is there a point at which the automated agent gets too good, or- or is there sort of a ... What would actually be the definition of almost a failure in this experiment, to some degree? Joshua Neil:I think one- one is to- to sort of interpret your question as it could be overfed. That is, if it's too good, it's too specific and not generalized. And as soon as you throw some different set of constraints or network at it, it fails. So I think that's a- that's a real metric of the performances. Okay, it- it learned on this situation, but how well does it do on the next one? Nic Filingham:Is there anything else, uh, either of you would like to add before we wrap up here? I feel like I've covered a lot of ground. I'm gonna go download CyberBattleSim and- and try and work out how to execute it. But a- anything you'd like to add, Christian? Christian Seifert:No, not from me. It was, uh, great talking to you.Natalia Godyla:Well, thank you Josh and Christian, for joining us on the show today. It was a pleasure. Christian Seifert:Oh, thanks so much. Joshua Neil:Yeah, thanks so much. Lots of fun. Natalia Godyla:Well, we had a great time unlocking insights into security, from research to artificial intelligence. Keep an eye out for our next episode. Nic Filingham:And don't forget to tweet us at MSFTSecurity, or email us at securityunlocked@microsoft.com, with topics you'd like to hear on a future episode. Until then, stay safe. Natalia Godyla:Stay secure.
4/28/2021

Knowing Your Enemy: Anticipating Attackers’ Next Moves

Ep. 25
Anyonewho’severwatched boxing knows that great reflexes can be the difference between achampionshipbeltand a black eye.The flexing ofan opponent’s shoulder, the pivot of theirhip-a good boxer will know enoughnot only topredictand avoidthe incoming upper-cut, but willknow how to turn the attack back on theiropponent.Microsoft’s newestcapabilities in Defender puts cyber attackers in the ring and predicts theirnext attacks as the fight is happening.On today’s episode,hosts Nic Fillingham and Natalia Godyla speak with ColeSodja, Melissa Turcotte, and Justin Carroll(and maybe even a secret, fourth guest!)abouttheirblogposton Microsoft’s Security blogabout the new capabilities of using an A.I.to see the attacker’s next move.In This Episode, You Will Learn:• What kind of data is needed for this level of threat detection and prevention?• The crucial nature of probabilistic graphical modeling in this process• The synergistic relationship between the automated capabilities and the human analystSome Questions We Ask:• What kind of modeling is used and why?• What does the feedback loop between program and analyst look like?• What are the steps taken to identify these attacks?Resources:Justin, Melissa’s, and Cole’s blog post:https://www.microsoft.com/security/blog/2021/04/01/automating-threat-actor-tracking-understanding-attacker-behavior-for-intelligence-and-contextual-alerting/Justin Carroll’s LinkedIn:https://www.linkedin.com/in/justin-carroll-20616574/Melissa Turcotte’s LinkedIn:https://www.linkedin.com/in/mturcotte/ColeSodja’sLinkedIn:https://www.linkedin.com/in/cole-sodja-a255361b/Joshua Neil’s LinkedIn:https://www.linkedin.com/in/josh-neil/NicFillingham’sLinkedIn:https://www.linkedin.com/in/nicfill/NataliaGodyla’sLinkedIn:https://www.linkedin.com/in/nataliagodyla/Transcript[Full transcript at https://aka.ms/SecurityUnlockedEp25]Nic Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft Security engineering and operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories for Microsoft Security, deep dive into the newest threat intel, research and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security.Natalia Godyla:And now, let's unlock the pod. Welcome, everyone, to another episode of Security Unlocked, and hello, Nic, how's it going?Nic Fillingham:It's going well, good to see you on the other side of this Teams call. Although, you and I were in person not 24 hours ago. You were here in Seattle, we were filming some more episodes of the Security Show. I don't think we've really given listeners of the podcast a full, meaty introduction to the Security Show, have we? Do you wanna let listeners know what they can find?Natalia Godyla:We play games and hang out with experts in the industry and we've done everything from building robots with folks, to building blocks, to painting our nails. You can find the Security Show on our YouTube channel, so, YouTube.com/MicrosoftSecurity or you can go to aka.ms/securityshow. We talk with Chris Wysopal, the CTO and co-founder of Veracode on modern secure software development, and Dave Kennedy, who comes to talk to us about SecOps and everything you need for a survival kit in SecOps, so come come check them out.Nic Fillingham:Bad news is you, you have to deal with, uh, Natalia and I on another, uh, media format. But before you go there, make sure you listen to today's episode of Security Unlocked. We have a couple of returning guests. We have Cole and Justin, who have been on before, as well as Josh Neil, who comes on in the, in the last few minutes. And new guest, Melissa. They're all from the Microsoft 365 Defender research team, and they all co-authored a blog from April 1st called Automating Threat Actor Tracking, Understanding Attacker Behavior for Intelligence and Contextual Alerting, which is exactly what it is but I think it buries the lead. Natalia, you had a great TL;DR, what did they do?Natalia Godyla:The team used statistics to predict the threat actor group and the next stage in the attack and really early in the attack, so that we could identify the attack and inform customers so that they could stop it. I think what's really incredible here is, not only the ability to predict that information, but to just do it so early in kill chain. Nic Fillingham:Within two minutes after an attack begin, using this model, Microsoft threat experts were able to send a notification to the customer to let them know an attack was underway. The customer was able to do, you know, the necessary things to get that attack shut down. We'd love, as always, your feedback. Send us emails, securityunlocked@microsoft.com. Hit us up on the Twitters. On with the pod. Natalia Godyla:On with the pod. Nic Fillingham:Well, welcome back to the Security Unlocked podcast, Cole and Justin, and welcome to the Security Unlocked podcast, Melissa. Thanks for joining us today. We have three wonderful guests, with maybe a, a fourth special guest appearing at the end. And today we're gonna be talking about a blog post appearing on the Security blog from April the 1st, called Automating Threat Actor Tracking, Understanding Attacker Behavior for Intelligence and Contextual Alerting. All of the authors from that blog are here with us. Cole, if I could start with you, if you could sort of reintroduce yourself to the audience, give us a little bit, uh, about your role, what you do at Microsoft, and then perhaps hand off to one of your colleagues for the next intro.Cole Sodja:Sure. Will do, thank you. So, hi, I'm Cole. I work in the Microsoft 356 Defender group. I'm a statistician. Primarily my responsibilities are driving, kind of, research and innovation in general, with supporting threat analytics, threat hunting, threat research in general. Yeah, been doing that for about three years now, and I love it, and I that's a little bit about myself, I'll hand it over to Melissa. Melissa Turcotte:All right. My name's Melissa, I work with Cole, so in the same group, Microsoft 365 Defender. I'm also a statistician by background. I've been in the cyber domain for about probably seven years now. I was working for Department of Energy research laboratory in their cyber research group for five years, and I joined Microsoft a year ago. I like all sorts of problems related to cyber. My expertise probably would be in anomaly detection, but anything related to cyber, and there's data in a problem, I like to be involved.Nic Fillingham:And Justin.Justin Carroll:Hey. I also work in the Microsoft 365 Defender team, doing threat intelligence. My main focus is uncovering new threats and actor groups and understanding what they're doing, different modifications to how they're conducting their attacks, and the outcomes of those attacks, and then figuring out the most effective ways to either, communicate that out to customers or action on detection capabilities to stop them from succeeding.Nic Fillingham:Listeners of the podcast will note that you have a super sweet ninja turtles tattoo, is that correct? Justin Carroll:This is accurate, this is definitely accurate. Nic Fillingham:And, and we may or may not have a super secret fourth guest on this episode, who may join us towards the end, who you would, you would know from an very early episode of the podcast, but perhaps we'll keep them secret until the very end. Thank you all for joining us, thank you for your time. Again, we're referring to a, a blog post that, that all of you authored from April 1st. This is a, quite a complex, and, and sort of technical blog post, which I know a lot of our audience will love. Nic Fillingham:I got a little lost in the math, but I, I absolutely was enthralled by what you all have undertaken here. Cole, if I could start with you, can you give us, give us an overview of what's covered in this blog post, and sort of what this project was, how you tackled it, and what we're gonna talk about, uh, on this episode today.Cole Sodja:Yeah. So if I step back, being someone kind of still fairly new in learning, uh, to cyber security, uh, I approached things pretty much with just using data, right? Doing data driven imprints, as I'd say. And through my research, what I started to, um, kinda ask myself is, can we kinda get ahead of cyber security attacks, you know, from a post-breach perspective? Once we see an adversary in a network, can we start to make some predictions, basically, on what they're likely gonna do? Who is the adversary, or is it human operated, is it an automated script, for example. And then if we recognize the adversary, kinda recognize their tactics, their techniques, their procedures, can we say, okay, we're, we're likely gonna see they're gonna ransom this enterprise, for example.Cole Sodja:So I tried to look at it as more of a data mining exercise initially, it's like, can I recognize these type of patterns, and then how predictive are these patterns that we're seeing in terms of what likely is gonna occur. Or put it another way, what type of threat is this, essentially, to the enterprise? So, so that's kinda the background, the motivation. Now, when I started this project, back with Justin and then with Melissa, it started really as let's look for particular, uh, threat actors that we're aware of, that we recognize, that we know about, and see, like, can we start, from a data perspective, classifying okay, is it this group, is it that group, and what does this group tend to do? Cole Sodja:And one of the challenges in that is, is sparsity. Basically, we don't have a lot of labels sitting around out there saying, it's threat actor group A, B, C, D, and so on. We have handfuls of those. Some of these actors, they don't tend to do attacks very frequently, right? They're extremely sparse. So, so one challenge of this, and one the motivation is, how can we actually partner with threat intelligence, for example, and our threat hunters, to try and essentially encode or extract some of their information to help us build models, to help us reason over the uncertainty, essentially. Cole Sodja:And when we say probabilistic modeling, that's what we mean. It's how do we actually quantify this uncertainty, both in what we believe about the actors, or the adversaries in general, as well as what they're gonna do, right, once they've breached your network. So that's kinda how it started, and what this blog's really about is kinda giving a walk-through, essentially, of what we did initially with this research. It started with, and Justin will talk about this in a moment, it started with looking at few, select threat actors that are very serious. Cole Sodja:We started to understand their behaviors more and more and we thought it was a good opportunity, initially, to try and build a model to, again, understand what they're doing, track what they're doing, because they do change their tactics over time, as well as just see if we could get ahead of them. Can we actually notify a customer in advance, before, uh, for example, their organization's ransomed? So, so that's one part of the blog that we'll discuss, and I'll hand it over to my good friend Justin to take it from here.Justin Carroll:So, like, one of the, the main challenges that we kinda face in the intelligence sphere is understanding the particulars of an actor and when they are present in an environment. A lot of times, you'll see the intelligence is really focused on a very particular indicator such as, like, a known IP address that's malicious, or a single behavior. But it's kinda difficult to frequently pivot them out to understand when a suspected attacker is in an environment. A lot of that is due because they don't always do the exact same behaviors when they are compromising... Organization or device. There will be some variation and it basically requires manual enrichment a lot of the times of devices to try and understand the specifics of the attacks and what Justin Carroll:... the final outcomes o- wh- out of that attack, so this opportunity presented one to work with data scientists to, like, really supercharge our efforts so that we could kinda come in understanding a much bigger picture and knowing, essentially, what behaviors that we saw occur and then which ones we might suspect. A lot of times with these human operated ransomware ones, the time to alert, to notify of the expected outcome is often fairly short, in particular with, uh, one of the ones that we worked on to kinda test this method out. We had seen very short instances from time to compromise to ransom, so, um, this was to try and see if we could have a, a highly confident method of enriching that intelligence, um, and then working with other teams to get those alerts out.Natalia Godyla:If I could jump in here for a moment. So, at the beginning of your description, you noted that typically you'd use manual enrichment. Can you talk a little bit about that? So prior to this probabilistic model, how did you go through that manual enrichment process to try to, uh, predict what threat actors they were or determine what stage of an attack it was?Justin Carroll:It would be something along the lines of, let's say, you had intelligence from either a partner team or open source intelligence that says, you know, "These threat actors are using this IP address as part of their attack," and then looking for the presence of that and then finding out what actually occurred on those devices to understand the entirety of the attack, or looking more generically and saying, like, "Okay, we know these attackers like to use a particular behavior as part of their credential theft," and then so looking for all sorts of instances of that credential theft and then kinda continuing to pivot down into one that is leading to the behavior that y- you're looking for. One of the difficulties that you'll see in particular with this and other actors is, like, they will use multiple shared open source tools and payloads. Um, many of them aren't even malware, they're clean tools with legitimate purposes, so it can make it difficult to try and suss out the ones from malicious versus administrative use, so you have to look for that combination of different behaviors to indicate something malicious is afoot.Nic Fillingham:Justin, if I look at the blog, I think it might be the first chapter here, there's a MITRE ATT&CK framework diagram, Figure One, and it, uh, outlines sort of the steps taken here for how this model was able to, with high confidence, identify the, the actor and, uh, send an alert to the customer who was able to shut it down. I wonder if you could sort of, could you walk us through this, these sort of six steps as an example of, of how this work, how this worked in, in sort of real life?Justin Carroll:Yeah. I can walk through basically from a model's perspective, essentially, how it works. Timing, that's more a function of, like, how the attack, uh, typically progresses with this actor. Technically speaking, what the model's really doing is it's encoding each behavior we have, in this case, each MITRE technique in particular in terms of what's the confidence that once we see, for example, initial access follow... Under, let's say, RDP brute force, followed by lateral tool transfer with subset of tools recognized, that particular sequence right there, that's where the model would be like, "Okay, the probability that it's this particular threat actor group conditional on those two things occurring in sequence will be X," and that sequence could occur in a matter of minutes or even days and weeks, dependent on the actor, of course, we're talking about. Justin Carroll:With the, the actor we're showing in this graph, this actor typically will penetrate a network through RDP brute force, but then w- sometimes the, they won't immediately transfer their tools. They might wait a day or two, or sometimes they'll, they'll do it very fast, like, once they basically compromise a log-in then, uh, they'll, they'll go to that machine, there might be some, um, discovery related commands before they transfer or they might just transfer their tools and then that will be the attack box, basically, in which they stage their attack, and then they'll do some additional things.Justin Carroll:So at each step, basically, or each stage of the attack, as we like to call it, the model is basically gonna then update its probabilities and say, "Okay, based on all the information I've seen up to this stage, the probability that it's this actor is P and now, conditional that it's this actor with probability P, the probability that we'll now see, for example, defense evasion and this 'tack will be Q," or, or we could even go further in the attack stage to say, "Now, given all this, what's the probability that we'll see, for example, ransomware or inhibit system recovery in the coming hour? Or in the coming, you know, X time?" Justin Carroll:So the model's able to do that, but it's primarily conditional on the stages it's observed up to a point in time, not so much in terms of the time it takes for the actors to do X.Natalia Godyla:So, in this blog and in our discussion today, we're gearing up to talk about probabilistic graphical modeling as a way to address the challenge that, Cole and Justin, you've set up for us today, and, and for any of our listeners who'd like to follow along in the blog, the blog is titled "Automating threat actor tracking: Understanding attacker behavior for intelligence and contextual alerting" and you can find it on the Microsoft Security blog. I'd love to dive into the probabilistic graphical modeling and perhaps start with a definition of what that means. So, M- Melissa, could you give us an overview of this approach?Melissa Turcotte:Yeah. We have this problem which what they are essentially saying is, we have a collection of things which... I'm a statistician so I often call them variables, but, you know, features, if you will, if that's m- more easy for you to understand, but we, th- these TTPs, th- right. The sets of things that the actors are doing, and we have a collection of them. And given some collection of these, we wanna make a statement about whether or not it's ransomware or whether it's not a specific threat actor, or a group of actors. Right? And this is, this is, like, a perfect, um, example of where probability can help you make these decision, and one thing I'd like to stress is that no one of these features gives you enough information about whether or not it's this actor or this, this group of actors, or it's ransomware, you know, whatever your variable interest is.Melissa Turcotte:It really is the collection of these together that, you know, kind of in Justin's mind, as an analyst, he's, he's making these connections in his head, and I wanna be able to replicate that in some sense, I wanna take into account his knowledge and kind of his decision making process, combined with the data that I have, to make these probabilistic statements about what I think is happening. And graphical models are really great here, probabilistic graphical models in particular, as they kind of provide this joint probability distribution over all these features, and the variable of interest, in this case, is kind of, maybe is it this actor, but not necessarily. I mainly wanna know something about any one of these other features. I may already know it's this actor, and I may wanna be like, "Wh- what are the common things I see this actor do?"Melissa Turcotte:So, so graphical models really shine in this case where you have this collection of things that you are observing, and you kind of want to ask questions about any subset of them. Given some observations of others, and so th- this is a really great tool to use in this setting, and it's also quite interpretable. So if you kind of look, if you're looking at the blog and you see this Figure Two, which is a toy example, but y- you kind of, as a human, you can look at that and you can kind of understand that, "Okay, so I'm seeing transfer tools and lateral movement are related." Um, and you can kind of understand sort of wh- what the relationships the model is making. Um, and so that kind of provides this extra, you know, benefit of this in that, yeah, I can talk an analyst through what this kind of is showing and then i- it's quite interpretable for them even if they don't understand the underlying maths, and that's kind of something we really wanna strive for. Um, you shouldn't have to understand the underlying maths to kind of understand the decisions that are being made.Melissa Turcotte:It's really attractive in this sense, and then the Bayesian networks, why I really like it is kind of, the Bayesian paradigm is... So you, you have, you know, statistics, generally, or data science, you have some data and you're kind of, you know, making inference given the set of data to make statements about things of interest. So the data tells you something about your beliefs and the state of the world, but you have your own subjective beliefs about wh- what you think could and could not happen. The, the Bayesian paradigm kind of combines those two things, so it's, you have your beliefs and then you have what the data is telling you, a- and your ultimate kind of predictions are based on the combination of those things. And generally, the, the way it works is the more data you have, the data will always win through.Melissa Turcotte:So this problem, bringing it back to attacker prediction, is a case where we don't have a lot of data, right? We don't... Companies get attacked... Or we say, companies get attacked all the time but not at the scale at which we collect the underlying data, so like, you know, we have, you know, you as a user are performing actions, logging into computers you use... You know, this shows up in the data thousands of times a day, whereas an attack happens kind of, like, on a monthly scale, so c- the scales of attacks to the data we're getting is just really small, and then when you go into attacks that kind of we've labeled as being attributed to a threat actor, I mean, that's even way smaller. So it's, it's kind of a small data problem, uh, in terms of the number of labels you have.Melissa Turcotte:But what we do have is this analysts who have spent years tracking these people and have their kind of, you know, beliefs about what they do and how they changed over time. And so we Melissa Turcotte:Wanna capture that. We definitely want to include the evidence we see and the data, but we wanna capture that really rich knowledge that we get from the analysts. And so kind of that's where the Bayesian network part becomes attractive because it, it provides a very principled way to, to capture the analysts' expertise, combine that information with the data we're seeing to make these ultimate predictions.Natalia Godyla:For our audience, could you really quickly describe a Bayesian network?Melissa Turcotte:So, a Bayesian network is a way of building a model for a collection of variables whereby the idea is that you have different variables which are related to each other. It, it, it kind of helps draw out or show what those relationships are so, like, in the graph, you know, if there's an arrow from impact... Or from transfer tools to impact that's saying if I see transfer tools, that has a direct impact... I'm gonna use the word impact twice here. Has a direct impact on whether or not I'm going to see impact. So, so it's kind of the way the variables relate to each other and the way the probabilities change according to those relationships. And so a Bayesian network encodes all this information. Nic Fillingham:If I can take another swing at that one... Thank you, Melissa. I'm wondering what were some of the other, uh, techniques that you either considered for this approach? Like, did you experiment with other methods and then ultimately chose Bayesian?Melissa Turcotte:Yes, um, in fact, uh, so the initial kind of... The perhaps most obvious thing to do is to c- to think of decision trees, right? You s- you're, you're, you're seeing, you know, these things over time. Okay, I saw, um, what was the first one? Initial access with this... You don't go as broad as initial access, but I saw initial access using this, you know, minor technique. And so you can kind of think, like, you, you, you have a tree that's kind of... Okay, I saw this, I didn't see this, but I saw this and I didn't see this, so now I think it's this actor. But kind of where this is preferable is the fact that, as Paul says, we don't want to see the whole attack happen before we make a statement about what we think it is. And Bayesian networks work really well in, in the absence of some observed variables. Cole Sodja:Yeah, I'll just quickly chime in. I agree with Melissa. So, I did experiments, for example, with several models including decision trees. Even, um, different forms of Bayesian decision trees like BART for example. And in addition to what Melissa is saying where, for example, predicting the probability that it's threat actor conditioned on certain variables we saw, uh, we might also, as Melissa pointed out, want to say, okay, let's predict, for example, that this threat actor is going to do impact or a certain form of impact. And with decision trees, that means basically you're building multiple decision trees to do that. You can't just build one decision tree... Well, let's put it this way. You can't easily build one decision tree to have multiple target variables. That's something you get for free with the Bayesian network. Another thing I'll say in addition to what, um... To marginalization is the Bayesian network is more general. So, it could actually handle kind of a broader graphical structure. The decision tree is a specific graph. Cole Sodja:So, it kind of already inhibits you, if you will, to learning a certain structure over the data. Whereas the Bayesian nets, they could give you a little more general structure. We could also build these models that are time dependent, what are called dynamic Bayesian networks. That's something much harder to do with tree models. So, it's just a more flexible model as well as I would say. In my experiments, the Bayesian network did perform better on average than the set of decision trees I considered.Nic Fillingham:I'd like to better understand the relationship between this model and folks like Justin. So, is Justin, as a very experienced threat analyst, is Justin helping you define labels and helping you sort of build some of the initial... I'm, gonna get the taxonomy wrong here, so please correct me. But the initial sort of properties of the model? Or is, is Justin, as an analyst, interpreting what you sort of think you have in the model? How, how do I understand the relationship between the analyst and, and how they're providing their expertise into, into this model?Melissa Turcotte:All three.Nic Fillingham:Oh, great. (laughs)Melissa Turcotte:All three things you said is actually correct. So, so hopefully we, we've explained it somewhat well. So, yes. The first stage, right Justin? The analysts are providing us our label data. So, yes. That's the first thing. And then they also help us kind of, you know, you have the raw data, but that's kind of... There's so much data processing that goes... That, that happens before it's kind of... This data's kind of in this tabular forms that's like, yes, we... You know, these are the features we are tracking, so think of your TTPs, the different notes in your graph. Getting the data into that, kind of that schema, the threat analysts help with. So, you know, help define what, what these tactics, techniques, and procedures are that we should track. Like you said, you, you can't be super broad. Lateral movement doesn't really have a lot of meaning, um, to kind of like the different ways in which someone can do lateral movement and how granular w- you want to go. Melissa Turcotte:So, we discuss with the analysts all the time to kind of build up, you know, the ontology, if you will. And then, you know, as a first stage, like I said, it's a small data sample, so we're like... Justin helps inform what the model thinks about in a probabilistic sense. So, you... One thing I might ask him, I, I would be like... If I saw net... you know I'm borrowing from our toy example, but if I saw a network scanning modify system process, transfer tools, but didn't see any of the others, do you think it would be this actor X? Or do you think it would be ransomware? And he would be like, hmm, I would probably 60% certain. I can take that information and encode that directly so that, in the absence of any data, the model would return 60%. It would... If I didn't see any data, it would return what Justin believed was the probability in the presence of a certain number of variables. Melissa Turcotte:And then we kind of see data and we update our beliefs over time based on that. And then, also, after we've kind of trained these things, I go back to Justin and say does this make sense to you? So, he, he's kind of involved in all three, the whole process.Nic Fillingham:Melissa, I think you're telling me you've built a virtual Justin. Melissa Turcotte:We... That, that is what we are literally trying to do. And back it up... And, you know, and back it up with data as well. I'd, I'd like to like... You know, I'm a firm believer that everyone has their subjective beliefs, Justin has beliefs as well. Oftentimes, I can prove analysts wrong. Be like, they think something, I'm like, well, the data is telling me something else. So, we need to figure out, you know, that discrepancy. But, yes. We are essentially trying to build virtual Jus- uh, Justins. Although, like, th- there... I don't think there's any stage upon which we won't need the analysts to constantly feed back in with the new information they have. Nic Fillingham:Got it. And then can it come full circle? Justin, how do you as an analyst, how do you get smarter and better at what you do by what this model is, is telling you? What's the feedback loop look like here for you?Justin Carroll:It's one of those where, basically, using the model kind of super-charged my abilities where, instead of having to look at this very granular kind of like ad hoc, oh, this may be interesting, now I have the instances already serviced to me, and I have a good understanding of what success rate through the kill chain the attacker was able to get. And maybe figure out which ones that I needed to enrich more to understand was there data that we can add into the model because they've done something different that we need to capture and then look for opportunities in that way. So, really, it's basically... It made it where, give or take, sometimes it would take anywhere from 10 to 20 minutes sometimes to try and figure out, like, is this who I think it is? And like, what have they done? What are their goals? To just looking at the result from the model. And within usually seconds, being like, yeah, that looks exactly right. That's... It's confirmed, I think that's spot on. Natalia Godyla:So, Justin, was there something that was the most surprising in working with this model? Something that the model taught you either about threat actors or any details about the features? Justin Carroll:One of the things was kind of reexamining My confidence levels on different parts of the attack. Um, where Melissa was stating, for instance, you know, the data suggesting this and the models coming to this conclusion, uh, you know, thinking that it's this probability, and there would be times where I'd have to kind of reevaluate and think, like, hmm, I might've been missing something or overestimating the prevalence of a particular thing and saying it's related to such. Like, uh, I can tend to get very biased based on my narrow scope of the attacks that I'm looking at and think that it's related to this thing, but the model was able to provide a lot of clarity to some of the behaviors that maybe I didn't think were as confident a signal or extremely confident signal and I wasn't giving them the appropriate weight. That's one of the advantages of using it to understand what the attacker's doing, is I let it do much of the leg work once everything's kind of coded in. And then occasionally, like if we found opportunities where it was like, hmm, this still isn't quite right, then it could be tuned as a c- um, as necessary. Justin Carroll:I think that was probably one of the biggest ones of kind of trying to work through and actually spell out, like, my own thinking processes when I'm evaluating the data. It was something that you just kind of do without thinking, where you're constantly, as an intelligence analyst, looking at data and making conclusions on that data. But you're not usually saying, like, okay, I saw this so I'm gonna give it a 60% probability that it's this. And like, you're, you're just kind of sometimes it's either gut intuition or working on it that way. But actually having the model encode and return back what it was understanding made a, a pretty big impact in trying to understand how my own decision processes work and basically how best to kind of think Justin Carroll:About these different, wide array of attacks that we're constantly investigating.Nic Fillingham:The types of indicators that you're building this model on, again please correct me on my taxonomy here, but you're not looking for, you know, NFO files or like ASCII art or, you know, the actual threat actors name being sort of hidden somewhere in the jpeg that they drop as a, as a for the LOLs, like, they're... You're not looking for a sort of a literal signature of these threat actor groups, you're, you're, what you're, what you're doing is you're, you're seeing the actions that have been taken and without any other way of attributing them to an individual group, you're piecing them together. Nic Fillingham:And as you, as you get more actions and you piece them together based on the, the labels that you get from people like Justin, you're able to, to ultimately have a high probability that it's this threat group actor and they're doing this thing and they're likely to do this thing next. Have I got that right? You're, they're... In no way shape or form are you actually finding a secret text file that has the name, you know, the, the, the handles for all the hackers who are doing it for the LOLs.Cole Sodja:So let me just quickly jump in, you pretty much nailed it. I'll say this, so, we wanted to do both actually, right, because we don't want to restrain the model if it's, if core's gonna add predictive power, so like you said, we're not actually searching, grepping for example, for a threat actor name and some file or image, certainly not that level. But, for example, some of the actors, maybe they have common infrastructure, maybe they use particular types of tools in their attack typically, right? Like, maybe there's a SHA-1 out there they've used a lot in their attack, or, or recurring IP addresses they use as part of brute forcing. Cole Sodja:Those are there, but those are very specific and if you just relied on those, like Melissa was saying, either one or a few of those, you're not gonna generalize. You'll probably miss that attacker, right? But we certainly don't want to exclude it from the model because, um, if we happen to see that, the model will, uh, come back with a different type of probability, right? It'd be like, okay. Now the model might be more confident early, rather than waiting to see how the rest of the kill chain progresses. On the more general side, we probably won't go to the MITRE categories, 'cause they're a little too general, right? But if we go to some of the sub techniques, we don't actually have to look at the particular types of executables, or tools, or IPs used. Cole Sodja:Sometimes just the timing and sequencing is enough actually, to narrow down to, maybe not a particular threat actor, but a group of actors or, more generally, we can say with high competence, you know, this is a human adversary. They're taking this amount of time to do discovery commands, they're, they're doing lateral these type of ways. And the model could recognize that, even without knowing the particular commands, it's just seeing the more general techniques involved, right? So we do a bit of both, actually. We tend to want to rely more on, kind of, the general attacks or indicators as you're saying, that's right. But, we certainly don't want to throw away specifics that are reuse because we could get ahead of the attack much earlier too. So it's a bit of both at the end of the day.Melissa Turcotte:So yes, Nic, if, if, if you have an evil bit, look for the evil bit. You don't need data science for that. Nic Fillingham:(laughs)Natalia Godyla:And how is this model being used today, meaning is this a model that's being used by our internal security team to protect Microsoft and its customers, is it being used by a Microsoft threat experts group or is this actually embedded in some of our solutions today, and our customers are feeling that benefit? And what is the future intent of the model?Justin Carroll:One of those... So, there are multiple uses that are in place for the model. So one of the big things for me, so in my own selfish interest, it's intelligence, it's one of the easiest ways that I can keep tabs on the attacker and continually build new profiles and understand, basically, reports out, this is what they're doing, this is how they're doing it, this is how active they are. Like, are we seeing, you know, large volumes of their attack, are they taking a break, that kinda stuff. Then, the Microsoft threat experts are using it as a signal to help understand attacks early on in the kill chain so that they can get those notifications out ideally before the ransom, which can be quite difficult a lot of the times depending on the adversary and how quickly they seek to ransom. A lot of times there isn't a great deal of time.Cole Sodja:Yeah, there's other products, for example, M365D. So, um, there are plans, uh, it requires some engineering, ultimately, because this is a big product, um, huge customer base and so on. But there are already plans in motion to take what we've built already, as part of this framework, and integrate that into that product. There's other products as well, both from a threat intelligence perspective, and possibly kind of from SOC alerting perspective as well, that I'm in active discussions with other products across Microsoft to do the POC, make sure it works with their data, make sure they're comfortable and then work with their engineering team to at least get that in the plan. Those are ongoing discussion but M365D does have, kinda, I'll say, in their planning cycle, to get this in the product. Nic Fillingham:I wonder if this might be a good time to bring our secret special guest on microphone, Josh, if you're there, I think I might ask, uh, might wonder if you could jump in on this one. I think you've understated the power of what you've built here. From everything that you've just explained, you know, within a couple of minutes of a threat actor getting initial access to have a high probability index to be able to contact the customer and say, here's who we think is inside your network, here's what we think they're gonna do next, so they can shut it down. This is the next level, right? And, and Josh, when we interviewed you on episode three, you were hinting at this, if I'm not mistaken. Is this, is this sort of what you guys have been working on?Joshua Neil:Yeah, I'm so proud that we, that we took it from concept to realized value for the customers and, and at this point we've had that impact with your customers in stopping human operations. And, and so it's really exciting and, and it's, it's on the journey but, you know, if I extract an overall theme from this, it's consistent with that podcast that we had before because I was sort of complaining about AI. And I was sort of complaining about what we see in some of the, in some of the branding and marketing that, that folks do in, in cyber security. And I think this team and, and the work they've done exemplifies the right applications of data driven methods. Joshua Neil:There is no magical, artificial intelligence today. What there is is, and this is a, an experience that all of us on the data science team have had over the, over the past few years, and really for me about 20 years, is we can use data and some mathematics and some computing to begin to automate and accelerate what the humans are doing. And so, by sitting very closely with, and working very hard with the human experts like Justin, we're explicitly encoding their knowledge into models. So that's one thing is that the data science we're doing is to automate some of the stuff they're doing today. But the intention is not to solve the world, not to give our customers a license to solve security, we're, we're not gonna be able to do that. What we are able to do is uplift the sophistication of our customers operations. Joshua Neil:So, you know, what Justin sort of reflected on, uh, he's able to do a more interesting job, a more sophisticated job, because we're taking the data and his knowledge and encoding it and accelerating and automating some of the stuff that he's having to do manually now. And that's where the real nuts and bolts, you know, and the real rubber meets the road here, is that there's no magic gun that's gonna blow away all the adversaries with, with AI. What there is is hard work between data scientists and threat expertise to uplift their capabilities and accelerate their effectiveness in the face of the adversary. And that's what I would like to get across to the, to the listeners, is that by hard work and careful and close collaboration between data science and threat expertise, that's how we really make progress in this space.Nic Fillingham:Thank you so much Josh. And I just wanted to quickly clarify, from a previous comment from Cole, so this model is in use now, correct? Folks like Justin, Microsoft threat analysts, they are using this model now to make the model better, and to be able to get that additional information and those confidence levels in, in, in doing their analyst work. And so Microsoft threat expert customers are directly benefiting from this work, as of today. That's correct, is it?Joshua Neil:That's correct. We've sent targeted attack notifications to customers based on this model.Nic Fillingham:You've all been very, very, generous. Natalia Godyla:Thank you for that. And, and thank you to the whole team here for joining us on the show today. Melissa Turcotte:Absolutely.Cole Sodja:My pleasure.Joshua Neil:It was a lot of fun as always. And, and thank you, Nic and Natalia for this.Natalia Godyla:Well, we had a great time unlocking insights into security, from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us at MSFTSecurity or email us at securityunlocked@microsoft.com with topics you'd like to hear on future episode. Until then, stay safe...Natalia Godyla:Stay secure.
4/21/2021

Below the OS: UEFI Scanning in Defender

Ep. 24
All of us have seen– or at least, are familiar with – the antics of Tom and Jerry or Road Runner and Wile E. Coyote. In each one the coyote or the cat set up these elaborate plans to sabotage their foe, but time and time again, the nimble mouse and the speedy birdareable tooutsmart their attackers.In our thirdepisode discussing Ensuring Firmware Security,hosts Nic Fillingham and Natalia Godylaspeak withShweta JhaandGowtham Reddyabout developing thetoolsthat allow for them to stay one step ahead ofcybercriminals in the cat & mouse game that is cyber security.In this Episode You Will Learn:• Thenewcapabilities within MicrosoftDefenderto scan theUnified Extensible Firmware Interface (UEFI)• How theLoJaxattack compromised UEFI firmware • How UEFI scanning emerged as a capabilitySome Questions that We Ask:• Has UEFI scanning always been possible?• What types of signals is UEFI scanning searching for?• What are the ways bad actors may adjust to avoid UEFI scanning?Resources:Shweta Jha’sLinkedIn:https://www.linkedin.com/in/jhashweta/Gowtham Reddy’sLinkedIn:https://www.linkedin.com/in/gowtham-animi/Defender Blog Post:https://www.microsoft.com/security/blog/2020/06/17/uefi-scanner-brings-microsoft-defender-atp-protection-to-a-new-level/NicFillingham’sLinkedIn:https://www.linkedin.com/in/nicfill/NataliaGodyla’sLinkedIn:https://www.linkedin.com/in/nataliagodyla/Transcript[Full transcript can be found at https://aka.ms/SecurityUnlockedEp24]Nic Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft's security, engineering, and operations teams. I'm Nic Fillingham-Natalia Godyla:And I'm Natalia Godyla. In each episode we'll discuss the latest stories from Microsoft Security, deep dive into the newest threat intel, research and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security.Natalia Godyla:And now, let's unlock the pod.Natalia Godyla:Hello Nic. Welcome to Episode 24. How's it going with you today?Nic Fillingham:Going well, thank you, Natalia. Yes, uh, welcome to you and welcome to listeners to Episode 24 of Security Unlocked. On today's podcast, we speak with Shweta Jha and Gowtham Reddy from the Microsoft Defender for Endpoint engineering team about capabilities in MDE to scan down into the UEFI layer. Now this is the third of three conversations we have that started back in Episode 11 with Nazmus Sakib where we talked about secure core PCs and, and firmware integrity. Then in Episode 14 we spoke with Peter Waxman about the Pluton processor and some of the new work that's happening there to imbed more security tech into sorta silicon onto the actual CPU die itself. And today we're sort of rounding that conversation out with Shweta and Gowtham to talk about how Microsoft Defender for Endpoint can now scan down or can scan down into the UEFI layer. You're gonna hear a bunch jargon, a bunch of technical terms like, I guess, UEFI. That's, we, we could start there.Natalia Godyla:Yes. And UEFI is the Unified Extensible Firmware Interface, so it is the software interface that lies between an operating system and firmware, and is an evolution of BIOS. And we'll also talk about MosaicRegressor which, for those of you that don't know, is the second ever UEFI rootkit which was discovered in 2020, but was used in an attack against NGOs in 2019. And, and for me, the timeline is shocking, second ever in the past year. Normally we hear about the continuous increase of a certain type of attack over the years, and here we're just at the second ever.Nic Fillingham:Yeah. It's a real interesting part of the conversation where we talk about the history of BIOS attacks, firmware attacks, UEFI attacks, and to learn that this has been sort of traditionally a pretty challenging area for attackers to, to breech and compromise. But, you know, Shweta and Gowtham have been, you know, very much ahead of the curve and, and being ahead of, of attackers in, in being able to develop these new capabilities to, from the operating system, scan down to the UEFI layer and look for malware, look for compromise. And it's a, it's a fascinating conversation. Again, it's sort of a completion of three episodes starting with Episode 11 and 14. So if you haven't listened to those, I recommend you add them to the queue. But I guess on with the pod.Natalia Godyla:On with the pod.Nic Fillingham:Welcome to the Security Unlocked podcast. Shweta Jha and Gowtham Reddy, welcome both of you. Thanks for being here.Gowtham Reddy:Thank you.Shweta Jha:Thank you so much for having us. We're so very excited.Nic Fillingham:I'm very excited, too. Now this is gonna be the third conversation in a sort of a mini series that we're running here on the podcast. We started with Nazmus Sakib who introduced us to the idea of secure core PCs and we talked about some of the challenges of firmware integrity and keeping firmware safe. Then we spoke Peter Waxman in another episode to learn about Pluton, the history of, of that technology and sort of what's coming for the Pluton processor. And today we're actually gonna talk about some new capabilities, or newish as of 2020, in Defender to scan down into the UEFI layer. Before we jump into to all that, let's just do some introductions for the audience. Shweta if we could start with you. Who are you? What is your role? What do you do day-to-day at Microsoft? Tell us, what you like the audience to know about you?Shweta Jha:Absolutely. Thank you, Nic. My name Shweta Jha. I am a program manager with Microsoft Defender for Endpoints, and I've been building security solutions features and products, and I'm super excited about it because security is the need today for our, uh, customers. And a few of the features that I built with my team were part of anti-tampering. Investment that we did, EDR block as part of be able to blocking and containment. And then we are gonna talk a lot about UEFI scanner. So pretty much around building security solution and features in this team and helping our customers.Nic Fillingham:Fantastic. And, and Gowtham, welcome to the podcast. If you could also introduce yourself. Uh, tell us about your role. What does your day-to-day look like?Gowtham Reddy:Hi. This is Gowtham Reddy. I'm an engineering manager in Microsoft Defender, uh, Endpoint. So before engineering manager, so I was working as an engineer in the same team for last six years. So I work on, uh, many of the rootkit technologies, the Defender, uh, has and, uh, the remediation technologies to remediate many of the malwares that are present on the system. I have been where I working on this fantastic team, developing like durable protection features that were, and compliment the ever changing malware fields.Nic Fillingham:That's great. So, again, welcome to both of you. Thanks for your time. One of the things we do on the, uh, Security Unlocked podcast here is we, we don't necessarily cover the latest announcements. We, we sort of look back over the last sort of three to six months for interesting sort of technology, interesting advancements in, in security technology, and we bring experts on to, to talk about these new features and capabilities after them sort of being in the wild. Today we're talking about the UEFI scanning capabilities that are in Microsoft Defender, and there's a blog post that, that both of you helped author back in, in June of 2020, which feels like a decade ago, but I guess it's more like six or seven months. So I wondered if one of you might be able to just walk us through. What was that announcement made in that blog post? What was sort of the news? And then I think maybe if the other one or maybe just following, I'll, I'll leave it to how we, how we split this up. But what was announced back in June? And sort of what's happened since then? How have those new capabilities sort of rolled out and what are we seeing with customers actually using them?Shweta Jha:So I, I guess I can get us started, and then I'll hand it over to Gowtham definitely to talk more on the technical details and the, the attacks that we see in the wild, and that's why we kind of built this UEFI scanner. So as you understand, this is a journey, right? We built a layered defense in that security solutions. And when we build any security solution, we need to make sure that we take a holistic approach. So if you look at the operating level of security solutions, we've been getting pretty great at operating level security solutions. And it's not only Microsoft. If you see other security providers as well, they have been doing great, too.Shweta Jha:So what does that mean? It means that because the operating system level security solution is really great, it does making difficult for attackers to not get detected at that level. It's a constant battle, so they have been looking into other means where they can go into the system undetected, and that's where if you look at the data, you would find that in recent past the attacks across hardware and firmware level has been on the rise. So we built UEFI scanner keeping in mind that we should be able to detect those type of attacks, because those type of attacks are not only very dangerous, but often time they are not detected. They persist even if you reboot the system. So the nature of these type of attacks is very dangerous, and keeping that in mind, we decided to build UEFI scanner.Gowtham Reddy:So I can add like why we did, uh, build the UEFI scanner. So because of the operating system security features that Microsoft is constantly working on, the bad guys are trying to go in, down and down in the layered architecture. And so some of the traits of the ia64 went onto the BIOS, tampering the BIOS and, uh, tampering the MBR, the master board required and, uh, VBR based bootkits. So Defender has evolved into that space of counting the MBR and, uh, detecting the bootkits and the boot time. Gowtham Reddy:So as a logical evolution the bad guys are, uh, from the stage of Colonel to the MBR, MBR to the UEFI. So we were anticipating that this kind of evolution is quite possible and the UEFI implants were not very far. So that's the time we found the first UEFI implant called LoJax. So that was a triggering point when we completely committed to ourselves to expand our root kit technology, to detect any kind of rotates presence in the UV. So that was our core idea of expanding or rotating to the layer much below the operating system. So there were some challenges Natalia Godyla:If you don't mind me jumping in, I had a question around that. So...Gowtham Reddy:Mm-hmm (affirmative)Natalia Godyla:... the way you're framing it is that when we started to notice the threat landscape moved to this layer, we decided to invest in this type of technology. What about the technology itself? Had there always been this opportunity to tackle UEFI scanning, or is there something new that we're leveraging in order to solve this problem Now that might not have been around beforehand? Gowtham Reddy:That's a good question. So there was always a chance to exploit the UEFI, but it's about the timing of the attackers to get at target this space because the rest of the platform and ecosystem is getting more and more secure. So the UEFI is not new. So it was there a decade ago, but the implants are new because of the advances in the operating system. Nic Fillingham:So Gowtham, tell us about the LoJax attack that happened. Was it the first or it was one of the first detected compromises of the UEFI firmware? Can you tell us some more about, about that? If folks aren't familiar with it like me? Gowtham Reddy:Mm-hmm (affirmative).So that definitely some theoretical researcher driven, before the LoJax, but the LoJax is a fast known exploitation instance where we know we found it in the wild. It is quite possible even before that a UEFI implant demonstrated in many of the black hat conferences, but those are theoretical in nature. So the research had access to the device and they demonstrated it. But LoJax's is the one where from operating system level. So a particular malware, I would say it as a root kit, which has tried to intrude from kernel mode to the UEFI, and they have installed a UEFI driver. So if we consider the operating system as a drivers, even the firmware itself had some drivers. So they were able to install a driver which actually in turn drops the another kernel mode driver, advanced operating system boots up. it's about the boot sequence. Gowtham Reddy:So first the firmware starts running and it initializes all the system, and then it invokes operating system. So in the LoJax's case, after the firmware is completed, it has already dropped the kernel driver on the operating system, if it is not present. So that means by the end of the firmware sequence, so we have a presence of a kernel driver. And when that kernel driver starts, that is a user mode, malware starts, kicks in. So this keeps repeating even after you were re-install the wares, even if you change the hard disc, the same pattern will be fought. So that's how the LoJax's type work. Nic Fillingham:And I wonder, do we know, what was the breakthrough that made LoJax possible? UEFI has been around for a while. UEFI for probably predates LoJax. And obviously before UEFI, there was sort of the more standard sort of BIOS that probably most folks are familiar with. Can we talk a little bit more about how LoJax came about and sort of what maybe changed or what the breakthrough was on the attacker side? Gowtham Reddy:I would say that there were a couple of open source read-write drivers, which has a capability to access the firmware, using a special interface called SPI. SPI is a something called serial peripheral interest. So using the serial peripheral interface, any kernel driver can instruct the platform hardware layer to read and write any content in the flash. So I think like many of the security industry knows a driver called a read drive, everything, they call it as RWE. So this is the driver using which anybody can read any offset, any device memory, and write. I think this is, the prevalence of this kind of open source tools might be help attackers to develop this kind of ecosystem of all the sequence of the malware, the root kits. Shweta Jha:In addition to what Gowtham said, definitely the work that researchers were doing in this space, it always starts with researcher trying to do something and then attackers trying to find other means. So here are the things. Attackers usually do exploit things that are not done in a right way. So in this case, for example, if there are certain configuration that you need to, or your partner needs to make sure that those are in place, for example, rewrite where you are not providing writing access, just the reading access, and so on. Shweta Jha:So typically in all these type of attacks would see that misconfigured devices are exploited the most, and that misconfiguration happens at the time when the devices are getting built. So that is another factor why these attacks are very successful, because there are misconfigured devices, because while building the devices, somebody messed to configure it and right way. And if you look at the journey, that's where you have a secure core PC, which is designed be secured, making sure that the things that are needed to protect the computer against these types of attacks that are there out from the first day. Natalia Godyla:So my question is about the application of this new technology. So I really appreciate you walking through that attacker workflow. So what type of signals is UEFI scanning, looking for? What is it using to enrich the context of the existing end point data?Gowtham Reddy:That's a very good question. So basically the level of details that UEFI scanner can get is enormous. So this is the area where like the defender has a content scanning. So, uh, we have, uh, extended our content scanning to every file that is present inside the firmware. So this help the defender research to write any kind of content scanning signatures to detect any bad content. So that means in this case, if research knows any implant, so we have a capability to scan the 600 million devices to know if any of our customers have impacted with the specified malicious file. Gowtham Reddy:And this is just one part of our UEFI scanner. And the other part of it is detecting any anomalous behavior inside the firmware. For example, in many of the supply chain attacks like Solarigate, it's quite possible that some of the OEMs channels were compromised and the deliver the firmware updates with the malicious modules in it. Gowtham Reddy:So in this case, our UEFI scanner collects all the metadata about the new for- firmware update and we run heavy amount models in our cloud. And that will tell us if there is an unknown anomaly that exists in this particular firmware update. Instead of a known malware implant so that the UEFI scanner has the two capabilities. One is detecting a known malicious implant, and the other one is anomalous from where presence of a fax. So in this case, we act both ways. Nic Fillingham:What does an anomaly look like in this context? Gowtham Reddy:Anomalies look like, for example, if you have a firmware is a, firmware is a file system, like a typical drive. A presence of an driver file, probably a hedge P driver file or an unsigned driver file. On a Dell OEM is constrained to the anomaly. Because we have trained the model of all the known Dell firmwares with them, a ML model. So any new image with the unexpected file, it will be immediately flagged. Nic Fillingham:And why is ML the sort of approach you've taken here versus sort of heuristics? I would have thought that there's a pretty limited set of content. They could make up sort of firmware and firmware instructions. Obviously, I don't know anything about this space, so I'll caveat that there, but, um, could you talk about why ML versus heuristics versus something else?Gowtham Reddy:In the days of, uh, BIOS, so you are a expectation was right. The bast consists of a series of micro code, Gowtham Reddy:... which is, uh, very limited. And, uh, in the context of UEFI, you have a full file system, uh, which has, like, uh, thousands of files; individual files. And, uh, this causes... Uh, creates, uh, basically a huge amount of, uh, the vectors space, which to scan or to collect the metadata. Gowtham Reddy:So it's not just simple collection of mecra- microcodes. It contains the drivers, it contains the services, it contains a lot of other things. It's a file system like NTFS.Nic Fillingham:Got it. So because UEFI is, as you say, a file system as opposed to... What was BIOS? BIOS was not a file system? BIOS was, uh, sort of a discreet, sort of, low level executable?Gowtham Reddy:Yeah, i- i- it is just a sequence of, uh, microcode instructions that will be run on the firmware. So basically, i- it has a s- uh, fi- se- set of microcodes. Nic Fillingham:So the machine learning models that you reference, w- where are they running? Are some of them running locally? Are they all running in the Cloud? Is it a mixture of the two?Gowtham Reddy:They're all running in the Cloud for now. So we have MDATP Cloud services where we run all this clo- uh, demo models. So our models are really very effective. So recently, we got in, uh, so- so, uh, the UEFI alert by, uh, mal- model. Apparently, it's a kind of, um, true positive because, um, there was a Microsoft engineer who was working on a hardware space.Gowtham Reddy:So he take, uh, firmware. And he kept a developer driver and he flashed on his own device. And, uh, our UEFI scanner immediately caught it and we... the security administrator got an alert and there was an investigation happen. So we are pretty ready to catch any kind of such things now.Natalia Godyla:So we all know it's a cat and mouse game with the threat actors. So what is the team anticipating in terms of how the actors will adjust their processes to evade this new UEFI scanning technology?Gowtham Reddy:That's a good question. We're trying to validate something in a- a lower level of trust, the lower level of ring other than the kernel. So definitely, there is a chance that attacker can modify the firmware presence. Uh, he can spoof the content when defender tries to scan. So this is quite, uh, possible. But we are already working on mitigating that kind of an attacks. Nic Fillingham:So now that this feature, these capabilities, have been live in the product for, uh, I guess over six months at this point, w- what have you learnt? What have you seen in the telemetry? What have you seen in the types of attacks and, I guess, even sort of false positives that have- have come through from- from this new, uh, capability?Gowtham Reddy:Uh, that's a very good question. So we learnt a lot of things. The UEFI file system has never scanned before. So we got some false positives on the content that we scan but we immediately fine-tuned our signatures.Gowtham Reddy:Back in... Six months before, when we published a blog, we only know the first UEFI known implant called LoJax but often we share... There was a second implant called Public. That's called MosaicRegressor and our UEFI scanner has well detected the MosaicRegressor implant. Uh, the- the telemetry count was small. So we did, uh, able to detect the mi- MosaicRegressor.Nic Fillingham:So in this first six months, as well as the LoJax campaign, uh, what's the taxonomy here? How do we f- refer to it?Gowtham Reddy:Uh, we can consider... W- we are, uh, tracking them as an UEFI implant malware or UEFI rootkit. So this is the category we are looking at. So right now, we have, uh, LoJax and we have a MosaicRegressor as, uh, two big families in this space.Nic Fillingham:Big families. Got it. Shweta Jha:Yeah, about MosaicRegressor, I wanted to add a little bit more just to complement what, uh, Gowtham mentioned, how powerful this tool is. And how powerful this particular feature is. So if you read through the MosaicRegressor, uh, breach, it was a nationwide targeted attack.Shweta Jha:This was targeted for diplomats. And this attack, as Gowtham described, first they would insert one module. Uh, that one module would get undetected and then that module would try to do other stuff, like try to, uh, get in touch with command and control and get another, uh, module and so on.Shweta Jha:So the entire c- chain is so very interesting. And I'm glad that we built this feature and we were able to detect it because it's so powerful. Most of the security solution, they're not able to detect because they don't have this, uh, such great capabilities.Shweta Jha:But look at the way this attack was carried. It was pretty much targeted, pretty much nationwide for a few countries, originated from one country. So the sophistication level in the nature itself speaks for it and I'm glad that we, as in our product, we have this capability which can even, you know, unknown, first seen, it can detect those type of attacks as well.Natalia Godyla:In the process of developing this new technology, where were there false starts? What techniques did you try but didn't work to solve this problem?Shweta Jha:Little bit on the journey, right? We have been working on it. Um, so Gowtham explained about how we have rootkit, bootkit level and then we went to the UEFI site and we had to be extremely careful because it's, like, uh, it has a high integrity and high severity of going wrong.Shweta Jha:So we had to be very careful making sure that the running system is not damaged and at this point, I'll hand it over to Gowtham because he can explain, in detail, each and every pieces that we took into consideration to making sure that our customers' device remain intact. So go ahead Gowtham.Gowtham Reddy:Yeah. Thanks Shweta. So, uh, we have indeed explored, uh, many mechanisms like accessing the PCI space from the operating system itself, which we didn't continue to proceed because of some of the pushback from the kernel team to update the haul.Gowtham Reddy:So actually, uh, to accessing any peripheral device from the PCI bus, there are a couple of complications because the peripherals have, uh, specific implementation of Reads and Writes, the bus Reads and Writes. So, uh, the approach we took was, uh, using the SBI interface, which is pretty much, kind of, an, uh, universal interface which is developed by Motorola by a long time ago.Gowtham Reddy:So luckily, what worked in our favor was most of the Intel p- s- uh, chipsets, they support the SBI based access. So they support the SBI, uh, using which we can use the memory map mechanisms to access the PCI space.Gowtham Reddy:So basically, here, what happened was instead of directly using the hardware primitives, we used, uh, software primitives because the chipsets are well supporting the SBI interface. So that's how we landed in our approach. Nic Fillingham:I wanted to circle back to the use of machine learning here in- in solving this problem. How big are the signal sets that you're getting to train the model? How big is the model?Nic Fillingham:Is the model that you use here, to detect anomalies in the firmware layer, is it as sophisticated and large as something as, like, looking for malware on endpoints? Or are we talking, like, a much sort of smaller more, sort of, n- nuance. No, that's not the right word. Sort of a smaller bespoke model?Gowtham Reddy:Uh, I can take that question. So u- usually, uh, in the endpoint when- when applying the malware, um, in machine learning models, we heavily focus on the individual file properties, like file headers, file footers and some file p- properties and so on. Gowtham Reddy:But UEFI case, we built a brand new machine learning model based on the properties of the UEFI image itself. So thanks to David, from our MDATP team. So he come up with a model where... which takes input signals as specific to the UEFI firmware image.Gowtham Reddy:To give some examples, each firmware drive has a lot of GUIDs, called firmware GUIDs. And then they have some properties called, uh, file types and properties. Every property that we took was specific to the firmware. So they are not generic to the specific malware files that we see regular malware detections. So these are highly tailored to the signals from the UEFI firmware image. Nic Fillingham:And were you able to reuse some of the anomaly detection Nic Fillingham:Algorithms are purchased from other parts of the defender engineering org, or did you have to sort of build a brand new model and a brand new way to detect anomalies? Shweta Jha:Yeah. So, we definitely used our existing infrastructure. So, as you know? Uh, we have a massive backend system where we get tons of signals and we run tons and tons of AI and ML model to detect the anomalies and to detect the new trends and so on. So, as Gordon was talking, for this particular UV, AI and ML model, even though where we had to tweak it to make sure that we capture the inputs that are UV specific, the models were used, the pipeline to collect the data that were used and the channel where we surface it to our customers. So, if you look at the end to end story, the way we do things are we detect, we remediate, and we also notify to our SecOps that, "Hey, these are the things that happened in your environment." And that goes in the form of alerts or incidents and so on. So, we used exactly same infrastructure, same pipeline, but specific to UV. Natalia Godyla:So, I know a little earlier in this episode, we talked about the learnings after being in market. What about the impact to SecOps teams? Do we have any early numbers to talk through about what this has raised for our customers? Shweta Jha:That's a great question. We do see here and there, though the number is not pretty high on the implant, but we do see in numbers there, like, as Gordon mentioned about a mosaic regression. We did find that and there are few others also. But I think the most important aspect of this unique feature is that, just a little bit forget about this feature and see that today's world, today, there is no UV scanner, the security admins or SecOps, they, they don't know what is happening at this level. They have tons of device in their organization. And these devices are at this level is completely black box for them, because they don't know whether it is configured well. They don't know if there are implants there. They don't know if there are vulnerabilities that could be exploited. Shweta Jha:So, there's the power of this UV scanner. One is, you know, so we, we built a solution keeping in mind that we will not only detect, we will bring these, these things where they don't have visibility today to understand what is going on. So, the focus area, and then the objective that we have is to detect the implant, either using the heuristic detection or the AI, ML but also read through each and every configuration that are happening at this level and the vulnerabilities that exist at this level and bring that to the, SecOps attention, so that when they look at it, they can take appropriate action to remediate it. So, that's the next step. And that is the work right now, we are currently doing. We do not have, in the form of report, we do see it in our data and we want to make sure that these are available to our SecOps. But just to tell you, there are tons and tons of misconfigured device out there. And it's, it's a little tricky.Gowtham Reddy:To add more about the misconfiguration. So, it's about like the PC settings, like a UV, the BIOS read-write or whatever the settings we'll use to see in when we go to the BIOS in the past. So, the UV must be configured well to support the secure boot, to use the TPM and to use any of the hardware provided features, it must be configured well. If it is misconfigured, you won't get any protection. So, if you have a helmet in your backseat, when you are driving, it won't help you. So, you had to keep it on your head. Shweta Jha:(laughs). That's a great analogy. Nic Fillingham:That leads us to, what is the guidance here for Sec admins and security teams out there? How do they enable this functionality? Is it on by default in, in certain places? What do we need to do to make sure that, that customers are getting the full protection from this capability? Shweta Jha:So, uh, this, this feature is enabled by default on all the devices. Um, we made sure that this is available. And the great news is that it is not only, you know, Windows 10, it is available for servers, download as well. So, that's the power that we have in our solution. Ultimately, if you look at what is the future that are gonna look like, secure core PC is the future we should be heading towards. But because enterprises and customers are not there yet, uh, we have UV scanner to compliment it. The other thing, if we have to talk about the futuristic roadmap, right now, we built the scanner for UV, but there are other network devices like network adapter and things like that. There is a scope to extend these types of capability to those devices as well, because those, there is a possibility to get those devices exploited too. So, that's something we are considering to work through. Nic Fillingham:Got it. So, just to confirm there, so, this new capability is on by default in any device that is being protected by the defender service. Is, is it, is it as simple as that or is there sort of more to it?Shweta Jha:Yes. Any device which is having defender antivirus running.Natalia Godyla:Thank you for that. That was super helpful. And thank you both for joining us on the show today. Shweta Jha:Thank you, Natalia. It was pleasure to be here and talking with our customers. Thank you so much for hosting us. Gowtham Reddy:Thank you Natalia and Nick for hosting us. So, it's been wonderful time talking to you about UV scanner. Thank you so much. Nic Fillingham:Thank you both for your time. Thanks for bringing great innovation to the security space. Shweta Jha:Absolutely. It's a constant journey and we're on it. Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode. Nic Fillingham:And don't forget to tweet us @msftsecurity or email us @securityunlocked@microsoft.com with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure.