Security Unlocked

Share

Ready or Not, Here A.I. Come!

Ep. 26

Remember the good ole days when we spent youthful hours playing hide and seek with our friends in the park? Well it turns out that game of hide and seek isn’t just for humans anymore. Researchers have begun putting A.I. to the test by having it play this favorite childhood game over and over and having the software optimize its strategies through automated reinforcement training.  

In today’s episode, hosts Nic Fillingham and Natalia Godyla speak with Christian Seifert and Joshua Neil about their blog post Gamifying machine learning for stronger security and AI modelsand how Microsoft is releasing this new open-sourced code to help it learn and grow.  


In This Episode, You Will Learn:

  • What is Microsoft’s CyberBattleSim? 
  • What reinforcement learning is and how it is used in training A.I. 
  • How the OpenAI Gym allowed for AI to be trained and rewarded for learning  

Some Questions We Ask:

  • Is an A.I. threat actor science fiction or an incoming reality? 
  • What are the next steps in training the A.I.? 
  • Who was the CyberBattleSim created for? 


Resources:

OpenAI Hide and Seek: 

OpenAI Plays Hide and Seek…and Breaks The Game! 🤖 

Joshua and Christian’s blog post: 

Gamifying Machine Learning for Stronger Security and AI Models 

Christian Seifert’s LinkedIn

https://www.linkedin.com/in/christian-seifert-phd-6080b51/ 

Joshua Neil’s LinkedIn

https://www.linkedin.com/in/josh-neil/ 

Nic Fillingham’s LinkedIn

https://www.linkedin.com/in/nicfill/ 

Natalia Godyla’s LinkedIn

https://www.linkedin.com/in/nataliagodyla/ 

Microsoft Security Blog:

https://www.microsoft.com/security/blog/


Related:

Security Unlocked: CISO Series with Bret Arsenault

https://SecurityUnlockedCISOSeries.com


Transcript

[Full transcript at https://aka.ms/securityunlockedep26]


Nic Filingham:

Hello and welcome to Security Unlocked! A new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft Security Engineering and Operations Teams. I'm Nic Filingham.


Natalia Godyla:

And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft Security. Deep dive into the newest threat intel, research, and data science.


Nic Filingham:

And profile some of the fascinating people working on artificial intelligence in Microsoft Security.


Natalia Godyla:

And now, let's unlock the pod.


Nic Filingham:

Hello, Natalia! Hello, listeners! Welcome to episode 26 of Security Unlocked. Natalia, how are you?


Natalia Godyla:

Thank you, Nic. And welcome to all our listeners for another episode of Security Unlocked. Today, we are chatting about gamifying machine learning, super cool, and we are joined by Christian Seifert and Joshua Neil who will share their research on building CyberBattleSim, which investigates how autonomous agents operate in a simulated enterprise environment by using high-level obstruction of computer networks and cyber-security concepts. I sounded very legit, but I did just read that directly from the blog.


Nic Filingham:

I was very impressed.


Natalia Godyla:

(laughs)


Nic Filingham:

If you had not said that you read that from the blog, I would've been like, "Wow". I would to like to subscribe to a newsletter.


Natalia Godyla:

(laughs)


Nic Filingham:

But this is a great conversation with, with Christian and Joshua. We talked about what is reinforcement learning. Sort of as a concept and how does that gonna apply to security. Josh and Christian also walked us through sort of why this project was created and it's really to try and get ahead of a future where, you know, malicious actors have access to some level of automated, autonomous tooling. Uh, and so, this is a new project to sort of see what a future might look like when there all these autonomous agents out there doing bad stuff in the cyber world.


Natalia Godyla:

And there are predecessors to this work, at least in other domains. So, they used a toolkit, a Python-based Open AI Gym interface to build this research project but there have been other applications in the past. OpenAI is, uh, well-known for a hide-and-seek. There is a video on YouTube that shows how the AI learned over time different ways to obstruct the agent and the simulated environment. Things like, blocking them off using some pieces of the wall or jumping over the wall.


Nic Filingham:

The only thing we should point out is that this CyberBattleSim is an open source project. It's up on GitHub and attained very much want researchers, and really anyone who's interested in this space to go and download it, go and run it, play around with it, and help make it better. And if you have feedback, let us know. There is contact information, uh, through the GitHub page but you can also contact us at Security Unlocked at Microsoft dot com and we can make sure you, uh, get in contact with the team. And with that, on with the pod?


Natalia Godyla:

On with the pod!


Nic Filingham:

Welcome to Security Unlocked, new guest, Christian Seifert. Thanks for joining us and welcome returning guest, Josh Neil, back to the podcast. Both of you, welcome. Thanks for being on Security Unlocked.


Christian Seifert:

Thanks for having us!


Joshua Neil:

And thanks, Nic.


Nic Filingham:

Christian, I think as a, as a new guest on the podcast, could we get a little introduction for our listeners? Tell us about, uh, what you do at Microsoft. Tell us about what a day to day look like for you.


Christian Seifert:

Sure, so I'm a, uh, research lead on the Security and Compliance team. So our overall research team supports a broad range of enterprising consumer products and services in the security space. My team in particular is focused on protecting users from a social engineering attack. So, uh, think of, like, fishing mails for instance. So we're supporting Microsoft Defender for Office and, um, Microsoft Edge browser.


Nic Filingham:

Got it, and Josh, folks are obviously familiar with you from previous episodes but a, a quick re-intro would be great.


Joshua Neil:

Thanks. I currently lead the Data Science team supporting Microsoft threat experts, which is our managed hunting service, as well as helping general res... cyber security research for the team.


Nic Filingham:

Fantastic, uh, again, thank you both for your time. So, today in the podcast, we're gonna talk about a blog post that came out earlier in this month, on April 8, called Gamifying Machine Learning for Stronger Security in AI Models, where you talk about a new project that has sort of just gone live called CyberBattleSim. First off, congratulations on maybe the coolest name? For, uh, sort of a security research project? So, like, I think, you know, just hats off there. I don't who came up with the name but, but great job on that. Second of all, you know, Christian if, if I could start with you, could you give us a sort of an introduction or an overview what is CyberBattleSim and what is discussed in this blog post?


Christian Seifert:

As I... before talking about the, the simulator, uh, the... let me, let me kind of take a step back and first talk about what we tried to accomplish here and, and why. So, if you think about the security space and, and machine learning in particular, a large portion of machine-learning systems utilized supervised, uh, classifiers. And here, essentially, what we have is, is kinda a labeled data set. So, uh, for example, a set of mails that we label as fish and good. And then, we extract, uh, threat-relevant features. Think of, like, maybe particular words in the body, or header values we believe that are well-suited to differentiate bad mails from good mails. And then our classifiers able to generalize and able to classify new mails that come in.


Christian Seifert:

There's a few, uh, aspects to consider here. So, first of all, the classifier generalizes based on the data that we present to it. So, it's not able to identify completely unknown mails.


Christian Seifert:

Second, is that usually a supervised classification approach is, is biased because we are programming, essentially, the, the classifier and what it, uh, should do. And we're utilizing domain expertise, red teaming to kind of figure out what our threat-relevant features, and so there's bias in that.


Christian Seifert:

And third, a classifier of who has needs to have the data in order to make an appropriate classification. So, if I have classifier that classifies fish mail based on the, the content of the mail but there is the threat-relevant features are in the header, then that classifier needs to have those values as well in order to make that classification. And so, my point is these classifiers are not well-suited to uncover the unknown unknowns. Anything that it has not seen, kinda new type of attack, it is really blind to it. It generalizes over data that, that we present to it.


Christian Seifert:

And so, what we try to do is to build a system that is able to uncover unknown attacks with the ultimate goal then to, of course, develop autonomous defensive component to defend against those attacks. So, that gives it a little bit of context on why we're pursuing this effort. And this was inspired by reinforcement learning research and the broader research community, mostly that is currently applied kinda in the gaming context.


Christian Seifert:

So OpenAI actually came out with a neat video a couple of years ago called Hide and Seek. Uh, that video is available on YouTube. I certainly encourage listeners to check it out, but basically it was a game of laser tag where you had a kinda, uh, a red team and a blue team, uh, play the game of laser tag against each other. And at first they, of course, randomly kind of shoot in the air and run around and there is really no order to the chaos. But eventually, that system learned that, “Hey, if a red team member shoots a blue team member, there's a reward.” and the blue team member also learned while running away from the red team member is, is probably a good thing to do.


Christian Seifert:

And so, OpenAI kinda, uh, established the system and had the blue team and the red team play against each other, and eventually what that led to is really neat strategies that you and I probably wouldn't have come up with. 'Cause what the AI system does, it explores the entire possible actions base and as result comes up with some unexpected strategies. So for instance, uh, there was a blue team member that kinda hid in a room and then a red team guy figured, “Hey, if I jump on a block then I can surf in that environment and get into the room and shoot the blue team member”. So that was a little bit an inspiration because we wanted to also uncover these unknown


Christian Seifert:

Unknownst in the security context.


Nic Filingham:

Got it. That's great context. Thank you Christian. I think I have seen that video, is that the one where one of the many unexpected outcomes was the, like, one of the, the, blue or red team players, like, managed to sort of, like, pick up walls and used them as shields and then create ramps to get into, like, hidden parts of the map? Uh, uh, am I thinking about the right video?


Christian Seifert:

Yes, that's the right video.


Nic Filingham:

Got it. So the whole idea was that that was an experiment in, in understanding how finding the unknown unknowns, using this game, sort of, this lazar tag, sort of, gaming space. Is, is that accurate?


Christian Seifert:

That's right, and so, they utilized reinforcement learning in order to train those agent. Another example is, uh, DeepMind's AlphaGo Zero, playing the game of Go, and, and here, again, kind of, two players, two AI systems that play against each other, and, over time, really develop new strategies on how to play the game of Go that, you know, humans players have, have not come up with.


Christian Seifert:

And it, eventually, lead to a system that achieved superhuman performance and able to beat the champion, Lisa Dole, and I think that was back in 2017. So, really inspiring work, both by OpenAI and DeepMind.


Nic Filingham:

Got it. I wonder, Josh, is there anything you'd like to- before we, sort of, jump into the content of the blog and, and CyberBattleSim, is there anything you'd like to add from your perspective to, to the context that Christian set us up on?


Joshua Neil:

Yeah. Thanks, Nic. I, I mean, I think we were really excited about this because... I think we all think this is a natural evolution of, of our adversaries, so, so, currently, our adversaries, the more sophisticated ones, are primarily using humans to attack our enterprises and, that means they're slow and they can make mistakes and they don't learn from the large amount of data that's there in terms of how to do attacks better, because they're humans.


Joshua Neil:

But I think it's natural, and we just see this, uh, everywhere and, all of technology is that people are bringing in, you know, methods to learn from the data and make decisions automatically, and it's- so it's a natural evolution to say that attackers will be writing code to create autonomous attack capabilities that learn while they're in the enterprise, that piece of software that's launched against the enterprise as an attack, will observe its environment and make decisions on the fly, automatically, from code.


Joshua Neil:

As a result, that's a frightening proposition because, I think the speed at which these attacks will proceed will be a lot, you know, a lot more quick, but also, being able to use the data to learn effective techniques that get around defenses, you know, we just see data science and machine learning and artificial intelligence doing this all over the place and it's very effective that the ability to consume a large amount of data and make decisions on it, that's what machine learning is all about. And so, we at Microsoft are interested in exploring this ourselves because we feel like the threat is coming and, well, let's get ahead of it, right? Let's go experiment with automated learning methods for attacks and, and obviously, in the end, for defense that, by implementing attack methods that learn, we then can implement defensive methods that will, that will preempt what the real adversaries are doing, eventually, against our customers.


Joshua Neil:

So, I think that's, sort of, a philosophical thing. And then, uh, I love the OpenAI Hide-and-Seek example because, you know, the analogy is; Imagine that instead of, they're in a room with, um, walls and, and stuff, they're on a computer network, and the computer network has machines, it has applications, it has email accounts, it has users, it's got a cloud applications, but, in the end, you know, an attacker is moving through an environment, getting blocked in various ways by defenses, learning about those blockings and detections and things and finding gaps that they can move through in, in very similar ways. So, I just, sort of, drawing that analogy back, Hide-and-Seek, it is what we're trying to do in cyber defense, you know, is, is Hide-and-Seek. And so the, I think the analogy is very strong.


Nic Filingham:

Josh, I just wanna quickly clarify on something that, that you said there. So, it sounds like what you're saying is that, while, sort of, automated AI-based attacking, attackers or attacking agents maybe aren't quite prevalent yet, they're, they're coming, and so, a big part of this work is about prepping for that and getting ahead of it. Is, is, is that correct?


Joshua Neil:

That's correct. I, I'm not aware of sophisticated attack machinery that's being launched against our enter- our customers yet. I haven't seen it, maybe others have. I think it's a natural thing, it's coming, and we better be ready.


Christian Seifert:

I mean, we , we see some of it already, uh, in terms of adversarial machine learning, where, uh, our machine learning systems are getting attacked, where, maybe the input is manipulated in a way that leads to a misclassification. Most of that is, is currently more, being explored in the research community.


Natalia Godyla:

How did you apply reinforcement learning? How did you build BattleSim? In the blog you described mapping, some of the core concepts of reinforcement learning to CyberBattleSIm, such as the environment, that action space, the observation space and the reward. Can you talk us through how you translated that to security?


Christian Seifert:

Yeah. So, so first let, let me talk about reinforcement learning to make sure, uh, listeners understand, kinda, how that works. So, as I mentioned, uh, earlier in the supervised case, we feed a label data set to a learner, uh, and then it able to generalize, and we reinforcement learning works very differently where, you have an agent that sits within an environment, and the agent is, essentially, able to generate the data itself by exploring that environment.


Christian Seifert:

So, think of an agent in a computer network, that agent could, first of all, scan the network to, maybe, uncover notes and then they're, maybe, uh, actions around interacting with the notes that it uncovers. And based on those interactions, the agent will, uh, receive a reward. That reward actually may be delayed by, like, there could be many, many steps that the agent has to take before the reward, uh, manifests itself. And so, that's, kinda, how the agent learns, it's, e- able to interact in that environment and then able to receive a reward. And so that's, kinda, what, uh, stands, uh, within the core of the, the CyberBAttleSim, because William Bloom, who is the, the brains behind the simulation, has created an environment that is compatible with, uh, common, uh, reinforcement learning tool sets, namely, the OpenAI Gym, that allows you to train agents in that environment.


Christian Seifert:

And so, the CyberBattleSim represents a simple computer network. So, think of a set of computer nodes, uh, the, the nodes represent a computer, um... Windows, Mac OS, sequel server, and then every node exposes a set of vulnerabilities that the agent could potentially exploit. And so, then, as, kind of, the agent is dropped into that environment, the agent needs to, first, uncover those nodes, so there's a set of actions that allows to explore the state space. Overall, the environment has a, a limited observability, as the agent gets dropped into the environment, you're not necessarily, uh, giving that agent the entire network topology, uh, the agent first needs to uncover that by exploring the network, exploiting nodes, from those nodes, further explore the network and, essentially, laterally move across the network to achieve a goal that we give it to receive that final reward, that allows the agent to learn.


Natalia Godyla:

And, if I understand correctly, many of the variables were predetermined, such as, the network topology and the vulnerabilities, and, in addition, you tested different environments with different set variables, so how did you determine the different environments that you would test and, within that particular environment, what factors were predetermined, and what those predetermined factors would be.


Christian Seifert:

So we, we determined that based on the domaine expertise that exists


Christian Seifert:

... is within the team, so we have, uh, security researchers that are on a Red Team that kind of do that on a day-to-day basis to penetration tests environments. And so, those folks provided input on how to structure that environment, what nodes should be represented, what vulnerabilities should be exposed, what actions the agent is able to take in- in terms of interacting and exploring that, uh, network. So our Red Team experts provided that information.


Nic Filingham:

I wonder, Christian, if you could confirm for me. So there are elements here in CyberBattleSim that are fixed and predetermined. What elements are not? And so, I guess my question here is if I am someone interfacing with the CyberBattleSim, what changes every time? How would you sorta define the game component in terms of what am I gonna have to try and do differently every time?


Christian Seifert:

So the- the CyberBattleSim is this parametrized, where you can start it up in a way that the network essentially stays constant over time. So you're able to train an agent. And so, the network size is- is something that is dynamic, that you can, uh, specify upon startup. And then also kinda the node composition, as well as ... So whether ... how many Windows 10 machines you have versus [inaudible 00:19:15] servers, as well as the type of vulnerabilities that are associated with each of those nodes.


Nic Filingham:

Got it. So every time you- you establish the simulation, it creates those parameters and sort of locks them for the duration of the simulation. But you don't know ... The agent doesn't know in advance what they will d- they will be. The agent has to go through those processes of discovery and reinforcement learning.


Christian Seifert:

Absolutely. And- and one- one tricky part within reinforcement learning is- is generalizability, right? When you train an agent on Network A, it may be able to learn how to outperform a Red Team member. But if you then change the network topology, the agent may completely flail and not able to perform very well at all and needs to kind of re- retrain again. And that- that's a common problem within the- the re- reinforcement learning research community.


Natalia Godyla:

In the blog you also noted a few opportunities for improvement, such as building a more realistic model of the simulation. The simplistic model served its purpose, but as you're opening the project to the broader community, it seems l- that you're endeavoring to partner with the other researchers to create a more realistic environment. Have you given some early thought as to how to potentially make the simulation more real over time?


Christian Seifert:

Absolutely. There is a long list of- of things that we, uh, need to think about. I mean, uh, network size is- is one component. Being able to simulate a- a regular user in that network environment, dynamic aspects of the network environment, where a node essentially is added to the network and then disappears from the network. Uh, all those components are currently not captured in the simulation as it stands today. And the regular user component is an important one because what you can imagine is if we have an attacker that is able to exploit the network and then you have a defender agent within that network as well, if there is no user component, you can very easily secure that network by essentially turning off all the nodes.


Christian Seifert:

So in- a defender agent needs to also optimize, uh, to keep the productivity of the users that are existing on the network high, which is currently not- not incorporated in- in the simulation.


Nic Filingham:

Oh, that's w- that's amazing. So there could be, you know, sort of a future iteration, sort of a n- network or environment productivity, like, score or- or even a dial, and you have to sort of keep it above a particular threshold while you are also thwarting the advances of the- of the agent.


Christian Seifert:

Absolutely. And I mean, that is, I think, a common trade off in the security space, right? There are certain security m-, uh, measures that- that make a network much more secure. Think of like two-factor authentication. But it does u- add some user friction, right? And so, today we're- we're walking that balance, but I'm hoping that there may be new strategies, not just on the attacker's side, but also on the defender's side, that we can uncover that is able to provide higher level of security while keeping productivity high.


Nic Filingham:

I think you- you- you have covered this, but I- I'd like to ask it again, just to sort of be crystal clear for our audience. So who is the CyberBattleSim for? Is it for Red Teams? Is it for Blue Teams? Is it for students that are, you know, learning about this space? Could you walk us through some of the types of, you know, people and- and roles that are gonna use CyberBattleSim?


Christian Seifert:

I mean, I think that the CyberBattleSim today is- is quite simplistic. It is a simulated environment. It is not ... It'-s it's modeled after a real world network, but it is far from being a real world network. So it's, uh, simplistic. It's simulated, which gives us some advantages in terms of, uh, scalability and that learning environment. And so at this point in time, I would say, uh the simulation is really geared towards, uh, the research community. There's a lot of research being done in reinforcement learning. A lot of research is focused on games. Because if you think about a game, that is just another simulated environment. And what we're intending to do here with- with some of the open source releases is really put the spotlight on the security problem. And we're hoping that the- the reinforcement learning researchers and the research community at large will pay more attention to this problem in the security domain.


Nic Filingham:

It's currently sort of more targeted, as you say, as- as researchers, as sort of a research tool. For it to be something that Red Teams and Blue Teams might want to look at adopting, is that somewhere on a road map. For example, if- if you had the ability to move it out of the simulation and into sort of a- a- a VM space or virtual space or perhaps add the ability for users to recreate their own network topology, is that somewhere on your- your wishlist?


Christian Seifert:

Absolutely. I think there's certainly the goal to eventually have these, uh, autonomous defensive agent deployed in real world environments. And so in order to get to that, simulation needs to become more and more realistic in order to achieve that.


Joshua Neil:

There's a lot of work to be done there. 'Cause reinforcement learning on graphs, big networks, i- is computationally e- expensive. And just a lot of raw research, mathematics and computing that needs to be done to get to that real- real world setting. And security research. And in incorporating the knowledge of these constraints and goals and rewards and things that ... T- that takes a lot of domain research and getting- getting the- the security situation realistic. So it's hard.


Christian Seifert:

In the simulation today, it provides the environment and ability for us to train a Red Team agent. So an agent that attacks the environment. Today, the defender is very simplistic, modeled probabilistically around cleaning up machines that have been exploited. So as kinda the next point on the wishlist is really getting to a point where we have the Red Team agent play against a Blue Team agent and kinda play back and forth and see kinda how that influences the dynamic of the game.


Natalia Godyla:

So Christian, you noted one of the advantages of the abstraction was that it wasn't directly applicable to the real world. And because it wasn't approved as a safeguard against nefarious actors who might use CyberBattleSim for the wrong reason. As you're thinking about the future of the project, how do you plan to mitigate this challenge as you drive towards more realism in the simulation?


Christian Seifert:

That is certainly a- a- a risk of this sort of research. I think we are still at the early stages, so I think that risk is- is really nonexistent as it stands right now. But I think it can become a risk as the simulation becomes more sophisticated and realistic. Now, we at Microsoft have the responsible AI effort that is being led at the corporate level that looks at, you know, safety, reliability, transparency, accountability, e- et cetera, as kind of principles that we need to incorporate into our AI systems. And we, early on, engaged the proper committees to help us shape the- the solution in a responsible fashion. And so at this point in time, there weren't really any concerns, but, uh, as the simulation evolves and becomes more realistic, I very much expect that we,


Christian Seifert:

... be, uh, need to employ particular safeguards to prevent abuse.


Nic Filingham:

And so without giving away the battle plan here, wh- what are some other avenues that are being, uh, explored here as part of this trying to get ahead of this eventual point in the future, where there are automated agents out there in the wild?


Joshua Neil:

This is the- the core effort that we're making, and it's hard enough. I'll also say I think it's important for security folks like us, especially Microsoft, to try hard things and to try to break new ground and innovation to protect our customers and really the world. And if we only focus on short-term product enhancements, the adversaries will continue to take advantage of our customers' enterprises, and we really do need to be taking these kind of risks. May not work. It's too ... It's really, really hard. And t- and doing and in- in purposefully endeavoring to- to- to tackle really hard problems is- is necessary to get to the next level of innovation that we have to get to.


Christian Seifert:

And let me add to that. Like, we have a lot of capabilities and expertise at Microsoft. But in the security space, there are many, many challenges. And so I don't think we can do it alone. Um, and so we also need to kinda put a spotlight on the problem and encourage the broader community to help solve these problems with us. And so there's a variety of efforts that we have pursued over the last, uh, couple of years to do exactly that. So, about two years ago we published a [inaudible 00:28:52] data science competition, where we provided a dataset to the broader community, with a problem around, uh, malware classification and machine risk identification and basically asked the community, "Hey, solve this problem." And there was, you know, prize money associated with it. But I really liked that approach because we have ... Again, we have a lot of d- expertise on the team, but we're also a little bit biased, right, in- in terms of kinda the type of people that we have, uh, and the expertise that we have.


Christian Seifert:

If you present a problem to the broader research community, you'll get a very different approaches on how people solve the problems. Most likely from com- kind of domains that are not security-related. Other example is an RFP. So we funded, uh, several research projects last year. I think it was, uh, $450,000 worth of research projects where, again, we kind of laid out, "Here are some problems that are of interest that we wanna put the spotlight on, and then support the- the research community p- to pursue research in that area."


Nic Filingham:

So what kind of ... You know, you talk about it being, uh, an area that we all sort of collectively have to contribute to and sort of get b- behind. Folks listening to the podcast right now, going and reading the blog. Would you like everyone to go and- and- and spin up CyberBattleSim and- and give it a shot, and then once they have ... Tell us about the- the types of work or feedback you'd like to see. So it's up on GitHub. What kind of contributions or- or feedback here are you looking for from- from the community?


Christian Seifert:

I mean, I'd really love to have, uh, reinforcement learning researchers that have done research in this space work with the CyberBattleSim. Kinda going back to the problem that I mentioned earlier, where how can we build agents that are generalizable in a way that they're able to operate on different network topology, different network configuration, I think is an- an- an exciting area, uh, that I'd love to see, uh, the research community tackle. Second portion is- is really enhancing the simulation. I mentioned a whole slew of features that I think would be beneficial to make it more realistic, and then also kinda tackle the problem of- of negatively impacting potential productivities of- of users that operate on that network. So enhancing the- the simulation itself is another aspect.


Nic Filingham:

Josh, anything you wanted to add to that?


Joshua Neil:

Yeah, I mean, I- I think those were the- the major audiences we're hoping for feedback from. But a- al- also like Christian said, if a psychologist comes and looks at this and has an idea, send us an email or something. You know, that multidisciplinary advantage we get from putting this out in the open means we're anticipating surprises. And we want those. We want that diversity of thought and approach. A physicist, "You know, this looks like a black hole and here's the m- ..." Who knows? You know, but that's- that's the kind of-


Nic Filingham:

Everything's a black hole to a physicist-


Joshua Neil:

(laughs) Yeah.


Nic Filingham:

... so that's, uh ...


Joshua Neil:

So, you know, I think that diversity of thinking is what we really solicit. Just take a look, yeah. Anybody listening. Download it. Play with it. Send us an email. We're doing this so that we get your- your ideas and thinking, for us and for the whole community. Because I think we- we also believe that good security, uh, next generation security is developed by everybody, not just Microsoft. And that there is a- there is a good reason to uplift all of humanity's capability to protect themselves, for Microsoft but for everybody, you know?


Natalia Godyla:

So Christian, what are the baseline results? How long does it take an agent to get to the desired outcome?


Christian Seifert:

So the s- simulation is designed in a way that also allows humans to play the game. So we had one of our Red Teamers to actually play the game and it took that person about 50 operations to compromise the entire network. Now when we take a- a random agent that kinda uninformed takes random actions on the network, it takes about 500 steps. So that's kind of the- the lower baseline for an agent. And then we trained, uh, a Deep Q, uh, reinforcement learning agent, and it was able to accomplish, uh, the human baseline after about 50, uh, training iterations. Again, network is quite simple. I wouldn't expect that to hold, uh, as kinda the- the simulation scales and becomes more complex, but that was, uh, certainly an encouraging first result.


Joshua Neil:

And I think the- the significant thing there is, even if the computer is- takes more steps than the human, well, we can make computers run fast, right? We can do millions of iterations way faster than a- than a human and they're cheaper than humans, et cetera. It's automation.


Nic Filingham:

Is there a point at which the automated agent gets too good, or- or is there sort of a ... What would actually be the definition of almost a failure in this experiment, to some degree?


Joshua Neil:

I think one- one is to- to sort of interpret your question as it could be overfed. That is, if it's too good, it's too specific and not generalized. And as soon as you throw some different set of constraints or network at it, it fails. So I think that's a- that's a real metric of the performances. Okay, it- it learned on this situation, but how well does it do on the next one?


Nic Filingham:

Is there anything else, uh, either of you would like to add before we wrap up here? I feel like I've covered a lot of ground. I'm gonna go download CyberBattleSim and- and try and work out how to execute it. But a- anything you'd like to add, Christian?


Christian Seifert:

No, not from me. It was, uh, great talking to you.


Natalia Godyla:

Well, thank you Josh and Christian, for joining us on the show today. It was a pleasure.


Christian Seifert:

Oh, thanks so much.


Joshua Neil:

Yeah, thanks so much. Lots of fun.


Natalia Godyla:

Well, we had a great time unlocking insights into security, from research to artificial intelligence. Keep an eye out for our next episode.


Nic Filingham:

And don't forget to tweet us at MSFTSecurity, or email us at securityunlocked@microsoft.com, with topics you'd like to hear on a future episode. Until then, stay safe.


Natalia Godyla:

Stay secure. 

 

More Episodes

6/2/2021

Pearls of Wisdom in the Security Signals Report

Ep. 30
It’s our 30thepisode! And in keeping with the traditional anniversary gift guide, the 30thanniversary means a gift of pearls.Sofrom us to you, dear listener, we’ve got an episode with somepearlsofwisdom!On today’s episode, hostsNic FillinghamandNataliaGodylabringback returning champion,Nazmus Sakib, to take us through the newSecurity Signals Report. Sakib walks us through why the reportwasdoneand then helps us understand the findings and what they mean for security.In This Episode You Will Learn:How pervasive firmware is in our everyday livesWhy many people were vulnerable to firmware attacksHow companies are spending the money they allocate towards digitalprotectionSome Questions We Ask:What was the hypothesis going into the Security Signals Report?How do we protect ourselves from vulnerabilities that don’t exist yet?Wereany of the findings from the report unexpected?ResourcesNazmusSakib’sLinkedIn:https://www.linkedin.com/in/nazmus-sakib-5aa8a6123/Security Signals Report:https://www.microsoft.com/security/blog/2021/03/30/new-security-signals-study-shows-firmware-attacks-on-the-rise-heres-how-microsoft-is-working-to-help-eliminate-this-entire-class-of-threats/Nic Fillingham’sLinkedIn:https://www.linkedin.com/in/nicfill/NataliaGodyla’sLinkedIn:https://www.linkedin.com/in/nataliagodyla/Microsoft Security Blog:https://www.microsoft.com/security/blog/Related:Security Unlocked: CISO Series with Bret Arsenaulthttps://SecurityUnlockedCISOSeries.com
5/26/2021

Securing Hybrid Work: Venki Krishnababu, lululemon

Ep. 29
On this week’s Security Unlocked we’re featuring for the second and finaltime,a special crossover episode of our sister-podcast, Security Unlocked: CISO Series with Bret Arsenault.Lululemon has been on the forefront of athleisure wear since its founding in 1998,but while many of its customers look atitexclusively as a fashionbrand,ata deeper level thisfashion empire is bolstered by a well thought out and maintained digital infrastructure that relies on ahard workingteam to run it.On today’s episode, Microsoft CISO Bret Arsenault sits down with VenkiKrishnababu, SVP of Global Technology Services at Lululemon.Theydiscuss the waysin whichtechnology plays into the brand, how Venkileada seamless transition into the remote work caused by the pandemic, and how he’s using the experiences of the past year to influence future growth in the company.In This Episode You Will Learn:Why Venkifeels sopassionatelyabout leading withempathyWhy Venki saw moving to remote work as only the tip of the iceberg; and how he handled whatlaidbelow.Specific tools and practices that haveleadto Venki’ssuccessSome Questions We Ask:What is the biggest lesson learned during the pandemic?How doesone facilitate effective management during this time?Howdoes Lululemonviewthe future of in-person versus remote work?Resources:VenkiKrishnababu’sLinkedIn:https://www.linkedin.com/in/vkrishnababu/Brett Arsenault’s LinkedIn:https://www.linkedin.com/in/bret-arsenault-97593b60/Nic Fillingham’sLinkedIn:https://www.linkedin.com/in/nicfill/NataliaGodyla’sLinkedIn:https://www.linkedin.com/in/nataliagodyla/Microsoft Security Blog:https://www.microsoft.com/security/blog/Related:Security Unlocked: CISO Series with Bret Arsenaulthttps://SecurityUnlockedCISOSeries.com
5/19/2021

Contact Us; Phish You!

Ep. 28
Threat actors arepeskyand, once again,they’reup to no good.A newmethodologyhas schemers compromising onlineformswhere userssubmittheir information like their names, email addresses,and, depending on the type of site, some queries relating totheir life.This new methodindicatesthat the attackers have figured out away around the CAPTCHA’s that have been making us all provewe’renot robotsbyidentifyingfire hydrantssince 1997.Andwhat’smore,we’renot quite surehowthey’vedone it.In this episode, hosts NataliaGodylaand Nic Fillingham sit down with Microsoftthreat analyst, Emily Hacker, to discuss what’s going on behind the scenes as Microsoft begins todigintothis new threat and sort through how best to stop it.In This Episode You Will Learn:Why this attack seems to be more effective against specificprofessionals.Why this new method of attack has a high rate ofsuccess.How to better prepare yourself for this method of attackSome Questions We Ask:What is the endgame for these attacks?What are we doing to protect againstIceIDin these attacks?Are we in need of a more advanced replacementforCAPTCHA?Resources:Emily Hacker:https://www.linkedin.com/in/emilydhacker/Investigating a Unique ‘Form’ of Email Delivery forIcedIDMalwarehttps://www.microsoft.com/security/blog/2021/04/09/investigating-a-unique-form-of-email-delivery-for-icedid-malware/Nic Fillingham’sLinkedIn:https://www.linkedin.com/in/nicfill/NataliaGodyla’sLinkedIn:https://www.linkedin.com/in/nataliagodyla/Microsoft Security Blog:https://www.microsoft.com/security/blog/Related:Security Unlocked: CISO Series with Bret Arsenaulthttps://SecurityUnlockedCISOSeries.comTranscript[Full transcript can be found athttps://aka.ms/SecurityUnlockedEp26]Nic Fillingham: (00:08)Hello and welcome to Security Unlocked, a new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft security, engineering and operations teams. I'm Nick Fillingham.Natalia Godyla: (00:20)And I'm Natalia Godyla. In each episode we'll discuss the latest stories from Microsoft Security, deep dive into the newest threat intel, research, and data science.Nic Fillingham: (00:30)And profile some of the fascinating people working on artificial intelligence in Microsoft Security.Natalia Godyla: (00:36)And now, let's unlock the pod.Nic Fillingham: (00:40)Hello, the internet. Hello, listeners. Welcome to episode 28 of Security Unlocked. Nic and Natalia back with you once again for a, a regular, uh, episode of the podcast. Natalia, how are you?Natalia Godyla: (00:50)Hi, Nic. I'm doing well. I'm stoked to have Emily Hacker, a threat analyst at Microsoft back on the show today.Nic Fillingham: (00:58)Yes, Emily is back on the podcast discussing a blog that she co-authored with Justin Carroll, another return champ here on the podcast, called Investigating a Unique Form of Email Delivery for IcedID Malware, the emphasis is on form was, uh, due to the sort of word play there. That's from April 9th. Natalia, TLDR, here. What's, what's Emily talking about in this blog?Natalia Godyla: (01:19)In this blog she's talking about how attackers are delivering IcedID malware through websites contact submission forms by impersonating artists who claim that the companies use their artwork illegally. It's a new take targeting the person managing the submission form.Nic Fillingham: (01:34)Yeah, it's fascinating. The attackers here don't need to go and, you know, buy or steal email lists. They don't need to spin up, uh, you know, any e- email infrastructure or get access to botnets. They're, they're really just finding websites that have a contact as form. Many do, and they are evading CAPTCHA here, and we talk about that with, with, with, uh, Emily about they're somehow getting around the, the CAPTCHA technology to try and weed out automation. But they are getting around that which sort of an interesting part of the conversation.Nic Fillingham: (02:03)Before we get into that conversation, though, a reminder to Security Unlock listeners that we have a new podcast. We just launched a new podcast in partnership with the CyberWire. It is Security Unlocked: CISO Series with Bret Arsenault. Bret Arsenault is the chief information security officer, the CISO, for Microsoft, and we've partnered with him and his team, uh, as well as the CyberWire, to create a brand new podcast series where Bret gets to chat with security and technology leaders at Microsoft as well as some of his CISO peers across the industry. Fantastic conversations into some of the biggest challenges in cyber security today, some of the strategies that these big, big organizations are, are undertaking, including Microsoft, and some practical guidance that really is gonna mirror the things that are being done by security teams here at Microsoft and are some of Microsoft's biggest customers.Nic Fillingham: (02:52)So, I urge you all to, uh, go check that one out. You can find it at the CyberWire. You can also go to www.securityunlockedcisoseries.com, and that's CISO as in C-I-S-O. CISO or CISO, if you're across the pond, securityunlockedcisoseries.com, but for now, on with the pod.Natalia Godyla: (03:12)On with the pod.Nic Fillingham: (03:18)Welcome back to the Security Unlocked Podcast. Emily Hacker, thanks for joining us.Emily Hacker: (03:22)Thank you for having me again.Nic Fillingham: (03:24)Emily, you are, uh, coming back to the podcast. You're a returning champion. Uh, this is, I think your, your second appearance and you're here-Emily Hacker: (03:30)Yes, it is.Nic Fillingham: (03:30)... on behalf of your colleague, uh, Justin Carroll, who has, has also been on multiple times. The two of you collaborated on a blog post from April the 9th, 2021, called Investigating a Unique Form-Emily Hacker: (03:43)(laughs)Nic Fillingham: (03:43)... in, uh, "Form", of email delivery for IcedID malware. The form bit is a pun, is a play on words.Emily Hacker: (03:51)Mm-hmm (affirmative).Nic Fillingham: (03:51)I- is it not?Emily Hacker: (03:53)Oh, it definitely is. Yeah.Nic Fillingham: (03:54)(laughs) I'm glad I picked up on that, which is a, you know, fascinating, uh, campaign that you've uncovered, the two of you uncovered and you wrote about it on the blog post. Before we jump into that, quick recap, please, if you could just reintroduce yourself to the audience. Uh, what, what do you do? What's your day-to-day look like? Who do you work with?Emily Hacker: (04:09)Yeah, definitely. So, I am a threat intelligence analyst, and I'm on the Threat Intelligence Global Engagement and Response team here at Microsoft. And, I am specifically focused on mostly email-based threats, and, as you mentioned on this blog I collaborate with my coworker, Justin Carroll, who is more specifically focused on end-point threats, which is why we collaborated on this particular blog and the particular investigation, because it has both aspects. So, I spend a lot of my time investigating both credential phishing, but also malicious emails that are delivering malware, such as the ones in this case. And also business email, compromise type scam emails.Nic Fillingham: (04:48)Got it. And so readers of the Microsoft Security Blog, listeners of Security Unlocked Podcast will know that on a regular basis, your team, and then other, uh, threat intelligence teams from across Microsoft, will publish their findings of, of new campaigns and new techniques on the blog. And then we, we try and bring those authors onto the podcast to tell us about what they found that's what's happened in this blog. Um, the two of you uncovered a new, a unique way of attackers to deliver the IcedID malware. Can you walk us through this, this campaign and this technique that you, you both uncovered?Emily Hacker: (05:21)Yeah, definitely. So this one was really fun because as I mentioned, it evolved both email and endpoint. So this one was, as you mentioned, it was delivering IcedID. So we initially found the IcedID on the endpoint and looking at how this was getting onto various endpoints. We identified that it was coming from Outlook, which means it's coming from email. So we can't see too much in terms of the email itself from the endpoint, we can just see that it came from Outlook, but given the network connections that the affected machines were making directly after accessing Outlook, I was able to find the emails in our system that contains emails that have been submitted by user 'cause either reported to junk or reported as phish or reported as a false positive, if they think it's not a phish. And so that's where I was actually able to see the email itself and determined that there was some nefarious activity going on here.Emily Hacker: (06:20)So the emails in this case were really interesting in that they're not actually the attacker sending an email to a victim, which is what we normally see. So normally the attacker will either, you know, compromise a bunch of senders and send out emails that way, which is what we've seen a lot in a lot of other malware or they'll create their own attacker infrastructure and send emails directly that way. In this case, the attackers were abusing the contact forms on the websites. So if you are visiting a company's website and you're trying to contact them a lot of times, they're not going to just have a page where they offer up their emails or their phone numbers. And you have to fill in that form, which feels like it goes into the void sometimes. And you don't actually know who it went to in this case, the, the attackers were abusing hundreds of these contact forms, not just targeting any specific company.Emily Hacker: (07:08)And another thing that was unique about this is that for some of the affected companies that we had observed, I went and looked at their websites and their contact form does require a CAPTCHA. So it does appear that the attackers in this case have automated the filling out of these contact forms. And that they've automated a way around these CAPTCHAs, just given the, the sheer volume of these emails I'm seeing. This is a good way of doing this because for the attacker, this is a much more high fidelity method of contacting these companies because they don't have to worry about having an incorrect email address if they have gotten a list off of like Pastebin or a list, you know, they purchased a list perhaps from another criminal. Emily Hacker: (07:52)A lot of times in those cases, if they're emailing directly, there's gonna be some, some false emails in those lists that just don't get delivered. With the contact form, they're designed to be delivered. So it's gonna give the attacker a higher chance of success in terms of being delivered to a real inbox.Natalia Godyla: (08:11)And so when we, we talk about the progression of the attack, they're automating this process of submitting to these contact forms. What are they submitting in the form? What is the, and what is the end goal? So there's malware somewhere in their-Emily Hacker: (08:27)Mh-mm-hmm (affirmative).Natalia Godyla: (08:27)... response. What next?Emily Hacker: (08:29)Yeah. It's a really good question. So the emails or rather the contact form submissions themselves, they're all containing a, a lore. So the contents themselves are lore that the attacker is pretending to be a, um, artist, a photographer, and illustrator, something along those lines. There's a handful of different jobs that they're pretending to be. And they are claiming that the company that they are contacting has used an image that belongs to the artist, illustrator, photographer on their website without permission. And so the attacker is saying, "You used my art without permission. I'm going to sue you if you don't take this down, if you wanna know what aren't talking about, click on this link and it'll show you the exact art that I'm talking about or the exact photo." What have you, all of the emails were virtually identical in terms of the content and the lore.Emily Hacker: (09:21)The attacker was using a bunch of different fake emails. So when you fill out a contact form, you have to put your email so the, the company can contact you, I guess, in reply, if they need to. And the attackers, almost every single email that I looked at had a different fake attacker email, but they did all follow a really consistent pattern in terms of the, the name, Mel and variations on that name. So they had like Melanie, I saw like Molina, like I said, there was hundreds of them. So the email would be Mel and then something relating to photography or illustration or art, just to add a little bit more credence, I think to their, to their lore. It made it look like the email address was actually associated with a real photographer. The, the attacker had no need to actually register or create any of those emails because they weren't sending from those emails. They were sending from the contact form. So it made it a lot easier for the attacker to appear legitimate without having to go through the trouble of creating legitimate emails. Emily Hacker: (10:16)And then the, um, the email itself from the recipients view would appear other than the fact that it felt fishy, at least to me, but, you know, I literally do this for a living. So maybe just everything feels fishy to me. Other than that, the email itself is going to appear totally legitimate because since it's coming through the contact form, it's not going to be from an email address. They don't recognize because a lot of times these contact forms are set up in a way where it'll send from the recipient's domain. So for example, a contact form, I don't know if this is how this works, but just as an example at Microsoft might actually send from Microsoft.com or the other large percentage of these that I saw were sent from the contact form hosting provider. So there are a lot of providers that host is kind of content for companies. And so the emails would be coming from those known email addresses and the emails themselves are gonna contain all of the expected fields, all in all. It's basically a legitimate email other than the fact that it's malicious.Nic Fillingham: (11:17)And, and just reading through the sample email that you, that you have in the blog post here, like sort of grammatically speaking it's, it reads very legitimately like, the-Emily Hacker: (11:26)Mh-mm-hmm (affirmative).Nic Fillingham: (11:27)... you know, the s- the, the grammar and the spelling is, it's colloquial, but it's, but it seems, you know, pretty legitimate. The idea of a photographer, a freelance photographer, stumbling upon their images being used without permission. You know, you hear stories of that happening. That seems to be somewhat plausible, not knowing how to contact the, the infringing organization. And then therefore going to the generic contact us form like this all, this all seems quite plausible. Emily Hacker: (11:52)And, definitely. And it's als one of those situations where even though, like I said, I do this for a living, so I read this and I was like, there's no way that's legit. But if my job was to be responsible for that email inbox, where stuff like this came in, it would be hard for me to weigh the consequences of like, is it more likely that this is like a malicious email? Or is it yeah. Is it possible that this is legit? And if I ignore it, my company is gonna get sued. Like, I feel like that kind of would give the recipient that, that weird spot of being like, "I don't want to infect the company with malware, or, you know, I don't wanna click on a phishing link if that's what this is, but also if I don't and then we get sued, is it my fault?"Emily Hacker: (12:33)I just, I, I feel for the recipient. So I, I understand why people would be clicking on this one and infecting themselves. And speaking of clicking on that is the other thing that's included in this email. So that was the last bit of this email that turns us from just being weird/legitimate, to totally malicious. All of the emails contain a link. And, um, the links themselves are also abusing legitimate infrastructure. So that's, uh, the next bit of abused, legitimate infrastructure that just adds that next bit of like believability if that's a word to this campaign.Nic Fillingham: (13:05)It is a word.Emily Hacker: (13:06)Okay, good believability. Is that the, the links, you know, we're, if you don't work insecurity, and even if you do work in security, we're all kind of trained like, "Oh, check the links, hover over the links and make sure it's going somewhere that you expect and make sure it's not going to like bad site dot bad, dot bad or something," you know, but these don't do that. All of the emails contained a sites.google.comm link. And I've looked at literally hundreds of these, and they all contain, um, a different URL, but the same sites.google.com domain. If you click on the link, when you receive the email, it'll take you actually to a legitimate Google authentication page that'll ask you to log in with your Google credentials, which again, every step along the way of this, of the email portion of this, of this attack, the attacker just took extra steps to make it seem as real as possible, or to almost like every piece of security advice. Emily Hacker: (14:01)I feel like they did that thing. So it seemed more legitimate because it's not a phishing page. It's not like a fake Google page that's stealing your credentials. It's a real where you would log in with your real Google credentials. Another thing that this does outside of just adding an air of legitimacy to the emails, it also can make it difficult for some security automation products. So a product that would be looking at emails and detonating the link to see if they're malicious and this case, it would detonate the link and it would land on, you know, a real Google authentication page. And in some cases it may not be able to authenticate. And then it would just mark these as good, because it would see what it expected to see. So, outside of just seeming legit, it also makes, you know, security products make this think it's more legit as well. But from there, the, uh, user would be redirected through a series of attacker own domains and would eventually download a zip file, which if they unzipped, they would find the IcedID payload.Emily Hacker: (15:06)So in this case, it's delivering IcedID, although this technique could be used to deliver other stuff as well, but it's not necessarily surprising that it's delivering IcedID right now, because pretty much everything I feel like I'm seeing lately as I study. And I don't think I'm alone in that there's murmurings that IcedID might be replacing Emotets now that you Emotet has been taken down in terms of being, you know, the annoyingly present malware. (laughs) So this is just one of many delivery methods that we've seen for IcedID malware lately. It's certainly in my opinion, one of the more interesting ones, because in the past, we've seen IcedID delivered a lot via email, but, um, just delivered via, you know, the normal type of malicious email if you will, with a compromised email sending with a, a zip attachment, this is much more interesting.Emily Hacker: (15:56)But in this case, if the user downloaded the payload, the payload would actually do many things. So in this case, it was looking for machine information. It was looking to see what kind of security tools were in place to see what kind of antivirus the machine was running. It was getting IP and system information. It was getting, you know, domain information and also looking to access credentials that might be stored in your browser. And on top of that, it was also dropping Cobalt Strike, which is another fun tool that we see used in every single incident lately. It feels like, um, which means that this can give attacker full control of a compromised device.Natalia Godyla: (16:38)So, what are we doing to help protect customers against IcedID? In the blog you stated that we are partnering with a couple of organizations, as well as working with Google.Emily Hacker: (16:52)Yes. So we have notified Google of this activity because it is obviously abusing some of their infrastructure in terms of the sites at Google.com. And they seem to be doing a pretty good job in terms of finding these and taking them down pretty quickly. A lot of times that I'll see new emails come in, I'll go to, you know, click on the link and see what it's doing. And the site will already be taken down, which is good. However, the thing about security is that a lot of times we were playing Catch Up or like, Whack-A-Mole, where they're always just gonna be a step ahead of us because we can't pre block everything that they're going to do. So this is still, um, something that we're also trying to keep an eye on from, from the delivery side as well. Emily Hacker: (17:34)Um, one thing to note is that since these are coming from legitimate emails that are expected is that I have seen a fair bit like, uh, a few of these, uh, actually, um, where the, the customers have their environment configured in a way where even if we mark it as phish, it still ends up delivered. So they have a, what is like a mail flow rule that might be like allow anything from our contact form, which makes sense, because they wouldn't wanna be blocking legitimate requests from co- from customers in their contact form. So with that in mind, we also wanna be looking at this from the endpoint. And so we have also written a few rules to identify the behaviors associated with the particular IcedID campaign. Emily Hacker: (18:16)And it will notify users if the, the behaviors are seen on their machine, just in case, you know, they have a mail flow rule that has allowed the email through, or just in case the attackers change their tactics in the email, and it didn't hit on our rule anymore or something, and a couple slipped through. Then we would still identify this on the endpoint and not to mention those behaviors that the rules are hitting on are before the actual IcedID payload is delivered. So if everything went wrong in the email got delivered and Google hadn't taken the site down yet, and the behavioral rule missed, then the payload itself is detected as I study by our antivirus. So there's a lot in the way of protections going in place for this campaign.Nic Fillingham: (18:55)Emily, I, I wanna be sort of pretty clear here with, with folks listening to the podcast. So, you know, you've, you've mentioned the, the sites.google.com a, a couple of times, and really, you're not, you're not saying that Google has been compromised or the infrastructure is compromised simply that these attackers have, uh, have come up with a, a, you know, pretty potentially clever way of evading some of the detections that Google, uh, undoubtedly runs to abuse their, their hosting services, but they could just evasively has been targeting OneDrive or-Emily Hacker: (19:25)Mh-mm-hmm (affirmative).Nic Fillingham: (19:25)... some other cloud storage.Emily Hacker: (19:25)That's correct. And we do see, you know, attackers abusing our own infrastructure. We've seen them abusing OneDrive, we've seen them abusing SharePoint. And at Microsoft, we have teams, including my team devoted to finding when that's occurring and remediating it. And I'm sure that Google does too. And like I said, they're doing a pretty done a good job of it. By the time I get to a lot of these sites, they're already down. But as I mentioned, security is, is a game of Whack-A-Mole. And so for, from Google point of view, I don't envy the position they're in because I've seen, like I mentioned hundreds upon hundreds of these emails and each one is a using a unique link. So they can't just outright block this from occurring because the attacker will just go and create another one.Natalia Godyla: (20:05)So I have a question that's related to our earlier discussion. You, you mentioned that they're evading the CAPTCHA. I thought that the CAPTCHA was one of the mechanisms in place to reduce spam. Emily Hacker: (20:19)Mh-mm-hmm (affirmative).Natalia Godyla: (20:19)So how is it doing that? Does this also indicate that we're coming to a point where we need to have to evolve the mechanisms on the forms to be a little bit more sophisticated than CAPTCHA?Emily Hacker: (20:33)I'm not entirely sure how the attackers are doing this because I don't know what automation they're using. So I can't see from their end, how they're evading the CAPTCHA. I can just see that some of the websites that I know that they have abused have a CAPTCHA in place. I'm not entirely sure.Nic Fillingham: (20:52)Emily is that possible do you think that one of the reasons why CAPTCHA is being invaded. And we talked earlier about how the, sort of the grammar of these mails is actually quite sophisticated. Is it possible? This is, this is a hands on keyboard manual attack? That there's actually not a lot of automation or maybe any automation. And so this is actually humans or a human going through, and they're evading CAPTCHA because they're actually humans and not an automated script?Emily Hacker: (21:17)There was another blog that was released about a similar campaign that was using the abusing of the contact forms and actually using a very similar lore with the illustrators and the, the legal Gotcha type thing and using sites.google.com. That was actually, it was very well written and it was released by Cisco Talos at the end of last year, um, at the end of 2020. So I focused a lot on the email side of this and what the emails themselves looked like and how we could stop these emails from happening. And then also what was happening upon clicks over that, like I said, we could see what was happening on the endpoint and get these to stop. Emily Hacker: (21:55)This blog actually focused a lot more on the technical aspect of what was being delivered, but also how it was being delivered. And one thing that they noted here was that they were able to see that the submissions were performed in an automated mechanism. So Cisco Talos was able to see that these are indeed automated. I suspected that they were automated based on the sheer volume, but I Talos is very good. They're very good intelligence organization. And I felt confident upon reading their blog that this was indeed automated, how it's being captured though, I still don't know.Natalia Godyla: (22:35)What's next for your research on IcedID? Does this round out your team's efforts in understanding this particular threat, or are, are you now continuing to review the emails, understand more of the attack?Emily Hacker: (22:52)So this is certainly not the end for IcedID. Through their Microsoft Security Intelligence, Twitter account. I put out my team and I put out a tweet just a couple of weeks ago, about four different IcedID campaigns that we were seeing all at the same time. I do believe this was one of them. They don't even seem related. There was one that was emails that contained, um, zip files. There was one that contained emails that contained password protected zip files that was targeting specifically Italian companies. There was this one, and then there was one that was, um, pretending to be Zoom actually. And that was even a couple of weeks ago. So there's gonna be more since then. So it's something that, like I mentioned briefly earlier, IcedID almost feels to be kind of, it feels a little bit like people are calling it like a, the next wave of replacement after Emotech are taken down. Emily Hacker: (23:43)And I don't know necessarily that that's true. I don't know that this will be the new Emotech so to speak, Emotech was Emotech And IcedID is IcedID but it does certainly feel like I've been seeing it a lot more lately. A lot of different attackers seem to be using it and therefore it's being delivered in different ways. So I think that it's gonna be one that my team is tracking for awhile, just by nature of different attackers using it, different delivery mechanisms. And it'll be, it'll be fun to see where this goes.Nic Fillingham: (24:13)What is it about this campaign or about this particular technique that makes it your Moby Dick-Emily Hacker: (24:17)(laughs) Nic Fillingham: (24:17)... if I may use the analogy.Emily Hacker: (24:20)I don't know. I've been thinking about that. And I think it has to do with the fact that it is so, like, it just feels like a low blow. I don't know. I think that's literally it like they're abusing the company's infrastructure. They're sending it to like people whose job is to make sure that their companies are okay. They're sending a fake legal threat. They're using legit Google sites. They're using a legit Google authentication, and then they're downloading IcedID. Like, can you at least have the decency, descend to crappy like unprotected zip attachment so that-Nic Fillingham: (24:49)(laughs)Emily Hacker: (24:49)... we at least know you're malicious, like, come on. It's just for some reason it, I don't know if it's just 'cause it's different or if it's because I'm thinking back to like my day before security. And I, if I saw this email as this one that I would fall for, like maybe. And so I think that there's just something about that and about the, the fact that it's making it harder to, to fully scope and to really block, because we don't want to block legitimate contact emails from being delivered to these companies. And obviously they don't want that either. So I think that's it.Nic Fillingham: (25:22)What is your guidance to customers? You know, I'm a security person working at my company and I wanna go run this query. If I run this, I feel like I'm gonna get a ton of results. What do I do from there?Emily Hacker: (25:33)That's a good question. So this is an advanced hunting query, which can be used in the Microsoft Security portal. And it's written in advanced hunting query language. So if a customer has access to that portal, they can just copy and paste and search, but you're right. It is written fairly generically to a point where if you don't have, you know, advanced hunting, you can still read this and search and whatever methodology, whatever, you know, searching capabilities you do have, you would just have to probably rewrite it. But what this one is doing the top one, 'cause I, I have two of them written here. The first one is looking specifically at the email itself. So that rejects that's written there is the, um, site.google.com.Emily Hacker: (26:16)All of the emails that we have seen associated with this have matched on that rejects. There was this morning, like I said, I was talking to a different team that was also looking into this and I'm trying to identify if she found, um, a third pattern, if she did, I will update the, um, AHQ and we have, we can post AHQ publicly on the Microsoft advanced hunting query, get hub repo, which means that customers can find them if we, if we change them later and I'll be doing that if that's the case, but point being this rejects, basically it takes the very long, full URL of this site.google.com and matches on the parts that are fairly specific to this email.Emily Hacker: (27:02)So they all contain, you know, some of them contain ID, some of them don't, but they all contain that like nine characters, they all contain view. It's just certain parts of the URL that we're seeing consistently. And that's definitely not by itself going to bubble up just the right emails, which is why have it joined on the email events there. And from there, the, I have instructed the users to replace the following query with the subject line generated by their own contacts, their own websites contact submission form. What I have in there are just a few sample subject lines. So if your website contact form generates the subject line of contact us or new submission or contact form, then those will work. But if the website con-, you know, contact form, I've seen a bunch of different subject lines. Then what this does is that it'll join the two. So that it's only gonna bubble up emails that have that sites.google.com with that specific pattern and a subject line relating to the contact form. Emily Hacker: (28:02)And given the searching that I've done, that should really narrow it down. I don't think there's going to be a ton in the way of other contact emails that are using sites.google.com that are showing up for these people. I wouldn't be surprised if this did return one email and it turned out to be a malicious email related to this campaign. But if the contact form generates its own subject line per what the user inputs on the website, then, you know, the screenshots that are in the blog may help with that, but it might be more difficult to find in that case. There's a second advanced hunting query there, which we'll find on the endpoint.Natalia Godyla: (28:37)And I know we're just about at time here, but one quick question on endpoint security. So if a customer is using Microsoft Defender for endpoint, will it identify and stop IcedID?Emily Hacker: (28:49)Yes, it will. The IcedID payload in this case, we're seeing Defender detecting it and blocking it. And that was what, one of the things I was talking about earlier is that Defender is actually doing such a good job. That it's a little bit difficult for me to see what's, uh, gonna happen next because I'm limited to, um, seeing kind of what is happening on customer boxes. And so, because our products are doing such a good job of blocking this, it means that I don't have a great view of what the attacker was going to do next because they can't, 'cause we're blocking it. So it's of mostly a win, but it's stopping me from seeing if they are planning on doing, you know, ransomware or whatever, but I'd rather not know if it means that our customers are protected from this.Nic Fillingham: (29:32)Well, Emily Hacker, thank you so much for your time. Thanks to you and Justin for, for working on this. Um, we'd love to have you back again on Security Unlocked to learn more about some of the great work you're doing.Emily Hacker: (29:41)Definitely, thank you so much for having me.Natalia Godyla: (29:47)Well, we had a great time unlocking insights into security, from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham: (29:54)And don't forget to tweet us @msftsecurity or email us at securityunlockedatmicrosoft.com, with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla: (30:05)Stay secure.