Security Unlocked

Share

Threat Modeling for Adversarial ML

Ep. 7

How ready is your corporate security team to handle AI and ML threats? Many simply don’t have the bandwidth or don’t see it as a priority. That’s where security engineers like Microsoft’s Andrew Marshall step in. In this episode, hosts Nic Fillingham and Natalia Godyla speak with Andrew about just what his team is doing to teach security professionals and policy makers about the dangers of AI and ML attacks, and walks through some of the documentation, available for free online, that can help guide the response. Plus, why he really, really doesn’t want to talk about Windows Vista. 

  

Nic and Natalia then explore what it’s like to hunt down threats with Sam Schwartz, a program manager with Microsoft Threat Experts. She came to Microsoft right out of college and didn’t even know what malware was. Now, she’s helping coordinate a team of threat hunters on the cutting edge of attack prevention. 

  

In This Episode, You Will Learn:   

• Why data science and security engineering skills don’t necessarily overlap 

• How attackers are using ML to change decision making 

• What security teams are doing to protect AI and ML systems 

• How threat hunters are tracking down the newest security risks 

• Why Microsoft Threat Experts are focused on human adversaries, not malware 

  

Some Questions We Ask:   

• What does the ML landscape look like at Microsoft? 

• How are ML attacks evolving? 

• What is ‘data poisoning’? 

• Why do threat hunters need to limit the scope of their work? 

• What skills do you need to be a security program manager? 


Resources


Threat Modeling AI Systems and Dependencies 


Andrew’s LinkedIn:

https://www.linkedin.com/in/andrew-marshall-47334969/


Sam’s LinkedIn:

https://www.linkedin.com/in/scschwa/


Nic’s LinkedIn:

https://www.linkedin.com/in/nicfill/ 


Natalia’s LinkedIn:

https://www.linkedin.com/in/nataliagodyla/


Microsoft Security Blog:

https://www.microsoft.com/security/blog/


Transcript

(Full transcript can be found at https://aka.ms/SecurityUnlockedEp07)


Nic Fillingham:

Hello, and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft security engineering and operations teams. I'm Nic Fillingham.


Natalia Godyla:

And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft security, deep dive into the newest threat intel research and data science.


Nic Fillingham:

And profile some of the fascinating people working on artificial intelligence in Microsoft security. If you enjoy the podcast, have a request for a topic you'd like covered, or have some feedback on how we can make the podcast better...


Natalia Godyla:

Please contact us at securityunlocked@microsoft.com, or via Microsoft Security on Twitter. We'd love to hear from you.


Nic Fillingham:

Hello, Natalia, welcome to episode seven of Security Unlocked. How are you?


Natalia Godyla:

I'm doing well. Refreshed after Thanksgiving break. What about yourself? Did you happen to eat the pork that you were planning? Those bratty pigs?


Nic Fillingham:

The bratty pigs actually have survived for another day. They live to tell another tale to eat more of my home delivered fresh produce, but we did eat a duck that we farmed on our farm. So that's the second time we've been able to enjoy some meat that we grew on the farm, the little mini farm that we live on, so that was pretty exciting.


Natalia Godyla:

Yeah. That's been the goal all along, right? To be self-sustaining?


Nic Fillingham:

To some degree. Yeah. So we achieved that a little bit over Thanksgiving which was cool. How about you, what'd you do over your Thanksgiving break?


Natalia Godyla:

Well, I made apple bread. Oh, there's a special name for the Apple bread, but I forgot it. Pull-apart Apple bread. And I spent a lot of time on WolframAlpha.


Nic Fillingham:

You spent a lot of time on WolframAlpha? Did your firewall break and it was the only accessible website? How did you even get to WolframAlpha?


Natalia Godyla:

WolframAlpha is like Sporcle. It's like if you have time, you get to play with their technology and they've got...


Nic Fillingham:

Sporcle? Sorry, Sporcle?


Natalia Godyla:

What? Do you not know Sporcle?


Nic Fillingham:

I'm really old. You'll have to explain that one to me. Is this a millennial thing?


Natalia Godyla:

Wow. Okay.


Nic Fillingham:

Bring me up to speed on Sporcle.


Natalia Godyla:

Sporcle is like fast, quick trivia games that you play with a group in one person just types in the answers while you're running through it.


Nic Fillingham:

I thought it was when you take a fork and a spoon and you dip them in glitter. Anyway, so you're on Sporcle, and you're like, "I've completed Sporcle. What's next?"


Natalia Godyla:

And you go to WolframAlpha. That's the next step?


Nic Fillingham:

So, what did you pose to WolframAlpha?


Natalia Godyla:

All right, at what point does a cat's meow become lethal to humans? Good question, right?


Nic Fillingham:

At what point does a cat's meow become lethal to a human? When it's connected to a flame thrower? When the meow is a series of poison darts? What does that mean?


Natalia Godyla:

There are a lot of use cases for cats. In this one, it's how high the decibel of their meow is, because that can eventually hurt a human. But it's really about spacing. Where you put the cat is very critical.


Nic Fillingham:

The question was how loud can I make a cat's meow, so that it hurts a human?


Natalia Godyla:

A well-trained army of cats that meow at the exact same time, synchronized cats.


Nic Fillingham:

Oh, a synchronized army of cats, all directed at a single person. Would their collective uber meow, would that serve as a rudimentary weapon? That was your question?


Natalia Godyla:

Yes.


Nic Fillingham:

And? Answer?


Natalia Godyla:

Theoretically, but it depends on how far away all the cats end up being. I'm now thinking that I should have just like planned to capture the cat's meows in a can or something.


Nic Fillingham:

Capture them in a can. What does that mean?


Natalia Godyla:

Like a can of whoopass.


Nic Fillingham:

Who would capture a cats meow in a can? Okay, Professor Farnsworth.


Natalia Godyla:

You can tell I'm not the expert on these podcasts.


Nic Fillingham:

So hang on, did you work out how many cats you needed in a single location to produce a loud enough meow to hurt somebody? Do you know the answer to this question?


Natalia Godyla:

No. No. I was more focused on total and I don't also know the answer to the question.


Nic Fillingham:

All right, to all the mathematicians out there and audiologists who have dual specialties into the capturing of cat meows into cans, and then the math required to multiply them into a focused beam of uber meow as a rudimentary weapon, please send us an email, securityunlocked@microsoft.com. Oh, Oh, segue, segue. We have email, we have email. We got messages from people who've been listening to the show and they send some very nice things, which is great. And they also gave us some topics they would like us to cover on the show, and we're going to cover one of them today.


Nic Fillingham:

Shout out to Ryan and to Christian and to Tyler who all asked us to continue to thread on adversarial ML and protecting AI systems. We're doing that exactly today on this episode. We have Andrew Marshall joining us, who is going to absolutely continue to thread that Sharon Xia started a couple episodes back talking about protecting AI systems in the MDDR report, and then who are we talking to Natalia?


Natalia Godyla:

Sam Schwartz. So she is a security PM at Microsoft and works directly with the Microsoft Threat Experts Team to deliver managed services to our customers. So she helps to provide threats Intel back to customers and is working on scaling that out, so that more and more customers can benefit from the billions of signals that we have, that we then apply to the data that we get from customers, in order to help identify threats. On to the podcast.


Nic Fillingham:

Welcome to the Security Unlocked Podcast, Andrew Marshall. Thank you for joining us.


Andrew Marshall:

Thank you. It's great to be here. Appreciate you having me on today.


Natalia Godyla:

Yeah, definitely. So why don't we start off by chatting a little bit about your role at Microsoft. Can you let us know what your day to day looks like?


Andrew Marshall:

Sure. So I'm a Principal Security Program Manager in the Customer Security and Trust Organization at Microsoft. My role is a little bit different from a lot of people who are security engineers. I'm not part of a product group. Instead, I work across the company to solve long-tail security engineering problems that maybe one particular group may not have the authority to lead all up. So I do a variety of different things, like killing off old cryptographic protocols, where we have to bring the entire company together to solve a problem.


Andrew Marshall:

And lately, I'd say the past two or three years in particular, my focus has been AI and ML. In particular, the security issues that are new to the space, because it brings an entirely new threat landscape that we have to deal with. And we have to do this as an entire company. So it was another one of those cross-company security engineering challenges that I really enjoy to tackle.


Natalia Godyla:

And what does the ML landscape look like in Microsoft? So if it's cross-company how many models are you looking at? How many different groups are using ML?


Andrew Marshall:

It's a really all over the place. And by that, I mean everybody's using it. And it really is pretty much in universal usage across the engineering groups. And while there's been a big focus to everybody, whether it's in Microsoft or elsewhere, everybody's been interested in jumping on this bandwagon. But as the past couple of years, we've started to see that there are specific security issues that are unique to AI and machine learning, that we're only now, as an industry, are starting to see come out of the world of research-driven, proof of concept contrivances, where somebody created a research paper and a vulnerability that they had to make a bunch of leaps to justify. The pivot is occurring now from that into actual weaponized exploitation of these attacks.


Andrew Marshall:

So what we're trying to solve here from a security perspective is with this worldwide rush to jump on the AI and ML bandwagon, what is the security debt around that? What are the new products and features and detections and mitigations that we need to build as a company to solve these issues for ourselves and for the world? One of those things is really focused on education right now, because we've published a series of documents that we made, we can publish them externally. We've got a machine learning threat taxonomy, which covers the intentional and unintentional threats that are specific to machine learning. We've got some documents that were built on top of that. One of which was called Threat Modeling AI/ML Systems and Dependencies.


Andrew Marshall:

And this is a foundational piece of security engineering education work that's being used at Microsoft right now. The issue being security engineers, who have been... you can be a great security engineer, with tons of experience. You could have been doing this for 15 years, or more, but it most likely also means you don't have any data science expertise, or familiarity. So security engineers and data scientists are not two skillsets that often overlap. Ann Johnson calls them, "platinum unicorns", because that's just this mythical creature that nobody really seems to see. But the idea here is that we want all of our security engineers across the company to be intimately familiar with these net new security threats, specific to AI and ML.


Andrew Marshall:

But here's the problem with all of that. This is such a nascent field, still, especially machine learning specific InfoSec, that if you are going to address these problems today, what you need is you need automation. You need new development work to be able to detect a lot of these attacks, because of the way that they occur. They can either be direct attacks against our model, or they can be attacks against the data that is used to create the model. The detections are very primitive, if they exist at all, and the mitigations are very bespoke. So that means if you find a need to mitigate one of these machine learning threats right now, it means you're probably going to have to design that detection or that mitigation specific to your service in order to deal with that issue. That's not a scalable solution for any company.


Andrew Marshall:

So where we need to be is we need to get the detections and mitigations for these machine learning specific threats, get them to be transparent, on by default, inherited by the nature of using the. Platform where it just works under the hood, and you can take it for granted, like we take for granted all of the compiled in threat mitigations that you get when you build code in Visual Studio. So for example, Visual Studio, if you build code there, you inherit all of these different compiled in threat mitigations. You don't have to be a security engineer or know anything about this stuff, but you get all of that goodness just by nature of using the platform. It's on by default and you're oblivious to it. And that makes it easy to use. So, that's where we need to get with this threat landscape too. That's just a very exciting, very challenging space to be a part of.


Nic Fillingham:

Well, I think we're done. Thanks very much, Andrew. No, joking. Wow, so much there. Thank you for that intro. So I think my first question is this conversation we're having is following one that we have with Sharon Xia recently talking about the machine learning insecurity section that was in the recently published Microsoft Digital Defense Report. You're referring to the threat modeling AI systems and dependencies work that's up on the docs page. We'll put a link to that in show notes. When we spoke to Sharon, she really called out upfront, and I think you've just really emphasized that the sort of awareness... This is a very nascent topic. And especially at the customer level, awareness is very low and there needs to be awareness in this field. So I think what is Microsoft doing... First, maybe what is Microsoft's role in promoting awareness of this new category and what are we doing there?


Andrew Marshall:

So we have a role on a couple of fronts here, both within the company and more broadly, within industry and with different governments and customers around the world. So our responsibility is to act... Internally, we'll help shaping not only the educational efforts within the company, but also the research and engineering investments that are made in order to address these issues and solve these problems in this space. There's a policy shaping side of that as well, which is working with governments and customers around the world to help them shape meaningful, actionable policy. That policy in any kind of space can be a dumping ground for good intentions. So whenever people are working on some kind of new policy or some kind of new standard, we always want to make sure that everything is as practical and as actionable as it can be with... And has to be really crisp because you can't have ambiguous goals. You have to have exit criteria for all of these things.


Andrew Marshall:

And the reason I'm elaborating on that is because my team in the company owns the security development lifecycle. And we're very, very careful about new security requirements that get introduced into that so much so to the point that we try not to introduce new security requirements there, unless we've got some kind of automation already ready to roll for people to use. And that way, we can just tell them, "Hey, this is now a mandatory thing that you have to do, but it's really just run this tool and fix these errors. It's not some kind of new manual attestation or big painful exercise to go through." And that's how we can keep adapting the SDL policy. On the responsible AI side and AI and ethics, we've got... This responsible AI standard that we're working on is basically the guiding principles around responsible AI for Microsoft in terms of how we deal with bias and fairness and transparency and reliability and safety issues as they relate to AI, as well as to security. And this is another element of policy that's being shaped within the company.


Nic Fillingham:

So you mentioned that very few of these guidances have been automated. Obviously, one of the goals is probably, I assume, to get them automated into toolsets and into SDL. So let's... I'm going to put a customer hat on. I'm a customer of Microsoft. How should I feel about the work that Microsoft is doing to secure its own AI and ML systems? So obviously, we're practicing what we preach here and putting these guidances into place. How is success being measured? Or what are the metrics that we're using to, be it manually or automated, to make sure that our own AI and ML systems are protected?


Andrew Marshall:

We're spinning up significant research and engineering investments across the company specifically to tackle these kinds of problems. Part of that is largely security. And it's part of this broader series of AI and ethics investments that we're making, but the security issues in particular, because we know that we've got customers reporting these kinds of things, and because we know that we've got our very own specific concerns in this space, we're putting together roadmaps to deal with these kinds of issues as specific sets of new product features and threat detections and mitigations in this space.


Andrew Marshall:

We understand that you can't catch any of these things manually. It takes automation to catch any of this stuff. So that gives us a roadmap of engineering investments that we can prioritize and work directly with engineering groups across the company to go solve that. And the idea here being that when we deliver those solutions, they're not just available to Microsoft services, but they'll be made available to customers of Microsoft as well.


Natalia Godyla:

So, Andrew, how are we seeing these attacks start to evolve already? So if you could talk us through a couple of examples, like data poisoning, that would be awesome.


Andrew Marshall:

Oh, I'd love to. So data poisoning is something that we've seen our own customers impacted by because as we point out in our threat modeling guidance, there's a tremendous over-reliance on using public uncurated data feeds to train machine learning models. Here's an example of a real situation that did happen. So a customer was aggregating trading data feeds for a particular futures market. Let's just say it was oil. And they're feeding these training data feeds from different trading institutions, brokerages, or trading websites or whatever. They're taking all this stuff over a secure channel, they're generating a machine learning model from it. And then they're using that ML model to make some really high consequence decisions like is this location a good place to drill for oil or bid on rights by which you can drill for oil? Or do we want to take a long position or a short position in the oil futures market?


Andrew Marshall:

So they're essentially trusting the decisions that come out of this machine learning model. And what's the nature of futures trading data feeds there's new data every day, so they're constantly incorporating this new data. Talking about the blind reliance on this untrusted data, even though it was over a secure channel, one of the training data providers was compromised not in a way that resulted in the website being shut down, but what happened was their data was contaminated. The data that they were sharing with everybody else. Unless you're actively monitoring for something like this as the provider of that data, there's no way that you're going to know that you're sending out that data to everybody else.


Andrew Marshall:

So if the providers are unaware of the compromise, then the consumer of the data is also going to be equally as oblivious to the fact. So what happens is over time, that data became trusted high confidence garbage within that trading data model. So then that led to these decisions like drilling for oil in the wrong place or longing the futures market when they should have been shorting it and vice versa. So the point here is without automation to detect that kind of data poisoning attack, you don't know anything went wrong until it blows up in your face.


Natalia Godyla:

It really gives you perspective because I feel like normally when you're hearing about cyber attacks, you are hearing about data being stolen and then sold or money itself being stolen. But in the case that you just explained, it's really about altering decision-making, it wasn't just direct money stealing.


Andrew Marshall:

That was an interesting case because we're also thinking, all right, well, was it a targeted attack against the consumer or the people building machine learning models? How did the attacker know that? Were they looking to see what kind of outcomes this would generate? Is this the only place that they were doing that? Of course the data provider doesn't know. That's one of the more interesting, more insidious attacks that we've seen because we've got to create new types of tools and protections in order to even detect that kind of stuff in the first place. So you're looking for... As your machine learning models are being built, you're looking at taking on new data and looking for statistically significant drift in certain parts of the data that deviate from what looks normal and the rest of your data, and we're looking at ways of solving that. And that's an interesting space. So, yeah.


Natalia Godyla:

So you noted that one of the potential reasons that the threat actor was playing around with that ML system for that customer example was because they were also just trying to figure out what they could do. So if it's so nascent then threat actors, are they in a similar place as us? Are they ahead of us?


Andrew Marshall:

Well, we've already had that pivot from contrived research exploits where people are just trying to show off. We've already had that pivot into actual exploitation. So I don't know how to go back and attribute the level of attacker sophistication there. I don't think it was actually... In the attack that I mentioned here, the oil company scenario, that was compromised through traditional security vulnerabilities of that data provider. And I think the jury is still out on the final attribution of all of that, as well as the level of attacker sophistication or if... What would be even more interesting than all of that is really what other customers of that data provider were compromised in this and building machine learning models that were contaminated by that data. Think about hedge funds, who else was compromised by this and never even found out? Or who else had a model blow up in their face? That'd be a very interesting thing to see.


Nic Fillingham:

The question I wanted to wrap up with, Andrew, is make me feel like we're on a good path here. Like, can we end on a high note? We talked about a lot of very serious scenarios and the consequences for adversarial ML. And obviously it's very important and very nascent, but should I feel like the good guys are winning? Should I feel like we've got good people on this? We're making great progress? That we should feel confident in AI and ML systems in-


Andrew Marshall:

Yeah, absolutely.


Nic Fillingham:

The second half of 2020?


Andrew Marshall:

That's our entire focus with the AI and ethics and engineering and research group. We are bringing the entire weight of Microsoft to bear around these issues from a research, engineering, and policy perspective. And we want to solve all these issues so that you do have trustworthy interactions with all of our products. And that's an essential thing that we realized collectively as a company that has to happen where people won't use these kinds of products. If it doesn't generate an outcome that you can trust is going to be accurate and free of bias and something that you can rely on, then people just won't use those things. So we've got the AI and security centers of gravity working across the company with research and policy experts to tackle these issues. It's a fascinating time to be a part of this. I think that... I just had my 20 year anniversary last month, and I think this is about the most fun I've had period in the past 20 years working on this stuff now.


Nic Fillingham:

It wasn't the launch of Windows Vista?


Andrew Marshall:

I have so many horror stories from that. We really don't want to air those.


Nic Fillingham:

Well, that's awesome. Gosh, what was I... I had this great question I was going to ask you and then the Vista joke popped in and now my brain is mulched.


Natalia Godyla:

I love how that took priority.


Nic Fillingham:

Like the most intelligent question I'm going to ask the entire interview and it's like just a joke bonk.


Andrew Marshall:

I have some very, very funny stories from Vista, but none that are appropriate for here.


Nic Fillingham:

Well, we may have to bring you on another time, Andrew, and try and sanitize some of those stories because the statute of limitations has surely run out on having to revere every single release of Windows. Surely we can make fun of Vista soon, right?


Andrew Marshall:

I'm sure we can.


Nic Fillingham:

So, Andrew, final question, where do you recommend folks go to learn more about this space and keep up to speed with any of the advancements, new taxonomy, new guidelines that come out?


Andrew Marshall:

I would definitely keep tabs on the Microsoft Security blog. That's going to be the place where we drop all of the new publications related to anything in this space and connect you with security content more broadly, not just AI and ML specific, but yeah, the Microsoft Secure Blog, that's where you want to be.


Nic Fillingham:

Great. Thanks Andrew Marshall for your time. We'll also put a link up to the guidelines on the doc's page.


Andrew Marshall:

All right. Thank you very much for having me today. It's been great.


Natalia Godyla:

And now let's meet an expert in the Microsoft Security Team to learn more about the diverse backgrounds and experiences of the humans creating AI and tech at Microsoft.


Natalia Godyla:

Hello everyone. We have Sam Schwartz on the podcast today. Welcome Sam.


Sam Schwartz:

Hi, thanks for having me.


Natalia Godyla:

It's great to have you here. So you are a security PM at Microsoft. Is that correct?


Sam Schwartz:

That is correct.


Natalia Godyla:

Awesome. Well, can you tell us what that means? What does that role look like? What is your day-to-day function?


Sam Schwartz:

Yeah, so I support currently a product called the Microsoft Threat Experts and what I am in charge of is ensuring that the incredible security analysts that we have, that are out saving the world every day, have the correct tools and processes and procedures and connections to be the best that they can be.


Natalia Godyla:

So what are some of those processes look like? Can you give a couple examples of how you're helping to shape their day to day?


Sam Schwartz:

Yeah. So what Microsoft Threat Experts does, is it as a managed threat hunting service provided by Microsoft defender ATP product and what they do is our hunters will go through our customer data in a compliant safe way, and they will find bad guys, human adversaries inside of the customer telemetry. And then they notify our customers via a service called the targeted attack notification service.


Sam Schwartz:

So we'll send an alert to our customers and say, "Hey, you have that adversary in your network. Please go do these following things. Also, this is the story about what happened, how they got there and how you can fix it."


Sam Schwartz:

So what I do is I try to make their lives easier by initially providing them with the best amount of data that they can have when they pick up an incident.


Sam Schwartz:

So when they pick up an incident, how do they have an experience where they can see all of the data that they need to see, instead of just seeing one machine that could have potentially been affected, how do they see multiple machines that have been affected inside of a single organization? So they have an easier time putting together the kill chain of this attack.


Sam Schwartz:

So getting the data and then also having a place to visualize the data and easily make a decision as to whether or not they want to tell a customer about it, does it fit the criteria? Does it not? Is this worth our time? Is this not worth our time? And then also providing them with a path to, with that data quickly create an alert to our customers so that they know what they're doing.


Sam Schwartz:

So rather than our hunters, having to sit and write a five paragraph essay about what happened and how it happened, have the ability to take the data that we already have, create words in a way that are intuitive for our customers, and then send it super quickly within an hour to two hours of us finding that behavior.


Sam Schwartz:

So all of those little tools and tracking, and metrics and easier, like creating from data, creating words, sending it to the customers, all of that is what I plan from a higher level to make the hunters be able to do that.


Nic Fillingham:

And to better understand the scale of what's happening here, like with a typical customer, what is the volume of signal or alerts or, I'm not sure what the correct taxonomy is, but what's the volume of stuff that's being monitored from the customer and then is being synthesized down to a bunch of alerts that then go and get investigated by a hunter?


Sam Schwartz:

So I don't have a per customer basis, but we have about, I think it's either 450 customers currently enrolled in our program. And unfortunately, we can't take everyone that would like to join us. Our goal is that we will eventually be able to do that, but we don't have enough people and we're still building our tooling to allow us to scale.


Sam Schwartz:

So with our 450 customers, we have every month, about 200,000 incidents that get created and we then bring that down. So some of those incidents don't get investigated because they don't meet our bar. Some of those incidents get investigated, but aren't interesting enough to actually have an alert created. And some of them even, although the alert is created, it's not actually interesting enough to send, or we've already sent something similar and it's not worth it.


Sam Schwartz:

So from those 200,000, we send about 200 to like 250 alerts a month about, but it also depends on the landscape. Like it depends on what's going on that-


Nic Fillingham:

And if I go even higher up the funnel, so before the 200,000 is it, what's the right taxonomy, is it an alert?


Sam Schwartz:

Incidents. We call them incidents.


Nic Fillingham:

... What's above an incident. What is, because I assume it's just tons and tons and tons of network logs and smaller signals that end up getting coalesced into an incident. Is that correct?


Sam Schwartz:

Yeah. So what we do is we call them traps. So what they are is they're queries that run over data that finds something super interesting. And you can think about these as similar to alerts that customers get, but much, much, much lower fidelity.


Sam Schwartz:

So for us, for our products, a trap, if it fires a hundred times and of that a hundred times, 99 of them are false positives, 99% of them are not super helpful for the customer, we're not going to send that to the customer. That's bothering them 99 times that they don't need to be bothered. But for our service, our whole thing is that we are finding that 1% that our customer doesn't know about.


Sam Schwartz:

So we have extremely low fidelity traps. Some of them are high fidelity that it can run a thousand times and only one time is it important? We want to see every a thousand times because that one time is worth it. So we have traps, I think we have about 500 of them. Some of them return thousands of results a day. Some of them won't return results for months.


Sam Schwartz:

And if that gets a hit, then those are the things that get bubbled up into our incidents. We cluster all of those trap results into the incidents, so that's ensuring that our hunters get all the information that they need when they log on, so the signals are massive. There's a massive amount. I don't even have a number.


Natalia Godyla:

I have literally so many questions.


Sam Schwartz:

Oh my God, happy to help.


Natalia Godyla:

So you said earlier, there's a bar for what the Microsoft Threat Experts will focus on. So what is in scope for them? What meets the criteria?


Sam Schwartz:

We are focusing on human adversaries. So we're not focusing much on commodity malware, as much as we are focusing on a hands-on keyboard attacker. So there are some traps that are, some of them are commodity malware, but paired with other traps so paired with other signals, that could be a hands-on keyboard person. And those are things we look at, but then maybe some of the traps on their own don't meet a bar for us to go look at.


Nic Fillingham:

Is that because commodity malware is basically covered by other products, other services?


Sam Schwartz:

(Affirmative). It's covered by our defender ATP product in general. So our hunters wouldn't be adding. Our whole point is that we have hunters who are adding context and value to the already incredible ATP product. And since ATP is already alerting and covering that, we'd rather find the things that aren't being covered.


Nic Fillingham:

So Sam, let's go back in time a little bit, so tell us about how you found yourself in the security space and maybe it's a separate story maybe it's the same story and how you got to Microsoft. We'd love to learn your journey, please.


Sam Schwartz:

It is the same story. Growing up, I loved chemistry.


Nic Fillingham:

That's too far back.


Sam Schwartz:

I know.


Nic Fillingham:

Oh, sorry. Let's start there.


Sam Schwartz:

I loved Chemistry. I loved like molecules and building things and figuring out how that all works. So when I went to college, I was like, I want to study chemical engineering. So I through my education became a chemical engineer, but I found that I really liked coding. We had to take a fundamentals class at the beginning and I really enjoyed the immediate feedback that you got from coding. Like you did something wrong, it tells you immediately that you messed up.


Sam Schwartz:

And also when you mess up and you're super frustrated and you're like, why didn't this work? Like I did it right. You didn't do it right, it messed up for a reason. And I really liked that. And I thought it was super interesting. And I found myself like gravitating towards jobs that involved coding.


Sam Schwartz:

So I worked for Girls Who Code for a summer. I worked for a Dow Chemical Company, but in their robotics division. So I was still like chemical engineering, but I got to do robots. And then when I graduated, I was like, I think I want to work in computer science. I don't like this chemical engineering. It was quite boring, even though they said it would get more fun, it never did. We ended up watching water boil for a lot of my senior year of college. And I was like, I want to join a tech company.


Sam Schwartz:

And I looked at Microsoft and they're one of the only companies that provide a program management job for college hires. So a lot of PM positions because there's a lot of high level thinking, coordinating and collaboration. A lot of PM positions are one of those, like you need experience, but in order to get experience, you have to do the job and it's like one of those weird circles and Microsoft allows college hires to do it.


Sam Schwartz:

So when I interviewed, I was like, I want to be a PM. It sounds fun to get to hang out with people. And I ended up getting the job, which is awesome.


Nic Fillingham:

Is that all you said in the interview? Just, it sounds fun to get to hang out with people?


Sam Schwartz:

Yes. I was like, this is it, this is my thing. What they did is they, in my interviews, they asked me a bunch of, they asked me a very easy coding question, I was so happy. I was so nervous that I wasn't going to get a pass that one, but that was easy. And then they asked me a design question. They asked me, "Pick your favorite technology." And me, I'm sad to say it. I feel like I'm better now looking back on myself, but I'm really not good with technology in general.


Sam Schwartz:

So they're like pick your favorite technology. And I was like, I'm going to pick a chemical engineering plant because I didn't know anything. So I picked an automation plant as my favorite technology. And they asked me a lot of questions around like, who are the customers? What would you do to change this to affect your customers? Who gets changed? How would you make it better?


Sam Schwartz:

Then I was talking specifically about a bottling plant, just because that's easy to understand. And I left that interview and my interviewer was like, I didn't know, he said, "I didn't know anything that you were talking about, but everything you said made perfect sense because it's about how can you take inputs, do something fun and then have an output that affects someone. And that's everything that we do here. Even though it's a bit obfuscated and you have a bunch of data and bad guys and hunters hunting through things, it's taking an input and creating something great from it."


Sam Schwartz:

And that's what we learned in our chemical engineering world. And I ended up getting this job and I walked on my first day and my team and they're like, "You're on a Threat Intelligence Team." I was like, "What does that mean?" And-


Nic Fillingham:

Oh, hang on. So did you not know what PM role you were actually going to get?


Sam Schwartz:

No. They told me that I was slated for the Windows. I was going to be on a Windows team. So in my head like that entire summer, I was telling people I was going to work on the start button just because like, that's what... I was like, "If I'm going to get stuck anywhere, I'm going to have to do the start button. Like that's where my-"


Nic Fillingham:

That's all there is. Windows is just now a start button.


Sam Schwartz:

I was like that what... I was guaranteed, I'm going to get the star button or like Paint. Actually, I probably would have enjoyed Paint a lot, but the start button and I came and they were like, "You're on Threat Intelligence Team." And I was like, "Oh, fun."


Sam Schwartz:

And it was incredible. It was an incredible start of something that I had no idea what anyone was talking about, when they were first trying to explain it to me in layman's terms, they're like, oh, well, there's malware and we have to decide how it gets made and how we stop it. And I was like, what's malware? I was like, you need to really dumb it down, I have no idea what we're talking about. And initially when I started on this threat intelligence team, there were only five of us. So I was a PM and they had been really wanting a PM, and apparently before they met me were happy to get a PM, but weren't so happy it was a college hire. They're like-


Nic Fillingham:

Who had never heard of malware.


Sam Schwartz:

We need structure.


Nic Fillingham:

And thought Windows was just a giant anthropomorphic start menu button.


Sam Schwartz:

They're like, we need structure, we need a person to help us. And I was like, hi, nice to meet you all. And so we had two engineers who were building tools for our two analysts and it was, we called ourself a little startup inside of security research inside of the security and compliance team, because we were figuring it out. We were like, threat intelligence is a big market, how do we provide this notion of actionable threat intelligence? So rather than having static indicators of compromise, how do we actually provide a full story and tell customers to configure, to harden their machines and tell a story around the acts that you take to initiate all of these. These configurations are going to help you more than just blocking IOCs that are months old. So figuring out how to best give our analyst tools, our TI analysts, and then allow us to better Microsoft products as a whole.


Sam Schwartz:

So based on the information that our analysts have, how do we spread that message across the teams in Microsoft and make our products better? So we were figuring it out and I shadowed a lot of analysts and I read a lot of books and watched a lot of talks. I would watch talks and write just a bunch of questions. Then finally, as you're around all these incredibly intelligent security people, you start to pick it up, and after about a year or so I would send meetings and I would listen to myself speak and I was like, did I say that? Was that me that one, understood the question that was asked of me and then also was able to give an educated answer? It was very shocking and quite fun. And I still feel that way sometimes, but I guess that's my journey into security.


Natalia Godyla:

Do you have any other suggestions for somebody who is in their last years of college or just getting out of college and they're listening to this and saying, heck yes, I want to do what Sam's doing. Any other applicable skills or tricks for getting up to speed on the job?


Sam Schwartz:

I think a lot of the PM job is the ability to work with people and the ability to communicate and understand what people need and be able to communicate that in a way that maybe they can't communicate. See people's problems and be able to fix them. But I think a lot of the PM skills you can get by working collaboratively in groups, and that you can do that in jobs, you can do that in classes. There's ample opportunity to work with different people, volunteering, mentoring, working with people and being able to communicate effectively and connect to people and understand, be empathetic, understand their issues and try to help is something that everyone can do and I think everyone can be an effective PM. On the security side, I think reading and listening. Even the fact that, the hypothetical was someone listening to this podcast that are already light years ahead of I was when I started, but just listening, keeping up to date, reading what's going on in the news, understanding the threats, scouring Twitter for all the goodness going on.


Sam Schwartz:

That's the way to stay on top.


Nic Fillingham:

Tell us about your role and how you interface with data scientists that are building machine learning models and AI systems. Are you a consumer of those models and systems? Are you contributing to them? Are you helping design them? How do you fit into that picture?


Sam Schwartz:

So a little bit of all of the things that you mentioned, being a part of our MTE service, we have so many parts that would love some data science, ML, AI help, and we are both consumers and contributors to that. So we have data scientists who are creating those traps that I was talking about earlier for us, who are creating the indicators of malicious anomalous behavior that our hunters then key off of. Our hunters also grade these traps. And then we can provide that back to the data scientists to make their algorithms better. So we provide that grading feedback back to them to have them then make their traps better. And our hope is that eventually their traps, so these low fidelity signals, become so good and so high fidelity that we actually don't even need them in our service, we can just put them directly in the product. So we work, we start from the incubation, we provide feedback, and then we hopefully see our anomaly detection traps grow and become product detections, which is an awesome life cycle to be a part of.


Nic Fillingham:

I want to change topics then, but this one's going to need a little bit of context setting because you are famous inside of Microsoft for anyone that has completed one of our internal compliance trainings. I don't even know how to describe this to people that haven't experienced it. Natalia, we've both done it. So there's this thing at Microsoft called Standards of Business Conduct, it's like a internal employee compliance. This is how you should behave, this is how you should function as a responsible employee and member of the Microsoft family, but then also how we work with customers and everything. And it's been going on for a few years. Sam, you had a cameo, you were the only non-professional actor in the recent series, that's correct?


Sam Schwartz:

I was, I was, I'm famous, I will be signing headshots when we're all back in the office.


Nic Fillingham:

So tell us about how did this happen?


Sam Schwartz:

So I, as anyone who has seen the Standards of Business Conduct videos, I wouldn't call them a training, I would call them a production.


Nic Fillingham:

An experience. Or production.


Sam Schwartz:

An experience, yeah. An experience.


Nic Fillingham:

They're like a soap opera. It's almost like Days of Our Lives. They really stir the emotion and we get attached to these characters and they go on wild journeys in a very short space of time.


Natalia Godyla:

I was just watching an episode and I literally got stressed.


Sam Schwartz:

Yeah, you're so invested in these characters and their stories and you're rooting for them to do the right thing. And you're like, come on, just be compliant. And in my first week on the job I was telling, I watched this training as everyone who starts Microsoft has to do and I was telling my team that I was obsessed with the main character who has his own trial and tribulations throughout the entire series. And I just thought it was fun and I was like, how do I get on it? That was my thing when I first joined, how do I get on Standards of Business Conduct? And every year, Microsoft is super passionate about giving, giving back, donating money, and every October we have this thing called the Give Campaign where every employee is encouraged to give back to their community.


Sam Schwartz:

And one of the ways that they do is they have an auction. So some of the auction things are, you get lunch or golf with Satya, or you get assigned, I don't know, computer or X-Box from Phil Spencer or whatever it is. I made those up.


Nic Fillingham:

You get to be the Windows start button for a day.


Sam Schwartz:

You get to be the Windows start button for a day. And one of those is a cameo in Standards of Business Conduct. And you can donate a certain amount of money and there's a bid going, where the person who donates the most money is at the leaderboard and then if you donate more money, you got on top. So a silent auction before giving back and donating. And I saw that last year on the gift campaign, but I didn't think much of it. It had a high price tag and I didn't want to deal with it. And then a couple of months later, I had just gotten back from vacation and my skip level was like, hey, I missed you a lot, let's get lunch. And I was like, okay, great, I love that.


Sam Schwartz:

And he was like, I want to go somewhere fun, I want to go to building 35, which is the executive nice cafeteria building at Microsoft, which is not near our office. And I was like, okay, weird, he wants to go to another building for lunch, but we can go do that. So I went with him and it was five to 10 minutes into our lunch and these people come up to our table and they're like, can we sit with you? And I'm looking around and there are tons of tables, I'm like, what are these people encroaching on my lunch for? I just want to have lunch and chat and these people want to come sit at my table, but of course, we're going to let them sit at our table. And I look over at the guy who's sitting next to me and it's the main character from Standards of Business Conduct. It is the actor, it is-


Nic Fillingham:

It's Nelson.


Sam Schwartz:

It's Nelson. And I fan girled over him for a year and a half now, I've seen all his work, I'm a huge fan.


Nic Fillingham:

Please tell me it was a Beatles on their first tour to America moment. Please tell me there was screaming, there was fainting.


Sam Schwartz:

I blacked out.


Nic Fillingham:

That's the picture in my head.


Sam Schwartz:

I don't remember. I don't remember what happened because I actually blacked out. And there's a video of this and you can see my body language, when I realized you can see me grab the arms of the chair and my whole body tenses up, and I'm looking around frantically, like, what's happening. And the woman who was sitting next to my skip-level, she actually created Standards of Business Conduct and she's in a lot of the videos, her name is Rochelle. And she's like, your team has pulled together their money and bought you a cameo in our next Standards of Business Conduct. And I turned around and my entire team is on the balcony of the cafeteria filming me and it was very cute and very emotional. And I got to see Nelson and then I got to be in Standards of Business Conduct, which is awesome. And it was a super fun experience.


Nic Fillingham:

So in the Microsoft cinematic universe, what is your relationship to Nelson? Are you a colleague?


Sam Schwartz:

We're all on the same team.


Nic Fillingham:

So you're a colleague.


Sam Schwartz:

We are on the same team.


Nic Fillingham:

So forever, you're a colleague of Nelson's.


Sam Schwartz:

Yeah, I am. And he knows who I am and that makes me sleep well at night.


Natalia Godyla:

Thank you, Sam, for joining us on the show today, it was great to chat with you.


Sam Schwartz:

Thank you so much for having me. I've had such a fun time.


Natalia Godyla:

Well, we had a great time unlocking insights into security, from research to artificial intelligence. Keep an eye out for our next episode.


Nic Fillingham:

And don't forget to tweet us @MSFTSecurity, or email us at SecurityUnlocked@microsoft.com with topics you'd like to hear on a future episode. Until then, stay safe.


Natalia Godyla:

Stay secure.

More Episodes

4/7/2021

The Language of Cybercrime

Ep. 22
How many languages do you speak?The average person only speaks oneor twolanguages, and for most people that’s plentybecause even as communities arebecoming more global, languages are still very much tied to geographic boundaries.Butwhat happens when you go on the internet where those regions don’t exist the same way they do in real life?Because the internet connects people from every corner of the world, cybercriminals canperpetratescamsin countriesthousands of miles away. So how doorganizationslike Microsoft’s Digital Crime Unit combatcybercrimewhen they don’t even speak the language of the perpetrators?On today’s episode ofSecurity Unlocked, hostsNic FillinghamandNataliaGodylasit down withPeterAnaman,Principal Investigator on the Digital Crimes Unit,to discusshowPeterlooks at digital crimes inavery interconnected world and how language and culture play into the crimes being committed, who’s behind them, and how to stop them.In This Episode, You Will Learn:• Some of the tools the Digital Crime Unit at Microsoft uses to catch criminals.• How language and culturalfactors into cyber crime• Whycyber crimehas been onthe rise since Covid beganSome Questions We Ask:• How has understanding a specific culture helped crack a case?• How does a lawyer who served as an officer in the French Army wind up working at Microsoft?• Are there best practices for content creators to stay safe fromcyber crime?ResourcesPeterAnaman’s LinkedIn:https://www.linkedin.com/in/anamanp/NicFillingham’s LinkedIn:https://www.linkedin.com/in/nicfill/NataliaGodyla’s LinkedIn:https://www.linkedin.com/in/nataliagodyla/Microsoft Security Bloghttps://www.microsoft.com/security/blog/Transcript[Full transcript can be found at https://aka.ms/SecurityUnlockedEp22]Nic:(music)Nic:Hello and welcome to Security Unlocked. A new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft's Security Engineering and Operations Teams. I'm Nic Fillingham.Natalia:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft's Security. Deep dive into the newest threat intel, research and data science.Nic:And profile some of the fascinating people working on artificial intelligence in Microsoft Security.Natalia:And now, let's unlock the pod.Natalia:Hello, Nic. How is it going?Nic:Hello, Natalia. I'm very well, thank you. I'm very excited for today's episode. We talk with Peter Anaman, who is a return guest. Uh, he was on an earlier episode where we talked about business email compromise and some of the findings in the 2020 Microsoft Digital Defense Report. And Peter had such great stories that he shared with us in that conversation, that we thought let's bring him back. And let's, let's get the full picture. And wow, did we cover some topics in this conversation. I don't even know where to begin. How would, what's your TLDR for this one, Natalia?Natalia:Well, whenever your friends or family think about cyber security, this is it. One of the stories that really stuck out to me is, Peter went undercover, and has actually gone undercover multiple times, but in this one instance he used the cultural context from his family history, as well as the languages that he knows to gain trust with a bad actor group and catch them out. It's incredible. He speaks so many languages and he told so many stories about how he applies that to his day-to-day work in such interesting ways.Nic:Yeah, I love, for those of you who listened to the podcast, Peter really illustrates how knowledge of multiple cultures, knowledge of multiple languages, understanding how those cultures and languages can sort of intersect and ebb and flow. Peter has used that as powerful tools in his career. I think it's fascinating to hear those examples. Other listeners of the podcast who, who do have more than one language, who do understand and have experience across multiple cultures, maybe oughta see some, uh, some interesting opportunities for themselves in, in, in cyber security maybe moving forward.Nic:I also thought it was fascinating to hear Peter talk about working to try and get funds and sort of treasures and I think gold, l-literal gold that was taken during the second world war. And getting them back to it's original owner. Sort of like, a repatriation effort. As you say, Natalia, these are all things that I think our friends and family think of when they hear the words cyber security. Oh, I'm in cyber security. I'm an investigator in cyber security. And they have this sort of, visions, these Hollywood visions. Nic:This is, that's Peter. That's what he's done. And he's, he talk about it in his episode. It's a great episode.Natalia:And with that, on with the pod.Nic:On with the pod. Nic:(music)Natalia:Welcome back to Security Unlocked, Peter Anaman.Peter:Thank you very much. Thanks for having me back.Natalia:Well, it was a pleasure to talk to you, first time around. So I'm really excited for the second conversation. And in this conversation we really love to chat about your career in cyber security. How you got here? Um, what you're doing? So let's kick it off with a little bit of a refresher for the audience.Natalia:What do you do at Microsoft and what does your day-to-day look like?Peter:So in Microsoft, I work within the legal department. Within a group called the Digital Crimes Unit. We are a team of lawyers, investigators and analysts who look at protecting our customers and our online services from, um, organized crime or attacks against the system. And so we, we bring, for example, civil and criminal referrals in order to do that action. On a day-by-day basis, it's very, very varied. I focus more on business email compromise present with some, with some assistance on ransomware attacks and looking at the depths and the affiliates there. As well as looking at some attacks against the infrastructure based on automated systems. Peter:So it's kind of varied. So on a day, I could, for example, be running some crystal queries or some specialized database queries in order to look for patterns in unauthorized or illegal activity taking place in order to quickly protect our customers. At the same time, I have to prepare reports. So there's a lot of report writing just to make sure that we can articulate the evidence that we have. And to ensure we respect privacy and all the other rules, you know, when we present the data.Peter:And also, in addition to that, uh, big part of it is actually learning. So I take my time to look at trends of what's going on. Learn new skills in order to know that I can adapt and automate some of the processes I do.Nic:Peter, as someone with an accent, uh, I'm always intrigued by other people's accents. May I inquire as to your accent, sir. Um, I'm hearing, I think I'm hearing like, British. I'm hearing French. There's other things there.Peter:(laughs)Nic:Would you elaborate for us?Peter:Yes, of course. Of course. Oh so, I was born in Ghana, West Africa and spent my youth there. And later on went to the UK where I learned that, I had to have elocution lessons to speak like the queen. And so I had lesson and my accent became British. So but at the same time, I'm actually a French national. Um, I've been in the French army as an officer. And so, that's where the French part is. And throughout, I've lived in different countries doing for work. Uh, so I've learned a bit of German, a bit of Spanish on the way.Nic:I, I actually cheated. I looked at your, um, LinkedIn profile and I see you have six languages listed.Peter:Yes.Nic:The two, the two that you didn't mention, I am embarrassingly ignorant of Fante? And T-Twi, Twi? What are they?Peter:Twi and Fante are two of the languages that are spoken in Ghana. They're local languages. And so growing up, I always had that around me. When I went to my father's village where his, we communicate in that language. English is kind of the National Language but within the country, people really speak their own languages. So I've ticked it off now. Can I speak fluently in, in it? No, I've been away for too long. But if you put me there, I would understand everything they're saying. Nic:What are the roots of those two languages? Are they related at all? Or are they completely separate?Peter:They are related but one, one person cannot always understand the other. If you look more broadly, you look at for example, the African continent all are, you'll find that there are over, from what we understand, over, what was it? 2,000 languages are spoken on the continent. So sometimes a person, say on the east coast doesn't understand the person in the west coast, you know. And, and it's fascinating because, you know, when we look at cyber crime, we are facing a global environment. Which is actually pretty carved out, right? The physical world is still pretty segmented.Peter:And so when, for example, investigating some crimes taking place in Nigeria, well they speak pidgin English. And so we have to try and adapt to that to understand, what do they really mean when they say, X or Y? And so, you know, it kind of opens our mind at, as we're doing the investigations. So we have to really try and understand the local reality because the internet is not just one place. And I think, you know, working for, you know, Microsoft and with such an amazing diverse team, we've been able to share knowledge.Peter:So for example, in the case I mentioned, I went to my colleague in Lagos, Abuja. He went, oh, that's what it means. And we're like, okay great. That one makes a lot more sense. And so we can move on. So we have this kind of richness in the team that allows us to lean on each other and, you know, sort of drive impact. But yeah, language is very important. (laughs)Natalia:I was gonna ask, do you have any interesting examples in which the culture was really important to cracking in the case or understanding a specific part of a case that you were working?Peter:Yes. So there was one case I worked on earlier on which was in Lithuania. And in Lithuania, for a very long time, this group had been under investigation but they were very good at their Op Sec and used some, uh, different types of encryption and obsolete, obsolete communication to hide themselves. But what I learned from the chats and when I was, this was in an IRC, it started in IRC channels and then moved out of there afterwards. But I noticed that there was a lot of Italy. There was a lot of Italian references. And my grandfather was Sicilian so I've spent time in Italy. So I kind of understood that they traveled to Italy.Peter:So in part of the persona, I made reference to Sicily. And I just said, you know, that's where my grandfather's from. And this, didn't give a name obviously, but it kind of brought them closer, right? Because like, oh, yeah we, we get it. And after about two, three months, I was able to get them to send me pictures of them going on vacation in Italy. And unfortunately for them, the picture had geo-location on it. And also, we were able to blow it up to get the background of where they were in the airport and using the camera from the airport, we were able to identify who they were. And then go back to the passport, find their path and they got arrested a few weeks later. Peter:So but to get that picture, to get that inner information required a kind of, trust that was being built in the virtual world and that comes from trying to understand the culture. By teasing out, asking questions about who are you and what do you like. So that's just one example.Nic:N-no pressure in answering this question and we'll even, we'll even cut it out of the edit if it's one you don't wanna go with.Peter:(laughs) Sure.Nic:If you're good with it. But um, uh, I heard you now talk about personas and identities and y-you just sort of hinted at it in the answer to the previous question. It sounds like some of the work that you have done in the past has been about creating and adopting personas in order to go and learn more information about bad actors and groups out there in, uh, in cyber land. Is that accurate and are you able to talk about what that role and that sort of, that work look like, when you're performing it?Peter:Yeah. So before you have Peter:...persona, you have to understand where that persona's gonna be acted, right?Peter:And I'll give you an, an example of a story. Once I had to go to LA to give a presentation and when I got to the airport I got a cab. And in the cab I looked at the guy's, the license plate of the, of the person. And I said, I bet you, I can guess, which country you were born in. He was like, an African American kind of person. He goes, impossible. No one has guessed it, you will never know. I was, all right. Are you ready? You're from Ghana. And his mind was blown. He was like, how, how did you pin that to one country? I was like, well, in your name, you have Kwesi. And I know if you're born in a country, in Ghana and have Kwesi, it means you're born on a Sunday. So that fact that you have your, that name there, that means you were born from Ghana. He goes, you are right. And so that was that. Peter:And I said, I miss some food, the cuisine from my, from, from Ghana. And he goes, oh, I know a great place. It's in Compton. I said, go. Uh, when? So I went into my restroom, showered, go ready, try to g-got into a taxi and he goes, I'm not going into Compton. I was like, well, why not? I wanna go to that restaurant. And he goes, oh, no, no, no. I'm going to get robbed or something bad is going to happen to me. I was like, but it- By the way, he left, he went, I had a great meal. Afterwards, I spent two hours in the restaurant 'cause no taxi would come and pick me up. And eventually, the waitress took me to a local casino. And I got a cab there and I got back.Peter:Where, where I'm going with this story is about the environment. I didn't know what Compton meant, right? So if I created a persona that went there that didn't know the environment, they would not succeed. They would stick out like a sore thumb. They would, they would fail. So the first idea, is always to understand what are the different protocols.Peter:If I'm looking at, for example, FTP or IRC, the different peer-to-peer networks. Or I'm looking at NNTP and the old internet, you know. All of those work, you need different tools to work there. Different ways to collect evidence and different breadcrumbs you could leave that you need to know it may be needed. Because when you're there, you're there, right? And it's, you're leaving, you're leaving a mark. Also some people say, use proxies. Well, the problem with proxies that someone could know you got a proxy on. Because well, there's lots of systems out there. So it's about using the system. Understanding how it's interconnected so that when you show up, you show up without too much suspicion.Peter:The other thing I learned is that the personas have to, have to be kind of, sad. 'Cause what I found is that when they were a bit sad, like, I'm happy with your work and things like that. What I found, that's me, right? I found that people were more interested because people are kind by nature, right? And so when they see that you're sad, they're more likely to communicate with you. While, while if you're too confident, I can do everything. They're like, uh, no, that person. Peter:So I try to like, psychologically look at ways to make the person as real as possible, based on my experience, right, because if it was based on me, I would be called out. Because I will be inventing a character that's, was not real. If you try to give me a trick question, because it's based on me, the answer's gonna be the same. I've got, the persona is me. It's just different. And so that's how I took my time to understand it. I spend a lot of time learning the internet, the protocols, you know, how does P2P actually work. When I, going to an IRC channel or when I'm looking at the peer-to-peer network and looking at the net flow. So the data which is passing from my computer upload. What other information is flowing. Peter:Because if I can see it, they can see it, right? And at the same time I have to have the tools. So I was very fortunate to have, for example, some tools that can switch my IP address with any country, like, every minute. So I could really change personas and change location really rapidly and no one would know better 'cause I'm using different personas in different contexts, right?Peter:Now, I never lie. One of, one of the clear things is that you never, I never try and do anything illegal because I have to assume that law enforcement is on the other side. And that's not what I'm trying to do. So I'm not gonna commit the crime. I'm not going to encourage you to do the crime. I'm just listening and just being curious about you. But then people make mistakes because they share, they over share sometimes without knowing. Maybe they're too tired or something. Natalia:I have a bit of a strange question. So with the lockdown, culturally, people are expressing publicly that they feel like they're over sharing. Because they're all locked indoors. They have, their only outlet is to share online. So have you noticed that in your work in security? Do, are people over sharing in that underground world as well? Or there, there hasn't been an equal shift?Peter:No, I, I, I, actually think it's getting worse. Um, and part of the reason is, as more people go online, they're speaking more about how to be anonymous. So for example, I've seen a rapid increase in BackConnect. These are residential IP addresses used as proxies. Well 'cause now they're communicating to each other, saying, hey, we're all online and this is how you can get found out. And so there actually there's more sharing going on. You know, I look at this, many more VPN services out there. It just seems, they're better prepared. Now, obviously, we see a lot more, right? So I'm definitely seeing more sophistication because people are spending more time online. So they, they're not walking around waiting for the bus. They're reading, they're learning, they're adapting. They communicate with each other. Peter:I've even found like, cyber crime as a service, we've found clusters of groups of people. And when you look at that network, you could see. They're saying, oh, I offer phishing pages or I offer VPN. They become specialized. So now you have people that are saying, I am just gonna focus on getting your, for example, some exploits. Or I'm just gonna focus on getting you, um, some red team work so that you can go and drop your ransomware. You know what, they, they've become more specialized actually because they're online. And they've got the time to learn.Nic:Peter, you mentioned earlier, some time you spent in, I think, was it the French army, is that correct?Peter:Yes, that's correct.Nic:Do you want to talk about that? Was that your foray into security? Did it, did it begin with your career in the army? Or did it begin before then?Peter:Hmm. I think it started probably before then. In a sense that, once I left high school, I decided I wanted to study law. Because I wanted the system that I was gonna be working in. And so I went to law school, uh, in the UK. And when I came out, unfortunately, the market was not as good. So I couldn't get a job. And when I looked around at what other trenches I had. I found there was an accelerated cause to become an officer in the French Army. It's a bit like, West Point in the US. Or, and so to do that, it was basically two years, it a two year program condensed into four months. It was hard. And so (laughs) I-Nic:It was what? No sleep? Is that what it was? (laughs)Peter:Ahhh. I've lived through little sleep.Nic:No sleep before meals.Peter:Yeah. I had to, you know, even- Well one time, I even had to evacuated because I got hyperten- you know, uh, hypothermia. (laughs) It was, uh, sort of a character build, character builder, I like to call it that. Uh, but really I think that started the path. Uh, but for the security side was, was after that. So, 'cause of my debts from law school, I, I left the army and I went to, back to the UK. And there, the first job I found was to be a paralegal, photocopying accounts, bank accounts opened between 1933 and 1947. It was part of something called a survey. And it actually had something to do with the Nazi gold.Peter:So what happened is that during the second world war, a lot of peop- uh, people of Jewish origin, saw that they were gonna be persecuted and took their money to, uh, Switzerland and put them in numbered accounts. And kept the number in their head. While unfortunately, so many of them sadly, uh, were victimized, they died. And the number died with them. Well, the money stayed in the accounts and over time because the accounts were dormant, well, you had charges. And so the money left. Peter:And so this was something that Paul Volcker, I believe it was, started the survey to get the Swiss banks to comply and give the money back to the families as result. So I was part of a team investigating one of the banks there. And although I started photocopying, I looked at, using my military skills, to be very efficient. So I was the best photocopier.Natalia:(laughs)Peter:And uh, and we were five levels underground. And that's what I did and I worked hard. And then after a few weeks, I got promoted to manage, uh, photocopiers. The people photocopying. We were a great team. And after that, they realized I was still hanging around because everyone was sleeping. 'Cause working five levels underground is a bit depressing sometimes. Peter:And so eventually, I became a data analyst. And so now I had to do the research on the accounts to try and find someone writing in pen, oh, this number is related to this other main account. Or this there piece of evidence is linked to this name. And so basically, for about, I think about three years, I basically, I eventually ran the French team and we looked at all the French cards opened from that period. And that started the investigations and sort of, trying to think deeper into evidence and how to make it work. Natalia:I really didn't think of myself as being cool before this, but I'm definitely not cool after hearing this. It's been validated, these stories are way beyond me. Peter:(laughs) Well, no. Just stories.Natalia:(laughs) So what brought you to Microsoft? That how did you go from piracy investigation to working at Microsoft as an investigator?Peter:So what took place was actually, my troubles created by Microsoft. So back in 2000 it was Microsoft who actually saw that the internet was becoming something that could really hurt internet commerce and e-commerce of role and wanted to make sure Peter:But they could contribute to it, and participate by building this capacity. And all the way through, they were one of my clients, at, essentially. And at some point, I realized that in my career, working for different customers, clients is great, because you learn, you don't have something different. So, for example, a software company is very different to a games company. Is different to a publishing company, is different to a mo- motion picture company, although it's digital piracy, it's actually very different in many respects. And I have- I saw how Microsoft was investing more in the cloud at that time, and I saw that as a big opportunity to really help a bigger threat to the system, right? Peter:And when I say to the system, E-commerce, 'cause everything was booming, this was in like 2008. And so, I decided that I would work for them. And actually, they offered me the job. So, I- I didn't, you know, I'm very privileged to be where I am now. But the, the, the way they positioned it is that they were looking for someone to help develop systems to map out, create a heat map of online piracy. I was like, "Wow, this is a global effort." So, uh, that's what I came on board with. And I built actually, a, a system similar to Minority Report, whereby I got basically these crawlers that I built that would go out and visit all these pirate sites. And you'll find this fascinating 'cause... Well, I found it fascinating, in some cases- Natalia:(laughs). Peter:... as we accessed the forums that we're offering, you know, download sale, RapidShare was one of the companies at the time, as we shut them down, they have crawlers in the forum, which will go and replace them. So, we had machine or machine wars, where we would shut down a URL, and then they would put another one. The problem is that our system was infinite. That is, we can, the machine can keep clicking. For them, they had about 10 groups of files. And so once they reached number 10, that was it. So, I found a way to automate the systems. And then after that using the, the Kinect, do you remember the Xbox Kinect? Nic:Cer- certainly. Peter:Managed to hack that, and the way it happened is that I built a map on Bing, whereby the Kinect could look in my body structure. And as I moved my hand, it would drill in to a country. And when I pushed, it would create, like, a, a table on the window with the number of infringements, what products were offered, when was the last time it was detected. And then, I could just wave it away and it would go, and then I could spin the world, it was a 3D map to go to another country and say, "What are the concentrations of piracy?" In this way, we had a visualized way of looking at crime as they were taking place online, and then zoom in and say, "We need to spend more effort here." Right? Peter:So, as well, just getting data analytics, but in a 3D format. And so, that was part of the excitement when I joined, is how to do that. Another example is, I found that, I read some research where it said that basically humans only spend a minute and a half on any search query. You know, in itself it doesn't mean much. But imagine you have a timer and it's one second, two seconds, three seconds, right? You're waiting for a minute and a half, right? So, 90 seconds, let's double that and say 180 seconds. Basically, let's say three minutes, it means that if you go to anyone you know, and ask them, "Go and search for Britney Spears downloads." And you look too, go, do, do the search, and they will click a link, nothing. Go next, click next, and they'll keep going. Peter:Before the three minute mark, they'll stop. They'll change the query, they'll do something different. Because they wouldn't get a result. Which means that when you do a search, and a search has got a million results, uh, it doesn't really matter. People are not going to go through the million. So, I started to think about the problems that when executives and people were saying, "Oh, I go on the internet, and I can find bad stuff." I was like, "Okay, but you can do like in three minutes. How about I build a robot that will pretend to be you, and go and find the infringements within that three minute window? Which is about 400 URLs. But I'm going to hit it with like send 100 queries, distributed." Peter:All of a sudden, we were finding the infringements before anyone could click on it, because we would report it to Google, Bing, Yandex, Baidu. And they would remove it from the, from the search results. And then, we had a measurement system, which would check and see, if I was a human, how many seconds would it take before I found an active download? Right? You could automate it. And so, we had a dashboard that could show that, and it worked. You know, we could, we saw a decline in the number of complaints because, well, it wasn't as visible. Now, if you knew where the pirate bay was, yeah, okay. But that wasn't really what we were doing. We were looking at protecting people from getting downloads which contain malware, or something nefarious, right? And, and, so we built these systems to protect consumers, essentially.Natalia:So, is there a connection, or maybe a community behind the work that you've done in piracy and the world of copyright? Uh, any, any best practices that are shared with content creators who are equally concerned with a malware being in their content, or just the sheer, the sheer fact that someone is pirating their content?Peter:I think from a contents per- perspective, and there are several amazing organizations out there, such as the BSA, Business Software Alliance, you have the MPAA, you know, you have the RIAA, and also IACC, the International Anti-Counterfeiting Coalition. Who have just incredible guidance for their members, which are specialized. So, for example, when you look at counterfeit goods, that's a very different thing to like, say, video, because video is distributed in a diff- different way. But one thing, which I think is important is that you don't just leave your, your house open, you lock it with a key, otherwise, someone will just come in and take your stuff. Peter:So, I think the same with contents, that when we create content, we have to find a way to work not only with different organizations that are looking to protect those rights, but also assume your own responsibility of locking your door. For example, what security could you put on it? Right? To maintain it? And how could you work with law enforcement who are there to protect the law, right? There are, I think there are different things that could be considered but most of it really, I would say the best is to start with the industry association, because they are much more specialized, and can give better advice, depending on the nature of the content that the person has. Peter:But, you know, when we were looking at online piracy, it wasn't just online piracy, because, you know, Microsoft participated in something called Operation Pangea. This was an Interpol driven operation where we found that a Russian organization that was distributing software for download in the millions of dollars, we took action to dismantle their payment mechanism. So, Visa and MasterCard would stop the payment on their website. So, they moved to prescription drugs, and they started selling prescription drugs. And so, for certain, it's really not in Microsoft's mandate to do that, right? Peter:But what we did is that we provided the expertise, and the knowledge we have to law enforcement to detect these websites. There were about 10,000 of them, and then drill down to say, "What's the payment gateway?" Because that's a choke point, you know, a criminal, definitely does what he does for the money. You know, you're not gonna rob a bank if there's no money there, right? So, with that in mind, they were able to do really, massively disrupt this organization. And that's because Microsoft looks at providing its expertise, and also learning from other people's expertise, right? But to tackle this bigger problem that impacts all of us.Nic:Peter, I'd love to circle back to language for a sec here. And when you were talking about the languages that you speak, and, and the importance of understanding culture. From your perspective, do you think there are countries, language groups, ethnic groups that are disproportionately... Well, I'm trying to think of the most elegant way to say, not protected or not protected as well as they could because they speak a language that is, you know, not as prevalent? So, you know, I looked at, you know, I'd never heard of the two, the two, uh, Ghanaian languages that you had on your- Peter:Mm-hmm (affirmative). Nic:... on your profile there, I'm not even gonna say them right, but Fante and- Peter:(laughs), so, it's Fante and Twi. Nic:Fante and Twi. So- Peter:Perfect. Nic:... native Fante, and Twi, I'm, I'm assuming there's, there's hundreds of thousands, maybe even millions of speakers of those- Peter:Yeah. Yes, absolutely.Nic:... two languages?Peter:Yes, yeah. Nic:Do AI and ML systems allow for supporting people that, you know, either don't speak English, or a sort of major international language?Peter:You're touching on something, which is very near and dear to me, 'cause it's a whole different conversation. And if you look at the history of language, there's, a, a great group of seminars written about it. It's actually I think, I believe, somewhere, I read somewhere that 60% of languages are actually not written. Right? And yes, you can go and see Microsoft has, translates between say, 60 or 100 pairs of languages, and Google the same. But what about the others? What about the thousands of others, that I think there are over 6,000 languages in the world. You're right. I mean, earlier this year, if I may be personal, I'm trying to adopt a baby girl. And so, I went to Ghana to try and manage the situation, which is very slow. Peter:And when I was there, I just saw the reality that, you know, they don't have access to resources, right? Because a book costs money. And so even for AI, how would they even know what AI is? So, I think there is an increasing gap, which is taking place. We can't keep build, building bigger walls, because it's just not going to work. We gotta be, we gotta think bigger than that. And so, one of the ideas is that when we look at some of the criminals, like I've had quite a few of them, a lot of them go to the same technical universities, for example, in West Africa. Well, why is that? It's cause I think they develop skills, and then they leave, and they can't get a job. And so, they end up being pulled into a life of cybercrime. So, culture Peter:It's I think becoming an important thing is that, there is a bigger and bigger divide 'cause not as many people have access to the resources, and how can we as a community who do have access, sort of proactively contribute to that? 'Cause we can't, there's no way you can, you know, just Nigeria has 190 million people. That's a lot of people, that's a lot people. The African continent has 1.2 billion. Asia, four billion, was like, um, I think it's like, is it two, three billion? No, two billion? Something like that but it's a lot people- Nic:It's a lot. Peter:... outside, right? (laughs). And so I think, I'm glad you brought that up 'cause I think it's a- an interesting conversation that we need to develop even, even more. Natalia:So, just trying to distill some of that down. So, are, are you saying then that, uh, at least when we're looking at language, there is a greater diversity of threat actors than there are targets? That those targets are centralized more around English speakers, but because of disproportionate opportunities in other parts of the world, we see threat actors across a number of different languages, across a number of different cultures? Peter:Yes. I, I think that's, that's a goo- uh, kind of a good summary of that, but I'll probably take it a step further and say, from my vantage point, again, you know, there are many other more brilliant people out there than me, I can only speak of what I've seen. I still find there are concentrations, right? When you look at business email compromise, and you go and pick up a newspaper and say, "Show me all articles about BEC, the biggest crime right now in the world, and show me all the people who've been arrested." Guess what? They're all from one place, West Africa. Why? Because if you look at the history of that crime, BEC, it was a ruse. Before that it used to be called, it was all under the category of Advanced E-fraud, but it used to be a lottery scam. Oh, the Bill and Melinda Gates lottery, you've won $25 million, or, uh, the Nigerian prince, right?Peter:Some people call 419 which is a criminal code in Nigeria. And then it went further back, they used to send faxes. Or, a lot of people developed a culture called the Yahoo boys, right? They it called Yahoo-Yahoo. And what they do is you go on YouTube, and you search for Yahoo-Yahoo, you'll see them like there's a whole culture behind that. They're dancing, they say, "This is my Monday car, my Tuesday car." And because they're making money and their communities are not, the community helps them because they get money. The stolen money is shared, and so now it becomes harder to break that because it becomes part of a culture. And so, that's why we see a lot more there I think than for example, in the US, or in Russia or in other countries it's 'cause I think there was, there's a, they have this kind of lead way that they'd be doing it for a lot longer and have a better sense of how to be sly. Nic:It sounds like the, the principles of reducing crime apply just as generally in the cyberspace as they do too in the, the non-cyber space. Whereas if you can give opportunities and lu- you know, um, lucrative opportunities to people, to utilize the skills that they've developed, both sort of in an orthodox or in an unorthodox fashion- Peter:Mm-hmm (affirmative). Nic:... then they're gonna put those skills to good use. But if you, if you train them up and then don't give them any way of using those skills to, to go, you know, ma- make a living in a, in a positive sense, they're, they're gonna turn to other, other avenues. Sounds like in, in, in parts of West Africa, that is business email compromise.Peter:Right, it is. And if I could just add two things there, one is that, you know, when I started looking at how to address cyber, online criminality, I have to look at the physical part of it. And in the physical world, there's actually, I call them neighborhoods. You have good neighborhoods, and bad neighborhoods, right? There are some neighborhoods you go to, no one's going to pick pockets you, right? Everyone's got a nice car or whatever. The other neighborhoods you go to, and there are some shady people in the corner, probably selling drugs or something. You know, uh, I'm, I'm being very simplistic, but I'm just trying to say, there are differences in neighborhoods in the physical world, and those need to be looked at as well. Because even if you gave education or a job to someone in a bad neighborhood, because of the environmental pressure, they may not be able to leave that neighborhood because they could be pressured into it. Peter:Online it's the same, I found that you see there are clusters of criminal activities that happen. And in those virtual they're interconnected, it's like, like two, or three levels, they know each other mostly. And so, we can have this kind of, we have to think more holistically, I suppose. I'm trying to say, Nic, that, it, we also have to look at the neighborhood and how do you make sure, for example, that neighborhood they have a sports field or the streets are clean because it makes you feel good, right? There's, there are other environmental factors that I think we may need to consider in a more holistic way. We, we can move much faster that way, because there are different factors, uh, which contribute to this.Nic:So, Peter, I honestly feel like we could keep chatting for the next four hours, right? Natalia:(laughs), I know. Peter:(laughs). Nic:We, we, (laughs). We, we've already, (laughs), eaten up a, a lot of your time, and we've covered a lot of ground. I'd love to circle back one final time to, to language and really sort of ask you is, eh, maybe it's not language, but is there something that you sort of feel particularly passionate about in your career at Microsoft? What you've done so far, what you're working on, and what you hope to do moving forward, is language and opening up accessibility through language, and other sort of cultural diversity? You, you, you, spoke a lot about that in the last sort of, you know, 45 minutes. Is that, is that something that you're personally, uh, invested in, and would like to work more on in the future? And, and if not, what other areas are you, are you looking forward to in the future? Peter:It's, it's absolutely something I'm, I'm very passionate about. And within Microsoft, as an example, the company has invested a lot in diversity and inclusion and equity, and it ended last year, but I was the president of the Africans in Microsoft employee resource group, for example, which has close to a thousand people. And all of it is about helping, working in a two way street, where we help our community, who are at times new in the country. And so, don't understand the cultural differences and how do we help them better, not integrate, but be themselves. And also, allow others that don't understand that they may be a minority, but there's so much richness to that diversity and how it makes teams stronger, because then you're not all looking through the same lens and you can bring in, you know, different perspectives about it. So, I'm absolutely invested in that, not just here in the US but also, you know, the African continent. Peter:And, and I'm very fortunate to be working in a company that's actually pushing me to do that. You know, the company is, is doing amazing things when it comes to diversity and inclusion. And yes, there's room to be made, but at least they're active. Going back really quickly to what you mentioned about language and AI, when we look at the internet, the internet is still zeros and ones. So, when you look at machine learning models, a lot of it is looking for like over 250 signals, right? In a, in one site. And it's not just about the language, it's about different languages, computer code and human code. And so, the machines are bringing those two together, which can help better secure platforms. Natalia:And just as we wrap up here, is there anything you want to plug? Any resources, any groups that you'd like to share with our audience? Peter:I think for me, you know, always try and keep updated on security. So, you know, the Microsoft Security Bulletin is a, is a great source for, uh, up-to-date information. Also, I think there are many other organizations that people can search for and reach out to me on the antenna. If you're not a bad guy or girl, I'll- Natalia:(laughs). Peter:... I'll share, (laughs), we, we can, um, actually, you know, I try to mentor as many people in our industry because, eh, together we become stronger. So, do reach out if you want to. Natalia:Awesome. Thank you for that, Peter. It was great having you on the show again, and I can honestly say, we'd be happy to have you back, and it was infinitely fascinating. Peter:Thank you very much for the invitation again. And, uh, it was a pleasure participating. Natalia:By the way, [foreign language 00:38:17]. Peter:Uh, there you go. Natalia:If you ever want to. Peter:(laughs). Natalia:(laughs). Peter:(laughs). Nic:Natalia, I didn't know you speak Spanish.Natalia:(laughs). Peter:(laughs). Natalia:Well, we had a great time unlocking insights into security from research to artificial intelligence, keep an eye out for our next episode. Nic:And don't forget to tweet us @msftsecurity or mail us at securityunlockedatmicrosoft.com with topics you'd like to hear on a future episode. Until then, stay safe.Natalia:Stay secure.
3/31/2021

The Human Element with Valecia Maclin

Ep. 21
For Women’s History Month, we wanted to share the stories of just a few of the amazing women who make Microsoft the powerhouse that it is. To wrap up the month, we speak with Valecia Maclin, brilliant General Engineering Manager of Customer Security & Trust, about the human element of cybersecurity.In discussion with hosts Nic Fillingham and Natalia Godyla, Valecia speaks to how she transitioned into cybersecurity after originally planning on becoming a mechanical engineer, and how she oversees her teams with a sense of humanity - from understanding that working from home brings unique challenges, to going the extra mile to ensure that no member of the team feels like an insignificant cog in a big machine - Valecia is a shining example of what leadership should look like, and maybe humanity too.In this Episode You Will Learn:• The importance of who is behind cybersecurity protocols• How Microsoft’s Engineering, Customer Security & Trust team successfully transitioned to remote work under Valecia’s leadership• Tips on being a more inclusive leader in the security spaceSome Questions that We Ask:• What excites Valecia Maclin about the future of Cybersecurity• How does a mechanical engineering background affect a GM’s role in Infosec• How Valecia Maclin, General Manager of Engineering, Customer Security & Trust, got to where she is todayResources:Valecia’s LinkedIn:https://www.linkedin.com/in/valeciamaclin/Advancing Minorities’ Interest in Engineering:https://www.amiepartnerships.org/SAFECode:https://safecode.org/Microsoft’s TEALS:https://www.microsoft.com/en-us/tealsMicrosoft’sDigiGirlz:https://www.microsoft.com/en-us/diversity/programs/digigirlz/default.aspxNic’s LinkedIn:https://www.linkedin.com/in/nicfill/Natalia’s LinkedIn:https://www.linkedin.com/in/nataliagodyla/Microsoft Security Blog:https://www.microsoft.com/security/blog/Transcript[Full transcript can be found athttps://aka.ms/SecurityUnlockedEp21]Nic Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft security engineering and operations teams. I'm Nic Fillingham. Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft security, deep dive into the newest threat intel research and data science. Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft security. Natalia Godyla:And now let's unlock the pod. Hey Nic, welcome to today's episode. How are you doing today? Nic Fillingham:Hello Natalia, I'm doing very well, thank you. And very excited for today's episode, episode 21. Joining us today on the podcast is Valecia Maclin, general manager of engineering for customer security and trust someone who we have had on the shortlist to invite onto the podcast since we began. And this is such a great time to have Valecia come and share her story and her perspective being the final episode for the month of March, where we are celebrating women's history month. So many incredible topics covered here in this conversation. Natalia, what were some of your highlights? Natalia Godyla:I really loved how she brought in her mechanical engineering background to cybersecurity. So she graduated with mechanical engineering degree and the way she described it was that she was a systems thinker. And as a mechanical engineer, she thought about how systems could fail. And now she applies that to cybersecurity and the- the lens of risk, how the systems that she tries to secure might fail in order to protect against attacks. And I just thought that that was such a cool application of a non-security domain to security. What about yourself? Nic Fillingham:Yeah. Well, I think first of all, Valencia has a- a incredibly relatable story up front for how she sort of found herself pointed in the direction of computer science and security. I think people will relate to that, but then also we spent quite a bit of time talking about the importance of the human element in cybersecurity and the work that Valecia does in her engineering organization around championing and prioritizing, um, diversity inclusion and what that means in the context of cybersecurity. Nic Fillingham:It's a very important topic. It's very timely. I think it's one that people have got a lot of questions about, like, you know, we're hearing about DNI and diversity and inclusion, what is it? What does it mean? What does it mean for cybersecurity? I think Valecia covers all of that in thi- in this conversation and her perspective is incredible. Oh, and the great news is, as you'll hear at the end, Valecia is hiring. So if you like me are inspired by this conversation, great news is actually a bunch of roles that you can go and, uh, apply for to go and work for Valecia on her team.Natalia Godyla:On with the pod?Nic Fillingham:On with the pod. Valecia Maclin, welcome to the Security Unlocked podcast. Thank you so much for your time. Valecia Maclin:Thank you, Nic and Natalia. Nic Fillingham:We'd love to start to learn a bit about you. You're, uh, the general manager of engineering for customer security and trust. Tell us what that means. Tell us about your team, us about the amazing work that you and- and the people on your team do. Valecia Maclin:I am so proud of our customer security and trust engineering team. Our role is to deliver solutions and capabilities that empower us to ensure our customers trust in our services and our products. So I have teams that build engineering capabilities for the digital crimes unit. We build compliance capabilities for our law enforcement and national security team. And our team makes sure that law enforcement agencies are in compliant with their local regulatory responsibilities and that we can meet our obligations to protect our customers. Valecia Maclin:I have another team that provides on national security solutions. We do our global transparency centers on where we can ensure that our products are what we say they are. I have two full compliance engineering teams that build capabilities to automate our compliance at scale for our Microsoft security development lifecycle, as well as, uh, things like, uh, advancing machine learning, advancing open source security, just a wealth of enterprise wide, as well as stakeholder community solutions. Um, I could go on and on. We do digital safety engineering, so a very broad set of capabilities all around the focus and the mission of making sure that the products and services that we deliver to our customers are what we intend and say that they are Nic Fillingham:Got it. And Valencia so how does your engineering org relate to some of the other larger engineering orgs at Microsoft that are building, uh, security compliance solutions?Valecia Maclin:So our other Microsoft organizations that do that are often building those capabilities within a particular product engineering group. Um, customer security and trust is actually in our corporate, external and legal affairs function. So we don't have that sales obligation. Our full-time responsibility is looking across the enterprise and delivering capabilities that meet those broad regulatory responsibility. So again, if we think about our digital crimes unit that partners with law enforcement to protect our customers around the world, well building capabilities for them or digital safety, right? If you think about the Christ church call and what happened in New Zealand, we're building capabilities to help with that in partnership with what those product groups may need to do. So, um, so we're looking at compliance more broadly. Nic Fillingham:Got it. And does your team interface with some of the engineering groups that are developing products for customers? Valecia Maclin:Absolutely. So when you think about the work that we do in the open source security space, our team is kinda that pointy end of the spear to do, um, that assessment and identify here where some areas are that we need to put some focus and then the engineering, the product engineering groups will then and build, go and build that resiliency into the systems. Nic Fillingham:To follow up questions. One is on the podcast, we've actually spoken to some- some folks that are on your team. Uh, Andrew Marshall was on an earlier episode. We spoke with Scott Christianson, we've had other members of the digital crimes unit come on and talk about that work, just a sort of a sign post for listeners of the podcast. How does Andrew's work, uh, fit in your organization? How does Scott's work fit into your organization? Valecia Maclin:So, um, both Andrew and Scott are in a team, um, within my org, uh, that's called security engineering and assurance, and they're actually able to really focus their time on that thought leadership portion. So again, if you think about the engineering groups and the product teams, they have to, you know, really focus on the resiliency of the products, what our team is doing is looking ahead to think about what new threat vectors are. So if you think about the work that Andrew does, he partnered with Harvard and- and other parts of- of Microsoft to really advance thought leadership and how we can interpret adversarial machine learning. Valecia Maclin:Um, when you think about some of our other work in our open source security space, it is let's look forward at where we need to be on the edge from a thought leadership perspective, let's prototype some capabilities operationalizes, so that it's tangible for the engineering groups that then apply and then, uh, my guys will go and partner with the engineering groups and gi- and girls, right? So- so, um, we will then go and partner with the product groups to operationalize those solutions either as a part of our security, um, development life cycle, or just a general security and assurance practices. Nic Fillingham:Got it. And I think I- I can remember if it was Scott or Andrew mentioned this, but on a previous podcast, there was a reference to, I think it's an internal tool, something called Liquid. Valecia Maclin:Liquid, yes, uh, yeah. Nic Fillingham:Is that, can you talk about that? Cause we, uh, it was hinted at in the previous episode? Valecia Maclin:Absolutely. Yes. Yeah. So Liquid, um, actually have a full team that builds and sustains Liquid. It is a, um, custom built capability that allows us to basically have sensors within our built systems. Um, and so when you think about our security development life cycle, and you think about our operational security requirements, it's given us a way to automate not only those requirements, but you know, ISO and NIST standards. Um, and then that way, with those hooks into the build systems, we can get a enterprise wide look at the compliance state of our bills as they're going on. Valecia Maclin:So a developer in a product group doesn't have to think about, am I compliant with SDL? Um, what they can do is, you know, once the- the data is looked at, we can do predictive and reactive analysis and say, hey, you know, there's critical bugs in this part of the application that haven't been burned down within 30 days. And so rath- rather than a lot of manual and testation, we can do, um, compliance a scale. And I- I just mentioned manual and testation of security requirements. Oh, one of my other teams, um, has recently just launched Valecia Maclin:.. the capability that we're super excited about that leverages what we call Coach UL or used to be called Simile. That again, is automating kind of on the other edge, right? So, with liquid, it's once we pulled in the build data. Um, we're working with the engineering groups in Microsoft now to, um, do the other edge where they don't have to set up a test that they're compliant with security requirements. Um, we're, we're moving very fast to, um, automate that on behalf of the developer, so that again, we're doing security by design. Nic Fillingham:So, how has your team had to evolve and change, uh, the way that they, they work during this sort of the COVID era, during the sort of work from home? Was your team already set up to be able to securely work remotely or were there sort of other changes you had to make on the fly? Valecia Maclin:So, you know, uh, as we've been in COVID, my team does respond to phenomenally. We were actually well positioned to work from home and continue to function from home. You know, there were some instances where from an ergonomic perspective, let's get some resources out to folks because maybe their home wasn't designed for them to be there, you know, five days a week. So, the, the technical component of doing the work, wasn't the challenge. What I, as a leader continuously emphasized, and it's what, what my team needed, frankly, is making sure we stayed with the connectedness, right?Valecia Maclin:How do we continue to make sure that folks are connected, that they don't feel isolated? That, you know, they feel visibility from their, from their managers? And consider I had, I had 10 new people start in the past year, entirely through COVID including three new college hires. So, can you imagine starting your professional-Nic Fillingham:Wow.Valecia Maclin:... career onboarding and never being in the office with your peers or colleagues and, and, you know, and the connected tissue you would typically organically have to build relationships. And so through COVID, during COVID, we've had to be very creative about building and sustaining the connective tissue of the team. Making sure that we were understanding folks, um, personal needs and creating a safe space for that. You know, I was a big advocate way back in August where I said, Hey folks, you know, 'cause the sch- I knew the school year was starting. And even though we hadn't made any statements yet about when returned to work would, you know, would advanced to, I made a statements to my team of, Hey, it's August, we've been at this for a few months. It's not going anywhere anytime soon. Valecia Maclin:So, I don't want us carrying ourselves as if we're coming back to the office tomorrow. Let's, you know, give folks some space to reconcile what this is gonna look like if they have childcare, if they have elder care, if they're just frozen from being in- indoors this amount of time. Let's make sure that we're giving each other space for that. Also during the past year, you know, certainly we had, I would say, parallel once in a generation type events, right?Valecia Maclin:So, we had COVID, but we also had, uh, increased awareness, you know, of, of the racial inequities in our country. And for me as a woman of color that's in cybersecurity, I've spent my entire career being a, a series of first, um, particularly at the executive table. And so, you know, so it was a, an opportunity we also had in the past year to advance that conversation so that we could extend one another grace, right? So I personally was touched by COVID. I, I lost five people in the past year. Um, and I was also-Nic Fillingham:I'm so sorry. Valecia Maclin:Yeah. (laughs) And you keep showing up, right? And I was personally touched as a black woman who once again, has to be concerned about, you know, I have, uh, I have twin nephews that are 19, one's autistic and the other is not, but we won't allow him to get a driver's license yet 'cause he, my, my sister's petrified because, you know, that's a real fear that a young man who's 6'1", sweetest thing you would ever see, soft-spoken, um, but he's 6'1". He has, you know, dreadlocks in his hair or locks. He would hate to hear me say they were dreads. He has locks in his hair. Um, and he dresses like a 19 year old boy, right?Valecia Maclin:But on spot, that's not what the world sees. And so, um, that's what we're all in. Then you think about what's happening now with our Asian-American community. That's also bundled with folks who are human, having to be isolated and endorse, which that's not how humanity was designed. And so we have to remember that that shows up. And, and when you're in, in the work of security, where you're always thinking about threat actors, and I often say that some of our best security folks have kind of some orthogonal thinking that's necessary to kind of deal with the different nuances.Valecia Maclin:When you, when you are thinking about how do you build resiliency against ever evolving threats, (laughs) not withstanding the really massive one that, you know, was the next one we, we dealt with at the end of the last calendar year. Those are all things that work in the circle. And I always say that people build systems, they don't build themselves. And in this time more than ever, hopefully, as security professionals, we're remembering the human element. And we're remembering that the work that we do, um, has purpose, which is, you know, why I entered this space in, in the first and why I've spent my career doing the things I've done is because we have a phenomenal responsibility increasingly in a time of interconnectedness from a technology perspective to secure our way of life. Nic Fillingham:Wow. Well, on, on that note, you talked about sort of why you went into security. I'd love to sort of, I'd love to go there. Would you mind talking us through how you sort of first learnt of security and, and why you're excited about it, and how you made the decision to, to go into that space? Valecia Maclin:Absolutely. So, mine actually started quite awhile ago. I was majoring in mechanical engineering and material science, uh, at Duke university. I was in my junior year and, um, I should preface it with, I did my four year engineering degree in three and a half years. So, my, my junior year was pretty intense. I worked, was working on a project for mechanical engineering that I'd spent about seven hours on and I lost my data.Nic Fillingham:Ah!Valecia Maclin:I was building a model, literally, I sat at the computer because, you know, you know, back then, you know, there weren't a whole lot of computer resources, so you try to get there early and, and, and snag the computer so that you could use it as long as you needed to. I went in actually, on a holiday because I knew everybody would be gone. So, if I, I could have the full day and not have to give up the computer to someone. So, I'd spend seven hours building this model and it disappeared. Valecia Maclin:And it was the, you know, little five in a 10 floppy, I'm pulling it out, I'm looking at the box (laughs). It's gone. The, the, the model's gone. I was gonna have to start all over. I started my homework over again, but then I said, I will never lose a homework assignment like that again. So, I went and found a professor in the computer science school to agree to do an independent study with me, because as a junior, no one was gonna allow me to change my major for mechanical engineering that far in, at Duke University. So, (laughs) not, not my parents, anyway. So, I, um, did an independent study in computer science and taught myself programming. So, I taught myself programming, taught myself how to understand the hardware with, with my professors help, of course. But it was the work I did with that independent study that actually led to the job I was hired into when I graduated. Valecia Maclin:So, I've never worked as a mechanical engineer. I immediately went into doing national security work, um, where I worked for companies that were in the defense industrial base for the United States. And so I, I started and spent my entire career building large scale information systems for, you know, the DOD, for the intelligence community, and that vectored into my main focus on large, um, security systems that I was developing, or managing, or leading solutions through. So, it started with loss data, right? (laughs) You know, which is so apropos for where we are today, but it started with, you know, losing data on a software, in a software application and me just being so frustrated Valecia Maclin:Straight and said, that's never gonna happen to me again (laughs) that, um, that led me to pursue work in this space. Natalia Godyla:How did your degree in mechanical engineering inform your understanding of InfoSec? As you were studying InfoSec, did you feel like you were bringing in some of that knowledge? Valecia Maclin:One of the beautiful things and that was interesting is I would take on new roles, I'll, I'll never forget. Um, I, I got wonderful opportunities as, as my career was launched and folks would ask me, well, why are you gonna go do that job? You've never done that before, you know, do you know it? (laughs) And so what that taught me is, you know, you don't have to know everything about it going in, you just need to know how to address the problem, right? So, I consider myself a systems thinker, and that's what my mechanical engineering, um, background provided was look at the whole system, right? And so how do you approach the problem? And also because I also had a material science component, we studied failures a lot. So, material failure, how that affected infrastructure, you know, when a bridge collapse or, or starts to isolate. Um, so it was that taking a systems view and then drilling down into the details to predictively, identify failures and then build resiliency to not have those things happen again. Is that kind of that, that level of thinking that played into when I went into InfoSec. Natalia Godyla:That sounds incredibly fitting. So, what excites you today about InfoSec or, or how has your focus in InfoSec changed over time? What passions have you been following? Valecia Maclin:So, for me, it's the fact that it's always going to evolve, right? And so, you know, obviously the breaches make the headlines, but I'm one, we should never be surprised by breaches, just like we shouldn't be surprised by car thefts or home invasions, or, you know, think about the level of insurance, and infrastructure, and technology, and tools and habits (laughs) that we've, uh, we've developed over time for basic emergency response just for our homes or our life, right? Valecia Maclin:So, for me, it's just part of the evolution that we have, that there's always gonna be something new and there's always gonna be that actor that's gonna look to take a shortcut, that's gonna look to take something from someone else. And so in that regard, it is staying on the authence of building resiliency to protect our way of life. And so I, I am always passionate and again, it's, it's likely how I, you know, spent almost, you know, over 27 years of my career is protecting our way of life. But protecting it in a way where for your everyday citizen, they don't have to go and get the degree in computer science, right? Valecia Maclin:That they can have confidence in the services and the, the things that they rely on. They can have confidence that their car system's gonna break, that the brakes are gonna hit, you know, activate when they hit it. That's the place I wanna see us get to as it relates to the dependency we now have on our computer systems, and in our internet connected devices and, and IOT and that sort of thing. So, that's what makes me passionate. Today it may look like multi-factored authentication and, you know, zero trust networks, but tomorrow is gonna look like something completely different. And what I, where I'd love to see us get is, you know, think about your car. We don't freak out about the new technologies that show up in our car, you know, 'cause we know how, we, we, we get in and we drive and, and we anxiously await some people.Valecia Maclin:I, I'm kind of a control freak, I wanna still drive my car. I don't want it to drive itself (laughter). Um, but nevertheless, with each, you know, generational evolution of the car, we didn't freak out and say, Oh my gosh, it's doing this now. If we can start to get there to where there's trust and confidence. And, and that's why I love, you know, what my org is responsible for doing is, you know, that there's trust and confidence that when Microsoft, when you have a Microsoft product or service, you, you, you can trust that it's doing what you intend for it to do. And, and that's not just for here, but then, you know, when you're again, whether it's the car, or your refrigerator, or your television, that's where I'd love to, that's where I want to see us continue to evolve. Not only in the capabilities we deliver, but as a society, how we expect to interact with them. Natalia Godyla:Are you particularly proud of any projects that you've run or been part of in your career? Valecia Maclin:I am. And it's actually what led me to Microsoft, I had my greatest career success, but it, it came also at, at a time of, of, of my greatest personal loss. Literally they were concurrent on top of each other. And so I was responsible, I was the, the business executive responsible for the cybersecurity version of, of, of the JEDI program. Uh, so I was the business executive architecting our response to that work that was what the department of Homeland Security. I worked for a company that at the time wasn't known for cybersecurity, and so it was a monumental undertaking to get that responsibility. And the role was to take over and then modernize the cybersecurity re- system responsible for protecting the .gov domain. So, it was tremendously rewarding, especially in the optic that we have today. I received the highest award that my prior company gives to an individual. Valecia Maclin:I was super proud of the team that I was able to lead and, and keep together during all the nuances of stop, start, stop, start that government contracting, um, does when there's protests. But during that same time, you know, 'cause it was, so it was one of those once in a career type opportunities, if you've ever done national security work, to actually usher an anchor in a brand new mission is how we would label it, um, that you would be delivering for the government. But at the same time, that, that wonderfully challenging both technically and from a business perspective scenario was going on, I, in successive moments, lost my last grandparent, suddenly lost my sister. 12 months later, suddenly lost my mother, six months later had to have major surgery. So, that all came in succession while I was doing this major once in a career initiative that was a large cyber security program to protect our government. Valecia Maclin:And I, I survived, (laughs) right? So, um, the, the program started and did well, but I, I then kind of took a step back, right? Once I, I, uh, I'd promised the company at the time of the government that I would, I would give it a year, right? I would make sure the program transitioned since we'd worked so hard to get there. And then I took a step back and said, Hmm, what do I really wanna do? This was a lot (laughs). And so I did take a step back and got a call from Microsoft, actually, um, amongst some other companies. Uh, I thought it was gonna take a break, but clearly, um, others had, had different ideas. And so, um, (laughter) I had, I had multiple opportunities presented to me, but what was so intriguing and, and what drew me to Microsoft was first of all, the values of the company. You know, I'm a values driven person and the values, um mean a lot and I'm gonna come back to that in a moment. Valecia Maclin:But then also I, I mentioned that the org I lead is in corporate external and legal affairs. It's not within the product group. It's looking at our global obligations to securing our products and services from a, not just a regulatory perspective, but not limited by our, our sales target. And so the ability to be strategic in that way is what was intriguing and what, what drew me. When you think about the commitments the company has made to its employees and to its vendors during a time, um, that we've been in, it says a lot about the fabric of, of who we are to take that fear of employability insurance and those sorts of things that are basic human needs, to recall how early on we still had our cafeteria services going so that they could then go and provide meals for, for students who would typically get school meals. And at the same Valecia Maclin:... time it meant that those vendors that provide food services could continue to do their work. When you think about our response to the racial inequity and, and justice, social justice initiative, and the commitments were not only, not only made, but our, our keeping is the fabric of the company and the ability to do the work that I'm passionate about, that, that drew me here. Nic Fillingham:You talked about bringing the human element to security. What does that mean to you and how have you tried to bring that sort of culturally into your organization and, and, and beyond?Valecia Maclin:So, if you think about the human element of security, the operative word is human. And so as humans, we are a kaleidoscope of gender, and colors, and nationalities and experiences. Even if you were in the same town, you have a completely different experience that you can bring to bear. So, when I think about how I introduce, um, diversity, equity and inclusion in the organization that I lead, it is making sure that we're more representative of who we are as humans. And sometimes walking around Redmond, that you don't always get that, but it's the, you know, I, I come from the East Coast. So, you know, one of the going phrases I would use a lot is, I'm not a Pacific Northwestner or I don't have this passive aggressiveness down, I'm pretty direct (laughs). And so that's a different approach, right, to how we do our work, how we lean in, how we ask questions. Valecia Maclin:And so I am incredibly passionate about increasing the opportunities and roles for women and underrepresented minorities, underrepresented, uh, minorities in cybersecurity. And so we've been very focused on, you know, not just looking at internal folks that we may have worked on, worked on another team, you know, for years, and making sure that every opportunity in my organization is always opened up both internally and externally. They're always opened up to make sure that we're, we're looking beyond our mirror image to, um, hire staff. And it's powerful having people that think the same way you do, because you can coalesce very quickly. But the flip side of that is sometimes you can lose some innovation because everybody's seeing the same thing you see. And, and it's so important in, in security because we're talking about our threat actors typically having human element, is making sure that we can understand multiple voices and multiple experiences as we're designing solutions, and as we're thinking about what the threats may be. Natalia Godyla:So, for women or, uh, members of minority groups, what guidance do you have for them if they're not feeling empowered right now in security, if they don't know how to network, how to find leaders like yourself, who are supporting DNI? Valecia Maclin:One of the things I always encourage folks to do, and, and I mentor a lot is, just be passionate about who you are and what you contribute. But what I would say, uh, Natalia, is for them to take chances, not be afraid to fail, not be afraid to approach people you don't know, um, something that I got comfortable with very early as if I was somewhere and heard a leader speak on stage somewhere, or I was, uh, you know, I saw someone on a panel internally or externally, I would go up to them afterwards and introduce myself and ask, you know, would you be willing to have a career discussion with me? Can I get 30 minutes on your calendar? And so that was just kind of a normal part of my rhythm, which allowed me to be very comfortable, getting to meet new executive leaders and share about myself and more importantly, hear about their journeys. Valecia Maclin:And the more you hear about other's journey, you can help cultivate a script for your own. And so, so that's what I often encourage 'cause a lot of times folks are apr- afraid, particularly women and, and minorities are afraid to approach to say, think, well, you know, I don't know enough, or I don't know what to ask. It can be as simple as, I heard you speak, I would love to hear more about your story. Do you have time? Do you have 20 minutes? And then let, you know, relationships start from there and let the learning start from there. Nic Fillingham:As a leader in the security space, as a leader at Microsoft, what are you excited about for the future? What what's sort of coming in terms of, you know, it could be cultural change, it could be technology innovation. What, what are you sort of looking and seeing in the next three, five, 10 years? Valecia Maclin:For me it the cultural change. I'm looking forward and you heard me kind of allude to a little bit of this of, you now have the public increasingly aware of what happens when there's data loss. I'm so excited to look forward to that moment when that narrative shifts and the public learns and knows more of security hygiene, cyber security hygiene. And, and not, you know, both consumer and enterprise, because we take for granted that enper- enterprises have nailed this. And, and we're in a unique footing as a company to have it more part of our DNA, but not every company does. And so that's what I'm looking forward to for the future is the culture of that young person in the midst of schooling, not having to guess about what a cybersecurity or security professional is, much like they don't guess what a lawyer or a doctor is, right? So, that's what I look forward to for the future. Nic Fillingham:Any organizations, groups that you, you know, personally support or fans of that you'd also like to plug? Valecia Maclin:Sure. So, I actually support a, a number of organizations. I support an organization called Advancing Minorities in Engineering, which works directly with historically black colleges and universities to not only increase their learning, but also create opportunities to extend the representation in security. I also am a board member of Safe Code, which is also focused on advancing security, design, hygiene across enterprises, small midsize and large businesses. And so, so those are, are certainly, uh, a couple of, of organizations that, you know, I dedicate time to.Valecia Maclin:I would just encourage folks, you know, we have TEALS, we have DigiGirlz. everyone has a role to play to help expand the perception of what we do in the security space. We're not monolithic. The beauty of us as a people is that we can bring our differences together to do some of the most phenomenal, innovative things. And so that would be my ask is in, whatever way fits for where someone is, that they reach out to someone and make that connection. I v- I very often will reach down and, uh, I'll have someone, you know, a couple levels down and say, Oh my gosh, I can't believe you called and asked for a one-on-one. Valecia Maclin:So, I don't wait for folks to ask for a one-on-one with me. I, I'll go and ping and just, you know, pick someone and say, Hey, you know, I wanna, I just wanna touch base with you and see how you're doing and see what you're thinking about with your career. All of us can do that with someone else and help people feel connected and seen. Natalia Godyla:And just to wrap here, are you hiring, are there any resources that you want to plug or share with our audience, might be interested in continuing down some of these topics? Valecia Maclin:Absolutely. Thank you so much. Um, so I am hiring, hiring data architects, 'cause you can imagine that we deal with high volumes of data. I'm hiring software engineers, I'm hiring, uh, a data scientist. So, um, data, data, and more data, right?Natalia Godyla:(laughs).Valecia Maclin:And, um, and software engineers that are inquisitive to figure out the, the right ways for us to, you know, make the best use of it. Natalia Godyla:Awesome. Well, thank [crosstalk 00:35:11] you for that. And thank you for joining us today, Valecia.Valecia Maclin:Thank you, Natalia. Thank you, Nic. I really enjoyed it.Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us @msftsecurity or email us at securityunlocked@microsoft.com with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure.
3/24/2021

Identity Threats, Tokens, and Tacos

Ep. 20
Every day there are literally billions of authentications across Microsoft – whether it’s someone checking their email, logging onto their Xbox, or hopping into a Teams call – and while there are tools like Multi-Factor Authentication in place to ensure the person behind the keyboard is the actual owner of the account, cyber-criminals can still manipulate systems. Catching one of these instances should be like catching the smallest needle in the largest haystack, but with the algorithms put into place by the Identity Security team at Microsoft, that haystack becomes much smaller, and that needle, much larger.On today’s episode, hostsNic Fillingham and NataliaGodyla invite back Maria Puertos Calvo, theLeadDataScientistin Identity Security and Protection at Microsoft,to talk with us about how her team monitors such amassive scale of authentications on any given day.Theyalsolookdeeper into Maria’s background and find out what got her into the field of security analytics andA.I. in the first place, and how her past in academiahelpedthattrajectory.In this Episode You Will Learn:• How the Identity Security team uses AI to authenticate billions of logins across Microsoft• Why Fingerprints are fallible security tools• How machine learning infrastructure has changed over the past couple of decades at MicrosoftSome Questions that We Ask:• Is the sheer scale of authentications throughout Microsoft a dream come true or a nightmare for a data analyst?• Do today’s threat-detection models share common threads with the threat-detection of previous decades?• How does someone become Microsoft’s Lead Data Scientist for Identity Security and Protection?Resources:#IdentityJobs at Microsoft:https://careers.microsoft.com/us/en/search-results?keywords=%23identityjobsMaria’s First Appearance on Security Unlocked, Tackling Identity Threats with A.I.: https://aka.ms/SecurityUnlockedEp08Maria’s Linkedin: https://www.linkedin.com/in/mariapuertas/Nic’s LinkedIn:https://www.linkedin.com/in/nicfill/Natalia’s LinkedIn:https://www.linkedin.com/in/nataliagodyla/Microsoft Security Blog:https://www.microsoft.com/security/blog/Transcript[Full transcript can be found at https://aka.ms/SecurityUnlockedEp20]Nic Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft security engineering and operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft security, deep dive into the newest threat intel, research, and data science. Nic Fillingham:And profile some of the fascinating people working on Artificial Intelligence in Microsoft security. Natalia Godyla:And now, let's unlock the pod.Nic Fillingham:Hello, Natalia. Welcome to episode 20 of Security Unlocked. This is, uh, an interesting episode. People may notice that your voice is absent from the... This interview that we had with Maria Puertos Calvo. How, how you doing? You okay? You feeling better?Natalia Godyla:I am, thank you. I'm feeling much better, though I am bummed I missed this conversation with Maria. I had so much fun talking with her in episode eight about tackling identity threats with AI. I'm sure this was equally as good. So, give me the scoop. What did you and Maria talk about?Nic Fillingham:It was a great conversation. So, you know, this is our 20th episode, which is kind of crazy, of Security Unlocked, and we get... We're getting some great feedback from listeners. Please, send us more, we want to hear your thoughts on the... On the podcast. But there've been a number of episodes where people contact us afterwards on Twitter or an email and say, "Hey, that guest was amazing," you know, "I wanna hear more." And Maria was, was definitely one of those guests who we got feedback that they'd love for us to invite them back and learn more about their story. So, Maria is on the podcast today to tell us about her journey into security and then her path to Microsoft. I won't give much away, but I will say that, if you're studying and you're considering a path into cyber security, or you're considering a path into data science, I think you're gonna really enjoy Maria's story, how she sort of walks through her academia and then her time into Microsoft. We talk about koalas and we talk about the perfect taco.Natalia Godyla:Yeah, to pair with the guac which she covered the first time around. Now tacos. I feel like we're building a meal here. I'm kind of digging the idea of a Security Unlocked recipe book. I, I think we need some kind of mocktail or cocktail to pair with this.Nic Fillingham:Yeah, I do think two recipes might not be enough to qualify for a recipe book. Natalia Godyla:Yeah, I mean, I'm feeling ambitious. I think... I think we could get more recipes, fill out a book. But with that, I, I cannot wait to hear Maria's episode. So, on with the pod?Nic Fillingham:On with the pod.Nic Fillingham: Maria Puertos Calvo, welcome back to the Security Unlocked podcast. How are you doing?Maria Puertos Calvo:Hi, I'm doing great, Nic. Thank you so much for having me back. I am super flattered you guys, like, invited me for the second time.Nic Fillingham:Yeah, well, thank you very much for coming back. The episode that we, we, we first met you on the podcast was episode eight which we called Tackling Identity Threats With AI, which was a really, really popular episode. We got great feedback from listeners and we thought, uh, let's, let's bring you back and hear a bit more about your, your own story, about how you got into security, how you got into identity, how you got into AI. And then sort of how you found your way to Microsoft. Nic Fillingham:But since we last spoke, I want to get the timeline right. Did you have twins in that period of time or had the twins already happened when we spoke to you in episode eight?Maria Puertos Calvo:(laughs) No, the twins had already happened. They-Nic Fillingham:Got it.Maria Puertos Calvo:I think it's been a few months. But they're, they are nine, nine months old now. Yeah.Nic Fillingham:Nine months old. And, and the other interesting thing is you're now in Spain.Maria Puertos Calvo:Yes.Nic Fillingham:When we spoke to you last, you were in the Redmond area or is that right?Maria Puertos Calvo:Yes, yes. The... Last time when we, we spoke, I, I was in Seattle. But I was about to make this, like, big trip across the world to come to Spain and, and the reason was, actually, you know, that the twins hadn't met my family. I am originally from Spain, and, and my whole family is, is here. And, you know, because of COVID and everything that happened, they weren't able to travel to the US to see us when they were born. So, my husband and I decided to just, like, you know, do a trip and take them. And, and we're staying here for a few months now. Nic Fillingham:That's awesome. I've been to Madrid and I've been to... I think I've only been to Madrid actually. Where, where... Are you in that area? What part of Spain are you in?Maria Puertos Calvo:Yes, yes. I'm in Madrid. I'm in Madrid. I, I'm from Madrid.Nic Fillingham:Aw- awesome. Beautiful city. I love it. So, obviously, we met you in episode eight, but if you could give us, uh, a little sort of mini reintroduction to who you are, what's your job at Microsoft, what does your... What does your day-to-day look like, that'd be great.Maria Puertos Calvo:Yeah. So, I am the lead data scientist in identity secure and protection, identity security team who... We are in charge of making sure that all of the users who use, uh, Microsoft identity services, either Azure Active Directory or Microsoft account, are safe and protected from malicious, you know, uh, cyber criminals. So, so, my team builds the algorithms and detections that are then put into, uh, protections. Like, for example, we build machine learning for risk based authentication. So, if we... If our models think an authentication is, is probably compromised, then maybe that authentication is challenged with MFA or blocked depending on the configuration of the tenet, et cetera. Maria Puertos Calvo:So, my team's day-to-day activities are, you know, uh, uh, building new detections using new data sets across Microsoft. We have so much data between, you know, logs and APIs and interactions b- between all of our customers with Microsoft systems. Uh, so, so, we analyze the data and, and we build models, uh, apply AI machine learning to detect those bad activities in the ecosystem. It could be, you know, an account compromised a sign-in that looks suspicious, but also fraud. Let's say, like, somebody, uh, creates millions of spammy email addresses with Microsoft account, for example to do bad things to the ecosystem, we're also in charge of detecting that.Nic Fillingham:Got it. So, every time I log in, or every time I authenticate with either my Azure Active Directory account for work or my personal Microsoft account, that authentication, uh, event flows through a set of systems and potentially a set of models that your team owns. And then if they're... And if that authentication is sort of deemed legitimate, I'm on my way to the service that I'm accessing. And if it's deemed not legitimate, it can go for a challenge through MFA or it'll be blocked? Did, did I get that right?Maria Puertos Calvo:You got that absolutely right.Nic Fillingham:So, that means... And I think we might've talked about this on the last podcast, but I still... I... As a long-term employee of Microsoft, I still get floored by the, the sheer scale of all this. So, there's... I mean, there's hundreds of millions of Microsoft account users, because that's the consumer service. So, that's gonna be everything from X-Box and Hotmail and Outlook.com and using the Bing website. So, that's, that's literally in the hundreds of millions realm. Is it... Is it a billion or is it... Is it just hundreds of millions?Maria Puertos Calvo:It depends on how you count them. Uh, if it's per day, it's hundreds of millions, per month I think it's close to a billion. Yes, for... Of users. But the number of authentications overall is much higher, 'cause, you know, the users are authenticating in s- in s- many cases, many, many times a day. A lot of what we evaluate is not only, like, your username and password authentications, there's also the, you know, the model authe- authentication particles that have your tokens cash in the application and those come back for request for access. So, the... We evaluate those as well. Maria Puertos Calvo:So, it's, uh... It's actually tens of billions of authentications a day for both the Microsoft account system and the Azure Active Directory system. Azure Active Directory is also a... Really big, uh, it's almost... It's, it's getting really close to Microsoft account in terms of monthly, monthly active users. And actually, this year, with, you know, COVID, and everybody, you know, the... All the schools, uh, going remote and so many people going to work from home, we have seen a huge increase in, in, in monthly active users for Azure Active Directory as well.Nic Fillingham:And do you treat those two systems separately? Uh, or, or are they essentially the same? It's the same anomaly detection and it's the same sort of models that you'd use to score and determine if a... If an authentication attempt is, is, uh, is legitimate or, or otherwise?Maria Puertos Calvo:It's, like, theoretically the same. You know, like, we, we use the same methodology. But then there are different... The, the two systems are different. They live in different places with different architectures. The data that is logged i- is different. So, these, these were initially not, you know... I- identity only, uh, took care of those two systems, like, a few years ago, before they w- used to be owned by different teams. So, the architecture underneath is still different. So, we still have to build different models and maintain them differently and, you know, uh, uh, tune them differently. So, so it is more work, but, uh, the, the theory and the idea, their... How we built them is, is very similar.Nic Fillingham:Are there some sort of trends that have, you know, appeared, having these two massive, massive systems sort of running in parallel but with the same sort of approach? What kind of behaviors or what kind of anomalies do you see detected in one versus the other? Do they sort of function sort of s- similar? Like, similar enough? Or do you see some sort of very different anomalies that appear in one system and, and not another.Maria Puertos Calvo:They're, interestingly, pretty different. Uh, when we see attack spikes and things like that, they don't always reflect one or the other. I think the, the motivation of the people that attack enterprises and organizations, it's, it's definitely from the, the hackers that are attacking consumer accounts. I think they're, you know, they're so in the black market separately, and they're priced separately, you know, and, and differently. And I think they're, they're generally used for different purposes. We see sometimes spikes in correlation, but, but not that much.Nic Fillingham:Before we sort of, uh, jump in to, to your personal story into security, into Microsoft, into, into data science, is the... You know, these... Talking about these sheer numbers, talking about the hundreds of millions of, of authentications, I think you said, like, tens of billions that are happening every day. Is that a dream for a data scientist to just have such a massive volume of data and signals at your fingertips that you can use to go and build models, train models, refine models? Is that, you know... Is this adage of more signal equals better, does that apply? Or at some point do you now have challenges of too much signal and you're now working on a different set of problems?Maria Puertos Calvo:That's a great question. It is an absolute dream and it's also a nightmare. (laughs) So, yeah. It is... It... And I'll tell you why for both, right? Like, a... It is a great dream. Like, obviously, you bet... The, the sheer scale of the data, the, you know, the, the fact... There are a lot of things that are easier, because sometimes when you're working with data and statistics, you have to do a lot of things to estimate if, Maria Puertos Calvo:... it's like the things that you're competing are statistically significant, right? Like, do I have enough data to approach that this sample, it's going to be, uh, reflection of reality, and things like that. With the amount of data that we have, with the amount of users that we have, it's the, we don't have that, we, we don't really have that problem, right? Like we are able to observe, you know, the whole rollout without having to, to figure out if what we're seeing, you know, it's similar to the whole world or not. Maria Puertos Calvo:So that's really cool. Also, because we're, you know, have so many users, then we also have, you know, we're a big focus for attackers. So, so we can see everything, you know, that happens in, in, in the cybersecurity world and like the adversary wall, we can find it in, in our data. And, and that is really interesting. Right. It's, it's really cool. Nic Fillingham:That sounds fascinating. But let, let, let's table that for a second. 'Cause I'd love to sort of go back in time and I'd love to learn about your journey into security, into sort of computer science, into tech, where did it all start? So you grew up in Madrid, is that right? Maria Puertos Calvo:Yes. I grew up in Madrid and when I was finishing high school and I was trying to figure out like, why do I do, I just decided to study telecommunication engineering, it's what's called a Spain, but it's ev- you know, the, the equivalent who asked degrees electrical engineering. Because I was actually, you know, really, really interested in math and science and physics. They were like my favorite subjects in high school. I was pretty, really good at it actually. Maria Puertos Calvo:And, but at the same time, I was like, well, this, you know, an engineering degree sounds like something that I could apply all of this to. And the one that seems like the coolest and the future and like I, I, is electrical engineering. Like I, at that time, computer science was also kind of like my second choice, but I knew that in electrical engineering, I could also learn a lot of computer science. Maria Puertos Calvo:It w- it has like a curriculum that includes a lot of computer science, but also you learn about communication theory and, you know, things like how do cell phones work? And how does television work? And you can learn about computer vision and image processing and all, all kinds of signal processing. I just found it fascinating. Maria Puertos Calvo:So, so I, I started that in college and then when I finished college, it was 2010. So it was right in the middle of the great recession, which actually hits Spain really, really, really badly when it came to the, the labor market, the unemployment back then, I think it was something like 25%-Nic Fillingham:Wow.Maria Puertos Calvo:... and people who were getting out of school, even in engineering degrees, which were traditionally degrees that would have, you know, great opportunities. They were not really getting good jobs. People, only consulting firms were hiring them, um, and, and really paying really, really little money. It was actually pretty kind of a shame. So I said, what, what, what should I do? And I, I had been a good student during college, so, and I had a professor that, you know, he, that I had done my kind of thesis with him and his research group. Maria Puertos Calvo:And he said, "Hey, why didn't you just like, continue studying? Like, you can actually go for your PhD and, because you have really good grades, I'm sure you can just get it full of finance. You can get a scholarship that will like finance, you know, four years of PhD. And you know, that way you don't have to pay for your studies, but also you kind of like, you're like a researcher and you have, uh, like money to live." And I was like, well, that sounds like a really good plan.Nic Fillingham:Sounds good.Maria Puertos Calvo:Like I actually, yeah. So, so I could do in that. And, and I, you know, then my master said, this masters say, wasn't computer science, but it was very pick and choose, right? Like, like you could pick your branch and what classes you took. And so the master's was the first half of the PhD was basically getting all your PhD qualifying courses, which also are equivalent to, to doing your masters. Maria Puertos Calvo:So I picked kind of like the artificial intelligence type branch, which had a lot of, you know, classes on machine learning and learn a lot of things that are apply that are user apply machine learning, it's like, uh, natural language processing and speech and speaker recognition and biometrics and computer vision. Basically, all kinds of fields of artificial intelligence, where, where in the courses that I took. And, and I really, really fou- found it fascinating. There wasn't, you know, a data science degree back then, like now everybody has a data science degree, but this is like 10 years ago. Uh, at least, you know, in Spain, there wasn't a data science degree.Maria Puertos Calvo:But this is like the closest thing, uh, that, and that was my first contact with, uh, you know, artificial intelligence and machine learning. And I, I loved it. And, and then I did my masters thesis on, uh, kind of like, uh, biometrics in, in terms of applying statistical models to forensic fingerprints to, to understand if a person can be falsely, let's say, accused of a crime because their fingerprint brand only matches a fingerprint that is found in a crime scene. Maria Puertos Calvo:So kind of try to figure out like, how likely is that. Because there have been people in the past that having wrongly convicted, uh, because of their fingerprints have been found in a crime scene. And then after the fact they have found the right person and then, you know, like, uh, it's not a very scientific method, what is followed right now. So that, that was a really cool thing too, that then I never did anything related to that in my life, but, but it was a very cool thing to study when I was in, in school. Nic Fillingham:Well, that, that's fair. I've, I've got some questions about that. That's fascinating. So how did you even stumble upon that as a, as a, as a, as a research focus? Was there a, a particular case you might've read in the, in the news or something like, I, I think I've never heard of people being falsely accused or convicted through having the same fingerprints, I guess, unless you're an identical twin. Maria Puertos Calvo:Mm-hmm (affirmative). (laughs) Actually, I can tell you because I have identical twins, but also that, because I studied a lot of our fingerprints is that identical twins do not have the same fingerprints.Nic Fillingham:Wow.Maria Puertos Calvo:Uh, because fingerprints are formed when you're in the womb. So they're not, they're not like a genetic thing. They happen kind of like, as a random pattern when, when your body is forming in the womb, and they happen, they're different. Uh, so, so humans have unique fingerprints and that's true, but the problem with the, the finger frame recognition is that, it's very partial, and is very imperfect because the, the late latent, it's called the latent fingerprint, the one that is found in a crime scene is then recovered, you know, using like some powder, and it's kind of like, you, you just found some, you know, sweaty thing and a surface, and then you have to lift that from there. Right. Maria Puertos Calvo:And, and that has imperfections in, and it only, it's not going to be like a full fingerprint. You're going to have a partial fingerprint. And then, then you, basically, the way the matching works is using this like little poin- points and, and bifurcations of the riches that exist in your fingerprint. And, and then, you know, looking at the, the location and direction of those, then they're matched with other fingerprints to understand if they're the same one or not. But the, because you don't have the full picture, it is possible that you make a mistake. Maria Puertos Calvo:The one case that it's been kind of really, really famous actually happened with the Madrid bombings that happened in 2004, where, you know, they, they blew up, uh, some trains and, and a couple of hundred people died. Then they, they actually found a fingerprint in one of the, I don't remember, like in the crime scene and it actually match in the FBI fingerprint database. It matched the fingerprint of a lawyer from Portland, Oregon, I believe it's what it was. And then he was initially, you know, uh, I don't know if you ended up being convicted, but, but you know, it wasn't-Nic Fillingham:He was a suspect.Maria Puertos Calvo:... it was a really famous case. Yes. I think he was initially convicted. And then, but then he was not after they found the right person and they, they actually found that yeah, both fingerprints, like the, the guy whose fingerprint it really was. And these other guys, they, their fingerprints both match the crime scene fingerprint, but that's only because it was only a piece of it. Right. You, you don't put your finger, like, you don't roll it left to right. Like when you arrive at the airport, right. That they make you roll your finger, and lay have the whole thing it's, you're maybe just, you know, the, the, the criminal fingerprint is, is very small.Nic Fillingham:Was that a big part of the, the research was trying to understand how much of a fingerprint is necessary for a sort of statistically relevant or sort of accurate determination that it belongs to, to the, to the right person?Maria Puertos Calvo:Yeah. So the results of the research they'd have some outcome around, like, depending on how many of those points that are used for identification, which are called minutia, depending on how, how many of those are available, it changes the probability of a random match with a random person, basically. So the more points you have, the less likely it is that will happen. Nic Fillingham:The one thing, like, as, as we're talking about this, that I sort of half remember from maybe being a kid, I don't know, growing up in Australia is don't koalas have fingerprints that are the same as humans. Did I make that up? Do you know anything about this? Maria Puertos Calvo:(laughs) I'm sure, I have no idea. (laughs) I have never heard such a thing. Nic Fillingham:I have a-Maria Puertos Calvo:Now I wanna know. Nic Fillingham:...I'm gonna have to look this up.Maria Puertos Calvo:Yeah.Nic Fillingham:I have a feeling that koa- koalas, (laughs) have fingerprints that are either very close to or indistinguishable from, from humans. I'm gonna look this one up. Maria Puertos Calvo:I wonder if like a koala could ever be wrongly convicted of a crime. Nic Fillingham:Right, right. So like, if I want to go rob a bank in Australia, all I need to do is like, bring a koala with me and leave the koala in the bank after I've successfully exited the bank with all the gold bars in my backpack. And then the police would show up and they arrest the koala and they'd get the fingerprints and they go, well, it must be the koala. Maria Puertos Calvo:Exactly. Nic Fillingham:This is a foolproof plan. Maria Puertos Calvo:(laughs)Nic Fillingham:I'm glad I discussed this with you on the podcast. Thank you, Marie, for validating my poses.Maria Puertos Calvo:Now, now you can't publish this.Nic Fillingham:Oh, we talked about fingerprints. Oh, crumbs you're right. Yeah. Okay. All right. We have to edit this out of the, (laughs) out of there quick. Maria Puertos Calvo:(laughs)Nic Fillingham:Um, okay. I didn't realize we had talked so much about fingerprints. That's my fault, but I found that fascinating. Thank you. So what happens next? Do you then go to Microsoft? Do you come straight out of your education at university in Madrid, straight to Microsoft? Maria Puertos Calvo:Kind of and no. So what happens next is that while I, I finished the master's part of this PhD, and at this time I'm actually dating my now husband, and he's an American, uh, working in Washington D.C. as an electrical engineer. So I, you know, I finished my master's and my, I say, why, why do I kind of wanna go be in the US uh, so I can be with him. And, you know, I have the space, the scholarship they'll actually lets me go do research abroad and you know, like kind of pays for it. So Maria Puertos Calvo:Find, um, another research group in the University of Maryland, College Park, which is really, really close to, to DC. And, and I go there to do research for, uh, six months. So, I spent six months there also doing research. Uh, also using, uh, machine learning for, for a different around iris recognition. And, you know, the six months went by and I was like, "Well, I want to stay a little longer," like, "I, you know, I really like living here," and I extended that, like, another six months. I... And at that point, you know, I wasn't really allowed to do that with my scholarship, so I just asked my professor to, you know, finance me for that time. And, and, uh, and at that time, I decided, like, you know, I, I actually don't think I wanna, like, pursue this whole PHD thing. Maria Puertos Calvo:So, so I stayed six more months working for him, and then I decided I, I, I'm not a really big fan of academia. I went into research in, in grad school in Spain mostly because there weren't other opportunities. I was super, you know, glad I did 'cause I, I love all the research and the knowledge that I gained with all... You know, with my master's where I learned everything about Artificial Intelligence. But at this point, I really, really wanted to go into industry. Uh, so I applied to a lot of jobs in a lot of different companies. You know, figuring out, like, my background is in biometrics and machine learning. Things like that. Data science is not a word that had ever come to my mind that I was or could be, but I was more, like, interested in, like, you know, maybe software roles related to companies that did things that I had a similar background in.Maria Puertos Calvo:For like a few months, I was looking in... I, I didn't even get calls. And I had no work experience other than, you know, I had been through college and grad school. So, I had... You know, and, and I was from Spain and from a Spanish university, and there was really nothing in my resume that was, like, oh, this is like the person we need to call. So, nobody called me. (laughs) And, and then one day, uh, I, I received a LinkedIn message from a Microsoft recruiter. And she says, "Hey, I have... I'm interested in talking to you about, uh, well, Microsoft." So I said, "Oh, my God. That sounds amazing." So, she calls me and we talk about it, and she's like, "Yeah, there's like this team at Microsoft that is like run mostly by data scientists and what they do is they help prevent fraud, abuse, and compromise for a lot of Microsoft online services." Maria Puertos Calvo:So, they, they basically use data and machine learning to do things like stopping spam for Outlook.com, doing, like, family safety like finding, like, things on the web that, that should be, like, not for children. They were also doing, like, phishing detection on the browser. Um, like phishing URL detection on the browser and a co- compromise detection for Microsoft Account. And so I was like, "Sure, that sounds amazing." You know? "I would love to be in the process." And I was actually lying because I did not want to move to Seattle. (laughs) Like, at that time, I was so hopeful that I will find a job at, you know, somewhere in DC on the east coast, which is like closer to Spain and where, where we lived in. But at the same time, you know, Microsoft calls and you don't say no mostly when nobody else is calling you. Maria Puertos Calvo:Um, so, so I said, "Sure, let's, you know, I, uh... The, the least I can do is, like, see how the interview goes." So, I did the phone screen and then I... They, they flew me to Seattle and I had seven interviews and a lunch inter- and a lunch kind of casual interview. So, it was like an eight hour interview. It was from 9:00 to 5:00. And, you know, everything sounded great, the role sounded great. Um, the, the team were... The things that they were doing sounded super interesting. And, to my surprise, the next day when I'm at the airport waiting for my flight to, to go back to DC, the recruiter calls me and says, "Hey, you, you know, you passed the interview and we're gonna make you an offer. You'll have an offer in the... In the mail tomorrow." I was like, "Oh, my God." (laughs) "What?" Like, I could not... This... It's crazy to me that this was, like, only seven years ago, it... But yeah.Nic Fillingham:Oh, this is seven... So, this was 2014, 2013?Maria Puertos Calvo:Uh, actually, when I did the interview, it was... It was more, more... It was longer. It was 2012. Nic Fillingham:2012. Got it.Maria Puertos Calvo:And then I... And then starting my Microsoft in 2013.Nic Fillingham:Got it.Maria Puertos Calvo:I started as a... I think at that time, they called us analysts. But it was funny because the, the team was very proud on the, the fact that they were one of the first teams doing, like, real data science at Microsoft. But there were too many teams at Microsoft calling themselves, and basically only doing, like, analytics and dashboards and things like that. So, because of that, the team that I was in was really proud, and they didn't want to call themselves data scientists, so they... I don't know. We called ourselves, like, analysts PMs, and then we were from that to decision scientists, uh, which I never understood the, the name. (laughs) Uh, but yeah. So, that's how I started.Nic Fillingham:Okay, so, so that first role was in... I heard you say Outlook.com. So, were you in the sort of consumer email pipeline team? Is that sort of where that, that sat?Maria Puertos Calvo:Yeah. Yeah, so, uh, the team was actually called safety platform. It doesn't exist anymore, but it was a team that provided the abuse, fraud, and, and, like, malicious detections for other teams that were... At the time, it was called the Windows live division.Nic Fillingham:Yes.Maria Puertos Calvo:So, all the... All the teams that were part of that division, they were like the browser, right? Like, Internet Explorer, Hotmail, which was after named Outlook.com. And Microsoft Account, which is the consumer ecosystem, we're all part of that. And our team, basically, helped them with detections and machine learning for their, their abusers and fraudsters and, and, you know, hackers that, that could affect their customers. So, my first role was actually in the spam team, anti-spam team. I was on outbound, outbound spam detection. So, uh, we will build models to detect when users who send spam from Outlook.com accounts out so we could stop that mail basically.Nic Fillingham:And I'd loved to know, like, the models that you were building and training and refining then to detect outbound spam, and then the kinds of sort of machine learning technology that you're, you're playing today. Is there any similarity? Or are they just worlds apart? I mean, we are talking seven years and, you know, seven years in technology may as well be, like, a century. But, you know, is there common threads, is there common learnings from back there, or is everything just changed?Maria Puertos Calvo:Yes, both. Like, there, there are, obviously, common threads. You know, the world has evolved, but what really has evolved is the, the, the underlying infrastructure and tools available for people to deploy machine learning models. Like, back then, we... The production machine learning models that were running either in, like, authentication systems, either in off- you know, offline in the background after the fact, or, or even for the... For the mail. The Microsoft developers have to go and, like, code the actual... Let's say that you use, like, I don't know, logistic regression, which is a very typical, easy, uh, machine learning algorithm, right? They had to, like, code that. They had to, you know... There wasn't like a... Like, library that they could call that they would say, "Okay, apply logistic regression to, to this data with these parameters. Maria Puertos Calvo:Back then, it was, like... People had to code their own machine learning algorithms from, like, the math that backs them, right? So, that was actually... Make things so much, you know, harder. They... There weren't, like, the tools to actually, like, do, like, data manipulation, visualization, modeling, tuning, the way that we have so many things today. So, that, you know, made things kind of hard. Nothing was... Nothing was, like, easy to use for the data scientists. It... There was a lot of work around, you know, how do you... Like, manual labor. It was like, "Okay, I'm gonna, like, run the model with these parameters, and then, like, you know, b- based on the results, you would change that and tweak it a little bit. Maria Puertos Calvo:Today, you have programs that do that for you. And, and then show you all the results in, like, a super cool graph that tells you, uh, you know, like, this is the exact parameters you need to use for maximizing this one, uh, you know, output. Like, if you want to maximize accuracy or precision or recall. That, that is just, like, so much easier.Nic Fillingham:That sounds really fascinating. So, Maria, you now... You now run a team. And I, I would love to sort of get your thoughts on what makes a great data scientist and, and what do you look for when you're hiring into, into your team or into sort of your, your broader organization under, uh, under identity. What perspectives and experience and skills are you trying to sort of add in and how do you find it? Maria Puertos Calvo:Oh, what a great question. Uh, something that I'm actually... That's... The, the answer of that is something I'm refining every day. The, you know, the more, uh, experience I get and the more people I hire. I, I feel like it's always a learning process. It's like, what works and what doesn't. You know, I try to be open-minded and not try to hire everybody to be like me. So, that's... I'm trying to learn from all the people that I hire that are good. Like, what are their, you know... What's, like, special about them that I should try to look in other people that I hire. But I would say, like, some common threads, I think, it's like... Really good communication skills. Maria Puertos Calvo:Like, o- obviously the basics of, you know, being... Having s- a strong background in statistical modeling and machine learning is key. Uh, but many people these days have that. The, the main knowledge is really important in our team because when you apply data science to cyber security, there are a lot of things that make the job really hard. One of them is the, the data is... What... It's called really imbalanced because there are mostly, most of the interactions with, with the system, most of the data represents good activities, and the bad activities are very few and hard to find. They're like maybe less than 1%. So, that makes it harder in general to, to, to get those detections. Maria Puertos Calvo:And the other problem is that you're in an adversarial environment, which means, you know, you're not detecting, you know, a crosswalk in, in a road. Like, it's a typical problem of, of computer vision these days. A crosswalk's gonna be a crosswalk today or tomorrow, but if I detect an attacker in the data today and then we enforce... We do something to stop that attacker or to... Or to get them detected, then the next day they might do things differently because they're going to adapt to what you're doing. So, you need to build machine learning models or detections that are robust enough that use, use what we call features or, or that look at data that it's not going to be easy... Easily gameable. Maria Puertos Calvo:And, and it's really easy to just say, "Oh, you know, there's an attack coming from, I don't know, like, pick a country, like, China. Let's just, like, make China more important in our algorithm." But, like, maybe tomorrow that same attacker just fakes IP addresses Maria Puertos Calvo:Addresses in, in a bot that, that is not in China. It's in, I don't know, in Spain. So, so, you just have to, you know, really get deep into, like, what it means to do data science in our own domain and, and, and gain that knowledge. So, that knowledge, for me, is, is important but it's also something that, that you can gain in the job. But then things like the ability to adapt and, and then also the ability to communicate with all their stakeholders what the data's actually telling us. Because it's, you know... You, you need to be able to tell a story with the data. You need to be able to present the data in a way that other people can understand it, or present the results of your research in, in a way that other people can understand it and really, uh, kind of buy your ideas or, or what you wanna express. And I think that that is really important as well.Nic Fillingham:I sort of wanted to touch on what role... Is there a place in data science for people that, that don't have a sort of traditional or an orthodox or a linear path into the field? Can you come from a different discipline? Can you come from sort of an informal education or background? Can you be self-taught? Can you come from a completely different industry? What, what sort of flexibility exists or should there exist for adding in sort of different perspectives and, and sort of diversity in, in this particular space of machine learning?Maria Puertos Calvo:Yes. There are... Actually, because it's such a new discipline, when I started at Microsoft, none of us started our degrees or our careers thinking that we wanted to go into data science. And my team had people who had, you know, degrees in economics, degrees in psychology, degrees in engineering, and then they had arrived to data science through, through different ways. I think data science is really like a fancy way of saying statistics. It's like big data statistics, right? It's like how do we, uh, model a lot of data to, like, tell us to do predictions, or, or tell us like what, how the data is distributed, or, or how different data based on different data points looks more like it's this category or this other category. So, it's all really, like, from the field of statistics.Maria Puertos Calvo:And statistics is used in any type of research, right? Like, when you... When people in medicine are doing studies or any other kind of social sciences are doing studies, they're using a lot of that, and, and they're more and more using, like, concepts that are really related to what we use in, in data science. So, in that sense, it's, it's really possible to come to a lot of different fields. Generally, the, the people who do really well as data scientists are people who have like a PhD and have then this type of, you know, researching i- but it doesn't really matter what field. I actually know that there, there are some companies out there that their job is to, like, get people that come out of PhD's programs, but they don't have like a... Like a very, you know, like you said, like a linear path to data science, and then, they kind of, like, do like a one year training thing to, like, make them data scientists, because they do have, like, the... All the background in terms of, like, the statistics and the knowledge of the algorithms and everything, but they... Maybe they're, they've been really academic and they're not... They don't maybe know programming or, or things that are more related to the tech or, or they're just don't know how to handle the data that is big. Maria Puertos Calvo:So, they get them ready for... To work in the industry, but the dat- you know, I've met a lot of them in, in, in, in my career, uh, people who have gone through these kind of programs, and some of them are PhDs in physics or any other field. So, that's pretty common. In the self-taught role, it's also very possible. I think people who, uh, maybe started as, like, software engineers, for example, and then there's so much content out there that is even free if you really wanna learn data science and machine learning. You can, you know, go from anything from Coursera to YouTube, uh, things that are free, things that are paid, but that you can actually gain great knowledge from people who are the best in the world at teaching this stuff. So, definitely possible to do it that way as well.Nic Fillingham:Awesome. Before we let you go, we talked about the perfect guacamole recipe last time because you had that in your Twitter profile.Maria Puertos Calvo:Mm-hmm (affirmative). (laughs)Nic Fillingham:Do you recall that? I'm not making this up, right? (laughs)Maria Puertos Calvo:I do. No. (laughs)Nic Fillingham:All right. So, w- so we had the perfect guacamole recipe. I wondered what was your perfect... I- is it like... I wanted to ask about tacos, like, what your thoughts were on tacos, but I, I don't wanna be rote. I don't wanna be, uh, too cliché. So, maybe is there another sort of food that you love that you would like to leave us with, your sort of perfect recipe?Maria Puertos Calvo:(laughs) That's really funny. I, I actually had tacos for lunch today. That is, uh... Yeah. (laughs)Nic Fillingham:You did? What... Tell me about it. What did you have?Maria Puertos Calvo:I didn't make them, though. I, I went out to eat them. Uh-Nic Fillingham:Were they awesome? Did you love them?Maria Puertos Calvo:They were really good, yeah. So, I think it's-Nic Fillingham:All right. Tell us about those tacos.Maria Puertos Calvo:Tacos is one of my favorite foods. But I actually have a taco recipe that I make that it's... I find it really good and really easy. So, it's shrimp tacos.Nic Fillingham:Okay. All right.Maria Puertos Calvo:So, it's, it's super easy. You just, like, marinate your shrimp in, like, a mix of lime, Chipotle... You know those, like, Chipotle chilis that come in a can and with, like, adobo sauce?Nic Fillingham:Yeah, the l- it's got like a little... It's like a half can. And in-Maria Puertos Calvo:Yeah, and it's, like, really dark, the sauce, and-Nic Fillingham:Really dark I think. And in my house, you open the can and you end up only using about a third of it and you go, "I'm gonna use this later," and then you put it in the fridge.Maria Puertos Calvo:Yes, and it's like-Nic Fillingham:And then it... And then you find it, like, six months later and it's evolved and it's semi-sentient. But I know exactly what you're talking about.Maria Puertos Calvo:Exactly. So that... You, you put, like, some of those... That, like, very smokey sauce that comes in that can or, or you can chop up some of the chili in there as well. And then lime and honey. And that's it. You marinate your shrimp in that and then you just, like, cook them in a pan. And then you put that in a tortilla, you know, like corn preferably. But you can use, you know, flour if that's your choice. Uh, and then you make your taco with the... That shrimp, and then you put, like... You, you pickle some sliced red onions very lightly with some lime juice and some salt, maybe for like 10 minutes. You put that on... You know, on your shrimp, and then you can put some shredded cabbage and some avocado, and ready to go. Delicious shrimp tacos for a week night.Nic Fillingham:Fascinating. I'm gonna try this recipe. Maria Puertos Calvo:Okay.Nic Fillingham:Sounds awesome.Maria Puertos Calvo:Let me know.Nic Fillingham:Maria, thank you again so much for your time. This has been fantastic having you back. The last question, I think it's super quick, are you hiring at the moment, and if so, where can folks go to learn about how they may end up potentially being on your team or, or being in your group somewhere?Maria Puertos Calvo:Yes, I am actually. Our team is doubling in size. I am hiring data scientists in Atlanta and in Dublin right now. So, we're gonna be, you know, a very, uh, worldly team, uh, 'cause I'm based in Seattle. So, if you go to Microsoft jobs and search in hashtag identity jobs, I think, uh, all my jobs should be listed there. Um, looking for, you know, data scientists, as I said, to work on fraud and, and cyber security and it's a... It's a great team. Hopefully, yeah, if you're... If that's something you're into, please, apply.Nic Fillingham:Awesome. We will put the link in the show notes. Thank you so much for your time. It's been a great conversation.Maria Puertos Calvo:Always a pleasure, Nic. Thank you so much. Natalia Godyla:Well, we had a great time unlocking insights into security, from research to Artificial Intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us @msftsecurity or email us at securityunlocked@microsoft.com with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure.