Security Unlocked

Share

Tackling Identity Threats With AI

Ep. 8

The last thing we all need this year is an identity crisis. Fear not, hosts Nic Fillingham and Natalia Godyla are here with Maria Puertas Calvo, Data Science Lead of Microsoft’s Identity Security and Protection Team, to learn how AI is being used to protect our personal identities. Maria also reveals previously undisclosed information – her favorite food and her famous top-secret recipe, so get ready to take notes! 


Later, the hosts bring back a previous guest, Geoff McDonald, ML Research Lead at Microsoft to unpack his career in cybersecurity and how game hacking led him to where he is now. 


In This Episode, You Will Learn:

• How offline detections are used for account compromise prevention 

• The importance of multi-factor authentication 

• How Microsoft is taking a new approach with AI to identify threats with real-time prevention  

• The problem with adversaries and malware attackers 


Some Questions We Ask: 

• How is Microsoft applying AI to solve problems for account compromise prevention? 

• How do humans play a role in labeling data sets? 

• How is Microsoft measuring success of their new enhanced AI? 

• What is the future for neural networks? 


Resources

 

Maria’s Blog 

https://techcommunity.microsoft.com/t5/azure-active-directory-identity/enhanced-ai-for-account-compromise-prevention/ba-p/1994653 

 

Microsoft Security Blog 

https://www.microsoft.com/security/blog/ 

 

Nic’s LinkedIn 

https://www.linkedin.com/in/nicfill/ 

 

Natalia’s LinkedIn 

https://www.linkedin.com/in/nataliagodyla/ 

 

Maria’s LinkedIn 

https://www.linkedin.com/in/mariapuertas/ 

 

Geoff’s LinkedIn 

https://www.linkedin.com/in/geoff-mcdonald-76655029/ 


Transcript

(Full transcript can be found at https://aka.ms/SecurityUnlockedEp08)


Nic:

Hello and welcome to Security Unlocked. A new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft Security engineering and operations teams. I'm Nick Fillingham.


Natalia:

And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft Security, deep dive into the newest threat Intel research and data science.


Nic:

And profile some of the fascinating people working on artificial intelligence in Microsoft Security. If you enjoy the podcast, have a request for a topic you'd like covered or have some feedback on how we can make the podcast better-


Natalia:

Please contact us at securityunlocked@microsoft.com or via Microsoft Security on Twitter. We'd love to hear from you.


Nic:

Hello, Natalia. Welcome to episode eight of Security Unlocked. How are you?


Natalia:

I'm doing great. We're right about at Christmas. I am feeling it in my onesy right now.


Nic:

You're feeling Christmas in your onesy? Is it a Christmas onesy?


Natalia:

No. I feel like onesys just highlight the Christmas spirit. I mean, you're in PJs all weekend.


Nic:

We've been in work from home for seven years now. We're all in perpetual onesy land.


Natalia:

Well, I mean, I try to put in effort. I don't know about you.


Nic:

I don't put any effort. I wonder if we should issue a subscriber challenge. I wonder if we could hit 1000 subscribers. We might make a security unlocked onesy. I wonder what other swag we could do? What would be good for a security unlocked podcast?


Natalia:

All right. I mean, I guess I'm a little biased but the security blanket is clever. The ones that Microsoft gives away.


Nic:

I don't think I have one of those.


Natalia:

It's a blanket with security images on it.


Nic:

Images of security in it? Just images of very strong passwords. Images of two factor authentication. What about a horse blanket? Like a blanket you put over your horse?


Natalia:

What does that have to do with security?


Nic:

Under the saddle. I'm just following the blanket thread, that's all. I'm just thinking different types of blankets. In two episodes have already talked about the bratty pigs. I wonder if we could turn the bratty pigs into our mascot and on the security blanket there could be like an animated picture of the bratty pigs running away with a padlock and key or something.


Natalia:

Have I not, and excuse the pun, unlocked the new technology in blankets and animated pictures? Is that possible on blankets now?


Nic:

Did I say animated? I meant illustrated, I'm sorry. Oh wow, I bet you there's some brand new piece of printing technology that's over in like Japan or South Korea that we haven't got over here yet where they've got animation on their blankets, that would be good. What about one of those automatic cat feeders for when you go away on holiday and it dumps a little bit of dry food into their bowl every 12 hours? And then we just put Security Unlocked on the side of it.


Natalia:

As long as it has our logo on, it fits.


Nic:

You know what? Also, this is our last episode for 2020.


Natalia:

How'd you feel about it?


Nic:

About this episode or about the year of 2020?


Natalia:

Well, the year 2020 is probably too much to unpack. What about our podcast adventure in 2020?


Nic:

Yeah, I've enjoyed it greatly. I listened to the first couple of episodes just the other day. And while they were great, I certainly heard an evolution in just eight episodes from that humble first back in October. So yeah, I've definitely enjoyed the trip. I'm very much looking forward to 2021. What about you?


Natalia:

I feel like our guests are making me smarter. With each new episode. I've got a few more terms under the belt. Terms I'd heard before but never got that clarity from experts and what the definition is especially as they're moving around. We see that with a lot of the machine learning and AI terms. Like neural networks when we're talking to experts, they have different lenses on what that should mean.


Nic:

The other thing that I found fascinating is everyone that you and I have reached out to internally, Natalia, and said, "Hey, do you want to be a part of this podcast?" Everyone said, Yes. Everyone has said, "Yeah, I'd love to share my story of how I got into security. I'd love to share my story of how I got to Microsoft." I love that we've spoken to such a incredible variety of people that have come to security and to Microsoft from just... I mean, everyone has a completely different story and everyone's been so willing to tell it. So I'm just very, very happy that we've been able to meet these great people and have these conversations.


Natalia:

Yes. And even in their diversity, I've been happy to see that there are really positive themes across the folks that wants to be in security that are in the security space. They're all so passionate about what they do and really believe in the mission, which is just great to see. And like you said, there's just awesome community. The fact that they want to go out and have these conversations and are always open to receiving questions from you guys. So please keep them coming. Our experts are equally as hungry as we are to hear not just feedback but questions on the topics that we discuss.


Nic:

So on today's episode, we chat with Maria Puertas Calvo. Fantastic conversation, very excited to have Maria on the podcast. I'm not sure if many folks picked up but a lot of the experts we've spoken to so far have been more on the endpoint detection side of the house. We've talked to folks over in the defender team and those who sort of look at the email pipeline. Maria and her team focused on identities, so protecting identities and protecting our identity platforms. And so she's going to talk about how AI and ML are used to protect identity. And then after Maria, we talked to...


Natalia:

Jeff McDonald. So he is a member of the Microsoft defender for endpoint research team. And he's joined us on a previous episode to talk about unmasking malicious threats with MC and ML. And today, he's chatting with us about his career in cybersecurity, which started with game hacking. So making changes in the game to get more skills, get new characters and he's got some amusing stories as to how far he took that. But it's also a theme we're seeing across a few of our guests that game hacking seems to be a gateway to cyber security.


Nic:

Yeah, hopefully the statute of limitations on game hacking has well and truly expired on the various games that Jeff mentions in his interviews. I hope we're not getting him in trouble. Enjoy the pod, and we'll see you all in 2021.


Nic:

Maria Puertas Calvo, thank you so much for joining us. Welcome to the Security Unlocked podcast.


Maria Puertas Calvo:

Hi, thank you for having me.


Nic:

If you could tell us about your role at Microsoft and what your day to day looks like in the team you're in. The mission and sort of scope of that work, that'd be great.


Maria Puertas Calvo:

Yeah, absolutely. So I am a principal data science manager in identity security and protection. So I lead a team of five data scientists that work within a big engineering team. And our big mission is to protect all of Microsoft's users from account compromise and other things like the abuse and fraud. As a data science team, we just analyze and look through all the huge amount of data that we get from all our customer logs and everything. And then we use that to build automated statistical based models or machine learning models or heuristic made models that are trying to detect those bad actions in our ecosystem. So compromised attacks or malicious bots that are trying to do bad things in our identity systems.


Natalia:

And Maria, we understand that your team also recently authored a blog on enhanced AI for account compromise prevention. So can you talk a little bit about what that blog entails, how we're applying AI to start solving some of these problems?


Maria Puertas Calvo:

Yeah, we're actually really excited about this work. But it just went into production recently and it has really enhanced what we call the bread and butter of really what we do. Which is trying to prevent compromise from happening in the ecosystem. Basically, we have been using artificial intelligence and AI to build detections for a pretty long time. And everything that we do, we try to start with whatever the long hanging fruit. We do offline detections, which are basically using the data after authentications or attacks already occurred and then detect those bad attacks and then we will inform the customer or make the customer reset their password or do some type of remediation.


Maria Puertas Calvo:

But being able to put AI at the time of authentication and so meeting that end goal that we're trying to not just detect when a user has been compromised and remediate it but we're actually able to prevent the compromise from happening in the first place. So this new blog talks about this new system that we've built. We already had real time compromised detection but it wasn't using the same level of artificial intelligence.


Natalia:

So is it correct to say then that in the past we had been doing is identifying a known attack, a known threat, and then producing detections based on that information and now we're trying to preempt it? So with this even more intelligent AI, we're trying to identify the threat as it's happening, is that correct?


Maria Puertas Calvo:

Yeah, that's correct. So we did already have real time prevention but most of our artificial intelligence focus used to be in the, after the fact. Now we have been able to move this artificial intelligence focus also to the real time prevention. And what we have achieved with this has really improved the accuracy and the precision of this detection itself. Which means now we're able to say that the signings that we say are risky, they're way more likely to actually be bad than before. Before we would have more noise and more false positives and then we would also have some other bad activities that would go undetected.


Maria Puertas Calvo:

With this new artificial intelligence system, we have really increased the precision. Which means, now if a customer says, "Oh, I want to block every single medium risk login that comes my way that is trying to access my tenant." Now, fewer of their real users are going to get blocked and more actual attackers are going to get blocked. So we've really improved the system by using this new AI.


Natalia:

What's changed that's increasing the precision?


Maria Puertas Calvo:

Yeah, so we actually published another blog with the previous system which was mostly using a set of rules based on user behavior analytics. So the main detection before was just using a few features of the signing itself and comparing them to the user history. So if you're coming from a new IP address, if you coming from a new location, if you're coming from a new device, there was like a deterministic formula. We were just using a formula to calculate a score which was the probability of how unfamiliar that finding was. Now we're taking way more inputs into account. So we're using... It depends on which protocol you're using.


Maria Puertas Calvo:

It has more intelligence about the network, it has some intelligence about what's going on. for example, if you're coming from an IP address that has a lot of other traffic that AD is seeing, it has also information about what AD is saying from that IP address. Does it have a lot of failed logins or is it doing something weird? And then instead of us manually setting a mathematical formula or rules in order to build that detection, what we do is we train an algorithm with what is called label data. So label data is just a set of authentications, some are good and some are bad and they're labeled as such. So we use that label data to tell the algorithm, "Hey, use this to learn," Right? That's how machine learning works.


Maria Puertas Calvo:

So the algorithm trains and then it's able to use that data to decide in real time if the authentication is good or bad.


Nic:

Yeah, thank you. And then where, if any, do human analysts or humans in specialty roles, if it's data science or analytics, when do they come in to either verify the results or help with labeling new sets of data? So you've got your known goods, you've got your known bads and I assume you end up with a bunch of unknowns or difficult to classify one way or the other. Is that a role for a human analyst or human data scientists to come in and create those new labels?


Maria Puertas Calvo:

Yeah, even though getting all this labels is extremely important. That is not really what... The data scientist is not there just classifying things as this is good, this is bad, just to get labels to feed it to the algorithm, right? What the data scientist does that is very crucial is to build the features and then train this machine learning model. So that is the part that is actually really important. And I always really try to have everybody in my team to really understand and become a great domain expert on two things, One is the data that they have to work with. It is not enough to just get the logs as they come from the system, attach the label to it and then feed it to some out of the box classifier to get your results.


Maria Puertas Calvo:

That is not going to work really well because those logs by themselves don't really have a lot of meaning. If the data scientist is able to really understand what each of the data points that are in our laws, sometimes those values, they're not coded in there to be features for machine learning. They're just added there by engineers to do things like debugging or showing locks to the user. So the role of the data scientist is really to convey those data points into features that are meaningful for the algorithm to learn to distinguish between the attack or the good. And that is the second thing that the data scientist needs to be really good at. The data scientist needs to have a very good intuition of what is good and how that looks in the logs versus what is bad and how the looks in the logs.


Maria Puertas Calvo:

With that knowledge basically knowledge of what the data in the logs mean and the knowledge of what attack versus good look in that data, then that is the feature engineering role. You transform those logs into all their data points that are calculations from those logs that are just going to have a meaning for the algorithm to learn if something is good or an attack. So I can give an example of this, it's very abstract. For example, when I see an authentication in Azure AD logs maybe one of the columns that I'd want him to know is like IP address, right? Every single communication over the internet comes from some client IP address which will be the IP address that's assigned to the device that you are on at the time that you're doing an authentication.


Maria Puertas Calvo:

There are billions, if not trillions of IP addresses out there. And each one is just some kind of number that is assigned to you or to your device and it doesn't really have any meaning on its own. It's just like if you have a phone number, is that a good or a bad phone number? I don't know, that's just not going to help me. But if I can actually go and say, "Okay, this is an IP address but is this an IP address that Nick use yesterday or two days ago? How often have I seen Nick in this IP address? What was the last time I saw Nick in this IP address?" If you can just play with those logs to transform it into this more meaningful data, it's really going to help the model understand and make those decisions, right?


Maria Puertas Calvo:

And then you also end up with fewer things to make decisions on, right? Because if I just had that one IP address to train the model, maybe my model will become really good at understanding which IP addresses are good and bad but only among the ones that we have used to train that model. But then when a new one comes in, the model doesn't know anything about that IP address, right? But if we instead change that into saying, "Okay, this is a known IP address versus an unknown IP address," And then now, instead of having trillions of IP addresses, we just have a value that says, Is it known or unknown. Then for every single new log in that comes in, we're going to be able to know if it's known or unknown.


Maria Puertas Calvo:

We don't really need to have seen that IP address before, we just need to compare it to the user history and then make that determination of it is this known or unknown and that ends up being much more valuable for the model.


Natalia:

So just mapping out the journey you've talked about. So we've gone from heuristics signature based detections to user analytics and now we're in a space where we're actively using AI but continuously optimizing what we're delivering to our customers. So what's next after this new release of enhanced AI? What is your team working on?


Maria Puertas Calvo:

So lots of things but one thing that I am really interested in that we're working on is making sure that we're leveraging all the intelligence that Microsoft has. So for example, we built a system to evaluate in real time, the likelihood that a finding is coming from an attacker. But all of that is just using the data that identity processes like Azure Active Directory sign ins and what's happening the Azure Active Directory infrastructure. But there's so much more that we can leverage from what is happening across the ecosystem, right? Like the user who signs into Azure Active Directory is probably also coming in from a Windows machine that probably has Microsoft dependent Defender ATP installed on it. That it's also collecting signal and it's understanding what it's happening to the endpoint.


Maria Puertas Calvo:

And at the same time, when the sign in happens then the sign in doesn't happen just to go to Azure AD, right? Azure AD is just the door of entry to everything, Usher, Office, you name it. Third party applications that are protected by things like Microsoft Cloud App Security. And all of the security features that exist across Microsoft are building detections and collecting data and really understanding in that realm, what are the security threats and what's happening to that user? So there is a journey, right? Of that sign in. It's not just what's happening in Azure AD but it's everything that's happening in the device. What's happening in the cloud and in the applications that are being accessed after.


Maria Puertas Calvo:

So we're really trying to make sure that we are leveraging all that intelligence to enhance everything that we detect, right? And that way, the Microsoft customer will really benefit from being a part of the big ecosystem and having that increased intelligence should really improve the quality of our risk assessment and our compromise detections.


Nic:

Maria, how much of this work that you talked about in the blog and the work that your team does is trying to mitigate the fact that some folks still don't have multi factor authentication? Is any of this a substitute for that?


Maria Puertas Calvo:

We know from our own data studies that accounts that are protected by multi factor authentication, which means every time they log in, they need to have a second factor, those accounts are 99.9% less likely to end up compromised because even if their password falls in the hands of a bad actor or get gassed or they get phished, that second factor is going to protect them and it's way more likely to stop the attack right there. So definitely, this is not supposed to be a substitute of multi factor authentication. Also, because of that, our alerts do not... They still will flag a user if the sign in was protected by multi factor authentication but the password was correct. Because even if there's multi factor authentication, we want to make sure that the user or the admin know that the password was compromised so they're able to reset it.


Maria Puertas Calvo:

But the multi factor authentication is the tool that is going to prevent that attack. And you asked earlier about what's next in other feature things and one thing that we're also really working on is, how do we move past just detecting these compromises with the password of using multi factor authentication as a mitigation of this risk, right? Like the way a lot of the systems are implemented today is if you log in and we think your log in is bad but then you do MFA. That is kind of like a reassuring things that we committed a mistake, that was a false positive and that's a remediation event. But the more people move to more MFA and more password less, our team is starting to think more and more of what's the next step?


Maria Puertas Calvo:

How are attackers are going to move to attacking that multi factor authentication. It is true that multi factor authentication protects users 99.9% of the time today but as more people adopt it, attackers are going to try to now move to get to bypass our multi factor authentication. So there's many ways but the most popular multi factor or second factor that people have in their accounts is telephone based. So there's SMS or there's a phone call in which you just approve the Sign In. There are phishing pages out there that are now doing what is called real time men in the middle attack in which you put your username and password, the attacker grabs it, puts it in the actual Azure AD site and then now you're being asked to put your SMS code in the screen. So the attacker has that same experience in their phishing site, you put in your code and the attacker grabs the code and puts it in Azure AD sign in page and now the attacker has sign in with your second factor, right?


Maria Puertas Calvo:

So two challenges that we're trying to tackle is, one, how do we detect that this is happening? How do we understand that when a user uses their second factor, that is not a mitigation of the risk? It's more and more possible with time that attackers are actually also stealing this second credential and using it, right? So we need to make more efforts in building those detections. And the second really big thing is, what then, right? Because if we actually that the attacker is doing that, then what is the third thing that we asked you? Now you've given us a password, you've given us a second factor, if we actually think that this is bad, but it is not. What is the way for the user to prove that it's them, right?


Maria Puertas Calvo:

So we need to move and I think this is extremely interesting, we need to move to from a world in which the password is the weak crab and everything else is just considered good. which today, it's very true. If you have a second factor, that is most likely going to be just fine but in the future, we we need to adapt to future attacks in which this won't be the case. So we need to understand what is the order of security of the different credentials and what is the remediation story for attacks that are happening with these second factors.


Nic:

I'd like to propose that third challenge, that third factor, should be a photograph of you holding today's newspaper doing the floss or some other sort of dance craze that's currently sweeping the nation.


Maria Puertas Calvo:

Sure, we'll add it to the bar code.


Nic:

I think that would just stamp out all identity theft and fraud. I think I've solved it.


Maria Puertas Calvo:

You did. I think so.


Natalia:

I think you'll be bringing back newspapers along with it.


Nic:

Yes. Step one is to reinvigorate the print newspaper industry. That's the first step of my plan but we'll get there.


Natalia:

So Maria, in your endeavors? How are you measuring success, for instance, of the new enhanced AI that your team has developed?


Maria Puertas Calvo:

Yeah, so our team is extremely data driven and metric driven and everything we do, we're trying to improve on one metric, right? The overall team mission really is to reduce the amount of users who fall victims of compromised account or what we call unauthorized access. So we have a metric that we all review every single day, we have a huge dashboard that is everybody's homepage in which we see in the last three months, what percentage of our monthly active users fell victim to compromised account and our main goal is to drive that metric down. But that is really the goal of the whole team including the people who are trying to make users adopt MFA and conditional access and other types of security measures.


Maria Puertas Calvo:

When we look into detection metrics and the ones like the AI detection metrics, we mostly play with those precision and recall metrics that are also explained in the blog. So precision is the percentage of all of the detected users or detected signings that you detected as bad that are actually bad, right? Out of everything that, let's say, you would block, how many of those were actually bad? So it really also tells you how much damage you're doing to your good customers. And the other one is recall and recall is out of all the bad activities that are out there, so let's say all the bad sign ins that happen in a day, how many of those that your system catch?


Maria Puertas Calvo:

So it's a measure of how good you are at detecting those bad guys. And the goal is to always drive those two numbers up. You want to be really high precision and you want to be really high recall. So every time we'll have a new system and a new detection or whatever it is or we perform improvements in one of our detection, those are the two metrics that we use to compare the old and the new and see how much we've improve.


Natalia:

And how are we getting feedback on some of those measures? And what I mean by that is the first one you mentioned. So precision, when you're saying how many were actually bad and we need to figure out how many were the true positive? How do we know that? Are we getting customer feedback on that or is there a mechanism within the product that lets you know that it was truly a bad thing that was caught?


Maria Puertas Calvo:

Yeah, so the same label and mechanisms that I was talking about earlier that we need both labels to be able to train or supervise machine learning models, we also need those labels in order to be able to evaluate the performance of those machine learning models. So knowing at least for a set of our data, how much is good and how much is bad and understanding what our systems are doing to detect the good and the bad. So one of the mechanisms is, as I was talking, the manual labeling that we have in place but the other one you mentioned is customer feedback, absolutely. Actually, one of the first thing we did when we launched editor protection is to include feedback buttons in the product.


Maria Puertas Calvo:

All of our detections actually go to an Azure Portal UX in the identity protection product and admins there can see all of the risky sign ins and all of the risky users and why they were detected as risky. Everything that my team is building gets to the customer through that product. And that's where the admin can click buttons like confirm safe or confirm compromised. Those are labels that are coming back to us. And users now also, there's a new feature in entity protection called My Finance. And users can go to my sign ins and look at all their recent signings that they did and they can flag the ones that they think it wasn't them. So if they were compromised, they can tell us themselves, this was not me.


Maria Puertas Calvo:

So that is another avenue for us to understand the quality of our detections. And then we're extremely customer obsessed as well. So even, it's not just the PMs in our team who have customer calls. The data scientists, many, many times get on calls with customers because the customers really want to understand what's the science behind all of these detections and they want to understand how it works. And the data science teams also wants the feedback and really understand what the customer thinks about the detection. If we're having false positives, why is that? It's really challenging too in the enterprise world because every tenant may have a different type of user base or different type of architecture, right?


Maria Puertas Calvo:

We had a time that we were tracking... We always track what are the top 10 tenants that get flagged by the detections. For example, airlines used to be a big problem for us because they had so much travel that we had a lot of false positives, right? We were flagging a lot of these people who because they're flying all over the world and signing in from all over the world. So it would trigger a lot of detections but there are other customers in which this is not the case at all. All of their users stay put and they're just only logging in from the corporate network because it's a very protected environment. So this quality of detections and this precision and recall can really vary customer by customer.


Maria Puertas Calvo:

So that is another challenge that I think we need to focus more in the future. How do we tune our detections in order to make more granular depending on what the industry is or what type of setup the customer or the tenant has.


Nic:

Changing subjects just a little bit and maybe this is the last question, Maria. I noticed on your Twitter profile, you refer to yourself as a guacamole eater. I wondered if you could expand upon that. There are very few words in your bio but there's a lot of thought gone into those last two words. Tell us about eating guacamole.


Maria Puertas Calvo:

Well, what can I say? I just really love guacamole. I think I may have added that about a year ago, I was pregnant with my twins who were born five months ago and when you're pregnant with twins they make you eat a lot of calories, about 3000 calories a day. So one of the foods that I was eating the most was guacamole because it's highly nutritious and it has a lot of calories. I went on a quest to finding the best recipe for guacamole and-


Nic:

Okay, walk us through your best guacamole recipe. What's in it?


Maria Puertas Calvo:

Absolutely. So the best guacamole recipe has obviously avocado and then it has a little bit of very finely chopped white onion, half jalapeno, cilantro and lime and salt. That's it.


Nic:

No tomatoes?


Maria Puertas Calvo:

No tomatoes. The tomatoes only add water to the guacamole, they don't add any flavor.


Nic:

What about then a sun dried tomato? No liquid, just the flavor? Is that an acceptable compromise?


Maria Puertas Calvo:

Absolutely not. No tomatoes in guacamole. The best way to make it is, you first mash the jalapeno chili with the cilantro and the onion almost to make a paste and then you mix in the avocado and then you finally drizzle it with some lime and salt.


Nic:

Hang on. Did you say garlic or no garlic?


Maria Puertas Calvo:

No garlic, onion.


Nic:

No garlic, I see. So the onion is the substitute for I guess that's a savoriness? I don't know how you classify... What's garlic? Is it Umami? I don't know the flavor profile but no garlic? Wow, I'm making guacamole when I'm at my house.


Natalia:

Well, you heard it here first guys. Maria's famous guacamole recipe.


Nic:

I think we'll have to publish this on Twitter as a little Easter eggs for this episode. It'll be Maria's definitive guacamole recipe.


Maria Puertas Calvo:

Now the secret is out.


Nic:

Well, Maria, thank you so much for your time. This has been a fantastic chat I think. I have a feeling we're going to want to talk to you again on the podcast. I think we'd love to hear a bit more about your personal story and I think we'd also love to learn more about some of the AI techniques that you talked to us about but thank you so much for your time.


Maria Puertas Calvo:

Yeah, of course, this was a pleasure. I had a great time and I'll come back anytime you want me. Thank you.


Natalia:

And now let's meet an expert from the Microsoft Security Team to learn more about the diverse backgrounds and experiences of humans creating AI and tech at Microsoft. Today, we're joined by Jeff McDonald, who joined us on a previous episode, unmasking malicious scripts with machine learning to talk to us about anti-malware scan interface or AMC. Thank you for joining us again on the show, Jeff.


Geoff McDonald:

Yeah. Thank you very much. I really enjoyed being here last time and excited to be here again.


Natalia:

Great. Well, why don't we start by just giving a quick refresher to our audience? Can you share what your role and day to day function is at Microsoft?


Geoff McDonald:

I lead a team of machine learning researchers and we build our machine learning defenses for Microsoft Defender antivirus product. So we built lightweight machine learning models which go into the antivirus product itself which run on your device with low memory and lower CPU costs for inference. We also deploy a lot of machine learning models into our cloud protection platform where we have clusters of servers in each region around the world. So that when you're scanning a file or behavior on your device, it sends metadata about the encounter up to our cloud protection in real time to the closest cluster to you. And then we do real time running of all of our machine learning models in the cloud to come back with a decision about whether we should stop the behavior or attack on your device.


Geoff McDonald:

So we're a small team of probably about five of us. We're a mix of threat researchers and machine learning and data science experts. And we work together to design new protection scenarios in order to protect our customers using machine learning.


Nic:

Jeff, when you go to a security conference, some kind of industry get together, do you describe yourself as a machine learning engineer? What do you use when you're talking to other security professionals in your field? Is machine learning... Is it sort of an established subcategory or is it still sort of too nascent?


Geoff McDonald:

Yeah. I used to call myself maybe a threat researcher or a security researcher when I would present at conferences and when I would introduce myself. But I'd say nowadays, I'd be more comfortable introducing myself as a data scientist because that's my primary role now. Although I come from a very strong background in the security and security research aspect, I've really migrated to an area of work where really machine learning and data science is my primary tool.


Natalia:

What's driven that change? What prompted you to go deeper into data science as a security professional?


Geoff McDonald:

So when I first started at Microsoft, I was a security researcher. So I would do a reverse engineering of the malware itself. I would do heuristics, deep analysis of the attacks, and threat families and prepare defenses for them. So I think learning pretty early on while doing all the research in response to these attacks, it was very clear that the human analysis and defense against all these attacks was really not scalable to the scale that we needed. So it really had to be driven by automation and machine learning, in order to be able to provide a very significant protection level to our customers. So I think that really drove the natural solution where all these human resources, these manual analysis doesn't scale to where we need it to be and where we want our protection level to be.


Geoff McDonald:

So it really encouraged finding the automation and machine learning solution. And I have previously had some experience with machine learning. At the time, it was kind of a natural fit where I began a lot of exploration of the machine learning application to protect it against these threats and then pivoted into that as my primary role eventually, as it was quite successful.


Natalia:

So your unique set of skills, data science and security, is one that's definitely sought after in the security space. But considering the fact that we're still trying to fill just security jobs, it's definitely a challenge. So do you have any recommendations for companies that are looking for your set of skills and can't find a unicorn like yourself that has both? And if were looking for multiple people, how should these teams interact so that they're leveraging both skills to protect companies?


Geoff McDonald:

When we look to fill new positions on our team, we try to be really careful to try to be as inclusive as possible to a lot of different candidates. So when we're pushing our new data science positions where we're looking for the data science experience, like in the machine learning and data science application, you'll see in our job applications, we don't actually require cybersecurity experience for our job positions. We're really looking for someone who has a really great understanding of the data and good understanding of ML. And being able to have a strong coding background in order to be able to implement these pipelines and machine learning models and try out their experiments and ideas in ways that they can implement and take them end to end to deploying them.


Geoff McDonald:

So really, for people that were looking to join our team, often, you don't actually necessarily have to have a background in cybersecurity for all of our positions. Sometimes we're looking for really strong data scientists who can pick up the basics of security and apply it in a very effective way. But we would also want our team have different sets of people who are more experienced in the security background to help drive some of the product and feature and industry and security trends for the team as well. Our team currently has quite a mix of backgrounds where there's some threat researchers and there's some pure data scientists who have come from related fields who actually haven't come from a cybersecurity background specifically.


Nic:

I wonder if we can back it up. If we can go back in time and start with you, your story, how did you first get into security, get interested in security? Did it start in elementary school? Did it start in high school? Did it start in college? Did you go to college? Can we back up and learn about the young Jeff McDonald?


Geoff McDonald:

I grew up in a small town near Calgary, Alberta, Canada. I guess it started with my family being a software developing family, I would say. Like my dad had his own software company and as a result, we were really lucky to have the opportunity to learn to code from a young age. So, we would see our dad coding, we knew that our dad coded so we're really interested in what he was doing and we wanted to be able to learn and participate.


Nic:

When was that Jeff? We're talking in 80s, 90s?


Geoff McDonald:

So that would be when I was probably around 10 years old when I started coding. And that would be I guess, 96 or so.


Nic:

I'm trying to learn like was that on some cool, old Commodore 64 hardware or were we well and truly in the x86 era at that point?


Geoff McDonald:

Yeah. I mean, an x86 I do believe. So it's just Visual Basic which is very simple coding language. The classic Visual Basic 6.0, we're really lucky to be able to learn to code at a pretty young age, which is awesome. And although my brother went more into... My older brother was about two years older, a big influence on me coding wise as well. He was really into making, you might say, malware. We both had our own computers, we had often tried to break into each other's computers and do things. My brother created some very creative hacks, you can say. Like, one thing I remember is he burned a floppy disk, which would have an autorun on it and the way that I'd protect my computer is a password protected login.


Geoff McDonald:

But back in those days, I think it was windows 98 at the time, it really wasn't a secure way of locking your computer where you have to type in your password. You can actually insert a diskette and would run the autorun and you could just terminate the active process. So my brother created this diskette and program, which would automatically be able to bypass my security protocols and my computer, which I thought was pretty funny.


Nic:

Is he still doing that today? Is he still red teaming you?


Geoff McDonald:

No. Not red teaming me anywhere, luckily.


Natalia:

So what point were you like, "Well, all of these things that I've been doing actually apply to something I want to be doing for a career?"


Geoff McDonald:

Yeah. So although was in a really software development friendly household. My dad was really concerned about the future of software development. He was discouraging us from going into software development as a primary career path at the time. Going into university I was mostly considering between engineering and business. I ended up going into engineering because I really liked the mathematical aspect of my work and it is a mix of coding and math, which is kind of my two strong suites. So I went into electrical engineering program, during my electrical engineering for four years is when I really changed from doing game hacking as my hobby to doing software development for reverse engineering tools. So as my hobby, I would create a reverse engineering tools for others to use in order to reverse engineer applications. So I went to universities in Calgary, Alberta there. And in Alberta, the primary industry of the province is oil and-


Nic:

Is hockey.


Geoff McDonald:

Good one. Yeah. So in Alberta, the primary industry in the sector is really oil and gas. There's a lot of oil and gas, pretty much all engineers when they graduate, the vast majority go into the oil and gas industry. So really, that's what I was thinking of that I'd probably be going into after I graduate. But either way, I continued the reverse engineering tool development, I did some security product kind of reverse engineering ideas as well. Approaching graduation, I was trying to figure out what to do with my life. I loved control systems, I loved software development, I loved the mathematical aspects and I want to do grad school. So then I looked at programs in security because my hobby of reverse engineering security, I didn't really take very seriously as a career.


Geoff McDonald:

I didn't think it could be a career opportunity, especially being in Alberta, Canada where oil and gas is the primary sector, there's not much in the way of security industry work to be seen as far as I could tell at the time in the job postings and job boards. So I ended up going for a master's in control systems continuing electrical engineering work. So basically, it's more like signal processing work where you're doing analyzing signals doing fault detection, basically, mount vibration sensors to rotating machines was my research. And then from the vibration signal, you're trying to figure out if there's a fault inside the motor or the centrifuge or the turbine or whatever it's attached to.


Geoff McDonald:

And in that field, there was a lot of machine learning in the research area. So that's where I got my first exposure to machine learning and I loved machine learning but that wasn't my primary research focus for my topic. And then approaching graduation, I started looking at jobs and I happen to get really lucky at the time that I graduated because there happened to be a job posting from Symantec in Calgary. And when looking at the requirements for the job postings, it had all of the reverse engineering tools and assembly knowledge and basically everything I was doing as a hobby, had learned through game hacking and developing these reverse engineering tools. It was looking for experience in only debug assembly. I'm like, "Oh, my goodness. I have all those skills. I can't believe there's actually a job out there for me where I could do my hobby as a career." So I got really lucky with the timing of that job posting and so began my career in cybersecurity instead of oil and gas.


Nic:

So you talked about the adding sensors parts to, I guess, oil and gas related sort of instrumentation. And then there was some machine learning involved in there. Is that accurate? So can you expand upon that a little bit, I'd love to learn what that look like.


Geoff McDonald:

So basically, the safety of rotating machines is a big problem. There was an oil and gas facility actually in Alberta which has centrifuges which spins the... I'm sure I'm not using the right terminology, but it spins some liquid containing gas to try to separate the compounds from the water, I think. And they had one of these... Actually, the spindle of the centrifuge broke and then it caused an explosion in the building and some serious injuries. So it was really trying to improve the state of the art of the monitoring of the health of a machine from the mounted accelerometers to them.


Geoff McDonald:

Two of the major approaches were machine learning, where you basically create a whole bunch of handcrafted features based on many different techniques and approaches and then you apply a neural network or SVN or something like that to classify how likely it is that the machine is going to have a failure or things like that. Now, I think at the time the machine learning was applied but it wasn't huge in the industry yet because machine learning in application to signals, that was, especially in convolutions, not as mature as it is now. The area I was working on was de-convolutions. A lot of machine learning models involve doing... At least a lot of machine learning models nowadays would approach that problem as a convolutional neural network. The approaches that I was working on next one was called a de-convolution approaches.


Geoff McDonald:

So I was able to get a lot of very in depth research into convolutions and what the underlying mean. And that has helped a lot with the latest model architectures where a lot of the state of the art machine learning models are based on convolutions.


Natalia:

So what was that a convolutional neural network? Can you define what that is?


Geoff McDonald:

So convolution is basically where you're applying a filter across the signal. It could be an image or it could be a one dimensional signal. So in this case, it's a one dimensional signal where you have... Well, at least it's a one dimensional signal if you have a single accelerometer on a single axis for the machine. You think of it like the classic ECG where you have a heartbeat going up and down. It's kind of like that kind of signal you can imagine which is the acceleration signal. And then you basically learn to apply a filter to it in order to maximize something. What filter you apply can be learned in different ways. So in a convolutional neural network, you might be learning the weights of that filter, how that filter gets applied based on back propagation through whatever learning goal you're trying to solve.


Geoff McDonald:

In a typical CNN model, you might be learning something like 1000 of these filters where you're adjusting the weights of all these filters through back propagation according to... To try to minimize your loss function. I guess in my research area, I was working to maximize, design a filter through de-convolution to maximize the detection of periodic spikes in the vibration signal. Meaning that something like an impact is happening every cycle of the rotor, for example.


Nic:

Well, so convolution is a synonym for sort of complexity. So de-convolution, is that a oversimplification to say that it's about removing complexity and sort of filtering down into a simpler set, is that accurate?


Geoff McDonald:

I wouldn't say it's so similar to the English language version of it. It's a specific mathematical operator that we apply to a signal. So it's kind of like you're just filtering a signal. And de-convolution is sort of like de-filtering it. It's my best way to describe it.


Nic:

Oh, right. Okay, interesting. De-filtering it. Could you take a stab at just giving us your sort of simplest if possible definition of what a neural network is?


Geoff McDonald:

Okay. A simplest stab of a neural network, okay.


Nic:

And Jeff, there's very few people have asked that question of but you're one of them.


Geoff McDonald:

Okay, cool. When you look at the state of the art, you'll actually find that neural networks themselves are not widely used for a lot of the problems. So when it comes to like a neural network itself, the best way I might describe it is that it's basically taking a bunch of different inputs and it's trying to predict something. It could be trying to predict the future stock price of Tesla, for example, if they're trying to predict whether Tesla's going to go up or down or they could be trying to predict it. Especially in our Microsoft defender case, we're trying to predict, "Based on these features, is this malicious or not?" Is our type of application.


Geoff McDonald:

So it's going to mean taking a whole bunch of inputs like, "Hey, how old is this file in the world? how prevalent is this file in the world? What's its file size? And then what's the file name?" Well, maybe I'll say, "Who's the publisher of this file?" Well, it's going to take a whole bunch of inputs like that and try to create a reasoning... It's going to try to learn a reasoning from those inputs to whether it's malware or not as the final label. We do it through a technique called back propagation because we have imagined a million encounters where we have those input features. So then we use these known outputs and inputs in order to learn a decision logic to best learn how to translate those inputs to whether it's Malware or not.


Geoff McDonald:

So we do this through a lot of computers or sometimes GPUs as well in order to learn that relationship. And a neural network is able to learn nonlinear relationships and co-occurrences. So for example, it's able to learn a logic like is it more than 10,000 file size? And is the publisher not Microsoft? And the age is less than seven days, then we think it's 70% malicious. So it's able to learn sort of more complex logic like that, where it can create and conditions and create more complex logic depending on how many layers you have in that neural network.


Natalia:

Do you think there's a future for neural networks? It sounds like right now you see a specific set of use cases like image recognition but for other use cases it's been replaced. Do you think the cases you described right now like image recognition will eventually be replaced by other techniques other than neural networks?


Geoff McDonald:

I think they'll always play a role or derivatives of them will play a role. And it's not to say that we don't use neural networks at all. Like in our cloud protection platform, you'll find tons of logistic regression single neuron models, you'll find GBM models, you'll find random forest models. And we've got our first deep learning models deployed. Some of our feature sets have a lot of rich information to them and are really applicable to the CNN, the convolutional neural network model architecture and for those, we will have a neural network at the end of the month. So it still definitely plays its specialty role but it's not necessarily what's driving the bulk of protection. And I think you'll probably find the same for most machine learning application scenarios around the industry. That neural network is not key to most problems and that it's not necessarily the right tool for most problems but it does still play a role and it definitely will continue to play a role or derivatives of it.


Nic:

My brain's melting a bit.


Natalia:

I want to ask for a definition of almost every other term but I'm trying to hold back a bit.


Nic:

Yeah, I've been writing down like 50 words that Jeff has mentioned like, "Nope, I haven't heard that one before. Nope, that one's new." I think, Jeff, you've covered such a lot of fascinating stuff. I have a feeling that we may need to come back to you at other points in the future. If we sort of look ahead more in general to your role, your team, the techniques that you're sort of fascinated in? What's coming down the pike? What's in the future for you? Where are you excited? What are you focused on? What are you going to see in the next six, 12 18, 24 months?


Geoff McDonald:

One of the big problems that we have right now is adversaries. So what malware attackers do is that they build new versions of their malware then they check if it's detected by the biggest antivirus players. And then if it's detected by our AV engines, what they do is they keep building new versions of their malware until it's undetected. And then once it's undetected, they attack or customers with it and then repeat. So this is been the cat and mouse game that we've been in for years, for 10 years at least. Now, what really changed about six years ago is that we put most of our protection into our cloud protection platform. So if they actually want to check again, so like our full protection, and especially our machine learning protection, they have to be internet connected so they can communicate with a real time Cloud Machine Learning protection service.


Geoff McDonald:

And what this means is if they want to test their malware against our defenses before they attack our customers, it means that they're going to be observable by us. So we can look at our cloud protection logs and we can see, "Hey, it looks like someone is testing out their attack against our cloud before they attack our customers." So it makes them observable by us because they can't do it in a disconnected environment. Originally, when we came out with cloud protection, it seems like the adversaries were still testing in offline environments. Now we've gotten to the point where so many of the advanced adversaries as well as commodity adversaries are actually pre-testing their attacks against our cloud defenses before the attack our customers. And this introduces a whole bunch of adversarial ML and defensive strategies that we're deploying in order to stay ahead of them and learn from their attacks even before they attack our customers.


Geoff McDonald:

So we have a lot of machine learning and data science where we're really focused on preventing them from being able to effectively test with our cloud as a way to get an advantage when attacking customers. So that's one that we have a lot of work going into right now. A second thing that I really worry about for the future, this is like the really long term future, hopefully it won't be a problem for at least another decade or two or even hopefully longer. But having reinforcement learning, if we have some big breakthroughs, where we're able to use reinforcement learning in order to allow machine learning to learn new attacks by itself and carry out attacks fully automated by itself by rewarding it.


Geoff McDonald:

Luckily, right now, our machine learning or reinforcement learning state of the art is not anywhere close to the technology that would be needed to be able to teach an AI agent to be able to learn new attacks automatically and carry them out effectively. At least nowhere close to the effectiveness of a human at this point. But if we get to the level of effectiveness where we can teach an AI to come up with and explore new attack techniques and learn brand new attack techniques and carry out the attacks automatically, it could change the computing world forever, I think. We might be almost going back to the point where we have to live on disconnected computers or extremely isolated computers somehow but it would be kind of like a worst case scenario where machine learning has allowed the attackers to get to the point where they can use AI to automate everything and learn new attack techniques, learn new exploits, and et cetera, entirely by itself which would be a humongous problem for defensiveness.


Geoff McDonald:

And there's a lot of ongoing research in this right now but it's very much on the defensive side where, "Hey, we're going to use reinforcement learning to teach an attacker so that we can learn from defending against it automatically." That hypothesis is great but it's been created with the goal of trying to improve our defenses. But actually, it's also building the underlying methods needed in order to carry out attacks automatically by itself. And I think if we get to that point, it's a really big problem for security. It's going to revolutionize the way computer security works.


Nic:

Well, hopefully, Jeff, you and your colleagues remain one or two steps ahead in that particular challenge?


Geoff McDonald:

Yeah, we will.


Nic:

I hope you share that goal. Jeff, what are you and your team doing to make sure that you stay ahead of your sort of adversarial counterparts that are looking to that future? What gives you hope that the security researchers, the machine learning engineers, the data scientists are, hopefully, multiple steps ahead of adversaries out there?


Geoff McDonald:

I think our adversary situation is much better than it used to be back in the day. Back in the day, they'd be able to fully test our defenses without us even being able to see it. And now that we've forced them into the game of evading our cloud protection defenses, it allows us to observe them even before they attack our customers. So the defenses we have in place that we've already shipped as well as a lot of what we have planned is really going to be a real game changer into the way that we protect our customers where we can actually protect them even before our customers are attacked. So we're in a much better defensive situation since we're able to observe them before the attack our customers nowadays.


Natalia:

Thank you, Jeff, for joining us on today's show. As always, it was fantastic chatting with you and like Nick said, definitely need to have you back on the show.


Geoff McDonald:

Thank you very much. really love being on here.


Natalia:

Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.


Nic:

And don't forget to tweet us @MSFTsecurity or email us at securityunlocked@microsoft.com with topics you'd like to hear on a future episode. Until then, stay safe...


Natalia:

Stay secure.

More Episodes

4/7/2021

The Language of Cybercrime

Ep. 22
How many languages do you speak?The average person only speaks oneor twolanguages, and for most people that’s plentybecause even as communities arebecoming more global, languages are still very much tied to geographic boundaries.Butwhat happens when you go on the internet where those regions don’t exist the same way they do in real life?Because the internet connects people from every corner of the world, cybercriminals canperpetratescamsin countriesthousands of miles away. So how doorganizationslike Microsoft’s Digital Crime Unit combatcybercrimewhen they don’t even speak the language of the perpetrators?On today’s episode ofSecurity Unlocked, hostsNic FillinghamandNataliaGodylasit down withPeterAnaman,Principal Investigator on the Digital Crimes Unit,to discusshowPeterlooks at digital crimes inavery interconnected world and how language and culture play into the crimes being committed, who’s behind them, and how to stop them.In This Episode, You Will Learn:• Some of the tools the Digital Crime Unit at Microsoft uses to catch criminals.• How language and culturalfactors into cyber crime• Whycyber crimehas been onthe rise since Covid beganSome Questions We Ask:• How has understanding a specific culture helped crack a case?• How does a lawyer who served as an officer in the French Army wind up working at Microsoft?• Are there best practices for content creators to stay safe fromcyber crime?ResourcesPeterAnaman’s LinkedIn:https://www.linkedin.com/in/anamanp/NicFillingham’s LinkedIn:https://www.linkedin.com/in/nicfill/NataliaGodyla’s LinkedIn:https://www.linkedin.com/in/nataliagodyla/Microsoft Security Bloghttps://www.microsoft.com/security/blog/Transcript[Full transcript can be found at https://aka.ms/SecurityUnlockedEp22]Nic:(music)Nic:Hello and welcome to Security Unlocked. A new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft's Security Engineering and Operations Teams. I'm Nic Fillingham.Natalia:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft's Security. Deep dive into the newest threat intel, research and data science.Nic:And profile some of the fascinating people working on artificial intelligence in Microsoft Security.Natalia:And now, let's unlock the pod.Natalia:Hello, Nic. How is it going?Nic:Hello, Natalia. I'm very well, thank you. I'm very excited for today's episode. We talk with Peter Anaman, who is a return guest. Uh, he was on an earlier episode where we talked about business email compromise and some of the findings in the 2020 Microsoft Digital Defense Report. And Peter had such great stories that he shared with us in that conversation, that we thought let's bring him back. And let's, let's get the full picture. And wow, did we cover some topics in this conversation. I don't even know where to begin. How would, what's your TLDR for this one, Natalia?Natalia:Well, whenever your friends or family think about cyber security, this is it. One of the stories that really stuck out to me is, Peter went undercover, and has actually gone undercover multiple times, but in this one instance he used the cultural context from his family history, as well as the languages that he knows to gain trust with a bad actor group and catch them out. It's incredible. He speaks so many languages and he told so many stories about how he applies that to his day-to-day work in such interesting ways.Nic:Yeah, I love, for those of you who listened to the podcast, Peter really illustrates how knowledge of multiple cultures, knowledge of multiple languages, understanding how those cultures and languages can sort of intersect and ebb and flow. Peter has used that as powerful tools in his career. I think it's fascinating to hear those examples. Other listeners of the podcast who, who do have more than one language, who do understand and have experience across multiple cultures, maybe oughta see some, uh, some interesting opportunities for themselves in, in, in cyber security maybe moving forward.Nic:I also thought it was fascinating to hear Peter talk about working to try and get funds and sort of treasures and I think gold, l-literal gold that was taken during the second world war. And getting them back to it's original owner. Sort of like, a repatriation effort. As you say, Natalia, these are all things that I think our friends and family think of when they hear the words cyber security. Oh, I'm in cyber security. I'm an investigator in cyber security. And they have this sort of, visions, these Hollywood visions. Nic:This is, that's Peter. That's what he's done. And he's, he talk about it in his episode. It's a great episode.Natalia:And with that, on with the pod.Nic:On with the pod. Nic:(music)Natalia:Welcome back to Security Unlocked, Peter Anaman.Peter:Thank you very much. Thanks for having me back.Natalia:Well, it was a pleasure to talk to you, first time around. So I'm really excited for the second conversation. And in this conversation we really love to chat about your career in cyber security. How you got here? Um, what you're doing? So let's kick it off with a little bit of a refresher for the audience.Natalia:What do you do at Microsoft and what does your day-to-day look like?Peter:So in Microsoft, I work within the legal department. Within a group called the Digital Crimes Unit. We are a team of lawyers, investigators and analysts who look at protecting our customers and our online services from, um, organized crime or attacks against the system. And so we, we bring, for example, civil and criminal referrals in order to do that action. On a day-by-day basis, it's very, very varied. I focus more on business email compromise present with some, with some assistance on ransomware attacks and looking at the depths and the affiliates there. As well as looking at some attacks against the infrastructure based on automated systems. Peter:So it's kind of varied. So on a day, I could, for example, be running some crystal queries or some specialized database queries in order to look for patterns in unauthorized or illegal activity taking place in order to quickly protect our customers. At the same time, I have to prepare reports. So there's a lot of report writing just to make sure that we can articulate the evidence that we have. And to ensure we respect privacy and all the other rules, you know, when we present the data.Peter:And also, in addition to that, uh, big part of it is actually learning. So I take my time to look at trends of what's going on. Learn new skills in order to know that I can adapt and automate some of the processes I do.Nic:Peter, as someone with an accent, uh, I'm always intrigued by other people's accents. May I inquire as to your accent, sir. Um, I'm hearing, I think I'm hearing like, British. I'm hearing French. There's other things there.Peter:(laughs)Nic:Would you elaborate for us?Peter:Yes, of course. Of course. Oh so, I was born in Ghana, West Africa and spent my youth there. And later on went to the UK where I learned that, I had to have elocution lessons to speak like the queen. And so I had lesson and my accent became British. So but at the same time, I'm actually a French national. Um, I've been in the French army as an officer. And so, that's where the French part is. And throughout, I've lived in different countries doing for work. Uh, so I've learned a bit of German, a bit of Spanish on the way.Nic:I, I actually cheated. I looked at your, um, LinkedIn profile and I see you have six languages listed.Peter:Yes.Nic:The two, the two that you didn't mention, I am embarrassingly ignorant of Fante? And T-Twi, Twi? What are they?Peter:Twi and Fante are two of the languages that are spoken in Ghana. They're local languages. And so growing up, I always had that around me. When I went to my father's village where his, we communicate in that language. English is kind of the National Language but within the country, people really speak their own languages. So I've ticked it off now. Can I speak fluently in, in it? No, I've been away for too long. But if you put me there, I would understand everything they're saying. Nic:What are the roots of those two languages? Are they related at all? Or are they completely separate?Peter:They are related but one, one person cannot always understand the other. If you look more broadly, you look at for example, the African continent all are, you'll find that there are over, from what we understand, over, what was it? 2,000 languages are spoken on the continent. So sometimes a person, say on the east coast doesn't understand the person in the west coast, you know. And, and it's fascinating because, you know, when we look at cyber crime, we are facing a global environment. Which is actually pretty carved out, right? The physical world is still pretty segmented.Peter:And so when, for example, investigating some crimes taking place in Nigeria, well they speak pidgin English. And so we have to try and adapt to that to understand, what do they really mean when they say, X or Y? And so, you know, it kind of opens our mind at, as we're doing the investigations. So we have to really try and understand the local reality because the internet is not just one place. And I think, you know, working for, you know, Microsoft and with such an amazing diverse team, we've been able to share knowledge.Peter:So for example, in the case I mentioned, I went to my colleague in Lagos, Abuja. He went, oh, that's what it means. And we're like, okay great. That one makes a lot more sense. And so we can move on. So we have this kind of richness in the team that allows us to lean on each other and, you know, sort of drive impact. But yeah, language is very important. (laughs)Natalia:I was gonna ask, do you have any interesting examples in which the culture was really important to cracking in the case or understanding a specific part of a case that you were working?Peter:Yes. So there was one case I worked on earlier on which was in Lithuania. And in Lithuania, for a very long time, this group had been under investigation but they were very good at their Op Sec and used some, uh, different types of encryption and obsolete, obsolete communication to hide themselves. But what I learned from the chats and when I was, this was in an IRC, it started in IRC channels and then moved out of there afterwards. But I noticed that there was a lot of Italy. There was a lot of Italian references. And my grandfather was Sicilian so I've spent time in Italy. So I kind of understood that they traveled to Italy.Peter:So in part of the persona, I made reference to Sicily. And I just said, you know, that's where my grandfather's from. And this, didn't give a name obviously, but it kind of brought them closer, right? Because like, oh, yeah we, we get it. And after about two, three months, I was able to get them to send me pictures of them going on vacation in Italy. And unfortunately for them, the picture had geo-location on it. And also, we were able to blow it up to get the background of where they were in the airport and using the camera from the airport, we were able to identify who they were. And then go back to the passport, find their path and they got arrested a few weeks later. Peter:So but to get that picture, to get that inner information required a kind of, trust that was being built in the virtual world and that comes from trying to understand the culture. By teasing out, asking questions about who are you and what do you like. So that's just one example.Nic:N-no pressure in answering this question and we'll even, we'll even cut it out of the edit if it's one you don't wanna go with.Peter:(laughs) Sure.Nic:If you're good with it. But um, uh, I heard you now talk about personas and identities and y-you just sort of hinted at it in the answer to the previous question. It sounds like some of the work that you have done in the past has been about creating and adopting personas in order to go and learn more information about bad actors and groups out there in, uh, in cyber land. Is that accurate and are you able to talk about what that role and that sort of, that work look like, when you're performing it?Peter:Yeah. So before you have Peter:...persona, you have to understand where that persona's gonna be acted, right?Peter:And I'll give you an, an example of a story. Once I had to go to LA to give a presentation and when I got to the airport I got a cab. And in the cab I looked at the guy's, the license plate of the, of the person. And I said, I bet you, I can guess, which country you were born in. He was like, an African American kind of person. He goes, impossible. No one has guessed it, you will never know. I was, all right. Are you ready? You're from Ghana. And his mind was blown. He was like, how, how did you pin that to one country? I was like, well, in your name, you have Kwesi. And I know if you're born in a country, in Ghana and have Kwesi, it means you're born on a Sunday. So that fact that you have your, that name there, that means you were born from Ghana. He goes, you are right. And so that was that. Peter:And I said, I miss some food, the cuisine from my, from, from Ghana. And he goes, oh, I know a great place. It's in Compton. I said, go. Uh, when? So I went into my restroom, showered, go ready, try to g-got into a taxi and he goes, I'm not going into Compton. I was like, well, why not? I wanna go to that restaurant. And he goes, oh, no, no, no. I'm going to get robbed or something bad is going to happen to me. I was like, but it- By the way, he left, he went, I had a great meal. Afterwards, I spent two hours in the restaurant 'cause no taxi would come and pick me up. And eventually, the waitress took me to a local casino. And I got a cab there and I got back.Peter:Where, where I'm going with this story is about the environment. I didn't know what Compton meant, right? So if I created a persona that went there that didn't know the environment, they would not succeed. They would stick out like a sore thumb. They would, they would fail. So the first idea, is always to understand what are the different protocols.Peter:If I'm looking at, for example, FTP or IRC, the different peer-to-peer networks. Or I'm looking at NNTP and the old internet, you know. All of those work, you need different tools to work there. Different ways to collect evidence and different breadcrumbs you could leave that you need to know it may be needed. Because when you're there, you're there, right? And it's, you're leaving, you're leaving a mark. Also some people say, use proxies. Well, the problem with proxies that someone could know you got a proxy on. Because well, there's lots of systems out there. So it's about using the system. Understanding how it's interconnected so that when you show up, you show up without too much suspicion.Peter:The other thing I learned is that the personas have to, have to be kind of, sad. 'Cause what I found is that when they were a bit sad, like, I'm happy with your work and things like that. What I found, that's me, right? I found that people were more interested because people are kind by nature, right? And so when they see that you're sad, they're more likely to communicate with you. While, while if you're too confident, I can do everything. They're like, uh, no, that person. Peter:So I try to like, psychologically look at ways to make the person as real as possible, based on my experience, right, because if it was based on me, I would be called out. Because I will be inventing a character that's, was not real. If you try to give me a trick question, because it's based on me, the answer's gonna be the same. I've got, the persona is me. It's just different. And so that's how I took my time to understand it. I spend a lot of time learning the internet, the protocols, you know, how does P2P actually work. When I, going to an IRC channel or when I'm looking at the peer-to-peer network and looking at the net flow. So the data which is passing from my computer upload. What other information is flowing. Peter:Because if I can see it, they can see it, right? And at the same time I have to have the tools. So I was very fortunate to have, for example, some tools that can switch my IP address with any country, like, every minute. So I could really change personas and change location really rapidly and no one would know better 'cause I'm using different personas in different contexts, right?Peter:Now, I never lie. One of, one of the clear things is that you never, I never try and do anything illegal because I have to assume that law enforcement is on the other side. And that's not what I'm trying to do. So I'm not gonna commit the crime. I'm not going to encourage you to do the crime. I'm just listening and just being curious about you. But then people make mistakes because they share, they over share sometimes without knowing. Maybe they're too tired or something. Natalia:I have a bit of a strange question. So with the lockdown, culturally, people are expressing publicly that they feel like they're over sharing. Because they're all locked indoors. They have, their only outlet is to share online. So have you noticed that in your work in security? Do, are people over sharing in that underground world as well? Or there, there hasn't been an equal shift?Peter:No, I, I, I, actually think it's getting worse. Um, and part of the reason is, as more people go online, they're speaking more about how to be anonymous. So for example, I've seen a rapid increase in BackConnect. These are residential IP addresses used as proxies. Well 'cause now they're communicating to each other, saying, hey, we're all online and this is how you can get found out. And so there actually there's more sharing going on. You know, I look at this, many more VPN services out there. It just seems, they're better prepared. Now, obviously, we see a lot more, right? So I'm definitely seeing more sophistication because people are spending more time online. So they, they're not walking around waiting for the bus. They're reading, they're learning, they're adapting. They communicate with each other. Peter:I've even found like, cyber crime as a service, we've found clusters of groups of people. And when you look at that network, you could see. They're saying, oh, I offer phishing pages or I offer VPN. They become specialized. So now you have people that are saying, I am just gonna focus on getting your, for example, some exploits. Or I'm just gonna focus on getting you, um, some red team work so that you can go and drop your ransomware. You know what, they, they've become more specialized actually because they're online. And they've got the time to learn.Nic:Peter, you mentioned earlier, some time you spent in, I think, was it the French army, is that correct?Peter:Yes, that's correct.Nic:Do you want to talk about that? Was that your foray into security? Did it, did it begin with your career in the army? Or did it begin before then?Peter:Hmm. I think it started probably before then. In a sense that, once I left high school, I decided I wanted to study law. Because I wanted the system that I was gonna be working in. And so I went to law school, uh, in the UK. And when I came out, unfortunately, the market was not as good. So I couldn't get a job. And when I looked around at what other trenches I had. I found there was an accelerated cause to become an officer in the French Army. It's a bit like, West Point in the US. Or, and so to do that, it was basically two years, it a two year program condensed into four months. It was hard. And so (laughs) I-Nic:It was what? No sleep? Is that what it was? (laughs)Peter:Ahhh. I've lived through little sleep.Nic:No sleep before meals.Peter:Yeah. I had to, you know, even- Well one time, I even had to evacuated because I got hyperten- you know, uh, hypothermia. (laughs) It was, uh, sort of a character build, character builder, I like to call it that. Uh, but really I think that started the path. Uh, but for the security side was, was after that. So, 'cause of my debts from law school, I, I left the army and I went to, back to the UK. And there, the first job I found was to be a paralegal, photocopying accounts, bank accounts opened between 1933 and 1947. It was part of something called a survey. And it actually had something to do with the Nazi gold.Peter:So what happened is that during the second world war, a lot of peop- uh, people of Jewish origin, saw that they were gonna be persecuted and took their money to, uh, Switzerland and put them in numbered accounts. And kept the number in their head. While unfortunately, so many of them sadly, uh, were victimized, they died. And the number died with them. Well, the money stayed in the accounts and over time because the accounts were dormant, well, you had charges. And so the money left. Peter:And so this was something that Paul Volcker, I believe it was, started the survey to get the Swiss banks to comply and give the money back to the families as result. So I was part of a team investigating one of the banks there. And although I started photocopying, I looked at, using my military skills, to be very efficient. So I was the best photocopier.Natalia:(laughs)Peter:And uh, and we were five levels underground. And that's what I did and I worked hard. And then after a few weeks, I got promoted to manage, uh, photocopiers. The people photocopying. We were a great team. And after that, they realized I was still hanging around because everyone was sleeping. 'Cause working five levels underground is a bit depressing sometimes. Peter:And so eventually, I became a data analyst. And so now I had to do the research on the accounts to try and find someone writing in pen, oh, this number is related to this other main account. Or this there piece of evidence is linked to this name. And so basically, for about, I think about three years, I basically, I eventually ran the French team and we looked at all the French cards opened from that period. And that started the investigations and sort of, trying to think deeper into evidence and how to make it work. Natalia:I really didn't think of myself as being cool before this, but I'm definitely not cool after hearing this. It's been validated, these stories are way beyond me. Peter:(laughs) Well, no. Just stories.Natalia:(laughs) So what brought you to Microsoft? That how did you go from piracy investigation to working at Microsoft as an investigator?Peter:So what took place was actually, my troubles created by Microsoft. So back in 2000 it was Microsoft who actually saw that the internet was becoming something that could really hurt internet commerce and e-commerce of role and wanted to make sure Peter:But they could contribute to it, and participate by building this capacity. And all the way through, they were one of my clients, at, essentially. And at some point, I realized that in my career, working for different customers, clients is great, because you learn, you don't have something different. So, for example, a software company is very different to a games company. Is different to a publishing company, is different to a mo- motion picture company, although it's digital piracy, it's actually very different in many respects. And I have- I saw how Microsoft was investing more in the cloud at that time, and I saw that as a big opportunity to really help a bigger threat to the system, right? Peter:And when I say to the system, E-commerce, 'cause everything was booming, this was in like 2008. And so, I decided that I would work for them. And actually, they offered me the job. So, I- I didn't, you know, I'm very privileged to be where I am now. But the, the, the way they positioned it is that they were looking for someone to help develop systems to map out, create a heat map of online piracy. I was like, "Wow, this is a global effort." So, uh, that's what I came on board with. And I built actually, a, a system similar to Minority Report, whereby I got basically these crawlers that I built that would go out and visit all these pirate sites. And you'll find this fascinating 'cause... Well, I found it fascinating, in some cases- Natalia:(laughs). Peter:... as we accessed the forums that we're offering, you know, download sale, RapidShare was one of the companies at the time, as we shut them down, they have crawlers in the forum, which will go and replace them. So, we had machine or machine wars, where we would shut down a URL, and then they would put another one. The problem is that our system was infinite. That is, we can, the machine can keep clicking. For them, they had about 10 groups of files. And so once they reached number 10, that was it. So, I found a way to automate the systems. And then after that using the, the Kinect, do you remember the Xbox Kinect? Nic:Cer- certainly. Peter:Managed to hack that, and the way it happened is that I built a map on Bing, whereby the Kinect could look in my body structure. And as I moved my hand, it would drill in to a country. And when I pushed, it would create, like, a, a table on the window with the number of infringements, what products were offered, when was the last time it was detected. And then, I could just wave it away and it would go, and then I could spin the world, it was a 3D map to go to another country and say, "What are the concentrations of piracy?" In this way, we had a visualized way of looking at crime as they were taking place online, and then zoom in and say, "We need to spend more effort here." Right? Peter:So, as well, just getting data analytics, but in a 3D format. And so, that was part of the excitement when I joined, is how to do that. Another example is, I found that, I read some research where it said that basically humans only spend a minute and a half on any search query. You know, in itself it doesn't mean much. But imagine you have a timer and it's one second, two seconds, three seconds, right? You're waiting for a minute and a half, right? So, 90 seconds, let's double that and say 180 seconds. Basically, let's say three minutes, it means that if you go to anyone you know, and ask them, "Go and search for Britney Spears downloads." And you look too, go, do, do the search, and they will click a link, nothing. Go next, click next, and they'll keep going. Peter:Before the three minute mark, they'll stop. They'll change the query, they'll do something different. Because they wouldn't get a result. Which means that when you do a search, and a search has got a million results, uh, it doesn't really matter. People are not going to go through the million. So, I started to think about the problems that when executives and people were saying, "Oh, I go on the internet, and I can find bad stuff." I was like, "Okay, but you can do like in three minutes. How about I build a robot that will pretend to be you, and go and find the infringements within that three minute window? Which is about 400 URLs. But I'm going to hit it with like send 100 queries, distributed." Peter:All of a sudden, we were finding the infringements before anyone could click on it, because we would report it to Google, Bing, Yandex, Baidu. And they would remove it from the, from the search results. And then, we had a measurement system, which would check and see, if I was a human, how many seconds would it take before I found an active download? Right? You could automate it. And so, we had a dashboard that could show that, and it worked. You know, we could, we saw a decline in the number of complaints because, well, it wasn't as visible. Now, if you knew where the pirate bay was, yeah, okay. But that wasn't really what we were doing. We were looking at protecting people from getting downloads which contain malware, or something nefarious, right? And, and, so we built these systems to protect consumers, essentially.Natalia:So, is there a connection, or maybe a community behind the work that you've done in piracy and the world of copyright? Uh, any, any best practices that are shared with content creators who are equally concerned with a malware being in their content, or just the sheer, the sheer fact that someone is pirating their content?Peter:I think from a contents per- perspective, and there are several amazing organizations out there, such as the BSA, Business Software Alliance, you have the MPAA, you know, you have the RIAA, and also IACC, the International Anti-Counterfeiting Coalition. Who have just incredible guidance for their members, which are specialized. So, for example, when you look at counterfeit goods, that's a very different thing to like, say, video, because video is distributed in a diff- different way. But one thing, which I think is important is that you don't just leave your, your house open, you lock it with a key, otherwise, someone will just come in and take your stuff. Peter:So, I think the same with contents, that when we create content, we have to find a way to work not only with different organizations that are looking to protect those rights, but also assume your own responsibility of locking your door. For example, what security could you put on it? Right? To maintain it? And how could you work with law enforcement who are there to protect the law, right? There are, I think there are different things that could be considered but most of it really, I would say the best is to start with the industry association, because they are much more specialized, and can give better advice, depending on the nature of the content that the person has. Peter:But, you know, when we were looking at online piracy, it wasn't just online piracy, because, you know, Microsoft participated in something called Operation Pangea. This was an Interpol driven operation where we found that a Russian organization that was distributing software for download in the millions of dollars, we took action to dismantle their payment mechanism. So, Visa and MasterCard would stop the payment on their website. So, they moved to prescription drugs, and they started selling prescription drugs. And so, for certain, it's really not in Microsoft's mandate to do that, right? Peter:But what we did is that we provided the expertise, and the knowledge we have to law enforcement to detect these websites. There were about 10,000 of them, and then drill down to say, "What's the payment gateway?" Because that's a choke point, you know, a criminal, definitely does what he does for the money. You know, you're not gonna rob a bank if there's no money there, right? So, with that in mind, they were able to do really, massively disrupt this organization. And that's because Microsoft looks at providing its expertise, and also learning from other people's expertise, right? But to tackle this bigger problem that impacts all of us.Nic:Peter, I'd love to circle back to language for a sec here. And when you were talking about the languages that you speak, and, and the importance of understanding culture. From your perspective, do you think there are countries, language groups, ethnic groups that are disproportionately... Well, I'm trying to think of the most elegant way to say, not protected or not protected as well as they could because they speak a language that is, you know, not as prevalent? So, you know, I looked at, you know, I'd never heard of the two, the two, uh, Ghanaian languages that you had on your- Peter:Mm-hmm (affirmative). Nic:... on your profile there, I'm not even gonna say them right, but Fante and- Peter:(laughs), so, it's Fante and Twi. Nic:Fante and Twi. So- Peter:Perfect. Nic:... native Fante, and Twi, I'm, I'm assuming there's, there's hundreds of thousands, maybe even millions of speakers of those- Peter:Yeah. Yes, absolutely.Nic:... two languages?Peter:Yes, yeah. Nic:Do AI and ML systems allow for supporting people that, you know, either don't speak English, or a sort of major international language?Peter:You're touching on something, which is very near and dear to me, 'cause it's a whole different conversation. And if you look at the history of language, there's, a, a great group of seminars written about it. It's actually I think, I believe, somewhere, I read somewhere that 60% of languages are actually not written. Right? And yes, you can go and see Microsoft has, translates between say, 60 or 100 pairs of languages, and Google the same. But what about the others? What about the thousands of others, that I think there are over 6,000 languages in the world. You're right. I mean, earlier this year, if I may be personal, I'm trying to adopt a baby girl. And so, I went to Ghana to try and manage the situation, which is very slow. Peter:And when I was there, I just saw the reality that, you know, they don't have access to resources, right? Because a book costs money. And so even for AI, how would they even know what AI is? So, I think there is an increasing gap, which is taking place. We can't keep build, building bigger walls, because it's just not going to work. We gotta be, we gotta think bigger than that. And so, one of the ideas is that when we look at some of the criminals, like I've had quite a few of them, a lot of them go to the same technical universities, for example, in West Africa. Well, why is that? It's cause I think they develop skills, and then they leave, and they can't get a job. And so, they end up being pulled into a life of cybercrime. So, culture Peter:It's I think becoming an important thing is that, there is a bigger and bigger divide 'cause not as many people have access to the resources, and how can we as a community who do have access, sort of proactively contribute to that? 'Cause we can't, there's no way you can, you know, just Nigeria has 190 million people. That's a lot of people, that's a lot people. The African continent has 1.2 billion. Asia, four billion, was like, um, I think it's like, is it two, three billion? No, two billion? Something like that but it's a lot people- Nic:It's a lot. Peter:... outside, right? (laughs). And so I think, I'm glad you brought that up 'cause I think it's a- an interesting conversation that we need to develop even, even more. Natalia:So, just trying to distill some of that down. So, are, are you saying then that, uh, at least when we're looking at language, there is a greater diversity of threat actors than there are targets? That those targets are centralized more around English speakers, but because of disproportionate opportunities in other parts of the world, we see threat actors across a number of different languages, across a number of different cultures? Peter:Yes. I, I think that's, that's a goo- uh, kind of a good summary of that, but I'll probably take it a step further and say, from my vantage point, again, you know, there are many other more brilliant people out there than me, I can only speak of what I've seen. I still find there are concentrations, right? When you look at business email compromise, and you go and pick up a newspaper and say, "Show me all articles about BEC, the biggest crime right now in the world, and show me all the people who've been arrested." Guess what? They're all from one place, West Africa. Why? Because if you look at the history of that crime, BEC, it was a ruse. Before that it used to be called, it was all under the category of Advanced E-fraud, but it used to be a lottery scam. Oh, the Bill and Melinda Gates lottery, you've won $25 million, or, uh, the Nigerian prince, right?Peter:Some people call 419 which is a criminal code in Nigeria. And then it went further back, they used to send faxes. Or, a lot of people developed a culture called the Yahoo boys, right? They it called Yahoo-Yahoo. And what they do is you go on YouTube, and you search for Yahoo-Yahoo, you'll see them like there's a whole culture behind that. They're dancing, they say, "This is my Monday car, my Tuesday car." And because they're making money and their communities are not, the community helps them because they get money. The stolen money is shared, and so now it becomes harder to break that because it becomes part of a culture. And so, that's why we see a lot more there I think than for example, in the US, or in Russia or in other countries it's 'cause I think there was, there's a, they have this kind of lead way that they'd be doing it for a lot longer and have a better sense of how to be sly. Nic:It sounds like the, the principles of reducing crime apply just as generally in the cyberspace as they do too in the, the non-cyber space. Whereas if you can give opportunities and lu- you know, um, lucrative opportunities to people, to utilize the skills that they've developed, both sort of in an orthodox or in an unorthodox fashion- Peter:Mm-hmm (affirmative). Nic:... then they're gonna put those skills to good use. But if you, if you train them up and then don't give them any way of using those skills to, to go, you know, ma- make a living in a, in a positive sense, they're, they're gonna turn to other, other avenues. Sounds like in, in, in parts of West Africa, that is business email compromise.Peter:Right, it is. And if I could just add two things there, one is that, you know, when I started looking at how to address cyber, online criminality, I have to look at the physical part of it. And in the physical world, there's actually, I call them neighborhoods. You have good neighborhoods, and bad neighborhoods, right? There are some neighborhoods you go to, no one's going to pick pockets you, right? Everyone's got a nice car or whatever. The other neighborhoods you go to, and there are some shady people in the corner, probably selling drugs or something. You know, uh, I'm, I'm being very simplistic, but I'm just trying to say, there are differences in neighborhoods in the physical world, and those need to be looked at as well. Because even if you gave education or a job to someone in a bad neighborhood, because of the environmental pressure, they may not be able to leave that neighborhood because they could be pressured into it. Peter:Online it's the same, I found that you see there are clusters of criminal activities that happen. And in those virtual they're interconnected, it's like, like two, or three levels, they know each other mostly. And so, we can have this kind of, we have to think more holistically, I suppose. I'm trying to say, Nic, that, it, we also have to look at the neighborhood and how do you make sure, for example, that neighborhood they have a sports field or the streets are clean because it makes you feel good, right? There's, there are other environmental factors that I think we may need to consider in a more holistic way. We, we can move much faster that way, because there are different factors, uh, which contribute to this.Nic:So, Peter, I honestly feel like we could keep chatting for the next four hours, right? Natalia:(laughs), I know. Peter:(laughs). Nic:We, we, (laughs). We, we've already, (laughs), eaten up a, a lot of your time, and we've covered a lot of ground. I'd love to circle back one final time to, to language and really sort of ask you is, eh, maybe it's not language, but is there something that you sort of feel particularly passionate about in your career at Microsoft? What you've done so far, what you're working on, and what you hope to do moving forward, is language and opening up accessibility through language, and other sort of cultural diversity? You, you, you, spoke a lot about that in the last sort of, you know, 45 minutes. Is that, is that something that you're personally, uh, invested in, and would like to work more on in the future? And, and if not, what other areas are you, are you looking forward to in the future? Peter:It's, it's absolutely something I'm, I'm very passionate about. And within Microsoft, as an example, the company has invested a lot in diversity and inclusion and equity, and it ended last year, but I was the president of the Africans in Microsoft employee resource group, for example, which has close to a thousand people. And all of it is about helping, working in a two way street, where we help our community, who are at times new in the country. And so, don't understand the cultural differences and how do we help them better, not integrate, but be themselves. And also, allow others that don't understand that they may be a minority, but there's so much richness to that diversity and how it makes teams stronger, because then you're not all looking through the same lens and you can bring in, you know, different perspectives about it. So, I'm absolutely invested in that, not just here in the US but also, you know, the African continent. Peter:And, and I'm very fortunate to be working in a company that's actually pushing me to do that. You know, the company is, is doing amazing things when it comes to diversity and inclusion. And yes, there's room to be made, but at least they're active. Going back really quickly to what you mentioned about language and AI, when we look at the internet, the internet is still zeros and ones. So, when you look at machine learning models, a lot of it is looking for like over 250 signals, right? In a, in one site. And it's not just about the language, it's about different languages, computer code and human code. And so, the machines are bringing those two together, which can help better secure platforms. Natalia:And just as we wrap up here, is there anything you want to plug? Any resources, any groups that you'd like to share with our audience? Peter:I think for me, you know, always try and keep updated on security. So, you know, the Microsoft Security Bulletin is a, is a great source for, uh, up-to-date information. Also, I think there are many other organizations that people can search for and reach out to me on the antenna. If you're not a bad guy or girl, I'll- Natalia:(laughs). Peter:... I'll share, (laughs), we, we can, um, actually, you know, I try to mentor as many people in our industry because, eh, together we become stronger. So, do reach out if you want to. Natalia:Awesome. Thank you for that, Peter. It was great having you on the show again, and I can honestly say, we'd be happy to have you back, and it was infinitely fascinating. Peter:Thank you very much for the invitation again. And, uh, it was a pleasure participating. Natalia:By the way, [foreign language 00:38:17]. Peter:Uh, there you go. Natalia:If you ever want to. Peter:(laughs). Natalia:(laughs). Peter:(laughs). Nic:Natalia, I didn't know you speak Spanish.Natalia:(laughs). Peter:(laughs). Natalia:Well, we had a great time unlocking insights into security from research to artificial intelligence, keep an eye out for our next episode. Nic:And don't forget to tweet us @msftsecurity or mail us at securityunlockedatmicrosoft.com with topics you'd like to hear on a future episode. Until then, stay safe.Natalia:Stay secure.
3/31/2021

The Human Element with Valecia Maclin

Ep. 21
For Women’s History Month, we wanted to share the stories of just a few of the amazing women who make Microsoft the powerhouse that it is. To wrap up the month, we speak with Valecia Maclin, brilliant General Engineering Manager of Customer Security & Trust, about the human element of cybersecurity.In discussion with hosts Nic Fillingham and Natalia Godyla, Valecia speaks to how she transitioned into cybersecurity after originally planning on becoming a mechanical engineer, and how she oversees her teams with a sense of humanity - from understanding that working from home brings unique challenges, to going the extra mile to ensure that no member of the team feels like an insignificant cog in a big machine - Valecia is a shining example of what leadership should look like, and maybe humanity too.In this Episode You Will Learn:• The importance of who is behind cybersecurity protocols• How Microsoft’s Engineering, Customer Security & Trust team successfully transitioned to remote work under Valecia’s leadership• Tips on being a more inclusive leader in the security spaceSome Questions that We Ask:• What excites Valecia Maclin about the future of Cybersecurity• How does a mechanical engineering background affect a GM’s role in Infosec• How Valecia Maclin, General Manager of Engineering, Customer Security & Trust, got to where she is todayResources:Valecia’s LinkedIn:https://www.linkedin.com/in/valeciamaclin/Advancing Minorities’ Interest in Engineering:https://www.amiepartnerships.org/SAFECode:https://safecode.org/Microsoft’s TEALS:https://www.microsoft.com/en-us/tealsMicrosoft’sDigiGirlz:https://www.microsoft.com/en-us/diversity/programs/digigirlz/default.aspxNic’s LinkedIn:https://www.linkedin.com/in/nicfill/Natalia’s LinkedIn:https://www.linkedin.com/in/nataliagodyla/Microsoft Security Blog:https://www.microsoft.com/security/blog/Transcript[Full transcript can be found athttps://aka.ms/SecurityUnlockedEp21]Nic Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft security engineering and operations teams. I'm Nic Fillingham. Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft security, deep dive into the newest threat intel research and data science. Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft security. Natalia Godyla:And now let's unlock the pod. Hey Nic, welcome to today's episode. How are you doing today? Nic Fillingham:Hello Natalia, I'm doing very well, thank you. And very excited for today's episode, episode 21. Joining us today on the podcast is Valecia Maclin, general manager of engineering for customer security and trust someone who we have had on the shortlist to invite onto the podcast since we began. And this is such a great time to have Valecia come and share her story and her perspective being the final episode for the month of March, where we are celebrating women's history month. So many incredible topics covered here in this conversation. Natalia, what were some of your highlights? Natalia Godyla:I really loved how she brought in her mechanical engineering background to cybersecurity. So she graduated with mechanical engineering degree and the way she described it was that she was a systems thinker. And as a mechanical engineer, she thought about how systems could fail. And now she applies that to cybersecurity and the- the lens of risk, how the systems that she tries to secure might fail in order to protect against attacks. And I just thought that that was such a cool application of a non-security domain to security. What about yourself? Nic Fillingham:Yeah. Well, I think first of all, Valencia has a- a incredibly relatable story up front for how she sort of found herself pointed in the direction of computer science and security. I think people will relate to that, but then also we spent quite a bit of time talking about the importance of the human element in cybersecurity and the work that Valecia does in her engineering organization around championing and prioritizing, um, diversity inclusion and what that means in the context of cybersecurity. Nic Fillingham:It's a very important topic. It's very timely. I think it's one that people have got a lot of questions about, like, you know, we're hearing about DNI and diversity and inclusion, what is it? What does it mean? What does it mean for cybersecurity? I think Valecia covers all of that in thi- in this conversation and her perspective is incredible. Oh, and the great news is, as you'll hear at the end, Valecia is hiring. So if you like me are inspired by this conversation, great news is actually a bunch of roles that you can go and, uh, apply for to go and work for Valecia on her team.Natalia Godyla:On with the pod?Nic Fillingham:On with the pod. Valecia Maclin, welcome to the Security Unlocked podcast. Thank you so much for your time. Valecia Maclin:Thank you, Nic and Natalia. Nic Fillingham:We'd love to start to learn a bit about you. You're, uh, the general manager of engineering for customer security and trust. Tell us what that means. Tell us about your team, us about the amazing work that you and- and the people on your team do. Valecia Maclin:I am so proud of our customer security and trust engineering team. Our role is to deliver solutions and capabilities that empower us to ensure our customers trust in our services and our products. So I have teams that build engineering capabilities for the digital crimes unit. We build compliance capabilities for our law enforcement and national security team. And our team makes sure that law enforcement agencies are in compliant with their local regulatory responsibilities and that we can meet our obligations to protect our customers. Valecia Maclin:I have another team that provides on national security solutions. We do our global transparency centers on where we can ensure that our products are what we say they are. I have two full compliance engineering teams that build capabilities to automate our compliance at scale for our Microsoft security development lifecycle, as well as, uh, things like, uh, advancing machine learning, advancing open source security, just a wealth of enterprise wide, as well as stakeholder community solutions. Um, I could go on and on. We do digital safety engineering, so a very broad set of capabilities all around the focus and the mission of making sure that the products and services that we deliver to our customers are what we intend and say that they are Nic Fillingham:Got it. And Valencia so how does your engineering org relate to some of the other larger engineering orgs at Microsoft that are building, uh, security compliance solutions?Valecia Maclin:So our other Microsoft organizations that do that are often building those capabilities within a particular product engineering group. Um, customer security and trust is actually in our corporate, external and legal affairs function. So we don't have that sales obligation. Our full-time responsibility is looking across the enterprise and delivering capabilities that meet those broad regulatory responsibility. So again, if we think about our digital crimes unit that partners with law enforcement to protect our customers around the world, well building capabilities for them or digital safety, right? If you think about the Christ church call and what happened in New Zealand, we're building capabilities to help with that in partnership with what those product groups may need to do. So, um, so we're looking at compliance more broadly. Nic Fillingham:Got it. And does your team interface with some of the engineering groups that are developing products for customers? Valecia Maclin:Absolutely. So when you think about the work that we do in the open source security space, our team is kinda that pointy end of the spear to do, um, that assessment and identify here where some areas are that we need to put some focus and then the engineering, the product engineering groups will then and build, go and build that resiliency into the systems. Nic Fillingham:To follow up questions. One is on the podcast, we've actually spoken to some- some folks that are on your team. Uh, Andrew Marshall was on an earlier episode. We spoke with Scott Christianson, we've had other members of the digital crimes unit come on and talk about that work, just a sort of a sign post for listeners of the podcast. How does Andrew's work, uh, fit in your organization? How does Scott's work fit into your organization? Valecia Maclin:So, um, both Andrew and Scott are in a team, um, within my org, uh, that's called security engineering and assurance, and they're actually able to really focus their time on that thought leadership portion. So again, if you think about the engineering groups and the product teams, they have to, you know, really focus on the resiliency of the products, what our team is doing is looking ahead to think about what new threat vectors are. So if you think about the work that Andrew does, he partnered with Harvard and- and other parts of- of Microsoft to really advance thought leadership and how we can interpret adversarial machine learning. Valecia Maclin:Um, when you think about some of our other work in our open source security space, it is let's look forward at where we need to be on the edge from a thought leadership perspective, let's prototype some capabilities operationalizes, so that it's tangible for the engineering groups that then apply and then, uh, my guys will go and partner with the engineering groups and gi- and girls, right? So- so, um, we will then go and partner with the product groups to operationalize those solutions either as a part of our security, um, development life cycle, or just a general security and assurance practices. Nic Fillingham:Got it. And I think I- I can remember if it was Scott or Andrew mentioned this, but on a previous podcast, there was a reference to, I think it's an internal tool, something called Liquid. Valecia Maclin:Liquid, yes, uh, yeah. Nic Fillingham:Is that, can you talk about that? Cause we, uh, it was hinted at in the previous episode? Valecia Maclin:Absolutely. Yes. Yeah. So Liquid, um, actually have a full team that builds and sustains Liquid. It is a, um, custom built capability that allows us to basically have sensors within our built systems. Um, and so when you think about our security development life cycle, and you think about our operational security requirements, it's given us a way to automate not only those requirements, but you know, ISO and NIST standards. Um, and then that way, with those hooks into the build systems, we can get a enterprise wide look at the compliance state of our bills as they're going on. Valecia Maclin:So a developer in a product group doesn't have to think about, am I compliant with SDL? Um, what they can do is, you know, once the- the data is looked at, we can do predictive and reactive analysis and say, hey, you know, there's critical bugs in this part of the application that haven't been burned down within 30 days. And so rath- rather than a lot of manual and testation, we can do, um, compliance a scale. And I- I just mentioned manual and testation of security requirements. Oh, one of my other teams, um, has recently just launched Valecia Maclin:.. the capability that we're super excited about that leverages what we call Coach UL or used to be called Simile. That again, is automating kind of on the other edge, right? So, with liquid, it's once we pulled in the build data. Um, we're working with the engineering groups in Microsoft now to, um, do the other edge where they don't have to set up a test that they're compliant with security requirements. Um, we're, we're moving very fast to, um, automate that on behalf of the developer, so that again, we're doing security by design. Nic Fillingham:So, how has your team had to evolve and change, uh, the way that they, they work during this sort of the COVID era, during the sort of work from home? Was your team already set up to be able to securely work remotely or were there sort of other changes you had to make on the fly? Valecia Maclin:So, you know, uh, as we've been in COVID, my team does respond to phenomenally. We were actually well positioned to work from home and continue to function from home. You know, there were some instances where from an ergonomic perspective, let's get some resources out to folks because maybe their home wasn't designed for them to be there, you know, five days a week. So, the, the technical component of doing the work, wasn't the challenge. What I, as a leader continuously emphasized, and it's what, what my team needed, frankly, is making sure we stayed with the connectedness, right?Valecia Maclin:How do we continue to make sure that folks are connected, that they don't feel isolated? That, you know, they feel visibility from their, from their managers? And consider I had, I had 10 new people start in the past year, entirely through COVID including three new college hires. So, can you imagine starting your professional-Nic Fillingham:Wow.Valecia Maclin:... career onboarding and never being in the office with your peers or colleagues and, and, you know, and the connected tissue you would typically organically have to build relationships. And so through COVID, during COVID, we've had to be very creative about building and sustaining the connective tissue of the team. Making sure that we were understanding folks, um, personal needs and creating a safe space for that. You know, I was a big advocate way back in August where I said, Hey folks, you know, 'cause the sch- I knew the school year was starting. And even though we hadn't made any statements yet about when returned to work would, you know, would advanced to, I made a statements to my team of, Hey, it's August, we've been at this for a few months. It's not going anywhere anytime soon. Valecia Maclin:So, I don't want us carrying ourselves as if we're coming back to the office tomorrow. Let's, you know, give folks some space to reconcile what this is gonna look like if they have childcare, if they have elder care, if they're just frozen from being in- indoors this amount of time. Let's make sure that we're giving each other space for that. Also during the past year, you know, certainly we had, I would say, parallel once in a generation type events, right?Valecia Maclin:So, we had COVID, but we also had, uh, increased awareness, you know, of, of the racial inequities in our country. And for me as a woman of color that's in cybersecurity, I've spent my entire career being a, a series of first, um, particularly at the executive table. And so, you know, so it was a, an opportunity we also had in the past year to advance that conversation so that we could extend one another grace, right? So I personally was touched by COVID. I, I lost five people in the past year. Um, and I was also-Nic Fillingham:I'm so sorry. Valecia Maclin:Yeah. (laughs) And you keep showing up, right? And I was personally touched as a black woman who once again, has to be concerned about, you know, I have, uh, I have twin nephews that are 19, one's autistic and the other is not, but we won't allow him to get a driver's license yet 'cause he, my, my sister's petrified because, you know, that's a real fear that a young man who's 6'1", sweetest thing you would ever see, soft-spoken, um, but he's 6'1". He has, you know, dreadlocks in his hair or locks. He would hate to hear me say they were dreads. He has locks in his hair. Um, and he dresses like a 19 year old boy, right?Valecia Maclin:But on spot, that's not what the world sees. And so, um, that's what we're all in. Then you think about what's happening now with our Asian-American community. That's also bundled with folks who are human, having to be isolated and endorse, which that's not how humanity was designed. And so we have to remember that that shows up. And, and when you're in, in the work of security, where you're always thinking about threat actors, and I often say that some of our best security folks have kind of some orthogonal thinking that's necessary to kind of deal with the different nuances.Valecia Maclin:When you, when you are thinking about how do you build resiliency against ever evolving threats, (laughs) not withstanding the really massive one that, you know, was the next one we, we dealt with at the end of the last calendar year. Those are all things that work in the circle. And I always say that people build systems, they don't build themselves. And in this time more than ever, hopefully, as security professionals, we're remembering the human element. And we're remembering that the work that we do, um, has purpose, which is, you know, why I entered this space in, in the first and why I've spent my career doing the things I've done is because we have a phenomenal responsibility increasingly in a time of interconnectedness from a technology perspective to secure our way of life. Nic Fillingham:Wow. Well, on, on that note, you talked about sort of why you went into security. I'd love to sort of, I'd love to go there. Would you mind talking us through how you sort of first learnt of security and, and why you're excited about it, and how you made the decision to, to go into that space? Valecia Maclin:Absolutely. So, mine actually started quite awhile ago. I was majoring in mechanical engineering and material science, uh, at Duke university. I was in my junior year and, um, I should preface it with, I did my four year engineering degree in three and a half years. So, my, my junior year was pretty intense. I worked, was working on a project for mechanical engineering that I'd spent about seven hours on and I lost my data.Nic Fillingham:Ah!Valecia Maclin:I was building a model, literally, I sat at the computer because, you know, you know, back then, you know, there weren't a whole lot of computer resources, so you try to get there early and, and, and snag the computer so that you could use it as long as you needed to. I went in actually, on a holiday because I knew everybody would be gone. So, if I, I could have the full day and not have to give up the computer to someone. So, I'd spend seven hours building this model and it disappeared. Valecia Maclin:And it was the, you know, little five in a 10 floppy, I'm pulling it out, I'm looking at the box (laughs). It's gone. The, the, the model's gone. I was gonna have to start all over. I started my homework over again, but then I said, I will never lose a homework assignment like that again. So, I went and found a professor in the computer science school to agree to do an independent study with me, because as a junior, no one was gonna allow me to change my major for mechanical engineering that far in, at Duke University. So, (laughs) not, not my parents, anyway. So, I, um, did an independent study in computer science and taught myself programming. So, I taught myself programming, taught myself how to understand the hardware with, with my professors help, of course. But it was the work I did with that independent study that actually led to the job I was hired into when I graduated. Valecia Maclin:So, I've never worked as a mechanical engineer. I immediately went into doing national security work, um, where I worked for companies that were in the defense industrial base for the United States. And so I, I started and spent my entire career building large scale information systems for, you know, the DOD, for the intelligence community, and that vectored into my main focus on large, um, security systems that I was developing, or managing, or leading solutions through. So, it started with loss data, right? (laughs) You know, which is so apropos for where we are today, but it started with, you know, losing data on a software, in a software application and me just being so frustrated Valecia Maclin:Straight and said, that's never gonna happen to me again (laughs) that, um, that led me to pursue work in this space. Natalia Godyla:How did your degree in mechanical engineering inform your understanding of InfoSec? As you were studying InfoSec, did you feel like you were bringing in some of that knowledge? Valecia Maclin:One of the beautiful things and that was interesting is I would take on new roles, I'll, I'll never forget. Um, I, I got wonderful opportunities as, as my career was launched and folks would ask me, well, why are you gonna go do that job? You've never done that before, you know, do you know it? (laughs) And so what that taught me is, you know, you don't have to know everything about it going in, you just need to know how to address the problem, right? So, I consider myself a systems thinker, and that's what my mechanical engineering, um, background provided was look at the whole system, right? And so how do you approach the problem? And also because I also had a material science component, we studied failures a lot. So, material failure, how that affected infrastructure, you know, when a bridge collapse or, or starts to isolate. Um, so it was that taking a systems view and then drilling down into the details to predictively, identify failures and then build resiliency to not have those things happen again. Is that kind of that, that level of thinking that played into when I went into InfoSec. Natalia Godyla:That sounds incredibly fitting. So, what excites you today about InfoSec or, or how has your focus in InfoSec changed over time? What passions have you been following? Valecia Maclin:So, for me, it's the fact that it's always going to evolve, right? And so, you know, obviously the breaches make the headlines, but I'm one, we should never be surprised by breaches, just like we shouldn't be surprised by car thefts or home invasions, or, you know, think about the level of insurance, and infrastructure, and technology, and tools and habits (laughs) that we've, uh, we've developed over time for basic emergency response just for our homes or our life, right? Valecia Maclin:So, for me, it's just part of the evolution that we have, that there's always gonna be something new and there's always gonna be that actor that's gonna look to take a shortcut, that's gonna look to take something from someone else. And so in that regard, it is staying on the authence of building resiliency to protect our way of life. And so I, I am always passionate and again, it's, it's likely how I, you know, spent almost, you know, over 27 years of my career is protecting our way of life. But protecting it in a way where for your everyday citizen, they don't have to go and get the degree in computer science, right? Valecia Maclin:That they can have confidence in the services and the, the things that they rely on. They can have confidence that their car system's gonna break, that the brakes are gonna hit, you know, activate when they hit it. That's the place I wanna see us get to as it relates to the dependency we now have on our computer systems, and in our internet connected devices and, and IOT and that sort of thing. So, that's what makes me passionate. Today it may look like multi-factored authentication and, you know, zero trust networks, but tomorrow is gonna look like something completely different. And what I, where I'd love to see us get is, you know, think about your car. We don't freak out about the new technologies that show up in our car, you know, 'cause we know how, we, we, we get in and we drive and, and we anxiously await some people.Valecia Maclin:I, I'm kind of a control freak, I wanna still drive my car. I don't want it to drive itself (laughter). Um, but nevertheless, with each, you know, generational evolution of the car, we didn't freak out and say, Oh my gosh, it's doing this now. If we can start to get there to where there's trust and confidence. And, and that's why I love, you know, what my org is responsible for doing is, you know, that there's trust and confidence that when Microsoft, when you have a Microsoft product or service, you, you, you can trust that it's doing what you intend for it to do. And, and that's not just for here, but then, you know, when you're again, whether it's the car, or your refrigerator, or your television, that's where I'd love to, that's where I want to see us continue to evolve. Not only in the capabilities we deliver, but as a society, how we expect to interact with them. Natalia Godyla:Are you particularly proud of any projects that you've run or been part of in your career? Valecia Maclin:I am. And it's actually what led me to Microsoft, I had my greatest career success, but it, it came also at, at a time of, of, of my greatest personal loss. Literally they were concurrent on top of each other. And so I was responsible, I was the, the business executive responsible for the cybersecurity version of, of, of the JEDI program. Uh, so I was the business executive architecting our response to that work that was what the department of Homeland Security. I worked for a company that at the time wasn't known for cybersecurity, and so it was a monumental undertaking to get that responsibility. And the role was to take over and then modernize the cybersecurity re- system responsible for protecting the .gov domain. So, it was tremendously rewarding, especially in the optic that we have today. I received the highest award that my prior company gives to an individual. Valecia Maclin:I was super proud of the team that I was able to lead and, and keep together during all the nuances of stop, start, stop, start that government contracting, um, does when there's protests. But during that same time, you know, 'cause it was, so it was one of those once in a career type opportunities, if you've ever done national security work, to actually usher an anchor in a brand new mission is how we would label it, um, that you would be delivering for the government. But at the same time, that, that wonderfully challenging both technically and from a business perspective scenario was going on, I, in successive moments, lost my last grandparent, suddenly lost my sister. 12 months later, suddenly lost my mother, six months later had to have major surgery. So, that all came in succession while I was doing this major once in a career initiative that was a large cyber security program to protect our government. Valecia Maclin:And I, I survived, (laughs) right? So, um, the, the program started and did well, but I, I then kind of took a step back, right? Once I, I, uh, I'd promised the company at the time of the government that I would, I would give it a year, right? I would make sure the program transitioned since we'd worked so hard to get there. And then I took a step back and said, Hmm, what do I really wanna do? This was a lot (laughs). And so I did take a step back and got a call from Microsoft, actually, um, amongst some other companies. Uh, I thought it was gonna take a break, but clearly, um, others had, had different ideas. And so, um, (laughter) I had, I had multiple opportunities presented to me, but what was so intriguing and, and what drew me to Microsoft was first of all, the values of the company. You know, I'm a values driven person and the values, um mean a lot and I'm gonna come back to that in a moment. Valecia Maclin:But then also I, I mentioned that the org I lead is in corporate external and legal affairs. It's not within the product group. It's looking at our global obligations to securing our products and services from a, not just a regulatory perspective, but not limited by our, our sales target. And so the ability to be strategic in that way is what was intriguing and what, what drew me. When you think about the commitments the company has made to its employees and to its vendors during a time, um, that we've been in, it says a lot about the fabric of, of who we are to take that fear of employability insurance and those sorts of things that are basic human needs, to recall how early on we still had our cafeteria services going so that they could then go and provide meals for, for students who would typically get school meals. And at the same Valecia Maclin:... time it meant that those vendors that provide food services could continue to do their work. When you think about our response to the racial inequity and, and justice, social justice initiative, and the commitments were not only, not only made, but our, our keeping is the fabric of the company and the ability to do the work that I'm passionate about, that, that drew me here. Nic Fillingham:You talked about bringing the human element to security. What does that mean to you and how have you tried to bring that sort of culturally into your organization and, and, and beyond?Valecia Maclin:So, if you think about the human element of security, the operative word is human. And so as humans, we are a kaleidoscope of gender, and colors, and nationalities and experiences. Even if you were in the same town, you have a completely different experience that you can bring to bear. So, when I think about how I introduce, um, diversity, equity and inclusion in the organization that I lead, it is making sure that we're more representative of who we are as humans. And sometimes walking around Redmond, that you don't always get that, but it's the, you know, I, I come from the East Coast. So, you know, one of the going phrases I would use a lot is, I'm not a Pacific Northwestner or I don't have this passive aggressiveness down, I'm pretty direct (laughs). And so that's a different approach, right, to how we do our work, how we lean in, how we ask questions. Valecia Maclin:And so I am incredibly passionate about increasing the opportunities and roles for women and underrepresented minorities, underrepresented, uh, minorities in cybersecurity. And so we've been very focused on, you know, not just looking at internal folks that we may have worked on, worked on another team, you know, for years, and making sure that every opportunity in my organization is always opened up both internally and externally. They're always opened up to make sure that we're, we're looking beyond our mirror image to, um, hire staff. And it's powerful having people that think the same way you do, because you can coalesce very quickly. But the flip side of that is sometimes you can lose some innovation because everybody's seeing the same thing you see. And, and it's so important in, in security because we're talking about our threat actors typically having human element, is making sure that we can understand multiple voices and multiple experiences as we're designing solutions, and as we're thinking about what the threats may be. Natalia Godyla:So, for women or, uh, members of minority groups, what guidance do you have for them if they're not feeling empowered right now in security, if they don't know how to network, how to find leaders like yourself, who are supporting DNI? Valecia Maclin:One of the things I always encourage folks to do, and, and I mentor a lot is, just be passionate about who you are and what you contribute. But what I would say, uh, Natalia, is for them to take chances, not be afraid to fail, not be afraid to approach people you don't know, um, something that I got comfortable with very early as if I was somewhere and heard a leader speak on stage somewhere, or I was, uh, you know, I saw someone on a panel internally or externally, I would go up to them afterwards and introduce myself and ask, you know, would you be willing to have a career discussion with me? Can I get 30 minutes on your calendar? And so that was just kind of a normal part of my rhythm, which allowed me to be very comfortable, getting to meet new executive leaders and share about myself and more importantly, hear about their journeys. Valecia Maclin:And the more you hear about other's journey, you can help cultivate a script for your own. And so, so that's what I often encourage 'cause a lot of times folks are apr- afraid, particularly women and, and minorities are afraid to approach to say, think, well, you know, I don't know enough, or I don't know what to ask. It can be as simple as, I heard you speak, I would love to hear more about your story. Do you have time? Do you have 20 minutes? And then let, you know, relationships start from there and let the learning start from there. Nic Fillingham:As a leader in the security space, as a leader at Microsoft, what are you excited about for the future? What what's sort of coming in terms of, you know, it could be cultural change, it could be technology innovation. What, what are you sort of looking and seeing in the next three, five, 10 years? Valecia Maclin:For me it the cultural change. I'm looking forward and you heard me kind of allude to a little bit of this of, you now have the public increasingly aware of what happens when there's data loss. I'm so excited to look forward to that moment when that narrative shifts and the public learns and knows more of security hygiene, cyber security hygiene. And, and not, you know, both consumer and enterprise, because we take for granted that enper- enterprises have nailed this. And, and we're in a unique footing as a company to have it more part of our DNA, but not every company does. And so that's what I'm looking forward to for the future is the culture of that young person in the midst of schooling, not having to guess about what a cybersecurity or security professional is, much like they don't guess what a lawyer or a doctor is, right? So, that's what I look forward to for the future. Nic Fillingham:Any organizations, groups that you, you know, personally support or fans of that you'd also like to plug? Valecia Maclin:Sure. So, I actually support a, a number of organizations. I support an organization called Advancing Minorities in Engineering, which works directly with historically black colleges and universities to not only increase their learning, but also create opportunities to extend the representation in security. I also am a board member of Safe Code, which is also focused on advancing security, design, hygiene across enterprises, small midsize and large businesses. And so, so those are, are certainly, uh, a couple of, of organizations that, you know, I dedicate time to.Valecia Maclin:I would just encourage folks, you know, we have TEALS, we have DigiGirlz. everyone has a role to play to help expand the perception of what we do in the security space. We're not monolithic. The beauty of us as a people is that we can bring our differences together to do some of the most phenomenal, innovative things. And so that would be my ask is in, whatever way fits for where someone is, that they reach out to someone and make that connection. I v- I very often will reach down and, uh, I'll have someone, you know, a couple levels down and say, Oh my gosh, I can't believe you called and asked for a one-on-one. Valecia Maclin:So, I don't wait for folks to ask for a one-on-one with me. I, I'll go and ping and just, you know, pick someone and say, Hey, you know, I wanna, I just wanna touch base with you and see how you're doing and see what you're thinking about with your career. All of us can do that with someone else and help people feel connected and seen. Natalia Godyla:And just to wrap here, are you hiring, are there any resources that you want to plug or share with our audience, might be interested in continuing down some of these topics? Valecia Maclin:Absolutely. Thank you so much. Um, so I am hiring, hiring data architects, 'cause you can imagine that we deal with high volumes of data. I'm hiring software engineers, I'm hiring, uh, a data scientist. So, um, data, data, and more data, right?Natalia Godyla:(laughs).Valecia Maclin:And, um, and software engineers that are inquisitive to figure out the, the right ways for us to, you know, make the best use of it. Natalia Godyla:Awesome. Well, thank [crosstalk 00:35:11] you for that. And thank you for joining us today, Valecia.Valecia Maclin:Thank you, Natalia. Thank you, Nic. I really enjoyed it.Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us @msftsecurity or email us at securityunlocked@microsoft.com with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure.
3/24/2021

Identity Threats, Tokens, and Tacos

Ep. 20
Every day there are literally billions of authentications across Microsoft – whether it’s someone checking their email, logging onto their Xbox, or hopping into a Teams call – and while there are tools like Multi-Factor Authentication in place to ensure the person behind the keyboard is the actual owner of the account, cyber-criminals can still manipulate systems. Catching one of these instances should be like catching the smallest needle in the largest haystack, but with the algorithms put into place by the Identity Security team at Microsoft, that haystack becomes much smaller, and that needle, much larger.On today’s episode, hostsNic Fillingham and NataliaGodyla invite back Maria Puertos Calvo, theLeadDataScientistin Identity Security and Protection at Microsoft,to talk with us about how her team monitors such amassive scale of authentications on any given day.Theyalsolookdeeper into Maria’s background and find out what got her into the field of security analytics andA.I. in the first place, and how her past in academiahelpedthattrajectory.In this Episode You Will Learn:• How the Identity Security team uses AI to authenticate billions of logins across Microsoft• Why Fingerprints are fallible security tools• How machine learning infrastructure has changed over the past couple of decades at MicrosoftSome Questions that We Ask:• Is the sheer scale of authentications throughout Microsoft a dream come true or a nightmare for a data analyst?• Do today’s threat-detection models share common threads with the threat-detection of previous decades?• How does someone become Microsoft’s Lead Data Scientist for Identity Security and Protection?Resources:#IdentityJobs at Microsoft:https://careers.microsoft.com/us/en/search-results?keywords=%23identityjobsMaria’s First Appearance on Security Unlocked, Tackling Identity Threats with A.I.: https://aka.ms/SecurityUnlockedEp08Maria’s Linkedin: https://www.linkedin.com/in/mariapuertas/Nic’s LinkedIn:https://www.linkedin.com/in/nicfill/Natalia’s LinkedIn:https://www.linkedin.com/in/nataliagodyla/Microsoft Security Blog:https://www.microsoft.com/security/blog/Transcript[Full transcript can be found at https://aka.ms/SecurityUnlockedEp20]Nic Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft security engineering and operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft security, deep dive into the newest threat intel, research, and data science. Nic Fillingham:And profile some of the fascinating people working on Artificial Intelligence in Microsoft security. Natalia Godyla:And now, let's unlock the pod.Nic Fillingham:Hello, Natalia. Welcome to episode 20 of Security Unlocked. This is, uh, an interesting episode. People may notice that your voice is absent from the... This interview that we had with Maria Puertos Calvo. How, how you doing? You okay? You feeling better?Natalia Godyla:I am, thank you. I'm feeling much better, though I am bummed I missed this conversation with Maria. I had so much fun talking with her in episode eight about tackling identity threats with AI. I'm sure this was equally as good. So, give me the scoop. What did you and Maria talk about?Nic Fillingham:It was a great conversation. So, you know, this is our 20th episode, which is kind of crazy, of Security Unlocked, and we get... We're getting some great feedback from listeners. Please, send us more, we want to hear your thoughts on the... On the podcast. But there've been a number of episodes where people contact us afterwards on Twitter or an email and say, "Hey, that guest was amazing," you know, "I wanna hear more." And Maria was, was definitely one of those guests who we got feedback that they'd love for us to invite them back and learn more about their story. So, Maria is on the podcast today to tell us about her journey into security and then her path to Microsoft. I won't give much away, but I will say that, if you're studying and you're considering a path into cyber security, or you're considering a path into data science, I think you're gonna really enjoy Maria's story, how she sort of walks through her academia and then her time into Microsoft. We talk about koalas and we talk about the perfect taco.Natalia Godyla:Yeah, to pair with the guac which she covered the first time around. Now tacos. I feel like we're building a meal here. I'm kind of digging the idea of a Security Unlocked recipe book. I, I think we need some kind of mocktail or cocktail to pair with this.Nic Fillingham:Yeah, I do think two recipes might not be enough to qualify for a recipe book. Natalia Godyla:Yeah, I mean, I'm feeling ambitious. I think... I think we could get more recipes, fill out a book. But with that, I, I cannot wait to hear Maria's episode. So, on with the pod?Nic Fillingham:On with the pod.Nic Fillingham: Maria Puertos Calvo, welcome back to the Security Unlocked podcast. How are you doing?Maria Puertos Calvo:Hi, I'm doing great, Nic. Thank you so much for having me back. I am super flattered you guys, like, invited me for the second time.Nic Fillingham:Yeah, well, thank you very much for coming back. The episode that we, we, we first met you on the podcast was episode eight which we called Tackling Identity Threats With AI, which was a really, really popular episode. We got great feedback from listeners and we thought, uh, let's, let's bring you back and hear a bit more about your, your own story, about how you got into security, how you got into identity, how you got into AI. And then sort of how you found your way to Microsoft. Nic Fillingham:But since we last spoke, I want to get the timeline right. Did you have twins in that period of time or had the twins already happened when we spoke to you in episode eight?Maria Puertos Calvo:(laughs) No, the twins had already happened. They-Nic Fillingham:Got it.Maria Puertos Calvo:I think it's been a few months. But they're, they are nine, nine months old now. Yeah.Nic Fillingham:Nine months old. And, and the other interesting thing is you're now in Spain.Maria Puertos Calvo:Yes.Nic Fillingham:When we spoke to you last, you were in the Redmond area or is that right?Maria Puertos Calvo:Yes, yes. The... Last time when we, we spoke, I, I was in Seattle. But I was about to make this, like, big trip across the world to come to Spain and, and the reason was, actually, you know, that the twins hadn't met my family. I am originally from Spain, and, and my whole family is, is here. And, you know, because of COVID and everything that happened, they weren't able to travel to the US to see us when they were born. So, my husband and I decided to just, like, you know, do a trip and take them. And, and we're staying here for a few months now. Nic Fillingham:That's awesome. I've been to Madrid and I've been to... I think I've only been to Madrid actually. Where, where... Are you in that area? What part of Spain are you in?Maria Puertos Calvo:Yes, yes. I'm in Madrid. I'm in Madrid. I, I'm from Madrid.Nic Fillingham:Aw- awesome. Beautiful city. I love it. So, obviously, we met you in episode eight, but if you could give us, uh, a little sort of mini reintroduction to who you are, what's your job at Microsoft, what does your... What does your day-to-day look like, that'd be great.Maria Puertos Calvo:Yeah. So, I am the lead data scientist in identity secure and protection, identity security team who... We are in charge of making sure that all of the users who use, uh, Microsoft identity services, either Azure Active Directory or Microsoft account, are safe and protected from malicious, you know, uh, cyber criminals. So, so, my team builds the algorithms and detections that are then put into, uh, protections. Like, for example, we build machine learning for risk based authentication. So, if we... If our models think an authentication is, is probably compromised, then maybe that authentication is challenged with MFA or blocked depending on the configuration of the tenet, et cetera. Maria Puertos Calvo:So, my team's day-to-day activities are, you know, uh, uh, building new detections using new data sets across Microsoft. We have so much data between, you know, logs and APIs and interactions b- between all of our customers with Microsoft systems. Uh, so, so, we analyze the data and, and we build models, uh, apply AI machine learning to detect those bad activities in the ecosystem. It could be, you know, an account compromised a sign-in that looks suspicious, but also fraud. Let's say, like, somebody, uh, creates millions of spammy email addresses with Microsoft account, for example to do bad things to the ecosystem, we're also in charge of detecting that.Nic Fillingham:Got it. So, every time I log in, or every time I authenticate with either my Azure Active Directory account for work or my personal Microsoft account, that authentication, uh, event flows through a set of systems and potentially a set of models that your team owns. And then if they're... And if that authentication is sort of deemed legitimate, I'm on my way to the service that I'm accessing. And if it's deemed not legitimate, it can go for a challenge through MFA or it'll be blocked? Did, did I get that right?Maria Puertos Calvo:You got that absolutely right.Nic Fillingham:So, that means... And I think we might've talked about this on the last podcast, but I still... I... As a long-term employee of Microsoft, I still get floored by the, the sheer scale of all this. So, there's... I mean, there's hundreds of millions of Microsoft account users, because that's the consumer service. So, that's gonna be everything from X-Box and Hotmail and Outlook.com and using the Bing website. So, that's, that's literally in the hundreds of millions realm. Is it... Is it a billion or is it... Is it just hundreds of millions?Maria Puertos Calvo:It depends on how you count them. Uh, if it's per day, it's hundreds of millions, per month I think it's close to a billion. Yes, for... Of users. But the number of authentications overall is much higher, 'cause, you know, the users are authenticating in s- in s- many cases, many, many times a day. A lot of what we evaluate is not only, like, your username and password authentications, there's also the, you know, the model authe- authentication particles that have your tokens cash in the application and those come back for request for access. So, the... We evaluate those as well. Maria Puertos Calvo:So, it's, uh... It's actually tens of billions of authentications a day for both the Microsoft account system and the Azure Active Directory system. Azure Active Directory is also a... Really big, uh, it's almost... It's, it's getting really close to Microsoft account in terms of monthly, monthly active users. And actually, this year, with, you know, COVID, and everybody, you know, the... All the schools, uh, going remote and so many people going to work from home, we have seen a huge increase in, in, in monthly active users for Azure Active Directory as well.Nic Fillingham:And do you treat those two systems separately? Uh, or, or are they essentially the same? It's the same anomaly detection and it's the same sort of models that you'd use to score and determine if a... If an authentication attempt is, is, uh, is legitimate or, or otherwise?Maria Puertos Calvo:It's, like, theoretically the same. You know, like, we, we use the same methodology. But then there are different... The, the two systems are different. They live in different places with different architectures. The data that is logged i- is different. So, these, these were initially not, you know... I- identity only, uh, took care of those two systems, like, a few years ago, before they w- used to be owned by different teams. So, the architecture underneath is still different. So, we still have to build different models and maintain them differently and, you know, uh, uh, tune them differently. So, so it is more work, but, uh, the, the theory and the idea, their... How we built them is, is very similar.Nic Fillingham:Are there some sort of trends that have, you know, appeared, having these two massive, massive systems sort of running in parallel but with the same sort of approach? What kind of behaviors or what kind of anomalies do you see detected in one versus the other? Do they sort of function sort of s- similar? Like, similar enough? Or do you see some sort of very different anomalies that appear in one system and, and not another.Maria Puertos Calvo:They're, interestingly, pretty different. Uh, when we see attack spikes and things like that, they don't always reflect one or the other. I think the, the motivation of the people that attack enterprises and organizations, it's, it's definitely from the, the hackers that are attacking consumer accounts. I think they're, you know, they're so in the black market separately, and they're priced separately, you know, and, and differently. And I think they're, they're generally used for different purposes. We see sometimes spikes in correlation, but, but not that much.Nic Fillingham:Before we sort of, uh, jump in to, to your personal story into security, into Microsoft, into, into data science, is the... You know, these... Talking about these sheer numbers, talking about the hundreds of millions of, of authentications, I think you said, like, tens of billions that are happening every day. Is that a dream for a data scientist to just have such a massive volume of data and signals at your fingertips that you can use to go and build models, train models, refine models? Is that, you know... Is this adage of more signal equals better, does that apply? Or at some point do you now have challenges of too much signal and you're now working on a different set of problems?Maria Puertos Calvo:That's a great question. It is an absolute dream and it's also a nightmare. (laughs) So, yeah. It is... It... And I'll tell you why for both, right? Like, a... It is a great dream. Like, obviously, you bet... The, the sheer scale of the data, the, you know, the, the fact... There are a lot of things that are easier, because sometimes when you're working with data and statistics, you have to do a lot of things to estimate if, Maria Puertos Calvo:... it's like the things that you're competing are statistically significant, right? Like, do I have enough data to approach that this sample, it's going to be, uh, reflection of reality, and things like that. With the amount of data that we have, with the amount of users that we have, it's the, we don't have that, we, we don't really have that problem, right? Like we are able to observe, you know, the whole rollout without having to, to figure out if what we're seeing, you know, it's similar to the whole world or not. Maria Puertos Calvo:So that's really cool. Also, because we're, you know, have so many users, then we also have, you know, we're a big focus for attackers. So, so we can see everything, you know, that happens in, in, in the cybersecurity world and like the adversary wall, we can find it in, in our data. And, and that is really interesting. Right. It's, it's really cool. Nic Fillingham:That sounds fascinating. But let, let, let's table that for a second. 'Cause I'd love to sort of go back in time and I'd love to learn about your journey into security, into sort of computer science, into tech, where did it all start? So you grew up in Madrid, is that right? Maria Puertos Calvo:Yes. I grew up in Madrid and when I was finishing high school and I was trying to figure out like, why do I do, I just decided to study telecommunication engineering, it's what's called a Spain, but it's ev- you know, the, the equivalent who asked degrees electrical engineering. Because I was actually, you know, really, really interested in math and science and physics. They were like my favorite subjects in high school. I was pretty, really good at it actually. Maria Puertos Calvo:And, but at the same time, I was like, well, this, you know, an engineering degree sounds like something that I could apply all of this to. And the one that seems like the coolest and the future and like I, I, is electrical engineering. Like I, at that time, computer science was also kind of like my second choice, but I knew that in electrical engineering, I could also learn a lot of computer science. Maria Puertos Calvo:It w- it has like a curriculum that includes a lot of computer science, but also you learn about communication theory and, you know, things like how do cell phones work? And how does television work? And you can learn about computer vision and image processing and all, all kinds of signal processing. I just found it fascinating. Maria Puertos Calvo:So, so I, I started that in college and then when I finished college, it was 2010. So it was right in the middle of the great recession, which actually hits Spain really, really, really badly when it came to the, the labor market, the unemployment back then, I think it was something like 25%-Nic Fillingham:Wow.Maria Puertos Calvo:... and people who were getting out of school, even in engineering degrees, which were traditionally degrees that would have, you know, great opportunities. They were not really getting good jobs. People, only consulting firms were hiring them, um, and, and really paying really, really little money. It was actually pretty kind of a shame. So I said, what, what, what should I do? And I, I had been a good student during college, so, and I had a professor that, you know, he, that I had done my kind of thesis with him and his research group. Maria Puertos Calvo:And he said, "Hey, why didn't you just like, continue studying? Like, you can actually go for your PhD and, because you have really good grades, I'm sure you can just get it full of finance. You can get a scholarship that will like finance, you know, four years of PhD. And you know, that way you don't have to pay for your studies, but also you kind of like, you're like a researcher and you have, uh, like money to live." And I was like, well, that sounds like a really good plan.Nic Fillingham:Sounds good.Maria Puertos Calvo:Like I actually, yeah. So, so I could do in that. And, and I, you know, then my master said, this masters say, wasn't computer science, but it was very pick and choose, right? Like, like you could pick your branch and what classes you took. And so the master's was the first half of the PhD was basically getting all your PhD qualifying courses, which also are equivalent to, to doing your masters. Maria Puertos Calvo:So I picked kind of like the artificial intelligence type branch, which had a lot of, you know, classes on machine learning and learn a lot of things that are apply that are user apply machine learning, it's like, uh, natural language processing and speech and speaker recognition and biometrics and computer vision. Basically, all kinds of fields of artificial intelligence, where, where in the courses that I took. And, and I really, really fou- found it fascinating. There wasn't, you know, a data science degree back then, like now everybody has a data science degree, but this is like 10 years ago. Uh, at least, you know, in Spain, there wasn't a data science degree.Maria Puertos Calvo:But this is like the closest thing, uh, that, and that was my first contact with, uh, you know, artificial intelligence and machine learning. And I, I loved it. And, and then I did my masters thesis on, uh, kind of like, uh, biometrics in, in terms of applying statistical models to forensic fingerprints to, to understand if a person can be falsely, let's say, accused of a crime because their fingerprint brand only matches a fingerprint that is found in a crime scene. Maria Puertos Calvo:So kind of try to figure out like, how likely is that. Because there have been people in the past that having wrongly convicted, uh, because of their fingerprints have been found in a crime scene. And then after the fact they have found the right person and then, you know, like, uh, it's not a very scientific method, what is followed right now. So that, that was a really cool thing too, that then I never did anything related to that in my life, but, but it was a very cool thing to study when I was in, in school. Nic Fillingham:Well, that, that's fair. I've, I've got some questions about that. That's fascinating. So how did you even stumble upon that as a, as a, as a, as a research focus? Was there a, a particular case you might've read in the, in the news or something like, I, I think I've never heard of people being falsely accused or convicted through having the same fingerprints, I guess, unless you're an identical twin. Maria Puertos Calvo:Mm-hmm (affirmative). (laughs) Actually, I can tell you because I have identical twins, but also that, because I studied a lot of our fingerprints is that identical twins do not have the same fingerprints.Nic Fillingham:Wow.Maria Puertos Calvo:Uh, because fingerprints are formed when you're in the womb. So they're not, they're not like a genetic thing. They happen kind of like, as a random pattern when, when your body is forming in the womb, and they happen, they're different. Uh, so, so humans have unique fingerprints and that's true, but the problem with the, the finger frame recognition is that, it's very partial, and is very imperfect because the, the late latent, it's called the latent fingerprint, the one that is found in a crime scene is then recovered, you know, using like some powder, and it's kind of like, you, you just found some, you know, sweaty thing and a surface, and then you have to lift that from there. Right. Maria Puertos Calvo:And, and that has imperfections in, and it only, it's not going to be like a full fingerprint. You're going to have a partial fingerprint. And then, then you, basically, the way the matching works is using this like little poin- points and, and bifurcations of the riches that exist in your fingerprint. And, and then, you know, looking at the, the location and direction of those, then they're matched with other fingerprints to understand if they're the same one or not. But the, because you don't have the full picture, it is possible that you make a mistake. Maria Puertos Calvo:The one case that it's been kind of really, really famous actually happened with the Madrid bombings that happened in 2004, where, you know, they, they blew up, uh, some trains and, and a couple of hundred people died. Then they, they actually found a fingerprint in one of the, I don't remember, like in the crime scene and it actually match in the FBI fingerprint database. It matched the fingerprint of a lawyer from Portland, Oregon, I believe it's what it was. And then he was initially, you know, uh, I don't know if you ended up being convicted, but, but you know, it wasn't-Nic Fillingham:He was a suspect.Maria Puertos Calvo:... it was a really famous case. Yes. I think he was initially convicted. And then, but then he was not after they found the right person and they, they actually found that yeah, both fingerprints, like the, the guy whose fingerprint it really was. And these other guys, they, their fingerprints both match the crime scene fingerprint, but that's only because it was only a piece of it. Right. You, you don't put your finger, like, you don't roll it left to right. Like when you arrive at the airport, right. That they make you roll your finger, and lay have the whole thing it's, you're maybe just, you know, the, the, the criminal fingerprint is, is very small.Nic Fillingham:Was that a big part of the, the research was trying to understand how much of a fingerprint is necessary for a sort of statistically relevant or sort of accurate determination that it belongs to, to the, to the right person?Maria Puertos Calvo:Yeah. So the results of the research they'd have some outcome around, like, depending on how many of those points that are used for identification, which are called minutia, depending on how, how many of those are available, it changes the probability of a random match with a random person, basically. So the more points you have, the less likely it is that will happen. Nic Fillingham:The one thing, like, as, as we're talking about this, that I sort of half remember from maybe being a kid, I don't know, growing up in Australia is don't koalas have fingerprints that are the same as humans. Did I make that up? Do you know anything about this? Maria Puertos Calvo:(laughs) I'm sure, I have no idea. (laughs) I have never heard such a thing. Nic Fillingham:I have a-Maria Puertos Calvo:Now I wanna know. Nic Fillingham:...I'm gonna have to look this up.Maria Puertos Calvo:Yeah.Nic Fillingham:I have a feeling that koa- koalas, (laughs) have fingerprints that are either very close to or indistinguishable from, from humans. I'm gonna look this one up. Maria Puertos Calvo:I wonder if like a koala could ever be wrongly convicted of a crime. Nic Fillingham:Right, right. So like, if I want to go rob a bank in Australia, all I need to do is like, bring a koala with me and leave the koala in the bank after I've successfully exited the bank with all the gold bars in my backpack. And then the police would show up and they arrest the koala and they'd get the fingerprints and they go, well, it must be the koala. Maria Puertos Calvo:Exactly. Nic Fillingham:This is a foolproof plan. Maria Puertos Calvo:(laughs)Nic Fillingham:I'm glad I discussed this with you on the podcast. Thank you, Marie, for validating my poses.Maria Puertos Calvo:Now, now you can't publish this.Nic Fillingham:Oh, we talked about fingerprints. Oh, crumbs you're right. Yeah. Okay. All right. We have to edit this out of the, (laughs) out of there quick. Maria Puertos Calvo:(laughs)Nic Fillingham:Um, okay. I didn't realize we had talked so much about fingerprints. That's my fault, but I found that fascinating. Thank you. So what happens next? Do you then go to Microsoft? Do you come straight out of your education at university in Madrid, straight to Microsoft? Maria Puertos Calvo:Kind of and no. So what happens next is that while I, I finished the master's part of this PhD, and at this time I'm actually dating my now husband, and he's an American, uh, working in Washington D.C. as an electrical engineer. So I, you know, I finished my master's and my, I say, why, why do I kind of wanna go be in the US uh, so I can be with him. And, you know, I have the space, the scholarship they'll actually lets me go do research abroad and you know, like kind of pays for it. So Maria Puertos Calvo:Find, um, another research group in the University of Maryland, College Park, which is really, really close to, to DC. And, and I go there to do research for, uh, six months. So, I spent six months there also doing research. Uh, also using, uh, machine learning for, for a different around iris recognition. And, you know, the six months went by and I was like, "Well, I want to stay a little longer," like, "I, you know, I really like living here," and I extended that, like, another six months. I... And at that point, you know, I wasn't really allowed to do that with my scholarship, so I just asked my professor to, you know, finance me for that time. And, and, uh, and at that time, I decided, like, you know, I, I actually don't think I wanna, like, pursue this whole PHD thing. Maria Puertos Calvo:So, so I stayed six more months working for him, and then I decided I, I, I'm not a really big fan of academia. I went into research in, in grad school in Spain mostly because there weren't other opportunities. I was super, you know, glad I did 'cause I, I love all the research and the knowledge that I gained with all... You know, with my master's where I learned everything about Artificial Intelligence. But at this point, I really, really wanted to go into industry. Uh, so I applied to a lot of jobs in a lot of different companies. You know, figuring out, like, my background is in biometrics and machine learning. Things like that. Data science is not a word that had ever come to my mind that I was or could be, but I was more, like, interested in, like, you know, maybe software roles related to companies that did things that I had a similar background in.Maria Puertos Calvo:For like a few months, I was looking in... I, I didn't even get calls. And I had no work experience other than, you know, I had been through college and grad school. So, I had... You know, and, and I was from Spain and from a Spanish university, and there was really nothing in my resume that was, like, oh, this is like the person we need to call. So, nobody called me. (laughs) And, and then one day, uh, I, I received a LinkedIn message from a Microsoft recruiter. And she says, "Hey, I have... I'm interested in talking to you about, uh, well, Microsoft." So I said, "Oh, my God. That sounds amazing." So, she calls me and we talk about it, and she's like, "Yeah, there's like this team at Microsoft that is like run mostly by data scientists and what they do is they help prevent fraud, abuse, and compromise for a lot of Microsoft online services." Maria Puertos Calvo:So, they, they basically use data and machine learning to do things like stopping spam for Outlook.com, doing, like, family safety like finding, like, things on the web that, that should be, like, not for children. They were also doing, like, phishing detection on the browser. Um, like phishing URL detection on the browser and a co- compromise detection for Microsoft Account. And so I was like, "Sure, that sounds amazing." You know? "I would love to be in the process." And I was actually lying because I did not want to move to Seattle. (laughs) Like, at that time, I was so hopeful that I will find a job at, you know, somewhere in DC on the east coast, which is like closer to Spain and where, where we lived in. But at the same time, you know, Microsoft calls and you don't say no mostly when nobody else is calling you. Maria Puertos Calvo:Um, so, so I said, "Sure, let's, you know, I, uh... The, the least I can do is, like, see how the interview goes." So, I did the phone screen and then I... They, they flew me to Seattle and I had seven interviews and a lunch inter- and a lunch kind of casual interview. So, it was like an eight hour interview. It was from 9:00 to 5:00. And, you know, everything sounded great, the role sounded great. Um, the, the team were... The things that they were doing sounded super interesting. And, to my surprise, the next day when I'm at the airport waiting for my flight to, to go back to DC, the recruiter calls me and says, "Hey, you, you know, you passed the interview and we're gonna make you an offer. You'll have an offer in the... In the mail tomorrow." I was like, "Oh, my God." (laughs) "What?" Like, I could not... This... It's crazy to me that this was, like, only seven years ago, it... But yeah.Nic Fillingham:Oh, this is seven... So, this was 2014, 2013?Maria Puertos Calvo:Uh, actually, when I did the interview, it was... It was more, more... It was longer. It was 2012. Nic Fillingham:2012. Got it.Maria Puertos Calvo:And then I... And then starting my Microsoft in 2013.Nic Fillingham:Got it.Maria Puertos Calvo:I started as a... I think at that time, they called us analysts. But it was funny because the, the team was very proud on the, the fact that they were one of the first teams doing, like, real data science at Microsoft. But there were too many teams at Microsoft calling themselves, and basically only doing, like, analytics and dashboards and things like that. So, because of that, the team that I was in was really proud, and they didn't want to call themselves data scientists, so they... I don't know. We called ourselves, like, analysts PMs, and then we were from that to decision scientists, uh, which I never understood the, the name. (laughs) Uh, but yeah. So, that's how I started.Nic Fillingham:Okay, so, so that first role was in... I heard you say Outlook.com. So, were you in the sort of consumer email pipeline team? Is that sort of where that, that sat?Maria Puertos Calvo:Yeah. Yeah, so, uh, the team was actually called safety platform. It doesn't exist anymore, but it was a team that provided the abuse, fraud, and, and, like, malicious detections for other teams that were... At the time, it was called the Windows live division.Nic Fillingham:Yes.Maria Puertos Calvo:So, all the... All the teams that were part of that division, they were like the browser, right? Like, Internet Explorer, Hotmail, which was after named Outlook.com. And Microsoft Account, which is the consumer ecosystem, we're all part of that. And our team, basically, helped them with detections and machine learning for their, their abusers and fraudsters and, and, you know, hackers that, that could affect their customers. So, my first role was actually in the spam team, anti-spam team. I was on outbound, outbound spam detection. So, uh, we will build models to detect when users who send spam from Outlook.com accounts out so we could stop that mail basically.Nic Fillingham:And I'd loved to know, like, the models that you were building and training and refining then to detect outbound spam, and then the kinds of sort of machine learning technology that you're, you're playing today. Is there any similarity? Or are they just worlds apart? I mean, we are talking seven years and, you know, seven years in technology may as well be, like, a century. But, you know, is there common threads, is there common learnings from back there, or is everything just changed?Maria Puertos Calvo:Yes, both. Like, there, there are, obviously, common threads. You know, the world has evolved, but what really has evolved is the, the, the underlying infrastructure and tools available for people to deploy machine learning models. Like, back then, we... The production machine learning models that were running either in, like, authentication systems, either in off- you know, offline in the background after the fact, or, or even for the... For the mail. The Microsoft developers have to go and, like, code the actual... Let's say that you use, like, I don't know, logistic regression, which is a very typical, easy, uh, machine learning algorithm, right? They had to, like, code that. They had to, you know... There wasn't like a... Like, library that they could call that they would say, "Okay, apply logistic regression to, to this data with these parameters. Maria Puertos Calvo:Back then, it was, like... People had to code their own machine learning algorithms from, like, the math that backs them, right? So, that was actually... Make things so much, you know, harder. They... There weren't, like, the tools to actually, like, do, like, data manipulation, visualization, modeling, tuning, the way that we have so many things today. So, that, you know, made things kind of hard. Nothing was... Nothing was, like, easy to use for the data scientists. It... There was a lot of work around, you know, how do you... Like, manual labor. It was like, "Okay, I'm gonna, like, run the model with these parameters, and then, like, you know, b- based on the results, you would change that and tweak it a little bit. Maria Puertos Calvo:Today, you have programs that do that for you. And, and then show you all the results in, like, a super cool graph that tells you, uh, you know, like, this is the exact parameters you need to use for maximizing this one, uh, you know, output. Like, if you want to maximize accuracy or precision or recall. That, that is just, like, so much easier.Nic Fillingham:That sounds really fascinating. So, Maria, you now... You now run a team. And I, I would love to sort of get your thoughts on what makes a great data scientist and, and what do you look for when you're hiring into, into your team or into sort of your, your broader organization under, uh, under identity. What perspectives and experience and skills are you trying to sort of add in and how do you find it? Maria Puertos Calvo:Oh, what a great question. Uh, something that I'm actually... That's... The, the answer of that is something I'm refining every day. The, you know, the more, uh, experience I get and the more people I hire. I, I feel like it's always a learning process. It's like, what works and what doesn't. You know, I try to be open-minded and not try to hire everybody to be like me. So, that's... I'm trying to learn from all the people that I hire that are good. Like, what are their, you know... What's, like, special about them that I should try to look in other people that I hire. But I would say, like, some common threads, I think, it's like... Really good communication skills. Maria Puertos Calvo:Like, o- obviously the basics of, you know, being... Having s- a strong background in statistical modeling and machine learning is key. Uh, but many people these days have that. The, the main knowledge is really important in our team because when you apply data science to cyber security, there are a lot of things that make the job really hard. One of them is the, the data is... What... It's called really imbalanced because there are mostly, most of the interactions with, with the system, most of the data represents good activities, and the bad activities are very few and hard to find. They're like maybe less than 1%. So, that makes it harder in general to, to, to get those detections. Maria Puertos Calvo:And the other problem is that you're in an adversarial environment, which means, you know, you're not detecting, you know, a crosswalk in, in a road. Like, it's a typical problem of, of computer vision these days. A crosswalk's gonna be a crosswalk today or tomorrow, but if I detect an attacker in the data today and then we enforce... We do something to stop that attacker or to... Or to get them detected, then the next day they might do things differently because they're going to adapt to what you're doing. So, you need to build machine learning models or detections that are robust enough that use, use what we call features or, or that look at data that it's not going to be easy... Easily gameable. Maria Puertos Calvo:And, and it's really easy to just say, "Oh, you know, there's an attack coming from, I don't know, like, pick a country, like, China. Let's just, like, make China more important in our algorithm." But, like, maybe tomorrow that same attacker just fakes IP addresses Maria Puertos Calvo:Addresses in, in a bot that, that is not in China. It's in, I don't know, in Spain. So, so, you just have to, you know, really get deep into, like, what it means to do data science in our own domain and, and, and gain that knowledge. So, that knowledge, for me, is, is important but it's also something that, that you can gain in the job. But then things like the ability to adapt and, and then also the ability to communicate with all their stakeholders what the data's actually telling us. Because it's, you know... You, you need to be able to tell a story with the data. You need to be able to present the data in a way that other people can understand it, or present the results of your research in, in a way that other people can understand it and really, uh, kind of buy your ideas or, or what you wanna express. And I think that that is really important as well.Nic Fillingham:I sort of wanted to touch on what role... Is there a place in data science for people that, that don't have a sort of traditional or an orthodox or a linear path into the field? Can you come from a different discipline? Can you come from sort of an informal education or background? Can you be self-taught? Can you come from a completely different industry? What, what sort of flexibility exists or should there exist for adding in sort of different perspectives and, and sort of diversity in, in this particular space of machine learning?Maria Puertos Calvo:Yes. There are... Actually, because it's such a new discipline, when I started at Microsoft, none of us started our degrees or our careers thinking that we wanted to go into data science. And my team had people who had, you know, degrees in economics, degrees in psychology, degrees in engineering, and then they had arrived to data science through, through different ways. I think data science is really like a fancy way of saying statistics. It's like big data statistics, right? It's like how do we, uh, model a lot of data to, like, tell us to do predictions, or, or tell us like what, how the data is distributed, or, or how different data based on different data points looks more like it's this category or this other category. So, it's all really, like, from the field of statistics.Maria Puertos Calvo:And statistics is used in any type of research, right? Like, when you... When people in medicine are doing studies or any other kind of social sciences are doing studies, they're using a lot of that, and, and they're more and more using, like, concepts that are really related to what we use in, in data science. So, in that sense, it's, it's really possible to come to a lot of different fields. Generally, the, the people who do really well as data scientists are people who have like a PhD and have then this type of, you know, researching i- but it doesn't really matter what field. I actually know that there, there are some companies out there that their job is to, like, get people that come out of PhD's programs, but they don't have like a... Like a very, you know, like you said, like a linear path to data science, and then, they kind of, like, do like a one year training thing to, like, make them data scientists, because they do have, like, the... All the background in terms of, like, the statistics and the knowledge of the algorithms and everything, but they... Maybe they're, they've been really academic and they're not... They don't maybe know programming or, or things that are more related to the tech or, or they're just don't know how to handle the data that is big. Maria Puertos Calvo:So, they get them ready for... To work in the industry, but the dat- you know, I've met a lot of them in, in, in, in my career, uh, people who have gone through these kind of programs, and some of them are PhDs in physics or any other field. So, that's pretty common. In the self-taught role, it's also very possible. I think people who, uh, maybe started as, like, software engineers, for example, and then there's so much content out there that is even free if you really wanna learn data science and machine learning. You can, you know, go from anything from Coursera to YouTube, uh, things that are free, things that are paid, but that you can actually gain great knowledge from people who are the best in the world at teaching this stuff. So, definitely possible to do it that way as well.Nic Fillingham:Awesome. Before we let you go, we talked about the perfect guacamole recipe last time because you had that in your Twitter profile.Maria Puertos Calvo:Mm-hmm (affirmative). (laughs)Nic Fillingham:Do you recall that? I'm not making this up, right? (laughs)Maria Puertos Calvo:I do. No. (laughs)Nic Fillingham:All right. So, w- so we had the perfect guacamole recipe. I wondered what was your perfect... I- is it like... I wanted to ask about tacos, like, what your thoughts were on tacos, but I, I don't wanna be rote. I don't wanna be, uh, too cliché. So, maybe is there another sort of food that you love that you would like to leave us with, your sort of perfect recipe?Maria Puertos Calvo:(laughs) That's really funny. I, I actually had tacos for lunch today. That is, uh... Yeah. (laughs)Nic Fillingham:You did? What... Tell me about it. What did you have?Maria Puertos Calvo:I didn't make them, though. I, I went out to eat them. Uh-Nic Fillingham:Were they awesome? Did you love them?Maria Puertos Calvo:They were really good, yeah. So, I think it's-Nic Fillingham:All right. Tell us about those tacos.Maria Puertos Calvo:Tacos is one of my favorite foods. But I actually have a taco recipe that I make that it's... I find it really good and really easy. So, it's shrimp tacos.Nic Fillingham:Okay. All right.Maria Puertos Calvo:So, it's, it's super easy. You just, like, marinate your shrimp in, like, a mix of lime, Chipotle... You know those, like, Chipotle chilis that come in a can and with, like, adobo sauce?Nic Fillingham:Yeah, the l- it's got like a little... It's like a half can. And in-Maria Puertos Calvo:Yeah, and it's, like, really dark, the sauce, and-Nic Fillingham:Really dark I think. And in my house, you open the can and you end up only using about a third of it and you go, "I'm gonna use this later," and then you put it in the fridge.Maria Puertos Calvo:Yes, and it's like-Nic Fillingham:And then it... And then you find it, like, six months later and it's evolved and it's semi-sentient. But I know exactly what you're talking about.Maria Puertos Calvo:Exactly. So that... You, you put, like, some of those... That, like, very smokey sauce that comes in that can or, or you can chop up some of the chili in there as well. And then lime and honey. And that's it. You marinate your shrimp in that and then you just, like, cook them in a pan. And then you put that in a tortilla, you know, like corn preferably. But you can use, you know, flour if that's your choice. Uh, and then you make your taco with the... That shrimp, and then you put, like... You, you pickle some sliced red onions very lightly with some lime juice and some salt, maybe for like 10 minutes. You put that on... You know, on your shrimp, and then you can put some shredded cabbage and some avocado, and ready to go. Delicious shrimp tacos for a week night.Nic Fillingham:Fascinating. I'm gonna try this recipe. Maria Puertos Calvo:Okay.Nic Fillingham:Sounds awesome.Maria Puertos Calvo:Let me know.Nic Fillingham:Maria, thank you again so much for your time. This has been fantastic having you back. The last question, I think it's super quick, are you hiring at the moment, and if so, where can folks go to learn about how they may end up potentially being on your team or, or being in your group somewhere?Maria Puertos Calvo:Yes, I am actually. Our team is doubling in size. I am hiring data scientists in Atlanta and in Dublin right now. So, we're gonna be, you know, a very, uh, worldly team, uh, 'cause I'm based in Seattle. So, if you go to Microsoft jobs and search in hashtag identity jobs, I think, uh, all my jobs should be listed there. Um, looking for, you know, data scientists, as I said, to work on fraud and, and cyber security and it's a... It's a great team. Hopefully, yeah, if you're... If that's something you're into, please, apply.Nic Fillingham:Awesome. We will put the link in the show notes. Thank you so much for your time. It's been a great conversation.Maria Puertos Calvo:Always a pleasure, Nic. Thank you so much. Natalia Godyla:Well, we had a great time unlocking insights into security, from research to Artificial Intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us @msftsecurity or email us at securityunlocked@microsoft.com with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure.