Tanya Janca of Semgrep

Jeremy at FireTail (00:02.672)
All right, welcome back to another episode of Modern Cyber. I'm your host Jeremy coming to you as always and looking forward to a conversation with somebody I've had the pleasure of knowing kind of virtually for a little while, but I've had the pleasure of hosting on the podcast before talking to learning from and just really enjoying the conversation. I am joined today by the one and only Tanya Janca. Tanya, thank you so much for taking the time to join us today.

Tanya Janca (00:26.334)
my gosh, thanks for having me.

Jeremy at FireTail (00:28.304)
Yes, well it is our pleasure. And for those who don't know Tanya, which I think is probably going to be like a very small subsection of our audience, Tanya is also known as SheHacksPurple, the bestselling author of Alice in Bob Learn application security. She's also the head of education and community at Semgrep, sharing content and training that revolves around teaching everyone to create secure software. Tanya has been coding and working in IT for over 25 years. She's won countless awards.

I think she's spoken at pretty much every security event that you ever would have heard of, but she is an award -winning public speaker and active blogger. She's delivered hundreds of talks across six continents. I don't know why she hasn't gotten to Antarctica yet, but she values diversity, inclusion, kindness, which shines through in her countless initiative. And she's just a lovely human being to talk to. So again, thank you, Tanya, for making the time.

Tanya Janca (01:18.062)
My pleasure. I actually had a plan to speak in Antarctica in 2020 and then COVID happened. Yeah, there was this big cruise ship going and then COVID started to break out and I just freaked out and canceled because I was worried that no one on the ship got COVID. So I would have been safe, but I'm a scaredy cat, so.

Jeremy at FireTail (01:23.471)
No way. Is that true story? that's insane.

Jeremy at FireTail (01:36.462)
I was gonna say...

Well, but especially at that time, I remember in some of the initial kind of COVID reports about the number of cases, you would sometimes see cruise ships identified as like separate countries when they were doing these assessments of number of cases per territory. And you'd see like the, you know, the SS such and such. Yeah. So I understand the decision. I probably would have made a similar decision in your shoes, although I'm really curious what kind of cruise is like a cybersecurity cruise that

Tanya Janca (01:57.357)
ship.

Jeremy at FireTail (02:08.847)
has speakers coming in and talking about application security topics.

Tanya Janca (02:11.597)
It was actually just, it was a whole bunch of speakers got together and booked it and then invited people to apply and I applied and they're like, yeah, definitely you're coming.

And so it's all technology and mostly software development. And so they're like, yes, we'll check the security box of Tanya. And then my friend Shira was gonna come who also speaks about cyber. And then as it got closer and closer, both her and I were like, is it worth potentially dying for? I don't think so. But they had a great time and they're like, we're not gonna do it again because we've all been to Antarctica now. Darn, I missed my shot.

Jeremy at FireTail (02:32.014)
Yeah, yeah. Yeah.

Jeremy at FireTail (02:41.679)
Yeah, yeah, yeah.

Jeremy at FireTail (02:49.648)
Yeah. I don't, hey, look, maybe we start a Kickstarter, you and me, and we organize round two of this so we can all get that checkbox on our speaker charts for going down there. But let's dive into a couple things today. There's a couple things that, you know, we're talking about a little bit off air before we started recording that you've mentioned and that I want to dive into. And one is around the concept of secure guardrails. And, you know,

Guardrails are something that comes up as a security, I don't know, guidance, principle, foundational element, et cetera. And I'm just curious, like from your perspective, what are they? Because I've heard them mean many things to many people.

Tanya Janca (03:34.221)
Yes, so think about a real guard wear.

think about a real guardrail on a road. So I used to be in a band and I played music. And I remember once we were leaving a concert, it was really late. And I sang to my bandmate, because all of us were going in one vehicle and him in another vehicle. And I was like, don't fall asleep. And he's like, I'm not going to blah, blah, blah. And so obviously he fell asleep. Right. And he hit the guardrail. It scared the poop out of him, which is good. He woke up. He and he just had a scratch on his car.

Jeremy at FireTail (04:01.935)
Yep. Yep. Yep. Yep. Yep.

Tanya Janca (04:09.614)
and he lived, right? It informed him of his error and helped him go back onto the road, the paved road where we want them to go. And so a lot of us talk about secure defaults or sometimes people will call it a paved road. So this is the secure way. This is the way we want you to do it. Like we want you to use this API gateway. We want you to use this form of authentication, et cetera, et cetera. So this is the way we want you to do it. We try to make it as easy as possible. But the different, so a guard rail,

The idea is when you go off the paved road, it informs you, hey buddy, you're doing something different than the rest of the company. One of these things is not like the other. What are you doing? And so it informs them and it tries to kind of nudge them back onto the road. So then they have a choice, right? And almost all of them go back onto the paved road. They're like, yeah, I'm not supposed to use, you know, the with statement. Oops, that's a boo -boo in JavaScript. I'm not supposed to do that. I'll go back and use the thing I know I'm supposed to.

Jeremy at FireTail (04:47.214)
Yeah. Yeah.

Jeremy at FireTail (05:04.365)
Yep, yep, yep.

Tanya Janca (05:09.341)
But sometimes they want to choose to do the thing. So the first time I hit a guardrail or a secure guardrail was at Microsoft. So I was making insecure demos on purpose. And so I made a connection string in my code because I wanted to demonstrate finding the secret and how this was bad. And I go to check it in with Azure DevOps and Azure DevOps is like, no, there's a secret in your code. And I was like, yeah, I know.

Jeremy at FireTail (05:37.453)
Yeah, I did it on purpose.

Tanya Janca (05:37.867)
And so I was like, please, yeah, I was like check it in anyway. And they're like, we need an explanation. So I put the explanation like, I'm doing this. And it's like, no, I still don't think you should. And so then I'm a dev. So you know what I did? I freaking crushed it and made it go in anyway. And then I pushed it to prod because it was a demo. It was in my own separate network that was like had firewalls around it. There was no data in that database, et cetera, right? So I felt I was not causing organizational risk, but.

Jeremy at FireTail (05:58.509)
Yeah. Yeah. Yeah.

Tanya Janca (06:06.954)
Imagine the viewpoint from someone else. So my boss calls me a few minutes later. He's like, Taya, what are you doing? Did you know Azure can make phone calls? I did not know. And so apparently Azure, there was this automated call that said a dev just did this. And I explained what we were doing and then we had this good laugh. He's like, ha ha, Azure's angry with you. And...

And we're joking, we thought it was very funny. And then I'm like, I have another call coming through, I'm gonna let you go. And it was the Microsoft incident response team. That is not the way you wanna meet those people. So these are normally very nice people, but they're not very impressed when you wake them up late at night for a pretend security incident. So.

Jeremy at FireTail (06:32.812)
Yeah, yeah.

Jeremy at FireTail (06:48.812)
Yeah.

Tanya Janca (06:49.898)
I had a lot of apologizing to do. But the point is, is that there I am being a dev. I just put it in anyway. And so a lot of times I'll work with companies and they're like, we're so excited to be able to break the build. We're just going to stop the devs from doing bad things. But very quickly, there'll be a false positive. So I felt that that was sort of a false positive because I knew what I was doing. I knew I wasn't causing organizational risk. I was making the demo on purpose to teach customers and the public about what not to do. Right.

And so to me, I was like, no, I'm I know I'm going off the paved road. I've been informed. I can sense to this. I'm aware of the risk and I'm accepting this risk. Right. But so as a lot of developers I see, they'll have a and it's especially static analysis tools or even software composition analysis tools where they're like, this is a false positive. I'm going to go to prod anyway. Right. And then they have enough false positives. They just turn the whole thing off.

Jeremy at FireTail (07:20.363)
Right.

Tanya Janca (07:49.659)
So a lot of security teams are like, we're going to break the build and then they're going to stop and it's going to be great. And I'm like, devs are like water. We can go around anything. Like I remember writing a decrypter so that we could just get the keys to the things and update stuff in prod because I wanted to and customers wanted things within an hour and making a request to the security team and doing all this stuff and then having it go to the DBA took 24 hours and then customers would yell at me. So we just wrote a decrypter and then

Jeremy at FireTail (07:55.147)
Yeah, yeah.

Jeremy at FireTail (07:59.852)
Yeah.

Tanya Janca (08:19.515)
we would just do it. We would break the policy so we could please the customer. And obviously that's not like a good thing. We shouldn't have done that. But it's because the security was in the way of availability. Right? And.

Jeremy at FireTail (08:30.731)
Yeah.

Tanya Janca (08:31.945)
developers will just go around whatever you've got. And so a lot of customers or people I consult with, they're very excited at first with breaking the build. And I'm like, if you break the build all the time, everyone's just going to be ticked off at you and they're going to turn your tools off. You need to be very serious when you break the build because they can go around you. And so if whenever possible, you use a guardrail to nudge them back to where you'd like them to be. So like, let's say you have,

Jeremy at FireTail (08:47.403)
Yeah. Yeah.

Tanya Janca (09:01.193)
an IDE plugin with a stack analysis tool. And then you have certain rules to enforce your coding guideline to, you know, this looks like a vulnerability, please don't do this, et cetera. And it does a little red squiggly when you go off the line.

So that would be one type of guard rule you could do then another guard rule could be okay, so I'm checking my code in Hey, there's a secret there that doesn't look so good And then maybe you have a linter run and it's like hey, you know what your API definition file It's not doing these three things that we kind of agreed we would do buddy And so you're trying to nudge them in the right direction and then by the time they're actually doing the build They're sending it to production through the release pipeline if you're breaking something that they've previously hit the guard rail on

like they've probably decided they they know they're off they're off -roading. I heard people call it off -roading but whatever it is like they're very confident that they're doing that thing and if you've warned them two or three times along the way if you break the bill they don't care right like they're like there's a reason they're doing it so I do think we should break the build on very important things but the rest of the time if we could nudge them to do the right thing I find we get better results personally.

Jeremy at FireTail (09:50.41)
Yep, yep.

Jeremy at FireTail (10:01.578)
Yeah. Yeah.

Jeremy at FireTail (10:07.722)
Yeah.

Tanya Janca (10:14.057)
especially earlier in the SDLC. Like if it's while they have the context open, they have the code open and it's like, hey, like don't use this function, use that function. It's like, okay, sure. Versus six weeks later after they've released a prod, we have like these pen test results and they're like, what was I working on?

Jeremy at FireTail (10:18.602)
Yeah.

Jeremy at FireTail (10:25.225)
Yeah, yeah.

Jeremy at FireTail (10:31.369)
Well, I think even two weeks later when it's like getting ready to go into staging or UAT, you're gonna get a lot more pushback, right? Because I've already built this code and if you're telling me I need to go back and rewrite this function or change to this library or whatever, now I've got to reconsider everything that I've worked on for the last couple of weeks. But you raise a point there that I wanna dig into because I'm really curious what your experience has been around this kind of, let's say, organizational friction.

more the organizational friction than the technical control. Because I think the technical control is pretty easy and it's even easy, conceptually easy, right? It's easy to just have the idea that like, hey, rule violations of type X will fail the build. Or, you know, these are the rules that are in that group of things that are like the, you know, the 10 commandments carved in stone that thou shall not violate them or build will fail whatever. But there's like, there's a couple of questions that come to my mind.

First one is, how do you figure out that threshold for what qualifies as fail build, do not pass go? And second is, how do you differentiate the signal that is coming out of the systems so that I, as a developer, know, no, this is a nudge control, this is a fail control? And then, kind of follow up to that is, how do you make it so that I am not going to tune out?

the fail controls over time in the same way that to your point, all these false positives would cause alert fatigue and I'm just gonna switch off mentally from paying attention to any of it.

Tanya Janca (12:07.305)
Okay, so there's, you have so many questions. Okay, so the first question, how do you decide what you fail on versus what you nudge on? So I would say I would nudge on everything. Like if it's a soft, it's like a notification, it's like there's the red squiggly underneath and I'm going to look at all the errors. I'm going to look at all the compile errors. I'm going to look at all the red squigglies and my VS code, whatever, and I'm going to check them all out. And...

Jeremy at FireTail (12:09.96)
I do, yeah.

Tanya Janca (12:34.634)
for breaking the build, I would have it generally start just be either critical or high reports, right? And where the confidence is high as well. The confidence needs to be medium or above. So if the confidence is low, no, I'm not breaking the build. If it's a medium thing, no, unless you're there, unless you already have fixed all the highs and criticals and you're working on breaking down the mediums, I wouldn't break on it. So to start. Yeah.

Jeremy at FireTail (12:41.352)
Okay.

Jeremy at FireTail (12:58.889)
Yep. And these are hyzing criticals like as defined by a vendor or hyzing criticals as defined by like the tool that you're using or you know what defines that.

Tanya Janca (13:08.841)
That's the best question ever. So one of the problems with all these automated tools, and I know I work for one of them, so.

Jeremy at FireTail (13:16.872)
And I build one too, so yeah.

Tanya Janca (13:17.885)
Yeah, exactly. Like, is it some of them don't have very much context. So the more context you can put together, the better. So what I've seen, so at one company, what I did was, is I took the results of all the different scanners and the scanners I had there weren't super accurate. They were pretty old. And there are more modern tools now that exist, but I didn't have an option. So I had a first gen SAST. I had a dynamic scanner with very little

customization and so I would have it all go into Defect Dojo which is a free tool from OWASP and then I had a policy set and it would know this app is public facing or not. It would know it contains sensitive information or not and so it would have a different more strict policy if there was higher risk associated with the app and then it would take all these results and it would basically tell me like pass or fail and I'd have the CI check that. It's like thumbs up or thumbs down. That took a long time.

Jeremy at FireTail (14:16.137)
Yep. Mm. Okay.

Tanya Janca (14:17.628)
to set up to be quite frank. But then my check in the CIA was like a couple minutes instead of like the sass was just so slow. And the DAS we just it just kept crashing. So we didn't put it in there. And so that's one thing you could do to get some context.

Jeremy at FireTail (14:28.84)
Yeah, yeah.

Tanya Janca (14:36.681)
Other context could be having a different policy for each app based on, like I said, the risk. So public facing, sensitive data, is it mission critical, et cetera? Is legislation applied to this or compliance, like PCI, et cetera? So then you know to be more strict or less strict. But on top of that,

Jeremy at FireTail (14:56.775)
Yep. Yep.

Tanya Janca (14:59.145)
tuning your tools as best you possibly can to make sure you're getting more accurate results. So for instance, I see it like a lot of companies that sell Dast and they're like, we're great with APIs. And I'm like, no, you're not. You're great with web apps. You're like, you find lots of stuff in web apps. And I totally still use one all the time for that. But unless you have like a pen tester sitting there, like in doing like a web proxy and manual testing, I find that they're not great. I'd rather use like an API specific.

Jeremy at FireTail (15:12.488)
Yeah, yeah.

Tanya Janca (15:29.051)
dynamic scanner, like an API specific fuzzer or whatever. And so adjusting your tool set so it's as accurate as humanly possible. And then I just, I only in the CI test for the super serious things I really care about. So like definitely injection, cross -eye scripting, like OWASP top 10 -esque types of vulnerabilities. I know with the new top 10, we can't test for design flaws automatically, but all the,

Jeremy at FireTail (15:30.408)
Yep. Yep.

Jeremy at FireTail (15:57.479)
Yeah, yeah, yeah, yeah, yeah.

Tanya Janca (15:58.907)
all the all the ones that like really scare me how about that

Jeremy at FireTail (16:04.199)
Yeah.

Tanya Janca (16:04.201)
And then you do rely on the tool, right? And so as soon as a false positive comes through, you need to figure out how to suppress that. And only suppress if it's a false positive. So I'm, I got banned last week on Stack Overflow like permanently because I tried to answer all the sum grab questions in one day and they did not like that. And one of the questions was, how do I suppress this result in sum grab? But then I looked at it and like, no, that's actually XSS. You need to fix that, don't suppress it. Here's how you fix it.

Jeremy at FireTail (16:19.111)
Right.

Tanya Janca (16:34.155)
they're like, you didn't answer his question. Four down votes in like two minutes. Yeah, so I'll not see you on Stack Overflow.

Jeremy at FireTail (16:36.646)
my gosh.

Jeremy at FireTail (16:41.158)
Yeah.

Jeremy at FireTail (16:47.11)
Yeah, well, fair enough. I mean, what's really funny and kind of like crazy timing around the day we're recording this episode also was a headline of how Stack Overflow is the latest place where quote unquote helpful Stack Overflow answers are actually have been found to be people attempting to plant malware in various applications.

So there are people on there who are answering questions like, hey, Tanya, you have a problem with this? you should grab the latest version such and such of this package. Let me point you to it. And that is a package that has embedded malware. Look, I tend to think that site's been in decline for a little while, unfortunately. And I know our own dev team doesn't use it nearly as much as any dev team did like three, four years ago. But yeah.

Anyway, okay, so that's one aspect of it. But then, again, if I'm the dev, how do you suggest that security teams engage with them to help them understand, like, hey, we've got this thing in place. Sometimes it's going to tell you this, and that is just a nudge. And here's how we would, we as the security team would like you to deal with it.

Tanya Janca (17:52.521)
not overwhelmed.

Jeremy at FireTail (18:09.573)
Nevermind, by the way, all the baggage that goes into that conversation that you probably already hate us and consider us the Department of No. But then like, but then there are going to be these other things that fail. And by the way, like what's in those two different buckets is different for to your point, Crown Jewel applications that are regulated or financial or healthcare or whatever, that's going to be a much more, you know, a much higher, lower threshold towards failure of a build, etc.

Because I find that that is one of the biggest challenges that you have to figure out is just kind of like, what's the communication so that it is received appropriately and responded to appropriately?

Tanya Janca (18:50.473)
Okay, excellent question. So one of the things I talk and teach about a lot is the idea of advocacy. So like connecting with software developers, building trust with them, educating them on the things you expect them to do. So I've seen a lot of security teams where they're like, this is our SaaS, that's your problem now. It's going to scan things and send you these giant reports and we plan to not help you. And guess how that goes poorly, right? So usually,

Jeremy at FireTail (19:18.118)
Yeah, yeah, yeah, terribly, of course, yeah.

Tanya Janca (19:20.426)
I try to socialize the idea that a new tool is coming. Then I have certain key developers help me choose the tool. So I gather requirements from lots of teams, so like which languages, frameworks, which CIs does it need to work with, which repos, et cetera. And then some, hopefully, developers that like me and are willing to work with the security team, who I've built trust over time, and I get them to help me choose it.

Jeremy at FireTail (19:42.981)
Yeah, yeah. Yeah.

Tanya Janca (19:47.433)
And I remember showing a SaaS tool to this group of developers and we're playing with it. And I, and you know, like I'm just talking about, and one of them said, can we, can we keep it? And then I knew I found the right one, right? They're like, we think it's great. Like you're going to buy it for us, right? I'm like, yeah, yeah, yeah. God, if you'll use it, yeah, I'll get whatever you want. Like this is so awesome. And.

Jeremy at FireTail (20:02.308)
Yeah, yeah.

Jeremy at FireTail (20:07.14)
Right, right. Yeah. Yeah. Yeah, yeah.

Tanya Janca (20:12.809)
And so then having like a workshop showing everyone how to use it, making sure I'm available for questions. The first time I had run a scan, then I would send it the results and I'd say, hey, do you want to meet and go through the results together and talk it out? Or are you like, go away, Tanya, I got this. And half of them are like, go away. And they would just fix things when they're ready. And the other half were like, meet with me. This is new. Walk me through. And then they would start fixing things. And we got a lot of stuff.

stuff fixed in the first three months, like a surprising amount with tons of communication. Like I remember I was like, okay guys, we're doing this. And they're like, my gosh, you've told us a hundred times. And I was like, yes. So I have communicated enough, right? And.

Jeremy at FireTail (20:44.356)
Yeah. Yeah.

Jeremy at FireTail (20:54.691)
Yeah, yeah. Yeah, yeah, yeah.

Tanya Janca (20:58.696)
And you want to make sure that the tool makes them feel smart and empowered, not stupid and held back. Right. And so that might mean that at first you just do the high priority checks and you get those things fixed. And then over time you add more checks and more checks over time. Because if you send them a report that's 200 pages long, I don't know about you, but like to me that feels like a slap in the face. 200, like 200 pages of bugs in my app. What? You think I'm an idiot?

Jeremy at FireTail (21:06.243)
Yeah.

Jeremy at FireTail (21:16.931)
Yeah.

Jeremy at FireTail (21:21.027)
Yeah.

Tanya Janca (21:28.603)
Right? Like, so sending them a well curated report that just shows things you're very confident about to start with. And then over time digging into more vague, you're not sure sorts of things. So I feel like so when I was a Deb, I wasn't necessarily the best communicator. And then I took a lot of management courses and communication courses because I was too direct. I would be extraordinarily direct. And I was at besides Van

Jeremy at FireTail (21:29.187)
Yeah.

Jeremy at FireTail (21:37.697)
Yeah.

Jeremy at FireTail (21:41.377)
Yep, yep, yep, yeah.

Tanya Janca (21:58.46)
this week and there are some people talking in the hallway right outside the door and my friend was giving a presentation. I just like walk up, I open the door and I'm like, come in or go talk somewhere else we can hear you, you're done now. And I was like, my my dev person's coming out but they did it right and later someone was like thank you so much like I was struggling to hear the speaker and they just didn't realize we could hear them right like I wasn't like f off but I'm like you're done now okay go or come in.

Jeremy at FireTail (22:17.698)
Yeah, yeah, yeah. Yeah. Yeah.

Yeah.

Tanya Janca (22:26.665)
Right? And they decide not to come in, which is fine. But like, it's like they couldn't make up their mind yet. Right? But I'm like, don't stand right next to the door. So sometimes I'm a little too direct with my feels. And so I had to learn to be...

Jeremy at FireTail (22:32.769)
Yeah.

Jeremy at FireTail (22:36.705)
Yep.

Tanya Janca (22:40.521)
less like learn to beat around the bush if that makes any sense and learn to soften the message and learn to repeat the message because some people need it to hear it multiple ways and like that was a big learning curve for me. But I did it and now I'm less abrupt.

Jeremy at FireTail (22:44.577)
It does.

Jeremy at FireTail (22:56.993)
Yeah.

Jeremy at FireTail (23:03.393)
Fair enough. Yeah. What's interesting in that, by the way, as you refer to this as your dev personality, I think most people would say that's a security personality. And, you know, classically, that is, you know, how a lot of people perceive security teams, especially kind of network and infrastructure security. You know, in many of the organizations I've worked with have that reputation of you're going to get dry as bone, crystal clear. It is, you know, binary zero or one fact.

Tanya Janca (23:08.97)
Yeah.

Tanya Janca (23:28.393)
Yeah.

Jeremy at FireTail (23:33.248)
type of communication at best and delivered with like no padding or any kind of fluff or niceness around it. Yeah, but go ahead.

Tanya Janca (23:43.754)
I feel like you can't do your job with AppSec if you're like that.

I feel like you're just not gonna get anywhere and maybe it's because I'm female I'm not sure but like when I was a pen tester I couldn't go up and be like your app sucks blah blah blah Like that's not gonna go well So I'd be like hey, I found something. Can we talk about it? And I remember some of the pen testers being like she takes too long She's wasting all this time like I saw her sitting and coding and I was like, yeah because I kept telling him You know doing a proof

Jeremy at FireTail (23:49.632)
Okay.

Jeremy at FireTail (24:04.095)
Mmm.

Jeremy at FireTail (24:07.711)
Yeah.

Tanya Janca (24:19.499)
list not a block list and he just kept doing block lists so I was like hey can I drive and he's like okay and then I just showed I just reversed the regex basically right and he's like right and so it turned out I just was a sucky pen tester and a much better app sec person and that's okay because I just kept doing app sec and like let's try model this and they're like there's no time for that Tanya.

Jeremy at FireTail (24:21.631)
Yeah.

Jeremy at FireTail (24:30.303)
Right, right.

Jeremy at FireTail (24:42.559)
Yeah, yeah, we've got to ship, must ship, must ship. Yeah, I turned out to be a sucky developer and that's how I ended up in security because I couldn't write good enough code. But there was another thing in what you said that I want to dig into, which is that like, when I think a lot of companies and it's like, it's really easy for me as somebody who is building a software company to fall into this trap as well. You got to remember that like the customer that you're working with, it's...

Tanya Janca (24:45.515)
My ship.

Jeremy at FireTail (25:12.161)
Yes, you want them to adopt it everywhere. And like, you know, you say, Hey, you're using our, in our case, API security tool. Awesome. Like apply it everywhere. But that may not actually be the best experience for that organization. And there actually is like a kind of a phase in process that is really important. And if you don't get it right, with the let's say with the initial implementation, and the first kind of workloads that you're onboarded with,

and then you try to make it bigger, you've just made a bad situation much larger and worse. Yeah, way worse, right? And so guess what? When it comes time for renewal and you look at the customer satisfaction scores, they're not gonna be that good. And so, you know, what you mentioned about kind of rolling a new practice out by, I think you said, you know, just start with your highs and your criticals, and then over time you can improve and improve and improve.

I think that is such a key point to bear in mind. And there was one other thing that I think you dug into that probably often goes kind of assumed, but not actually implemented in practice, which is, to the extent that security teams actually involve some of their secondary stakeholders in decision -making, I think you're gonna get much better outcomes, right? Because to your point, if you're the security team and you buy a security tool,

And the first thing that it does is just like puts all your developers on blast for how crappy their stuff is, your 200 page report, you know, no developer is gonna want to engage with that thing at all. So involving them in a decision making process, if you've got those types of relationships and you've got the opportunity to do that, can only lead to better outcomes.

Tanya Janca (26:58.443)
You know, what's funny is like when Semgrab does the sales process, we always like want to include devs and the security teams usually super resistant. And that's where we think we win, which is why we want them, right? But they're like, they're like, no, we don't, we don't need them. And it's like, yeah, but, but they're the, they're the actual user, right? Like, like you look at the dashboard and hopefully see the numbers go down, but they're the ones that have to like live and breathe it all day, every day.

Jeremy at FireTail (27:11.232)
Yeah. Yeah, yeah.

Jeremy at FireTail (27:16.768)
Yeah, yeah, because they...

Jeremy at FireTail (27:25.248)
Yeah, yeah.

Tanya Janca (27:28.35)
And so we're like, we really think like they should see it and almost none of them involve them. So if you're doing one with us, like we'd love to talk to the devs. But I just did it because I have been the person that had someone else shove the security thing down my throat that I did not like the tool. And actually one of the one of the places I went to work, there had been an AppSec guy there for a long time and he was super brilliant, but

very taken for granted. And he proudly told me in our first one -on -one meeting how he had...

Jeremy at FireTail (28:00.161)
Mmm.

Tanya Janca (28:06.091)
assessed that is not the place I work so I'm not going to name them because I don't want to ever say bad things publicly about other competitors. He had it run on every single build and it broke if there were mediums or highs and all his devs were doing so great and I was like this is awesome show me and so we look at the first one and he's like see it run and he's like it's disabled and I was like weird let's just look at another one.

Jeremy at FireTail (28:13.761)
Yep, yep.

Tanya Janca (28:34.475)
And then we ended up spending the hour realizing that people so he had spent two years manually adding it to every CI having a million meetings and all of them had been turned off. So either it didn't run at all or it ran but only an alerting and he was blocking nothing and almost all of it was just completely turned off. And then we went and looked and realized only 9 % of the developers had ever even logged in to see their bugs because you had to RDP into a server and then you had to do this and that and then you had an account etc.

Jeremy at FireTail (28:49.473)
Yeah. Yeah.

Jeremy at FireTail (28:58.465)
Yeah.

Jeremy at FireTail (29:02.369)
That's terrible user experience.

Tanya Janca (29:04.714)
Yeah, and and he I swear I so he ended up in the next couple months resigning of a broken heart He was just like he felt he's like I'm a fool. I'm too ashamed to show my face at work I'm like, no, you're awesome. It's not you and he left I was so sad because I thought he was really really smart very passionate person, but he's like my feelings are too hurt I don't want to work at a place where the devs don't respect me. And so he left I yanked that tool out. I put something else in with more true positives. Sure false positives

Jeremy at FireTail (29:12.449)
Yeah, yeah.

Jeremy at FireTail (29:26.689)
Yeah, yeah.

Yeah.

Tanya Janca (29:34.668)
a dashboard where they didn't have to hop onto five different boxes to get to it, etc. And then we start getting some results. But I just, I felt so bad for him.

Jeremy at FireTail (29:43.967)
Yeah. Yeah. Look, I mean, I think two lessons learned out of that story. One is you have to meet people where they work on a day -to -day basis. You know, every security vendor, us included to some extent, we're guilty of this as well. We love our dashboard. We love our UI. We put a lot of time and effort into making something pretty. But the truth of the matter is like most security teams, what they want to do is log in at the beginning.

go through implementation cycles, maybe some tuning, get it to where they want it to be exactly, and then wash their hands of it, walk away, and it's going to create a ticket for me, it's going to send me a Slack alert, it's going to pop up in my IDE, I don't know, it's gonna do the red squiggly underline, whatever it is that it's going to do, it should do that thing in a place that you're already going to be every day as you are doing your work and meet you where that is.

The second thing that I wanted to highlight from that story is, you know, there will be probably a temptation from some of our audience to listen to this and say like, well, you know, that person was too sensitive. That person was too, you know, took it too personally or whatever. It's just a job and you know, it's not them, it's the organization, it's the organization's fault, et cetera. But remember that your security programs are run by people at the end of the day and their job satisfaction.

directly influences your security outcomes. And so when you've got kind of broken organizational processes, whether it's communication, whether it's, I don't know, you know, this case, there was a whole lot going on, I would say, but whatever those things are, remember that like, you know, kind of that the the health, mental health and well being of the people running your security program, like I said, that has a direct correlation on your security outcomes.

Tanya Janca (31:21.77)
Yeah.

Tanya Janca (31:34.794)
Yeah, having two years of your work deleted behind your back like.

Jeremy at FireTail (31:40.736)
Yeah.

Tanya Janca (31:41.994)
I like as the meeting, like as us looking continued, I just started getting kind of uncomfortable because he wasn't like angry. He was hurt. He'd worked there 12 years. He felt these people were his peers. Like it was like betrayal. Like, and I felt upset and I had only worked there three weeks.

Jeremy at FireTail (31:53.152)
Yeah, yeah. Yeah.

Yeah, yeah, yeah.

Jeremy at FireTail (32:04.768)
Yeah. Yeah.

Tanya Janca (32:05.387)
And then he seemed so embarrassed in front of me and it was like, I still have complete respect for you. Like this is, yeah, so it's so true. I actually talk a lot about incident response and how, like if you just crush your teams with incidents all the time and they never have recovery, like guess what? They quit and they work somewhere else where people treat them well. And it's way cheaper to just not treat them like crap.

Jeremy at FireTail (32:26.559)
Yeah. Yeah, yeah. yeah. Yeah.

Tanya Janca (32:29.706)
and keep your team rather than burning them out and then having to find new ones all the time.

Jeremy at FireTail (32:35.487)
Yeah, that whole question of, and I've seen this a lot, especially at larger organizations where they need to maintain a certain amount of staffing, right? Where they've got tens of people or hundreds of people working SOCs and IR and so on, as opposed to like one or two where you can treat them individually. They'll often look at a decision like, well, I can't give Jeremy a raise as if level one incident response SOC analyst because that's going to disrupt.

the, I don't know, like the level or the band, the salary band for that position, well, I will tell you, hiring and retraining is way more expensive than retention, right? Yeah.

Tanya Janca (33:17.867)
Yeah. yeah. Yeah, we had I am I got hired once to mentor someone two to four hours per week for a year so that he could turn into an app sec professional and they were paying him something like 70 K a year. And so then at the end they give him a $10 ,000 raise and he's like, cool. I'm off demandy and for a 250 % raise by. I was just like,

Jeremy at FireTail (33:41.757)
my gosh.

Tanya Janca (33:44.01)
So it's like you really got to look at the pay scales and like what's going on in the world and talk with them. Yeah, that was scary.

Jeremy at FireTail (33:48.541)
Yeah, yeah. What's the reality? Yeah, yeah, yeah. Yeah. Well, we're coming close to time here. We've got a few minutes left. I've got I've still got a couple of questions. I have one question I kind of want to spring on you out of the blue, and I'll be very curious to hear your reaction to this. And I apologize in advance for not prepping you with this one. I recently had a conversation with the CEO of a really fast growing company in the security space who shall remain unnamed. All right. And

Tanya Janca (33:59.69)
Okay.

Jeremy at FireTail (34:18.556)
his position to me when we were chatting about what I'm working on on the API security space side. And by the way, I really appreciated all your comments earlier. For those for whom those comments resonated, please do check out FireTail. I think like all the things you said we have experienced and thought about in the design of our own product. So hopefully that comes through for you if you ever have a look at it. But.

When I talked about API security, the CEO was super dismissive and said to me, he made a comment on the order of, look, APIs are part of an application and AppSec just really isn't that important in the broad scheme of the cybersecurity landscape. And I dug into the comment with him and I was like, you're crazy because actually applications protect the data and the data is ultimately the crown jewel. So I'm...

And he brought up this counterpoint and his view was like, yeah, but if you look at where all the money goes, which is to say the spending in cybersecurity, there is way more money spent on network security and on things like infrastructure security. Like probably his, I think he said something like 20 to $1 spent on NetSec and InfraSec as opposed to AppSec. So first of all, like how do you refute that in like, you know, because you've been doing AppSec and kind of,

Tanya Janca (35:33.163)
Mm -hmm.

Tanya Janca (35:41.323)
Bye -bye.

Jeremy at FireTail (35:41.627)
you know, very well known and specialized in AppSec for a while, I happen to think he's wrong. And like the strategic importance is on the AppSec side. But I'm curious if you've gotten comments like that in the past before, and how do you, like, how do you explain to people the importance of AppSec?

Tanya Janca (36:00.14)
So I do agree with one part of what he said. Yes, organizations have traditionally spent significantly more on network security and infrastructure security. Yes, that's part of why AppSec is such a giant dumpster fire because we haven't been giving proper attention to it. And if you look at the Verizon breach report, which...

Jeremy at FireTail (36:16.154)
Okay.

Jeremy at FireTail (36:21.242)
Okay.

Tanya Janca (36:25.355)
I have looked at for many years, the number one cause of data breach is insecure software every year for many, many years. And so as a result, and data breaches are the most damaging, humiliating and expensive types of security incidents. So while phishing might be the number one cause of security incidents in general,

they're often significantly less expensive or they're part of a much bigger attack that actually is because of insecure software that the data breach occurs. And so that's part of why it's extremely important. The other part is this.

Jeremy at FireTail (36:59.258)
Mmm.

Tanya Janca (37:05.547)
So the confidentiality, integrity and availability of these data and systems are our responsibility to protect. And so if they're not available because there's a cyber attacker, because someone has been beating up those APIs with brute force attacks and there's no gateway protecting them or they didn't turn on the rate limiting or whatever, right? You can't get your work done and your job done. Right. And so I would say in having them be available, having the

Jeremy at FireTail (37:30.585)
Mmm.

Tanya Janca (37:35.453)
confidentiality of your data still protected, making sure the data you receive is actually accurate is quite important as well. I remember doing a threat model. So life labs, I don't know if you've heard of them, but they do basically all of the blood tests across Canada or a lot of them. And they had a data breach. And I remember someone saying, well, I'm not sick. So why do I care? Because I had a friend who had she had a brain tumor, but for 18 years. And so but it would show up on test results. Right.

Jeremy at FireTail (37:49.593)
Okay. Okay.

Jeremy at FireTail (37:57.369)
Yeah.

Jeremy at FireTail (38:02.713)
Mmm.

Tanya Janca (38:05.356)
And I'm like, well, we need to protect people like her, like the minority. And he's like, well, I don't have that problem. So why do I care? I'm like, you dummy. I could just go on to paste bin, copy it, add that you are HIV positive and then upload it somewhere else. And then boom, everyone in the world thinks you're HIV positive from now on. And he's like, and I'm like, and people are biased against people that are HIV positive. Like you might not get a job or you might not whatever. Right. Like or I could give you every single

type of bad health condition that exists like the dangerous ones like you have heart disease and you have everything you have all the cancers every single one of them right and like you're never gonna get insurance again right and he he was like and so when this when data gets leaked it it can literally result in death the integrity of the data can result in financial harm physical harm health harm etc like

Jeremy at FireTail (38:48.951)
Yeah.

Tanya Janca (39:04.971)
So I think that sometimes people also forget what runs software. So like if you have an insulin pump, imagine the integrity is broken. So it's still available, but it killed you. Right? And so when people are like, app sex not important. And I'm like, cool, try to live a single day without software. Just try it. Even if you're out camping.

Jeremy at FireTail (39:10.807)
Yeah.

Jeremy at FireTail (39:24.823)
Yeah, yeah, yeah.

Tanya Janca (39:28.01)
I bet you wouldn't believe how many of your little things have software in them. Or how'd you get to the campsite? Did you use Google Maps? Right. And so when they think about just how many times per day that they use software, like actually I'm looking, I have this ring. It's an hour ring and there's software in there. Right. And so when we think about all of that, yeah, that's part of, I don't know if I had the most concise answer for you, but I hope that it helped.

Jeremy at FireTail (39:34.615)
Yeah. Yeah, yeah, yeah.

Jeremy at FireTail (39:43.447)
Yep, yep, yep.

Jeremy at FireTail (39:55.125)
Well, look, I think this point about, you know, software runs in everything nowadays. And I think there was a I think it was the Marc Andreessen quote software is eating the world. I mean, it is quite literally true, aside from maybe eating, but it is powering pretty much everything in the world. And it is such a good point. You know, we have an advisor on our advisory board who wrote the book on on IoT and device security called If It's Smart, It's Vulnerable.

Tanya Janca (40:24.778)
yeah.

Jeremy at FireTail (40:24.853)
just remember like every piece of IoT equipment, every operational technology, every industrial control system, everything is running software. And these are systems that can't really be easily patched very often. You can't push over the air and update to a firmware or a controller or fix some software vulnerability. Like the importance of secure design, the importance of secure software is super, super critical. And I just...

I found it to be very dismissive in that conversation and I wanted to get your take as somebody who specializes in AppSec. So I appreciate you taking us through your thinking around that.

Tanya Janca (41:03.625)
I suspect that him and I could have a really good conversation.

Jeremy at FireTail (41:07.829)
Awesome, awesome. Well, I know there's a couple things I wanted to kind of ask you to finish up today. Number one is I hear you've got some fun stuff planned for the summer. You've recently opened a free academy. What's that all about?

Tanya Janca (41:21.993)
Yes, so I run the community for SumGraph. So SumGraph bought my company WeHack Purple last year and we had this giant community and giant academy and we're just slowly rebranding it and rebuilding it. So it's much bigger. And so the SumGraph Academy opened May 1st and it's free. So you quote unquote pay by giving us your email address and then we add you to our monthly newsletter, which me and the WeHack Purple crew write. So it's actually really fun. We had thousands of subscribers to our

Jeremy at FireTail (41:26.773)
Okay. Okay.

Tanya Janca (41:51.898)
previous newsletter, so it's actually really good. And you can just unsubscribe, but obviously you shouldn't. And so we have, so you can take courses on secure coding, functional programming, how to write rules in SumGrap, application security. I'm going to add an incident response for devs soon. We have an API security course. And so, and you get a certificate of completion whenever you finish a course. So you can show off to your friends what you've learned and the whole thing.

Jeremy at FireTail (41:57.653)
Why would you?

Jeremy at FireTail (42:13.045)
Excellent, excellent.

Tanya Janca (42:21.801)
The plan is to just have it permanently be free. And so there's that and that's awesome. And then I get to run community events. So I speak and participate in other community events like AppSec PNWs in two weeks in Vancouver. And I'm speaking at that and the SubGrab community is helping by like running their social media for them. We're running. So this summer we're running an AppSec trivia night in Las Vegas as part of the Hacker Summer Camp stuff. And so.

in between the B -Sides Las Vegas and Diana and Defcon and Black Hat, etc. There's going to be an AppSec trivia night and I believe we're going to partner with B -Sides Las Vegas. So it's sort of part of their conference, but run the SunGrab community. And I'm looking forward to being extremely silly with the trivia. I bought a 3D printer, so I'm going to print some very silly prizes.

Jeremy at FireTail (43:12.755)
Okay.

Jeremy at FireTail (43:19.315)
Excellent, excellent.

Tanya Janca (43:21.385)
Yeah, and I just...

I really like to have a good time, if that makes sense. And sometimes I feel security is a little serious and I used to do, as you know, music and I did comedy and all of this. So I want it to be sort of like, this is your like, you know, come and eat and drink, but also prepare to be very silly. Like I helped with the B -Sides Vancouver Island one that we had in 2019 and we that so we're not going to do this in Vegas. So don't worry. But we had this thing where if you answered wrong, you had to eat hot sauce.

Jeremy at FireTail (43:29.299)
yeah.

Tanya Janca (43:53.674)
I'd eat so much hot sauce that day. So whenever someone else got something wrong, we would scream hot sauce at them. And for like two years after on the island, if like someone said a fact wrong, we'd be like, hot sauce. And we just pointed them and then they have to eat hot sauce. But we won't hot sauce you this summer.

Jeremy at FireTail (43:56.787)
Hahaha

Jeremy at FireTail (44:09.883)
Fair enough, fair enough. By the way, I totally agree with your point. Like I think security is often way too serious and like it's important to have fun and to, you know, take not only pride, but take enjoyment in your work. And so I think, you know, with that, I will just say thank you again for taking so much time today. I think we went a little bit longer than we originally planned for. I appreciate the flexibility on that side.

To everybody listening, please, if you've got a moment, like, subscribe, follow, click all that good stuff, you know what I'm gonna say, rate, review, and share this episode with any of your friends. The one and only Tiny Agent Kit, thank you so much for taking the time to join us on this episode of Modern Cyber.

Tanya Janca (44:52.552)
Thank you so much for having me, Jeremy.

Jeremy at FireTail (44:55.219)
Awesome. I look forward to probably losing really badly in AppSec trivia this summer at Black Hat, but I will do my best to make it out there and participate if nothing else. And please do join us next time on another episode of Modern Cyber. Bye -bye.

‍