Protecting Your APIs with the FireTail Library

Timo Rüppell

‍

Cool cool stream has started. So let's begin.

‍

Ciarán Finn 

‍

Hello, everyone. I thank you for joining us for our webinar session today on protecting APIs with the FireTail open source libraries. My name is Ciarán and I'm a developer at FireTail. I have

been here since about March this year and as developer I've worked with will work with many

apis. They're kind of hard to avoid. So when I was looking at the Fire tail open source

library and how that fits in with APIs. I had questions about how, maybe, it works, what I can achieve, how it performs lots of things like this. 

‍

So when I started talking to our VP of product Timo here, we decided it seemed like a pretty good idea for a webinar kind of session as it's probably a bunch of questions that other developers that there are also going to have and so before moving on. Timo, do you want to say hello? 

‍

Timo Rüppell

‍

Yeah, thanks. Thanks Ciarán for that sort of Ingress. Yeah, my name is Timo. I'm the VP of product here at FireTail. I'm originally a classically trained theoretical physicist. So black holes string theory that sort of stuff and after I got my PhD, I decided to look for something hard to do and writing maintainable JavaScript seems like the ticket. So that's where I ended up. Doing JavaScript for startups and all the way up the challenge rating now designing services and and open source libraries indeed. 

‍

So one of the things that I'm particularly proud of FireTail is the open source library that we have. Um, they indeed are our approach to fixing some of the shortcomings that we think are present in traditional ways to protect APIs and hope to go through some of those examples and use cases today and give you a good idea of what you can actually do with those libraries.

‍

Ciarán Finn

‍

Sounds good. So yeah, just a quick overview of what we're gonna kind of touch on and hopefully answer questions on and the next kind of a little while. We have our introduction which we're kind of down through now and then we're gonna get on to the importance of API security why it's something that is worth having libraries installed for in that kind of thing the needs for application layer visibility how we deploy and use the libraries what seamless API wrapping and protection is and the FireTail API security platform. So with that Timo, the first question I have for it is what is the importance of API security?

‍

Timo Rüppell

‍

All right. 

‍

Ciarán Finn

‍

And what's FireTail? 

‍

Timo Rüppell 

‍

That's a very good question and sort of sets up the entire premise of this call, obviously. So the reason why FireTail works in this space is that we sort of think that APIs are increasingly more

and more important They are literally everywhere like I can guarantee that anybody watching

this video is using and has used apis even today possibly many hundred hundreds possibly thousands.

A recent study showed that over 80% of all internet traffic is API calls and with this sort of extremely large attack surface comes obviously, a lot of incidences. So apis are involved in a in a line share of current attacks and current incidents because of the prevalence of API calls and and sort of speaking to that point like we we really believe that sort of security methods that are sort of Traditionally at the network layer don't really help with a lot of the incidents that we are seeing.

‍

So what we are seeing is that these incidents are usually tied to business logic flaws or

flaws in. authorization that are all dealt with at the application layer and sort of like don't really have a bearing on for example having a firewall in front of your application.

‍

Ciarán Finn

‍

I see. So yeah, you're saying the application layers kind of something that's missing from a lot of

traditional stocks. stocks. Can you go into that a bit more maybe? 

‍

Timo Rüppell

‍

Sure. Yeah, so this is a very sort of like a miniature threat modeling exercise We have listed here these the current Owasp top 10 API security issues, and we've also got like a little schematic here of a traditional API call and included our stuff like the the sort of traditional Network layer security methods that I mentioned like the the internet connection is SSL protected and there's a Gateway or proxy that does rate limiting and has a WAF in front of it for example, but then there's also the other stuff that the sort of server or application layer does which is the sanitizing validate the requests and responses and thus authorization and authentication. 

‍

Obviously there's a Handler function there somewhere and possibly calls to other APIs, but if we take this view and look at where these sort of ovas items start lining up we see that okay. We've got BOLAs which are authorization issues. We've got broken authentication and again other issues related to business logic with the BOLA and profile and obviously unrestricted resource consumption can be dealt with with rate limiting.

‍

But again, this depends a lot of how the application is built if you can simply change a

request parameter to a very high value and get access to lots of data that way it doesn't really

need like there's no real way in which a rate limiter can help you there. right? And so going through these you you start seeing the picture come sort of forming up that that a ton of these things are all sitting in the application layer with authentication and authorization being obviously the most important but these are sort of this this is a good sort of overview and sort of a mental model to have when you're thinking about. Where do you put your effort or where you need to put your effort into protecting your APIs?

‍

Ciarán Finn

‍

Nice one. Thanks Timo. so I can see there like obviously we want to focus at the kind of the application level, but where does the firetail library fit there to how do I run it? Where does it sit?

‍

Timo Rüppell

‍

That's a good question. 

‍

Ciarán Finn

‍

What happens? 

‍

Timo Rüppell

‍

So the FireTail library is intended to be used as middleware. So we've written the party library in

several different languages JavaScript python Ruby Java go we've also written modules for

engine X, for example Kraken D. 

‍

And the idea is that it is a lightweight wrapper around your server that does all of the sort of authentication authorization validation both for request and response objects is very important people usually forget to do those things for the responses and that can then obviously lead to excessive data leakage for example, but the the general point is that this is a thing that that you provide a spec for your API a definition of how the API should work and the library sits in line in the application and actually takes care of dealing with these issues that are mostly sort of otherwise strewn around in different various pieces of your applications. 

‍

You have, like, a centralized place. that is well documented dealing with these things based on a documented format.

‍

Ciarán Finn

‍

Nice one. So I think this is a question that most developers would immediately start thinking about then is like oh is this gonna slow my app down? What's the overhead here? Even for requests or just…

‍

Timo Rüppell 

‍

Sure. It's a very valid question. Obviously whenever you add an additional dependency into your application. You start increasing the sort of resource footprint. I would say that our

libraries are designed to be very efficient and not sort of do stuff that can be sort of for example done asynchronously or shunted off to a separate thread in line. And so you do not get a huge amount of overhead. Also, you should understand that when you're doing authorization and authentication with our library or you do validation and sanitization of things that you should do anyway, so using a high performing library like ours will probably save you cycles instead of doing it like yourself in every single route handler, for example again other benefits. 

‍

Like I said already before is that you have this sort of authentication authorization code and the validation, etc centralized so that also saves you saves you resources and sort of like figuring out where to put these things. Also finally the main thing that you can get as a benefit. is that the library will terminate requests early. So if there's a problem or an issue with sanitization and validation of the parameters if somebody's sending bogus requests then the early sort of blocking at the library can save you cycles on that part.

‍

Ciarán Finn

‍

Very good. So this kind of starts into the world of sort of customizing how we might use the library and that last bit you touched on a bit blocking requests early and stuff like that is their way maybe I could run this library and not do the blocking. I've misconfigured

‍

Timo Rüppell

‍

Yeah, that's a very good point and to those concerns you can definitely run the library in non-blocking mode. So that's definitely there. Then you'll just get the logs to the SaaS platform and so it can run in this sort of pure observational mode. The other thing that I should say. is that there are custom sanitization settings as well. So that you actually control also what data gets actually forwarded to our SAS platform and that way you stay in control of what happens to your application. The library is really intended to be a sort of helping hand or sort of make it. Make your development much easier instead of sort of having you in or or requiring certain ways of operating.

‍

Ciarán Finn

‍

This is going to start blocking some requests or maybe returning certain errors. Am I able to control exactly what error message gets returned and how it looks to

‍

Timo Rüppell 

‍

Yeah, so obviously the library does a lot of things out of the box pre configured. For example, incorrect content type gives you a 415 not available content type 406 an authorized 401 or for three depending on the situation. So there's a lot of thought put into the library to be as standard compliant as possible, but you have the option of standardizing your error messages  that requires a little bit more work on your part. You need to write error handlers and register them with the library on setup, but it is easy to do. 

‍

Ciarán Finn

‍

Cool, and on a basic level is this easy for me to switch on and off this Library. Can I just..?

‍

Timo Rüppell

‍

Yeah, this is a hard question. No, it's not. Yes. It's very easy. You're obviously in full control. This is just an import that you add to your project. And since it is middleware, you can just comment it out is very easy to go from using the library to not using the library obviously once you start relying more and more on the various features of the library. You may not want to switch it off, but um, that is I think not that that huge of a problem there because you're already getting a lot of benefit then out of it.

‍

Ciarán Finn

‍

Cool, so you mentioned there that it sends data off to the very top platform like blogs and stuff like that. Am I able to see them locally or within my own sort of environment or was it only…?

‍

Timo Rüppell

‍

Yeah, so technically no, but again here I would stress that these are open source libraries. So it is very easy to modify them. It doesn't take a lot of work to have them write out logs to a local file. For example, and then you're in full control and can do with the logs As You Wish. Um, also the API that our SaaS platform used is documented. So if you want you can also write a Lambo function for example that you know parses the logs first before sending them out to us so that you can for example ensure your own sanitization. And sort of stripping sensitive data. Which you can then, you know, do so in general. It's intended to be an extendable system that'll again put you in control of where to send. what data and how to use the library.

‍

Ciarán Finn

‍

Nice one. So this is like an endpoint that you'll be sending your data to what happens if it's down and we all know there's many problems with networking around the world. It's possible. It'll happen.

‍

Timo Rüppell

‍

Absolutely.

‍

Ciarán Finn

‍

What happens with…

‍

Timo Rüppell

‍

Yeah, obviously we pride ourselves on having built a very resilient and redundant infrastructure. And but as you say like Network conditions can vary so the way that the library is work is they take this into account by actually pushing the logs into a locally held async queue which means that as long as your application stays up the library can retry to send these logger winds and and the queue will be flushed based on the size of the queue or a number of items. or period of time and those can also be adjusted easily in the code and that is basically how we deal with sort of like smaller hiccups or sort of like short-term unavailability of the network, for example.

‍

Ciarán Finn

‍

A couple of years down the line my company decides they don't want to use FireTail for some reason, is everything gonna stop working. Is it gonna break entirely or…?

‍

Timo Rüppell

‍

Uh, no, so it won't stop working. You can use the libraries again as I said already earlier. It is

absolutely possible to run their library without logging to the platform. The sanitization, validation, etc. will all continue to work, you will be losing out on features of the SaaS platform, which we’ll go through in a little bit of time later in this talk, but in general there's really no constraints to using the library. You don't need to be set up with us. Obviously. We do have free trial accounts if you just want to trust it out or try it out so that way you can also, you know just start out and test it out with the library and the sales platform at the same time without even having to pay anything.

‍

Ciarán Finn

‍

Cool, so you have me convinced Timo. 

‍

Timo Rüppell

‍

 Awesome.

‍

Ciarán Finn

‍

I want to know how do I set…

‍

Timo Rüppell 

‍

Well instead of going through like a big sort of in-depth coding demo. I just want to show this on a sort of very small basic toy application to give you an idea of the sort of steps needed to set this up at the minimum level. So here we have a picture of a very sort of simplistic flask application. Where you see that the app is registered and then and then returned and here you see sort of what the additional amount of code is if you are using FireTail, so it's an import and a setup and then your basically done obviously you do need to provide the open API specification for your for your API that will then be pointed at in the settings for the app for the library and the library will then use that document to do all of its validation, etc against the API.

‍

Ciarán Finn

‍

Slowly so from that point on what's gonna happen? How does this work? 

‍

Timo Rüppell

‍

All right. So, let's take a little bit more practical look at a couple of demos here that are done in the sort of JavaScript version of the library, so this is again the open API spec and we'll show the setup for JavaScript here and sort of in a very simplistic toy application that has just a couple of routes for a pet store here. You see the equivalent setup for the sort of JavaScript side and now with a slightly more realistic application. There is a slightly larger setup object. But otherwise, it really is again a single line of additional code. where you basically add the FireTail Library as middleware after it is set up and sort of this is really the very sort of simplest way of something to an existing application here is a sort of short overview of the two operations and sort of like some toy data that this application can provide.

‍

Ciarán Finn

‍

Nice. Yeah, it's good to see kind of a more flesh that example of things like even just the API keys and that…

‍

Timo Rüppell

‍

Yeah. 

‍

Ciarán Finn

‍

…kind of thing being passed in and endpoints. I think they are read from just environment variables, which is kind of nice and configurable.

‍

Timo Rüppell 

‍

Yeah, I mean obviously like the idea is for this to be a slightly more fleshed out demo and indeed this is now Postman and you can see that like this obviously then. Works in returning returning this object

‍

Ciarán Finn

‍

Probably so you guess that's like basic setup. It's installed. It's running and all that kind of

‍

Timo Rüppell

‍

Yes. 

‍

Ciarán Finn

‍

…thing. think.We're gonna hopefully see some features of that now. 

‍

Timo Rüppell 

‍

Yes, let's actually get on with the next thing which is sort of like how to deal with authentication. Right? So we did the little threat except a threat modeling exercise at the start and we saw that authentication is a really critical thing to get correct. 

‍

And so the Open API spec has a provision for authentication and the way it works. is that

you specify the authentication schema for a route and then you provide a handler for that schema and here we can see how this how the library will then use that Handler to, you know

authenticate in this case these API requests against the bearer token. that is present in the requests. So pretty pretty sort of standard stuff the nice thing about doing it this way in the

open API spec is that it makes it a lot easier to verify and see that all of your routes have

proper Authentication and then again if you're using these sort of SAS platform as well, you can see here a little preview of the sort of like additional context that you can have when these requests are logged to the platform.

‍

Ciarán Finn

‍

Nice one here, really kind of seems like a nice central feature and a nice way to see how it all works.

‍

Timo Rüppell

‍

Yeah. 

‍

Ciarán Finn

So what we

‍

Timo Rüppell

‍

Oh, so next next I think we can show a little bit of the sort of request body validation features. And sort of this again is something I'm sure you've worked on a hundreds if not thousands of Handler functions for various apis, right?

‍

Ciarán Finn

‍

Me, 

‍

Timo Rüppell

‍

…and sort of when I did that the most annoying thing was every single time You need to check whether the input parameters or the the sort of request parameters are exactly what they are or what they are supposed to be Etc, That is that is was for me the most annoying thing. So this is

like really nice here the the Open API spec lets you define those in in sort of schemas for example, here. We are defining the pet schema and you can see that that you can Define the type and format. For these parameters and then the library will do all of the rest. So here we see somebody's trying to register a giraffe and they get a they get a corresponding

error message that the name is not correct as well as other things like checking that required parameters are actually present in the request Etc. so this is I think one of my favorite features of the library. 

‍

Ciarán Finn

‍

Yeah, really shows the kind of pair of some simple config and then it being like properly addressed 

‍

Timo Rüppell 

‍

Yeah. So next I think is the the other side of the coin which I already mentioned earlier is the

response validation, So so this is a thing where I think like many people tend to forget that so you have like all of your inputs validated then you do a database request and you just push out the stuff that you get from the database straight into the response object and you send it off in your server like who hasn't done that but this is a is a disaster recipe for for having data leak, right? And so what the open API spec can do and is being demonstrated here is that you can set this additional properties false property on the schema and that will allow the library to then verify that what is sent out from The endpoint will actually conform exactly to the schema. So even though for example these pets here have an owner ID present, right or we we try to add like a full object here Fuki value here. It'll actually excuse me. It'll actually verify that the that the thing that is returned from the API the response will actually conform to the proper specification and then obviously like it'll also check for the required objects as well in a similar way to the request validation. 

‍

Ciarán Finn

‍

Nice, that's kind of a way of stopping you leaking out extra data that maybe should the user should never know is there.

‍

Timo Rüppell 

‍

Exactly like in in sort of like larger development teams. Somebody does a like changes the tables in the database and they're suddenly some internal uuides present That shouldn't be leaked. This is a perfect way of stopping that Yes, okay. Now we get to the sort of maybe more complicated meat and that is authorization. So authorization is a little bit special because it is actually not taken care of by the open API spec. So the open API spec has methods for authentication and not authorization And I I know that you are familiar with the difference and I hope that if somebody in the in the sort of audience watching this is not immediately familiar with the difference. This is something that is a really good thing to educate yourself about.

So what this basically means is that you need to make sure that the that the resource that

you're returning from the endpoint in these case the pets here actually the long to the logged in user here being presented by the user with the user ID one. Um, and and this is something that if this is not done properly, this is one of the leading causes of API incidences. So in order to sort of deal with this, we've added a vendor extension to the open API spec where you can again document where the user ID sits on the response object and where the user ID sits on the request object and then the library will make sure that these match for the data that is being returned out of the API. 

‍

Ciarán Finn

Yeah, seems super useful. And yeah, it's one of those things that if it's not baked into your system, it's very easy to miss and not realize that you're again kind of leaking data. You don't want to providing information to the wrong person and all those kind of things.

‍

Timo Rüppell

‍

Yeah, exactly, and it's so easy to to do when you're developing an endpoint Handler you just look at okay, where can I get the user ID from and and suddenly you're picking the user ID out of the query parameter instead of the of the place where the sort of authentication method has put it right and that will lead to an attacker being able to spoof spoof IDs, right? And that is a that is like those things have literally happened many examples of those. Yeah, okay. 

‍

Now the to make it take it one step further, right because not all authentication and authorization systems are that simple here we have a an example where now we're not going to authenticate against the owner ID that is present on the pet. We are going to authenticate against the tag that is present on the pet. And this is sort of like it's a more modern approach to authorization. It is called attribute based access control a back a similar variant is our back role-based access control. So these usually rely On these things that are called authorization graphs. Which we are running here now at toy version of it in in open fga that basically just has a model that attaches the sort of rights to see a pet and sort of to the tag and in order for this to work. You need to provide an access resolver that will actually then asynchronously ask this authorization provider for whether this user has this correct. Access Control access right to this specific tag or

the resource with the tag, right and again since the open API spec doesn't actually deal with this. We've made a vendor extension. that will let you that'll let you specify these things And here we now see that the sort of access control works now based on the tag and not on the ID.

‍

Ciarán Finn

‍

That's pretty cool Timo. I am I noticed I had some sort of was it an authorization graph I saw in there. What's that kind of representing? 

‍

Timo Rüppell

‍

Yeah, that's a I mean that is a good thing and this is sort of like a little bit more advanced already. So if you think about authorization as a sort of concept a higher level concept, it is always about three things. It is about who wants to do something. This is usually called the principle then it is about what they want to do, which is the action and then it is about what they want to do the action too. So the resource and and authorization graph. Such as open fga or there's there's a Zanzibar for Google. There are many Cloud providers have their own implementations of these and authorization providers. Simply is a very fast database to query for these tuples for these combinations of of principle action and resource obviously, they're sort of you can think of much much more complicated systems than we have here. Like documents could be owned by groups and groups could have members that have access to documents based on their status in the group and then maybe there's even some you know, temporal limitations there. So that documents should only be editable for like a a day after they're created and like it gets really complicated. But the main idea is that these authorization graphs allow you to store this information and sort of maintain it and query it. Yeah. 

‍

Ciarán Finn

‍

Lovely, that's query. Of course. Yes. And yeah, so I think I think we've seen pretty pretty good performance of how the library kind of will integrate with different applications what I can do and we've seen a little bit of this already with some logs I think but I'm with the firetail platform itself. Can you show me some more about how that would work and kind of in sync with the library in that kind of thing.

‍

Timo Rüppell

‍

Obviously sure yeah professors really by saying that this is now all the stuff that you get in addition to using the library there. You can use the Library without our platform. Although there are again a free tiers to sign up with and test it out and the general idea. Is that our platform functions as the centralized logging platform for your libraries and for your other APIs that you might have so the the way it works is that you up Integrations once you signed up and those Integrations can be code Integrations like the libraries or they can be Cloud Integrations. So if you've got your APIs deployed on AWS or Azure or gcp there are Discovery Integrations that will find the deployed apis and then there are also logging Integrations. that will start to know ship logs to our platforms. And what you can get out of that is sort of a good overview and a centralized view of all your logs that are then searchable and filterable and have also of context data attached to them and specifically we will also do login Richmond, which means that we will do stuff that we can't do in the library because of performance right so we can't do sort of like machine learning stuff. We can't do stuff like bot detection or pii detection Etc. And so that sort

of stuff is offloaded from the sort of your environment to our Cloud environment and we take care of that and give you just the sort of enriched data back by having these logs, you know, appropriately tagged and have sort of additional details available for each of these log items. so specifically the library gets more context in this case. 

‍

So for example, here we have on the left side. We see a response object that we are logging from just a normal Gateway integration again sort of the similar response if you're working with the library because we have access to the application layer information you get sort of the for in this case the original body and then you get the actual body that was returned in this case. You can see for example the sanitization working and stripping out the owner ID. That shouldn't be present. This can also give you much greater insight into why certain requests fail for example, and sort of overall gives you a sort of nice overview of what is happening with your API requests the sort of primary feature that comes after you have sort of gained your visibility of your API and observability of your apis is then posture management. So this means that we will discover your open API specifications that are the sort of basis of how your APIs operate and then we will give you we'll basically analyze them. and we will give you security recommendations and findings about that spec. So this is a tool that you can then use either as a development team to sort of check your work or as an operations team to sort of see if there are any apis that are deployed that might have issues with them, Right? So here we have these findings that you can see some examples of that are based on things. That might be wrong or might be problematic in your open API specification. 

‍

And finally, obviously the the nicest thing is that you can then tie this into our rich notification integration system and you can sort of have a almost fully automated way of somebody pushes a change into in GitHub. There's a hook that sends the open API spec to our platform it gets analyzed. There's some issues found that weren't present in the previous version and you get a slack message about that. And so the cycle of you know having corrective movement and sort of visibility over things happening in your API deployment is really the greatest advantage of using our SaaS platform.

‍

Ciarán Finn

‍

This one. That sounds great Timo. Thanks so much. So I think we've seen a pretty good overview of what the library is capable of How It's implemented how it will actually behave when it's running and then also how it interference with the firetale library and start with the firetail platform and all the kind of lovely features. You can take advantage of that for that point. Thank you very much.

‍

Timo Rüppell 

‍

My pleasure. It was great talking to you and I do hope that everybody here

watching this got some good ideas. Maybe some interesting things to start reading up on

and as always if you're interested in these things and want to talk about them. Just

feel free to reach out to us. either on social media or at contact@firetail.io or any of our personal FireTail emails as well. Timo@firetail.io. And Ciarán, you’re ciaran@firetail.io, right? 

‍

Ciarán Finn

‍

Correct. All right. Thanks very much. 

‍

Timo Rüppell

‍

Thank you.