API Security - The Next Generation with Elastic Beam

The Prime Directive - Open more technology doors without increasing risk.

I can distinctly recall being fresh in the Federally focused aspect of my career and having a conversation with a customer where he said to me "We don't measure failure in dollars, but in the loss of lives." It was a powerful statement that has stuck with me ever since that speaks to the sensitivity of the data and the critical nature of US Federal systems. The challenge becomes that this creates a situation where disruption can be far from ideal, and risk-adverse strategies with a strong focus on security are often paramount.  And yet, recent times have brought an explosion of new technologies and capabilities with tactical and practical value that cannot be ignored. Servicemen and women look to utilize their own devices to accessing PII focused data and services, or basic applications such as time sheets, without all the hassle of going through CAC-secured desktop systems. Officers in command want rapid risk assessment dashboards on mobile devices rather than bringing a razor to "briefs" because they take two days to complete. Operational readiness needs to be real-time; confirmed via onboard self-diagnostics and sensors, not simply scheduled maintenance checks. The possibilities are endless, but the requirements for high security, low risk deployments are still ever present, as well as the ever strengthening drive to embrace more agility in policy making and implementation to keep up with evolving technology. With these new technologies, come new threats and new requirements around security. Here at Axway Federal, we have a strong focus on working with US Government agencies to enable requirements around mobility, cloud migration, API Lifecycle Management, IoT, and more, but always cognizant of the weight and challenge of ensuring we can help accomplish implementing these disruptive new requirements with a risk-adverse mindset, utilizing best in breed security.

Axway can provide best practices and sample policies (check out our new Axway Marketplace if you haven't already), continuously adapt to evolving threats, and utilize signature based protections, but all this is highly reactive rather than real time or proactive. Often we find ourselves integrating into tools such as IPS/IDS systems to provide some of these capabilities, but these work at lower levels of the OSI hierarchy than we do. We see more and more requirements coming out to provide application layer (OSI Layer 7) type proactive analytics and dashboards with rapid risk assessment for evolving attacks. While the API Gateways have the ability to build policies to do this, they are only as good as today's knowledge and the knowledge of the implementer(s), and often not even to that extent when focus is on a minimal viable product (MVP) that simply checks the boxes for security controls. There is a need for real time, adaptive security with threat discovery, but that is not a simple ask. This goes into a whole new territory beyond policy building, and into machine learning (ML) and artificial intelligence (AI).

Boldly go where no API security solution has gone before!

A short time ago I was approached by the Axway Product Management Group to explore potential integration into an organization that had an interesting new solution to address this issue. An tool that processed application layer traffic and applied machine learning to identify threats that were not being addressed by current tooling. It went beyond layer 2-4 type IPS/IDS type security to discover advanced persistent threats (APT) utilizing complex mechanisms on traffic that was targeting APIs, using authentic but stolen credentials and tokens, and learning the behavior of APIs to be able to spot anomalies. My first thought was this sounded like User Behavior Analytics (UBA), but then it really clicked; if you can apply algorithms to learn similar things about humans (UBA) and lower network traffic (IPS/IDS), why can't you do it for APIs as well? This would open a door for a far deeper and more advanced level of security addressing the most prevalent architecture style of the day, RESTful APIs, in a way that no one else is currently addressing. These would not be redundant capabilities to anything else customers are using, but an entirely new layer of security to add to the arsenal to fortify the foundation of enterprise security by securing current API security gaps, and bring a next gen tool to the market to address security of the backbone of communication for all these new technologies and disruptive paradigms... APIs.

Looking into their capabilities, they had the technology to fulfill the requirements we needed to meet. Deep API traffic inspection, backed by a scalable and powerful machine learning engine, capable to determining API behavior and identifying anomalies to uncover threats that we had not built policies to protect against. the protections weren't just on inbound traffic either, but pre and post-login attacks to identify anomalies even after the attacker was already inside. Add to that the ability to perform forensic analysis after the attacks, autonomously block attacks, and display the results via an powerful and easy to read dashboard (common, everyone loves a slick UI)... needless to say, I jumped at the opportunity to work on an integration with these front runners exploring a new frontier, Elastic Beam (EB).

(Elastic) Beam me up Scotty!

After the initial meetings of the teams and getting all the prerequisite NDAs and such in place, we began to discuss what the integration would look like. I must say it has been a great pleasure working with the team at Elastic Beam, as they very much 'get it'. In much the same way as we at Axway might advise our customers when creating RESTful APIs (API-First), we started not with the actual data modeling and integration, but with the design. We quickly identified several key performance indicators (KPI) for the proof of concept (POC):

  • Simplicity: The more complex the integration, the more room for error and the lower the ease of use. The goal was to create something that could be used with a couple clicks of the mouse. We often say in regards to policy creation in the Axway API Gateway 'configure, don't develop' as the goal of our pre-built filters. Here the aim was to take that a step further and 'Import, don't configure'.
  • Uniformity: Simplicity and uniformity really go hand in hand. We determined the ultimate solution should be environment and customer agnostic to lower the level of effort as much as possible. Anyone should be able to implement the solution with minimal to no customization. Protocols, message schemas, content-types, HTTP Verbs... it should not matter. For the end customer, no matter what their use case or API, the solution should work the same with a single supported policy set. Similarly, we defined a schema to communicate between Axway APIM and EB to ensure that the characteristics of the backend resource would have no effect on the actual integration between the solutions.
  • Flexibility: While uniformity and simplicity may be paramount, a brittle solution is far from ideal. Much as monolithic web services have seen a transition to microservices, the goal was to create capabilities where individual components could be modified without the need to entirely refactor the solution. In a perfect world, everything adheres to today's standards and one size (policy) fits all. The reality is that there would be atypical use cases that would need addressed. 
  • Granularity: An extension to the flexibility requirement. When we discuss an API:first strategy, we talk about designing the API document first to allow bidirectional development (perhaps MAD in tandem with data modeling). Similarly, for all the value the integration may bring, there may be times when global enforcement is not desirable, such as during API development before production. It might also make sense to first engage in a POC within an enterprise against a handful of APIs before rolling it out en masse. Whatever the case, we wanted to allow granularity on the customers end to choose which APIs the solution was applied to at any given time.
The Axway API Management Suite provides capabilities around creating, controlling, and consuming APIs. Part of this solution includes the ability to create an API registry for versioning, discovery, testing, and consumption within the Axway API Manager. When registering and exposing APIs, you have the option to specify custom security, request, routing, and response policies, which we may often use to apply additional layers of fine grain security and auditing. As we discussed the initial architecture and integration components, employing the policies within the Axway API Manager as part of the API registration and exposure process seemed to be the most logical. This would force us to create policies that worked for any API. We could seperate out request and response handlers, which referenced supporting policies as required, to allow for flexibility within their processing. Users could add or remove request or response processing independently per API (even per API version or per API method) as desired. This also would allow for granularity if customization was needed, as they could simply clone and modify the base policies and apply them only where applicable without complex logic within the policies to identify the atypical scenarios; they even could use the standard request and a modified response, or visa versa. Finally, this would meet the simplicity requirement by simply requiring a user to click the request/response policy drop downs and selecting the policy to implement.

After defining the process, the next step was to actually create the policies within the Axway API Gateway. The EB solution has a powerful AI engine under the covers. Together we defined a JSON schema that would be dynamically populated with request and response metadata within the Axway API Gateway engine and delivered to the EB ASE endpoint to run through their ML engine algorithms, which publishes the results to Elasticsearch, and displays them in Kibana.

Ultimately the JSON message sent is a relatively small payload, maybe 10 lines long, which makes the ability they have to provide such powerful analytics quite impressive. Speaking with the EB team during this process about how the ML engine worked and determined the state of the transaction (attack, anomaly, normal) was quite eye opening.  While undeniably more complex under the covers, the general concepts around how they process the data seem relatively simple, and equally undeniably capable.

After a few working sessions and some testing, we had a working model. I often find myself getting overly excited about technology and realized at this point there were still some questions I needed to ask to understand the viability of the two products working in tandem that I probably should have asked earlier in the process. I crossed my fingers and asked, and was more than satisfied with the results. Regarding false positives, their solution has the ability to be tuned to meet the needs of an environment and find the happy medium between security and accessibility. They have a plan in place to be able to get feedback from their solution running in the industry and disseminate learned attack signatures, IP/URL reputation, and more to all their customers, with the option to turn that sharing and update process on and off as needed. They were able to run across physical, cloud, and hybrid architectures (with additional capabilities specific to IoT) and had no issues with scalability. Powerful REST APIs allowed for autonomous actions and integration into the CICD toolchain, as well as the ability to publish informative reports in a JSON format that could be incorporated into other tools, also via a REST API. Aside from simply answering queries such as these, we also discussed on of their other capabilities, such as publishing a Decoy API that is not used by application to capture only malicious traffic. Almost humorous in its simplicity, yet a great example of how the EB team is thinking outside the box to bring new types of API security to the table.

Perhaps the most interesting discussions were around what is next for the EB solution and team. They have explored and discovered new territory in the API security space, but keep a close eye on them (https://www.elasticbeam.com/), because unless I am off my mark, the best is yet to come.


Post a Comment

Popular posts from this blog

Firewall, IDS, IDP, WAF, API Gateway: Choose Your Shield

REST API Best Practices: HTTP Status Codes and You, Part 2(xx) - Status 200; Success!