The importance of API monitoring across the enterprise [Q&A]

Over the past few years, technology teams have split into smaller work groups with more focused tasks. The rise of the cloud has created the need for DevOps teams, and the gap has grown wider between teams that build products and teams that manage products.

At the same time, applications have become dramatically more complicated. This has given rise to specialized site reliability engineers who are well-versed in monitoring all application components, including APIs. However, focusing API resilience in one team has allowed organizations to treat the symptoms rather than the underlying problem.

We spoke to Jamie Beckland, chief product officer at APIContext to discuss why API monitoring should be a core function of product engineering, product management, site reliability, DevOps, and governance teams.

BN: What API monitoring metrics should organizations be tracking and why?

JB: API monitoring should be robust enough to give you a comprehensive understanding of your API resiliency. High-quality APIs have robust performance and strong security controls.

Here are the API metrics that matter for judging API quality:

1. Latency: This metric measures the time taken for an API request to be processed and responded to. High latency can indicate performance bottlenecks that need to be addressed to maintain a good service. Issues can manifest from anywhere across the organization's network or the wider internet. Each monitor trace should include an analysis of each component part of the API call, end to end. This is one area where synthetic monitoring is critical -- only when you instantiate the API call can you track DNS response time, network handshakes, and other components that are outside of the application.
2. Full Call Response (including errors): Tracking the frequency and types of errors (e.g., 4xx and 5xx status codes) is essential to identify issues that could degrade the user experience or signal potential security vulnerabilities. Many API issues present as intermittent gaps in specific use cases. Tracking the full response is the only way to ensure you can investigate when errors manifest.
3. Uptime and Availability: Ensuring that APIs are available and operational is critical. Monitoring uptime helps maintain service reliability and meet SLAs. For many API products, SLA violations carry financial costs. Even if there is no contractual SLA, users won't continue to leverage a flaky API, especially if it is required to run part of their own website or application. Monitoring cadence must align with your target SLA. Some synthetic monitoring is set up to test endpoints as often as every 5-10 seconds.
4. Dependency Performance: It's not enough to monitor your own APIs, you also must track the APIs that your application relies on. APIs often depend on other services like databases, authentication services, payment processors, and other third-party APIs). Monitoring the performance of these dependencies helps in diagnosing the root cause of performance issues. Often, when an application falters, it's because a key vendor has failed.
5. Conformance for Security: Often, APIs are not built as they were designed. When that happens, security by design principles are impossible to follow. API design specifications, like OpenAPI Specifications, ensure that API documentation is up to date. However, regressions can be introduced in every code push, so continuous conformance monitoring is necessary to ensure that production APIs do not deviate from their specification. This ensures that APIs are secure, and the lack of conformance testing is the main reason for so many API security issues today.

BN: Why is API performance important for modern businesses?

JB: APIs are crucial for modern business operations, connecting systems, enabling integrations, and facilitating data exchange. They account for over 75 percent of internet traffic, and without them, digital applications would cease to function. APIs directly impact critical business aspects including:

• Customer Experience: Modern apps require many APIs to function well and meet high customer expectations. Slow or unreliable APIs lead to poor user experiences, dissatisfaction, and churn. Fast APIs ensure reliability and smooth experiences, which are essential for retaining customers and building loyalty.
• Developer Experience: APIs are used to build internal and external applications. Developers rely on our APIs, and they face pressure to work faster and build resilient applications. Unreliable APIs result in a poor developer experience.
• Revenue Creation: APIs create digital products and services, driving revenue. Unreliable APIs mean unreliable revenue streams. Poor API performance affects revenue creation and delivery, impacting sales and processes like inventory management and payment processing. High-performing APIs enable rapid innovation and new service deployment, driving business growth.
• Operational Efficiency: APIs automate processes and enable seamless data exchange. Performance issues cause delays and inefficiencies, reducing productivity. High-performing APIs ensure smooth operations, reducing downtime and allowing focus on strategic tasks.
• Innovation: APIs impact future products, providing scalability and enabling businesses to adopt new technologies easily. They support rapid prototyping and deployment of new features, fostering innovation and competitiveness.

BN: How do APIs affect user experience, or how do they facilitate it?

JB: APIs are crucial in delivering application and website functionality and enhancing user experience. The goal of API-driven development is to enable seamless interactions between applications, services, and users.

APIs enable communication between the back end and the front end. Every web page that is rendered, every mobile app screen, and every connected device presentation relies on APIs that translate information in databases into something that a human can read. If you have ever noticed a piece of a website that is blank where text or images should be, that is likely because an API call failed.

APIs also allow applications to provide information in real time. Every streaming TV service uses APIs to call video files and render them in real time onto your TV or tablet. When you watch an online auction tick down, APIs process the bids and track the winner. News feeds with social media updates or breaking alerts on news websites also rely on real-time information delivery from APIs. Without APIs, real-time services would not be possible.

Personalization and customization: Some API-powered examples include e-commerce websites that remember what you have purchased in the past, reading recommendations from your library app based on what you enjoy reading, social media feed updates from your closest friends and family, and customized notification preferences from your bank.

These are just some of the improvements in user experience from APIs, there are more like improved performance, easier integration, and more advanced functionality.

BN: How do organizations achieve the necessary security for their API architecture?

JB: Securing APIs continues to be an issue even for the most mature teams. In part, this is due to the great breadth and depth of API functionality. But it's also due to the explosion in APIs over the past decade, at a time when API security was not well understood by most developers. As a result, more than half of teams know they have API security issues but do not have a clear understanding of how to remediate them.

The first step to securing APIs is to ensure that the architected API specification has been properly implemented in production APIs. More than 80 percent of API security issues are misconfigurations that would be solved by building the API as designed. The APIContext Conformance product compares OpenAPI Specifications against production APIs to identify misconfigurations. It is less burdensome and more effective than writing negative tests for every possible API exploit.

Next, improve the authentication and authorization mechanisms of your APIs. Many APIs use a basic authentication password or an API key that is provisioned once per user. More advanced authentication, such as oAuth 2, JWT tokens, or the financial-grade API (FAPI) standard, can dramatically reduce data leaks over APIs.

Leverage your API gateway and firewall to provide centralized points for managing API security. An API gateway can handle authentication, authorization, rate limiting, and logging while also providing a layer of abstraction that shields backend services from direct exposure. API firewalls further enhance security by filtering and blocking malicious traffic.

Synthetic API monitoring also allows you to see the entire API call from start to finish and log all data, even sensitive data that can't be logged from observed traffic. Analyzing observed and synthetic results together to understand potential security breaches means you can manage risks effectively as they arise.

Image Credit: Alexandersikov/Dreamstime.com