The key to developer job satisfaction: Give them a handle on observability

The developer talent gap is very real. According to IDC, there will be a global shortfall of four million developers by 2025. Other analyses are more dire, estimating the current shortage at 40 million developers worldwide and expected to reach more than 85 million by 2030.

While the U.S. Bureau of Labor Statistics currently indicates there are more than 1.6 million developers employed in the U.S., this number is expected to grow by 25 percent to more than two million by 2031, much faster than the average for all occupations. Despite this growth, developer demand is expected to exceed skills availability for many years to come. There are numerous causes for this, including the rapid growth of digital transformation, increasing software development complexity and more. But one thing is for certain - the fight for talent is going to be fierce, and it’s going to be essential for organizations to focus on keeping their developer talent happy and right where they are.

Identifying Avoidable Stressors

All of this is coming at a time when organizations are facing more extraordinary pressures than ever to digitize operations and create differentiated experiences. Ultimately, it is developers who bear responsibility for this, and according to statistics, 83 percent of them report feeling burnt out. Developers are the ones holding the key to a large portion of your institutional knowledge, so it is critical to retain them. In many cases, this means minimizing as many sources of avoidable stress and strife as possible.

Some sources of developer stress are inevitable -- there will always be tight deadlines, bugs to fix and demand for new features. But there are other stressors that are avoidable and that organizations may be grappling with unnecessarily. One is certainly outdated approaches to observability. More specifically, developer teams have to reactively sift through mountains of log data, searching for a "needle in the haystack" anytime there is an issue with their service. And given the massive spike in log data, there are many cases where they don’t have access to the data they need.

What is Observability?

Observability refers to the ability to infer the internal health of an app or system from externally-generated, available data. As little as five years ago, it may have been possible for developers to harness and leverage all of this data with the goal of advancing and delivering exceptional user performance (speed and availability), but this is changing.

Today’s ballooning data volumes are driven in large part by the growth of cloud and microservices; the number of global users being served; and increasingly composite applications (every API that exists within any single application will be generating data). It’s quickly becoming humanly impossible to harness and leverage this huge influx. Ironically, as data volumes have increased (which, one would think, would serve to better guide and inform teams in the event of a performance degradation), the number of prolonged outages lasting longer than 24 hours has actually risen sharply. Developers have so much data they often find themselves wading through it aimlessly, to no avail and without the faintest idea of where to begin looking for the root of performance problems.

The Way Forward

Fortunately, there are ways to deal with this morass, empowering developers to fully harness and leverage all of the data at their disposal to actually get faster and more efficient at finding and fixing user performance issues - and thus reducing a significant source of stress. Traditionally, observability approaches have entailed "pooling" all data in a central repository for analysis, but this approach is becoming both too costly (from a storage cost perspective) as well as too slow (as repositories grow fatter, they tend to slow to a crawl in returning queries).

The answer lies in flipping this observability paradigm on its head, analyzing data in smaller chunks all processing in parallel, as it’s being created at its point of origin. There are several benefits to this approach. First, when data is analyzed at the source, as it’s being created, the process of identifying the bottleneck becomes much more intuitive and quick. Organizations can effectively have an eye on all their data (which is important, as performance-impacting issues can crop up anywhere), while avoiding the heavy costs and latency that occurs when everything is stuffed into a central repository.

Second, by applying real-time analytics at the source, organizations can surface insights from huge volumes of data so developers don’t have to spend hours sifting through data to find what they’re looking for. This, of course, makes them much more efficient at detecting and responding to performance degradations.

One other important advantage in this era of the cloud and microservices is that real-time analytics can be applied automatically and start generating insights the second an instance is created or an application goes live. This is known as automatic discovery, and it can reduce a lot of the unnecessary toil associated with teams having to manually add new instances to their observability initiative, each and every time a new one is created. Without this capability, developers will always be one step behind because the new instances will be operational and generating data before that data has an opportunity to be captured and analyzed. This can leave a huge blind spot and cause teams to be caught flat-footed during the first few minutes or hours of a system being live, when performance issues are most apt to rear their ugly head.

Conclusion

As the battle for developer talent intensifies, organizations are going to need to do everything in their power to increase job satisfaction and retention. There is no question that these roles are inherently taxing, but there are definitely obstacles that can be eradicated in order to make developers feel less stressed, more efficient and productive, and more in control. Breaking down observability data is a prime example and it can go a long way towards avoiding developer burnout and helping organizations hang onto these most precious resources.

Photo Credit: ollyy/Shutterstock

Ozan Unlu is CEO, Edge Delta.