Mick McGuinness
Mick McGuinness DBmarlin Co-founder & Product Manager

Database observability

Database observability

Observability has gained a lot of popularity in recent years and many monitoring tool vendors now advertise their products as Observability platforms. In this blog we will take a look at what observability means and how it applies to databases.

What is observability?

In general, observability is the extent to which you can understand the internal state or condition of a complex system based only on knowledge of its external outputs. The more observable a system, the more quickly and accurately you can navigate from an identified performance problem to its root cause.

For observability tools this usually means that they collect 3 types of telemetry (metrics, logs and traces) sometimes referred to as ‘the 3 pillars of observability’. You might also hear these tools refer to collecting MELT data (metrics, events, logs and traces). In addition, some tools may do automatic service discovery and dependency mapping and some even have AIOps capabilities for automatic root cause detection.

What about database observability?

That is all well and good for distributed applications, but what about databases where performance problems often arise, since after all, it is where a company’s most valuable data lives and fast access to ever growing volumes of data is often a bottleneck for application performance and end-user experience.

Observability tools tend to give only a very high-level view of database performance. The same is true for APM (Application Performance Management) tools and for IPM (Infrastructure Performance Monitoring) tools.

The telemetry that these tools collect about database performance is mainly limited to database instance level metrics. These are things such as: the number of database connections, the number of query executions, the number of reads/writes and so on. While these are useful to see whether your instance is close to hitting any limits (e.g. are connections getting close to the max allowed connections?) they don’t let you see where the time is spent executing SQL statements in your database, where those statements came from, what resources they are using and what they are waiting on.

This is exactly what DBmarlin was designed for. It provides observability for databases that goes beyond what application observability platforms provide. DBmarlin of course collects database instance metrics and system metrics from the database host. Where DBmarlin’s visibility differs though, is in how it tracks active sessions in the database to see where they came from (user, client machine or IP address, program, session), what they were executing (SQL statement or Batch procedure) and where the time was spent (Using CPU, or waiting on something).

When we say ‘waiting on something’, that could mean any one of the several hundred ‘wait events’ which the database engines provide, to see the exact cause of contention whether that is a lock, latch, disk I/O or something else. DBmarlin’s knowledge-base has articles for over 1000 different wait events with helpful advice about what they mean and what can be done to minimise them. The knowledge-base is linked in-context within the product so you are only 1 click away from a deeper understanding of what a specific wait means and what to do about it.

Usually having identified what your database is waiting on and the statement(s) doing the waiting, you then have the task of tuning. Database tuning can often give massive benefits, with sometimes simple changes speeding things up by several orders of magnitude. See this case study for example. Tuning a SQL statement can often seem daunting. As a first step you need to understand the execution plan steps, which show how the database optimiser is going to execute the statement. DBmarlin provides the execution plans for your top statements and can even track when the plans have changed, which could have had an impact on performance. When analysing an execution plan we also provide in-context click through to our knowledge-base which provides more explanation of what each step means.

Summary

You can think of DBmarlin as an integral part of a wider end to end observability platform providing visibility into aspects of database performance that you don’t get from other observability tools. DBmarlin currently integrates with IBM Instana and we have more integrations planned. If you would like to try DBmarlin, at the time of writing we currently support the following (Oracle, SQL Server, PostgreSQL, MySQL, MariaDB, IBM Db2 and CockroachDB) running either self-hosted or in the Cloud.

Ready to try DBmarlin?

If you would like to find out more about DBmarlin and why we think it is special, try one of the links below.

dbmarlin-community-join