Mick McGuinness
Mick McGuinness DBmarlin Co-founder & Product Manager

The Modern DBA and the evolution of database management

The Modern DBA and the evolution of database management

My colleague Ian previously wrote about the role of a modern DBA about a year ago. As noted in that article, DBAs have been slowly declining, as a percentage of total IT staff from 3.3% in 2013 to around 2.9% in 2018 according to Computer Economics, while at the same time the volume of data needing to be managed is increasing.

How is it possible to manage more and more data with fewer DBAs?

Doing more with less has always be a goal for businesses by automating the manual tasks and replacing them with hardware or software.

The shift to Cloud databases, removes the need for some of the tasks that a traditional DBA would have once done, such as patching, upgrading and backups.

Another reason is the growing power of developers to do more for themselves, with ideas like self-service databases or throwaway databases that can be spun up, used and then destroyed once they are no longer needed. Usually, the databases in these cases, are managed by platform engineering teams and the role involved may be classed as a ‘modern DBA’ or may go by a newer title such as ‘Data Reliability Engineer’ or ‘DevOps Data Engineer’.

Differences between a modern DBA and a traditional DBA

The DBA role is evolving along with the processes to support modern data stacks. As with everything in IT and software, nothing ever stands still!

Whether you are a traditional DBA or a modern DBA, you have a shared goal to protect the data and stability of your organisation’s databases. Data is the number one asset of most organisations after all.

Historically, a ‘traditional DBA’ would specialise in a single database engine like Oracle or Db2 and could spend their whole working life as the guardian of those databases and everything that happened within them. Those roles are now dying out and except for traditional DBAs approaching the end of their career there will be pressure to change.

These days, ‘traditional DBAs’ need to evolve to become ‘modern DBAs’ by adopting new skills and responsibilities.

In a recent webinar by Liquibase and their customer, The Hartford, they explained how they see the role of the modern DBA. The need for change comes from seeing how the application teams have adopted ‘cool’ new technologies and processes (e.g. Git, CI/CD, Agile, DevOps, modern observability tools) and traditional DBAs feel that they don’t want to be left behind.

On the webinar the attendees were asked which words best described a modern DBA and number one was “innovator” while others suggested “multi-talented”, “automator” and “collaborator”. These allude to the fact that a new way of thinking is required to embrace new technologies and processes. All of this requires the modern DBA to become an expert in more than just one discipline and to be proficient in ideas like CI/CD and automation which their counterparts in app development are accustomed to.

Liquibase webinar

Key responsibilities of modern DBAs

Here are some key responsibilities of a modern DBA:

  1. Data Infrastructure Maintenance: Managing and maintaining data storage systems, databases, and data processing frameworks to ensure they are functioning correctly and efficiently.
  2. Data Pipeline Monitoring and Troubleshooting: Monitoring data pipelines to detect issues such as data ingestion failures, data transformation errors, and data quality problems. When issues arise, the Data Reliability Engineer works to resolve them promptly.
  3. Data Quality Assurance: Ensuring the quality, integrity, and accuracy of data by implementing data validation checks, data monitoring, and data governance processes.
  4. Performance Optimisation: Optimising data workflows, queries, and processing jobs to improve the performance and efficiency of data processing systems.
  5. Scalability and Resilience: Designing data systems to be scalable and resilient, handling large volumes of data and ensuring continuous data availability.
  6. Automation and Tooling: Developing and maintaining automation scripts and tools to streamline data operations and reduce manual intervention.
  7. Incident Response and Post-Mortems: Participating in incident response for data-related issues and conducting post-mortems to identify the root causes and prevent future incidents.
  8. Collaboration with Data Engineers and Data Scientists: Working closely with data engineering and data science teams to understand their requirements and provide data solutions that meet their needs.
  9. Security and Compliance: Ensuring that data systems meet security standards and compliance requirements to protect sensitive data.

Summary

In IT, nothing ever stands still. Whatever your role, you will at some stage in your career, need to adapt to new ways of working as well as new tools and technologies. If you are a veteran with decades in the industry, adopting new ways of working could be daunting, but also an exciting challenge. For those forging a new career in IT and already working in a ‘modern way’, enjoy it for now, but realise that in a few years, things will move on as they always do, and that’s what makes IT a great place to be.

Ready to try DBmarlin?

If you are looking for modern database observability solution that fits with new ways of working and can integrate with your CI/CD pipelines then take a look at DBmarlin:

dbmarlin-g2-review