Potential for the future after this year’s Data Platform announcements

Reading Time: 6 minutes

Introduction

In this post I want to cover the potential for the future after this year’s Data Platform announcements. Due to the fact it’s the time of the year when many peoples imaginations are stirred up.

In addition, this year there has been plenty of interesting Data Platform announcements. So, I thought I would stir up your grown-up imagination by covering the potentials for the future in this post.

Now there’s certainly been a fair few interesting Data Platform announcements this year. However, I am going to delve into some of the ones I personally find interesting. To clarify, I mean the Data Platform announcements below.

1. Amazon RDS being made available on VMWare
2. SQL Server 2019
3. IBM buying Red Hat
4. Hortonworks and Cloudera merger
5. AWS Outposts

1. Amazon RDS being made available on VMWare announcement

Now this announcement took me completely by surprise. A well-established cloud service by one vendor being made available on a virtualization platform from another vendor.

Currently, it is only available if you register for a preview. And at the moment only the Microsoft SQL Server, PostgreSQL, MySQL and MariaDB databases are available within it.

Meanwhile, AWS have also stated Oracle will happen in the future. You can read more about if from AWS’s perspective here.

What will happen when Oracle Databases are available within RDS and RDS becomes available within VMWare? Well we could see that large companies will make changes in their system management or/and decide to merge teams.

We also could see some changes to the AWS Schema Conversion Tool. In addition, it could also trigger Microsoft to make some changes to Azure Stack.

1.1 Management of multiple Data Platforms

Imagine if a large company that already uses VMWare decides to start using RDS by moving databases from different systems into it. This could change how they manage deployments, maintenance and backups.

1.2 Merged Data Platform teams

Potentially we might start seeing a merger of larger teams. If all the different databases are managed within RDS it might appear more sensible for management to combine the teams. Especially if the core management of the databases appears to be the same.

If companies do start moving to RDS will staff look to start preferring using one database platform in RDS over another? Certainly, it’s a possibility if members from one of the merged teams are more prominent or advanced.

Before anybody gets too excited about these prospects there are other things to consider. For example, if the company has invested heavily into Oracle and Oracle Exadata hardware do you think they will be that keen to give up on their investment?

1.3 AWS Schema Conversion Tool

I hope that because of this potential adoption the AWS Schema Conversion Tool will finally be updated. Especially in the way you can convert databases.

Currently, you only can convert Microsoft SQL Server to other databases. It would be more appealing to some if they can also convert the other way around

1.4 Data Platform within Azure Stack

Admittedly using RDS within VMWare currently looks more appealing to people then using SQL Resource Provider in Azure Stack. Maybe it will cause Microsoft to either make changes to it or look to integrate Azure SQL Database into the product instead.

For those wondering what Azure Stack is, I talked about it in a previous post here.

2. SQL Server 2019 announcement

As a lot of you Data Platform professionals know SQL Server 2019 is coming. There’s being a whole load of Data Platform announcements about what is new within it online.

Subsequently, with the push from Microsoft themselves to start using SQL Server on Linux there has been a rapid increase in interest in doing so.

However, does that mean more companies will be willing to start migrating their existing SQL Server databases on to a Linux distribution? Well like a lot of things it depends.

2.1 Data Platform migrations to Linux

For instance, if you work for a company with reasonably small number of SQL Server instances, I suspect the migration to Linux will be relatively simple.

However, one thing to keep in mind if you work for a larger company. If you are hoping to migrate to SQL Server on Linux soon, then be prepared for some political maneuvering.

I spoke to a Delivery Manager at a large company about this a while back. He highlighted something they probably must do due to having a large number of SQL Server instances.

They will probably evaluate if SQL Server is still the right database solution to use if they are moving to Linux.

Another thing to remember is that the larger the number of instances, the more testing will need to be done before migrations start taking place.

2.2 Kubernetes use in data Platform

The announcement about Big Data Clusters and High Availability being made available on SQL Server 2019 using Containers and Kubernetes is huge.

A lot more Data Platform experts have started taking an interest in Kubernetes and potentially a lot more will follow.

Chris Adkin had a bit of a head start as he’s been looking into Kubernetes for a while now. This week he started blogging about building a Kubernetes cluster for SQL Server 2019 Big Data Clusters. You can view his blog here.

2.3 Training for data Platform experts

Of course, with the increasing interest of running SQL Server on Linux it does mean more Data Platform experts will look to become Linux or Docker experts.

Subsequently, soon there’s the potential for a lot more Data Platform professionals gaining Linux certifications. In addition, expect more Data Platform experts to start attending Linux and Docker conferences.

I can tell you from experience learning Linux again after not using it for some time was certainly hard work. I talked about that in a previous post here.

Andrew Pruski has blogged about using Containers with SQL Server. So if you want to learn more about it you can read his blog here.

3. IBM buying Red Hat

Now this was another announcement at the end of October a lot of people did not see coming. And that includes a lot of Red Hat administrators.

It’s still early days yet. Nevertheless, there’s some potential for this merger to raise the stakes as far as Data Platform offerings in the cloud are concerned.

4. Hortonworks and Cloudera merger

Hortonworks and Cloudera announcement about their merger is certainly an interesting for the Big Data landscape. These two are thought to be the leaders in the Hadoop industry.

Undeniably, a lot of people have seen what these two Big Data giants have delivered over the years within the Hadoop ecosystem. So, this Data Platform announcement is interesting.

With this merger they are aiming to use their combined expertise to deliver an enterprise data cloud. We’ve already seen what Hadoop based cloud offerings like HDInsight are capable of, so the potential here is huge.

Certainly, there’s potential for this to have massive implications in the Big Data industry. And this merger could also encourage even more Data Platform offerings to emerge.

You can read more about their merger and their future aims here.

5. AWS Outposts

Last month AWS announced AWS Outposts. Which will be AWS’s hybrid cloud platform offering. Which means it will compete with Microsofts Azure Stack.

Like Azure Stack it will work by using dedicated hardware within your own environment. However, AWS are offering two variants of this. Allowing you to extend either AWS or VMWare cloud functionality depending on which variant you choose.

5.1 AWS native variant

Potentially using this variant means that companies who use AWS will be able to host a limited number of their services on site.

However, it does look like AWS will be extending the number of services they offer on there as soon as possible. On their site they mention they intend to add services like RDS.

If that happens that means there will be two separate ways to have RDS available locally. If this happens it will be interesting to see which option is most used.

5.2 VMWare variant

In addition to this companies might want to use the VMWare variant to ease the process of migrating Virtual Machines from their existing VMWare infrastructure to VMWare cloud.

I suspect doing this would give some companies peace of mind because their virtual machines would still be hosted in their environment.

It looks promising and looks to be available in the second half of next year. There’s a lot of potential with is as long as more services are made available in the future. For more details you can see the announcement here.

Final word

I hope this post about the potential for the future after this year’s Data Platform announcements has triggered your imagination.

In addition, I hope it urged you to think of potential future possibilities. Meanwhile, if there’s anything else on the horizon that you think is worth discussing feel free to add a comment.