My T-SQL Tuesday contribution for this month is about the using Notebooks more with Azure Data Engineering services. Which is hosted this month by Steve Jones.
Steve has invited us all to talk about how we have used, or would like to use, a Jupyter Notebook. You can find out more about the invite by clicking here or on the image below.
For those of you who are not aware, Notebooks are basically documents that can contain code and text. With the idea being that you can run code in separate sections that are known as cells. Notebooks are very popular with Data Science users and are becoming more popular elsewhere.
Using Notebooks more with Azure Data Studio
Now, I have been a bit of a fan of Jupyter Notebooks and for a while. I was really excited when they were first introduced in Azure Data Studio. Which I think is a great multi-platform product that is full of potential.
In fact, I have covered how you can use Notebooks in Azure Data Studio in multiple posts.
For example, the post I did about Azure Data Studio Insiders build here.
Plus, I did a review of the SQL Server 2019 Lab that was created by Microsoft. Because the SQL Server 2019 Lab shows how useful Notebooks can be. You can read that post in detail here.
When Big Data Clusters were first announced I was really excited about it. I had been using the original scripts in SQL Server documentation to create them.
However, as soon as I found out that there was a new wizard in Azure Data Studio Insiders build, I started to use that to test deployments as well. In fact, I used it to test a lot of deployments for a session that I use to present.
In addition, I also did a demo of how to deploy a Big Data Cluster using a Notebook in a video I did for the Azure Advent Calendar here. Before anybody says anything, we were all asked to wear Christmas jumpers for our contributions.
With this in mind, if you are new to Big Data Clusters and want to do an initial deployment, I highly recommend using the wizard in Azure Data Studio. Plus, I highly recommend looking at the other wizards as well. Like the ones relating to Azure Arc.
Viewing Notebooks in Azure DevOps
In fact, at one stage I also figured out how you could render Notebooks in Azure DevOps. So that you can view them in Azure DevOps itself. I shared it rather excitedly with my co-presenter. Afterwards, I shared it with the rest of the Data Platform community in another blog post here.
Using Notebooks more with Azure Data Engineering services
Anyway, I’m still a big fan of using Notebooks in Azure Data Studio. However, I will admit that these days I am using Notebooks more with Azure Data Engineering services.
For example, the Notebooks you can use within Azure Databricks and Synapse Studio. Due to my change of role.
I shared my thoughts about the increase in demand for Azure Data Engineering roles in a previous post here. Anybody who’s using Azure Data Engineering services like these will tell you how popular the Notebooks within them are.
It is one of the reasons I was glad to see so many submissions relating to these services for the next free DataWeekender conference. Which I am one of the organizers for. You can register to attend the third one which is happening on May 15th by clicking here or on the badge below.
Final word about using Notebooks more
I hope my contribution this month about using Notebooks more with Azure Data Engineering services is useful. Plus, I hope it encourages some of you to look into these services more.
Of course, if you have any comments or queries about this post feel free to reach out to me. I am also very curious to hear from people who are using SQL Server 2019 Big Data Clusters in production at the moment. Since this post has reminded me about past tests.
In addition, feel free to mention your own experiences of using Notebooks with a comment.
Be First to Comment