First impressions of the web terminal in Azure Databricks

Reading Time: 3 minutes

In this post I want to cover my first impressions of the web terminal in Azure Databricks. Because I had a legitimate use for it recently and I found it so interesting that I wanted to share my thoughts about it.

In reality, this post is also another first. Because it is my first ever post dedicated solely to Azure Databricks. So, I have had to create a new Azure Databricks category especially for this post.

For those of you who are not aware, a couple of years ago Databricks introduced the web terminal. Which basically allows you to log into your Spark cluster and run Linux commands.

I want to share my experience with web terminal to raise awareness about it. I suspect it will be of interest to others. Especially those who built Spark clusters by hand or have some other form of Linux experience. For example, those who learned Linux to support SQL Server and are now looking at Databricks.

First impressions of the web terminal

Anyway, a while back I had to investigate issues with mount points not appearing in Spark clusters. I wondered if there was a way to connect to the clusters behind the scenes to see what mount points existed.

That is when I discovered that I could use the web terminal to help with this. After enabling the web terminal for use I then went into the Compute section of the workspace. I then selected the relevant cluster and went into the Apps section.

I was very impressed when the web terminal opened up in a new tab. Even more so when I realized that I could run the findmnt command to view existing mountpoints. Like in the below example.

It helped with this particular issue. I must admit that my first impressions of the web terminal are that it can be useful to help investigate issues like this.

Plus, I found it a very interesting experience for other reasons as well. Including the fact that many years ago I built a physical Hadoop cluster that also ran Spark.

Additional tests in the web terminal in Azure Databricks

I tested doing other things in the web terminal. Because I was interested in finding out what else I could do whilst logged into the web terminal. It was interesting to see that a lot of standard Linux commands work in the terminal. Including the popular VIM editor.

I was also able to install applications as well. In fact, after a bit of tweaking I was even able to install a GitHub self-hosted runner on the node. However, I must stress that I do not recommend doing this.

Plus, I did notice some other interesting things as well. However, I will save that for another post.

Personally, I found doing all of this very interesting and I am glad that this feature was added.

Enabling the web terminal in Azure Databricks

You can enable the web terminal yourself if you have the right permissions in the Azure Databricks workspace. By going to Settings and selecting Admin Console.

From there you can select the Workspace settings tab, before scrolling down to the Advanced section to enable it.

Final words

I hope me talking about my first impressions of the web terminal in Azure Databricks has been of interest. Because I want to raise awareness of this feature.

Personally, I find it interesting to be able to log in to the web terminal and look around. However, I am aware there can be some implications with this. One of which I will save for another post.

Of course, if you have any comments or questions relating to this post feel free to reach out to me.