Recently a client mentioned he wanted to run spark in a container for testing Databricks code locally. He Was working from the basic docker-compose yaml file provided by bitnami. That way, the containers would run on his local machine. The problem was accessing the cluster from the host machine. I decided to take a day…
Category: Infrastructure
Network Infrastructure Updates
If you’ve tried accessing the site over the past couple of weeks, you might have noticed it wasn’t up 100% of the time. That’s because I was upgrading my network infrastructure. Before the migration, I ran the Ubiquiti EdgeRouter 6p. This is a tremendous little quad-core router that ran completely silently. I upgraded from the…
Docker and Watchtower
When I began work with SQL Server, I used to run a single Windows server with a single instance of SQL Server 2000. I used this instance to test code and configurations separate from my employer’s servers. It helped me verify if a change could help and by how much it could help. That single…
Alexa Solutions Infrastructure
With an understanding of the Alexa ecosystem and some of the terminology I’ll be using, let’s look under the hood a bit today. All Alexa devices depend on the cloud to do the heavy lifting. In the most simplified case, this is what’s under the hood of a simple Alexa skill. You speak to your…
Azure Data Lake Storage ACL Automation
In my last blog entry, we covered how to layout folders in your Data Lake Storage account based on a logical design. That’s only half the battle. You also need to set up Access Control Lists (ACLs). Setting up controls via the Azure portal is easy, but not something you can automate. Today, we’ll jump…
Azure Data Lake Storage Zone Layout Automation
After laying out the structure for our zones, my client quickly asked, is there a way we can automatically stand up this structure each time we bring a new Tenant into our solution. With a smile, I replied it was possible! In the short term, we would use PowerShell to stand up these folders every…