This entry picks up the story behind my first data science project predicting hail damage to farms. In this article we identify data quality issues in our first data source. Property and Crop Damage In the NOAA documentation these two columns were recorded to say how much property and crop damage occurred in a given…
Tag: SQLServer
Data Analysis…can we automate this?
As some of you know, I’ve moved from consulting back into a full-time employee for Crop Pro Insurance. There was so much opportunity in this role. First of all, this role gives me my first full-time data science credit. I also get to build a team to support data science projects. On top of that,…
Metadata Model Update
As I began learning Biml, I developed my original metadata model to help automate as much of my BI development as I could. This model still works today, but as I work with more file based solutions in Azure Data Lakes, and some “Big Data” solutions, I’m discovering it’s limitations. Today I’d like to talk…
Data Warehouse Efficiency
How quickly do you get from the business coming to you with “we need a data warehouse” to delivering that warehouse? If you’ve sat through any Biml talk, you’ve undoubtedly heard stories of thousands of staging packages being generated per hour. You may have even heard tales of source systems being analyzed in hours, rather than…