Temporal data storage, which was defined in the ANSI SQL:2011 standard, describes an architecture in which relational databases maintain row-level versioned copies of data. When using temporal data, the database does automatically what many DBAs and application developers have had to do manually for years to track point-in-time versioning for individual tables. Starting in 2016, Microsoft added SQL Server temporal…
The yearly PASS Summit is just around the corner, a little less than six weeks away. For those not familiar with this event, it is a gathering of several thousand data professionals for 4+ days of education and networking. The PASS Summit will take place in Seattle, Washington, from Tuesday, October 31st until Friday, November 3rd. Networking at the Summit…
Is it possible to be an effective data professional without knowing anything about the domain of data you support? I remember having a conversation a few years ago with a friend of mine, a fellow data practitioner. In his role as a database administrator, he asserted, he could do his job without knowing anything about the underlying data. His argument…
One of the more common questions I find in SQL Server SSIS forums is how to split apart a single text file into multiple outputs. Files extracted from foreign sources often arrive in nonstandard formats, and are too often unchangeable at the source and must be parsed during the import process. For unconventional custom transformations such as this, using the…
This year I’ve delivered my full-day Building Better SSIS Packages course a half dozen times across the country, but I hadn’t scheduled this class in the Dallas area this year. As of today, that has changed! I’m happy to announce that I’ll be delivering this class at the Microsoft office in Las Colinas next month, on Friday, August 25th. Registration…
When designing the ETL architecture for new or changing systems, one of the more common needs is to create a process that copies all of the data from a set of tables (perhaps even all tables) from a source system to a destination database. This is especially prevalent when building an operation data store (ODS), or building a set of…
In the previous post in this series, I addressed how to use an on-premises instance of SSIS to move data to and from Azure databases. If you’re running in a pure Azure environment without on-prem SQL Server, that load architecture would present some challenges. However, by running SSIS in an Azure VM, you can communicate move data into or out…
If you are building SQL Server Integration Services (SSIS) packages, using the SSIS catalog as a deployment target is usually the easiest and most efficient solution. However, the SSIS catalog is not created by default, even when you select the SSIS components during the SQL Server installation process. The good news is that creating the SSIS catalog is a quick…
Some time back, while pulling into a local coffee shop, I spotted a stranded motorist in the parking lot. Per the Texan code of ethics, I was duty bound to walk over and offer my assistance, and in doing so I discovered that someone else had already stopped to help. The motorist was in the driver’s seat turning the key…
Having the right tools for the job makes the work much more efficient. However, for those just starting out in SQL Server land, it may not be clear which tools are useful or appropriate for a given task. I regularly field questions about which tools I use on my development machines, and why. To that end, I’ve compiled a brief…