I’m going to Nottingham, England, fabled home of Robin Hood, where I’m going to eat fish and chips and talk about SSIS. How’s that for tying seemingly unrelated topics together? I’m happy to report that in addition to my full-day preconference seminar at SQLBits, I’ll also be delivering a regular session entitled Cleaning Up Dirty Data with SSIS on Friday,…
I’m happy to announce that I’ve been selected to deliver a full day of training at the upcoming SQLBits conference in Nottingham, England. This will be my first time taking part in SQLBits (in fact, it will be my first trip to the U.K.) and I’m very much looking forward to the experience. I’ll be delivering a full day presentation…
OK, I’m not really going to break it. I’m just going to beat it up a little bit. This Thursday evening, I’ll be working my home turf at the North Texas SQL Server User Group monthly meeting, presenting “When ETL Goes Bad: Handling Errors and Anomalies in SSIS” at 6:15 at the Microsoft campus. In this hour-long presentation, I’ll go…
Frequently I am asked how to properly use SQL Server Integration Services to handle ragged flat files, an example of which is shown below. In this small sample file, you can see that we are dealing with medical data with an inconsistent format. As shown, the rows in the pipe-delimited file can have as few as seven and as many…
I’m happy to announce that the book I’ve been working on for the past two years, SSIS Design Patterns, is complete and has been released for sale as of today. I got the unique privilege to work alongside SSIS rock stars Andy Leonard, Jessica Moss, Matt Masson, and Michelle Ufford to scribe what will hopefully be a must-have book for…
I’ve fielded a number of requests recently asking how to interrogate a file within SSIS and change the processing rules based on the metadata of said file. A recent forum poster specifically asked about using the foreach loop to iterate through the files in a directory and, on a per-file basis, either process the file or skip the file if…
So I’ve discovered another benefit of being a technical blogger. Not only do you get some kudos when you write something that helps someone else, but if you offer up a less-than-optimal solution, you’ll get some suggestions on how it can be done better. I’ve had my share of the former, but earlier this week I experienced the latter. Last…
A relatively common requirement in ETL processing is to break records into disparate outputs based on an alphabetical split on a range of letters. A practical example of this would be a work queue for collections staff based on last name; records would be pulled from a common source and then separated into multiple outputs based on a the Customer…
So you’ve got some packages that regularly extract data to one or more text files, but you know that from time to time some of the queries will not return any data. However, you find in SSIS that, in a flat file export package, the output file is created regardless of whether any rows are written to the file, and…
It has been well-documented through myriad blogs and forum posts about the case sensitivity of the comparisons in the SSIS lookup transformation (a good review can be found here). In a nutshell, a comparison using the lookup transformation is case sensitive when using the default setting of Full Cache, even if the values in the database are stored in a case…