Microsoft & ETL (SSIS)

So this post is a bit different as it doesn’t relate to mobile or web development particularly, but is more of a conceptualization and analysis.  Recently, I’ve been tasked to move and summarize large sets of data; we’re talking tens of millions of records, resulting in hundreds of gigabytes of space needed.  At first I took a rudimentary approach which involved in writing a console app in C# using ADO.NET.  I won’t even going into using an ORM (Object Relational Mapper) like Entity Framework, lets just say: they’re not efficient in this scenario.

Continue reading “Microsoft & ETL (SSIS)”