It’s Time for Your Company to Address Data Governance
by David Harper on January 11th, 2012
So I do most of my writing while I travel – specifically, while I am in flight. One topic that I have yet to talk about is data governance, specifically as it relates to business intelligence. Data governance is still somewhat of a new topic. It has existed for websites, and in Microsoft terms, for SharePoint for years now―but why not for data?
In my years of consulting, data ownership is somewhat a game of musical chairs―IT passes it to the business to own, the business passes it right back to IT to regulate, IT passes it back to the business to define, and the business passes it back to IT to standardize. Then the music stops and we see who is left holding the bag….
To be effective, data governance requires a coordinated approach among business and IT leaders. The IT goal should be to have a clear ownership chain and definition set for any piece of structured or unstructured data. The business goal should be to have ownership of the meaning and usage of this data.
With users demanding more self-service BI applications, the ownership of the source can sometimes become diluted. Thankfully, the technology is really picking up the pace to support this process.
The combination of SharePoint 2010 and SQL Server 2008 R2 make data governance a fairly streamlined process. Excel Services is a perfect example of how technology can help support data governance.
- The business defines the system of record and creates the definition of the data elements.
- IT creates the Office Data Connection (ODC) file and places it in a secured folder inside of SharePoint.
- Users point their Excel 2010/2007 client to the SharePoint document library to connect to the ODC.
- Users create pivots and charts and upload into SharePoint.
- Users connect to SharePoint to see the end result.
What is so simple and straightforward here is how data are being managed and governed. No more rogue reports that you have to go to Bill in Accounting and pay him in Skittles to unlock the power of data! Strong data governance protects everyone from everyone.
So my questions to you: does your organization have a data governance team? If so, who sits on it and how does it operate? If not – what’s stopping you?!
Winning Strategies are Founded on Data and Insights, not Gut Feelings
by David Harper on January 4th, 2012
In the not-too-distant past, companies had to get by without real-time information and business intelligence. It’s not that they didn’t have the information; often critical data resided in disparate systems across multiple departments. So it was extremely difficult to aggregate, synthesize, and analyze without committing significant time and energy.
Because of this dynamic, companies came to rely on people who had a strong gut instinct. In the heat of the moment, with make-or-break decisions, these individuals could be depended on to divine the right path based on their experience, knowledge of the business, and ability to read the tea leaves. (Malcolm Gladwell wrote a whole book on this topic.)
There are obvious problems with this scenario: what about the other 99.9 percent of us who don’t happen to have this gift? Or worse yet, what happens when the “gut feeling” turns out to be dead wrong?
Thankfully, technology has progressed to the point that companies no longer need to stake their future on the resident gunslinging clairvoyant.
How is this possible? Well, I am happy you asked. The answer: with Insights.
SQL Server 2012 features a range of new capabilities, including Data Insights. This tool will allow business users to combine data in many ways that ETL (extract, transfer, and load) developers have been able to use for years. They will then be able to take their data and, using the new enhancements of PowerPivot, exploit the data on their own. Want to share your findings with the rest of the team? Look no further than Power View.
With these new tools, not only can the right data be identified and translated into business insights, but every member on the team can access the information. This is definitely an exciting time for data and business intelligence.
I personally am super excited to be venturing to Redmond to learn from some of the product team all the new features of SQL Server 2012. I will report back with my findings. Our goal at Aspect is to be “production ready” by the time SQL Server 2012 is released to manufacturing. Stay tuned for more details.
Dispatches from the SQL Server 2012 BI Delta Force Ranger event
by David Harper on November 14th, 2011
I had the privilege to attend the BI Delta Force event at Microsoft in their Redmond campus. The event gives Microsoft partners access to various program and product managers who are working on the product release of SQL Server 2012, which will have a pervasive effect on the Microsoft BI community. You will have to wait until spring 2012, when it’s released to manufacturers. SQL Server 2012 is in Community Technology Preview 3 (CTP3) as I type this (with release candidates to follow).
What did we learn? Well, lots of great new information about the BI Semantic Model (BISM) and how it can be leveraged by true business users (versus waiting on some IT process). From a performance perspective, a few sessions were directed at the ColumnStore index, a new feature in this release. This index type will not fit all models and scenarios, but it does enable multi-million row tables (stored in traditional SQL tables) to be returned in sub-second response times.
From a visualization perspective, the buzz has been very loud around Power View. This SilverLight-based application, combined with either the BISM or PowerPivot workbooks published to SharePoint, allows business users to create and export their views to PowerPoint. (Yes, this is a very big deal!)
Finally, what makes all of this possible, and is new for SQL Server 2012, is Analysis Services running in tabular mode. The VertiPak engine allows for 10x and beyond data compression. What’s interesting about this mode is that the data are not stored in an Online Analytical Processing (OLAP) format but in tabular mode.
Whew…well, that was a a lot of information and very high level. However, this SQL release will bring lots of new, exciting features to an already-rich BI stack. With that said―when is your organization upgrading?!
Denali isn’t reinventing the wheel―or is it?
by David Harper on September 21st, 2011
Microsoft has been able to keep our SQL Server hearts content by releasing a new feature-filled version of the product every two to three years. However, every release can leave us stuck trying to figure out how to scale the mountain of knowledge yet again.
SQL Server Denali (SQL 11) will hopefully be released in the next four to five months, bringing with it a new suite of features and functionality. There are two changes/enhancements that have the business intelligence (BI) community particularly excited about this upcoming release. Reporting Services is adding some new functionality around Crescent, a reporting platform that lives in SharePoint and was built using slick front ends with SilverLight.
The new visualizations really look like Reporting Services is poised to compete with some of the new tablet-enhanced reporting engines. Crescent will utilize semantic models that can be built with Visual Studio 10 and leverage the Excel Services engine inside of SharePoint 2010. This will require the customer to be using the Enterprise version of SharePoint, but we have found many of our customers are already there as a result of other application needs. Information is changing constantly, so I will just link directly to the CodePlex site.
The other exciting new enhancement is around PowerPivot, which debuted with SQL 2008 R2 and SharePoint 2010 but is also being enhanced for the new SQL release. For those of you that have used or looked at PowerPivot, one of the more “frustrating” aspects of the product is the lack of ability to create hierarchies inside of your data using slicers. While true hierarchies still won’t be available in the upcoming release, Microsoft has allowed you to group filtering columns together to create a “faux-hierarchy.” The functionality is slick and really adds a lot to the product.
So is change bad? Absolutely not! As the title noted, we are constantly reinventing the wheel when it comes to delivery channels and data. Heck, if the v1 of any platform was “the best,” we would still be using Windows 1.0 and SQL Server 4.3 (You should be screaming in terror).
Now, for the good news. Your BI center of excellence should be all about your organization’s data. That is so important I will type it again―your BI center of excellence should be all about your organization’s data. At Aspect, we take a data-centric approach, ensuring that your company’s data can evolve over time as technology matures and changes. Aspect has a proven multistep process that allows us to adapt and accelerate organization’s BI initiatives.
Is your organization ready?
Data in today’s air travel
by David Harper on September 9th, 2011
I am blogging to you today from approximately 25,000 feet onboard an American Eagle regional jet. My trip through the airport was pretty standard. Fight to find a good parking spot―only to find one on the top level―BUT it was by the elevator. Walk to elevator only to discover the elevator is out of service. Wait in security line―only to be randomly selected for “enhanced” screening. As we approach the tenth anniversary of 9/11, I can’t help but think of how data has reshaped the way our airport interactions take place, and how this data has led us to new policies and procedures when flying.
So how is data used? Let’s start with data scrubbing at the airport. When I was new to the travel/consulting world (circa 2004), every time I printed a boarding pass I noticed it had “XXX” in the lower right. I quickly came to know that this was meant to tell the TSA agent that I should receive extra screening. But why? Well apparently, using some master database lists, there was a security threat with a name similar to mine―which meant I was to receive special treatment. (As a sidenote, I had to provide three forms of identification and a letter to the TSA to get off of this list.)
Why is my ticket so expensive and yours is so cheap? Timing is everything when it comes to purchasing tickets. Buy the last seat and you will pay more; buy a midweek ticket on off hours and you will pay less. I have worked on a few business intelligence projects in the past for the airlines. It is a very complicated algorithm that goes into calculating seat revenue and what to charge for each ticket. Again―data working for (or against) you here.
Why does my martini only have one olive? During the 1990s, American Airlines was aggressively looking for ways to cut costs. They calculated that if they cut one olive out of a martini it could save the company a million dollars a year. Is this the truth? I am really not sure, but I am sure the savings add up. For the record, I think that is also why they only give you a small glass of soda rather than the “whole can” on some flights!
Predictive analytics or data mining is an exciting area of the Microsoft Data Platform that is often underutilized. Mining models can be built against both relational and multidimensional sources to predict everything from what movie Amazon should recommend for you to whether the customer on the phone will buy what you are selling (based on demographics, geography, and even time of day).
So why are so few customers actively pursuing data mining? I have found that it requires more knowledge outside of data. Statistics play a HUGE role in data mining and understanding which mining model is the correct model to use in each scenario. At Aspect, we have successfully implemented a variety of data-mining scenarios. The first step is to understand the data―for example, is the data in a normal distribution where standard deviation would apply. Once the data is understood, the best model can be selected to model the predictive variables.
Is your organization using data mining? Are you thinking about data mining? I would love to hear your successes, or your concerns with this technology.
Hello World
by David Harper on August 17th, 2011
Hello World. Two simple words that often introduce a successful execution of a test computer program. As I introduce myself to Aspect blog readers, I can’t think of two better words to begin with. I have been asked to share my opinions, findings, musings, and other tantalizing tidbits with the rest of the world via this blog. But before we get to the good stuff, I think it only fair to tell you a little about how I arrived at this particular point.
I graduated from the University of Northern Iowa with a degree in Management Information Systems around the time that everyone thought their toaster would take over the world on January 1, 2000. I, along with everyone else, was happy to see that wasn’t the case and I could then further my career search.
My first job out of college was doing Web design – HTML, ASP, SQL Server, and one thing became very clear….I was HORRIBLE at it. The programming piece was the easy thing, but trying to figure out what color would complement orange was out of my comfort zone. I also realized fairly quickly that I really didn’t care.
What I did care about was data – and the power that data had. Our database administrator soon left, and I jumped at the opportunity to play with data all day long. Soon I was churning out reports and pivot tables (In Excel 2000), and keeping our sales and marketing teams apprised of what products were selling and what promotions were working well.
I was in this position for about three years before moving on to Wells Fargo and BIG data. I led the Debit Card MIS technical team and produced some very cool and compelling reports that went out to every Wells Fargo branch in the country. It was in this position that I realized there were entire careers out there that allowed people to focus just on data – and how data can (and should) be used as a competitive advantage.
After almost two years at Wells Fargo, I joined Quilogy (now Aspect) as a consultant on our Business Intelligence (BI) team. It was about the same time SQL Server 2005 was released, and it opened a whole new world of possibilities using Microsoft BI tools. Analysis Services had finally “grown up” and was ready to start playing in the big pool with the likes of Oracle and IBM. Our client list continued to grow, and our team’s head count and influence also grew at the same time. Before long, it was a weekly occurrence to start my week at the airport heading out on a new adventure.
Now, more than seven years later, I am leading the BI practice at Aspect. In this role, I not only assist our sales team in showcasing the technology but also help our other consultants and architects on various projects. I love my job – I love the technology, I love the people, and I STILL love the data. At heart, I am a GEEK; but I am a geek that is allowed to go out and share my geek-ness with others and ignite their love of data.
Here is a simple example: In my spare airplane time, I thought it would be fun to create an Analysis Services cube that tracked airline performance. I can now tell you every metric about every domestic flight that has flown since 2008. Try me! I know I have the answer!
Here is another interesting musing: we performed a customer engagement with a regional grocery store chain to do category analysis. Remember the old beer-and-diapers theory: if a dad goes into a grocery store for diapers, he is also likely to buy beer. We were able to prove this. What we found was that shoppers who purchased store-brand (“generic”) diaper wipes purchased more domestic beer, and those shoppers who purchased brand-name wipes purchased more imported beer. Does this describe you?
I intend to mix my blog with technical bits and bytes, along with industry best practices and other relevant information as it pertains to the Microsoft Data Platform. SQL Server Denali is right around the corner, so there will be LOTS to talk about. I am looking forward to interacting with many of you out there and feedback (good or bad) is always welcome.
Enough already! Hello World!
Improving the way companies communicate internally and externally with their customers is an important objective for all organizations. This blog is focused on levaraging the collective knowledge withing Aspect.