Our Interview with Director of Analytics Services at EDUCAUSE

Data and Analytics in Higher Education

About EDUCAUSE and Leah

EDUCAUSE is a higher education technology association and the largest community of IT professionals committed to advancing higher education. Analytics8 has been working with Educause since 2016 to improve their analytics offerings to their association members so that they can use their data more effectively.

As Director of Analytics Services at EDUCAUSE, Leah Lang leads the strategy and operations for EDUCAUSE’s suite of benchmarking services, products, and tools, which are used by the higher education community to inform decision-making about IT. Through working closely with the higher education community and staying current on trends in higher education, higher education IT, and technology in general, Leah ensures that IT leaders have the metrics they need to make the case for IT.  Leah’s expertise of IT metrics has positioned her as a leader in the higher education community.

Our Interview with Leah

• Why did EDUCAUSE want to improve its data and analytics capabilities? ↵
• What business or technical challenges did you need to overcome? ↵
• What KPIs were important to track in the solution? ↵
• What led you to choose Qlik? ↵
• What functionality makes it easy for your users to gain insight? ↵
• How did you encourage adoption? ↵
• How do you measure success? ↵
• What would you consider some of your biggest wins? ↵
• Any ideas for future functionality? ↵
• Any advice for other companies looking to implement a modern data and analytics solution? ↵

Why did EDUCAUSE want to improve its data and analytics capabilities?

Leah: “EDUCAUSE helps IT leaders in higher education make better decisions by providing peer data comparisons for key IT benchmarks. Colleges and universities use EDUCAUSE data to inform IT strategic planning and to answer questions like “how does my spending compare to my peers’?” or “how does my school’s ed tech or information security services compare to others?” To collect its data, EDUCAUSE sends surveys to thousands of institutions, students, and faculty throughout the world, each year. Results are then released through benchmarking dashboards members use to measure their departments against peer data provided by other institutions.

Our prior benchmarking solution was marked with manual processes for the EDUCAUSE analytics team and the output was cumbersome for members to analyze. We sought out to select a BI solution based on its ability to integrate multiple data sources, calculate benchmarks on the fly, and provide quick, interactive insights to our members. After a lot of analysis, QlikSense was the best product for our needs.”

Did you have any unique business or technical challenges that your analytics solution needed to overcome?  

Leah: “It was important to allow our members to create custom peer groups on the fly as part of our benchmarking solution. While predetermined peer groupings allow comparisons on many helpful data points, there are times when institutions wish to compare based on more specialized criteria related to their own department goals and changing benchmarking needs. With many BI solutions, peer groupings are handled by creating data cubes for predefined filter groups. This method didn’t allow our members the flexibility to create peer groups as needed.  With Qlik’s easy integration with multiple data sources and ability to summarize data without cubes, it was a great solution to meet our custom peer benchmarking need.

We also needed to ensure that our members’ dashboard served as a one-stop-shop for data entry, benchmarking, and analysis.  We didn’t want our users to respond to our survey in one place, access survey results in another, and create their peer groupings in another. By leveraging mashup technologies and APIs, users are able to access their reporting visualizations in the same environment where they complete our surveys.  As an added benefit, this simplifies their experience with the data.  Instead of overwhelming them with and training them on all of the advanced features in the out-of-the box Qlik user interface, they see only what they need to for quick analysis of key data points.  Our dashboards are customized for visual data discovery; users can filter results based on their peer groups, drill deeper into the analysis using responsive Qlik objects, and save their progress and return to it later.

Finally, we have strict restrictions on who can see our data and how it can be used; implementing business rules for data sharing was imperative. Thanks again to Qlik’s easy data integration, EDUCAUSE and its members can now easily define role access for different types of data.”

Given that your organization serves data professionals in higher education, what KPIs were important to track in the solution?  

Leah: “Our solution provides hundreds of data points around IT financials, staffing, services, and more. A typical member dashboard may include data points on the proportion of the institutional budget spent on IT, an institution’s analytics maturity, or some high-level percentages about the types of classroom technologies deployed on similar campuses.  All dashboards include the ability for institutions to compare themselves against institutions with similar characteristics, such as student body size, geography, and budget. By use of interactive visualizations, users can also see year-over-year changes and other trends depending on how they choose to segment the data.”

What led you to choose Qlik as the analytics platform?

Leah: “Qlik’s flexible method of calculating data summaries in memory and the ease of data source integration were big factors in our choice of BI platform. Also, Analytics8 has a lot of experience with Qlik implementations and is able to think outside of the box to continually add new functionality that we request. For example, they recently implemented extension objects to fill a few gaps on the frontend interface, such as customizing a Radar chart that was sourced online and using a list box to allow for default selections in the dashboard. They were also able to meet our need for fully leveraging the Qlik API to create a highly customized user experience.  I’m particularly impressed in their development of a dashboard tool that uses only a handful of Qlik objects to provide our users with unlimited dashboard pages to analyze countless questions.”

What functionality makes it easier for users to gain better insight into their performance?

Leah: “Hands down, the interactive, responsive data visualizations that automatically reflect how the user wants to drill into the data. Users appreciate the flexibility to start from high-level data summaries, make selections on one object based on what they are seeing in the data, and having that selection filter other visualizations in the same view. Once users find a result they need, they often want to return to it later.  The API integration allows a handy bookmarking feature so that users can save their analysis.”

How did you encourage adoption of your new analytics solution by your members?

Leah: “Designing our solution based on member needs was key to ensuring adoption and, in effect, seeing ROI on our analytics investments. Our members were clear in their feedback that our previous method of providing tons of raw data for them to benchmark against their peers was not adequate. They wanted a solution that combined all data in a single place and made it easy for them to freely explore and analyze.

We also place a lot of emphasis on education and training so that our members know the capabilities of our solution and that they feel confident using it. We offer several learning options, such as webinars, courses, videos, and an exhaustive user guide, which covers how to use the tool and common use cases. We’ve also presented “telling your story with data” sessions at professional development events, using the tool as an example, so that users can envision new ways to apply their data. Later this year, we plan to focus on data literacy training which will teach users about proper data entry, data cleaning, using dashboards, and most importantly, interpreting the results.

Our members also contribute to user adoption as they share their own success stories with the member community. We encourage members to share their tips and stories about what they can do now that they couldn’t previously, because we see that sharing success motivates others to do more with the data.”

How do you measure success in terms of your new analytics solution?

Leah: “We regularly send out  surveys to our members to gauge their satisfaction and receive feedback for improvement. Questions include everything from “what data is most valuable to you?” to “how was your experience today?” We also look at tool usage and the number of support requests which alert us to problems with the tool.

Internally, we also look at the cost of tool support. We built an intuitive solution that is easily managed in-house, which lowers the ongoing cost of hiring outside help for customization requests.”

What would you consider some of your biggest wins since the implementation of your new solution?

Leah: “We’ve seen a 28% increase in data use since launching our new solution to members! And funnily enough, we’ve also seen a 28% decrease in support requests, which indicates that this solution is much better at answering the questions our members have of their data.

There’s also been an increase in community chatter about the new solution. One member recently sent accolades to our CIO listserv that I couldn’t be happier about:”

I wanted to take a minute and publicly congratulate the CDS team at EDUCAUSE for the release of the new digital capabilities dashboards.  At [our University], we’ve incorporated two of these benchmarks into our balanced scorecard(s) and in a single stroke the CDS team has eliminated the need for what was a very manual and time-consuming benchmarking process.  If you’re a CDS contributor and you haven’t already, take a look at the new dashboards.  For me, these tools are just as, if not more, important than the financial benchmark data because they provide [University] with a standardized, benchmark-capable way to assess our maturity on a variety of capabilities. I’m already planning to use the information security capability dashboard to help provide context to our Board on the work we are doing to protect institutional systems and data.

Any ideas for future functionality or offerings?

Leah: “We want to create a peer recommender system that helps users select logical peer groups for benchmark analysis.  Right now, members filter through demographic information from 4,000+ institutions to pick and choose institutions to compare against. In the future, we want to apply data science techniques to create an automated scoring system that indicates how well institutions match up based on characteristics like centralized vs. distributed IT services and other to-be-determined predictors. This scoring system will make it much easier for members to select peer groups and give them piece of mind that their benchmarking analysis is done in the appropriate context.”

Any advice for other companies looking to implement a modern data and analytics solution?

Leah: “Your success is directly correlated with your vision. Having a clear vision will help you communicate with your consultant about your needs and the ultimate strategy to move forward. In the past, when we started work without clear vision, we set ourselves up for failure. The goals for our newest solution were clear from the start, and it shows.”

Read more about the work Analytics8 did with Educause to transform their Self-service Benchmarking Tool