The Path to Data and Analytics Modernization

Most organizations aim to be one step ahead of the competition—able to make more informed decisions about their business as well as their customers—so that they not only succeed but thrive in an everchanging business landscape. However, although many organizations have the data to do so, they often lack the technology, processes, and people to fully optimize its worth.

Tony Dahlager, VP of Data Management, and Kevin Lobo, VP of Analytics at Analytics8, laid out the case for how to build modern data and analytics solutions during “The Path to Data and Analytics Modernization” webinar. They discussed the business demands and industry shifts that are impacting the need to modernize, as well as the benefits of and the approach to data and analytics modernization, and the key pillars of modern data management.

It’s not just adopting new technology—modernization requires a balanced approach that includes people and processes as well.

What is Data and Analytics Modernization? 

Modernization isn’t just the newest buzzword—although it seems that everyone has their own way of defining it. Data modernization is also not about a single action or implementing some suite of tools. It is rethinking how you use data and analytics as a company. ​

Oftentimes people characterize data modernization as just moving to the cloud; but the approach you take and the advantages you realize go beyond just cloud adoption. Modern solutions expose advanced analytical capabilities that help your business users make smarter decisions.​ Data modernization requires modern data management principles. It involves moving from legacy databases and architectures to modern, cloud-based platforms and scalable architectures, and migrating to next-gen analytics tools. ​

What Are the Benefits of Going Down the Path of Data and Analytics Modernization?

As more data becomes available, and business users require better and more advanced analytics, your organization needs to be able to act quickly to meet those demands. Data modernization allows you to more easily adapt to changes as well as evolve as an organization.

Modern data and analytics solutions allow your organization to scale and be flexible, integrate new data sources, get to insights quicker, democratize your data, and effectively plan for the future of your business.

Legacy tools, on the other hand, lack the ability to solve modern data problems. So, what are the things that you can do today to plan for the future when it comes to how you’re approaching data and analytics, and how do you get there? To start, understand the types of modern problems your organization is facing today and how they impact your business.

Defining and Addressing Modern Data and Analytics Problems

There are lots of different challenges and industry shifts that are leading organizations down the path to data modernization, because old ways of approaching data and analytics are simply not keeping up with the technology and business demands we’re seeing today. Organizations have:

• Exponential Data Volumes: Across the board, companies are having trouble keeping up with the volume of data, and this volume is increasing exponentially every year. However along with this explosion of data volume and variety brings an opportunity for organizations. There are competitive advantages with the ability to access, use, store, transform, and analyze more data—including increasing data volumes, types, and sources.
• Different Types of Data: It’s not just relational sources anymore; there are different types of data popping up every day, and analytics systems can have trouble playing nicely with semi and unstructured data sources. New types of data include data from social media, customer engagement, marketing, and a variety of different channels—all which can be used to really understand your customers if you are able to harness it. However, these data sources all require integration, forcing businesses to adopt new technologies and processes to keep pace.
• Different Types of Consumers: People are using data for lots of different purposes now, and reporting, dashboarding, and Excel are no longer sufficient analysis tools. Streaming, real-time, near real-time, embedded use cases, advanced analytics, and bi-directional use cases are growing and becoming much more common for organizations—even for those that are less mature in their data analytics lifecycle.
• Cloud, On-Prem, and Hybrid Systems: We see lots of different types of architectures and they’re getting more and more complex. Data sources have been on-prem and in the cloud for a while now, but now organizations have source systems, data warehouses, and operational systems split between being hosted on-prem, in the cloud, in multiple clouds, or hybrid, which just adds to the modern problems that they’re facing today. How do you how do you manage all that architecture?
• A Need for Advanced Analytics: Advanced analytics is becoming more prevalent, even though it’s still on the horizon for many organizations. Companies at a minimum need the foundation in place to prepare them to perform advanced analytics, machine learning, and AI.
• A Need to Democratize Data: More people throughout the company want to get their hands on data. Analysis isn’t just happening in IT anymore—all departments and functions are becoming data literate. People throughout the organization want the ability to slice and dice the data.
• Scarce and Expensive Talent: With all this innovation and new tech, there is a scarcity of talent. It’s hard to find people to maintain your old stack, let alone keep up with the rapidly changing demands in the marketplace. Organizations are having trouble finding the right people to bring onboard to help.

So, knowing these modern demands—both from a technical and business perspective, how do you solve the modern data problems that you’re facing?

Data modernization isn’t just a technology play, it includes people and processes too. You need a balanced approach that includes these 5 pillars: data strategy, data architecture, data management and governance, analytics tools, and the right people and processes.

The Five Pillars of Data and Analytics Modernization

1.) Data Strategy

Anytime you start on a data and analytics initiative, it is key to start with a data strategy. It is the foundation of everything you do going forward. It will act as a guide for your organization in terms of how you approach data and analytics—not just from a technical perspective, but from a people and process perspective too. It will help you answer questions such as:

• What do the employees need in order to more effectively use data?
• What processes are required to ensure the data is high quality and accessible?
• What technology will enable the storage, sharing, and analysis of data?
• What data is needed. Where is it sourced? And is it of good quality?

Your data modernization initiative should be viewed as a high-stakes project driven by a long-term strategy. And if you have a data strategy in place already, now is a good opportunity to re-examine it and align it with any changes to what the business wants to achieve and how data can help you get there.

2.) Data Architecture

You need an agile, cloud-based, future-ready data backbone that enables easier, faster, and more flexible access to large volumes of data and different data sources. Consider your current data needs and choose an approach to data architecture that can expand with your needs over time. A modern data architecture includes:

• Alternate Less Governed, Less Latent Pathways to Data:Not all data needs to go through your data warehouse. In a modern data architecture, data can take different, less governed pathways—like through a data lake or a persistent staging layer.
• Data Warehouse and Data Lake: A data warehouse is still a central component of a modern data architecture. You want a place where you can bring disparate data together, apply business logic governance to your data, and make it available in pre-curated formats. Although this can be a more latent approach, with a data warehouse, you’ll be able to create opportunities to stream data and do real-time analytics.However, in a modern data architecture, data can also take a less governed approach and go through a data lake. A data lake can store a variety of large volumes of raw data that might not have a defined use. It does not necessarily require quality checks, a certain structure, type, or format constraint on load, so companies can store some of their data in a less governed way “just in case”, providing a means for innovation and agility.

  A common recommendation we make to our clients is to include a persistent staging layer in their architecture to replicate historical source system data using change data capture (CDC) or delta loads. This space gives power users the ability to innovate and prototype solutions outside of the governed data warehouse. Sometimes this historical layer takes the form of a data lake, and sometimes it exists as a layer in the data warehouse.

• Modular Approach: By designing your solution with components that are independent of each other (and play well with one another), you can be more resilient and opportunistic when new approaches or technologies emerge. Avoid “vendor lock” so that you can evolve faster and adopt new solutions that will benefit your organization without a lot of rework or re-architecture.

You can’t take a one-size-fits-all approach to a modern data architecture. And not all organizations require the same data architecture to be modern.

3.) Data Management & Governance 

Oftentimes when you hear the word data management, you think data architecture, but that’s just one part of it. Modern data management requires that your data be accurate and available to the right people at the right time. Although technology and architecture play an instrumental part in data management, you need to have principles defined for your data when embarking on a data  modernization initiative. Those principles include:

• All Data in One Place: Can you combine data from different sources and systems to see the big picture? There are different approaches that you can take today that allow you to be able to bring all of your data together from different places to gain more context. It also allows you to reduce risk associated with business users connecting to source systems directly to get data.
• Agility: How quickly can you make new data and information available to those who need it? Are you able to take advantage of new technologies and innovations as they become available?
• Risk Management: Mitigate the risk of bad decisions based on poor data with a holistic approach to data quality that spans the entire data lifecycle.
• Scalability, Stability, and Security: This comes down to architecture—whether your data is on the cloud, on-prem, or you have a hybrid landscape. Do you have confidence that your data will always be available and only to the right audience?

If data management is a means to maximize the value you create from data, then modern analytics can be viewed as the vehicle that turns data into meaningful information that you can act upon.

4.) Analytics Tools

Think of the reports and applications you use on a day-to-day basis to find actionable information. Migrating to newer, next-gen analytics tools will provide better analytics capabilities including real-time analysis, embedded analytics, enhanced collaboration, and more. But how do you pick the right tools to match your needs when there are so many options?

Don’t simply focus on the strengths and weaknesses of the technology.

• Consider Entire Architecture When Choosing Your Tool: One of the biggest mistakes an organization can make is going through a rigorous tool selection process only to choose a solution because it’s the most aesthetically pleasing, or the price point is too good to pass up. Its critical to remember your analytics solution isn’t simply a standalone tool but part of an end-to-end modern data stack. Consider your entire data architecture, do a bakeoff and take the tool for a test drive before making a final decision.
• Assess Skills Sets: Your people must be factored into the decision-making because they’re ultimately going to be the ones responsible for building and developing reports. A modernization project isn’t just a shift in technology; it’s a shift in the skillsets required within your organization. If your team of developers need to pivot from their legacy platform to a modern analytics solution, they need an enablement plan and a training plan in place to mitigate the learning curve.
• Focus on Short Term: Modernization for many companies often means there are years of analytics reporting that needs to be migrated to a new tool. A migration strategy will help you overcome this task by narrowing your focus on the short term: what will be migrated initially, and which apps and reports can be retired. Take this as an opportunity to triage and revisit requirements. Do you really need ten years of historical data, or does two years suffice?
• Put Together a Roll-Out Plan: When it comes to reporting, the modernization effort is as much educational as it is technical. In order to encourage adoption, a roll-out and training plan must be in place to educate end users on the feature functionality of your new analytics tool. Consider scheduling a one-day bootcamp training with your end users to educate them on their new analytics tool.

5.) The Right People and Processes

In a migration the natural focus tends to gravitate toward the technology itself. But its critical to remember that a modernization effort is not just a shift in technology; it’s also a shift in skillsets required within your organization. There are some key considerations that you should think about to ensure a successful migration:

• Training and Enablement: Training is core to any migration effort. There are two distinct pathways for training and enablement. The first is developer focused training for your folks on staff who will be building, maintaining, and owning your analytics solutions. Consider this your “how to build” learning track. The second is training and enablement targeted to end users. This is your “how do I use it” learning track. It’s important to delineate between these two tracks, and not adopt a one-size fits all approach in training.
• How Your Organization Receives Training: Knowing your audience is key to how your users and developers will receive training. Understanding how your people learn best—one-day bootcamp, classroom training, webinar, documentation-only, etc.—will allow you to deliver training that maximizes value. Also, consider if you have the capabilities to deliver that training internally, or if you need outside help.
• Load Balancing: In any migration initiative, there will be a period of time where your legacy reporting and modern analytics tool will likely both be live at the same time. Recognizing the demand on time this creates is essential and distributing the workload effectively across both tools is key.
• Don’t Just Lift and Shift: What works in one tool, is not simply transferable to the next. Recognize that your new analytics tool is a different platform at the outset, and that you need to build to the strengths and capabilities in your new tool, and not simply try to recreate concepts that worked in your old tool. Unlearn quickly, and continually ask yourself how you’re going to approach and do things differently in your new analytics tool.