Analytics Advantage – Part 1A: Democratization

This article is an excerpt from the first report: The Democratization of Analytics for Transportation and Logistics.
A copy of the full report can be downloaded here.

This Part 1A is the first in a multi-part series on using analytics in transportation and logistics to achieve a competitive advantage:

• Part One: The Democratization of Analytics for Transportation and Logistics — Here in Part One, we explore the kinds of data being produced by transportation and logistics systems, how that can be used to create a data-driven enterprise, the substantial obstacles to achieving an analytic advantage, and how those obstacles can be overcome.
• Part Two: Analytics for Private Fleet and Driver Performance — In Part Two, we look at how analytics uses real-time location data, combined with orders, plans, proof-of-delivery, vehicle data, and more to drive significant improvements to fleet and driver performance.
• Part Three: Analytics for Improving Carrier Performance and Leveraging Trade Data — In the third and final part, we discuss how analytics can create improvements to carrier performance while substantially reducing costs, as well as how trade data can help importers and exporters gain competitive insights, manage supply chain risk, optimize total landed costs, and more.

The Analytics Advantage

As companies continue to digitize their supply chain and logistics operations, they are beginning to recognize that they are sitting on a wealth of information that has an enormous, yet largely untapped potential to help increase their performance and improve their customer service. Increasingly, there are examples of how companies have gained significant value through the data that is available to them. Not only do their internal systems generate volumes of relevant data, but there are external sources that can be leveraged to give a comprehensive view of their business, supply chain, and even their entire industry. This level of insight is profoundly changing how companies use their supply chain and logistics operations and the value they deliver.

Supply chain and logistics systems generate a wealth of valuable data. Core planning and execution systems such as a transportation management system (TMS) or route planning, real-time visibility, mobile, and telematics are prime data generators. For companies that operate internationally, importing or exporting of goods often use global trade management systems to classify goods, submit filings, and ensure compliance, creating additional valuable supply chain and logistics data. All of the data from these systems need to be brought together into a cohesive interlinked set, in order to provide deep insights into operational performance and customer service. Companies that are able to intelligently use this data, become data-driven enterprises that realize a powerful ‘analytics advantage.’

Figure 1 – Using Logistics-related Data to Create an Analytics Advantage

Data-driven Organizations Realize an Analytics Advantage

Data-driven, analytics-capable organizations are able to ask the right questions and extract tremendous value from the mountains of data they already have. They are able to find very specific ways to improve their purchased transportation or fleet performance, evaluate their ability to better serve their customers and the cost to do it, understand and optimize the total landed cost of their international sources, improve how well they are complying with regulatory mandates, and much more. Becoming data-driven fundamentally changes an organization’s culture, decision-making processes, and the role of metrics.

Table 1 – Characteristics of Traditional vs. Data-Driven Organizations

The Democratization of Analytics

What really sets a data-driven organization apart is how widely and deeply diffused the use of data and analytics is throughout the enterprise. Analytics should not be the exclusive domain of a few data scientists isolated in their ivory towers. Data-driven insights and the ability to ask questions should be available throughout the organization, at the point wherever decisions are made. Analytics is democratized by infusing them into the systems and processes used by professionals throughout the organization.

Analytics should be accessible to all — the people at the front lines making the day-to-day decisions and getting things done, as well as to the managers and executives responsible for functional and business unit effectiveness. Broad access to good analytics ‘lifts the blinders off’ for an organization. Employees gain much more specificity into where and how performance can be improved, as well as the tools for early identification and correction of execution issues. The organization becomes much more agile and competitive — quickly sensing issues and quickly adjusting course, solving those issues before they become bigger problems. Employees are more empowered to solve problems directly and effectively. As a result, morale, individual performance, and job satisfaction improve. Democratization is a powerful force.

What Keeps Companies from Realizing Their Analytics Advantage?

As more and more competitors become data-driven, those who don’t do so will be at a major disadvantage. Many companies are constrained in their use of analytics due to a set of interrelated challenges:

• Data Science Talent Shortage — While there are shortages of IT talent across the board, data scientists and big data analytics specialists top the list as the scarcest.²
• The IT Bottleneck — For organizations lucky enough to have some in-house talent, the backlog³ of projects can stretch for months or years.
• Data Wrangling Challenges — Data wrangling typically consumes 50%-80% of data scientist/analytics project resources, with precious little time and resources left for building models and doing analysis.

Data Science Talent Shortage

The shortage of data scientists has made it hard for even deep-pocketed companies to hire and retain an adequately staffed in-house analytics team. LinkedIn’s 2018 Workforce Report said, “Demand for data scientists is off the charts.” The report singled out data scientists over all the other jobs.

The number of data scientist jobs grew by over 6.5X from 2012-2017⁴ and continues to grow at over 50%/year. Most estimates indicate there are more than twice as many data science job openings as there are qualified candidates. The consensus is that demand for these skills will continue to increase faster than the supply of qualified candidates for many years to come.

The IT Bottleneck

Hiring good data scientists is only part of the battle. Most IT departments are already stretched thin and being asked to do more with less. Over 70% of their time (on average) is consumed maintaining and supporting existing systems, rather than helping implement new functionality or delivery of new types of analytics and reports. IT department project backlogs often extend to over a year (in some cases several years). Analytics projects just have to get in line. That is the opposite of being agile for a business trying to keep up.

Data Wrangling Challenges

Data wrangling involves finding, accessing, organizing, cleaning, and enriching data from many different sources into a cohesive data set that is useable for analytics.⁷ The effort required to do this is usually vastly underestimated. For more details on the activities involved and what consumes all that time, see the sidebar What Makes Data Wrangling So Time-Consuming?

Given all these challenges, individuals often manually gather whatever data they can and ‘grind out answers’ in Excel. They can use spreadsheets to try to find underperforming drivers, carriers, and so forth, to get a better handle on what is going on. Yes, they can get answers this way, but it is a very manual approach. A more scalable and sustainable approach is needed. Partnering with the right transportation and logistics solutions provider can be the answer for many companies.

In Part 1B of this series, we look at how solution provider capabilities can be leveraged to overcome some of these challenges.

_____________________________________________________

¹ For more on these activities, see What is Data Wrangling and Why Does it Take So Long. — Return to article text above
² In the Harvey Nash/KPMG CIO survey, CIOs were asked “Which functions do you feel suffer from a skills shortage?” Presented with a list of 22 different IT-related functions, Big Data/Analytics topped the list in both last year’s survey (What’s Driving Data Science Hiring in 2019?) and this year’s survey (see chart in the results here-scroll down to question 16). — Return to article text above
³ IT backlogs are not just due to the workload on data scientists from analytics projects, but all of the IT staff from a variety of demands such as maintenance and support of existing systems (which consume on average over 70% of IT budget and resources), massive implementations or major upgrades to ERP or other systems, integration and data cleanup efforts, and so forth. — Return to article text above
⁴ See LinkedIn’s 2017 U.S. Emerging Jobs Report — Return to article text above
⁵ For example, suppose you have rich data about a subset of customers, but that subset is not a representative sample of your broader customer base. Perhaps you have certain data only from opt-in customers, and maybe they are younger or of a certain mix of persona types that is different from your overall base of customers. For more, see Picking Favorites: A Brief Introduction to Selection Bias — Return to article text above
⁶ Other types of bias in the data include seasonal bias, linearity bias, confirmation bias, recall bias, survivor bias, observer bias, and reinforcement bias. — Return to article text above
⁷ Data wrangling may also be used to prepare data for AI and machine learning uses. — Return to article text above

To view other articles from this issue of the brief, click here.