Going from Hindsight to Foresight 🚀 Understanding the analytic value escalator and Prescriptive Analytics

aking a decision is easy but knowing how to make the right decision at the moment of choice, now that is tricky. As the outcomes and consequences are only known after the decision has been made, we try hard to mitigate the risk of making a wrong one.  

Like a game of probability, we weigh different information and data, and play out the possible outcomes against each other to narrow down our choices, and, well, make a bet. Given the vast amount of information and data available, gathering the needed and relevant information can be a challenge. For the human mind it is impossible to grasp all inputs and data at once. And it is practically impossible. Additionally, as we learn new information, we may create new connections and gain new insights that open new possibilities. Which often leads to the question, "What if...?"

Lastly, before executing the decision, we weigh our options and evidence, and filter it through the personal and/or corporate value filter. By repeating this process, and adding a decision-review step, we learn how to make better decisions. The more we know, the more experience we have, the better our chances of making the best possible choice. And that is how it has been for the last ten of thousands of years.  

While we have evolved our ability to gather and access information with software, and made the analytical part simpler and more accessible, machine-assisted decision making and execution is about to change the decision-making process.

AI and decision making

The human brain can process 11 million bits of information per second, but our conscious minds can handle only 40 to 50 bits per second. And while we do not always forget, retrieving the right information at the right time is not straightforward.

Our ability to gather and analyse data is limited by our knowledge, time, and “computational power.” However, if we know what information we need, there are now thousands of tools that can help us gather the data and connect it with other data sources to uncover new insights and patterns.

Predicting the future based on historical patterns is not a complicated science, but rarely a trustworthy one. Machine learning algorithms have increased the accuracy and given us a better foresight of how decisions and events might unfold, making it possible to simulate different scenarios and study decision consequences without having to execute a decision. The possibility of setting up “What-if” scenarios and playing them against each other, pushes us closer of being able to make the right, rational decision.  

Building on the previous point about the importance of good data, let us talk about the challenge of data diversity. Machine learning models are only as good as the data they are trained on. If you train a model on a narrow dataset, it will only be able to make predictions that are relevant to that dataset. For example, an automated script writer that is only trained on movies and books written by Quentin Tarantino will always produce scripts that are similar to Tarantino's work. The same thing happens if you run your analytics only based on your company's internal data without considering external data such as market and competitor data.  

Powerful and accurate models combine data from a variety of sources to reduce bias, improve generalisation, and identify new patterns and insights. For example, a company that is developing a model to predict customer churn could combine data from its internal CRM system with data from external sources such as social media and customer reviews. This would help the company to identify patterns and insights that it would not be able to see by looking at its internal data alone.

Prescriptive analytics  

The one type of analytics that will profoundly change our decision-making process, and profoundly change how we work, is prescriptive analytics.

Prescriptive analytics is (currently) the final stage in the analytics spectrum, which includes descriptive, diagnostic, predictive, and prescriptive analytics. Descriptive analytics answers the question "What happened?", diagnostic analytics explain “Why it happened!”, predictive analytics addresses "What might happen?", and prescriptive analytics tackles "What should we do about it?", including all former analytics in its process.  

When we make decisions, all these analyses happen naturally in our brain and are part of our decision process. The extent of how much we analyse depends on the time we have, the number of people involved, and the consequences of the decision. If we have little time or the stakes are low, we may make a quick decision with minimal analysis. However, if we have more time or the stakes are high, we will spend more time trying to analyse the situation and considering our (imagined) options.

If we turn to machine-assisted decision making powered by prescriptive analytics many of parts of decision process become automated. Using machine learning, algorithms, and computational modelling, prescriptive analytics provide insights, simulates different scenarios, and suggest actionable steps in response to a predicted outcome or scenario.

For example, in supply chain management, prescriptive analytics might suggest optimal routes for delivery based on predicted weather conditions, anticipated traffic patterns, and historical accident data. Or, in finance, it could recommend investment strategies based on a forecasted economic downturn.

A New Paradigm of Decision-Making with Prescriptive Intelligence

A step-by-step decision-making process includes most commonly these seven parts:

1. Identify the decision
2. Gather information
3. Identify alternatives
4. Weigh the evidence
5. Choose among the alternatives
6. Take action
7. Review the decision

Imagine that you have a data foundation that gathers all your data in one place, both external open data (market, competitors etc.) and internal. You have billions of rows of present and historical data, cleaned, enriched, and contextualised. You are a Business Development Manager at a Last-Mile delivery company, and you are tasked with expanding sales to a new area. Where do you start?

1. Identify the decision

In which geographical area can we increase our revenue the most?  

2. Gather information

Where are our competitors present?  
What are our competitors' prices?  
Where are our terminals?  
How much are we today delivering in each area?
What delivery options are the most popular in which area?  
What investment will be needed for each area?  
Etc.  

3. Identify alternatives

All areas and options are listed. Business cases are presented.

4. Weigh the evidence

Alternatives are weighed against each other. Pros and cons are discussed.

5. Choose among the alternatives

Once you have weighed all the evidence, you are ready to select the alternative that seems best for the company. You may even choose a combination of alternatives.

6. Take action

You implement the chosen alternative. It is time for execution.

7. Review the decision

You review the results of the decision and see how your expansion plan is working out and iterate.  

With prescriptive intelligence in place, the machine assisted decision-making process is similar, but at the same quite different as the effort lies in the beginning, and not the collection of information. We assume here you have access to a tool that combines market data with internal data.

1. Identify the decision

In which geographical area can we increase our revenue the most?  

2. Goal formulation (prompting)

What are the results that you are looking to achieve and through what means. List interesting areas for exploration and factors you think are relevant.

3. Scenario evaluation

Alternatives and scenarios are simulated and presented by the AI describing the steps needed to reach formulated goal. Costs and risks are listed based on data that is available. You have the possibility to deep dive into areas to expand your analysis or follow the recommended path.

4. Weigh the scenarios

Recommendation is weighed against the other scenarios.

5. Scenario implementation

You implement the chosen scenario and measure against milestones and goals set by the AI.

6. Review the chosen scenario

The decision and chosen scenario are evaluated in real time with the AI to ensure ongoing learning and optimisation.

If we look past the fact that much of the decision-making process is automated, we move from hypothetical discussions around outcomes and consequences to an evaluation of the proposed steps to reach the decision and set goal. The proposed scenario is not unbiased and unemotional, it is guiding force explaining how to reach that goal with what is available.  

Science fiction?  

Prescriptive intelligence is not something we imagine anymore, it is being worked on today, and there are already solutions in the market for specific use cases. Our decision-making process will not only be faster (timewise), but we will also be able to be much more accurate in understanding outcomes and the decisions in between we need to make to reach a certain goal.

Finding the competitive edge

If everyone can afford the same tools and have access to the same data, isn't there a risk that we will all pull towards the same goals in our respective fields? Isn’t it all about increasing profit through expansion or decreasing costs?

The chances of that scenario are limited.

Not one company has the same data as another one. We can acquire datasets, predictions, but in the end how we operate, they people we employ, the decision we made, and our assets and business models are not the same. Each company has its own strategy, so even if we all access the same market intelligence, the outcome will be different. But just as generative AI has shown with ChatGPT and Midjourney, the playfield has become much more even.  

Market analysis and expensive data is becoming less expensive and available to a larger extent of companies, and not only the big ones.

Market intelligence

A general prescriptive analytics platform is still a couple of years in the future. At Tembi, we have built the data foundation for it, and are constantly working on adding new machine learning based prediction and econometric models to create better insights and foresights for our clients based on open data.

While companies have their internal data, we provide extensive access to open data, and ready-to-go-analytics – or market intelligence – that provide actionable insights to the decision-making process. Many of our clients use our API to connect their data with our data to examine and understand (i.e.) volume fluctuations (revenue drivers) with external events, and hence be able to understand how external factors impact their business, mitigate risk, or uncover new business possibilities.

The more we connect the world's Information the better we will understand the future, and the more impact our decisions will have. And that is why we work here at Tembi. Until we provide a general prescriptive intelligence platform for executing successful business decisions, we focus on providing market intelligence that is beyond what can be seen by a person online. We combine data from multiple industries and build market predictions models based on changes across different industries.  

‍

‍

n today's data-driven world, the abundance of information and the advancement of analytical tools have sparked a competitive quest for insights. As data becomes more affordable and accessible, the ability to use this data effectively becomes a decisive factor in staying ahead. But having data is one thing; making sense of it to predict the future is quite another. It is a complex task that goes beyond just crunching numbers—it is about weaving together diverse parts of information, both old and new, to form a clear picture of what lies ahead.

This article aims to untangle the concept of Predictive Market Intelligence, demonstrating how it operates and its value in a business context. We will look at how this approach to data can lead to smarter decisions and how it is shaping the way companies move forward.  

What is Predictive Market Intelligence?

Predictive Market Intelligence (PMI) stands at the confluence where big data analytics, artificial intelligence, and advanced market research meet. It is the art and science of collecting vast amounts of open data - from (i.e.) market trends, company behaviour, to global economic indicators - and analysing them to forecast future market conditions. The aim of PMI is not only to investigate information based on past market performance – historical data – but to forecast the evolution of markets, specific industries, or companies, by employing diverse analytical methods and algorithms.

Unlike traditional market research, Predictive Market Intelligence is dynamic, constantly refining its insights with a steady stream of real-time data. This process enables businesses to not just interpret the present but also to anticipate and prepare for future market developments, gaining foresight and deepening their understanding of potential future scenarios.

Applications of Predictive Market Intelligence

If companies can use Predictive Market Intelligence to gain foresight, can PMI be applied everywhere, or are there particular interesting applications of this approach to market analysis and strategy? Here are a couple of examples:

Enhanced Forecasting Abilities

• Anticipating Market Trends: Predictive Market Intelligence allows companies to not just understand current market dynamics but to anticipate future trends. By analysing patterns in data, businesses can foresee changes in consumer preferences, economic shifts, or industry disruptions. This foresight enables them to adapt their strategies proactively rather than reactively, staying ahead of the curve.

• Identifying Emerging Opportunities: With Predictive Market Intelligence, companies can spot emerging opportunities in their industry. This could include untapped market segments, new product possibilities, or innovative service offerings that have not yet been fully realised by competitors.

Data-Driven Decision Making

• Reducing Uncertainty: In business, uncertainty can be costly. Predictive Market Intelligence significantly reduces this uncertainty by providing data-backed insights. When decisions are based on solid data, the risks associated with them are significantly lowered.

• Strategic Alignment: Predictive Market Intelligence aligns various aspects of a business - from marketing and sales to product development and supply chain management - with the overall market dynamics. This alignment ensures that every part of the business is working towards a common, data-informed goal.

Improved Customer Understanding

• Tailored Customer Experiences: By understanding customer behaviour and preferences through Predictive Market Intelligence, companies can tailor their products, services, and marketing efforts to meet the specific needs and desires of their target audience.

• Building Customer Loyalty: Businesses that consistently meet or exceed customer expectations foster stronger customer loyalty. Predictive Market Intelligence plays a crucial role in enabling businesses to understand and predict what their customers want, often before the customers themselves know.

Operational Efficiency

• Streamlining Operations: Predictive Market Intelligence can identify inefficiencies in operations, supply chains, and production processes. By addressing these inefficiencies, companies can reduce costs and improve their overall operational effectiveness.

• Resource Optimisation: With Predictive Market Intelligence, businesses can allocate their resources more effectively, whether it's human resources, capital investment, or marketing spend, ensuring that every dollar spent is optimised for maximum return.

Competitive Analysis

• Benchmarking Against Competitors: Predictive Market Intelligence tools can analyze competitors' performance, strategies, and market position. This insight allows companies to benchmark their performance and strategise accordingly to gain a competitive advantage.

• Adaptive Strategies: In fast-paced industries, what works today might not work tomorrow. Predictive Market Intelligence empowers companies to quickly adapt their strategies in response to competitive moves or market shifts.

Technology Behind Predictive Market Intelligence

Retrieving Market Intelligence is a question of gathering data from various sources, organising the gathered data, and applying different technologies to validate, enrich and put the data into context. The last step is to apply different analytical models depending what outcome one is looking for. So, where the first step is about gathering (open) data, the second analytical step is the creation of synthetic data (programmatically generated data).  

Each step of the process, from open data to intelligence, uses different technologies. Each plays a unique role and function, but applied together, collectively, these technologies can create incredibly precise projections. Let us dive into a couple of them.

Data Mining and Aggregation

Central to Predictive Market Intelligence is the process of data mining and aggregation. This involves the meticulous gathering of vast volumes of data from a multitude of sources like public information, financial reports, and for example websites. The objective is to amass a comprehensive dataset that encapsulates the diverse aspects of the market and company behaviors. This rich tapestry of data forms the foundation upon which further analysis is built.

Artificial Intelligence and Machine Learning

Artificial Intelligence (AI) and Machine Learning (ML) stand at the core of Predictive Market Intelligence, processing and interpreting the extensive data collected. AI algorithms are adept at discerning complex patterns and relationships within the data, which are often imperceptible to the human eye. Simultaneously, ML models, with their ability to learn and improve from the data, continuously refine their insights, ensuring they remain relevant and accurate in a rapidly changing market.

Natural Language Processing

A key component in understanding context is Natural Language Processing (NLP). NLP technologies delve into text-based data, analysing news articles, pdfs, and websites. They are particularly effective in understanding the context of the written text, and being able to synthesis substantial amounts of data and help verify what the data is

Predictive Analytics

Predictive analytics brings a forward-looking perspective to Predictive Market Intelligence. By employing statistical and econometric models as well as forecasting algorithms, it anticipates future market behaviors, trends, and company needs. This facet of Predictive Market Intelligence is instrumental in risk assessment and scenario planning, allowing businesses to prepare for various future market scenarios.

Big Data Analytics and Cloud Computing

Big Data Analytics provides the muscle to process and analyze the immense datasets characteristic of Predictive Market Intelligence. It offers real-time analysis and sophisticated data visualization tools, making complex data understandable and actionable. Complementing this is cloud computing, which offers the necessary infrastructure for data storage and analysis. Its scalability ensures that businesses can adapt to varying data demands, while also offering cost-effective solutions compared to traditional in-house data centers.

For Experts and Beginners Alike

Predictive Market Intelligence is not only for experts. With platforms such as Tembi, PMI is today accessible for everyone, regardless of analytical skill set. While there are use-cases that require tailormade algorithms, predictions such as company growth, market trends and econometric forecasts are already available. And with decision-ready market insights, companies can quickly adapt to a data-driven decision process without heavy investments.

Predictive Market Intelligence for Experts

For the expert, Predictive Market Intelligence serves as an advanced tool that complements and elevates their analytical skills. PMI can be used to validate hypotheses, refine models, and conduct in-depth analyses that underpin robust, strategic decisions.

The technology used in Predictive Market Intelligence lets experts quickly sort through and understand huge amounts of data. This means they can get a clear picture of how markets are changing, what competitors are doing, and how companies are behaving. With this kind of intelligence, experienced professionals can make accurate predictions and find new business opportunities before anyone else does.

For less savvy analytical minds

For those new to Predictive Market Intelligence, it can seem both exciting and a bit overwhelming at first. But this technology simplifies the process of understanding the market by turning complicated ideas into clear insights. It provides easy-to-use tools and clear visuals that help make sense of complex data.

With Predictive Market Intelligence, even those just starting out can get a complete view of the market. They'll learn to spot the important signs that show changes in what consumers want or in the economy. This technology is like having a guide and a coach in one, helping new users think strategically and make decisions based on data.

A Convergence of Knowledge

Predictive Market Intelligence acts as a bridge between theory and practice, enabling a fluid exchange of knowledge across all levels of expertise. It is a field that values the knowledge of the expert and nurtures the growth of the newcomer. By fostering an environment where learning is continuous and insights are accessible, Predictive Market Intelligence ensures that all users, regardless of their level of expertise, can contribute to and benefit from the intelligence it provides.

What is next for Predictive Market Intelligence

The future of Predictive Market Intelligence looks particularly promising as cloud computing costs, which have been a significant factor in the past, are expected to continue their trend towards more economical and efficient services. As the price-performance ratio of technologies like GPUs improves, companies can leverage more powerful analytical capabilities at a lower cost. This could further democratize PMI, allowing smaller businesses to engage with what was only accessible to larger corporations. The integration of emerging technologies such as distributed cloud and advanced AI (Artificial Intelligence) algorithms will further enhance PMI's accuracy and speed, offering businesses of all sizes the predictive insights needed to stay ahead in an increasingly data-centric world.

What will be key, as always with the development of analytics and AI, is the quality and the amount of data. With a democratization of technology, the winners will be the ones that invest in good data gathering processes – both internal and external open data – and have solid data partnerships in place.

One thing is sure, we have only touched the very beginning of this approach. But already today, it is evident that companies that utilize external data in their decision process, have far better chances of making better decisions. Giving them a better competitive edge.

‍

‍

‍