Marketing mix modelling (MMM) is one of the most prominent methods in marketing data analysis. An entire industry is built around MMM guiding marketers through planning their marketing strategies. It is particularly well established in industries where individual customer tracking is difficult and, where mass media advertising dominates the media budget. These include in particular CPG, and some consumer services, such us entertainment or fast food restaurants or retail.
As a result of MMM widespread use and popularity, it has been attracting lot of criticism and scrutiny over the years. Some comments are valid and the potential issues can be resolved by experienced modellers. However, lots of criticism is driven by insufficient understanding of the methodology and/or its inadequate application.
We want to raise awareness of MMM through addressing the most commonly criticised issues. A reader can see how one can go about applying the methodology for the best results.
Let’s start with the most common criticism of MMM.
The statement that past performance is not an indication of the future is supposedly the most common criticism of MMM.
This is a valid observation. Modellers are well aware of this issue. In response, the methods used in MMM are explicitly designed to address it. Thanks to in-built robustness of regression techniques, the most persistent patterns are extracted and modelled. Even though some details are not captured in the process – robustness of remaining components gives modeller confidence in the interpretation of past results as well as in projecting the expected effect of future actions.
The underlying idea is not that past response patterns predict the future with certainty but that they contains valuable insights that might be instructive in the future planning. While no one can assert the future with certainty, marketers should not ignore the past entirely.
Focused on Explaining Past Rather Than Projecting Future?
It is often questioned, how suitable MMM methodology is for projecting future sales. This has strong implications for MMM actionability.
Reliability of projections greatly depends on the stability of market environment under study. With expected changes in either of them the forecast window is relatively short-term. In order to mitigate “time-myopia”, modelling and forecasting should become a continuous process. In this case important structural changes would be immediately captured in models and forecasts would be regularly updated.
After covering the two fundamental points let’s move on to the methodological and data-related objections to MMM. We discuss below how relevant these issues are and how modellers and marketers can handle them.
Causality vs. Spurious Correlation?
Some stakeholders put doubts on the model specification. In particular, they question whether estimated effects in MMM are indeed causal and they are not simply spurious correlation between two variables.
While causality implies that one process is driving another, correlation, and in particular,spurious correlation, means that two processes simply show a similar pattern.
If one wants to base decisions on the model results, it is important to ensure that only significant causal relationships are included.
With this in mind, the selection of potential factors needs to be guided by an economic or marketing theory and focus on those factors where previous research showed causal relationship.
In addition, modelling should be preceded by a detailed exploratory analysis. This includes plotting relationships, calculating correlations and, where appropriate, testing the direction of the relationship. Granger causality test is a well-established methodology to perform this exercise.
MMM Capturing Many Small Correlations Rather Than Robust Relationships?
MMM is based on the econometric regression, which is a parametric data modelling method. If the relationship between the explanatory variable and the rest of the model is not statistically significant (in plain English it means the relationship is not stable and equally strong over time), the modeller, after examining diagnostic parameters, excludes the variable from the model. The parametric methods provide clear rules how strong the relationship needs to be to include the respective factor in the model.
In multivariate regressions, the effect of the explanatory variables included in the model is determined simultaneously. The link between the magnitude of an impact and significance of the relationship can be small. One can find a very significant relationship with a small magnitude.
Measuring Only Short-Term Effects?
Measuring short- vs. long-term effects has implications for planning marketing communication. Models capturing only immediate effects of marketing and ignoring all its downstream consequences would favour activities with large immediate effects. Even if some might be less optimal in the longer term. On the other hand they may not give fair credit to the activities with gradually increasing positive effects. This applies to budget split recommendations between promotions and advertising. High immediate volume uplifts of price-promoted products might dominate smaller and gradual advertising effects, while potentially harmful longer-term effects of price promotions are not accounted for.
Well thought-through methodology is here the best remedy. First, modellers use one of several available methods to measure memory effects of advertising over time. These are adstocking or distributed lag models that enable to account for short- to medium-term effects of advertising. Second, given that most mix models cover the period of around three years, the possibilities of measuring long-term effects are to some extent limited. One can look at base level dynamics or construct a variable capturing lagged advertising spend level over time to account for longer-term advertising impact.
Similar applies to measuring the effects of price promotions. MMM allows assessing their immediate effect on sales as well as their potential post-promotion dips, and longer-term impact on base sales and price sensitivity.
All the effects are factored into recommendations on budget split for advertising and promotions. To sum up, mix models can measure both short and longer-term effects of marketing. The right model specification allows testing a wide range of hypotheses.
Incorrect Measurement Due to a Lack of Variable Variation?
We can divide this issue into two parts: (i) Low variation of modelled variable and, (ii) Low variation of explanatory factors.
Low Variation of Modelled Variable
Some big established brands with strong base can show only a small response to changes in advertising. One can think of low-involvement products that are essential part of consumption basket.
Nevertheless these brands can still benefit from MMM. The first thing is to reassess which KPI should be modelled. These can be the share of market (SOM) instead of, or along with, absolute sales volume. Measuring our results relative to competitors can be instructive for brand management. Second, more Test-and-Learn can be recommended for these brands to bring further dynamics. MMM enables disentangling the impact of individual activities and recommends the most viable candidates for scaling up.
Also, seemingly smooth large base can be driven by multiple positive and negative effects changing over time. Modelling helps understand these better.
Low variation of explanatory factors
Some important factors, such as distribution, may change only occasionally. This makes it difficult to incorporate them in the model.
Models in general cannot quantify the impact of those variables that were stable over the modelled period. Their effect is embedded within the base.
However, a small variation in some factors does not necessarily mean stable sales. Modelling can still provide lots of insights into the effect of external factors and interactions of our brand sales with competitor dynamics.
Omitted Variable Bias?
Omitting important variables from the model can incorrectly apportion their impact (or part thereof) to other factors. On the other hand, complex models with many variables might not work well. How to get this right?
The good news is that unlike more granular methods of marketing analysis, MMM can incorporate more factors and well account for wider macroeconomic and technological developments.
Modellers take several steps to ensure that no important factors are missing in the model.
First, before the modelling stage starts, the modeller gets input from stakeholders and uses own domain knowledge and experience to identify all potential factors. This is followed by detailed inspection of data, that includes correlation and sensitivity analysis and data visualisation to reveal data relationship which should be further tested. When modelling starts, the modeller has already got a fair amount of knowledge about the modelled brand and data, what variables should be tested and what is the expected outcome.
Therefore marketers should support modellers in their search for more complete data sources. Thorough data preparation is instrumental for good quality of the model.
Fail to Capture Synergies?
The modern markets are complex. Brand’s sales are driven by a range of factors, many of which interact. It is sometimes argued that models understate the value of marketing by failing to incorporate interaction effects.
One can account for interactions and synergies in MMM and most modellers do.
First, accounting for standard part of "synergies" is embedded in the multiplicative model form. This approach assumes that every additional unit of explanatory factor increases the modelled variable by x%. Basically the higher sales were already achieved, the stronger will be the absolute effect of an additional factor. The multiplicative models are essential if modelled KPI is driven by multiple overlapping factors. Second, where interactions between specific pairs of factors are expected, one builds variables to capture these effects. Since one does not have a prior knowledge of which pairs of factors interact, exhaustive search or evolutionary method can be used to identify most significant interactions.
Lack of Knowledge on External Factor Development?
Additional reason which may impair accuracy of projections is uncertainty of external factor development. Even though, we cannot control these factors, we can predict their development. If such exercise is beyond the scope of MMM project, one can compare the expected model outcome under different scenarios of external development.
For this purpose, MMM consultancies build decision-support tools. Based on the built models, the tools enable to simulate scenarios of different external developments and competitor activity levels. Through comparisons of expected outcomes, marketers can determine which strategies work better and how robust they are to external dynamics.
The goal of this note was to contribute our views to the discussion of some aspects of MMM. We agree that MMM, as any other methodology, has its pros and cons and some product categories can benefit from it more than others.
MMM is definitely very effective at providing "a big picture" of the main drivers of sales and incremental effect of media communication and price dynamics. It is one of the most suitable analytical methods that can measure well the impact of mass media advertising when individual customer data are not available.
For many brands, FMCG in particular, MMM is a very valuable instrument for informed planning of marketing activities. Adding further more granular methods to the analytical suite enables to compare and cross-check its results and improves the overall insight and understanding of major brand drivers. These, when turned into actions, bring brands closer to improving productivity of brand's marketing investment.
For marketers from data-rich industries, MMM provides an additional angle to assess marketing efforts through quantifying incremental value of marketing activities and recommendations for media channel mix improvements.
Sydney, 02 April, 2013,
Elena Yusupova, PhD