Marketing Mix Modeling (MMM) traditionally employs statistical metrics such as R-squared, Mean Absolute Percentage Error (MAPE), Root Mean Squared Error (RMSE), and Akaike Information Criterion (AIC) for model calibration and evaluation. These metrics, while insightful, often fall short in addressing the complexities of real-world scenarios. This report explores advanced analytical techniques, focusing on the Probability Integral Transform (PIT) residuals and Kullback-Leibler (KL) divergence, to enhance the calibration of MMM. Our findings indicate significant deviations from uniformity in the PIT residuals for both optimal and suboptimal models, with the best model demonstrating lower KL divergence, suggesting a closer fit to the expected uniform distribution. This study underscores the value of incorporating advanced metrics for a more nuanced understanding of MMM calibration, beyond conventional evaluation methods.

This report presents a novel approach to validate the efficacy of Marketing Mix Modelling (MMM) using Difference-inDifference (DID) technique. We use DID to measure the impact of MMM by comparing outcomes from two markets-one receiving increased marketing investments and the other maintaining existing strategies. The results confirm the causal effects of MMM interventions, demonstrating DID's utility in showing MMM's practical benefits and encouraging its broader adoption.

Feature Selection (aka covariate/independent variable selection) in any regression problem is a crucial task. Incorporating features in the model that best predicts or explains the dependent variable is both an art and science. There are many feature selection methods. In MMM, incorporating the right features in the model is a very tricky proposition. Traditionally, in MMM, feature selection is done using a combination of correlation approaches and domain informed judgments about the variables. However, this is not a robust approach since this can lead to Bias in the model. As a workaround, we have been constantly experimenting with other feature selection methods. In this paper, we propose that Granger Causality could be a potential feature selection method. We illustrate how the Granger Causality (if carefully used) can aide in feature selection in an MMM setup.

This report is an investigation into the trade-offs between Predictive Accuracy and Business Impact in Robyn, which uses the Nevergrad algorithm for optimizing (Normalized RMSE) and Business Error (Decomposed Residual Sum of Squares) as independent objectives. We examined models with the best and the worst Decomp.RSSD on the Pareto frontier, analyzing their performance through the lenses of Kullback-Leibler (KL) Divergence and Chebyshev's inequality. The aim is to explore how these models balance the dual objectives of error minimization and "Business Impact", highlighting the complexity of selecting the "Best" model when considering both statistical alignment and business relevance. Analysis revealed unexpected trends between the Best and Worst models in terms of KL Divergence and error clustering, highlighting a trade-off between minimizing business error and maintaining predictive accuracy, and pointing to the need for a nuanced model evaluation approach, and a delicate hand in choosing the final model.