Deep Learning does not Replace Bayesian Modeling : Comparing research
use via citation counting
Abstract
One could be excused for assuming that deep learning had or will soon
usurp all credible work in reasoning, artificial intelligence and
statistics, but like most ‘meme’ class broad generalizations the concept
does not hold up to scrutiny. Memes don’t generally matter since the
experts will always know better but in the case of Bayesian software
like Stan and PyMC3 even its developers and advocates bemoan the
apparent dominance of deep learning as manifested in popular culture,
breathtaking performance and most problematically from funding agency
peer review that impacts our ability to further advance the field. The
facts however do not support the assumed dominance of deep learning in
science upon closer examination. This letter simply makes the argument
by the crudest of possible metrics, citation count, that once Computer
Science is subtracted, Bayesian software accounts for nearly a third of
research citations. Stan and PyMC3 dominate some fields, PyTorch, Keras
and TensorFlow dominate others with lots of variation in between.
Bayesian and deep learning approaches are related but very different
technologies in goals, implementation and applicability with little
actual overlap so this is not a surprise. While deep learning is backed
by industry behemoths (Google, Facebook) the Bayesian efforts are not
and it would behoove funders to recognize the impact of Bayesian
software given its centrality to science.