This paper provides new perspectives on explainable machine learning and deep learning, from the view point of an explicit system-centric approach. Unlike most existing commentary on the said topic, this paper seeks to explain the fundamentals of the black box problem in machine learning by focusing on explicit system modeling. A convenient yet interesting example is used to illustrate the ideas in a tutorial-like manner thereby making the article accessible to readers from diverse backgrounds.