According to WHO, COVID-19 is an infectious disease and has a significant social and economic impact. The main challenge in ?fighting against this disease is its scale. Due to the imminent outbreak, the medical facilities are over exhausted and unable to accommodate the piling cases. A quick diagnosis system is required to address these challenges. To this end, a stochastic deep learning model is proposed. The main idea is to constrain the deep representations over a gaussian prior to reinforce the discriminability in feature space. The model can work on chest X-ray or CT-scan images. It provides a fast diagnosis of COVID-19 and can scale seamlessly. This work presents a comprehensive evaluation of previously proposed approaches for X-ray based disease diagnosis. Our approach works by learning a latent space over X-ray image distribution from the ensemble of state-of-the-art convolutional-nets, and then linearly regressing the predictions from an ensemble of classifi?ers which take the latent vector as input. We experimented with publicly available datasets having three classes { COVID-19, normal, Pneumonia. Moreover, for robust evaluation, experiments were performed on a large chest X-ray dataset with fi?ve different very similar diseases. Extensive empirical evaluation shows how the proposed approach advances the state-of-the-art.