This paper presents a novel Graph-Convolutional-Network Based Deep-Learning (GCDL) approach to effectively determine estrogen-receptor-status (ERS) in patients with invasive breast cancer, from their Hematoxylin-and-Eosin (H&E) stained Tissue Microarray (TMA) images. Exploring previously overlooked correlation between ERS and nuclei spatial properties from H&E images, we use a set of novel methods to 1) construct twolevel cell-graphs from breast cancer H&E-stained tissue images and a novel graph node pooling, 2) extract two-level graph-metrics, 3) find the feature distribution statistics on low-level graph, 4) combine the features from 2) and 3) with conventional nuclei morphometric features to form composite node embeddings on high-level subgraph, and 5) associate a single-layer graph convolution with a stack of several nonlinear dense layers to create a GCDL classifier for enhanced performance. The proposed GCDL approach is tested and compared with a popular residual based Convolutional-Neural-Network (ResNet) method on a dataset of 960 patients with their ERSs diagnosed by pathologists. The test results show that the GCDL approach outperforms ResNet, achieving the best accuracy by far in ERS classification using a single H&E-stained TMA image per patient. Moreover, the GCDL approach requires much less training data and training time to achieve competitive results.