Power distribution grids host an increasing amount of distributed renewable generators, electric vehicles, and heat pumps worldwide. Distribution grids, however, were not designed with the goal of incorporating large shares of these technologies. These soaring challenges demand realistic grid models to assess the need for operation strategies and reinforcements that ensure reliable and economic management. Nevertheless, real models are often unavailable due to privacy and security concerns or a lack of digitized data from distribution system operators. To address this issue, we present a framework for large-scale inference of geo-referenced low- and medium-voltage grid models using publicly accessible information. First, we develop a clustering algorithm, which detects load areas served by distribution grids. Then, we obtain the graphical grid layout, i.e., a graph with the street and pathway geometries and the load point connections inside the load area. Next, we introduce a selection method for line types that assigns cost-effective conductors to grid lines while ensuring operational constraints. We demonstrate the framework’s effectiveness by inferring the low- and medium-voltage infrastructure in a study zone with a population of 8.7 million people.