The purpose of a state estimation (SE) algorithm is to estimate the values of the state variables considering the available set of measurements. The centralised SE becomes impractical for large-scale systems, particularly if the measurements are spatially distributed across wide geographical areas. Dividing the large-scale systems into clusters (\ie subsystems) and distributing the computation across clusters, solves the constraints of centralised based approach. In such scenarios, using distributed SE methods brings numerous advantages over the centralised ones. In this paper, we propose a novel distributed approach to solve the linear SE model by combining local solutions obtained by applying weighted least-squares (WLS) of the given subsystems with the Gaussian belief propagation (GBP) algorithm. The proposed algorithm is based on the factor graph operating without a central coordinator, where subsystems exchange only “beliefs”, thus preserving privacy of the measurement data and state variables. Further, we propose an approach to speed-up evaluation of the local solution upon arrival of a new information to the subsystem. Finally, the proposed algorithm provides results that reach accuracy of the centralised WLS solution in a few iterations, and outperforms vanilla GBP algorithm with respect to its convergence properties.