Lidar odometry allows for precise and reliable estimation of a vehicles movement over short durations, but suffers from drift over long trajectories. In this work, we propose an approach for reducing this drift by incorporating publicly available data from OpenStreetMap (OSM). Our approach is an extension to conventional Iterative Closest Points (ICP)-based lidar odometry with an additional registration against a map derived from the building outlines of OSM. For this, we propose a complementary filter inspired fusion step, capable of reducing the drift of lidar odometry while simultaneously avoiding the integration of erroneous map data. A detailed evaluation of our method on the KITTI odometry and Boreas data sets, as well as on our own recorded data, shows our approach to be applicable in a wide variety of urban and suburban situations, capable of reducing the drifting lidar odometry and yielding state-of-the-art results when compared to other OSM map-assisted visual odometry approaches.