Object manipulation for construction assembly using a crane is a control problem with highly challenging dynamics, merging contact-rich manipulation, high dynamics uncertainty from the outdoors environment, coarse actuators, and an underactuated system. The inevitable contact from assembly presents both a potential to exacerbate small errors as well as an opportunity to limit uncertainty by restricting the possible configuration space. For a controller to take advantage of this beyond simple lift-up and lay-down operations, a system that is capable of considering how uncertainty evolves under contact dynamics is required. We approach this problem by learning the dynamics of the payload’s possible occupancy distribution using a visual foresight inspired model, then performing learning-based model predictive control with this learned model as the dynamics model, with the objective of creating possible occupancy that corresponds to one payload instance in its target position. We evaluate this system in simulation on the problem of I-beam assembly, specifically, aligning and inserting a horizontal box member between flanges of two opposing vertical I-beams.