Combining direct imaging astrometry and long-baseline radial velocity (RV) measurements of stellar binaries can provide precise constraints on their 3D orbits and yield dynamical masses for both components. We applied the combination of these methods to study HD104304, a binary system with a decades-long orbit containing a G8IV subgiant and a recently-discovered M dwarf companion. Using radial velocities collected over a timespan of two decades by Keck/HIRES and astrometry calculated from adaptive optics images taken by Keck/NIRC2, we explored models to jointly fit the astrometric orbital motion and RV trend. Previous studies of this system (Howard & Fulton, 2016) were unable to distinguish between two and three body solutions using RVs alone. However, we are able to break this degeneracy by incorporating images into the fit. We make 13-sigma dynamical mass measurements of the primary and secondary, and find that a slightly eccentric solution (e=0.3) is required. However, the dynamical mass we measure for the primary (~1.8 solar masses) is significantly higher than its well constrained spectroscopic mass of 1.02 solar masses. This hints at the need for a three-body solution to accurately model the observed trend in the HD 104304 system.