Research has shown that the quality of mother-infant interactions, such as the way the mother and the infant are in synchrony during interactions, is associated with infants’ vagal tone, a physiological indicator of emotion regulation. However, little is known about the association between the infant’s vagal tone and the quality of father-infant interactions. The existing literature suggests that the greater quantity of time a father spends with the infant, either alone (dyadic time) or with others (social time) may strengthen the association between the quality of father-infant interactions and infant emotion regulation. In this study, we aimed to investigate the association between father-infant synchrony and infants’ vagal tone during interactions and whether dyadic time and social time have a moderating role. Fathers (N = 77) and 3-month-old infants were observed interacting and the infants’ electrocardiogram was recorded to compute indices of vagal tone. Fathers reported dyadic and social time in questionnaires. Results showed that father-infant synchrony has no direct associations with infants’ vagal tone. Moderation analyses revealed that more social time increases the association between synchrony and vagal tone. This study suggests that the interactive processes during shared family time may influence dyadic father-infant interactions and the infant’s vagal regulation.