not-yet-known not-yet-known not-yet-known unknown To be successful a virus must maintain high between-host transmissibility while also effectively adapting within hosts. The impact of these potentially conflicting demands on viral genetic diversity and adaptation remains largely unexplored. These modes of adaptation can induce uncorrelated selection, bring to high frequency mutations that enhance certain fitness aspects at others’ expense, and contribute to the maintenance of genetic variation. The vast wealth of SARS-CoV-2 genetic data gathered from within and across hosts offers an unparalleled opportunity to test the above hypothesis. By analyzing a large set of SARS-CoV-2 sequences (~ 2 million) collected from early 2020 to mid-2021, we found that high frequency mutations within hosts are sometimes detrimental during between-host transmission. This highlights potential inverse selection pressures within- versus between-hosts. We also identified a group of nonsynonymous changes likely maintained by pleiotropy, as their frequencies are significantly higher than neutral expectation, yet they have never experienced clonal expansion. Analyzing one such mutation, spike M1237I, reveals that spike I1237 boosts viral assembly but reduces in vitro transmission, highlighting its pleiotropic effect. Though they make up about 2% of total changes, these types of variants represent 37% of SARS-CoV-2 genetic diversity. These mutations are notably prevalent in the Omicron variant from late 2021, hinting that pleiotropy may promote positive epistasis and new successful variants. Estimates of viral population dynamics, such as population sizes and transmission bottlenecks, assume neutrality of within-host variation. Our demonstration that these changes may affect fitness calls into question the robustness of these estimates.