Several medications and treatments are being investigated for their potential effectiveness against coronavirus disease 2019 (COVID-19), including androgen and other sex hormones. However, the causal relationships between serum sex hormones and COVID-19 susceptibility and severity, particularly with regards to potentially sex-specific effects, remain largely unknown. In this study, we used the latest data from the UK Biobank (up to 424,907 individuals) and COVID-19 Host Genetics Initiative (up to 1,878,143 individuals) to systematically assess the sex-specific causal effects of serum sex hormone levels on COVID-19 outcomes within a two-sample Mendelian randomization (MR) framework. The inverse-variance weighted method was used in the main MR analysis. We additionally performed a series of sensitivity analysis to assess the robustness of MR effect estimates to potentially invalid genetic variants. Our MR analysis revealed novel causal associations between serum estradiol and bioavailable testosterone levels and SARS-CoV-2 infection in women, but not men, except for a suggestive inverse causal association between estradiol levels and COVID-19 severity in men. These novel findings improve our understanding of the sex-specific causal nature of sex hormones in relation to COVID-19 outcomes, and suggest that sex hormones may serve as potential therapeutic targets for preventing SARS-CoV-2 infection and improving patient outcomes.