The evolutionary direction of gonochorism and hermaphroditism is an intriguing mystery to be solved. The special transient hermaphroditic stage makes the little yellow croaker (Larimichthys polyactis, L. polyactis) an appealing model for studying the formation of hermaphrodites. On the other hand, as the most famous commercial fish species in East Asia, the origin and evolutionary relationship of L. polyactis and Larimichthys crocea remain unclear. Here, we report the genome sequence of L. polyactis, with a size of ~706 Mb (contig N50 = 1.21 Mb and scaffold N50 = 4.52 Mb) and 25,233 protein-coding genes. Phylogenomic analysis suggests that L. polyactis diverged from the common ancestor of Larimichthys crocea ~25.4 million years ago. Our high-quality genome assembly enabled comparative genomic analysis, which revealed a number of within-chromosome rearrangements and translocations without major chromosome fission or fusion events between the two species. The dmrt1 gene was identified as the candidate sex determination gene in L. polyactis. The expression of dmrt1 and its upstream regulatory gene rnf183 were both sexually dimorphic in the transcriptome analysis. Rnf183, unlike its two paralogues rnf223 and rnf225, is only present in Larimichthys but not in other teleost species, suggesting that it originated from a lineage-specific duplication or was lost in other teleosts. Phylogenetic analysis shows that the hermaphrodite stage in male L. polyactis may be explained by the sequence evolution of dmrt1. Decoding the L. polyactis genome not only provides insight into the genetic underpinnings of hermaphrodite evolution but also provides valuable information for enhancing fish aquaculture.