Abstract The three-dimensional (3D) organization of chromosomes is crucial for packaging a large mammalian genome into a confined nucleus and ensuring proper nuclear functions in somatic cells. However, the packaging of the much more condensed sperm genome is challenging to study with traditional imaging or sequencing approaches. In this study, we develop an enhanced chromosome conformation capture assay, and resolve the 3D whole-genome structures of single mammalian sperm. The reconstructed genome structures accurately delineate the species-specific nuclear morphologies for both human and mouse sperm. We discover that sperm genomes are divided into chromosomal territories and A/B compartments, similarly to somatic cells. However, neither human nor mouse sperm chromosomes contain topologically associating domains or chromatin loops. These results suggest that the fine-scale chromosomal organization of mammalian sperm fundamentally differs from that of somatic cells. The discoveries and methods established in this work will be valuable for future studies of sperm related infertility.