Atrial fibrillation (AF) is a prevalent and morbid abnormality of the heart rhythm with a strong genetic component. Here, we meta-analyzed genome and exome sequencing data from 36 studies that included 52,416 AF cases and 277,762 controls. In burden tests of rare coding variation, we identified novel associations between AF and the genes MYBPC3, LMNA, PKP2, FAM189A2 and KDM5B. We further identified associations between AF and rare structural variants owing to deletions in CTNNA3 and duplications of GATA4. We broadly replicated our findings in independent samples from MyCode, deCODE and UK Biobank. Finally, we found that CRISPR knockout of KDM5B in stem-cell-derived atrial cardiomyocytes led to a shortening of the action potential duration and widespread transcriptomic dysregulation of genes relevant to atrial homeostasis and conduction. Our results highlight the contribution of rare coding and structural variants to AF, including genetic links between AF and cardiomyopathies, and expand our understanding of the rare variant architecture for this common arrhythmia.