The evolution of linkage disequilibrium between genes underlying mate choice and ecological traits is thought to be a fundamental step in the course of speciation with gene flow. Here, we capture this process in the hamlets, a group of closely related reef fishes from the wider Caribbean that differ essentially in colour pattern and are reproductively isolated through strong visually-based assortative mating.Using full-genome analysis, we identify four narrow genomic intervals that are consistently differentiated among sympatric species in a backdrop of extremely low genomic divergence. These four intervals include genes involved in pigmentation (sox10), axial patterning (hoxc13a), photoreceptor development (casz1) and visual sensitivity (SWS and LWS opsins), respectively, that develop islands of long-distance and inter-chromosomal linkage disequilibrium as species diverge.The relatively simple genomic architecture of species differences allows linkage to be maintained in the presence of gene flow by a combination of assortative mating and natural selection.