Right whales (Eubalaena glacialis) were extirpated from the eastern North Atlantic by commercial whaling. Gray whales (Eschrichtius robustus) disappeared from the entire North Atlantic in still-mysterious circumstances. Here we test the hypotheses that both of these species previously occurred in the Mediterranean Sea, an area not currently considered part of their historical range. We used ancient DNA barcoding and collagen fingerprinting methods to taxonomically identify a rare set of 10 presumed whale bones from Roman and pre-Roman archaeological sites in the Strait of Gibraltar region, plus an additional bone from the Asturian coast. We identified three right whales, and three gray whales, demonstrating that the ranges of both of these species historically encompassed the Gibraltar region, and likely including the Mediterranean Sea as calving grounds. Our results significantly extend the known range of the Atlantic gray whale, and suggest that 2,000 years ago right and gray whales were common when compared to other whale species. The disappearance of right and gray whales from the Mediterranean region is likely to have been accompanied by broader ecosystem impacts, including the disappearance of their predators (killer whales) and a reduction in marine primary productivity. The evidence that these two coastal and highly accessible species were present along the shores of the Roman Empire raises the hypotheses that they may have formed the basis of a forgotten whaling industry.