By combining metal nodes and organic linkers one can make millions of different metal-organic frameworks (MOFs). At present over 90,000 MOFs have been synthesized and there are databases with over 500,000 predicted structures. This raises the question of whether a new experimental or predicted structure adds new information. For MOF-chemists the chemical design space is a combination of pore geometry, metal nodes, organic linkers, and functional groups, but at present we do not have a formalism to quantify optimal coverage of chemical design space. In this work, we show how machine learning can be used to quantify similarities of MOFs. This quantification allows us to use techniques from ecology to analyse the chemical diversity of these materials in terms of diversity metrics. In particular, we show that this diversity analysis can identify biases in the databases, and how such bias can lead to incorrect conclusions. This formalism provides us with a simple and powerful practical guideline to see whether a set of new structures will have the potential for new insights, or constitute a relatively small variation of existing structures.