Graph neural networks are a popular deep-learning architecture in applications to materials and molecules, and the most widespread implementations rely on interatomic distances as geometric descriptors. Unfortunately, GNNs based on distances are not complete, i.e. there are geometries, corresponding to molecules and/or periodic structures, that are indistinguishable by the GNN. For these, the corresponding machine-learning models will be unable to learn differences in the properties of the "degenerate" structures. This dataset contains a collection of molecular and solid structures that cannot be discriminated by distance-based graph neural networks, together with example code showing how to parse them and use to demonstrate the shortcomings of this class of machine-learning algorithms.