Small binary asteroid systems and pairs are thought to form through fission induced by spin up via the Yarkovsky-O'Keefe-Radzievskii-Paddack (YORP) effect. This process is expected to depend on their structural strength, hence composition. We aim to determine how taxonomic classes, used as a proxy for composition, distribute amongst binary asteroids and asteroid pairs compared to the general population. We compare the distribution of taxonomic classes of binary systems and pairs with that of a reference sample of asteroids. We build this sample by selecting asteroids to reproduce the orbital and size distribution of the binaries and pairs to minimize potential biases between samples. A strong deficit of primitive compositions (C, B, P, D types) among binary asteroids and asteroid pairs is identified, as well as a strong excess of asteroids with mafic-silicate rich surface compositions (S, Q, V, A types). Conclusions. Amongst low mass, rapidly rotating asteroids, those with mafic-silicate rich compositions are more likely to form multiple asteroid systems than their primitive counterparts.