A previous study showed that gains in visuospatial memory from first to fifth grade predicted end-of-fifth grade mathematics but not reading achievement, controlling other factors. In this follow up study, these relations were assessed from sixth to ninth grade, inclusive (n = 145). The results showed that growth in visuospatial memory across the elementary school years was related to growth in mathematics achievement after fifth grade, controlling intelligence, the central executive and phonological memory components of working memory, in-class attentive behavior, parental education, and fifth grade mathematics achievement. As found for fifth grade, this relation was not found for reading achievement after fifth grade. In total, the results suggest that visuospatial memory has a unique influence on ease of learning some types of mathematics and that this influence becomes more important across successive grades.