In Irrgang et al. (2020), we have trained a convolutional neural network to perform a so-called downscaling task. This downscaling aims to recover the fine-structure continental water storage distribution on the South American continent from coarse-resolution space-borne gravimetry observations. Here, we share data sets that were used for training the neural network, namely (1) monthly pairs of gridded terrestrial water storage anomalies (TWSA) of the South American continent and (2) surface water storage anomalies (SWSA) in the Amazonas region for the time period 2003-2019. TWSAs were used as target (output) values of the neural network and were derived from the Land Surface Discharge Model (LSDM, Dill, 2008). The corresponding input values were calculated by spatially smoothing the TWSA fields with a 600 km Gaussian filter. After training the neural network over the time period of 2003 to 2018, its performance was tested and compared to LSDM for the subsequent year 2019.