An understanding of the spatial dimension of economic and social activity requires methods that can separate out the relationship between spatial units that is due to the effect of common factors from that which is purely spatial even in an abstract sense. The same applies to the empirical analysis of networks in general. We use cross-unit averages to extract common factors (viewed as a source of strong cross-sectional dependence) and compare the results with the principal components approach widely used in the literature. We then apply multiple testing procedures to the de-factored observations in order to determine significant bilateral correlations (signifying connections) between spatial units and compare this to an approach that just uses distance to determine units that are neighbours. We apply these methods to real house price changes at the level of Metropolitan Statistical Areas in the USA, and estimate a heterogeneous spatio-temporal model for the de-factored real house price changes and obtain significant evidence of spatial connections, both positive and negative.