Chromatin has been shown to undergo diffusional motion, and during active gene transcription the RNA polymerase activity negatively affects chromatin motion. However, the relationship between chromatin motion and other genomic processes remains unclear. Hence, we set out to label the DNA directly in a sequence unbiased manner and recorded labeled chromatin dynamics in interphase human cells expressing GFP-tagged PCNA, a cell cycle marker and core component of the replication machinery. We detected decreased chromatin mobility during S-phase compared to G1 and G2 phases using automated particle tracking. To gain insight into the organization and dynamics of the genome during DNA replication, we analyzed labeled chromatin domain size and motion in replicating cells. By correlating chromatin mobility with proximity to sites of active DNA synthesis, we show that chromatin motion is locally constrained at the sites of DNA replication. Furthermore, inhibiting DNA synthesis activity leads to increase loading of DNA polymerases and further restricts local chromatin motion. The polymerases under stress no longer reel DNA through, as they normally do during DNA synthesis, which may explain the further restriction of chromatin motion and the fact that loading the helicase/polymerase already restricts it during active DNA synthesis. We, therefore, propose that increased polymerase loading but not their catalytic activity reduces genome dynamics and its accessibility and interaction with other genomic regions.