Polymers are candidate materials for a wide range of sustainability applications such as carbon capture and energy storage. However, computational polymer discovery lacks automated analysis of reaction pathways and stability assessment through retro-synthesis. Here, we report the first extension of transformer-based language models to polymerization reactions for both forward and retrosynthesis tasks. We curated a polymerization dataset for vinyl polymers covering reactions and retrosynthesis for representative homo-polymers and co-polymers. Overall, we report a forward model accuracy of 80% and a backward model accuracy of 60%. We further analyse the model performance on a set of case studies by providing polymerization and retro-synthesis examples and evaluating the model's predictions quality from a materials science perspective.