In Western societies, many polarized debates extend beyond the area of opinions, having consequences for social structures within society. Such segmentation of society into opinion-based groups may hinder communication, making it difficult to reconcile viewpoints across group boundaries. In three representative samples from Australia and the Netherlands (N = 1,206), we examine whether perceived polarization predicts the quality (harmony, comfort, and experience of negative emotions) and quantity (avoidance of the issue) of communication with others in the community. We distinguish between perceived opinion differentiation (i.e., the extent to which opinions in society are divided) and perceived structural differentiation (i.e., the extent to which society fissions into subgroups). Results show that although opinion differentiation positively predicts the discussion of societal issues, the belief that these opinions reflect a deeper societal divide predicts negative communication expectations and intentions. We discuss how polarization perceptions may reinforce communicative behaviors that catalyze actual polarization processes.