The fact with difficult conversations is that you can delay them, but you cannot delay the negative effects of the situation that made them necessary in the first place.

If a colleague is under performing and you have to pick up their slack, silence will not improve things. If your boss is not giving you what you demand, silence will not make them change. If the team you are working in has a toxic culture, silence will not make that more digestible.

Also, more likely than not, eventually the outcome you fear and that justified the delay is going to materialize no matter what. That colleague is probably going to be fired anyway, your boss is going to get rid of you, or you are going to get rid of them, the team will have to make some drastic changes one way or the other.

So, at the end of the day it is mainly a matter of being an agent of change or a spectator. The former makes you waste a lot less time, and you have no time to waste.