The things you say no to – and the things you say yes to as well of course – they signal what you care about. In the day to day, for most things, it might seem not too important (it is). But when you are in a leadership position, there is no excuse. You might have very good reasons to dedicate your attention or energy to this or that, yet eventually you are telling those around you what matters and what you all together are about.
Tread carefully with these kind of decisions, they do not affect yourself only.