A gallery, a private guide, a concert, theatre, sport or special access can shape the whole visit. Capacity, dates and the movement around them make early thought useful, particularly for a short stay.
This is not an argument for filling every hour. It is about protecting the one or two things that make the city worth being in, then allowing the rest of the day to work naturally around them.