Early season I prefer the mornings because it's cooler and the deer are moving. Once the temps reach 50-60's in the afternoon, deer movement stops.
During the rut, sit all day if you can.
Late season I like dusk. The deer have been chased for 2 months now. They have learned to hunker down all day unless pushed. They get up at dusk to move to feed. Hopefully I can catch one before dark.