There are all types of button-down shirt guys, from dressy to messy, and (at least) two who fit into the casual bucket are mr downtown and mr uptown.
doesn’t mind wrinkles
doesn’t like wrinkles
centered on trousers
Of course there is crossover. Both are interested in style and like to buy premium. For the shirt, it’s in the details and how it’s worn.
Let’s face it, both are cool dudes.