I always take the band off before hand, only because I put it in a bag with the rest of them to make something cool at some point.
Was smoking with a pal the other day who asked me if I knew why the band was originally placed on a cigar? I didn't, and he informed me that it was to keep tobacco from staining my white gloves, or in my case ungloved hands.
No idea if this is true, but by his account it may have been developed as a way for snooty folks to keep their white gloves clean. I plan to keep the band on the next time I happen to be smoking in white gloves.