Because it's not just for looks. Fines can and will clog pots.
I have a couple of little ones that have developed a solid layer at the bottom of their pot. The mix used in those was thrown together quickly - unsifted Turface and some potting soil that was mostly pine bark. A crust has formed on the bottom and the drainage is awful. I'm repotting them this weekend (unless it get colder than expected), and I'll be sifting everything & cutting way back on the organics.
edit: Obviously there are plenty of places on-line where the sift/don't sift debate has been taken up by various folks. I just wanted to add a personal example that helped convince me.