I understand the desire to quickly develop a tree towards exhibition quality. If you are in a hurry to get there, yes indeed you should remove flowers.
However, I'm in the ''other camp''. To me the flowers are the rent the tree has to pay to get me to water it over the year. If I didn't get flowers, I would get bored and forget about it. The reason don't have more than one boxwood is that trees that stay plain green all year bore me. I need to see at least some flowers every year. So I never remove all the flowers. On a weak tree I might thin the numbers down to just a few, but generally I let most flowering trees bloom as much as they want. Flowers are not a big metabolic expense in quince. - but the fruit are a bigger metabolic expense. Even so my crab apples in training I do allow then to keep a few fruit every year.