I can't say that I have a least favorite. My favorites are tropicals, but I can't think of anything that I don't like.
Honestly IMO if there is/was something I didn't like it would be because I myself simply did not prepare enough to take care of it rather than the tree being something I don't like or something that was too hard to take care of.
The Fukien Tea gets tons of hate here it seems, and that's fine, to each his own. But my FT's do great (and I'm in Michigan) they just take a bit of a game plan to overwinter them inside is all. If you don't plan ahead for them though then I'm sure it would be easy to not like them.