I agree with Eric: the soil is not drying out enough between waterings. You need to let the surface of the soil dry completely, then water slowly and thoroughly. Let the surface dry again, water slowly and thoroughly, and so on.
If the tree is still struggling or if what you call mildew is still there after a week of that treatment, let the soil dry out a little more between waterings. (It may be mildew, tho it looks like a fungus to me. Either way, it means the soil is staying too wet.)
Also, get the tree into good, bright light and make sure there is some air circulation around it.
Knowing what it is, as Eric asked, and knowing where you are (in general terms) will let us give you better advice.