That is fungus caused by poor conditions. Some people here go for the poison cabinet, that can reduce the symptoms for now but that doesn't help in the long run.
Make sure the leaves do not stay wet. Don't water the leaves and place the tree where the wind can go through the crown (so not near a wall) and it gets plenty of sun. Under those conditions the fungi hardly stand a chance.
Why did you remove the leaves? >90% of the leave is healthy and doing photosynthesis. These fungi are everywhere, thinking you can avoid your tree getting exposed by removing infected leaves is an illusion.