Glad to see your azalea is doing better.
Organic material is one thing. Too much organic material on the surface will eventually clog the interstitial spaces between media particles and prevent percolation. Doing soji will restore the percolation of the media. However if one has to go down more than a 1/2” on an azalea one ends up damaging the important surface roots. In this case one is better off repotting.
Moss is entirely different. Moss protects the media particles from washing away during watering. Moss slows evaporation, provides insulation, protecting the roots from cold and heat. Moss filters the fine organic material, rejects the larger particles. Finally moss wicks water slowly out of the media. So clearing moss away for the reasons given is actually working against best practice. (…Believe this is is mostly data from Rick Garcia’s book).
Cheers
DSD sends