Typically you don't want to leave foliage only on the ends of the branching. It gives the tree a topiary kind of pom pom look. Healthy ficus can be cut back and back bud very well if it doesn't fill back out on its own. Looks like you've been keeping it healthy so don't worry about it. That lowest branch should help the base grow a bit, so I don't think I would mess with it.