Yes that's some kind of fungus. It's normally a sign of healthy soil biology. I'd just leave them, unless they grew to the point that they were clogging drainage etc, or were a known harmful species.
This could very well be the case! I'm not familiar with UK species of fungi. However, I did notice that the soil in the pot looks high in organics.
There are a bunch of 'cultivars' of elf cups with orange-ish colors, so I figured that it's possible that they're elf cups. I'm no mycologist though, so I can agree with any other ID that has a better fit.