Your issue is mostly likely with the manure and possibly the granite dust. Manure, even composted manure, breaks down into extremely fine particles. Those particles can clog any drainage you've got. It can also be EXTREMELY difficult to re-wet after drying out once. It will actually shed water without rehydrating--unless you soak the pot underwater for an hour--which of course, will only worsen the drainage issue, as the manure holds onto water.
Granite dust, especially flakes with flattish sides, compacts. Granite dust is used as a compaction layer underneath paver stones. It serves to drain water from underneath pathways and such. IT IS MILLED TO compact easily--not what you want in a bonsai soil. If you use granite or other crushed stone, the granules must be irregular in shape (not flakes, which suggests flattened sides) so they cannot fit snugly against surrounding particles.
Since boxwood like very well drained soil, but are also prone to die back if their roots dry out, I'd re-think the manure entirely. It's got too many things that make it difficult to manage effectively in a soil mix. I'd also switch to crushed granite or flint used for poultry consumption. That's what we use here in the States.
For the last ten years or more, I've been keeping boxwood in soils that have only 10-20 percent SIFTED GRANULAR organic components (like composted pine bark mulch, or baby orchid mix--fine fir bark), and 80 percent soil-less ingredients like crushed granite, haydite or even turface.