How long have you been overwintering the tree inside? This the second year, third?
It has a long and varied history. When I lived in California, I had it inside for probably 3 winters (it was inside all year then) and then I moved it outside, where it was outside for probably 3 winters (in zone 10).
When I moved to Virginia (zone 7) last summer, I let it go dormant and brought it inside when temperatures were going below 15F and kept it inside until spring.
In all those changes, I've never seen it get unhappy.