I've some experience with Wisteria so I thought I would try to give advice..
I am currently growing Wisteria from a seed, Chinese Wisteria. Firstly, and importantly, they are a vine.. not a normal tree that grow straight up and thicken well. They do not thicken well, compared to others but not to say they can't thicken in time.
I've mine in a few different situations, but by far the best was the one I left to grow up a trellis. That thing scaled and consumed that 6ft, 2.5ft wide trellis in a season and as a result of the tight winding, and grip it had, the trunk thickened. Compared to those without structure to climb up (but were planted in a BIG pot of organic soil to simulate ground growing), its perhaps 50% thicker. Compared to the one I left in a small pot, with no structure? Waste of a years growth, it did next to nothing. No thickening.
If you really really want rid of the grafting mark, which to be fair, it is ugly, you could ground layer it. It will result in a nice radial root spread that you can then work on for a few years and sort, so that when you ground grow it and leave it to thicken and get big, it will develop a nice nebari. Commonly done for these reasons. Mostly, I would say let this thing grow. Stick it next to something it can grab and grow up and leave it for a number of years.
The thickness of that wisteria trunk, not including the grafted base, is the about the thickness of a primary or secondary branch on my wisteria, after it was left to grow up a trellis for a year. Same other rules apply, water it well, feed it VERY well. Aside from that, it can withstand our temperatures, winds and heat. Your climate is mostly similar to mine, I'm just west of you, across the river.