320kbps re-encoding question?

Your logic is incorrect. It's like taking 128ml of water from a 128ml container and then pooring it into a 320ml container. The 320ml will still be larger than the 128ml container, despite the fact that you won't get any additional liquid.