You might get an image of an entirely different person. The multimodal image generation models know how to preserve the result that you wanted, and just change it in one single small way ...