This is not intuitive but here goes: With the current photo displayed, left click "change or delete" at the bottom. Then left-click on the photo. Then left-click "cancel". Now the photo has gone and you can go through the normal upload for a new photo.