Aaaaah I just tested a lot
And I found out that if you click the overwrite text it works, but if you click the question mark or between 'overwrite' and question mark it doesn't work.
And as you mis-click it just closes the save-screen again, just like when saving worked.
However you don't get to see all keybinds for 0.2 second like you see when overwriting has succeeded.
So I guess this might have been the problem. You expect all the text to be a button, but the buttin is smaller then you'd expect and you don't get very clear visual feedback on wether you clicked right or not.
I was convinced that I clicked correctly, but this could also be the explanation...