You need to disable extension hiding in Windows.
All you have to do is click Tools -> Folder Options on the menu bar of an open folder. Then click the View tab on the window that pops up. You should see a list like this:
Locate "Hide extensions for known file types" and make sure it is NOT checked. Then click OK and you're done!