Welcome to the forums! How energy efficient are the windows? Storms will help keep winter weather at bay, but if they are single paned units, you may reap more of a savings over time to suck it up and replace the windows. Sills generally are separate from the window frame and are replaceable, believe it or not. You can pry the sill out from the frame, buy a replacement, cut it and nail it in place, caulk and prime/paint. As for the exterior jambs rotting, if they aren't too large you may find epoxy a way to go. If not, then you will be removing the windows anyway, why not replace them?

Thats not caulk, well, not like most people think. It's an adhesive sealant thats put in a groove in the window frame when it is assembled at the factory. The screws on the inside are covered by plastic caps which almost always get boogered up when you try to remove them. They don't screw to the door, they screw to the outside frame which causes it to sandwich the door between the halves. Best thing to do is take an ice cube (or wait for a cold winter day) to harden the sealant as much as possible, then take a single edge razor blade and run it between the plastic frame and the goo. If you are lucky it may peel off as a single strip. Likely you will have an exposed portion of the door skin underneath which will be the original door color.

I agree with Vic. It should have been removed prior to the door being painted. I've never used an ice cube but can verify that the 'caulk' removes easier in cold weather, it gets kind of gooey when it's hot. I suspect that removing the caulk will reveal some of the factory primer