There is one thing though that you failed to mention. Return of investment. Today's games cost a lot more than they used to to develop and produce. Sadly you have to cater for the current gaming demographic to guarantee the game will sell. If they had left the game using the same gameplay and style as the original games it would have pleased the original fan base, but that would not have been a big enough number of sales to make a decent return from the investment spent. They have to develop games that they are more certain a wide range of gamers will buy.

Think of it much like an artist or designer. They all want to paint or produce a design for themselves, using their own ideas and imagination, but those won't have much commercial value or interest due to the work being individual and personal. They instead need to produce work for others that cater for their needs and ideas, even if the artist or designer doesn't personally like it. It's how money is made.