Eh, I'd never go on easy for a game. Half the time, typical normal mode feels like it's on easy. My first playthrough though, I just stay on normal, it's easier to see the finer details in a game and get a feel of the actual gameplay. Then I'd replay it on the highest difficulty for trophies and such.
I suppose it's why I always like games that don't have a changeable difficulty, but meh.
Some games are kind of impossible to play with higher difficulties though. Either the combat itself makes it too pointless or it can't be balanced enough. Good example would be ES: Oblivion, where it gives enemies insane amounts of health that makes you spend literally half an hour fighting the guy, while no matter your current level, it still takes forever.