I think it depends. There are a lot of adult orientated video games and anime on the market.
In my personal opinion if you are a grown adult and still like playing Pokemon and watching Sailor Moon and My Little Pony, yea there may be some crossed-wires occurring.
Lolicon is just for degenerates period.