Then what can you know about how people who are not rich feel? Your life is not representative.
As someone who has been on pretty much all parts of the spectrum, let me tell you something: people adapt. People can be happy with many different lifestyles. What you, from your perspective of being rich, see as misery, the person themselves might see as quite a happy life. I know my income is low by American standards, but I'm genuinely happy with my life, I love my career and I go to the office every day with a smile. If you think I am some kind of an outlier, then read all kinds of surveys on that. And if you think that I am just faking being happy... Well, you are free to do so.
It is one of the factors, yes, but not the only one. Qatari are richer than Swiss, and yet they don't seem to be particularly happy!