Gaming is a hobby. Hobbies are what people do to pass the time, keeping themselves occupied.
Anyone can consider any hobby a waste of time. That is irrelevant to you as long as you enjoy it, and is done in a manner that isn't harmful or done compulsively. The fact that it is all YOU do is an issue with you, not gaming.