In my opinion, yes, you should look to increase your memory, unless you are looking to replace/upgrade the entire computer in the near future.
Just looking at your screenshot, you're trying to play a game that wants about 3 gigs of memory. Your operating system typically wants about 3-4 gigs of memory, and you have a system with 8. You just barely fit, assuming you're running nothing else at all.
You can do this. There is nothing inherently wrong with being right up against your memory limit, so you don't have to increase it until it starts to cause problems. A warning that you're getting low does not necessarily mean you're at a point of needing to upgrade -- but it does mean you are close.
"Swap" is hard drive space that doubles as memory. Your oS has logic to send stuff in memory that hasn't been used for a while out to disk to make room for something else. However, increasing your swap isn't a great solution. You're very close to actively using all of your available memory when you're playing a game. If you're actively using more memory than you have, your system starts
thrashing, which is a term that means it is constantly paging things into and out of memory to try to fit it. Problem is, accessing your hard drive is anywhere between about 3 and 100,000 times slower than accessing memory (yeah, no shit). When you're thrashing, the computer has no time to do anything else. In other words: It doesn't help.
If you want to increase your swap and see if it works for you, it's not hard and there's no nasty side effects.
This article should have instructions for your OS. Don't expect too much.
If Heroes is all you play and this is a typical level of memory usage for you, you're okay for now. You likely won't be okay much further into the future though, so if you plan on keeping the PC a while I would upgrade the memory.