Computer issues

[Deleted]

Hello!
I've just received my new rig:
Video: Gainward GeForce GTX 660 2GB DDR5 192-bit
Mainboard: Gigabyte GA-H81M-DS2
Processor: Intel Core i5 4440 3.1GHz box
Memory: KingMax 4GB DDR3 1600MHz
Hard disk: WD 500 GB SATA-III 64MB 7200rpm Caviar Black
Case: Floston Superior.
The issue is when I start it up the screen is black.
It has Ubuntu OS on it ( which btw gets stuck on the loading screen - where those 5 orange balls are), at which stage the red led is still flickering which means it's still loading something.
Would be ideal to get into BIOS and change BOOT order but spamming DEL key (got it from the mainboard manual) does nothing at start.
So apart from the stuck Ubuntu loading screen everything is black on screen ( no startup POST logo message , no BIOS keys, no nothing).
I even connected my old HDD with WIN XP on it and it still won't load and the whole screen will stay black.
Any further questions or solutions?
EDIT: My case has a 500 W source, you think that can be the issue here? I see on the GFX box that it requires 450 W or better, so that's just for one part. What do you think?

- - - Updated - - -

Update: Removed the GTX 660, leaving only the integrated video card from the mobo.
It works now, but I still don't get a beep after a restart and I still can't enter BIOS because that damn Ubuntu OS appears right from the start.
Should I remove Ubuntu? Format HDD? Or what ?