Im shamelessly just going to copy some stuff since i hardly know how software is written. But what i do gather from everything is the fact this game optimization is horrible. And it now turns out to be something that will not be easily fixable. If this is true this game if it survives the first year is going to be having huge problems in the future since the rigs those people use will be our laptops tomorrow. And if they dont fix that before we get (reasonably priced) the same tech, they are going to have lots of problems.
This is not from me!
And yes i used CODE tags otherwise this would be incredibly long sorry for this.
As a software engineer I am going to throw out my speculation based on some poking around using tools like Process Explorer.
The game is not CPU or GPU limited when in Warzones or other player heavy situations. Usually when you see this kind of behavior it means things are I/O bound. Both the CPU and GPU are waiting for either data from the hard drive or the network and doing nothing for short periods of time.
Games used to be single threaded and the same loop that does the rendering also processed network packets and read data from disk. If something was loaded from disk no frames where being rendering during that time, so most games had a loading screen and loaded everything into ram and made sure to never touch the disk again until the next load screen.
SWTOR is not single threaded, this is easy to know because it runs two processes, this means a minimum of two threads, but its actually much more, last time I looked both processes had 10+ threads each.
SWTOR is definitively loading assets from disk constantly probably because a whole planet cannot fit into the 2 gigabytes of address space a 32bit process has access to. A lot of software does this sort of thing, it typically called streaming, and there are a lot of approaches.
SWTOR is actually unusual in using two processes, I myself have never seen a game do this. In poking around its easy to see the main swtor.exe with the larger memory footprint is doing the network communication and playing the sounds while the secondary smaller swtor.exe is doing the direct3d calls and utilizing the GPU.
These two processes must communicate somehow and this is what makes things unusual for something like a game, because inter-process communication adds much overhead, even when using the fastest form called "shared memory".
It's seems no matter how fast your CPU or GPU is any time the disk is read in SWTOR the framerate will plummet, this is the random "hitching" you see when driving a speeder around, this means the disk access is blocking the rendering engine in some way. Other players exasperate the problem because of their varying outfits and models which must not be able to completely fit in ram and must be loaded/unloaded on demand, vs NPC which in a questing area are all wearing similar outfits etc.
You can tell SWTOR blocks on disk while a game like WoW doesn't because have you noticed in SWTOR you never see another player partially loaded? In WoW while the players assets are being loaded they may show up as just a "shadow" on the ground, you can see them moving around but the model hasn't been loaded yet. In SWTOR players "fade in" fully loaded which mean their models must be read from disk(or already be in ram if lucky) before the engine will continue. This may be intentional or a limitation in the engine design. I prefer WoW's approach of never blocking rendering even if on the rare occasion you might end up fighting nothing but a shadow while the model is loading(happens much less in wow because of simpler models allow more to stay in ram).
So the game is blocking on I/O, either network, IPC, or most likely disk leading to horrible FPS even on high end systems. SSD's will help the situation some, but even an SSD is still thousands of times slower than ram. And for those with 8-16gigs of ram, SWTOR is only 32bit with two processes, so it basically has a total of 4 gigs of usable address space and it seems that only one processes is really using it's 2 gigs, while the second renderer is only using about 300-400megs. There has been reports of setting up RAM disks for SWTOR's assets helping if you have lots of ram, which would line up with what I am seeing.
If this is the case what are the solutions?
1. Stop blocking on disk reads, this made leads to things like just seeing a blob shadow run by for a few seconds, but you can still control game and take action. This may or may not be easy to do given the engine design.
2. Merge the two processes. Again I have never seen a game do this, IPC adds overhead and latency that would not exist in a single process design.
3. Compile for 64bit. This may be difficult to do depending on the engine design, but a single 64bit process could use all the available system ram for caching assets greatly reducing disk I/O.
That is my educated guess based on what I know and what I have seen in game, it may be wrong, it would be nice to get a real response from a dev to clarify.
Wakantanka is right. I also am a software engineer and I noticed a few things that indicate corners were cut, such as the 2 processes.
Each x86 Application is limited to 2gb of ram and by having 2 it means they could not for what ever reason get memory utilization below 2 and cheated by having a second process. If they were willing to cut corners on this I can only imagine what other design corners they have cut and agree 100% this is a design flaw. I love this game but imagine this will kill it as they cant reproduce years of design fast enough to keep subscribers especially if they purchased the engine instead of writing it themselves. These issues are most likely an IO issue caused by reads from disk or network.
I worked on a project that encountered similar behavior after years of poor design flaws and cut corners that ultimately resulted in starting over from scratch as it was control software for industrial automation and crashes / poor performance issues like this that take down multi million dollar factory equipment is unacceptable to most customers as I assume it will be for a gaming community who spend thousands to just play a game.
Bottom line this is a software issue and it was not ready to be released. They should have put more effort into design and pushed the release back to focus on quality. I am usnure if it was a cash flow problem as most software projects go over budget and get released with some bugs. The fix for this could take months or even years depending on the severity of the engine problems (Full engine rewrite) or not come at all because the cost could be too high to be profitable and they might just scrap this project.
Sorry to seem pessimistic but I have seen less severe problems like this kill projects and while I remain hopeful they will fix this game as I love it and would love it more if it worked as intended but I remain skeptical it will be fixed fast enough to keep enough subscribers to keep this game going.