It's because they build alot of siege engines first, but ya - it's really annoying, I've only had one defense battle in black crag. (rebellion)
On the other hand, they are so easy to defend now, it's not that much fun anyway other than watching it because it looks cool.