Suppose the game UI is drawn at 1 pixel = 1pixel. The auction house is 1000 pixels wide. The 'action bar' on the bottom of the screen is even wider if you include the side art. That means people with very small screens (like a modern macbook air or a simliar class of hardware, or say a typical desktop computer from 2005) end up with an interface that doesn't quite fit on the screen.
The slider lets you can scale elements anywhere from ~2 UI pixels = 1 game pixel so that people with really tiny screens can have a UI that doesn't dominate their field of view. On the other end of the spectrum you can map 1 UI pixel = 2 game pixels so people with very high resolution screens (like a retina display macbook pro running windows, or some of the older very high resolution PC laptops that would be 1900x1200 on a 15" screen) can have a UI with buttons that are large enough to click on the first try. (those numbers are made up but they illustrate the purpose well enough). This is a somewhat crude version of the scaling that Mac OS X does on modern high pixel density displays.
When you look at the default UI style and the sorts of things the blizzard UI provides by default (ie: the wowapi button/scrollbar/etc classes) this sort of thing works really well - they designed the UI and the supporting artwork to scale reasonable well. You'll notice there are no 1 pixel lines that need to be precisely aligned to pixels. The artwork is "blurry" but never by more than a pixel which is typically just fine -- nobody complaints about cleartype text and this is basically the same thing because the UI elements just aren't that detailed.
When addon authors and UI compilers started making things that use very sharp/precise graphics that changed things. If "here is a 2 pixel line, draw it starting from pixel 5 to pixel 10" that's easy enough. When somebody has adjusted their UI so that 5-10 is now 7.5 - 15.5 (and the height of that same line is 2.3 pixels) you end up with wierdness. Do you draw the line 2 pixels high? 3? 2, blue a faded/blurry bit? Fractional pixels "don't make sense" so you have to find some way to accommodate it. Sometimes you want to be blury so you don't introduce a sharp line where the UI designer didn't intend one. Sometimes you do want a sharp line to avoid bluring where the UI designer didn't intend it. There's no way for blizzard to guess what you intended but their art and api seem to indicate you should always avoid sharp lines and make stuff that handles a bit of edge blurring. When addon authors and UI compilers decided to ignore this recomendation they gained an increase in flexibility (you can do new kinds of UI elements / styles that didn't work before) but it came at the cost of being inflexible (the entire effect depends on being carefully aligned to pixel grids).
I find it rather hard to criticize blizzard's decision here - it was pretty practical, works most of the time, and even when it breaks you can usually work your way out of the corner with a bit of effort. There are better systems possible but they'd be much more difficult to impliment and would have created a pile of extra work for addon creators too. I'm sure they'd make different choices today - but technology was very different 10 years ago and even choices made back then aren't completely terrible in light of contemporary hardware.