Including duration when calculating difficulty of an encounter is "out of left field"? (I'm assuming that's the phrase you were referring to) I consider it a major factor, and i dont consider "it took ages because there was lots of trash" to be an important factor when calculating the difficulty of a raid, as the concentration and intensity of those packs are nothing when compared to the boss fights themselves.
Honestly, if you cant accept that a 30 second zergfest with 2 mechanics and each dps literally standing still and pressing 1 button 80% of the time, and substituting in a second a few times over the 30 seconds is nothing in comparison to modern raiding, then there really is nothing further to discuss. Your argument can be reduced down to "there is lots of trash and it can sometimes take ages getting from boss to boss".
When completing an intense activity that requires very high levels of concentration and communication, duration is a major factor. When comparing mundane and simple tasks, the duration needs to be extreme before it becomes a factor, such as some of FF infamous fights that took (correct me if im wrong) 15-18 HOURS to down. The difficulty there comes from the endurance, and as such, the encounter difficulty (excluding the duration) is extremely low.
Vanilla / Classic is the triple whammy of easy - EXTREMELY short fights coming in at 5-10% of a retail encounter, and extremely simplistic encounter design, AND extremely simplistic class mechanics.