Are you suggesting majority of investment is done independently and not through a financial firm? Because the most likely investment source of avarage Americans is 401k... which is closer to being managed by "some weirdo bunch of rich guys rather than it just being pretty much everyone isn't broke as shit."
- - - Updated - - -
One of the largest losses was by Goldman... basically, banks that would see massive deregulation if Trump can continue his course, are seeing hits. Trump's suggested tax plan, should have and did boost the stock market, as it artificially increases corporate profits by 10%. If the stock market feels like that course might not come to fruition, prices would readjust to account for that 10% not happening.
TLDR: Wallstreet is reacting as if Trump cannot complete his goals.