Since we never got confirmation of the engine improvements from 1->2 (I know we never did in the first year or two) I have a sneaking suspicion they didn't fully happen and the reason they've taken down some of this content is simply a matter of practicality with the engine. Don't ask me about specifics, I'm nowhere near technically literate enough, but it seems to fit from what we know of issues past.
At first...because it was a rush job under Smith that was largely done in the final 9ish months.
Then because they were working to try to unfuck the game.
Then they started hitting their stride.
Then they broke off from Activision and lost support studios.
Then they decided to take a game that was never designed for longterm support and support it in the longterm.
All while frequently ignoring lessons learned in MMO's over the decades, and often even lessons they learned in Destiny 1. I don't have a particularly high opinion of Bungie as a developer outside of, as you say, the basic nuts and bolts. Gunplay, visuals, movement etc. are all solid as hell. It's the....everything else where they seem to consistently struggle.
Bonus points: If you missed the story back then you've gotta hunt it down in videos/elsewhere from what I know. I think they may have added some of it to the in-game codex, but damn if it's not poor form to take new players and drop them into a universe that's building off of years of story development they literally can't even access because it's all been removed.
I'm still genuinely sad that they're doing what I'd initially hoped and supporting the game for a long time. I had no clue they were so ill equipped for it, and desperately wish they'd start fresh with Destiny 3 with engine improvements, actual lessons learned, and plans to support the game for years to come out of the gate so they can start designing around that.