Not exactly. You wouldn't have to access your entire hard drive at once, just the stuff you're using. And once you access what you need it'll be stored locally in RAM. Your local device will probably need significantly more RAM, but RAM is pretty cheap. It's still a lot of data you'll have to pull down (probably 2GB-8GB), but not as much as you're thinking. With a gigabit connection that would take ~16-64 seconds. So yeah you would definitely need a fast internet connection for it to be practical.
They could go the route of putting most of the power on the cloud end or they could go the route of putting most of the power on the user device's end. I'm honestly not sure which way would be better. It probably depends on several factors.
With increasing internet speeds more and more stuff is moving to the cloud. If internet speeds reach a point where the average person has a 100 gigabit connection then cloud computing becomes extremely practical. And with cloud computing, whether computer processing is happening in the cloud or the cloud is just storing data and transmitting it to whatever device you're connected to, it won't be necessary to connect a device to another device. All you would do is log in and then you would have access to all of your data and personalized settings.