In practice practice that what happens but the VP isn't not obligated to hand off the gavel and can take it back whenever they want.
The VP sets the agenda, is the decider of points of order, and interprets Senate rules.
The Senate can override the presiding officer through vote, which is where the majority leaders implied power comes from as the acting leader of their party within the Senate. So the presiding officer could set an agenda but the majority leader who gets to sit on committees can kill stuff there OR depending on how strong party lines are can overrule the presiding officer by vote.
Example someone makes a point of order. - the majority leader for instance . The VP can deny that point of order. The majority leader can issue a rebuttal by having their party mates back the majority leader with a simple majority, thus overriding the presiding officer. So you can see how the VP might get tired playing parliamentary procedures and spend them time and effort with the executive branch.