You'll have an easier time than a woman trying to get her tubes tied.
https://en.wikipedia.org/wiki/Male_contraceptive There are a few other options.
But yes, every contraceptive method ever is iffy. You either take the risk and prepare to deal with them, or you take some fallback yourself to ensure that it doesn't happen. Nothing is perfect. Hence I am in favor of financial severence options for men, in the same style and time-frame as abortion, to prevent abuse of it as best as possible.
That's what sperm counts are for. You wank into a receptacle and have it checked for swimmers. No swimmers, no worries.