If they care about you they will understand. Ultimately you are the only one that can figure out what is best for you and yours. Now if you are talking about leaving a significant other behind, that might be the end of the relationship. If it is friends and family it shouldn't be. Unfortunately this is a part of adult life people often ignore, as you grow older people scatter to do what is best for them and theirs.