Nah. I think Pandaland had it right. Tie it to cooking only. Maybe even have a pond you can stock up with the types of fish you need.
House should be secondary professions only imo other than some decorations for your personal main ones. I spent tens of thousands of garrison resources to get Morketh as my blacksmith and Dagg as my miner so professions quests would probably be a good idea as well especially if you can customize the NPCs that hang out in your house. (The single biggest issue of ESOs housing is how dead the places feel due to no ability to place NPCs. Hearthfire in Skyrim nailed that so much better. Hell even Fallout 76 is bringing that to their trainwreck of a game)