The LUA file for that portion is ...ElvUI\modules\misc\raidUtility.lua
Lines 10-17 check to see if you are in a position that would warrant having the utility showing.
If you change the 'true' on line 13 of the file to 'false' it should never display the utility. Anytime you update Elv you will need to go back and change it back again. No guarantees, but that's the section that makes the most sense to me.
--Check if We are Raid Leader or Raid Officer
local function CheckRaidStatus()
local inInstance, instanceType = IsInInstance()
if ((IsInGroup() and not IsInRaid()) or UnitIsGroupLeader('player') or UnitIsGroupAssistant("player")) and not (inInstance and (instanceType == "pvp" or instanceType == "arena")) then