I realize I won't change your mind here, but one can justify any sort of environmental degradation or invasive species impact using the above logic.
The reasoning that 'Other things are worse' or just randomly assigning the professional opinion of "biologists would say..." just directly ignores or is meant to diminish the fact that hogs have negative affects on native species and this is proven empirically. If you don't value the preservation of native species, and native communities/landscapes they comprise, that's fine just say so, but recognize that's where the difference of opinion lies, as some people do value those things.
A main difference between hogs and general development, is that we're as a society, sort of dependent on some amount of development for meeting our basic needs. Hogs are an issue on lands we set aside specifically to be spared from development, so they're an additive threat. It's not a 'pick one or the other' as the bad guy situation.
And again, Im totally down with hog hunting as there's no negative impact from that. I'd just prefer a landscape without them entirely.