I also have a Colt Mk III in .357 that I used to carry in a shoulder holster while float hunting in the canoe for offside shots. Unfortunately I have never taken a shot at a deer with it so no help there. It has killed many, many rabbits though shooting .38 spl out of it. It was my primary weapon for rabbits (my winter season hunting fix) when there was still a decent population to hunt around here. So much easier to carry and shoot in the thick cover with both hands free.
New guys I'd go hunting with would look at me at chuckle while they were loading shells into their shotguns or .22 rifles and I was slipping the pistol into a shoulder holster. I distinctly remember one time I went hunting with a buddy and a friend of his from work. We're at the tailgate getting our weapons out and the guy turns to me showing a .22 rifle. "I like to give them a chance." he says to me with a serious look on his face as if apologizing in advance for his misses. "Me too." I replied as I slipped the pistol into the shoulder holster. He thought I was joking until the end of the day and I had my limit of five and those guys were skunked.
It was with this gun that I discovered that I'm way better at hitting moving targets than stationary ones while firing a pistol. I just have to get out of my own mind and let instinct take over.

I'm infamous with my buddies for completely missing sitting rabbits at 15ft only to roll them over on the second shot while they were running away.
EDIT: FWIW I also blame this pistol (really my young, dumb self) for most of my hearing loss and tinnitus I suffer from today. I never wore and hearing protection hunting with it in those days and my ears would often ring for days afterwards. Just too headstrong as young man to accept that there would be long term consequences to my actions.