Here's my 2 cents on your situation, what I would have done the same and what I would have done different. I am not writing anything to criticize you, I only hope it helps you for future hunts. There is definitely some overlap with a few things I've already read in here.
After the shot the first thing I would have done was mark the last spot I saw the deer. I try to pick out a tree as a landmark. Sometimes there won't be much of a blood trail initially and having this spot marked can be a huge help. Sometimes you'll walk over to this spot and they are laying right there. Then I'll mark where the shot was. In the heat of the moment sometimes this is really hard to remember even just minutes later. Again I pick a tree, rock, bush as a landmark. Depending on how everything went down determines what I'll do. If I know it was a gut shot, I just back out and don't even check out the arrow. I had one bed 30 yards into the thick stuff after the shot once and pushed it just by checking out the arrow at the shot location when I knew it was a gut shot and was going to back out. If I think its a shot I might was to start tracking right away, or want to consider my options I'll go over to the spot of the shot. Then I mark it with toilet paper. If I think its worth trailing right away I'll follow the blood trail and mark blood with TP. TP makes it easy to look back and see if the deer is trending in a direction. If I think I need to wait I back out. If the weather allows me, I am conservative. The buck I shot on Oct 30 was at noon. I went back in just before light on the 31st and found him right away. I have another buck I shot in 14 degree weather at 9 am. I went back at light the next morning and it took a few hours to find him due to a freak snow flurry but I found him at noon the next day. Both of these deer actually died right after the shot but they were still dead the next morning.
As far as going back and other hunters. The best solution is to hunt areas where the other hunters aren't going. Then you won't have to worry about it. But that is not always possible so this is what I would do. If I know I have to track a deer I'm probably up super early anyway cause I'm antzy to get on the trail. I would get in there to where I shot at least 1/2 hour before daylight and be ready to go at light. You can even start snooping around in the dark. I have a 500 lumen LED light that I recommend for tracking and I've spent many hours tracking in the dark with it. If you are there before the other hunters to track your deer they are probably not going to give you a hard time. This is where I differ on some of the other guys opinions about waiting a few hours until after light.
I do think you did the right thing by avoiding confrontation. In that case I probably would have waiting in my car or went to get breakfast and waited for the guy to be gone to try to track.
Hope this helps.
Pretty bad.
