I was waiting in a kind of rough bar around the corner from my office for a friend to show up, and a drunk at the next table started harassing me basically by saying I was a cop. I didn't know anyone in the place. There were no bouncers around. I was drinking vodka and orange juice. I said "I'm not a cop" in a tone of aggrieved innocence, as if I found cops as offensive as he did. Nothing aggressive or confrontational, just defending myself against the accusation. This guy left his table and when he returned, he said something else about cops, very aggressively, and jostled my table, upsetting my glass and spilling orange juice over my white Nikes. He still hadn't sat down.
I realized immediately any sign of cowardice would egg the guy on, so without any hesitation at all (this was a split-second decision) I rose to my feet with a very grim look on my face (I'm almost six foot one) not taking my eyes off his. I really didn't know what I was going to do next, but his demeanour changed immediately. "What are you gonna do now, arrest me?" - - but he said it in a whining, self-pitying way, like I was some kind of bully, and if I had any decency I'd just overlook the whole thing. Now he was scared. Luckily at just that moment, some bouncers appeared and hauled off his ass.
=======================
About two years ago, I was standing at the payphones and just starting to dial a number just inside the entrance of a huge internet cafe on 42nd Street, just a few doors west of Broadway and next to one of the big theatres and a McDonalds, and this husky black guy in his forties, in a blue denim jacket starts accusing me in a Jamaican accent of moving all his papers off the tray under the phone. Now in New York, ANY interaction with a total stranger is potentially a mugging or a scam, particularly in a tourist-rich environment like Times Square. This guy was definitely physically intimidating. If it was a mugging, there was probably nothing you could do but run like hell; OTOH, it was more likely to be just a scam, because of the heavy police presence, both uniformed and undercover, in which case the guy was more interested in creating the threat of physical violence rather than inflicting physical violence. Anyway, I sized it up as a scam. Again, the reaction has to be split-second. I immediately put the phone down, standing nose to nose with the guy and not backing away an inch, look him right in the eye and say "Get the fuck outta here. I didn't touch your stuff." Very calm, unhesitating, and a real ice-cold delivery. The "get the fuck outta here" was pure New York - - if the guy's a scammer, he's only interested in out-of-towners. The guy just disappeared for a minute. One minute he's there, then he's not. Came back a minute later and tried to make amends, but that 's another story.
I'm not a tough guy but I'm a fairly big guy. Never been in a fist-fight in my life and never had to, either. There were a few other times when I've been challenged but the secret is, I never back down. I look like I'm ready to go the distance and the guy who started it all invariably backs off. The closest I ever came was an altercation with a Puerto Rican cab-driver just outside the El San Juan Hotel about 11:00 PM, and he only backed off because he was scared of losing his taxi licence, I realized later. He was probably armed with a knife, a pipe or a screwdriver, and I could have been seriously fucked up if he hadn't have come to his senses. I was really stupid that time, the obvious solution was to walk away and look for a cop, but I was in terrible pain from an ear-ache and he was trying to rip me off on a ride to the hospital, so I was just not thinking straight. I provoked him by deliberately slamming his door as I got out and then I just stood there waiting for him when he got out and said he was gonna kill me. Dumber than that you do not get.