It certainly isn't the Man's place to interrupt anyone's discussion. He makes deals, not a sense of rudeness. He merely takes the moment to continue eating his ham sandwich.

...Yet, as the discussion continues, the Man does keep in mind that he has work to do. Work that can't be interrupted for long. Eventually, he does volunteer some words.

"May I help you?"