I told it, he didn't underestimate Kalin. It wasn't likely that he would come out as victorious from such a duel. But it would effect his image in front of the masters. So it just wasn't beneficial to fight openly.
But it is also not beneficial if he is constantly interrupted in his plans. I would expect him to be highly annoyed to an aggressive point towards Kalin .