Reference: al-Hilyah – Volume 4, Page 359
Abdullaah ibn Abee al-Hudhayl said:
The time for prayer entered upon a group of scholars; So it was said to one of them to lead [the prayer], but he declined. It was said to him: What prevents you [from leading the prayer]? He said: I feared that a passerby would pass & say, certainly he was asked to lead [the prayer] because he was the best of them.