Reference: http://www.alfawzan.af.org.sa/node/14835
Question: Does a lot of whispering during the prayer nullify the prayer?
Response: No, it does not nullify the prayer, however, it reduces the reward; [because] the reward of the prayer is according to how much of the time your heart was attentive [during it].