I didn't think the Pharisees were so much condemning them for taking the corn, as they were for laboring on the Sabbath. That is a big no no.
And the temple shewbread:
Showbread, shewbread, Schaubrot, lechem (hap)pānīm(לחם פנים), in a biblical or Jewish context, refers to the cakes or loaves of bread which were always present on a specially dedicated table, in the Temple in Jerusalem as an offering to God. An alternative, and more appropriate, translation would be presence bread[1], since the Bible requires that the bread is constantly in the presence of Yahweh[2].
(via Wiki)
So they were eating the bread that was supposed to be an offering to God.
And Jesus said that was OK. No wonder they killed him.