Yes, there is an odd dilemma.
Fish & chips carry VAT, so why not a pasty? Yup, I get that.
Caviar is expensive and bought by rich people, so why not tax it? Yup, I get that too.
But we don’t want to tax all food and however you split VATable from non-VATable, there will always be anomalies, so you have to make a choice and take it on the chin from the people you upset. So Mr Cameron realised he could not keep everyone happy.
What I object to is that whenever a choice like this arises, he always falls the same way and decides to upset the generally weaker, easier targets that have less influence. This is why we are clearly not all in it (equally) together.