Two Labour frontbenchers have quit after party leader Jeremy Corbyn sacked two "disloyal" senior figures and promoted a Trident opponent in a so-called "revenge reshuffle".
However, in what could be a sign of further problems ahead, Mr Benn was forced to deny claims made by allies of Mr Corbyn that he had agreed to conditions in returning for staying in the role.
The Durham North MP also claimed Maria Eagle had wanted to remain Shadow Defence Secretary.
The party sources said the changes were never intended as a "Big Bang", but were "relatively modest".
Speaking after the 36-hour reshuffle, shadow chancellor John McDonnell said Mr Benn would be free to give his views on a vote but from the backbenches.
Hilary Benn will have to stand down if he wants to disagree on foreign policy with the Labour leader again as part of the deal to keep his job. Thornberry is, like Corbyn, against Trident.
He told Radio 4 Today programme: "We were promised when Mr Corbyn took over, there would be a more open more pluralist politics where people can speak their mind on things, that would take place ..."
Europe shadow secretary Pat McFadden also said he had been sacked in the first reshuffle since Corbyn was elected leader in September a year ago, in part due to comments McFadden made about terrorism which he said Corbyn interpreted as a challenge to his own response to the Paris attacks.
Ms Eagle, who had clashed with former London mayor Ken Livingstone over a defence policy review they were jointly overseeing, is said to have been keen to take on the culture brief.
Dugher, who had written a lengthy article in the New Statesman criticising the leader, told Sky News yesterday: "I think I tried to do my best in delivering some straight talking and honest politics, and I think it was a little too much for him, which is a shame".
Labour backbencher Wes Streeting said he was "gutted" by the news, branding the leadership "a shower".
Labour's frontbench team now has 17 women and 14 men.
"Mr McDonnell said: "(Mr Corbyn) is re-dictating the terms of what leadership is all about.
The Garston and Halewood MP has instead been handed her "dream" job as the party's culture brief, replacing the outspoken Michael Dugher.