In her speech, Badenoch also took aim at former politicans who “are in denial” and “don’t want to admit that immigration is too high”.
“Some people say that we need to split the difference and find the centre ground,” she said.
“It’s not about the centre ground, it’s about the common ground and it is about common sense.”
She added that “most people in our country want to control immigration” and the Tories were “a mainstream party”.
Taking questions from journalists after her speech, Badenoch was asked about a new political movement launched by former West Midlands mayor Sir Andy Street and former Scottish Conservative leader earlier this week.
The initiative – named Prosper UK – aims to win back centrist voters who feel “politically homeless”.
Asked how damaging the new movement could be to the Conservative Party, Badenoch said: “Anybody who is trying to push an agenda that is not the platform I stood on is not being helpful.”
She added: “If they’ve got new ideas which are on the right, not left-wing ideas, and we will welcome them. But right now we need to move away from any sort of factionalism or groupings.”
The Conservative leader said her party was “not a party of the left and we should not be accommodating people who want to paint that picture”.
Asked if this referred to Davidson and Sir Andy, Badenoch denied this, telling the BBC: “Ruth Davidson and Andy Street have been very supportive, but they are supporting the agenda which I’m delivering.”
However, she added that there were other people who had joined their movement who had previously defected to the Liberal Democrats and “anyone who wants a different agenda needs to understand that that’s not coming”.