The prime minister said that will be achieved in-part through ending Imprisonment for Public Protection sentences, or IPPs.
These were available between 2005 and 2012, and were given to offenders who pose a significant risk of causing serious harm to the public.
Once a minimum term has been served, the prisoner can be released only once the Parole Board is satisfied they no longer need to be confined for public safety, meaning many are being held indefinitely.
Burnham said he had applied “common sense” on the treatment of IPPs, which he had “long-standing concerns” about.
“I was faced with a situation where people on those sentences who have more than served their time, not being released when others who’ve committed very serious crimes being released early,” he told the BBC.
“The judgement I’ve made is the public, in my view, would prefer a situation where people who have served their time to be considered for release before those who haven’t.”
According to Ministry of Justice data, 2,271 people are serving IPP sentences as of June. Of those, 856 have never been released because the Parole Board thought it was not safe to do so.
Burnham conceded that of those offenders, there will be “individuals who will pose a risk to communities”, and said a cross-party oversight group would be established.
The government will table legislation to terminate the sentences by the “end of this Parliament”, he added.
Pressed on how many foreign prisoners could be returned, Burnham said it was “hundreds”.
The government has said powers will be implemented early next year to deport some foreign criminals immediately after sentencing.
Those powers would block foreign prisoners from ever returning to the country while “keeping those in prison for the most serious crimes locked up for a substantial period before they are removed”.
The government is not thought to have signed any additional prisoner returns agreements with other countries.
It is understood that work has also begun to create more cells in existing prisons by adapting space used for other purposes.
None of the measures announced by Burnham are likely to create much additional prison capacity by the time the first batch of prisoners are released early in October.
The prime minister is said to have taken a “political decision” to use some of the remaining capacity in the prison system to ensure the changes announced today could be implemented immediately.
In addition to those in jail for manslaughter, the prime minister said prisoners convicted of indecent assault offences and death by dangerous driving will also remain in prison.
Historic child sex offences and causing or allowing the death of a child will also be exempted from the early release scheme, while new “restriction zones” to better protect domestic violence survivors from perpetrators released early will be introduced.
Burnham said the additional changes announced on Sunday would result in “hundreds” more prisoners staying behind bars.
The justice secretary is expected to give a more precise figure for how many people will be affected when he gives a statement in Parliament on Tuesday.
Earlier this month, Burnham announced prisoners guilty of rape, serious child sex offences and grooming will be excluded from early release eligibility.