Ed Miliband has said his relationship with brother David has still not fully recovered, more than three years after their fight for the Labour leadership.
The Labour leader told BBC Radio 4's Desert Island Discs it was "healing", following his narrow victory in 2010,
On the programme Mr Miliband defended his party's handling of the candidate selection row in Falkirk.
And he said he was inspired by his late father, Marxist thinker Ralph Miliband, but they were different politically.
Mr Miliband insisted he had no regrets about any of the "big decisions" in his life and said David was still his best friend.
"It has been incredibly tough; really, really tough. I didn't take this decision lightly. I knew it would have an impact on my family and on him," he told presenter Kirsty Young.
"What I thought was: 'This is going to be hard but I've got things to say and things to do which I am in a better place than him or others to say and do about the way we need to move on from New Labour, about things that need to happen in the country'."
Asked about the most difficult part of his role as leader of the opposition, he said: "It's the 24/7 nature of the job.
"It's about the fact that I've got two small children, Daniel and Sam, and they're getting to the stage where they know if dad's around or not."
He was asked whether putting his family "front and centre" of his 2010 conference speech was a smart thing to do.
"Absolutely," he replied. "Because you can't understand me without understanding where I come from. My dad had a particular political outlook, it's not the same as mine, but it's a big part of what inspired me.
"In modern politics, who you are, who your family is, it's always going to be relevant and important to people, so it comes with the territory."
He added: "We live in a capitalist society - my dad thought you could abolish capitalism, I don't - but I think it throws up fundamental injustices.
"What motivates me as a politician is you see injustice and you seek to do something about it."
On the Falkirk constituency issue, he said: "The person around whom there was controversy is not going to be the candidate, there was a joining up scheme which we've closed down, none of the members who were joined up are going to be able to vote, we've had a police inquiry.
"So I honestly say to you we've acted really thoroughly to uphold the integrity of the party."
Speaking about one of his selections, the hymn Jerusalem, he said: "Being the son of refugees; my parents love this country.
"When the Mail wrote [about his father] 'man who hated Britain', famously, the reason I hated that so much was because it was so at odds with the way my parents think about this country."
He talked about how his parents fled from Europe to Britain to escape persecution from the Nazis in World War II, and about the death of relatives in concentration camps.
Mr Miliband said his wife Justine was "an amazing person" and his family "mean everything to me, it's the most important thing in my life".
Young said people were sometimes sceptical about politicians' choices on Desert Island Discs - and asked how many people had cast their eye over his list.
Mr Miliband replied: "It's absolutely my list. It's a list that is personal to me - some people will not like some of the songs and that's as it should be - but this is a list I chose."
The Labour leader's other selections included Angels by Robbie Williams and the left-wing anthem The Ballad of Joe Hill by Paul Robeson.
His desert island luxury was a regular chicken tikka curry takeaway and his book was Douglas Adams' The Hitchhiker's Guide to the Galaxy.
Prime Minister David Cameron appeared on Desert Island Discs in 2006 and Deputy Prime Minister Nick Clegg was on the programme in 2010.
Listen to Desert Island Discs on BBC Radio 4 at 11:15 GMT on Sunday 24 November and 09:00 GMT on Friday 29 November, or afterwards on the BBC iPlayer.