Previous employer reference has made me lose a job. What can I do?

Rather disappointed, was supposed to be starting a job on Monday after being out of work for several months. Seemed like a solid place plenty of room for progression etc.

They was adamant about me providing a reference from my last employer which I was hesitant about due to this happening.

I left on bad terms, didn’t have a good relationship with one of the directors who was a bully and his son in law, which I eventually got into a physical confrontation with.

The situation started as an argument which lead to his behaviour progressively getting more and more threatening, getting into my personal space after me backing away several times, raised voices etc.

He eventually was approaching me with his arms out shouting “come on then” I backed off several times until I couldn’t anymore and decided to take action. no real serious damage was caused to myself or him other than bruising.

Anyways i left as soon as this happened as i had enough of the place so it wasn’t exactly like i was sacked.

Anyways I’m assuming they have mentioned this situation and being a family run company have completely painted me as the bad guy.

Is there anything I can do?? Or is it a lost cause? Respectfully not looking for your opinion in what I did if it was right or wrong.