If a fighter is turned completely side on...

[Levity]

This isn't true. Before the Toney-Jirov fight the referee had a talk to Toney about this issue. He anticipated it because of Toney's style, saying "if you turn your back, it's a legal blow". Bernstein and the fellow commentator talk about it during the fight.

Is that in the rules anywhere? Or did the ref just make that up himself?

Presumably this referee would also allow full force shots to the back of the head if one fighter turned his back quickly, say after missing a shot wildly or stumbling off-balance. Ultimately, this means that it's about where one intends to land a blow and far less about where they actually land it, although I believe most sets of rules only address the latter.

If we can just make up the rules for ourselves, I would focus on warning and penalizing the fighter committing the initial infraction instead of letting those and the punches to the back of the head, low blows, etc. slide.

[Levity]

That's not true. A ref won't penalize you if the fighter ducks into it. Happens a lot. Some refs miss it, some don't and allow it. If you bend over and get hit in the back, it's legal if the ref properly sees that the fighter turned.

I meant to say there is nothing in the rules that qualifies it, and there isn't to my knowledge, at least in any notable written set of boxing rules. Obviously those are fixed during a certain bout, though referees enforce them differently, as you seem to note despite also making contradictory blanket statements ("A ref won't penalize you if the fighter ducks into it).

[Levity]

Also, is it even illegal to be completely side on? You can still land punches to the front and the side of the head if your opponent is in that position.