Neighbors aren’t strangers. When I say strangers I mean like “Person you have never seen before passing through”. There’s an issue with that called ‘moral hazard’ - when it’s your literal neighbor, if he wrongs you, you have forms of recourse. You literally know where he lives. If a stranger wrongs you, you have no idea where he is or where he’s gone to - hence the need for exchange to be immediate and roughly equivalent (barter). What you’re describing is known as a ‘reciprocity’ or a ‘gift economy’, depending on the exact details of the society in question.

These IOUs get formalized into quantifiable valued you can write down and that’s fiat money.

Not quite. When you start talking about quantifiable values, you start talking about units of value, which are less important in reciprocal or gift economies. When quantifiable debt is being discussed, you start heading into early market economies.

Fiat currency is money that’s given its purchasing power by the central government or by its use as a common (though pointless) medium of exchange - or both, depending on who you talk to.