Bittercress - Remove or keep? - eviltoast

Hi all. I have quite a lot of hairy bittercress in my garden. I think it rode in on the compost I got this year. Ive been removing it from directly around my pea plants but in the open spaces of the garden where its really taken off, Ive surrendered and let it be.

My question is this: Should I be removing it or should I leave it as a kind of live mulch. It doesnt appear to be hindering my peas or other plants in that bed but if it will do long term damage to the soil Id rather take care of it now.

Anyone have any experience with it. Is it worth the effort of removing or do I just let it be?

Thanks!