Edit: Looks like forcing the lock to stay up is the best option, plus adding a hook-and-eye latch to keep the cats (one of whom particularly reckless) from getting onto the roof by way of the balcony.
When I go onto the balcony and close the screen door behind me, the lock falls down into the locked position. (Conveniently, this mostly happens when I’m not wearing pockets and therefore don’t have my phone.)
The lock doesn’t seem to be loose, or at least the screw won’t go any tighter. I don’t think I’m closing the screen any more firmly than necessary.
Other than trying to remember to instead close the glass door behind me — passing through doors happens on autopilot so I’m very likely to forget — is there some way to prevent locking myself out? Or is it working as designed and I have a head-in-clouds problem rather than a flaky-door problem?
Are you actually up on a higher floor where there’s little chance of anyone trying to get in that way? If the lock has a key, maybe you could hide a copy on the balcony? Alternatively I would wedge something into the lock, it’s not like it actually DOES anything. Anyone who wants in your house will simply go through the screen without making enough noise to alert the neighbors.