In this specific example, my question is why pay farmers to not grow crops instead of encouraging them to create a surplus and just paying them the difference in the price drop?

The farmers are getting paid to do nothing ergo if you stop compensating them for doing nothing they will grow crops instead to make up for the difference. The government is whole reason this scenario is messed up in the first place.