That is a thing, but I know of exactly one person personally that has that. Most people here just have screw on filters that goes on their sink tap, or a fridge with a water dispenser, which already needs a water line for exactly the reason you guessed, ice cubes.

Most refrigerators here come with a built-in filter now at least to make ice. And often using a fridge filter or one that attaches to the sink tap is something anyone can do themselves, whereas an under the sink or whole-house system is something many people would hire someone to put in. So it’s probably a combination of convenience and cost.

We are on a well, so we have a whole-house filter system. But our fridge still came with a filter for its built-in tap.