Vending machine error reveals secret face image database of college students - eviltoast

Facial-recognition data is typically used to prompt more vending machine sales.

  • cmnybo@discuss.tchncs.de
    link
    fedilink
    English
    arrow-up
    178
    ·
    9 months ago

    Why the hell does a vending machine need a facial recognition camera to “activate the purchasing interface”?

    There should just be a set of buttons to select what you want and a window so you can see what items are available.

    • Aatube@kbin.social
      link
      fedilink
      arrow-up
      103
      ·
      9 months ago

      Stanley sounded alarm after consulting Invenda sales brochures that promised “the machines are capable of sending estimated ages and genders” of every person who used the machines without ever requesting consent.

      • NatakuNox@lemmy.world
        link
        fedilink
        English
        arrow-up
        19
        ·
        edit-2
        9 months ago

        Yup it’s for “advertising” say for example the Army wants to know which areas have the most fighting aged men. So posters and recruiters know where to hang out. (this is the most extreme example.)

        • darth_helmet@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          7
          ·
          9 months ago

          Vending machine company sells facial recognition and temporal location data to a data broker who enriches it to enhance identifiability and this data is sold to a stalker who uses it to murder people. That’s a more extreme, but certainly not most extreme, example.

    • LesserAbe@lemmy.world
      link
      fedilink
      English
      arrow-up
      16
      ·
      9 months ago

      I saw some posts about a similar technology in the meetings and events industry: a company is selling “facial analysis” not “facial recognition.” They try to get around privacy laws by saying “well our technology does scan every single face it sees, but it doesn’t store that image, it just determines age, gender, race and emotional sentiment and adjusts tallies for those categories in a database.”

      It’s still information gathering I didn’t consent to while attending a conference, and it’s a camera with the potential to be hacked.

      Of course it’s always about marketing and advertising. They want to have a heat map of which areas are popular and at what times. In the case of events so they can sell to sponsors and exhibitors. In this university it’s less clear. Do the vending machines have a space to sell ads? That would be my guess.

    • magnetosphere@kbin.social
      link
      fedilink
      arrow-up
      13
      ·
      9 months ago

      Because people are dumb. If the machine knows when someone is looking at it, it can stop doing whatever it does to try and get your attention, and put itself in “sales mode”.

      Still, you’re right. It seems like an overly complicated and expensive solution. Old-fashioned vending machines did the job just fine.

  • _number8_@lemmy.world
    link
    fedilink
    English
    arrow-up
    151
    arrow-down
    3
    ·
    9 months ago

    why do people think it’s okay to do this shit? if you’re coding facial recognition for a vending machine, that’s like 80 steps too far down the capitalism ladder

    if you took this machine back to the 1920s and told people what it was doing, they’d shoot at it. and probably you

    • random9@lemmy.world
      link
      fedilink
      English
      arrow-up
      23
      arrow-down
      3
      ·
      edit-2
      9 months ago

      80 steps too far down the capitalism ladder

      This is the result of capitalism - corporations (aka the rich selfish assholes running them) will always attempt to do horrible things to earn more money, so long as they can get away with it, and only perhaps pay relatively small fines. The people who did this face no jailtime, face no real consequences - this is what unregulated capitalism brings. Corporations should not have rights or protect the people who run them - the people who run them need to face prison and personal consequences. (edited for spelling and missing word)

    • Kornblumenratte@feddit.de
      link
      fedilink
      English
      arrow-up
      15
      arrow-down
      1
      ·
      9 months ago

      In the article is a sound explanation: the machine is activated by detecting a human face looking at the display.

      If this face recognition software only decides “face” or “not face” and does not store any data, I’m pretty sure this setup will be compatible with any data protection law.

      OTOH they claim that these machines provide statistics about age and gender of customers. So they are obviously recognising more than just “face yes”. Still – if the data stored is just a statistics on age and gender and no personalised data, I’m pretty sure it still complies even with 1920s data protection habits.

      I’m pretty sure that this would be GDPR conform, too, as long as the customer is informed, e.g. by including this info in the terms of service.

      • yuriy@lemmy.world
        link
        fedilink
        English
        arrow-up
        20
        ·
        9 months ago

        If I need to accept a TOS to use a vending machine, I don’t need to use that vending machine.

        • slumberlust@lemmy.world
          link
          fedilink
          English
          arrow-up
          5
          ·
          9 months ago

          Fear not, you agree to car ToS if you get in it as a passenger! Not sure how enforcable that is,but the fact they try is gross enough.

        • Kornblumenratte@feddit.de
          link
          fedilink
          English
          arrow-up
          2
          ·
          9 months ago

          I don’t know about the US, but in Germany, by using a vending machine, you are implicitely and automatically consenting with the ToS of the vendor by your action.

  • ChaoticNeutralCzech@feddit.de
    link
    fedilink
    English
    arrow-up
    104
    arrow-down
    3
    ·
    9 months ago

    The students should get together and jack the machine away into their hacking club and do some reverse engineering, so that we get more information on how the data collection worked as opposed to just trusting the company’s statements. If a hacking group like the German Chaos Computer Club got behind this, they could release their findings while keeping the perpetrators anonymous. However, I’m pretty sure the machine is just a frontend to a server, which got shut down as soon as the students complained, with no GDPR-like checkout being available in the jurisdiction.

      • KairuByte@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        8
        ·
        9 months ago

        No only was a person behind the decision, a person was also behind the dissemination of the requirements, the implementation of the change, the design of the hardware, and all steps in between.

    • Hamartiogonic@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      14
      ·
      9 months ago

      When you start tinkering with a machine learning model of any kind, you’re probably going to find some interesting edge cases the model can’t handle correctly. Maybe there’s a specific face that has an unexpected effect on the device. What if you could find a way to cheese a discount out of it or something?

      • redcalcium@lemmy.institute
        link
        fedilink
        English
        arrow-up
        17
        ·
        9 months ago

        Imagine a racist vending machine. The face recognition system think this customer is black with 81% confidence. Let’s increase the price of grape soda! Oh look, a 32 year old white woman (79% confidence). Better raise the price of diet coke!

        • SpaceCowboy@lemmy.ca
          link
          fedilink
          English
          arrow-up
          3
          arrow-down
          1
          ·
          9 months ago

          In Japan they had some kind of facial recognition on vending machines selling cigarettes that would determine the age of the person in attempt to prevent kids from buying cigarettes. But it only worked for Japanese people.

          Stupid racist vending machine wouldn’t sell me smokes!

            • SpaceCowboy@lemmy.ca
              link
              fedilink
              English
              arrow-up
              2
              arrow-down
              1
              ·
              9 months ago

              It’s cool, I quit years ago.

              Also I was in a diverse group of people and we were able to do some science. Fortunately we had a Japanese person in the group which allowed me to purchase the smokes. But yeah, it failed on everyone that wasn’t Japanese.

        • Hamartiogonic@sopuli.xyz
          link
          fedilink
          English
          arrow-up
          1
          ·
          9 months ago

          When you use a generated face with a mixture of white and black features, that’s when it gets interesting. Maybe you can even cause an integer overflow.

      • ChaoticNeutralCzech@feddit.de
        link
        fedilink
        English
        arrow-up
        5
        ·
        edit-2
        9 months ago

        I don’t think they’re doing dynamic pricing on an individual basis, that would be too obvious. But checking the demographics of each location or individuals’ shopping habits, and potentially adjusting the prices or offerings? Definitely.

        • Hamartiogonic@sopuli.xyz
          link
          fedilink
          English
          arrow-up
          1
          ·
          9 months ago

          So, if you show it 100 faces from group A and 4 faces from group B, that could start gradually shifting the prices in a specific direction. If you keep going, you might be able to make it do something funny like charging 0.1 € for a Pepsi and 1000 € for a Coke or something like that. If the devs saw that coming, they might have set some limits so that the price can’s spiral totally out of control.

          • ChaoticNeutralCzech@feddit.de
            link
            fedilink
            English
            arrow-up
            2
            ·
            9 months ago

            I am sure the profit margin is taken into account, so you won’t get an ultracheap Pepsi unless it expires soon. Similarly, I expect it to consider economic viability, so it won’t keep raising prices unless people are willing to pay them. Of course, you never know what the model actually does or what goals it follows (maximizing profit is a good guess, though), or how bad the coding is. The program might be very versatile and robust, or it may break when you show it a QR code - how can I know? Probably something in between.

  • Tristaniopsis@aussie.zone
    link
    fedilink
    English
    arrow-up
    68
    arrow-down
    3
    ·
    9 months ago

    “Where Cadillac Fairview was ultimately forced to delete the entire database, “

    LOL yeah right.

    “OK BUBBA! WE DONE DEE-LEETED THE ENTIRE THANG!!”

    Bollocks.

    They probably gave the ‘enforcement’ agency a blank hard drive and said “Well, gee, shucks. That’s all we had!”

  • theodewere@kbin.social
    link
    fedilink
    arrow-up
    63
    ·
    9 months ago

    “over 5 million nonconsenting Canadians” were scanned into Cadillac Fairview’s database

    fully scanned facially by automated kiosks in malls… the database was deleted only after an investigation…

    • pete_the_cat@lemmy.world
      link
      fedilink
      English
      arrow-up
      20
      ·
      9 months ago

      Tons of Point of Sale terminals run Windows instead of Linux for some reason, probably because the software they run is only written for Windows.

      • grandma@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        17
        arrow-down
        1
        ·
        9 months ago

        Makes sense, but a vending machine shouldn’t need a fully fledged OS in the first place imo

        • Flying Squid@lemmy.world
          link
          fedilink
          English
          arrow-up
          10
          arrow-down
          4
          ·
          9 months ago

          I don’t get it either. What do vending machines need to be computerized for at all? What was wrong with the old kind that was around for decades where you put your money in, pushed a button, and stuff came out? I certainly can’t think of a reason for a vending machine to have a camera. That’s nuts.

          • BreakDecks@lemmy.ml
            link
            fedilink
            English
            arrow-up
            11
            arrow-down
            1
            ·
            9 months ago

            It’s 2024, most people don’t carry cash, and the whole world runs on automation. These kinds of vending machines are completely over the top, but it’s actually a pretty bad idea to not use computers for this application. Just knowing when machines need to be refilled remotely saves more money than such an implementation would cost.

            • Flying Squid@lemmy.world
              link
              fedilink
              English
              arrow-up
              3
              arrow-down
              4
              ·
              9 months ago

              It’s not hard to know when machines need to be refilled. You just come regularly, take note of how much or little stock has been purchased, then adjust your refill amounts and times accordingly. This has to be done regardless of a handful of computerized machines because plenty of them still aren’t.

              Accepting a credit card or tap-to-pay would probably require computerization, but the technology should be no more complex than any other, similar piece of hardware and the machine should even be able to work if the card network is down and just accept cash if that happens.

              So sure, part of the machine should be computerized. The part that accepts money. The rest is unnecessary, probably raises the price of the machines unnecessarily and certainly never justifies a camera.

              • Rob T Firefly@lemmy.world
                link
                fedilink
                English
                arrow-up
                10
                ·
                9 months ago

                It’s not hard to know when machines need to be refilled. You just come regularly, take note of how much or little stock has been purchased, then adjust your refill amounts and times accordingly. This has to be done regardless of a handful of computerized machines because plenty of them still aren’t.

                I worked in the arcade/vending business in the 1990s. That blind maintenance model was a crapshoot for the machine owners. We had to routinely send a crew (usually me and one other person) to drive to a location - near or far - with games, photo booths, vending stuff, etc. just in case the supplies in some machine or another ran out, something needed fixing, etc. Sometimes we’d arrive and learn we have hours of refilling and/or maintenance work to do on a machine, sometimes it had been a slow week or two and a crew had just spent their whole workday and a tank of gasoline to collect $50 from the cash box and go home again. Remote administration really changed the game for that whole business.

              • BreakDecks@lemmy.ml
                link
                fedilink
                English
                arrow-up
                4
                ·
                9 months ago

                An IoT SIM costs a whole lot less than sending a technician to every machine to check stock. I’m not arguing in favor of facial recognition, I’ve already made that clear, but you are dead wrong if you don’t think automation at scale isn’t economical.

                If you’re already putting a modem in the box for credit cards, why not collect some telemetry? Sensors are cheap and effective.

                • Flying Squid@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  2
                  arrow-down
                  3
                  ·
                  9 months ago

                  They have to go to every machine to restock regardless. All they have to do is note down on a little notepad or even an app on their phone what sells, what doesn’t and how quick.

                  I’m sorry, I just can’t go along with internet-connected public vending machines. If you want to connect everything in your house to the internet, fine. But a machine that sells candy bars does not need to be connected to the internet just because it’s marginally more efficient to do so than the way it had been done previously for decades. Because it results in this sort of shit. And unnecessary price-gouging through selling a university expensive machines with an unnecessary connection to the internet instead of something that worked perfectly well already and didn’t cost as much money.

        • Appoxo@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          5
          arrow-down
          2
          ·
          9 months ago

          As if Linux based vending machines aint a full fledged OS even with a minimal installation?
          This aint embedded.

          • msage@programming.dev
            link
            fedilink
            English
            arrow-up
            1
            ·
            9 months ago

            No, Linux is a kernel.

            OS is a specific distribution, so like a Debian is the full fledged OS.

            So just write your inventory inside the file, and bind the vending machine keys to it, and ignore 99% of the OS. The coin slot I would expect runs its own validations.

          • grandma@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            2
            arrow-down
            2
            ·
            9 months ago

            Yes of course Linux is a fully fledged OS, my point was a vending machine should not need any OS, my bad if I didnt make that clear

            • herrvogel@lemmy.world
              link
              fedilink
              English
              arrow-up
              2
              ·
              9 months ago

              Why not? A full windows environment (though not really, because these things run what’s called the kiosk mode) can run on cheap SBCs and gives you a ton of hardware and software flexibility, and is also pretty convenient. It’s very commonly used for very good reasons.

              • grandma@sh.itjust.works
                link
                fedilink
                English
                arrow-up
                2
                arrow-down
                1
                ·
                edit-2
                9 months ago

                TIL about windows kiosk mode!

                I can understand it from the perspective of the developers who need to implement all this crazy tracking/advertising/graphics functionality, but imo a vending machine should only do three things:

                1. Let me see what is available (preferably using glass)
                2. Accept payment
                3. Give me what I paid for

                Vending machines have done this for decades without requiring an operating system. Keep it simple!

                • herrvogel@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  3
                  ·
                  9 months ago

                  Simple in what way?

                  You could make logistics simpler by giving these things networking capacity so you can remotely track their stock and cash levels.

                  If your software needs to run on multiple different device configurations, you can simplify development and deployment by letting the OS handle a lot of the low level stuff.

                  In other words, a simpler machine is not necessarily going to be simpler to operate for the company.

              • Miaou@jlai.lu
                link
                fedilink
                English
                arrow-up
                1
                ·
                9 months ago

                Let’s stop kidding ourselves, the “good reason” is the cto’s yearly Microsoft financed holidays and/or too much legacy code to restart from scratch. But from a purely technical aspect, there’s no reason to touch windows

        • Tbird83ii@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          2
          ·
          9 months ago

          It isn’t. It would most likely be windows IoT. it’s an embedded windows OS that allows for a single app instance to be running.

          You’d be surprised how many things run windows IoT right now…

      • Diplomjodler@lemmy.world
        link
        fedilink
        English
        arrow-up
        4
        arrow-down
        2
        ·
        9 months ago

        A low end Windows PC can be had very cheap these days. Why bother doing something proprietary, if you can just cobble together something from off the shelf parts?

        • BreakDecks@lemmy.ml
          link
          fedilink
          English
          arrow-up
          8
          ·
          9 months ago

          This isn’t even remotely true. Everyone knows that if you’re trying to do a cheap embedded product, you use SBCs and Linux. Using Windows for these kinds of applications is almost always the result of a company having a contract with Microsoft that leads their development strategy towards Microsoft’s offerings rather than the best offerings.

          Also, in what universe is a Linux platform more proprietary than Windows?

          • Miaou@jlai.lu
            link
            fedilink
            English
            arrow-up
            2
            ·
            9 months ago

            People here are delusional, booting windows eats more ram than your typical embedded product needs to run. Same goes the hard drive.

            And this people also think maintaining a yocto/build root image is proprietary 🤷‍♂️

  • db2@lemmy.world
    link
    fedilink
    English
    arrow-up
    29
    arrow-down
    3
    ·
    9 months ago

    To the people that allowed that gross invasion to happen:

    Oopsie woopsie, diddums make a widdle fucky wucky? Yes you did. Yes you did.

    Then do what you’d do to any other child: take away the toy they misbehaved with.

  • Kissaki@feddit.de
    link
    fedilink
    English
    arrow-up
    28
    arrow-down
    7
    ·
    9 months ago

    “facial recognition exe” doesn’t say anything about a “face image database” as this post title claims.

    • teamevil@lemmy.world
      link
      fedilink
      English
      arrow-up
      18
      arrow-down
      3
      ·
      9 months ago

      What the hell else could they be doing with the data? Scanning a face without a database is absolutely pointless.

      • Kissaki@feddit.de
        link
        fedilink
        English
        arrow-up
        18
        arrow-down
        8
        ·
        edit-2
        9 months ago

        The linked article tells you: Recognize when someone stands in front of the vending machine.

        “the data” is interpreted. Not stored or matched.

        • CaptPretentious@lemmy.world
          link
          fedilink
          English
          arrow-up
          19
          ·
          edit-2
          9 months ago

          Sure that’s their claim but they’re not asking ‘why have that type of tech anyways’.

          If it’s supposed to just act as a motion sensor, we’ve had those for decades. None of which needed to register if it was a face or not. Why isn’t the purchasing interface just always there, why is it an interface, and why is it not just a button that says press to start…

          Why is there a computer in there that’s been trained on how to recognize what a face is in order to open up a purchasing interface. What would be the point of investing that much research and development if it was just doing something that could have been accomplished in the '90s with tech that you could have bought it radio shack.

          • brianorca@lemmy.world
            link
            fedilink
            English
            arrow-up
            9
            arrow-down
            1
            ·
            9 months ago

            Article says “the machines are capable of sending estimated ages and genders” so it’s not recognizing individuals, but perhaps adjusting the sales pitch for who it sees walking by.

            (But it’s a collage campus, so most students will be around the same age. Maybe it pitches different things to teachers?)

          • AngryishHumanoid@reddthat.com
            link
            fedilink
            English
            arrow-up
            5
            arrow-down
            1
            ·
            9 months ago

            From the side of someone who works heavily in data analysis and application databases I can tell you it would be very, very easy to see if it was just a front end application using the data or storing it in a database. There are use cases for both setups, absolutely, but a cursory examination of the machine in question would make it abundantly clear which it was doing.

        • uis@lemm.ee
          link
          fedilink
          English
          arrow-up
          18
          arrow-down
          1
          ·
          9 months ago

          “Why do you need fingerprint reader?”

          “To recognize when someone touches the vending machine.”

  • devilish666@lemmy.world
    link
    fedilink
    English
    arrow-up
    19
    ·
    9 months ago

    Hmm… facial recognition vending machine huh…
    Finally it’s time for my jammer & some script from c/netsec to shine

  • pHr34kY@lemmy.world
    link
    fedilink
    English
    arrow-up
    16
    arrow-down
    4
    ·
    edit-2
    9 months ago

    I’d doubt it’s collecting or transmitting much. It’s probably just estimating age, sex, race etc. and using it to decide which promotion to put on screen. It’s possibly collecting these to determine what type of people use the machine. Similar to those billboards in shopping centres.

    Storing each individual to recognize later or identify online seems like a stretch.

    If it did have a user bio database, it would be centralised and not on the machine itself.

    • doctorcrimson@lemmy.world
      link
      fedilink
      English
      arrow-up
      8
      arrow-down
      1
      ·
      edit-2
      9 months ago

      I think the problem is that it is storing the user faces, at all. If it were simple identifying each person’s characteristics there would be no reason to save that data for later. Also, apparently the company advertises that the machine does transmit this data for estimating age and gender for every purchase.

      • dev_null@lemmy.ml
        link
        fedilink
        English
        arrow-up
        2
        ·
        9 months ago

        That’s your claim though. They are storing “male, 24” and that’s it, no face. Of course they could be lying and actually are storing faces, but it doesn’t look like it. And it’s also perfectly valid to object to them storing even “male, 24”.

  • AutoTL;DR@lemmings.worldB
    link
    fedilink
    English
    arrow-up
    11
    arrow-down
    1
    ·
    9 months ago

    This is the best summary I could come up with:


    The Reddit post sparked an investigation from a fourth-year student named River Stanley, who was writing for a university publication called MathNEWS.

    Where Cadillac Fairview was ultimately forced to delete the entire database, Stanley wrote that consequences for collecting similarly sensitive facial recognition data without consent for Invenda clients like Mars remain unclear.

    Stanley’s report ended with a call for students to demand that the university “bar facial recognition vending machines from campus.”

    Some students claimed on Reddit that they attempted to cover the vending machine cameras while waiting for the school to respond, using gum or Post-it notes.

    The technology acts as a motion sensor that detects faces, so the machine knows when to activate the purchasing interface—never taking or storing images of customers."

    It was only after closing a $7 million funding round, including deals with Mars and other major clients like Coca-Cola, that Invenda could push for expansive global growth that seemingly vastly expands its smart vending machines’ data collection and surveillance opportunities.


    The original article contains 806 words, the summary contains 166 words. Saved 79%. I’m a bot and I’m open source!

      • KuroeNekoDemon@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        1
        ·
        9 months ago

        Yeah but this is the University of Waterloo we’re talking about here. This hit Canadian mainstream media CTV News so I know that. Also for an university specializing in Engineering and Mathematics there’s a shit ton of cameras around

  • Snapz@lemmy.world
    link
    fedilink
    English
    arrow-up
    8
    arrow-down
    1
    ·
    9 months ago

    Those any combination coca cola machines have cameras on them.