@BreadstickNinja - eviltoast
  • 1 Post
  • 339 Comments
Joined 1 year ago
cake
Cake day: July 1st, 2023

help-circle








  • It’s a fun toy. It’s not a research aid, it’s not a productivity tool, and it’s not particularly useful in the workplace.

    It’s honestly very similar to the VR craze of a few years back. Silicon Valley invented a fun toy and then tried to convince everyone that it would transform the workplace. Meetings in VR and simulated workstations and all that. Ultimately everyone figured out that VR is completely useless in the workplace and Silicon Valley was just trying to find ways to sell their fun toy. Now we’re going through the same learnings with AI.


  • It’s actually not easy to ensure that an LLM will cite a correct source, in the same way it’s not easy to ensure that it will provide accurate information. It’s based on token probability, not deterministic lookups of “this data came from this source.” It could entirely make something up, then write “Source:” and then probabilistically write “Wikipedia” because those tokens commonly follow those for “Source.”

    If you have an AI bot that looks up information in real time, then that would be easy. But for a trained LLM, the training process is highly destructive. Original information is not preserved except in relationships based on probability.



  • You do it by comparing the state voting results to pre-election polling. If the pre-election polling said D+2 and your final result was R+1, then you have to look at your polls and individual polling firms and determine whether some bias is showing up in the results.

    Is there selection bias or response bias? You might find that a set of polls is randomly wrong, or you might find that they’re consistently wrong, adding 2 or 3 points in the direction of one party but generally tracking with results across time or geography. In that case, you determine a “house effect,” in that either the people that firm is calling or the people who will talk to them lean 2 to 3 points more Democratic than the electorate.

    All of this is explained on the website and it’s kind of a pain to type out on a cellphone while on the toilet.