How can I scrape every .sensorpanel attachment from this thread? - eviltoast
  • binaryriot@alien.topB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    A little bit shell magic around a little Python3 helper will do the job quickly: for i in $(seq 1 10); do grablinks.py --fix-links 'https://forums.aida64.com/topic/667-share-your-sensorpanel/page/'"${i}" --search 'file/attachment.php?' -f 'wget -c -O '\''%text%'\'' '\''%url%'\' | fgrep '.sensorpanel'; done | tee fetchscript.sh, then verify the generated shell script, and finally: sh fetchscript.sh.

    Happy waiting! :)

    You can grab my grablinks.py Python3 script from here: https://github.com/the-real-tokai/grablinks

    • ocp-paradox@alien.top
      cake
      OPB
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      Wow thanks. I’ll try this out tonight. Was just a last ditch effort here before I went and manually did 50 pages a day or something. Thanks again.