This post is an automated archive from a submission made on /r/DataHoarder, powered by Fediverser software running on alien.top. Responses to this submission will not be seen by the original author until they claim ownership of their alien.top account. Please consider reaching out to them let them know about this post and help them migrate to Lemmy.

Lemmy users: you are still very much encouraged to participate in the discussion. There are still many other subscribers on !datahoarder@selfhosted.forum that can benefit from your contribution and join in the conversation.

Reddit users: you can also join the fediverse right away by getting by visiting https://portal.alien.top. If you are looking for a Reddit alternative made for and by an independent community, check out Fediverser.

Gotta be a solution out there already right?

Have you tried wfdownloader, works well for scraping forums.

A little bit shell magic around a little Python3 helper will do the job quickly: for i in $(seq 1 10); do grablinks.py --fix-links 'https://forums.aida64.com/topic/667-share-your-sensorpanel/page/'"${i}" --search 'file/attachment.php?' -f 'wget -c -O '\''%text%'\'' '\''%url%'\' | fgrep '.sensorpanel'; done | tee fetchscript.sh, then verify the generated shell script, and finally: sh fetchscript.sh.

Happy waiting! :)

You can grab my grablinks.py Python3 script from here: https://github.com/the-real-tokai/grablinks