Mariana Bay

Join the Mariana Bay to take part in the discussion, we would love to have you on board! We have multiple boards for whatever your interests may be and our community will make you feel right at home, so what are you waiting for? Sign up today!

Technology Data-hoarding Thread (+guide)

Welcome to the data-hoarding thread (and guide), where I will explain what is data-hoarding, why you should do it, and how.

(This guide shall be edited with time. Consider it not entirely finished. The thread also is for general discussion of data-hoarding, the guide is just a bonus).

You might wonder:

What is data-hoarding? :newspaperpepe:

If you type "data-hoarding definition" on Goolag™, you'll be prompted with a somewhat negative definition, going as far as to qualify data-hoarding as a "mental disorder", "compulsive behavior":
According to Wikipedia (bad definition):
Digital hoarding
(also known as e-hoarding, e-clutter, data hoarding, digital pack-rattery or cyber hoarding) is defined by researchers as an emerging sub-type of hoarding disorder
characterized by individuals collecting excessive digital material which lead to those individuals experiencing stress and disorganization. :soyjakspeech:

In reality, data-hoarding is, in my own definition, the process of downloading and storing every single tidbits of internet media you like or come across, it is more of a value/principle and of a hobby. :mooman:

I didn't start getting into data-hoarding because I had some form of "compulsive disorder" but who knows. In simpler words, if you like something on any platform like TheyTube or a website that you like, series, movies, animes, imageboard threads, blogs, texts and such, you download it and store it on either a hard-drive or else (USB key storage is rare since not many have Terabytes of capacity, or are much more expensive, and are easier to lose in a corner of your room or else, also tend to dysfunction more often).

But then, it begs the following question:

Why should you do data-hoarding ? :pepechill:
"Why would I want to download EVERYTHING (or not) I like on the Internet?" :pepecringe: you might ask yourself. And that is a valid question.

Well, because of several reasons:


1. Because everything on the Internet gets deleted at some point.

That's right, contrary to what the (false) saying is "Nothing disappears from the Internet", everything does disappears at some point. For the sake of storage, or because the person(s) that uploaded a media suddenly want it deleted (which happens much, much more often than you think), either out of controversy related to it.

Or because the service that hosted it went bankrupt or faced legal issues (many file hosters recently were deleted, same for torrent websites like
rarbg that was recently shut down), or because they grew tired of it, obscure IRL reasons... there are millions of reasons really.

Or sometimes because things (such as softwares) were either only launchable or available on a very old OS/machine and are today nearly unfindable or do not work even on emulators (such is the case for the
iMac TTS, a text-to-speech software famous for its use in the anime Serial Experiment Lain [which i'm not necessarily a fan of, or its weird connection with troons]).


Literal hundreds of pentabytes (if not more) were lost with time since the beginning of the Internet. And to point out that you never know when things might get deleted, I at one point data-hoarded a TheyTube channel, and merely a week later the owner decided to delete all of the videos with no prior warnings, and I now only have 50% of the videos on hard-drive, sucks, although I probably am the only person with the videos still, which are on my hard drive.

You never know when the things you like can disappear.

Another example is the way imageboards function with threads. They have a limited number of threads active, only keeping those who are bumped/have the most interaction on, and deleting threads that went inactive for a long time, to save disk space. As such, an insane amount of threads were deleted to this day, fortunately with some being available on archive websites, torrent collections of threads, or rarely relayed with screenshots by other people somewhat randomly.


As a sidenote, Goolagis going to remove all TheyTube channels that are unactive for more than 2 years (goodbye pentabytes of Internet memes and history of Y2K) in december 2023. So another reason to start data-hoarding.

Data-hoarding is also often the only way lost media can ever get solved/found (another interesting topic you could be interested in, of which I will not talk about however, in this guide).

2. Because of censorship.

As most of you already know, platforms like TheyTube changed policies with time, progressively becoming more and more restrictive and enforcing globohomo agenda. As such, many (although not limited to) TheyTube channels were banned, videos censored (and more recently age-restricted for some), some deleted due to copyright enfringement, with no hope of ever recovering them (unless they were archive but more often than not they are not if the videos/channels don't go further than 5k or 10k subscribers), on top of the fact that TheyTubers generally do not store all of their videos on their hard-drive (to give you an example, I have some videos I recorded with OBS on my hard drive, and something as simple as gameplay of trolling in L4D2 in HD in 12 mins takes nearly 3 gigabytes, but that's just a single case and the result varies in general. Imagine the TheyTube channels that have hundreds of these videos, on top of having all TheyTube videos by default having 720p, with some having 2k, or even 4k quality available).

And anything that is remotely politically incorrect or contains swear words or "nudity" (more of an excuse since videos banned for that reason do not generally have explicit nudity despite e-whores never having any problems with their videos) gets banned too. :pepecringe:

3. Because you can watch/access the things you data-hoard whenever you want.

With data-hoarding, you can access everything you downloaded at whatever moment you want, with no bandwith, and without depending on some corporation's platform, websites and ToS to access, and (generally) no one can delete what you have on your hard drive, no matter what it is. You don't need to send information to anyone, not even your ISP if you look at the things you data-hoard (although the exception can be made that your OS technically could know with data collection, or possibly glowies since most CPUs and motherboards have backdoors inside of them :glowie:).

4. Because you can transfer the things you data-hoard on whatever devices you want.

If you downloaded some .mp4 or .mkv files (videos), you can transfer them from one disk to another, to your phone, your tablet, and share it with your friends if you want, since the things you have on your disk drive cannot be deleted (with the only exception that you still technically can lose it if your hard-drive is damaged, which i'll talk about later on).

5. Because it stands for web preservation and archiving.

If you're more of a moral/value person, it also stands for web preservation and archiving, allowing medias, history and such to be kept. Most of the internet's history/drama/else are generally written/shown by data-hoarders, which still are (to various degrees depending on the person) archivists.
??. And other hundreds of reasons.


There are many other reasons that exist for one to want to data-hoard, which either that currently do not come to my mind, or because others are unknown. The reasons I gave you are reasons that generally apply to me, and as such, there can be many other reasons for other persons.

Convinced? If so, onto the next and last section of the guide.


How to data-hoard? :hmmpepe:

"Data-hoarding is cool and all but you still didn't tell us how to do it. :soy:" Yup. And that's what i'm going to do. Keep in mind that I am nowhere a professional data-hoarder, I only started data-hoarding about 1-2 years ago which is VERY late for data-hoarding standards. What i'll give you is more of the way *I* data-hoard, and there may be many more softwares or websites I do not use or talk about, this guide is more of an introduction point do data-hoarding, a beginner's guide really (which will get updated with time).

Data-hoarding is primarily made by:
- The use of softwares, be it web crawlers, torrent softwares, downloading softwares
- Archive websites, sometimes torrent/file-sharing websites, and bunch of others.


I will now give the websites and softwares that I use to data-hoard, explain briefly how they work, with a few screenshots and small tutorials to get you started.


DISCLAIMER: I data-hoard on windows 10, so there may be small differences of performance or installation process or even software that are and can be used with Linux.

Jdownloader, or the data-hoarder's best friend:

Jdownloader is the software I use the most often. It is free, open-source, and with it you can download nearly everything you wish. It is written in Java (ew, i know). It can be used to download TheyTube links, files on file-hosting websites, it supports the use of accounts (for example for premium file hosting websites or private trackers or else), and you can tinker with the settings to make the downloads go wherever you want it to go, how the folders must be made, what quality settings you want it to use for TheyTube videos (by default it takes 720p if I remember correctly), it even downloads descriptions in .txt files, as well as subtitles in a .srt file (generally those automatically generated in the video).

All-in-all a very good data-hoarding software. It also has a (by default activated) clipboard function that grabs all the links you copy, so that it automatically puts it in the links to download (and you can turn that off if you want), it's perfect to download multiple youtube videos, you just copy the link and it grabs it easily, perfect for your liked TheyTube videos playlist (since it is by default private) where you can simply right click, copy and do that over and over (still quite a long process but much less compared to using third party websites to "convert" the video), although I never tried putting my account login in the software.

Link for Jdownloader: https://jdownloader.org/jdownloader2

Choose your OS, it will redirect you to MEGA and from here you just download and execute it.
Don't mind the very old layout and UI, it is a legit and good software.



How to use it:

Jdownloader is very easy to use and only revolves about three parts:
- Downloads section
- LinkGrabber
- Settings
.

Here is how the downloads look like (with copious amounts of sometimes unnecessary captions):


View attachment 881

View attachment 882

(Linkgrabber part above).


Then, the settings. You don't really need to tinker with it if you just want to simply download links and whatnot, but I modified one setting that allowed to name the file with the date of which
the original file (that I downloaded) was uploaded on the internet, useful to date TheyTube videos. IF you want to do that, go to the settings tab, Plugins, choose the TheyTube.com plugin, scroll down until you arrive at "filename & packagername", scroll some more and in filename for video files, make sure the following is written:
*3D* *360* *VIDEO_NAME* (*H*p_*FPS*fps_*VIDEO_CODEC*-*AUDIO_BITRATE*kbit_*AUDIO_CODEC*)*DATE_UPLOAD* *DATE*.*EXT*
and you should be good. But again that's not necessary and i've had a few problems with it, make a backup of the original line in case you have problems.

Now that Jdownloader is out of the way, let's now talk about the second thing I use the most. It technically isn't strictly made for data-hoarding.

Qbittorrent, or the free sailor's ship:

Qbittorrent of which most of you probably already know is a torrent software, it's once again free, and open-source, and without the sketchiness of the garbage-tier ex-crypto-miner malware that is utorrent. :swag:

But why would we want to use a torrent software for data-hoarding? :hmmpepe:

Well, because there are many torrents out there that are collections of different stuff, including for example things such as 20 Gb of 4chan threads between 2009-2012. If you search well you can download multiple collections of archived stuff, youtube videos, etc.

Or even simply download series, animes, and such, which TECHNICALLY still counts as data-hoarding.

Link for Qbittorrent: https://www.qbittorrent.org/download

Qbittorrent is even easier to use than Jdownloader, so I won't be providing any guide here. Just a tip however, I recommend once you downloaded Qbittorrent that you go in Tools (or just click the cogwheel, easier like that) -> Settings -> BitTorrent and tick "Enable anonymous mode".

Then, onto the third thing I use the most to data-hoard.


Archive.org, the heaven of all data-hoarders:

Archive.org (<- link), a website that devotes itself to web preservation, unfortunately facing legal problems (lawsuits) due to 'copyright issues'. Hopefully it will stick around, yet I recommend you still download as much stuff as you can there (and you can even couple it with Jdownloader). Always go from the principle that if it can be deleted, it will at some point.

You can find an INSANE amount of things there, be it books, youtube videos, even includes the
wayback machine which allows you to go to snapshots of websites (if users were kind enough to do snapshots of them), some trace back to early 2000s.

The only problem with archive.org is its difficulty to navigate and find the stuff you want to look for, the search bar isn't that great, and you'll definitely have to be patient (or lucky) to some degree.

Therefore, you also can use another alternative, which is to write in Goolag in quotation marks the things you want to find, for example "lost media" "playlist" site:archive.org. What it basically does is that it will only show the pages that contain the words in the quotation marks, and limits itself to the website archive.org. Much quicker alternative indeed.


HTTrack, the website downloading software:

Did you ever told yourself "man, i'd really like to have this whole website available at any time on my PC :apusad:" and thought it to be not possible? Sure you could archive.org, but it is clunky, long to use, and doesn't allow you to access the website offline, without third-parties and it doesn't give you the guarantee that archive.org will stay forever.

I present to you WinHTTrack, an open source offline browser (and web-crawler/downloader) for websites. It is quite old, and its last update goes back to 2017, but it still works perfectly.
:mooman:


Link for HTTrack: https://www.httrack.com/page/2/en/index.html

Both the UI from the website and the software are very old-looking, but it still works just fine and is legit. Do note that you need to make an individual folder for a website you wish to download. Since this software isn't too hard to use either, i'm not going to post a guide for the time being, it's very accessible anyway. Also, if you can't decide in action "download web site(s)" or "download web site(s) + questions" just choose the first option.

Tips for data-hoarding:

- ALWAYS backup the stuff you download on other disk drives. You never know when you can have a faulty disk drive that decides to die for some random reason. :trollface:
- IF you decide to use servers instead of disk drive storage (which also is possible in data-hoarding), you need to be prepared to handle security and put a solid password, otherwise there is a small probability that you will lose your files or possibly more. You'd be amazed at how many servers exist that are public for a decade and contains a lot of stuff everyone can download (even if it was not supposed to be so), with security issues. In my opinion disk drive storage without internet is the most secure and efficient way, but whatever floats your boat of course.
- Consider buying additional disk drives (externals are a good way to do so) if you aim to download a lot of stuff.

And that's basically it for the guide!

Don't hesitate to ask questions or even just generally talk about data-hoarding, that's why I made this thread after all, it's not just about the guide. :mooman:
 

Tedposter

Well-known member
Marianan ID
8
Joined
Apr 2, 2023
Threads
27
Messages
496
Reaction score
510
Awards
48
LV
3
 
Offline
>Asocial schizoid, data-hoarding/niche/obscure stuff fan, currently a NEET, truth-seeker, gateway experience student (i.e hemi-sync, remote viewing etc) pessimistic intellectual, minority-hater, y2k nostalgic, self-aware that no one gives a shit about any of this
 
Marianan ID
62
Joined
Jul 11, 2023
Threads
1
Messages
5
Reaction score
8
Awards
2
Location
L
LV
0
 
Offline
>Asocial schizoid, data-hoarding/niche/obscure stuff fan, currently a NEET, truth-seeker, gateway experience student (i.e hemi-sync, remote viewing etc) pessimistic intellectual, minority-hater, y2k nostalgic, self-aware that no one gives a shit about any of this
Yup.
 
Marianan ID
62
Joined
Jul 11, 2023
Threads
1
Messages
5
Reaction score
8
Awards
2
Location
L
LV
0
 
Offline

ignika98

Porn writer and wife beater
V.I.P.
Marianan ID
60
Joined
Jul 2, 2023
Threads
1
Messages
70
Reaction score
128
Awards
12
Website
bit.ly
LV
1
 
Offline
I've been thinking about doing something like this recently. Didn't know there was a term for it or any kind of community. I'll have to check it out sometime. Need to buy myself one of those multi terabyte storage hdds to back up all my shit onto though.
 
Marianan ID
62
Joined
Jul 11, 2023
Threads
1
Messages
5
Reaction score
8
Awards
2
Location
L
LV
0
 
Offline
Now this is a great thread , it gets my approval.

View attachment 903
Thanks a lot! I'm glad you like it :pepechill:

I'm going to edit it at some point to add two softwares, one of disk recovery data and one that pretty much makes it functional again (helps if there is an error where you can't reformat an USB key or else, I think those two work on phones too, i'll have to try it since I have stuff back from 2013 on broken ones, probably lost media too in there).

I've been thinking about doing something like this recently. Didn't know there was a term for it or any kind of community. I'll have to check it out sometime.
Nice! Yup there is a term for that indeed, as well as a 'community' on normieddit (even if it is very loose and people generally just come and go, asking questions and stuff then leaving), pretty dead too, w/ 692k members and only about a few dozens talk in threads.

Need to buy myself one of those multi terabyte storage hdds to back up all my shit onto though.
Yup, it's pretty much a necessity if you want to get into data-hoarding (or even just doing some backup). :mooman:

I don't have any particular recommendation on what you should choose (also depends on your budget), but personally I find that external HDDs like the Toshiba ones are decent enough, never had a problem with them yet. They're very small too and don't take too much (physical) space.

Also mathematically it seems cheaper for some reason if you go on amazon to buy several HDDs of 2 Teras rather than one of 4 Teras.

Also divides the data into several locations, decreasing the risk that you lose everything in one go. :pepechill:
 
Last edited:

Khastle

Herald of the Mariana
Janny
Marianan ID
4
Joined
Apr 2, 2023
Threads
197
Messages
2,029
Reaction score
1,091
Awards
80
Location
Dwayne's Basement, Mariana Bay
LV
4
 
Offline
So the original OP mysteriously disappeared and deleted all his stuff. He also tried nuking the thread and left the numbers "22" in its place, thankfully Xenforo archives saves previous post versions and reverting it back to normal was rather easy, this is a good asf thread and I ain't letting it go to waste. Couldn't recover the images sadly, can't have everything. @ThirdyAughtSix is it's new owner for archival purposes.
 

ThirdyAughtSix

Hells Janitor
Janny
Marianan ID
15
Joined
Apr 8, 2023
Threads
30
Messages
568
Reaction score
772
Awards
50
LV
3
 
Offline
So the original OP mysteriously disappeared and deleted all his stuff. He also tried nuking the thread and left the numbers "22" in its place, thankfully Xenforo archives saves previous post versions and reverting it back to normal was rather easy, this is a good asf thread and I ain't letting it go to waste. Couldn't recover the images sadly, can't have everything. @ThirdyAughtSix is it's new owner for archival purposes.
 

ignika98

Porn writer and wife beater
V.I.P.
Marianan ID
60
Joined
Jul 2, 2023
Threads
1
Messages
70
Reaction score
128
Awards
12
Website
bit.ly
LV
1
 
Offline
So the original OP mysteriously disappeared and deleted all his stuff. He also tried nuking the thread and left the numbers "22" in its place, thankfully Xenforo archives saves previous post versions and reverting it back to normal was rather easy, this is a good asf thread and I ain't letting it go to waste. Couldn't recover the images sadly, can't have everything. @ThirdyAughtSix is it's new owner for archival purposes.
OP really was a true schizo...
 

Punished_Senko

Well-known member
V.I.P.
Marianan ID
83
Joined
Aug 7, 2023
Threads
3
Messages
86
Reaction score
94
Awards
17
LV
1
 
Offline
Thank you so much for this thread, before i saw this i was still using a shitty youtube downloader site, and i already have like ~80GB of yt videos hoarded, one thing you should add to this thread is what hard drives are ideal for data-hoarding and what hard drives have the best $/GB ratio.

Also i would like to know if there is a offline installer for Jdownloader?
 

Khastle

Herald of the Mariana
Janny
Marianan ID
4
Joined
Apr 2, 2023
Threads
197
Messages
2,029
Reaction score
1,091
Awards
80
Location
Dwayne's Basement, Mariana Bay
LV
4
 
Offline
Thank you so much for this thread, before i saw this i was still using a shitty youtube downloader site, and i already have like ~80GB of yt videos hoarded, one thing you should add to this thread is what hard drives are ideal for data-hoarding and what hard drives have the best $/GB ratio.

Also i would like to know if there is a offline installer for Jdownloader?
Haha no worries, I'll have to look into hard drives as I'm no expert on them. Wish the original OP was still here but he disappeared into the night and tried to take the forum with it.

Not sure, I'll look into offline installers but there must be one out there.
 

Punished_Senko

Well-known member
V.I.P.
Marianan ID
83
Joined
Aug 7, 2023
Threads
3
Messages
86
Reaction score
94
Awards
17
LV
1
 
Offline
Haha no worries, I'll have to look into hard drives as I'm no expert on them. Wish the original OP was still here but he disappeared into the night and tried to take the forum with it.

Not sure, I'll look into offline installers but there must be one out there.
What happened with OP?
 

Khastle

Herald of the Mariana
Janny
Marianan ID
4
Joined
Apr 2, 2023
Threads
197
Messages
2,029
Reaction score
1,091
Awards
80
Location
Dwayne's Basement, Mariana Bay
LV
4
 
Offline
What happened with OP?
@Lokhir was a user that was very active and social on this forum for like a week until he suddenly dropped from the face of this forum and tried to delete this forum post alongside all his others. I managed to restore this thread as thankfully xenforo saves previous versions of threads but yeah his disappearance was very odd. I suspect Lokhir was either a paranoid individual or from another forum trying to mess with us, either way I'm grateful to him for this thread as its pretty good.

I gave this thread to @ThirdyAughtSix but yeah he ain't the OG OP.
 
Boatyard
Rules Help Users
  • No one is chatting at the moment.
      There are no messages in the current room.