Cheapest way to back up a *lot* of data?
from Inkstainthebat@pawb.social to selfhosted@lemmy.world on 11 Jan 12:23
https://pawb.social/post/37901756
from Inkstainthebat@pawb.social to selfhosted@lemmy.world on 11 Jan 12:23
https://pawb.social/post/37901756
Hello!! Some recent technical problems on my family’s NAS gave me a big scare and finally pushed me to figure out a way to back it all up. I’m asking here specifically because I really don’t know where to even starts because of the fact I’ve got just under 50 terabytes worth of data stored in a 7-disk RAID-5 and would prefer to keep it cheap. What are your suggestions?
Edit: thank you for all the suggestions, I’ll probably be considering using Backblaze for backups, or perhaps seeing if I can scrounge up old unused disks from people I know. Thank you all again <3
threaded - newest
How much of that 50 terabytes is media downloaded from the Internet? Because the cheapest way would be to trust that it’s already backed up on the Internet and then use one of the usual services like B2 by Backblaze or Storagebox by Hetzner to back the rest of it up.
Yeah, if 90% of that is movies/shows, then you really don’t need a backup of that as you can always re-download it. Then you have a 5TB backup problem which is much cheaper to solve.
This is a good compromise. When I was tight on backup space, I just had a “backup” script that ran nightly and wrote all the media file names to a text file and pushed that to my backup.
It would mean tons of redownloading if my storage array failed, but it was preferable to spending hundreds of dollars I didn’t have on new hardware.
If you back up the modern day Arrs databases then it’s essentially the same thing and already built into the software that will redownload them for you. That’s my solution. I backup my backups of those, of my home assistant, my Immich library, my Nextcloud, etc… Pirated media is, for the most part, out there backed up on several places already.
This is what I do - well, I back up there entire container. But functionally the same.
There’s only a few pieces of media that I have backed up manually due to their rarity, but even those I don’t really care about.
If you tested whether those files had healthy seeds on the Net, that would actually be a pretty cool idea. But I have some rather rare Linux ISOs that definitely don’t get seeded anymore so those I would want to not lose and actually backup.
That’s a shit load of downloading. LOL wow!
I have 45TB of data and the majority of that is definitely downloaded media. They call us data hoarders for a reason.
Oh sure I understand data hoarding. I was just thinking, to restore 50 tb from the internet is going to take more than a fortnight.
It’s literally downloading the same amount of data you would be backing up, and you won’t be charged hourly for downloading it from the internet as opposed to a large storage service.
Imagine having to back up 50Tb to S3 :p
Not everyone has a symmetric connection.
But… can we trust that we will have stuff available on the internet in the future?
All my TV shows and movies, I don’t bother. But my 150gb mp3 library I keep backed up because it’s much smaller and I know some of that stuff is not readily available online.
You can replicate across more than one provider and do automated regular monitoring that backups are still accessible.
If one goes down you hopefully have time to figure out a replacment before the other(s) do.
Probably not worth it for a bunch of xvid dvdrips or historical archives of full system-level backups but for critical data it’s sensible.
Exactly. Regimes want to kill this as fast as they can to milk us of every penny witghr their shitty services. I dont trust any sites will stay up.
It’s kinda complicated because a good chunk of that is data that is technically redownloadable, but has been tweaked (most of my movies are a multiplexed high-res eng version merged with audio from lower-res dub.) Either way, thank you for the suggestions
I suggest making a script that uses existing software (ie mkvtoolnix) to extract the dubbed audio and then backing that up and l leaving the high quality video to the Web to backup.
I know it’s less than ideal but you can automate both extracting it and muxing it back in. It may take some effort to setup, but it’s well worth the huge recurring costs incurred from backing up that amount of data.
Just an idea to consider.
My suggestion: Buy 3x 28TB drives. Mirror the data to them. Then move them off site.
The off-site location could either be a family member’s home where you can then sync to the drives over the internet. Or in a PO box nearby that you retrieve them from time to time to re-sync the data.
That is the cheapest option. Maybe the most convenient or most reliable option, but definitely the cheapest.
I would definitely keep them warm, as in a running machine.
Drives on a shelf die more often than always-on drives.
Really? Do you have any source on that?
If it’s true, I bet it’s only if they’re actually running without ever spinning down.
Nothng official, sorry, wish I did!
Mostly personal experience. But that experience is also shared among a group of peers and friends in the SMB space where their clients think they can keep stuff on externals in an office safe only to find they’ve gone tits up nearly every time they pull them out a couple years later. And not the enclosures, the drives themselves - they all have external drive readers for just these kinds of circumstances.
In the enterprise you’d get laughed out of a datacenter for even suggesting cold drives for anything. Of course that’s based around simple bit rot concerns, and why file systems like ZFS use a methodology to test/verify bits on a regular basis.
If nothing else, that bit rot should be enough of a reason to not store data on cold drives. It’s not what drives were designed (or tested) to do.
Edit: Everything I’ve read over the years suggests failures happen as much from things like lubricants hardening from sitting as from bit rot. I’ve experienced both. I’ve seen drives that spin up after ten years but have numerous data errors, and drives that just won’t spin up, while their counterparts that have run nearly continuously are fine (well, their bit-rot was caught by the OS and mitigated). With a running drive you have monitoring, so you know the state.
But what about Amazon Glacier? That’s exactly what they do. Cheap storage on cold drives.
Cold storage solutions would be cheapest if you don’t need to access it often, if you do then Backblaze b2.
Lastly you could do your own backup (drives sitting at a friends of family’s place?)
First: there is no cheap way to back this amount of data up. AWS Glacier would be about $200/mo, PLUS bandwidth transfer charges, which would be something like $500. R2 would be about $750/mo, no transfer charges. So assume that most companies with some sort of whacky, competing product would be billed by either of these companies with you as a consumer, and you can figure out how this is the baseline of what you’ll be getting charged from them.
50TB of what? If it’s just readily available stuff you can download again, skip backing that up. Only keep personal effects, and see how much you can reduce this number by.
Bb would be ~300$/month for 50tb
50TB on a Hetzner storage box would be $116/month, with unlimited traffic. It’d have to be split across three storage boxes though, since 20TB is the max per box. 10TB is $24/month and 20TB is $46/month.
They’re only available in Germany and Finland, but data transfer from elsewhere in the world would still be faster than AWS Glacier.
Another option with Herzner is a dedicated server. Unfortunately the max storage they let you add is 2 x 22TB SATA HDDs, which would only let you store 22TB of stuff (assuming RAID1), for over double the cost of a 20TB storage box.
You can have more than 2x 22tb, the SX line is full of storage servers!
SX65 2x 1TB nvme 4x 22tb (~104€ 0%VAT) SX135 2x 2tb nvme 8x 22tb (~204€ 0%VAT) SX295 2x 8tb nvme 14x 22tb (~384 0%VAT)
If you manually add the disks to configurations like AX-line, there is a limit, that may be bypassed if you contact hetzners support, but that’d be expensive compared to the SX line I reckon.
Also, Hetzner’s server auction has quite affordable 4x 16tb servers, starting from 63€ 0%VAT, that’d be pretty affordable RAID1- solution, ~1€ per TB/month.
Unlimited traffic on all of the above, and a sidenote: SX line servers have one-off setup fees, server auction has no setup fees.
Oops, I didn’t know about the SX line, and didn’t know they had auction servers with large amounts of disk space. Thanks!! I’m not familiar with all of Hetzner’a products.
For pure file storage (ie you’re only using SFTP, Borgbackup, restic, NFS, Samba, etc) I still think the storage boxes are a good deal, as you don’t have to worry about server maintenance (since it’s a shared environment). I’m not sure if supports encryption though, which is probably where a dedicated server would be useful.
Well, ideally you encrypt your data before transferring, so the provider never sees your data. I’m using a storagebox to backup btrfs incremental snapshots (using btrbk) and just AES encrypt them locally before sending them over, so I don’t care if the storagebox itself is encrypted.
Definitely… I use Borgbackup for my backups, which encrypts the backups before sending them to the remote server. Not all use cases can do that though, so sometimes it’s useful to have filesystem-level encryption.
Damn, I would have thought that glacier would be cheaper and they would claw your eyes out on egress and access.
The cheapest medium is tape. It is by far the lowest price per terabyte. The tradeoff is that the drives are pricey, even the used ones, and it’s a huge pain to read the data back out (but in a recovery scenario, that’s tolerable).
Backblaze personal is about the cheapest I know of: $99 per year unlimited. Caveats would be that the drives have to be physically connected to the computer doing the backup. Additionally, should you ever need to restore the backup, the best way would be to buy a 10 tb drive from Backblaze, restore the data, then send the drive back for a full refund x 5. Restoring 50 tb online would be excruciating.
I'm currently backing up my NAS to Backblaze Personal by mounting the drives using Dokany. They appear as local drives and the Backblaze client accepts them for backup.
That’s worth a bookmark. Thanks for the share.
But isn’t it available only for Windows/macOS?
Yes, however you could run it in Wine, or create a Windows VM.
Yeah, on Linux it is better to use their B2 service
Gee, that’s a lot of homework.
3 Ironwolf Pros for £1,200, before the AI bs ruins their prices too
Sort your data into stuff you absolutely need to keep (personal files and such) and stuff you'd be okay with losing (less important files, device backups, downloads you can redownload, etc). Then only back up the former. As for backup medium, ServerPartDeals often has some pretty good deals on storage; they were selling refurbished 12TB drives for $80 a pop a while back.
They haven’t been selling anything that cheap since the AI driven hard drive shortage. A refurbished 12TB drive is around $200 now.
In the backup world, 50TB isn’t really a lot, and you’re not really ready to talk about tape systems or maintaining an always-on disk system. Also, HDD’s have been getting more expensive due to AI idiocy. But, cheapest is probably a second raid system, like 4x 20tb drives. Do the backup at home and then move the backup system off site and either keep it spinning, or make sure to spin up and test the individual drives every so often.
4-bay DAS with a handful of big HDDs in RAIDZ1. Load it up, then store it in your office at work or at a friend or family member’s house. Retrieve, update, and scrub somewhere between once every few weeks to once every few months, depending on how often your critical data is changing.
Is it data you would trust in the hands of random strangers on the internet? If so, I can easily store 50TB for you, as long as it’s temporary.
Oh, and I have various storage solutions in various jurisdictions, so if you have any preferences as to places you do NOT want to store it, that’s something you need to hilight.
… this is odly nice of you. Sorry, but are you really willing to trust a random internet stranger to store up to 50TB of data on your machines? The data could be literally anything and might put you into trouble 😵💫
No offense, just curiosity
what’s the difference to storing it on a company’s platform?
sign a thing that says there’s nothing illegal in it. done.
Also, encrypt before sending. Plausible deniability is a real thing.
Yeah I’m not OP (and I’m not offering this service to the public) but anything I’m ever hosting for anyone else should be zero-knowledge encrypted. I don’t want to be able to know what’s in other people’s data. THAT makes me uncomfortable. As long as it’s encrypted, and I don’t have the key, it’s just random bits as far as I’m concerned.
I don’t care what you use your random bits for. None of my business.
Yup! Based.
If it’s encrypted how should I know?
Tape is still the cheapest and best archival medium. Drives are expensive, but the actual tape is cheap. But 50TB might not be enough to justify.
I’m heavily researching tape for my data, I currently sit around 400TBs Total but only around 200 in data that id actually want to backup and can’t just redownload.
Iirc the break even point 100-150TB
ETA: that break even point might actually be lower now that I think about it since that number is probably outdated when I did it and doesn’t account for the shortage crap
I just came here to say exactly the same thing. Tapes for the long term, but you also have to take reaaaaaly good care of how they’re stored ie. don’t store them under the kitchen sink in your bathroom
My kitchen sink in the kitchen is fine though? What about the bathroom sink?
/s
The tape drive costs more or less 2000€ (without VAT), the tapes cost about 80-100 for a 15tb drive (and compressed capacity doesnt count as the to be backed up data is probably not just a database or text.
Don’t think there are much economic options beside finding the cheapest S3 storage or a secondary backup server.
<img alt="If blank tapes aren’t cheap enough…" src="https://lemmy.world/pictrs/image/803f5155-fcaf-485b-b7f5-51c0da8927c6.gif">
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
5 acronyms in this thread; the most compressed thread commented on today has 12 acronyms.
[Thread #995 for this comm, first seen 12th Jan 2026, 03:05] [FAQ] [Full list] [Contact] [Source code]
Nobody said this here but storj is one of the cheapest storage out there and it’s has redundancy and is distributed. And if you have truenas, it’s kinda baked right in. They had some hiccups when communicating changes to the community, but overall nice folks.
$15/TB (I’m assuming USD) is incredible!
I am actualy paying about 5 bucks per terabyte, but I think I am still on old tarif… Oh wait they still offer it, its now called active archive and its 6 dollars per terabyte.
First thing is decide what data you have that can be readily replaced. e.g. publicly available Linux ISOs. Then remove that from your backup strategy. You may end up with a lot less data needing backup.
Exactly, you can just redownload them later. But pay attention if you have rare Linux ISOs, maybe because of their quality or if they’re in a specific language.
diskprices.com
is the best resource for finding cheap personal digital storage, last i looked into it not sure how up-to-date that site is kept though
Slightly provocative take:
Let it go, it doesn’t matter. The desire to hoard data is, like hoarding money, understandable but unnecessary. What do you do with all this media. Your time on this planet is finite.
so trash all family memories. ok.
50 terabyte of family memories? Maybe trash 49 then. :)
Tape Drive?
Install W11… your data will be backed up…just not the way you want it.