from diegantobass@lemmy.world to selfhosted@lemmy.world on 14 Jun 12:50
https://lemmy.world/post/31379291
Self-hosting services has been a life-changer. And I thank this community for helping me a lot recently. Not only did I learn a lot more about linux, network and docker, but it helped me understand better how platforms and advertising just f*cked up the internet I grew up with.
But I wonder: do any of you hate how self-hosting services like photo- or document-management systems, or even a simple rss tool, forces you to sort your stuff out, and put your decades old files in order?!
I’m in the process of migrating my web browser bookmarks to linkding because it’s a GREAT tool. But I have like 2k websites to manualy check wether they’re still there, wonder at how cool they still are, tag properly and archive with SingleFile!
And that’s just ONE service…
threaded - newest
I think that’s a feature 😅😅
Just kidding, I had no issue with my photos for example with Photoprism, but for streaming my music with gonic I need to make some modifications for all my album art to show up, and in some cases titles and album names…
It is indeed a great feature but how time-consuming haha :)
I inherited my music collection from my 20 years old limewire addicted self, so it’s a complete mess. I’m in the process of completing albums and using Picard to properly tag everything… 20 years of music collection…it’ll take me 20 more years!
Anyway, I guess it’s a warning for anyone starting to accumulate data: think about metadata, formats and data-management. NOW!
I think life is easier if you stop managing metadata and instead deal in folder structure. My music has never had consistent metadata and tagging, yet it’s never been a problem. I use Gonic and just browse my music by folder structure.
These days I try to do both, but recognize it’s an on-going thing that will never be done.
Sometimes folder structure can be a challenge because of extensive metadata. Where do parts of a compilation go, for example. At least with metadata, music players can show the tracks correctly.
The main mess is genres. That is non existent in the folder structure (I do have it: artist > albums), and still very hard with Picard etc. But genres is very convenient for Autoplay, auto generate shuffle playlists
I went through the exact same situation with my 20 year old music collection and MusicBrainz Picard and unfortunately this is just the state of music piracy even now as things are rarely tagged correctly or come from compilation albums, sample albums, or are just obscure and don’t have the proper tags available on the service. This is why I don’t mind paying for music streaming even though I selfhost most other media formats because you can put in days of work tagging songs and still have a jumbled mess at the end (Picard’s tagging template is also a huge PITA to use with some obscure legacy language).
I had some issues with my old kazaa era mp3s and figured it was sometimes easier to just download them again using soulseek…Now I have better formats and quality along with tagged files. Then I use navidrome to serve my music
Same boat. I put everything in Picard and let it analyze everything. It turned out about 95% perfect. Haven’t touched it since, and I’m using the metadata it generated.
Isn’t that the goal? If you have an old drawer full of unorganized stuff, implementing a selfhosted management tool is getting an organizer and thinking about how to fill it, but you still have to sort your stuff in.
The only selfhosted thing where I really have to re-organize is my documents in paperless but I’m so glad to finally have it all organized and searchable instead of some hot mess of an inconsistent folder structure.
I’m in the long process of paperlessing. It’s THE perfect example of that (not so) hidden cost. But there’s no lying or trying to sell you magic. You put effort in a systematization that empowered by a great tool and a well thought out and tried model, and voila, winning.
Yeah I think of it the other way round: I couldn’t get myself to organize them without combining it with a nice selfhosted tool. The goal is getting my stuff organized, the cost is doing work, which includes setting up a system. I can cheat on the cost a little by including a fun project in the cost part.
I do think there’s a hidden cost in selfhosting though and it’s maintenance. Fortunately, there’s selfhosted tools that help with that too :-)
Thinking back on your rhetorical question, I think it’s just it.
It’s the goal. The goal was always to try and make me think that I am not just simply taking care of my stuff (and by extension myself). Because taking care (of yourself) isn’t valorized in a capitalist society.
Fuck it all. I’m putting YEARS of work into just sorting myself out.
It's important to use services with a workflow that works for you; not every popular service is going to be a good fit for everyone. Find your balance between exhaustive categorization and meaningless pile of data, and make sure you're getting more out than you're putting in. If you do decide that an extensive amount of effort is worth it, make sure that the service in question is able to export your data in a data-rich format so that you won't have to do it all again if you decide to move to a different tool.
Thanks for mentioning SingleFile. I’m not using it right now.
The workflow with linkding and the linkding injector is gold.
If you really access them that infrequently, are they actually worth keeping?
Plus if they’re links, how many still work?
Actualy a lot more than I thought, and they bring joy, as souvenirs :)
There is a hidden cost to every hobby and everybody is willing to tolerate a certain degree of shittyness.
I have a friends that has a rather old car and something on it is always broken. But he has no problem having 20 different apps for appliances, instead of deploying home assistants. Or having ads everywhere and even trying pihole or at least NextDNS.
On the other hand, I see my car as a transportation tool and when I need it I want to use it without worrying about some random part exploding. But I have no problem running Proxmox and hosting tons of services for my family.
That said, I would definitely not self-host something like NextCloud or any business critical component for my business and just paid somebody for the service.
You understand the value of risk management.
I do both - older vehicles always needing attention, and self-hosting shit
Hate? Digital decluttering feels really good, for me anyway.
Pain feels good. It’s like sport, is it? Is it sport? I’m healthy.
You could use an llm with an mcp to the local filesystem and hope it can do it for you
Or I could not. Ever.
I know there is all of that AI hate, which i’m all for. But taking models to run locally does not benefit the AI companies. If anything this is the way to make something that is actually good out of that hot mess.
You’re right, but I’d need a graphic card < money.tar.gzip
Yeah, personally I just looked for second hand high vram gpus and waited. I got 2 titan Xp (12gb vram) for only $180 each.
I used phi3:mini-4k for tagging all my bookmarks and don’t think it was any worse than a big model for that kind of job. It will run on a 10 year old cpu and a few gb of ram. (note: ai tagging of bookmarks isn’t that great, regardless, but it helps with search).
I didn’t move shit haha. Dumped OneDrive onto the Nas and mounted it for next cloud, I didn’t even clean out the photos, which I copied into immich. I did move some ebooks, but that was very few things that I have
Make sure you check karakeep.app, because it has, at least, automatic tagging and full text search on the bookmarks
Simplify as much as you can.
And remember, if you’re also self-hosting for family, someone will need to take over all that software and digital clutter when you’re gone.
I’ve been trimming as much as I can on my NAS, including only keeping the most important self-hosted software and heavily purging old files and backups.
This. I’m not that old yet, but the realization hit me in the face pretty hard. And all the more reasons to sort it out. And definitely simplify. Or “make it usable” let’s say.
You don’t even have to be old. Death or serious illness/injury can affect us at any age, and it would suck if your family lost access to all the self-hosted photos and videos, for example.
“Make it usable” is a great idea.
Scaryyyy !
I just very recently discovered that bitwarden (vaultwarden) has this perfect feature like a “trusted contact” (not sure) where you can choose a person that can request access to your password vault, and if you DON’T answer in X days (configurable), they get access.
And you can put a secure note in there that has all the instructions necessary for them to access anything they might need (either by taking that note to someone skilled enough to follow the instructions, or by making it dead simple enough for them to just extract everything to an empty external ntfs hard drive in a simple file hierarchy).
Idk. My folders are always decently organized since I’ve been nutty about since I was a kid, but the specific file structures different services can demand is a headache. This is why I prefer more simplistic services without a database, but there’s always trade-offs to be had with both options.
I’m a bit split on it, but I do agree that it can be annoying and when you mess up, services and links you’ve sent to other people don’t work and it can be quite agonizing. It’ll probably get better for me as time goes on, but man it can bite at times.
Karakeep. It will throw an error if a website is down and you won’t get tags.
Have you considered the possibility that, if you have 2k bookmarks, this isn’t necessarily a self-hosting issue, but rather a bookmark hoarding issue :)
That’s why I don’t go back and reorganize old bookmarks. I just start fresh every time.
This is what I do. I keep the old ones around for a while, and every time I realize that I’m not missing anything, and delete them.
Worst case, I’ll have to root around in my backups. But it has never happened wrt browser bookmarks.
I don’t think I’ve even used bookmarks since Vista.
I know, right!? Do I have to let go? Yes! Am I defined as a person by the shit I accumulate? No!
Sometimes hitting delete is the best thing you can do. Especially bookmarks, how many of them is out of date, or not relevant to you any more. And if you needed some of it, you can find it again. Sure, there is a few things a bit harder to find, but it should take less time than sort through 2k bookmarks. 😀
Can’t you use a script for that? There is a method to bulk import into linkding from Firefox and a REST API for linkding that allows you to remove all expired links.
Yes and no. A lot of sorting and optimizing processes can be done via scripts. For example, I had chatgpt generate one that finds audio streams in videos that are not in the language I need. Manual verification and then let another script remove the remaining lists streams that I don’t need.
Yeah, I don’t bother sorting and organizing old files/bookmarks/whatever. Automatic tagging and full-text search solve that need. I try to keep recent stuff organized nicely though.
2k bookmarks? i would just automate the process of saving each of them locally and just forget it lol. if it’s somehow needed later search on the older archive
Actually, that is a thing I like. Going through this stuff can be tedious, but it brings a lots of memories, things that I forgot about, things I once wanted to do. And also, after cleaning my digital life I feel similar as after cleaning in the physical world - good - I did something, I made my world a tad bit more organized and a tad less overwhelming. (I should note that I am lazy and I always must force myself to clean, but I never regret doing that after I start 😀) P.S. as I wrote in one comment below, maybe bookmarks is not a necessarily a thing that you want to go through and sort. Here I am more writing about my notes, or photos, etc…
Definitely second your feeling. I am similar in my relationship to cleaning. It feels like a lot of effort, but efforts feel good afterwards.
Are you kidding me? True, there is time involved. My biggest ‘sin’ right now is “home gallery” for it works on MY directory structure which I won’t give up.
The geoguessing game that hides in it is superb ! I’m still amazed with the images I’ve been able to locate. Sometimes 40 years back.
Nothing better than a properly formatted data file.
Self hosting teaches you this
I guess the trick is to not look for stuff to host because you’ll end up with all kinds of things you weren’t doing in the first place.
And so little time!
<img alt="" src="https://lemmy.world/pictrs/image/978ff9e3-6f31-4772-a2c7-be1eeb425b41.png">
Just 2k in bookmarks? Pffft! Those are rookie numbers. Check back when you have 59k bookmarks. Currently there are 1.1k in the broken links category. The vast majority of the links are topics I research or have interest in, exterior of self-hosting. I do not consume TV data, but I do a ton of reading. I find that reading gives me better retention of the topic, and it’s rather easy to highlight & search for cross comparisons, and further research. Ever since I was a wee lad, barely able to read, I have had an insatiable lust for knowing. It is this that drives the link counts. LOL
Manual web crawler at that point
LOL Never thought of it like that, but yeah.
When I need something I’ll ask you instead of google
LOL
That’s the neat part…
This is so foreign to me. I never bookmark anything ever. I leave a few tabs open until I complete that task, read that article or decide I don’t care anymore.
I digitally collect odd things, selfhosted in several apps depending on if it’s for ‘read later and decide’ or preserve… For one, I like the etymology of words or phrases and how they’ve evolved in meaning, and in some instances bastardized the meaning. For another, I collect political cartoons from any country. I am fascinated how some of the ones I’ve read about, have changed some people’s minds. Things I find educational. Things that are totally polar opposite me. You’d be surprised what you learn even tho you may still remain opposed. So these are a back up of a backup which gets backed up, lol, It’s the source files if you will, and I archive them in another app however I still keep the source as a backstop.
I’ll end with this as an example since this might be misconstrued as not about selfhosting, As a wee lad, someone donated a set of Encyclopedia Britannica to us. I read those cover to cover many times. So, with the help of self hosting and dedicated devs around the globe, thank you so very much for being so generous with your skills and time, I can continue my quest to know.
TL:DR: I’m just a weird, old man.
I’m the same! I just don’t keep track of where all the useless knowledge in my head comes from.
When you’re 70, you need all the help in the remembering dept you can get.
I found that going back to bookmarking (and subscribing to RSS) is the best way to pull away from the algorithm-feed-trough of the social media websites and SEO bullshit. As I got more and more bookmarks of interesting sites, and found lots of feeds to subscribe too, I found I naturally gravitated away from the corporate web. It’s a requirement now if you are interested at all in indie-web type stuff, forums for esoteric hobbies or software communities, or personal web pages of interesting people -those things just don’t show up on search engines or social media anymore.
Lemmy does a good enough job of bringing content to me. But I appreciate your perspective. It’s definitely something to keep in mind as we get closer to the AI apocalypse.
What is this “sort” thing you speak of? I don’t sort anything, I have NextCloud syncing my entire photos, videos and documents folders and they are just as messy as ever. Granted, I do go through my photos and videos once a year and dump them in a folder named for the year they were taken. Occasionally, I’ll go hog wild and try to sort some of a year’s photos/videos into folders named after events. Though, that hasn’t happened in a number of years. I setup NextCloud so I could have everything synced to my own server and just forget, not have to deal with labeling my data.
As for bookmarks. I already keep those in folders; but, I don’t sync those. I use my desktop far more than I use my phone for web browsing. And the types of things I use my phone for (mostly recipes), I just keep bookmarked there.
It looks like we found another person that’s immune! Sample their blood
I’m frustrated that Immich doesn’t have a “back up new photos only” option.
All the photos on my phone are already in a huge external library with my backups from previous phones. I don’t want to delete them from my phone just so immich doesn’t freak out, and I don’t wanna have them on my NAS twice
Immich seems great, but this seems like the bread and butter migration path that nearly everyone would take.
When I got a new phone, immich was weird about the photos that came over when I transferred my phone data. After I confirmed they were all backed up (in like 3 places) I just nuked the copies pictures and videos from my phone.
I don’t want to nuke them from my phone though. I want them on my phone, and I want them on my NAS 1 time.
Content management makes up very little of what I self host.
That stuff’s just curated and is always been curated as I take new things that I need I curate them.
Then there’s another class of data I deal with which is synchronization. Synchronization as the wheat and the chaff in it and if any of it goes away I don’t really care because anything that was really worth keeping already got curated.
Yes. Some services are not good at accepting existing naming conventions, insist on their own naming and sorting conventions, or require a lot of third party services that will unfortunately rename your movie to something foreign or otherwise completely wrong.
It takes quite a bit of time to clean up titles and metadata that you may be migrating or adding from a personal collection. Sure, it’s free…but it doesn’t mean it isn’t frustrating or time consuming.
I hate having to run my own backups. That’s been a massively hidden cost behind self hosting that I did not originally account for. Anything sufficiently robust is expensive and anything cheap is unreliable (at least at the scales of data I have, 4k+ RAW videos and photos are massive).
Does it still count as “self hosting” if one of your backups uses something like restic to push to b2 or hetzner storage boxes? It’s not consumer point and click.
I have one copy going there, and one going to a $50 thinkstation usff connected to a single external hard drive. It’s not raid, but if it dies, it just gets quickly replaced while I rely on the hosted backup.
There’s also the slightly-less-hidden cost.
Electricity to run your home server(s).
I just moved 20k bookmarks from Pocket to Readeck, and can sympathize lol. A lot of the links are dead. I found a cleanup script I’m going to run but it’s still a huge curation challenge
I thought the hidden cost is my power bill by having a PC run 24/7…
I condensed down from a power hungry tower server to a couple of thinkstations and a nas. Much nicer on the power.
You should see if you can get all of your services to work on a Raspberry Pi. They hardly use any power.
Some things would, but not everything, so at best case, I would need to run 2 computers instead of one. I have a beefy spare M1 MacBook Pro, uses almost the same as the Raspberry but it’s horrible for selfhosting.
There are web extension that check for bookmakers that responds with 404 and automatically deletes them.
2000 Websites? WTF why?
IMO that’s not a hidden cost. That’s a decision you made. The actual hidden cost is the electricity and time you spend by configuring and mantaining your services.