ISO Opensource, Selfhosted, Web Trends Monitor
from irmadlad@lemmy.world to selfhosted@lemmy.world on 12 Dec 15:57
https://lemmy.world/post/40115593

Looking for a self hosted, web search trends monitor. I have looked at Plausible Analytics, OpenSearch, Matomo, and some other website analytics platforms, but I’m not necessarily wanting to monitor a specific website(s). Rather, I want to monitor what people are searching for on the internet.

Is such a thing possible?

#selfhosted

threaded - newest

frongt@lemmy.zip on 12 Dec 16:28 next collapse

How do you plan to get this data? Most search companies don’t share it.

irmadlad@lemmy.world on 12 Dec 16:38 collapse

Well, I’ve seen datasets from places like Common Crawl, Web Data Commons, Yahoo’s Webscope which can be integrated into something like MeiliSearch. I’m going in kind of blind on this, and I don’t know if it can be pulled off with the datasets that are publicly available. It’s just something that has captured my imagination, and so I am on a fishing trip.

otter@lemmy.ca on 12 Dec 16:30 next collapse

Your account is marked as a bot, you can change that toggle in your account settings

irmadlad@lemmy.world on 12 Dec 16:34 collapse

Weird. How about now? Thanks for the heads up.

maaaaaaaaat@jlai.lu on 12 Dec 17:39 collapse

Still marked as bot for me actually

irmadlad@lemmy.world on 13 Dec 02:25 collapse

<img alt="" src="https://lemmy.world/pictrs/image/0a1b53b7-25be-4a96-93c8-60f775934d13.jpeg">

How’s that?

testman@lemmy.ml on 13 Dec 03:56 collapse

Apparently not good enough lol. Still marked as bot.

irmadlad@lemmy.world on 13 Dec 09:00 collapse

Dang it. I unticked the bot box, saved, still no joy.

maaaaaaaaat@jlai.lu on 13 Dec 17:23 collapse

Maybe you should ask to admin of your instances

irmadlad@lemmy.world on 14 Dec 07:03 collapse

It’s worth a shot but they’re usually rather quiet.

GottaHaveFaith@fedia.io on 13 Dec 05:40 next collapse

That's something worth money so I don't think you're going to find anything good

JadedBlueEyes@programming.dev on 14 Dec 04:31 collapse

I’m pretty much sure only free option for finding out what other people search for is Google Trends. It’s very valuable data that is hard to get, so the companies that offer it charge quite a lot for it.