Introducing reitti: a selfhosted alternative to Google Timeline (github.com)
from danielgraf@discuss.tchncs.de to selfhosted@lemmy.world on 02 Jul 21:40
https://discuss.tchncs.de/post/39888199

Hey everyone!

I’m excited to introduce Reitti, a location tracking and analysis application designed to help you gain insights about your movement patterns and significant places—all while keeping your data private on your own server.

Core Capabilities:

Photo Integration:

Data Import Options:

Customization:

Self-hosting:

Reitti is still early in development but has already developed extensive capabilities. I’d love to hear your feedback and answer any questions to tailor Reitti to meet the community’s needs.

Hope this sparks some interest!

Daniel

#selfhosted

threaded - newest

otter@lemmy.ca on 02 Jul 21:49 next collapse

Cool!

I love the UI for this one, it’s unique compared to the others

danielgraf@discuss.tchncs.de on 02 Jul 22:40 collapse

Thanks otter, I tried to have a historical look. Like going to the past and revive some memories. Hope it worked out. I am actually pretty happy with it.

Lost_My_Mind@lemmy.world on 02 Jul 21:57 next collapse

Man. I have no use for this. I know where I go. I go to work. And then the gym. Almost every day. Because I work a lot.

It’s like my boss always says…“BACK TO WORK, DICKHEAD!”

Zwuzelmaus@feddit.org on 02 Jul 22:00 next collapse

I still have a phone containing Google, and cannot change this situation (maybe with my next phone). Usually, I switch off location services very often and I avoid such tracking apps because all my data goes to Google then.

Would it be advisable to use this at all before I get rid of Google?

danielgraf@discuss.tchncs.de on 02 Jul 22:13 collapse

Hello @Zwuzelmaus@feddit.org, i usually use GPSLogger for Android to track my location during the day and this periodically sends the data to reitti whenever i am back at home. I have no idea if you switch off location services what happens on the GPSLogger side of the chain. If it still be able to access GPS I see no problem, if not than this sadly will break the usecase for reitti.

It relies on a consistent GPS tracking data to be able to do its thing

Zwuzelmaus@feddit.org on 03 Jul 00:22 collapse

if you switch off location services

…then no app gets a location anymore.

Sibbo@sopuli.xyz on 02 Jul 22:02 next collapse

This looks amazing, and I will try it out once it has reasonable NixOS support.

I especially like the immich integration.

Does it increase battery consumption of the phone a lot?

danielgraf@discuss.tchncs.de on 02 Jul 22:16 next collapse

Reitti on its own does not run on your phone. It relies on external apps to track your GPS location like GPSLogger or Owntracks. They will increase the battery consumption but I think it is bearable. I personally have setup GPSLogger to fetch every 30 seconds a GPS location and then sends it every hour to reitti. With that i do not see that more power usage. Currently starting to test how the power drain is with a 15 seconds interval.

smiletolerantly@awful.systems on 02 Jul 22:18 next collapse

Better open a package request (or pull request :D) then 😄

beerclue@lemmy.world on 02 Jul 22:53 collapse

May I ask what you mean by NixOS support? There’s a docker compose you could use in their repo…

Sibbo@sopuli.xyz on 03 Jul 01:01 collapse

But then I’d have to combine that with existing services myself. NixOS ensures that there is only one postgres running for example, such that not every service starts its own postgres instance.

For immich, for example the following is enough for a most basic configuration:

services.immich.enable = true;
services.immich.port = 2283;
beerclue@lemmy.world on 03 Jul 02:20 collapse

I see what you mean, interesting. Didn’t really look at NixOS as a server os. I personally prefer using multiple compose files (in the process of migrating to k8s). I share resources too, like in your example, I just point to the existing DB instance, not create a new one for each new service.

MrQuallzin@lemmy.world on 02 Jul 22:13 next collapse

This is really awesome! I was just about to start looking for something like this, so great timing. Going to get this up on my Unraid server tomorrow and play around with it

Atropos@lemmy.world on 02 Jul 23:00 next collapse

This is really neat, I’m going to spin it up tomorrow!

danielgraf@discuss.tchncs.de on 02 Jul 23:08 collapse

Let me know how it worked out. I am deploying it to my server via docker without an problems, but maybe they are some quirks i missed. :D

Atropos@lemmy.world on 02 Jul 23:24 collapse

Of course, happy to do so. I’ll report back with my experience when I have it working!

Zagorath@aussie.zone on 02 Jul 23:22 next collapse

Fuck yeah this is awesome! The detail of Immich integration is just the icing on top of an awesome cake!

How demanding is it on server resources? Am I likely to be able to run it on an old Raspberry Pi that’s also running a couple of other relatively light tasks? How much storage does it end up using over time? I’m probably going to try and get it running either on my Pi or my Synology NAS, though the latter has had issues with Docker containers in the past depending on the container’s dependencies…

danielgraf@discuss.tchncs.de on 02 Jul 23:47 next collapse

I have no clue if a raspberry will handle it. There a a couple of services involved to make it fast, but they are then another burden like RabbitMQ. Which make ingesting data instantaneous but you need extra processing power to handle the queues. It all comes with a tradeoff.

For size, there is mainly the PostGIS DB. I just checked and my db is around 800 MB for roughtly 8 1/2 Years of data.

Photon (the reverse geocode enabled in the compose file) is another beast. For Germany it takes 14 GB of storage while running, if you let PARALLELL updates enabled you can double that every time the index is updated. But you can remove that from the compose file and rely on external Geocoders. It is described in github.com/dedicatedcode/reitti?tab=readme-ov-fil…

Zagorath@aussie.zone on 03 Jul 02:05 collapse

Oh interesting. I’ve just read through that link, and I was assuming that something similar to the “external only” option would have been the only way it worked. More specifically, I thought it’d just store a list of historical points and display those on an OSM overlay. But it seems like even “external only” is much more involved than that.

What happens with self-hosted Photon if you specify a country, but then also visit another country? (I assume in hybrid mode it’s as simple as "use Photon in your country, use Nominatim otherwise?)

But yeah, definitely sounds like a Pi is probably not gonna cut it. I’ll have to see if my Synology can do it, or if the weird OS restrictions Synology imposes prevent it.

danielgraf@discuss.tchncs.de on 03 Jul 03:06 collapse

If you use Photon and only have your main country available, it will fallback to the configured external Geo-coding-services since Photon will not return a result then. So the order of execution is:

  • first try Photon
  • if it does not return anything, try to call one or all of the available Geo coding services.
Zagorath@aussie.zone on 03 Jul 04:47 collapse

Is that true even if you’re not in hybrid mode?

danielgraf@discuss.tchncs.de on 03 Jul 05:01 collapse

Maybe the wording is confusing in the Readme. Reitti will try to fetch the data from a configured photon instance first, if this does not return anything and you have Geocoding services configured, it will try them. There is actually no switch for hybrid mode or only local. It depends on what is configured.

Photon Only: you have only photon configured and under Settings > Gecoding you deleted or disable every available service. Hybrid Mode: Photon is configured and under Settings > Geocoding there are Services available. That es is the one I use. Having Photon with the data for Germany and all the rest is handled by Geoapify.com.

External Only: You dropped Photon from the docker-compose file and only rely on services under Settings > Geocoding

If you do not configure anything, then Reitti will skip Geocoding and only display Unknown Place.

I will update the Readme to make that clear.

Zagorath@aussie.zone on 03 Jul 05:33 collapse

If you do not configure anything, then Reitti will skip Geocoding and only display Unknown Place.

Ah ok thanks. This is what I was wondering.

Two follow-ups:

Can you specify multiple COUNTRY_CODEs? (and if so, is the method

environment:
  - COUNTRY_CODE=country_one
  - COUNTRY_CODE=country_two

or

environment:
  - COUNTRY_CODE=[country_one, country_two]

or something else?)

And is this something that can seemlessly be retroactively changed? For example, if I set COUNTRY_CODE=au and it works fine for Australia, but then I move to NZ, can I add (assuming the answer to my first question is yes) or change to COUNTRY_CODE=nz and have all the NZ locations work on the already-recorded data, even if I made that change to my configuration after I had been in NZ for a few months?

danielgraf@discuss.tchncs.de on 03 Jul 05:38 collapse

Good question, afaik you can not enter multiple countries to Photon. I was hoping it would be possible but everything i saw was it is either one country or the whole world. But maybe you can have a look here: github.com/komoot/photon That is the service we are using.

Zagorath@aussie.zone on 03 Jul 08:09 collapse

I just took a really quick look at it, but under Importing data from Nominatim it says “-country-codes allows to filter the data to be imported by country. Set this to a comma-separated list of two-letter language codes.”

That’s a different section from the Importing data from a JSON dump section, which is where it only mentions -country-code. But even that does seem to suggest it takes “all the parameters of an import from a Nominatim database”. So it seems like either the documentation for one of them is wrong, or both are lacking (because in fact both the singular and plural work).

danielgraf@discuss.tchncs.de on 03 Jul 08:43 next collapse

I think this is not exposed when running the Docker container. But let me check later when i have time what happens if i put another country in that variable

danielgraf@discuss.tchncs.de on 03 Jul 09:32 collapse

I looked at the docker image i am using in the docker-compose file and this only supports having a single country code. The actual reason can be found here: github.com/rtuszik/…/start-photon.sh#L341C5-L342C…

It is probably possible if you deploy photon on its own and then import the data somehow. But that is to much hassle for me, i think and hope that most of the use case is handled by the current solution. At least for most of the potential users. But I get the point if someone is traveling a lot between countries.

If there is enough demand I could maybe try to create a PR for the Docker image to handle multiple country codes.

acchariya@lemmy.world on 02 Jul 23:52 collapse

Looks to be a java application with a number of services running alongside- I don’t think it’s going to be lightweight to run on resource constrained devices, but sweet project anyway! (Side note, no clue how you engineers find the time to hack on things like this, I feel like I’ve got so little time to myself I cannot imagine dedicating it to a project like this)

BlueEther@no.lastname.nz on 02 Jul 23:33 next collapse

This looks good, I’m currently using dawarich and owntracks to track our families 5 week road trip around the south island of New Zealand (no.lastname.nz/post/1468113). If I can find a spare moment in our travels and a good internet connection I’ll try and spin up an image and test it out.

One thing that I would love is to be able to have multiple people displayed on a single map (my wife [kids when they have their own phones with data], brother, mother and a few close friends - we already use google maps for this)

danielgraf@discuss.tchncs.de on 02 Jul 23:51 collapse

That sounds like an awesome idea, actually the data is stored for every user separately depending on the used api token or when you login and do a data import. I will create a feature request for it.

But bear in mind, reitti is not meant to be a real-time tracking app. At the moment, data is processed every 10 minutes.

BlueEther@no.lastname.nz on 03 Jul 01:27 collapse

Real time is nice, but not a must have at least for my use case.

corsicanguppy@lemmy.ca on 02 Jul 23:56 next collapse

Docker-dependent? It looks fantastic, but I have no containers in my home-lab – and it’s based on my time managing OS security for an OS. I’m stuck living vicariously through the rest of you, so report back often.

pemptago@lemmy.ml on 03 Jul 00:08 next collapse

This looks awesome! I’d just been collect my GPS data thinking one day I’d analyze/visualize it on my own-- not expecting anyone else to make a such a comprehensive program. The UI looks really slick. I’m looking forward to testing it out. Hopefully this weekend. Thank you for this!

danielgraf@discuss.tchncs.de on 03 Jul 00:22 collapse

Let me know how it works out for you. If you have the gpx files, you can simply import them inside the settings menu.

scarilog@lemmy.world on 03 Jul 00:21 next collapse

My only concern is battery usage. Google has the advantage of OS integration, which skims location data for timeline history even when another app accesses location, which uses essentially no battery (since you would’ve been using location for that other app anyway).

But it’s awesome that a tool like this exists anyways, great work.

danielgraf@discuss.tchncs.de on 03 Jul 01:35 collapse

Thank you :)

I understand your concerns, this is something every additional app would have to deal with.

For me it is ok to have GPSLogger running all the time, I think for what it is doing it is quite easy on the battery but I do not use my phone actively that much and I am happy if it survives a day which it does.

illusionist@lemmy.zip on 03 Jul 07:15 collapse

I log my position every 5 seconds for years and gpslogger is not among any of the battery consuming processes if you use the device. If you don’t use the phone at all, it will consume more power but that’s no reason not to use it.

dan@upvote.au on 03 Jul 00:41 next collapse

I’d love to see an integration with PhotoStructure in addition to Immich.

danielgraf@discuss.tchncs.de on 03 Jul 01:18 collapse

Never heard of PhotoStructure but if they provide an API where i can search assets for a day and it also returns the exif data especially latitutde and longitude it should be pretty straight forward to implement. Feel free to add a feature request when you got time and I will have a look at it :)

nieceandtows@programming.dev on 03 Jul 00:43 next collapse

I’ve been putting off setting up an immich server. Would this do well if hosted on the same machine?

Zagorath@aussie.zone on 03 Jul 02:09 next collapse

I love that it supports multiple formats for important location as well as multiple geocoders. But that makes me wonder, would it be feasible to support multiple image libraries? There’s a bunch of different FOSS photo libraries out there. I think Nextcloud is the main other one I’ve heard about ‘in the wild’, as it were. Or is there too much bespoke Immich code in there for that to be a simple plug-and-play option?

danielgraf@discuss.tchncs.de on 03 Jul 02:56 collapse

no, that would not be a problem as soon as the other image library has an api reitti could query. It just happens that I am settled with immich and had no other needs at the moment.

If you need a specific one, drop a feature request and I will have a look.

Zagorath@aussie.zone on 03 Jul 03:09 collapse

I don’t actually have any personally. I’m still with Google Photos for now and hadn’t decided what to switch to, with Immich, Nextcloud, and the non-open Synology Photos being the top of my list. Legitimately, what a tool like this supports could be a factor I use to help decide.

How complicated is the code interfacing with Immich? Is it a piece someone not familiar with your overall code base could relatively easily pick up and make a pull request for?

danielgraf@discuss.tchncs.de on 03 Jul 03:28 collapse

I do not think it is that complicated. The front-end sends a request to the back-end with the current selected day. This triggers a search in Immich returning all photos taken on that specific day. This is returned to the front-end and this than does the heavy lifting like filtering them to the current map bounds, displaying them on the map at a specific location. We proxy all request from the front-end through our server because of CORS issues and I did tried to avoid having to configure Immich besides creating a token for the API.

One would need to either create a specific IntegrationService like ImmichIntegrationService and then figure out a way how the user can configure that. The easiest would be that we just then call all available ones even if I do not see the use case of having multiple Photo-Servers. But it would make the code in Reitti cleaner and would not hurt if we do not configure 20 simultaneous servers :D

Kernal64@sh.itjust.works on 03 Jul 03:22 next collapse

This looks really cool! I just have one question. Is it possible to just install this like normal software on a Linux machine or does it require Docker?

danielgraf@discuss.tchncs.de on 03 Jul 03:34 collapse

Thank you.

At the moment i do not have any plans of providing a way of running it without docker. Mainly because of time to support that.

Since it is a Spring-Boot-Application it would be possible to create a jar file which you can execute or deploy as a service with systemd. But then you have to make sure all prerequisites are also running. That is the one thing I like about docker and especially docker compose.

But short answer: Yes, it is possible but you are on your own at the moment. I would help and maybe we can add a section to the readme how to do it.

kameecoding@lemmy.world on 03 Jul 06:54 collapse

Not sure if you know about it and I haven’t used it myself yet, but being able to create native executables could be relatively easy with this

docs.spring.io/spring-boot/…/index.html

danielgraf@discuss.tchncs.de on 03 Jul 07:06 collapse

I used that once on a past gig and it wasn’t very pleasant to use. Especially in combination with spring boot. But that is a couple of years ago. Maybe things have changed. I personally would prefer the executable jar from spring boot. With that you do not have to make that many steps to make it work. But thanks for the suggestion :)

SomeDudeFromSpace@lemmy.ml on 03 Jul 03:35 next collapse

Great project! Any recommendations for an iOS app for this? I’ve been using OwnTracks, but it works very unreliably. As far as I understand, it’s an OS problem since Apple allows for very limited background processing for the sake of saving battery, so there’s not much any logging apps can do, but I was just curious if someone found a way around it.

danielgraf@discuss.tchncs.de on 03 Jul 04:05 collapse

Not really, I stopped using IOS a year ago because of exactly this reason. Had a lot of problems syncing files because of the power saving. I understand why IOS is doing it and for a normal user I think it is the way to go. But anything beyond that, it only hinders the experience you get out of apps. Maybe someone here as any experience with an app which works reliably.

emrsmsrli@lemmy.world on 03 Jul 03:41 next collapse

What’s the difference from Dawarich, if I may ask? Beside from a better name :)

danielgraf@discuss.tchncs.de on 03 Jul 04:01 collapse

Thanks :) As a German I really like the name Dawarich. First it sound really nice for me but also that “Da war ich” means “There have i been” in german makes, at least for me, an awesome project name.

Take this with a grain of salt because I have no idea what the plans are for Dawarich or have ever been and this is solely based on my external view. For me the main differences are:

  • visits and trips are our main data, everything else is just the way to calculate them. For Dawarich it looks to me, that it is the other way around. It displays all the location data in good way with the heatmap and so on but visits or places seems so tacked on. This should not be an offense against it. I actually still have an instance running and it was the main pushing point to finally start working on Reitti.
  • the sleek UI but this depends on your taste

In the end, they are not that far off. Maybe a matter of taste.

emrsmsrli@lemmy.world on 03 Jul 07:20 next collapse

Thanks for the reply! I will give it a go :)

The word dawarich does not produce a polite sound in my main language, meant no offense to the project :)

pipariturbiini@sopuli.xyz on 03 Jul 14:24 collapse

Maybe slightly off-topic, but how did you end up with the name “reitti”? You say you’re German - do you have some sort of tie to the Finnish language?

danielgraf@discuss.tchncs.de on 03 Jul 19:54 collapse

Oh, i had the idea in mind what i want to create and than it was a matter of a couple of Google queries but in the end one of the LLM suggested a list of different names in foreign languages and reitti somehow sticked 😊

warmaster@lemmy.world on 03 Jul 04:35 next collapse

This looks amazing, congratulations and thank you for making it FOSS. I was wondering if you are considering integrating with Home Assistant.

danielgraf@discuss.tchncs.de on 03 Jul 04:41 collapse

Thanks :)

No, did not occur to me. What would the integration look like? Connecting it to the message bus to receive location updates? Honestly it is a couple years ago I played with HA.

rumba@lemmy.zip on 03 Jul 05:00 next collapse

Location sensor would be a good minimum bar.

A custom card for your app that is just basically a iframe into your app with auth would also be pretty decent. Your version of a map looks really nice.

Maybe surfacing metrics of distance traveled or number of geolocations.

I’ll have to install the app and play around with it to make other recommendations but those are the first things that come to mind.

pyr0ball@lemmy.dbzer0.com on 03 Jul 07:41 next collapse

Home automation using geofencing, and my partner likes to get a notification when I’m heading home from the office

warmaster@lemmy.world on 03 Jul 15:33 collapse

I have the HA app on my phone, it reports my location back to my HA server.

I would like if Reitti could retrieve my location from my HA server, instead of asking me to upload it again to Reitti. Uploading my location in short intervals drains the battery very fast, it’s something I want to avoid if I can.

Additiinally, I don’t want to expose anything to the internet. So I pay for the Home Assistant cloud subscription that does it for me in a more secure manner than what I could implement with the little free time I have. Reitti could retrieve my location more securely if it did so without exiting my LAN.

danielgraf@discuss.tchncs.de on 03 Jul 19:51 collapse

I had a similar setup with Home Assistant in the past so I understand your usecase. For Reitti to detect visits somewhat reliable it needs at least one datapoint of location data a minute. We build location clusters with minimum 5 points in 5 minutes. If HA tracks that often it should work. HA probably tracks more than that.

I could add an integration that Reitti fetches the data from Home Assistant. Do you mind in creating a feature request?

warmaster@lemmy.world on 04 Jul 18:45 collapse

Do you mind in creating a feature request?

Done 👍🏻

gullmar@feddit.it on 03 Jul 05:08 next collapse

Very very cool! I see that you can import GPX files: can you also do the opposite, i.e., export a GPX file for a selected time range?

danielgraf@discuss.tchncs.de on 03 Jul 05:33 collapse

I was thinking about that, but the main problem is that we do not store all the data which comes in.

If we ingest data from an app, I am pretty sure that the quality of the data is actually usable. But for example if we import an Records.json from Google Takeout. The quality of the earlier years is somewhat sketchy. For this we filter out some points like travelling with over 2000 km/h, sudden direction changes etc and they are lost forever. At least for Reitti they are unknown.

The feature would need a lot of explanation why the data we export is not the same we import.That is the reason I did not implemented it even if it would come in handy for testing stuff. Handling GPX files is a pita …

gullmar@feddit.it on 03 Jul 13:58 collapse

I understand. Currently, I’m using the NextCloud’s extension Phonetrack, and one on my main use cases is to export GPX files to use them in Darktable and add geotracking information to the pictures I shoot with my camera. It works well, actually, but Phonetrack is not too intuitive and does not offer many other features: that’s why I was looking with interest at this alternative!

wise_pancake@lemmy.ca on 03 Jul 06:08 next collapse

Visually this is gorgeous

Really nice work!

illusionist@lemmy.zip on 03 Jul 07:20 next collapse

That looks very neat. Thanks for starting this!

I guess this will directly compete with dawarich, right?

github.com/Freika/dawarich

danielgraf@discuss.tchncs.de on 03 Jul 07:39 collapse

I would not say compete. They are different in how things are done from my point of view. I want to focus more on the visits we have done in the past to relive some lost memories whereas Dwarich looks more “technical” for me. I have no better words for it, I hope you get my point in what i am trying to achieve with Reitti. So there should be enough room for both 🙂

I also do not have any intentions to offer a hosted version in the foreseeable future or even anytime.

illusionist@lemmy.zip on 03 Jul 09:04 collapse

I get what you mean. The focus is different of dawarich. I’m really looking forward into checking reitti out!

TheHobbyist@lemmy.zip on 03 Jul 10:26 next collapse

Congrats! Amazing project, exciting interface and you went the extra mile on the integration side with third parties. Kudos!

Edit: I’ll definitely have to try it out!

umbrella@lemmy.ml on 03 Jul 10:34 next collapse

can i use it to track vehicles too like traccar?

danielgraf@discuss.tchncs.de on 03 Jul 12:56 collapse

I have no experience with traccar but it seems that it supports live tracking. This is something Reitti does not support. Depends on your usecase, but i think traccar is better suited.

SkaveRat@discuss.tchncs.de on 03 Jul 14:28 next collapse

Saving for later

douglasg14b@lemmy.world on 03 Jul 15:05 collapse

Samesies

someacnt@sh.itjust.works on 03 Jul 16:25 next collapse

I am slightly bothered by how the logo is quite off-center, is it intentional?

danielgraf@discuss.tchncs.de on 03 Jul 20:00 collapse

It was not intentional but after bothering not about it because i had other things on my mind i got used to it and now like it the way it is.

But for everyone who is bothered by that. If Reitti reaches 1k stars on Github I will add a switch to use a centered one 😊

ada@piefed.blahaj.zone on 03 Jul 18:55 next collapse

I managed to break our instance. I imported several years worth of google takeout location data, and now the "stay-detection-queue" is stalled.

danielgraf@discuss.tchncs.de on 03 Jul 19:57 collapse

Congratulations 😆

To help with that I would need some information:

  • does it show anything in the logs?
  • what do you mean by several years or how big was the Records.json?

Thank you for testing 🙂

ada@piefed.blahaj.zone on 03 Jul 20:34 collapse

It's a 1gig json file that has about 10 years of data. I get multiple repeats of the rabbit timeout in the logs. The Job Status section tells me that it's got just under 9 hours of processing remaining for just over 16,000 in the stay-detection-queue. The numbers change slightly, so something is happening, but it's been going for over 12 hours now, and the time remaining is slowly going up, not down.

reitti-1 | 2025-07-04T03:06:08.820Z INFO 1 --- [ntContainer#2-1] c.d.r.s.p.VisitDetectionService : Detected 61806 stay points for user ada reitti-1 | 2025-07-04T03:06:17.848Z WARN 1 --- [ntContainer#2-1] o.s.a.r.l.SimpleMessageListenerContainer : Consumer raised exception, processing can restart if the connection factory supports it reitti-1 | reitti-1 | com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - delivery acknowledgement on channel 9 timed out. Timeout value used: 1800000 ms. This timeout value can be configured, see consumers doc guide to learn more, class-id=0, method-id=0) reitti-1 | at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.checkShutdown(BlockingQueueConsumer.java:493) ~[spring-rabbit-3.2.5.jar!/:3.2.5] reitti-1 | at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.nextMessage(BlockingQueueConsumer.java:554) ~[spring-rabbit-3.2.5.jar!/:3.2.5] reitti-1 | at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:1046) ~[spring-rabbit-3.2.5.jar!/:3.2.5] reitti-1 | at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:1021) ~[spring-rabbit-3.2.5.jar!/:3.2.5] reitti-1 | at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.mainLoop(SimpleMessageListenerContainer.java:1423) ~[spring-rabbit-3.2.5.jar!/:3.2.5] reitti-1 | at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1324) ~[spring-rabbit-3.2.5.jar!/:3.2.5] reitti-1 | at java.base/java.lang.Thread.run(Unknown Source) ~[na:na] reitti-1 | Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - delivery acknowledgement on channel 9 timed out. Timeout value used: 1800000 ms. This timeout value can be configured, see consumers doc guide to learn more, class-id=0, method-id=0) reitti-1 | at com.rabbitmq.client.impl.ChannelN.asyncShutdown(ChannelN.java:528) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | at com.rabbitmq.client.impl.ChannelN.processAsync(ChannelN.java:349) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | at com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:193) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | at com.rabbitmq.client.impl.AMQChannel.handleFrame(AMQChannel.java:125) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | at com.rabbitmq.client.impl.AMQConnection.readFrame(AMQConnection.java:761) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | at com.rabbitmq.client.impl.AMQConnection.access$400(AMQConnection.java:48) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:688) ~[amqp-client-5.25.0.jar!/:5.25.0] reitti-1 | ... 1 common frames omitted

danielgraf@discuss.tchncs.de on 03 Jul 21:01 collapse

Thanks for the information. I will try to recreate it locally. In my testing I used a 600MB file and this took maybe 2 hours to process on my server. It is one of these ryzen 7 5825U. Since Reitti tries to do these analysis on multiple cores we start it with 4 to 16 Threads when processing. But the stay detection breaks when doing it that way, so it is locking per user to handle that. If now one of them takes a long time the others will break eventually. They will get resheduled 3 times until rabbitmq gives up.

On what type of system do you run it?

I will add some switches so it is configurable how many threads are opened and add some log statements to print out the duration it took for a single step.

ada@piefed.blahaj.zone on 03 Jul 21:07 collapse

i7-8700 with 64GB of RAM

danielgraf@discuss.tchncs.de on 03 Jul 21:17 collapse

Hmm, I had hoped you say something like a Raspberry PI :D

But this should be enough to have it processed in a reasonable time. What I do not understand in the moment is, that the filesize should not affect it in any way. When importing it 100 Geopoints are bundled, send to RabbitMQ. From there we retrieve them, do some filtering and save them in the database. Then actually nothing happens anymore until the next processing run is triggered.

But this than works with the PostGis DB and not with the file anymore. So the culprit should be there somewhere. I will try to insert some fake data into mine and see how long it takes if i double my location points.

ada@piefed.blahaj.zone on 03 Jul 21:40 collapse

I was also trying to set up GPSLogger whilst it was crunching through the backlog, and I manually transferred a file from that app before I had autologging configured. Not sure if that could have done it?

The times don't overlap, as the takeout file is only up until 2023

danielgraf@discuss.tchncs.de on 03 Jul 21:56 collapse

Thanks for getting back to me. I can look into it. I don’t think it’s connected, but you never know.

The data goes the same way, first to RabbitMQ and then the database. So it shouldn’t matter, it’s just another message or a bunch of them in the queue.

ada@piefed.blahaj.zone on 04 Jul 00:13 collapse

Ok, so it may not be frozen. The numbers in the queue seem to imply it is, however, timelines and places are slowly filling out in my history. A couple of dates I had looked at previously were showing me tracklogs for the day, but not timeline information, and now, they're showing timelines for the day

danielgraf@discuss.tchncs.de on 04 Jul 00:29 collapse

That’s good, but I still question why it is so slow. If you receive these timeout exceptions more often, at some point the data will cease to be analyzed.

I just re-tested it with multiple concurrent imports into a clean DB, and the stay-detection-queue completed in 10 minutes. It’s not normal for it to take that long for you. The component that should take the most time is actually the merge-visit-queue because this creates a lot of stress for the DB. This test was conducted on my laptop, equipped with an AMD Ryzen™ 7 PRO 8840U and 32GB of RAM.

ada@piefed.blahaj.zone on 04 Jul 01:09 collapse

Since I last commented, the queue has jumped from about 9000 outstanding items, to 15,000 outstanding items, and it appears that I have timelines for a large amount of my history now.

However, the estimated time is still slowly creeping up (though only by a minute or two, despite adding 6000 more items to the queue).

I haven't uploaded anything manually that might have triggered the change in queue size.

Is there any external calls made during processing this queue that might be adding latency?

tl;dr - something is definitely happening

danielgraf@discuss.tchncs.de on 04 Jul 01:52 collapse

This process is not triggered by any external events.

Every ten minutes, an internal background job activates. Its function is to scan the database for any RawLocationPoints that haven’t been processed yet. These unprocessed points are then batched into groups of 100, and each batch is sent as a message to be consumed by the stay-detection-queue. This process naturally adds to the workload of that queue.

However, if no new location data is being ingested, once all RawLocationPoints have been processed and their respective flags set, the stay-detection-queue should eventually clear, and the system should return to a idle state. I’m still puzzled as to why this initial queue (stay-detection-queue) is exhibiting such slow performance for you, as it’s typically one of the faster steps.

ada@piefed.blahaj.zone on 05 Jul 13:49 collapse

Well, the last update seems to have cleared the queue, and all of my history from that 10 year import now shows, with trips and places identified!

But now, it's having issues with importing the new google format import. I've got a 34MB file there that goes back to 2017, and this data says that it has imported, but then never appears in my history.

If it's relevant, there is overlap in the data, as my 10 year takeout import went up to 2023, and my "new format" import starts in 2017 and went a couple of days ago. I changed my google account in 2017, but logged in to both on my phone simultaneously, so I was accruing location data on both accounts at the same time for a while before I turned it off on my old account.

danielgraf@discuss.tchncs.de on 05 Jul 20:31 collapse

I am glad it worked out for you in importing the first file. I am still puzzled why it took that long.

For the new format, did you have Android or iOS? With the timeline export from Google Maps on iOS, we can not do anything at the moment because there is actually no raw data in it. Only information like you stayed in this timeframe at that point and you traveled between these points. It’s actually a little bit funny that it aggregates to the same data Reitti uses in the end.

If you are on Android, it could also be a bug when importing that file. I only had a small one from one of my accounts to test. If you mind creating a bug report, I will have a look. If you do not want to attach the export file there, feel free to send it to daniel@dedicatedcode.com. I will have a look at it then privately. No problem.

For the overlap in exports, it depends. If the points are the same, meaning they have the same timestamp, then Reitti will discard them. If not, they will be handled like every other new data and will end in recalculating visits and trips around that particular time.

ada@piefed.blahaj.zone on 05 Jul 21:15 collapse

I'm on android. I'll raise a bug report.

pfr@lemmy.sdf.org on 03 Jul 20:53 next collapse

Hell yes! I turned off location data for immich but now I can use this!

danielgraf@discuss.tchncs.de on 03 Jul 21:20 collapse

It is actually awesome if you have some old photos with the geodata attached and scim through Reitti and suddenly one of them shows up :)

LazyToad@sopuli.xyz on 04 Jul 00:48 next collapse

Got it up and running, looks neat!

Is there a way to import old pictures from Immich? If I take new ones they do show up on Reitti, but none of the old ones are shown.

danielgraf@discuss.tchncs.de on 04 Jul 01:43 next collapse

Thank you for testing Reitti. 🙏

It depends on two key requirements for Reitti:

  1. First, it finds all photos from Immich taken on the day you selected.
  2. Then, it filters these photos based on the selected map bounds, using the embedded EXIF geolocation data (where the photo was shot).

If the EXIF data does not contain geolocation information, we currently cannot display those photos because their placement on the map cannot be determined.

Could you please verify in Immich if the expected photo has its location in the metadata? If it is available there, then the issue might lie in how Reitti is parsing that specific data.

LazyToad@sopuli.xyz on 04 Jul 02:40 collapse

Thanks for the explanation! Seems to work correctly after all, pictures do show up when I change the date on timeline instead of just staring at current date, 👍 Had just not understood how to use Reitti properly.

danielgraf@discuss.tchncs.de on 04 Jul 02:43 collapse

Glad I could help :)

danielgraf@discuss.tchncs.de on 07 Jul 04:59 collapse

Hi LazyToad,

it depends. Reitti on its own does not import any pictures. It only shows fetches them from Immich. For that to work it needs the geolocation where the image was taken in the exif data. You need to check if your expected image has this data in Immich. If it is available it shoud be displayed on the map if you select the day the picture was taken.

Does it show the location in Immich? You can verify this if you open the image in Immich and let it display the Info.

<img alt="" src="https://discuss.tchncs.de/pictrs/image/b4137818-977c-41f7-99de-b932e9a042b8.png">

If it is showing, then it still could be that it is a bug in Reitti. Feel free to open an issue then :)

shrink9313@lemmy.world on 06 Jul 11:36 next collapse

Dear OP,

I just spent some time setting up Reitti and I am very pleased with the results. Many thanks for all the work and great documentation.

I do have a few questions:

  1. You mention that it can take hours to days for Photon to setup. How do I know its done?
  2. The default external reverse geocoding is not doing anything for me, but there seems little to no info in the logs for me to try any remedy. Related to my previous comment, how would I know that photon is working correct?
  3. Is it possible to enjoy the map view in color in stead of the black and white?

Thanks for your time.

danielgraf@discuss.tchncs.de on 06 Jul 11:53 collapse

Cool, thanks for the feedback.

For: 1: There should be something about downloading in the Photon logs. You can check with docker compose logs photon. I do not remember how long it took to download the index for Germany, but it was noticeable. It depends a lot on how you are connected to the internet and then how long it takes to extract the index.

2: You should also see that in the log of Photon. If the default does not work reliably, you can create an account on Geoapify; it is free, and you get 3000 reverse geocode requests per day. I personally have Photon for DE and then the default service and additional Geoapify.

3: Yeah, I was also thinking about that. In the end, a per-user setting and then the overlay should be disabled, but it could take a while until I can get to it.

shrink9313@lemmy.world on 06 Jul 12:48 collapse

Thanks a bunch for your time and answers! Ill have another go later this week and try to report back here. GPS logger and immich work flawless!

ada@piefed.blahaj.zone on 06 Jul 16:12 next collapse

Having all of this data stored in a location I can control has been really good.

But yesterday, it was pointed out to me that the API reitti provides means I have access to over a decades worth of location history that I can use to geotag my photo collection! That is a game changer. I've been trying to find a way to pull that information from google location history for a long time, and it turns out, you've created it!

This is really valuable to me, so I want you to know just how much I appreciate the effort you've put in to making it. Thank you

danielgraf@discuss.tchncs.de on 07 Jul 03:56 collapse

Thank you very much for these kind words. Means a lot to me if my little tool brings value to you :)

ada@piefed.blahaj.zone on 07 Jul 16:44 next collapse

I've got a question about how reitti calculates significant places/visits.

I was thinking of adjust gps logger so that it doesn't log points if they're within 10m of the last point it logged. That will clear up the data when I'm at home or work, so that there is less of a random squiggle of location data. It will record me arriving at home, and leaving home, but not much in between.

Will that impact how reitti calculates locations though? Is it looking at the number of points, or is it simply a matter of duration within a particular vicinity?

danielgraf@discuss.tchncs.de on 07 Jul 20:12 collapse

Hi ada,

yes, this will impact the visit calculation. Visits are calculated in building clusters of at least 5 points in a range of 100m over a duration of at least 5 minutes. If there is only one point logged for example at work or home, Reitti is not able to detect when you left it because every point is just an instant in time and does not carry any duration information with it.

The more points we have to calculate Visits the more accurate it will be. I personally have set up GPSLogger to log every 30 seconds no matter how far I travelled but with at least 40m of accuracy.

Pandoras_Can_Opener@mander.xyz on 10 Jul 07:25 next collapse

This is a bit of a long shot. Does this work with tracking devices? I’m looking for a degoogled way to track my cat and somebody linked me here.

danielgraf@discuss.tchncs.de on 10 Jul 12:02 collapse

Greetings, @Pandoras_Can_Opener@mander.xyz, that sounds like a truly wonderful idea, and as a fellow cat owner, it brings me great joy to hear about it. 😻

In fact, I have recently changed the analysis of data, which is now performed in near real-time as soon as new data becomes available. I am currently working on the functionality to display multiple users (or, in your case, Pandora) on the map, which should be beneficial to your idea.

Now, the primary question is, how can we integrate the data from the tracker into Reitti. Thats something I have no idea at the moment. Do you have any Infos about that?

ada@piefed.blahaj.zone on 11 Jul 10:49 collapse

Is there any way of forcing it to rebuild places and trips? Somewhere along the line, most of mine disappeared. It's still building new ones just fine as GPS logger adds real time data, but most of my historical stuff is gone.

The actual data is there, and it shows up fine on the maps for each day, but the places/trips/stats etc are mostly empty.

I can't see anything in the logs that might explain it

danielgraf@discuss.tchncs.de on 11 Jul 12:22 collapse

If you are running Reitti with Docker, you can set the environment variable DANGEROUS_LIFE to true. This will enable a new section in the Settings. There, you can use the button ‘Clear and Reprocess’. This will delete all visits and trips and set all raw data points to unprocessed. During the next run, everything will be recalculated. In that settings section, you could also press the ‘Start Processing’ button if you don’t want to wait. Make sure to wait until the success message appears. It can take a while until all data is deleted and switched to unprocessed.