How to sync contacts and calendars between desktop and Android (without root)

Howto
1

This is a simple guide on how to synchronise contacts and calendars between a desktop operating system (Windows, macOS, Linux) and Android relying on a combination of Syncthing and Radicale. The main goal and idea is to have Radicale running on every system locally, so that you can connect with it even when there is no network connection available. This applies to both desktop operating systems and Android as well.

Desktop:

  1. Install and run Radicale following instructions from https://radicale.org.

  2. Use relative paths in the Radicale config file, so that everything is contained in the same folder.

  3. For reference, my Radicale config looks somewhat like this:

    [auth]
type = htpasswd
htpasswd_filename = users
htpasswd_encryption = plain

[server]
hosts = localhost:5232

[storage]
filesystem_folder = .
type=multifilesystem_nolock

    The last line, i.e. type=multifilesystem_nolock, is required when running Radicale on some filesystems, e.g. FAT on Android.

  4. Sync contacts via CardDAV and calendars via CalDAV between Radicale and a local client application, e.g. Thunderbird.

  5. Add the whole local Radicale folder to Syncthing.

  6. Add .Radicale.cache to the folder’s ignore patterns.

  7. Share the Radicale folder from the desktop to the Android device.

Android:

  1. Accept the shared Radicale folder, placing it somewhere on the internal storage, e.g. /sdcard/Syncthing/Radicale.

  2. Install Termux (https://wiki.termux.com/wiki/Installation).

  3. Install Termux:Boot (https://wiki.termux.com/wiki/Termux:Boot).

  4. In Android settings, give Termux permission to access files and also exclude the app from battery optimisation. The first is required for Radicale to access the Syncthing folder, and the second for DAVx⁔ to trigger Radicale sync when in background.

  5. Inside Termux, update repositories first:

    pkg up

  6. Install Python:

    pkg install python

  7. Install Radicale:

    pip install radicale

  8. Install Vim:

    pkg install vim

    Of course, you can use another text editor of choice instead.

  9. Create a new text file called radicale.sh at ~/.termux/boot:

    mkdir ~/.termux/boot
touch ~/.termux/boot/radicale.sh

  10. Edit the newly created file with Vim:

    vim ~/.termux/boot/radicale.sh

  11. Inside the file, write:

    #!/data/data/com.termux/files/usr/bin/sh
cd "/sdcard/Syncthing/Radicale"
python -m radicale --config="radicale.ini"

    Note: This assumes that your Radicale config file is called radicale.ini.

  12. From now on, Radicale should start automatically on boot, and you can connect to it via a Web browser the same way as you would on the desktop.

  13. Install DAVx⁔ (https://f-droid.org/packages/at.bitfire.davdroid) and use it to sync contacts and calendars between Radicale and Android.

    Note: if Data Saver is enabled in the OS, you need to exclude DAVx⁔ from it, or otherwise nothing will happen when trying to run the sync (despite the fact that it happens fully offline :crazy_face:).

While this method may sound complicated at first, I have been using it for years now with zero issues. I had also tried using DecSync (https://github.com/39aldo39/DecSync) before, however it proved to be very buggy for me, hence I switched to this method instead. The main advantage is that Radicale runs locally both on the desktop and Android, which makes it possible to add, edit, and delete items offline, and Syncthing will sync the changes once connected. Of course, as with any other type of sync, simultaneous modifications on multiple devices will lead to conflicts, so you mustn’t edit the same events or contacts without syncing the previous changes first.

I also understand that this short guide assumes the user has already got some basic knowledge on how to install software and operate the command line. I may try to make the guide more beginner-friendly later on, but unfortunately I’m very limited on time right now, so I’ve only wanted to lay out the basic steps to follow along.

I hope someone will find this guide useful :slightly_smiling_face:.

PS Inside Termux, you can test the script simply by executing it manually first, e.g. sh ~/.termux/boot/radicale.sh in order to verify that it actually works while watching the output live before trying to start in on boot (and possibly debug any errors there).

14 Likes
3

Nice tutorial!

I was wondering what problems you were having with DecSync, though? It seems to work fine for me


4

Thank you :slightly_smiling_face:.

I wasn’t able to import my existing contacts and calendars into it (see https://github.com/39aldo39/Radicale-DecSync/issues/6, the issue is still open), and also DecSync has a problem of creating a vast and always growing number of tiny files (see https://github.com/39aldo39/DecSync/issues/2), which is a problem for Syncthing specifically, because it slows down synchronisation, especially on Android (11 and newer), where dealing with thousands of files is very slow due to FUSE.

On the other hand, Radicale by itself only stores and syncs actual calendar event .ics files and contact .vcf files without any additional metadata attached to them separately (as long as you ignore the cache files as recommended in the instructions).

Of course, if DecSync works for you, then I’m not saying that there’s anything wrong with using it, especially considering that it’s much easier to set up on Android.

5

Thank you again for the guide. It works (with a+x for .sh and pwd file creation)! )

What do you do to encrypt content of collections so other apps can’t access it? I couldn’t easilty spot any config or plugin that enables it.

Reasoning

“All files” permission on Android is somewhat more widespread than Contacts. In my case, it’s: Contacts is only 6 of 50 apps allowed “Files and media”: 44 apps allowed out of 77 “All files”: 7 apps (termux+tasker+syncthing+“file manager”
) And that’s not counting “systems” apps like all those “nearby share” etc that has “all files” access.

6

I don’t think you can protect the data like that and still make it accessible to other applications. This is because the folder in question needs to be in a location that both Syncthing and Termux can read and write to.

Personally, I’m not really worried about other apps that much. I always try to use free and open source applications whenever possible, and if not, I stick to using the big names, avoiding unknown, closed source ones. I do use full disk encryption on all my devices to protect them from unauthorised physical access.

7

Lots of good work, and not easy.

Will set this aside in case it’s needed. Right now I have contacts and calendar synchronizing through Gmail because I’m an Android user.

Thunderbird works with Gmail natively so I can’t see a need for it at the moment but thank you.

8

I would like to thank you again for the guide, I’m piloting it for several users though still having troubles with how to avoid “allow all of them to use battery without any limits”.

Re encryption - it looks like one can create needed “Storage plugin” for Radicale similarly t what was done for ?esty. I’m not talking here about secure vault invisible to anyone, but at least not storing them in plain text in plain sight - similarly to what is done in KeePassDX where pwd DB is kept in encrypted manner in “publicly accessible” space.

9

Hi @tomasz86, very nice work. This is essentially what I was looking for, for a long time and is well presented for my level of knowledge.

However I am struggling to set up the android version. At the point, where your “radicale.ini” config comes into play, I dont know what to do. I cant find this file anywhere neither do I know how to create it. And if I just run it without a specification I cant get it to run the desired Syncthing Folder as its storage without crashing. Could you go a little bit into detail on the android installation? I would really love to make this work, since everything else is very smoothly running.

10

The question seems to be more about how to configure Radicale itself than related to Syncthing, however my radicale.ini basically looks like this:

[auth]
type = htpasswd
htpasswd_filename = users
htpasswd_encryption = plain

[server]
hosts = localhost:5232

[storage]
filesystem_folder = .
type=multifilesystem_nolock

and then the users file in the same folder:

user1:password1
user2:password2

Obviously, you should use your preferred usernames and passwords here.

filesystem_folder = . makes Radicale store all files inside the collection-root folder in the same path as radicale.ini is located.

This is basically it, but if you still have problems, please share screenshots of your configuration, including the exact path and structure of your Radicale folder that you sync with Syncthing.

1 Like
11

This is a really great idea, thanks so much for the guide. Unfortunately, termux can only be run from the primary user account. The bootstrap binaries have a hardcoded $PREFIX path that can not be installed to any other path than /data/data/com.termux/files/usr. I run multiple profiles on my phone, and leave the primary as bare as possible. As such, I can’t run termux in that profile. There does seem to be ways to bypass this restriction via binary patching, but that also requires root. Does anyone know of alternative methods for running radicale on android?

12

Very interesting. Out of curiosity: why did you opt for Syncthing when Radicale is actually about syncing contacts and calendar? (I am using Davx5 on Android connecting to a Radicale instance on the linux desktop).

13

I just prefer to have everything synced via P2P and self-contained :slight_smile:. No central server, etc. This is especially true, as I often need to operate with extremely poor Internet connectivity, or even no Internet at all.

3 Likes
14

Thank you, Tomasz, for your efforts! Since this seems to be the only solution for syncing calendars and similar data on Android and desktop without a central server and without relying on abandonware, I think it would deserve a dedicated wiki + forum. What do you think? For now, I’ll use the Syncthing forum.

I set up almost everything as described.

Added .Radicale.tmp-* to the Syncthing ignore list since I don’t think it makes sense to sync those.

For a reason I couldn’t find out, Radicale on Android doesn’t update mtimes on its ics files, at least for me. Because of that, locally updated events whose ics file sizes didn’t change (e.g., just moved to another day) were ignored by Syncthing. I cured this by adding

hook = touch --no-create "%(path)s"

to the [storage] section of the Radicale config file on Android. It works very well so far. Still feels weird that this was necessary at all. Could someone please check if this behavior is standard for Radicale on Android file systems or if I just did something wrong?

What I couldn’t cure was that on Android, DAVx5 + Radicale take a looong time to sync my calendars (roughly 3 min for 10000 events on a Pixel 8). Every time, no matter how few changes I made. Sometimes I even got timeouts, but I think I have never lost any data.

Can I somehow help DAVx5 and/or Radicale identify the events that have changed? Do they really have to check everything every time? If this can’t be changed, could I, at least with root access, use a faster file system somewhere on the Android device (ext4 instead of fuse)? I’m aware these are no Syncthing questions even though they’re on-topic for this thread.

15

Thank you for this guide, its working really well for me.

One thing I noticed is that if you delete a collection in Radicale on one device, Syncthing will refuse to delete the collection folder on your other devices because they still contain the ignored .Radicale.cache folder. Changing your ignore pattern to (?d).Radicale.cache should fix that.

At least on my Android device running LineageOS 22 (Android 15), Radicale is correctly updating mtimes.

16

Thanks for this detailed writeup, @tomasz86. I need some alternative because I’ve been using DecSync+Syncthing for years for my contacts and calendar, but it is abandoned and barely hanging on, with entries randomly disappearing.

I’d like to try this, but really don’t know if I am up for another Termux adventure. It just seems like way too many links in the chain. Why can neither Android or Thunderbird expose backend ics and vcf files that can be synced between devices with Syncthing? We had this kind of thing figured out decades ago and now have to make things complicated with so many needless SQLite databases and servers snooping on our data.

Okay, enough rant, but really, I would financially support a small project for a self-contained app or utility on each end to expose ics and vcf files that can be synced by Syncthing. Syncthing’s conflict avoidance is really adequate, if I have to manually resolve a conflict on a rare occasion, it’s not a big deal because they are human-readable.