Set rescan interval for a specific time of day?

Support
1

We are taxing the heck out of machine with 7 very large repos that get a large amount of changes every night (and on the weekends), as this machine catches all of the backups from various servers (not Syncthing backups, these are done with Robocopy). Plus we have a different backup process that provides more granular versioning on a different set of repos. If Syncthing starts rescanning the robocopy jobs slow down to a crawl and even fail. I already have Interval set to 86400 (once a day), but what would help if if I could have this only occur at 9AM (for example), when no other jobs are running.

2

In this case I’d recommend disabling the automatic rescan and triggering it externally. In v0.11, set the rescan interval to zero (in the config, the UI might not allow it); in v0.10 set it to a very large value (but don’t use v0.10).

Then put something like curl -X POST -H X-API-Key:yourapikey http://localhost:8384/rest/db/scan?folder=default into your crontab at some suitable time.

You’ll still get a rescan when syncthing starts up, no matter what.

Syncthing-inotify may be useful too, or maybe not depending on your modification pattern and number of files… But do investigate it.

1 Like
3

Thanks. Your example uses folder=default, but the documentation (https://github.com/syncthing/syncthing/wiki/REST-Interface) indicates that a specific folder has be indicated. Is default a valid value? Will I have to specify each one of the repos?

4

Default is the folder which gets added by default on first start, so default is a valid value but it refers to the specific folder called default. You’ll need a:

for FOLDER in foldera folderb folderc; do 
  curl -X POST -H X-API-Key:yourapikey http://localhost:8384/rest/db/scan?folder=$FOLDER
done
5

I figured as much, but was hoping that wasn’t true. That is going to be problematic to maintain. What if I just set the value to 0, and then just restart Syncthing once a day on a specific schedule?

6

Restarting causes a rescan by default, so it’s good enough just to restart. Ideally you might want to check that it’s not in progress of doing something maybe.

7

Or let the computer work for you.

#!/bin/bash

apikey=$(curl -s http://localhost:8080/rest/system/config | json gui.apiKey)
folders=$(curl -s http://localhost:8080/rest/system/config | json folders | json -a id)

for folder in $folders; do
    echo "Rescan of $folder"
    curl -X POST -H "X-API-Key: $apikey" \
        "http://localhost:8080/rest/db/scan?folder=$folder"
done

(requires https://github.com/trentm/json)

A restart will resend the full index right now, which is unfortunately inefficient. The above just does rescan and sends deltas.

1 Like
8

Are you certain setting that value to 0 disables the scan? The discussion around that is confusing.

9

Yes

12345678901234567890

10

Try as I might, I can’t get the REST API to work with cURL.

command: curl -X POST -H “X-API-KEY:xxxxxxxxxxxxx” https://localhost:8384/rest/system/config -k result: EOF

command: curl -X POST -H “X-API-KEY:xxxxxxxxxxxxxxxxxx” https://localhost:8384/rest/db/scan?folder=KASPERSKY01 -k result: Nothing? No response, does not initiate a scan

11

You’re posting in an empty config. I’m not sure what happens here. Maybe you get an error, maybe you actually erase the config.

This could be an effect of nuking the config in the previous step… Also, you can run with curl -i to get the response code - 200 is successful, anything else indicates an issue.

12

Luckily, we don’t nuke the config when the input is empty - there is a 500 error returned. :relieved:

The second command either returns empty and with code 200 or with an error and code 500. Are you sure the scan doesn’t happen?

13

Yikes, I misread the directions. Also, I thought that cURL without a command was a GET, and I had tried this and got nothing as well:

curl -X GET -H “X-API-KEY:xxxxxxxxxxxxxxx” https://localhost:8384/rest/system/config

Ok, it is a GET, I just tried this again and it got me the config (I swear it returned just “Found” last night):

curl -X GET -H “X-API-KEY:xxxxxxxxxxx” https://localhost:8384/rest/system/config

Yes, I believe the other command is still not initiating a scan. In fact, once last night and in my test this morning, the command seems to “hang” (not return, in fact I waited so long last night I eventually killed it.

14

Wait, is it not supposed to return until the scan is complete? I hope not, because that’s very long time.

15

Ok, yes, it is starting the scan - the GUI does not indicate that is occurring though - and the command finally returned, but took about 30-60 seconds.

16

It does not return until the scan is complete, that’s correct. Or until curl times out, which may be something to increase.

17