Continous disk activity

macpeterr · October 13, 2017, 12:40pm

I have a Synology DS916+ with the following DSM version installled.

majorversion="6" minorversion="1" productversion="6.1.3" buildphase="GM" buildnumber="15152" smallfixnumber="7" builddate="2017/10/07" buildtime=“06:50:48”

Syncthing v0.14.39, Linux (64 bit) has been installed through the package manager and already running for some months fine now. Since some time now, Syncting is continously performing disk activity even when no clients are available to sync data.

I tried to debug it by running it from the commandline

sudo su - syncthing
export STTRACE=all
/usr/local/syncthing/bin/syncthing -home=/usr/local/syncthing/var

This seems to start syncthing fine, but when started this way the web interface on port 7070 does not come available, also not after some time. When running this way the disk activity is low again

2017/10/13 14:37:31.285483 debug.go:39: DEBUG: Enabling lock logging at 100ms threshold
2017/10/13 14:37:31.292622 logfs.go:72: DEBUG: main.go:1163 basic /usr/local/syncthing/var/ MkdirAll . 448 <nil>
2017/10/13 14:37:31.292732 logfs.go:114: DEBUG: main.go:1168 basic /usr/local/syncthing/var/ Stat . {0xc4201a44e0} <nil>
[monitor] 2017/10/13 14:37:31.292981 monitor.go:95: INFO: Starting syncthing
2017/10/13 14:37:31.295854 debug.go:39: DEBUG: Enabling lock logging at 100ms threshold
2017/10/13 14:37:31.302691 logfs.go:72: DEBUG: main.go:1163 basic /usr/local/syncthing/var/ MkdirAll . 448 <nil>
2017/10/13 14:37:31.302770 logfs.go:114: DEBUG: main.go:1168 basic /usr/local/syncthing/var/ Stat . {0xc42007be10} <nil>
[start] 2017/10/13 14:37:31.303092 events.go:268: DEBUG: subscribe Unknown
[start] 2017/10/13 14:37:31.303219 events.go:268: DEBUG: subscribe Unknown
[HOVGC] 2017/10/13 14:37:31.318887 main.go:651: INFO: syncthing v0.14.39 "Dysprosium Dragonfly" (go1.9 linux-amd64) teamcity@build.syncthing.net 2017-09-25 06:05:21 UTC
[HOVGC] 2017/10/13 14:37:31.318959 main.go:652: INFO: My ID: 
[monitor] 2017/10/13 14:37:31.789906 internal.go:51: DEBUG: Dialer logging disabled, as no proxy was detected
[HOVGC] 2017/10/13 14:37:31.800303 internal.go:51: DEBUG: Dialer logging disabled, as no proxy was detected
[HOVGC] 2017/10/13 14:37:32.399683 sha256.go:92: INFO: Single thread SHA256 performance is 99 MB/s using minio/sha256-simd (73 MB/s using crypto/sha256).
[HOVGC] 2017/10/13 14:37:32.399831 events.go:231: DEBUG: log 1 Starting map[home:/usr/local/syncthing/var myID:=...]
[HOVGC] 2017/10/13 14:37:33.521006 main.go:711: INFO: Hashing performance with weak hash is 85.00 MB/s
[HOVGC] 2017/10/13 14:37:34.626629 main.go:713: INFO: Hashing performance without weak hash is 94.03 MB/s
[HOVGC] 2017/10/13 14:37:34.626693 main.go:719: INFO: Weak hash enabled, as it has an acceptable performance impact.

How can i run syncthing on a synology DS from the commandline right ? And how am i able to detect what is causing the disk activity ?

cheers, Peter

macpeterr · October 17, 2017, 9:38pm

Finally was able to enable debugging properly. In debug mode syncthing is continously spitting out debug log entries like below;

Seems to be a scanning activity which continously runs and causing the high disk activity. Any ideas ?

[HOVGC] 2017/10/17 23:34:55.116385 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/users/renatevanderhorst/Mijn Muziek" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.117109 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/attachmentType_attachment_Operations.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.118628 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/users/renatevanderhorst/Mijn Video's" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.119042 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/callbackType.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.120468 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Andale Mono.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.120757 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/callbackType_callback_Operations.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.122592 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Apple Chancery.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.122757 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/collectionTargetActionType.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.124387 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/AppleGothic.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.124732 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/collectionTargetActionType_action_Operations.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.126751 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/AppleMyungjo.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.127408 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/collectionTargetType.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.128706 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Arial Black.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.129607 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/collectionTargetType_collectionTarget_Operations.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.131006 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Arial Bold Italic.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.131650 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/commentType.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.134026 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Arial Bold.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.134663 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/commentType_userComment_Operations.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.136268 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Arial Italic.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.137043 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/customAttributesType.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.138451 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Arial Narrow Bold Italic.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.139243 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/customAttributesType_customAttributes_Operations.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.140357 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Arial Narrow Bold.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.140786 leveldb_transactions.go:178: DEBUG: new global after update: {{{[{7657566545331481920 4}]}, NJCSIM4-SMXWUA7-HHSGM3O-PGEGLY4-66IRMUR-SZEKS4D-BFZYU7L-FYTBFQJ}, {{[{7657566545331481920 3}]}, 7777777-777777N-7777777-777777N-7777777-777777N-7777777-77777Q4}}
[HOVGC] 2017/10/17 23:34:55.141571 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/displayInfoType.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.142675 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Arial Narrow Italic.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.143144 leveldb_transactions.go:78: DEBUG: insert; folder="eutzg-7ikm3" device=NJCSIM4-SMXWUA7-HHSGM3O-PGEGLY4-66IRMUR-SZEKS4D-BFZYU7L-FYTBFQJ File{Name:"Library/Caches/com.apple.Safari/WebKitCache/Version 11/Records/A856DC69C44662F18C9BE5119BDA9E1DE2A37CE9/Resource/78EE1ACA4C53D62C8AF7E1F8C86CF74E3D5DE895", Sequence:11826359, Permissions:0600, ModTime:2017-10-17 19:41:43.151869 +0200 CEST, Version:{[{NJCSIM4 17}]}, Length:1013, Deleted:false, Invalid:false, NoPermissions:false, Blocks:[Block{0/1013/0/9c8e0caa116bd75cac806bb7b97b9a3bd0f062ad2177346409e13aaffa069b9f}]}
[HOVGC] 2017/10/17 23:34:55.144151 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/displayInfoType_displayInfo_Operations.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.145034 leveldb_transactions.go:89: DEBUG: update global; folder="eutzg-7ikm3" device=NJCSIM4-SMXWUA7-HHSGM3O-PGEGLY4-66IRMUR-SZEKS4D-BFZYU7L-FYTBFQJ file="Library/Caches/com.apple.Safari/WebKitCache/Version 11/Records/A856DC69C44662F18C9BE5119BDA9E1DE2A37CE9/Resource/78EE1ACA4C53D62C8AF7E1F8C86CF74E3D5DE895" version={[{7657566545331481920 17}]}
[HOVGC] 2017/10/17 23:34:55.146356 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Arial Narrow.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.146903 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/documentType.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.148716 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Arial Rounded Bold.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.149337 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/documentType_document_Operations.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.150931 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Arial Unicode.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.151841 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/flexfieldMappingType.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.153089 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Arial.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.154052 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/flexfieldMappingType_flexfieldMappings_Operations.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.155094 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Ayuthaya.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.156057 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/getTaskDetails.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.157336 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Batang.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.158473 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/getTaskDetails_Operations.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.160049 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/BigCaslon.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.161063 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/identityType.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.162536 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Bodoni 72 Smallcaps Book.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.163191 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/identityType_assigneeGroups_Operations.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}
[HOVGC] 2017/10/17 23:34:55.164624 leveldb_dbinstance.go:504: DEBUG: need folder="mccqf-4ccvf" device=5ONXR2U-FWTEFWG-MVV5UNE-TZEXZNI-F2PCI6A-UP4V3BQ-RYTM6GB-EYKIQQ3 name="Library/Application Support/lexy.huiswerk_144250711230912/drive_c/windows/Fonts/Bodoni Ornaments.ttf" need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}
[HOVGC] 2017/10/17 23:34:55.165404 leveldb_dbinstance.go:504: DEBUG: need folder="ksqib-wf3xr" device=HOVGCY5-3ZZMHT4-KXJZMEA-3OCEXYD-YIZNQOF-P3P4J3U-IV2DGSH-DL2WUQG name="BPM12c AmazonCloud/Stage files/BPMSuite12cTraining/labfiles/solutions/afterLab3/SalesQuoteApplication/BusinessPracticesUI/adfmsrc/BusinessPracticesUI_BusinessPracticesReviewHumanTask/identityType_assigneeUsers_Operations.xml" need=false have=false haveV={[]} globalV={[{7657566545331481920 1}]}

imsodin · October 17, 2017, 10:13pm

This is just a short burst of very verbose logging from the db package, that happens whenever pulling or calculation completions - it does not show a problem in itself.

The first line is weird, as the global version seems to be lower than the local one, which shouldn’t ever be the case I think:

need=true have=true haveV={[{16977296168964114523 2}]} globalV={[{4299355875259078777 1} {16977296168964114523 1}]}

There’s no context though, so no idea what might have caused it and whether it is connected at all.

Either provide a full log, which is probably way too big, or run without STTRACE and provide that log, it might already contain clues.

AudriusButkevicius · October 17, 2017, 11:18pm

Local and global version doesn’t mean much as it’s the first 7 bytes of the device id, so those numbers are irrelevant.

The fact that you run with STTRACE=all makes it even worse, as it logs tons of crap that is completely irrelevant to your problem.

If you are unhappy about disks not spinning down, then I can tell you that the disks that are storing the database are never going to spin down, because there is periodic db compaction in the background and various other things.

If you are talking about disks in data directories then it’s most likely doing scans every 60 or so seconds, or if not scans, it does a periodic health check to verify that the folder is still there etc.

macpeterr · October 18, 2017, 6:25pm

I do no care about not spinning down of the disks, thats expected when running a diskstation and using it primarily for backup reasons. But what i noticed is that the disk activity increased and that is want back to much less activity when shutting down syncthing.

I installed dstat on the synology to measure the disk i/o and it seems that the disk read activity went from an avg. of 20MB/sec to 0,5MB/sec after shutting down syncthing. I am not concerned about some disk i/o but this does also impact other applications running on the diskstation. Also already have set the scan interval to 600 seconds to reduce scanning.

AudriusButkevicius · October 18, 2017, 6:44pm

You can run with STTRACE=filesystem for filesystem operations, yet if that doesn’t show anything obvious, it’s most likely the database. I guess it would be useful to understand where the writes are happening, not just the fact that they are happening.

macpeterr · October 18, 2017, 7:25pm

Does it continously check for mtime changes for the files ? After running with STTRACE=filesystem, 99% of the logentries are similar to the ones below.

[HOVGC] 2017/10/18 21:23:24.851022 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedir_imedia_imedia/ Lstat Library/Caches/Metadata/Safari/Bookmarks/DBB9AC99-6FF1-4D77-A360-3CCC27D468A8.webbookmark {0xc42bbeb380} <nil>
[HOVGC] 2017/10/18 21:23:24.851148 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedir_imedia_imedia/ Lstat Library/Caches/Metadata/Safari/Bookmarks/05E417B6-7B9C-4F6A-B1C3-8356DDFE03C7.webbookmark {0xc42bbeb520} <nil>
[HOVGC] 2017/10/18 21:23:24.851294 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedir_imedia_imedia/ Lstat Library/Caches/Metadata/Safari/Bookmarks/188701C2-F1A5-4057-9730-6B49CE2F4078.webbookmark {0xc42bbeb6c0} <nil>
[HOVGC] 2017/10/18 21:23:24.851433 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedir_imedia_imedia/ Lstat Library/Caches/Metadata/Safari/Bookmarks/0BD699FA-FF8C-4320-8080-527E0A39B42D.webbookmark {0xc42bbeb860} <nil>
[HOVGC] 2017/10/18 21:23:24.851584 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedir_imedia_imedia/ Lstat Library/Caches/Metadata/Safari/Bookmarks/D9293D29-FC92-4368-8666-63654B8A6D09.webbookmark {0xc42bbeba00} <nil>
[HOVGC] 2017/10/18 21:23:24.851739 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedir_imedia_imedia/ Lstat Library/Caches/Metadata/Safari/Bookmarks/B4E02525-CE51-4B9D-89DC-98DFF835B221.webbookmark {0xc42bbebba0} <nil>
[HOVGC] 2017/10/18 21:23:24.851867 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedir_imedia_imedia/ Lstat Library/Caches/Metadata/Safari/Bookmarks/480C962F-94DB-4D8F-87DA-97A44543F979.webbookmark {0xc42bbebd40} <nil>

AudriusButkevicius · October 18, 2017, 7:33pm

This looks just like debug output during a normal scan.

macpeterr · October 19, 2017, 6:06pm

Yes, i seems to retrieve the mtime for each file. And as you can see the READ is low in MB/s while retrieving the mtime for the files. This also not my concern. But while it is not scanning for files and no clients are synchronizing, the READ transfer rate goes over the 20MB/s continously.

[HOVGC] 2017/10/19 18:41:15.218065 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedir_renatevanderhorst_barcelona/ Lstat Library/Caches/Metadata/Safari/History/https:%2F%2Fwww.springest.nl%2Fopleiding-onderwijs%2Fleeractiviteiten-leerstijlen%2Fkolb-leerstijlen-test?sg_sessionid=1479214153_582b044923a4e1.22347601&__sgtarget=-1&__sgbrwsrid=9a8975cafe7772133fbbe0ca848d40ef%23sgbody-1032245.webhistory {0xc42e9b5790} <nil>
[HOVGC] 2017/10/19 18:41:15.218307 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedir_renatevanderhorst_barcelona/ Lstat Library/Caches/Metadata/Safari/History/http:%2F%2Fmens-en-samenleving.infonu.nl%2Fopleiding-en-beroep%2F2319-leerstijl-en-leren-leerfasencyclus-kolb.html.webhistory {0xc42e9b5930} <nil>
[HOVGC] 2017/10/19 18:41:15.218526 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedire_petervannes_chigatze/ Lstat .m2/repository/org/apache/maven/maven-repository-metadata/2.0.7/_maven.repositories {0xc423f88000} <nil>
[HOVGC] 2017/10/19 18:41:15.218670 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedir_renatevanderhorst_barcelona/ Lstat Library/Caches/Metadata/Safari/History/https:%2F%2Fwww.zalando.nl%2Fdames-home%2F?wmc=CRM31_TM_NL_TM18_018..&wt_cd=5A1DCD2E14593CBC3B9A0684168A6274&wt_t=1479215921525.webhistory {0xc42e9b5ad0} <nil>
[HOVGC] 2017/10/19 18:41:15.218802 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedire_petervannes_chigatze/ Lstat .m2/repository/org/apache/maven/maven-repository-metadata/2.0.8 {0xc423f881a0} <nil>
[HOVGC] 2017/10/19 18:41:15.219597 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/chigatze_iii/ Lstat VM_Standard_Setup/soa12cR2_12.2.1.3/oraclelinux/cdrtools/cdrtools-3.01/libparanoia/shlparanoia.mk {0xc421a19930} <nil>
[HOVGC] 2017/10/19 18:41:15.219824 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/chigatze_iii/ Lstat VM_Standard_Setup/soa12cR2_12.2.1.3/oraclelinux/cdrtools/cdrtools-3.01/librscg {0xc421a19ad0} <nil>
[HOVGC] 2017/10/19 18:41:15.227156 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/fotos_bibliotheek_renatevanderhorst_barcelona/ Lstat Pictures/Foto's Library.photoslibrary/Previews/2011/05/27/20110527-225114/P1050909.jpg {0xc421a19ba0} <nil>
[HOVGC] 2017/10/19 18:41:15.227568 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/fotos_bibliotheek_renatevanderhorst_barcelona/ Lstat Pictures/Foto's Library.photoslibrary/Previews/2011/05/27/20110527-225114/P1050910.jpg {0xc421a19d40} <nil>
[HOVGC] 2017/10/19 18:41:15.227816 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/fotos_bibliotheek_renatevanderhorst_barcelona/ Lstat Pictures/Foto's Library.photoslibrary/Previews/2011/05/27/20110527-225114/P1050923.jpg {0xc421a19ee0} <nil>
[HOVGC] 2017/10/19 18:41:15.237772 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedir_renatevanderhorst_barcelona/ Lstat Library/Caches/Metadata/Safari/History/http:%2F%2Fsinterklaasjournaal.ntr.nl%2Fassets%2Fknutselen%2Fsinterklaaskalender_2016.pdf.webhistory {0xc42e9b5c70} <nil>
[HOVGC] 2017/10/19 18:41:15.238010 logfs.go:60: DEBUG: mtimefs.go:55 basic /volume1/Syncthing/homedir_renatevanderhorst_barcelona/ Lstat Library/Caches/Metadata/Safari/History/http:%2F%2Fservice-mail.zalando.nl%2Foptiext%2Foptiextension.dll?ID=Y78bWH_T3x9W5mmhY6d1C3TOQfD2EFjRUGOGxed6gSmFbZR7rlCH0P_2qtxPLVfoo7YhyAnzamxK65FTZ_f48XG1qXgYV2GSaF.webhistory {0xc42e9b5e10} <nil>
^C
2918k 2122k|   0     0 | 186  18.0 
4085k 3738k|   0     0 | 269  99.8 
3674k   12M|  13k    0 | 226   411 
3549k  221k|   0     0 | 220  24.8 
3241k  838k|2458B    0 | 227  82.2 
4882k   17M|  30k 5734B| 290   473 
3329k   21k|   0     0 | 205  4.00 
2971k 2186k|   0     0 | 198  8.80 
3125k  420k|2458B    0 | 226  26.0 
3194k 1166k|   0     0 | 204  27.8 
3025k  299k|   0     0 | 211  39.2 
3597k  614B|8192B    0 | 238  0.40 
5017k   18M|   0     0 | 318   468 
3334k   13k|   0     0 | 217  3.20 
3201k 3277B|   0  5734B| 221  2.00 
3481k 1918k|   0     0 | 239  54.2 
3450k  604k|   0     0 | 221  31.4 
3486k    0 |   0     0 | 234     0 
4371k 5262k|   0     0 | 287   108 
2619k   16M|   0     0 | 169   365 
 812k 2222k|   0     0 |74.4  14.2 
 863k 3789k|   0     0 |75.0  17.6 
1421k 3930k|   0     0 | 131  66.6 
1722k  905k|  11k    0 | 170  80.6 
 834k  317k|   0     0 |73.4  14.8 
2733k   16M|   0     0 | 182   355 
3114k  282k|4096B    0 | 198  4.40 
3361k  141k|   0     0 | 222  18.0 
2946k  221k|   0     0 | 206  23.6 
3225k  437k|   0     0 | 225  36.8 
3546k   19k|   0     0 | 222  4.20 
3386k 6554B|   0     0 | 234  2.80 
4363k   24M|6554B    0 | 279   501 
3398k 2119k|   0     0 | 224  24.6 
3370k  546k|   0     0 | 235  55.8 
3037k  238k|   0     0 | 205  15.4 
3574k  414k|   0     0 | 238  30.4 
2874k   43k|   0     0 | 206  7.20 
5456k   33M|5734B    0 | 304   705 
4646k  532k|   0     0 | 229  44.6 
6510k 8192B|   0     0 | 352  2.40 
2283k  266k|5734B    0 | 159  9.00 
2656k 4915B|   0     0 | 171  2.00 
2192k  159k|   0     0 | 172  31.0 
2076k  952k|   0     0 | 150  6.60 
3246k   14M|   0     0 | 221   355 
2994k  197k|   0     0 | 172  24.8 
  14M   82k|   0     0 | 396  5.20 
  19M 1362k|   0     0 | 497  8.20 
  19M   15M|   0     0 | 533   418 
  19M 2326k|   0     0 | 517  31.0 
  14M 2174k|   0     0 | 398  56.8 
  18M   70k|   0     0 | 474  11.0 
  16M  485k|9830B    0 | 441  23.6 
  15M    0 |   0     0 | 408     0 
-dsk/total- ---paging-- --io/total-
 read  writ|  in   out | read  writ
  17M  213k|   0     0 | 447  10.8 
  18M  158k|  13k    0 | 464  21.2 
  15M  949k|   0     0 | 417  36.6 
  15M   18M|   0     0 | 444   464 
  17M  386k|   0     0 | 452  18.2 
  19M  239k|   0     0 | 477  3.00 
  18M  819B|   0     0 | 473  0.80 
  19M  102k|6554B    0 | 479  9.80 
  18M  290k|6554B    0 | 466  20.8 
  19M  193k|   0     0 | 505  31.0 
  15M   11M|  11k    0 | 509   439

This read activity continuous till syncthing is restarted.

  20M  354k|   0     0 | 514  23.0 
  19M  238k|   0     0 | 510  23.4 
  16M 7736k|   0     0 | 490   261 
  21M  229k|   0     0 | 538  10.4 
  23M    0 |   0  3277B| 562     0 
  16M    0 |  34k    0 | 479     0 
2874k  285k|   0     0 | 185  7.40 
9025k   10M| 819B    0 | 298   279 
  14M 5690k|   0     0 | 271   114 
  15M 3982k|   0     0 | 336  90.0 
  11M 2213k|2458B    0 | 287   119 
9768k 1917k|   0     0 | 266  75.6 
  12M  713k|   0     0 | 270  42.4 
6436k 1942k|1638B    0 | 209  28.0 
7015k 5748k|   0     0 | 224   157 
5804k   21M|   0     0 | 281   602 
2965k  724k|6554B    0 | 223  32.6 
3162k 3983k|4915B    0 | 192  50.4 
3452k  360k|   0     0 | 260  30.6 
3102k  721k|6554B    0 | 268  9.80 
-dsk/total- ---paging-- --io/total-
 read  writ|  in   out | read  writ
3158k 1674k|   0     0 | 290  43.2 
4679k  579k|   0     0 | 361  50.8

AudriusButkevicius · October 19, 2017, 6:18pm

So read rate is 20mbps, but nothing in logs at the point it happens? It’s most likely the database. Perhaps you should temporarily move the database/home directory to a different spindle or ramfs and see if that makes it go away.

macpeterr · October 20, 2017, 9:13am

That whould require a very large ramdisk. Almost more RAM than available in my diskstation.

root@somehost:/usr/local/syncthing/var# du -h index-v0.14.0.db/ 3.9G index-v0.14.0.db/ root@somehost:/usr/local/syncthing/var#

imsodin · October 20, 2017, 10:07am

I like fatrace (https://piware.de/2012/02/fatrace-report-system-wide-file-access-events/) for that kind of thing. It is a tiny tool that will print file access to stdout. Some distros (debian, Arch probably more) have it in their repos if you distrust a tool from an unknown (to me at least) source.

macpeterr · October 20, 2017, 10:08am

I used iotop which can be installed on synology using ipkg. 99% of the io was related to syncthing processes.

AudriusButkevicius · October 20, 2017, 10:10am

Sure, but that doesn’t explain where the IO is going to. If you had separate disks, you could verify its the database by moving it to a separate spindle. To be honest, just running inotify in db directory and data directories might explain where the IO is happening.

macpeterr · October 20, 2017, 10:14am

No can’t still be explained. Tried to erase the current index database and restart Syncthing for each folder. Also changed the rescan interval for the folders to 3600 (1h).

curl -X POST -H “X-API-Key: …” https://somehost:7070/rest/system/reset?folder=gktyk-zxmi5 --insecure &

This seems to reduce the I/O. Though it seems that the rescan interval is not persisted. After restarting it is back at default value (300s). (Yes i did hit the save button )

macpeterr · October 20, 2017, 10:15am

I already remove inotify some weeks ago

imsodin · October 20, 2017, 10:24am

As Audrius said, iotop gives you global disk activity by process, fatrace gives you a log of accessed/written/… files per process. Thus you will be able to tell what exactly is being accessed during the high activity phases reported by iotop. It is a very simple utility, just run it and look at it’s output. The purpose is the same as using notify (not syncthing-inotify) as Audrius proposed, just (arguably) easier.

Also rescan interval most certainly is persistent, it seems like there is something quite off with your setup. Please run the fatrace stuff and maybe give as a full non-debug log. Until now we only got to see very verbose but tiny log fragments which do not really say much.

AudriusButkevicius · October 20, 2017, 11:39am

Rescan interval can be overwritten by external tools if you use any.

macpeterr · October 21, 2017, 1:21pm

Tried to build version 0.11 as wel as 0.12 on the DiskStation. Both builds do fail and suggested use of pacman seems not possible. I am not able to install pacman throug ipkg.

cc -O2 -g -Wall -Wextra -Werror   -c -o fatrace.o fatrace.c
fatrace.c: In function 'print_event':
fatrace.c:138:13: error: In the GNU C Library, "major" is defined
 by <sys/sysmacros.h>. For historical compatibility, it is
 currently defined by <sys/types.h> as well, but we plan to
 remove this soon. To use "major", include <sys/sysmacros.h>
 directly. If you did not intend to use a system-defined macro
 "major", you should undefine it after including <sys/types.h>. [-Werror]
         snprintf (pathname, sizeof (pathname), "device %i:%i inode %ld\n", major (st.st_dev), minor (st.st_dev), st.st_ino);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                                                                                                                                                                                                        
fatrace.c:138:13: error: In the GNU C Library, "minor" is defined
 by <sys/sysmacros.h>. For historical compatibility, it is
 currently defined by <sys/types.h> as well, but we plan to
 remove this soon. To use "minor", include <sys/sysmacros.h>
 directly. If you did not intend to use a system-defined macro
 "minor", you should undefine it after including <sys/types.h>. [-Werror]
         snprintf (pathname, sizeof (pathname), "device %i:%i inode %ld\n", major (st.st_dev), minor (st.st_dev), st.st_ino);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                                                                                                                                                                                                        
cc1: all warnings being treated as errors
<builtin>: recipe for target 'fatrace.o' failed
make: *** [fatrace.o] Error 1

macpeterr · October 21, 2017, 1:38pm

Because i was unable to compile fatrace i tried two different versions of notify. Both versions do not support to trace the i/o activity base on a PID. How would you suggest (which arguments) to use notify to determine the file i.o?