Pull: Generic Error

lantrix · October 29, 2019, 5:21am

Running v1.3.0, macOS (64 bit) on both sync clients.

Client A: Send Only shows as up to date.
Client B: Showing some failed and with status “Failed Items”

Startup and (sanitised) Logs sample on Client A:

[start] 16:10:38 INFO: syncthing v1.3.0 "Fermium Flea" (go1.13.1 darwin-amd64) brew@Mojave.local 2019-10-01 05:34:58 UTC
[start] 16:10:38 INFO: Using large-database tuning
[SEFGJ] 16:10:39 INFO: My ID: SEFGJ5J-7AH3M2O-AIMGRXH-3OI47MV-CB3P6RI-xxxxxxx-xxxxxxx-xxxxxxx
[SEFGJ] 16:10:40 INFO: Single thread SHA256 performance is 99 MB/s using crypto/sha256 (81 MB/s using minio/sha256-simd).
[SEFGJ] 16:10:40 INFO: Hashing performance is 92.03 MB/s
[SEFGJ] 16:10:41 INFO: Ready to synchronize "Photos HDD" (ey6qw-v7cpn) (sendonly)
[SEFGJ] 16:10:41 INFO: Ready to synchronize "iPhoto & Photos HDD" (nfxrt-6pjz3) (sendonly)
[SEFGJ] 16:10:41 INFO: Ready to synchronize "RAW Scans HDD" (t53sl-dlnqc) (sendonly)
[SEFGJ] 16:10:41 INFO: Ready to synchronize "Default Folder" (default) (sendreceive)
[SEFGJ] 16:10:41 INFO: Overall send rate is unlimited, receive rate is unlimited
[SEFGJ] 16:10:41 INFO: Using discovery server https://discovery.syncthing.net/v2/?noannounce&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[SEFGJ] 16:10:41 INFO: Using discovery server https://discovery-v4.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[SEFGJ] 16:10:41 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[SEFGJ] 16:10:41 INFO: Using discovery server https://discovery-v6.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[SEFGJ] 16:10:41 INFO: TCP listener ([::]:22000) starting
[SEFGJ] 16:10:41 INFO: QUIC listener ([::]:22000) starting
[SEFGJ] 16:10:41 INFO: Completed initial scan of sendreceive folder "Default Folder" (default)
[SEFGJ] 16:10:41 INFO: quic://0.0.0.0:22000 detected NAT type: Full cone NAT
[SEFGJ] 16:10:41 INFO: quic://0.0.0.0:22000 resolved external address quic://119.xx.xx.xx:42449 (via stun.syncthing.net:3478)
[SEFGJ] 16:10:41 INFO: GUI and API listening on 127.0.0.1:22023
[SEFGJ] 16:10:41 INFO: Access the GUI via the following URL: https://127.0.0.1:22023/
[SEFGJ] 16:10:41 INFO: My name is "Lexx"
[SEFGJ] 16:10:41 INFO: Device IBIFF4U-HVZQ6YX-ZO2BQJK-KKCP7LI-I3H4UJJ-xxxxxxx-xxxxxxx-xxxxxxx is "remote.server.com" at [dynamic]
[SEFGJ] 16:10:42 INFO: Established secure connection to IBIFF4U-HVZQ6YX-ZO2BQJK-KKCP7LI-I3H4UJJ-xxxxxxx-xxxxxxx-xxxxxxx at 10.0.0.10:55944-119.xx.xx.xx:22000/tcp-client/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[SEFGJ] 16:10:42 INFO: Device IBIFF4U-HVZQ6YX-ZO2BQJK-KKCP7LI-I3H4UJJ-xxxxxxx-xxxxxxx-xxxxxxx client is "syncthing v1.3.0" named "remote.server.com" at 10.0.0.10:55944-119.xx.xx.xx:22000/tcp-client/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[SEFGJ] 16:10:42 INFO: Completed initial scan of sendonly folder "RAW Scans HDD" (t53sl-dlnqc)
[SEFGJ] 16:10:42 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/08"): scan: fcntl: too many open files
[SEFGJ] 16:10:42 INFO: Scanner (folder ey6qw-v7cpn, item "Slide Previews - Who and When"): scan: open /Volumes/photos/Slide Previews - Who and When: too many open files
[SEFGJ] 16:10:42 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/09"): scan: open /Volumes/iphoto/Apple Photos/Photos Library.photoslibrary/Masters/2016/02/09: too many open files
[SEFGJ] 16:10:42 INFO: Scanner (folder ey6qw-v7cpn, item "Vuescan"): scan: open /Volumes/photos/Vuescan: too many open files
[SEFGJ] 16:10:42 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/10"): scan: fcntl: too many open files
[SEFGJ] 16:10:42 INFO: Scanner (folder ey6qw-v7cpn, item "Wedding Stills"): scan: open /Volumes/photos/Wedding Stills: too many open files
[SEFGJ] 16:10:42 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/11"): scan: open /Volumes/iphoto/Apple Photos/Photos Library.photoslibrary/Masters/2016/02/11: too many open files
[SEFGJ] 16:10:43 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/17/20160217-233711"): scan: fcntl: too many open files
[SEFGJ] 16:10:43 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/19"): scan: fcntl: too many open files
[SEFGJ] 16:10:43 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/23"): scan: fcntl: too many open files
[SEFGJ] 16:10:43 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/24"): scan: fcntl: too many open files
[SEFGJ] 16:10:45 INFO: Completed initial scan of sendonly folder "Photos HDD" (ey6qw-v7cpn)

Startup and (sanitised) Logs sample on Client B:

[start] 16:13:58 INFO: syncthing v1.3.0 "Fermium Flea" (go1.13.1 darwin-amd64) brew@HighSierra.local 2019-10-01 05:34:58 UTC
[start] 16:13:58 INFO: Using large-database tuning
[IBIFF] 16:14:00 INFO: My ID: IBIFF4U-HVZQ6YX-ZO2BQJK-KKCP7LI-I3H4UJJ-xxxxxxx-xxxxxxx-xxxxxxx
[IBIFF] 16:14:01 INFO: Single thread SHA256 performance is 148 MB/s using minio/sha256-simd (122 MB/s using crypto/sha256).
[IBIFF] 16:14:01 INFO: Hashing performance is 125.79 MB/s
[IBIFF] 16:14:01 INFO: Ready to synchronize "iPhoto & Photos HDD" (nfxrt-6pjz3) (sendreceive)
[IBIFF] 16:14:01 INFO: Ready to synchronize "RAW Scans HDD" (t53sl-dlnqc) (receiveonly)
[IBIFF] 16:14:01 INFO: Ready to synchronize "Default Folder" (default) (sendreceive)
[IBIFF] 16:14:01 INFO: Ready to synchronize "Photos HDD" (ey6qw-v7cpn) (sendreceive)
[IBIFF] 16:14:01 INFO: Overall send rate is unlimited, receive rate is unlimited
[IBIFF] 16:14:01 INFO: Using discovery server https://discovery.syncthing.net/v2/?noannounce&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[IBIFF] 16:14:01 INFO: Using discovery server https://discovery-v4.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[IBIFF] 16:14:01 INFO: Using discovery server https://discovery-v6.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[IBIFF] 16:14:01 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[IBIFF] 16:14:01 INFO: QUIC listener ([::]:22000) starting
[IBIFF] 16:14:01 INFO: TCP listener ([::]:22000) starting
[IBIFF] 16:14:01 INFO: Completed initial scan of sendreceive folder "Default Folder" (default)
[IBIFF] 16:14:01 INFO: GUI and API listening on [::]:22023
[IBIFF] 16:14:01 INFO: Access the GUI via the following URL: https://127.0.0.1:22023/
[IBIFF] 16:14:01 INFO: My name is "remote.server.com"
[IBIFF] 16:14:01 INFO: Device SEFGJ5J-7AH3M2O-AIMGRXH-3OI47MV-CB3P6RI-xxxxxxx-xxxxxxx-xxxxxxx is "Lexx" at [dynamic]
[IBIFF] 16:14:02 INFO: quic://0.0.0.0:22000 detected NAT type: Full cone NAT
[IBIFF] 16:14:02 INFO: quic://0.0.0.0:22000 resolved external address quic://119.xx.xx.xx:22000 (via stun.syncthing.net:3478)
[IBIFF] 16:14:04 INFO: Established secure connection to SEFGJ5J-7AH3M2O-AIMGRXH-3OI47MV-CB3P6RI-xxxxxxx-xxxxxxx-xxxxxxx at 10.0.0.100:53956-10.0.0.10:22000/tcp-client/TLS1.3-TLS_AES_128_GCM_SHA256
[IBIFF] 16:14:04 INFO: Device SEFGJ5J-7AH3M2O-AIMGRXH-3OI47MV-CB3P6RI-xxxxxxx-xxxxxxx-xxxxxxx client is "syncthing v1.3.0" named "Lexx" at 10.0.0.100:53956-10.0.0.10:22000/tcp-client/TLS1.3-TLS_AES_128_GCM_SHA256
[IBIFF] 16:14:04 INFO: Completed initial scan of receiveonly folder "RAW Scans HDD" (t53sl-dlnqc)
[IBIFF] 16:14:07 INFO: Completed initial scan of sendreceive folder "Photos HDD" (ey6qw-v7cpn)
[start] 16:13:58 INFO: syncthing v1.3.0 "Fermium Flea" (go1.13.1 darwin-amd64) brew@HighSierra.local 2019-10-01 05:34:58 UTC
[start] 16:13:58 INFO: Using large-database tuning
[IBIFF] 16:14:00 INFO: My ID: IBIFF4U-HVZQ6YX-ZO2BQJK-KKCP7LI-I3H4UJJ-xxxxxxx-xxxxxxx-xxxxxxx
[IBIFF] 16:14:01 INFO: Single thread SHA256 performance is 148 MB/s using minio/sha256-simd (122 MB/s using crypto/sha256).
[IBIFF] 16:14:01 INFO: Hashing performance is 125.79 MB/s
[IBIFF] 16:14:01 INFO: Ready to synchronize "iPhoto & Photos HDD" (nfxrt-6pjz3) (sendreceive)
[IBIFF] 16:14:01 INFO: Ready to synchronize "RAW Scans HDD" (t53sl-dlnqc) (receiveonly)
[IBIFF] 16:14:01 INFO: Ready to synchronize "Default Folder" (default) (sendreceive)
[IBIFF] 16:14:01 INFO: Ready to synchronize "Photos HDD" (ey6qw-v7cpn) (sendreceive)
[IBIFF] 16:14:01 INFO: Overall send rate is unlimited, receive rate is unlimited
[IBIFF] 16:14:01 INFO: Using discovery server https://discovery.syncthing.net/v2/?noannounce&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-M[IBIFF] 16:14:13 INFO: New NAT port mapping: external TCP address 119.xx.xx.xx:38454 to local address 0.0.0.0:22000.
[IBIFF] 16:14:13 INFO: Detected 1 NAT service
[IBIFF] 16:14:35 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1979-01-xx-John_DOE-North_Albury-VS-RAW-4000-64b-AGFA-00067.dng"): pull: generic error
[IBIFF] 16:14:35 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/19xx-xx-xx-Historic_Building-Victoria-VS-RAW-4000-64b-00051.dng"): pull: generic error
[IBIFF] 16:14:36 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1974-03-07-John_DOE-VS-RAW-4000-64b-AGFA-00146.dng"): pull: generic error
[IBIFF] 16:14:42 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1977-xx-xx-Jane_DOE-Wedding-VS-RAW-4000-64b-AGFA-00121.dng"): pull: generic error
[IBIFF] 16:14:46 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/_prescantesting/slide-2018-12-30-0002.dng"): pull: generic error
[IBIFF] 16:14:52 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1974-12-xx-John_DOE-Visit_from_paternal_grandparents-VS-RAW-4000-64b-AGFA-00196.dng"): pull: generic error
[IBIFF] 16:14:54 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1973-05-xx-Unknown-VS-RAW-4000-64b-Kodachrome-00170.dng"): pull: generic error
[IBIFF] 16:14:59 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1963-xx-xx-Rob_DOE-Boating-Frankston_Victoria-VS-RAW-4000-64b-Kodachrome-00159.dng"): pull: generic error
[IBIFF] 16:15:06 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1967-05-xx-House-Noble_Park_Victoria-VS-RAW-4000-64b-Kodachrome-00166.dng"): pull: generic error
[IBIFF] 16:15:08 INFO: Joined relay relay://45.76.112.17:22067
[IBIFF] 16:15:12 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/197x-xx-xx-Jim_Jane_DOE-VS-RAW-4000-64b-AGFA-00006.dng"): pull: generic error
[IBIFF] 16:15:16 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1974-02-09-Joe_DOE-Wedding-VS-RAW-4000-64b-AGFA-00009.dng"): pull: generic error
[IBIFF] 16:15:19 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/_prescantesting/slide-2018-12-29-0006.dng"): pull: generic error
[IBIFF] 16:15:24 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1977-xx-xx-Jane_DOE-Wedding-VS-RAW-4000-64b-AGFA-00118.dng"): pull: generic error
[IBIFF] 16:15:28 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/197x-xx-xx-Jim_Jane_DOE-VS-RAW-4000-64b-AGFA-00007.dng"): pull: generic error

And so on. Client B (the Send/Receive) logs all the pull: generic error continuously while client A (Send Only) logs continuously scan: fcntl: too many open files on this and other shared folders.

calmh · October 29, 2019, 6:44am

This is bad. It can’t open files because it does not have available file descriptors. You’ll have to increase that limit (ulimit -n).

It’s asking for data from client A, who can’t serve it due to the error above and doesn’t really understand why it’s not working, hence returning a generic “could’t give you that data, sorry”.

lantrix · October 29, 2019, 11:20am

Thanks @calmh - The Mac soft limit is 256 files on Mac OS 10.14.

Found a great write-up on the specifics of setting it properly on a Mac.

Now I set it to 1024, that error went away and the Remote (Client B) synced everything happily.

system · December 6, 2019, 6:55am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.