Pull: Generic Error

Running v1.3.0, macOS (64 bit) on both sync clients.

  • Client A: Send Only shows as up to date.
  • Client B: Showing some failed and with status “Failed Items”

Startup and (sanitised) Logs sample on Client A:

[start] 16:10:38 INFO: syncthing v1.3.0 "Fermium Flea" (go1.13.1 darwin-amd64) brew@Mojave.local 2019-10-01 05:34:58 UTC
[start] 16:10:38 INFO: Using large-database tuning
[SEFGJ] 16:10:39 INFO: My ID: SEFGJ5J-7AH3M2O-AIMGRXH-3OI47MV-CB3P6RI-xxxxxxx-xxxxxxx-xxxxxxx
[SEFGJ] 16:10:40 INFO: Single thread SHA256 performance is 99 MB/s using crypto/sha256 (81 MB/s using minio/sha256-simd).
[SEFGJ] 16:10:40 INFO: Hashing performance is 92.03 MB/s
[SEFGJ] 16:10:41 INFO: Ready to synchronize "Photos HDD" (ey6qw-v7cpn) (sendonly)
[SEFGJ] 16:10:41 INFO: Ready to synchronize "iPhoto & Photos HDD" (nfxrt-6pjz3) (sendonly)
[SEFGJ] 16:10:41 INFO: Ready to synchronize "RAW Scans HDD" (t53sl-dlnqc) (sendonly)
[SEFGJ] 16:10:41 INFO: Ready to synchronize "Default Folder" (default) (sendreceive)
[SEFGJ] 16:10:41 INFO: Overall send rate is unlimited, receive rate is unlimited
[SEFGJ] 16:10:41 INFO: Using discovery server https://discovery.syncthing.net/v2/?noannounce&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[SEFGJ] 16:10:41 INFO: Using discovery server https://discovery-v4.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[SEFGJ] 16:10:41 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[SEFGJ] 16:10:41 INFO: Using discovery server https://discovery-v6.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[SEFGJ] 16:10:41 INFO: TCP listener ([::]:22000) starting
[SEFGJ] 16:10:41 INFO: QUIC listener ([::]:22000) starting
[SEFGJ] 16:10:41 INFO: Completed initial scan of sendreceive folder "Default Folder" (default)
[SEFGJ] 16:10:41 INFO: quic://0.0.0.0:22000 detected NAT type: Full cone NAT
[SEFGJ] 16:10:41 INFO: quic://0.0.0.0:22000 resolved external address quic://119.xx.xx.xx:42449 (via stun.syncthing.net:3478)
[SEFGJ] 16:10:41 INFO: GUI and API listening on 127.0.0.1:22023
[SEFGJ] 16:10:41 INFO: Access the GUI via the following URL: https://127.0.0.1:22023/
[SEFGJ] 16:10:41 INFO: My name is "Lexx"
[SEFGJ] 16:10:41 INFO: Device IBIFF4U-HVZQ6YX-ZO2BQJK-KKCP7LI-I3H4UJJ-xxxxxxx-xxxxxxx-xxxxxxx is "remote.server.com" at [dynamic]
[SEFGJ] 16:10:42 INFO: Established secure connection to IBIFF4U-HVZQ6YX-ZO2BQJK-KKCP7LI-I3H4UJJ-xxxxxxx-xxxxxxx-xxxxxxx at 10.0.0.10:55944-119.xx.xx.xx:22000/tcp-client/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[SEFGJ] 16:10:42 INFO: Device IBIFF4U-HVZQ6YX-ZO2BQJK-KKCP7LI-I3H4UJJ-xxxxxxx-xxxxxxx-xxxxxxx client is "syncthing v1.3.0" named "remote.server.com" at 10.0.0.10:55944-119.xx.xx.xx:22000/tcp-client/TLS1.3-TLS_CHACHA20_POLY1305_SHA256
[SEFGJ] 16:10:42 INFO: Completed initial scan of sendonly folder "RAW Scans HDD" (t53sl-dlnqc)
[SEFGJ] 16:10:42 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/08"): scan: fcntl: too many open files
[SEFGJ] 16:10:42 INFO: Scanner (folder ey6qw-v7cpn, item "Slide Previews - Who and When"): scan: open /Volumes/photos/Slide Previews - Who and When: too many open files
[SEFGJ] 16:10:42 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/09"): scan: open /Volumes/iphoto/Apple Photos/Photos Library.photoslibrary/Masters/2016/02/09: too many open files
[SEFGJ] 16:10:42 INFO: Scanner (folder ey6qw-v7cpn, item "Vuescan"): scan: open /Volumes/photos/Vuescan: too many open files
[SEFGJ] 16:10:42 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/10"): scan: fcntl: too many open files
[SEFGJ] 16:10:42 INFO: Scanner (folder ey6qw-v7cpn, item "Wedding Stills"): scan: open /Volumes/photos/Wedding Stills: too many open files
[SEFGJ] 16:10:42 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/11"): scan: open /Volumes/iphoto/Apple Photos/Photos Library.photoslibrary/Masters/2016/02/11: too many open files
[SEFGJ] 16:10:43 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/17/20160217-233711"): scan: fcntl: too many open files
[SEFGJ] 16:10:43 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/19"): scan: fcntl: too many open files
[SEFGJ] 16:10:43 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/23"): scan: fcntl: too many open files
[SEFGJ] 16:10:43 INFO: Scanner (folder nfxrt-6pjz3, item "Apple Photos/Photos Library.photoslibrary/Masters/2016/02/24"): scan: fcntl: too many open files
[SEFGJ] 16:10:45 INFO: Completed initial scan of sendonly folder "Photos HDD" (ey6qw-v7cpn)

Startup and (sanitised) Logs sample on Client B:

[start] 16:13:58 INFO: syncthing v1.3.0 "Fermium Flea" (go1.13.1 darwin-amd64) brew@HighSierra.local 2019-10-01 05:34:58 UTC
[start] 16:13:58 INFO: Using large-database tuning
[IBIFF] 16:14:00 INFO: My ID: IBIFF4U-HVZQ6YX-ZO2BQJK-KKCP7LI-I3H4UJJ-xxxxxxx-xxxxxxx-xxxxxxx
[IBIFF] 16:14:01 INFO: Single thread SHA256 performance is 148 MB/s using minio/sha256-simd (122 MB/s using crypto/sha256).
[IBIFF] 16:14:01 INFO: Hashing performance is 125.79 MB/s
[IBIFF] 16:14:01 INFO: Ready to synchronize "iPhoto & Photos HDD" (nfxrt-6pjz3) (sendreceive)
[IBIFF] 16:14:01 INFO: Ready to synchronize "RAW Scans HDD" (t53sl-dlnqc) (receiveonly)
[IBIFF] 16:14:01 INFO: Ready to synchronize "Default Folder" (default) (sendreceive)
[IBIFF] 16:14:01 INFO: Ready to synchronize "Photos HDD" (ey6qw-v7cpn) (sendreceive)
[IBIFF] 16:14:01 INFO: Overall send rate is unlimited, receive rate is unlimited
[IBIFF] 16:14:01 INFO: Using discovery server https://discovery.syncthing.net/v2/?noannounce&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[IBIFF] 16:14:01 INFO: Using discovery server https://discovery-v4.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[IBIFF] 16:14:01 INFO: Using discovery server https://discovery-v6.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW
[IBIFF] 16:14:01 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting
[IBIFF] 16:14:01 INFO: QUIC listener ([::]:22000) starting
[IBIFF] 16:14:01 INFO: TCP listener ([::]:22000) starting
[IBIFF] 16:14:01 INFO: Completed initial scan of sendreceive folder "Default Folder" (default)
[IBIFF] 16:14:01 INFO: GUI and API listening on [::]:22023
[IBIFF] 16:14:01 INFO: Access the GUI via the following URL: https://127.0.0.1:22023/
[IBIFF] 16:14:01 INFO: My name is "remote.server.com"
[IBIFF] 16:14:01 INFO: Device SEFGJ5J-7AH3M2O-AIMGRXH-3OI47MV-CB3P6RI-xxxxxxx-xxxxxxx-xxxxxxx is "Lexx" at [dynamic]
[IBIFF] 16:14:02 INFO: quic://0.0.0.0:22000 detected NAT type: Full cone NAT
[IBIFF] 16:14:02 INFO: quic://0.0.0.0:22000 resolved external address quic://119.xx.xx.xx:22000 (via stun.syncthing.net:3478)
[IBIFF] 16:14:04 INFO: Established secure connection to SEFGJ5J-7AH3M2O-AIMGRXH-3OI47MV-CB3P6RI-xxxxxxx-xxxxxxx-xxxxxxx at 10.0.0.100:53956-10.0.0.10:22000/tcp-client/TLS1.3-TLS_AES_128_GCM_SHA256
[IBIFF] 16:14:04 INFO: Device SEFGJ5J-7AH3M2O-AIMGRXH-3OI47MV-CB3P6RI-xxxxxxx-xxxxxxx-xxxxxxx client is "syncthing v1.3.0" named "Lexx" at 10.0.0.100:53956-10.0.0.10:22000/tcp-client/TLS1.3-TLS_AES_128_GCM_SHA256
[IBIFF] 16:14:04 INFO: Completed initial scan of receiveonly folder "RAW Scans HDD" (t53sl-dlnqc)
[IBIFF] 16:14:07 INFO: Completed initial scan of sendreceive folder "Photos HDD" (ey6qw-v7cpn)
[start] 16:13:58 INFO: syncthing v1.3.0 "Fermium Flea" (go1.13.1 darwin-amd64) brew@HighSierra.local 2019-10-01 05:34:58 UTC
[start] 16:13:58 INFO: Using large-database tuning
[IBIFF] 16:14:00 INFO: My ID: IBIFF4U-HVZQ6YX-ZO2BQJK-KKCP7LI-I3H4UJJ-xxxxxxx-xxxxxxx-xxxxxxx
[IBIFF] 16:14:01 INFO: Single thread SHA256 performance is 148 MB/s using minio/sha256-simd (122 MB/s using crypto/sha256).
[IBIFF] 16:14:01 INFO: Hashing performance is 125.79 MB/s
[IBIFF] 16:14:01 INFO: Ready to synchronize "iPhoto & Photos HDD" (nfxrt-6pjz3) (sendreceive)
[IBIFF] 16:14:01 INFO: Ready to synchronize "RAW Scans HDD" (t53sl-dlnqc) (receiveonly)
[IBIFF] 16:14:01 INFO: Ready to synchronize "Default Folder" (default) (sendreceive)
[IBIFF] 16:14:01 INFO: Ready to synchronize "Photos HDD" (ey6qw-v7cpn) (sendreceive)
[IBIFF] 16:14:01 INFO: Overall send rate is unlimited, receive rate is unlimited
[IBIFF] 16:14:01 INFO: Using discovery server https://discovery.syncthing.net/v2/?noannounce&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-M[IBIFF] 16:14:13 INFO: New NAT port mapping: external TCP address 119.xx.xx.xx:38454 to local address 0.0.0.0:22000.
[IBIFF] 16:14:13 INFO: Detected 1 NAT service
[IBIFF] 16:14:35 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1979-01-xx-John_DOE-North_Albury-VS-RAW-4000-64b-AGFA-00067.dng"): pull: generic error
[IBIFF] 16:14:35 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/19xx-xx-xx-Historic_Building-Victoria-VS-RAW-4000-64b-00051.dng"): pull: generic error
[IBIFF] 16:14:36 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1974-03-07-John_DOE-VS-RAW-4000-64b-AGFA-00146.dng"): pull: generic error
[IBIFF] 16:14:42 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1977-xx-xx-Jane_DOE-Wedding-VS-RAW-4000-64b-AGFA-00121.dng"): pull: generic error
[IBIFF] 16:14:46 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/_prescantesting/slide-2018-12-30-0002.dng"): pull: generic error
[IBIFF] 16:14:52 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1974-12-xx-John_DOE-Visit_from_paternal_grandparents-VS-RAW-4000-64b-AGFA-00196.dng"): pull: generic error
[IBIFF] 16:14:54 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1973-05-xx-Unknown-VS-RAW-4000-64b-Kodachrome-00170.dng"): pull: generic error
[IBIFF] 16:14:59 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1963-xx-xx-Rob_DOE-Boating-Frankston_Victoria-VS-RAW-4000-64b-Kodachrome-00159.dng"): pull: generic error
[IBIFF] 16:15:06 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1967-05-xx-House-Noble_Park_Victoria-VS-RAW-4000-64b-Kodachrome-00166.dng"): pull: generic error
[IBIFF] 16:15:08 INFO: Joined relay relay://45.76.112.17:22067
[IBIFF] 16:15:12 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/197x-xx-xx-Jim_Jane_DOE-VS-RAW-4000-64b-AGFA-00006.dng"): pull: generic error
[IBIFF] 16:15:16 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1974-02-09-Joe_DOE-Wedding-VS-RAW-4000-64b-AGFA-00009.dng"): pull: generic error
[IBIFF] 16:15:19 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/_prescantesting/slide-2018-12-29-0006.dng"): pull: generic error
[IBIFF] 16:15:24 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/1977-xx-xx-Jane_DOE-Wedding-VS-RAW-4000-64b-AGFA-00118.dng"): pull: generic error
[IBIFF] 16:15:28 INFO: Puller (folder "RAW Scans HDD" (t53sl-dlnqc), item "vuescan-raw/197x-xx-xx-Jim_Jane_DOE-VS-RAW-4000-64b-AGFA-00007.dng"): pull: generic error

And so on. Client B (the Send/Receive) logs all the pull: generic error continuously while client A (Send Only) logs continuously scan: fcntl: too many open files on this and other shared folders.

This is bad. It can’t open files because it does not have available file descriptors. You’ll have to increase that limit (ulimit -n).

It’s asking for data from client A, who can’t serve it due to the error above and doesn’t really understand why it’s not working, hence returning a generic “could’t give you that data, sorry”.

1 Like

Thanks @calmh - The Mac soft limit is 256 files on Mac OS 10.14.

Found a great write-up on the specifics of setting it properly on a Mac.

Now I set it to 1024, that error went away and the Remote (Client B) synced everything happily.