"Folder path missing" even the path exists (OneDrive Win10 Fall Creators Update)


(Jarno) #1

I am getting “Folder path missing” error message for an existing folder. The folder has the “.stfolder” -folder. The whole folder is locally synced OneDrive For Business folder that is being backed up to NAS using Syncthing.

I think the problem relates to the the Win 10 Fall Creators Update that introduced a new Onedrive “Files on demand”-feature that is turned on by default (as if I wanted to keep my files on cloud by default). I have switched this feature from Onedrive settings off, but it did not solve the issue with Syncthing. Any idea what to do?


(Jack Chidley) #2

I am having a similar issue to you: it seems that any OneDrive folder, including the sub-folders, will not sync.


(Jarno) #3

I have both Onedrive and Onedrive For Business folders synced with Syncthing. I had the problem with both the folders, but the disabling the “Files On Demand” fixed the Onedrive folder sync, but it did not help with the Onedrive for Business sync. There are some error messages related to “readonly” folder too.


#4

I have a similar(if not the same) issue as all these folks.

After some digging, I found that this is because OneDrive(After the new Creator’s Update) now makes the OneDrive folder a reparsepoint. To get around this, you’ll need to delete the reparsepoint on the OneDrive directory in order for Syncthing to recognize this directory again.

Specifically, you can follow these steps:

  1. Stop OneDrive Sync. (Not sure if this is required, but I Exited the OneDrive Application in your toolbar icons section as well.)
  2. Open the Command Prompt by clicking the start button and typing in “cmd” in your search bar.
  3. In the command prompt, type in fsutil reparsepoint delete C:\Users\<your username>\OneDrive
  4. The command should not return any message.
  5. Restart Syncthing and see if it’s able to scan the directory again.

(In my situation, I also disabled the new “Files on Demand” feature.)

The only downside is that you’ll need to redo this process every time you reboot or start OneDrive. OneDrive seems to recreate this reparsepoint when it starts. I haven’t tried removing the reparsepoint while OneDrive is also syncing… since I’m scared I might corrupt something.


(Jarno) #5

Thanks a lot for debugging this. I hope devs can develop a workaround since deleting the folder is not very handy after every boot.


#6

To be clear, you are NOT deleting the folder.

You are deleting the reparse point associated with the folder. It’s basically metadata that OneDrive is using somehow. It seems that this has to deal with lower-level NTFS features that are way beyond my understanding as of now… But you can read more on reparse points here: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365503(v=vs.85).aspx


(Jarno) #7

Thanks, Geeksquad. I created a bug report to GitHub.


(Audrius Butkevicius) #8

So I don’t think we should fix this. The problem comes from the fact OneDrive uses something funky, not us.


(Jakob Borg) #9

I think the fix should come from the Go standard library / runtime, much like the similar issue of NTFS deduplication. Both stem from the fact of the thing on disk being visibly something other than a normal file or directory, and we only sync files and directories - not reparse points or quantum teleportation tunnels or magic placeholders or whatever. If it’s supposed to act like a normal file or directory, it’s up to the OS or standard library to present it as such.

It might certainly be a bug in Go, though.


Edit: I see the NTFS dedup issue is closed Go-side now, so maybe that works for us. Or maybe it just exposes the deduplicated files as symlinks in which case we will legitimately still ignore them.


#10

Would it be possible to explain how syncthing reads a directory? (I’m sorry if I’m boring you, I don’t really know golang.)

I wanted to see if it’s indeed a golang issue, so I wrote this quick script to just list out the files & directories of the OneDrive folder. (https://gist.github.com/clee231/413ccd631d26685250d8ae4c72e5ece9)

From my testing, this script is able to get a listing of files and directories within the folder, with and without the reparse point metadata in place.


If we decide to not fix it, I’d suggest at least updating the error message with something more descriptive, since the folder isn’t actually missing. :wink:


(Simon) #11

Can you try this: https://gist.github.com/imsodin/6fe6d8c877a6a96b3da66f93aefaaf03

We could extend the message to say “folder path missing or inaccessible”, but that is still very general.


(Audrius Butkevicius) #12

So I am actually upgrading my windows to this Fall Creator Break All Your Stuff update. Will see what that returns once it’s done.


(Jakob Borg) #13

There’s also the thing that I’m sure it’s possible to read the directory and the files in it if you don’t look too closely at what you’re reading - just hold your nose, open the file, read the contents. But we take great care to not follow symlinks, not read or wipe out things like named sockets and pipes, and so on.

Hence, when the OS throws us something that is sort of like a file, sort of like a symlink, maybe neither and acts weird when you try to figure out what it is - we will take the safe route and skip it. Unison and other programs may have a different philosophy and take a different approach. That’s fine.

This is also a scariness around even trying to support syncing the OneDrive files. I assume that if you actually check that box, files will no longer be on disk unless they are in use, and then get automatically offloaded into the cloud. Syncthing will replicate that as a delete to everywhere. If that everywhere is another OneDrive folder, that will look like a regular user-initated delete to OneDrive. Poof, gone for real.

All in all, while it would be nice to support whatever funkiness Windows is throwing at us here, I think it’s probably a Bad Idea™ to sync your OneDrive folder using Syncthing.


(Audrius Butkevicius) #14

So stat/lstat on OneDrive returns fine, it does however show up as a symlink, which essentially do not exist for us in Windows.


(Jakob Borg) #15

So probably https://github.com/syncthing/syncthing/issues/4353 is the first issue to solve (fs regression on folder roots that are symlinks in windows, afaics).

Not saying you should, but if someone wants to tackle it. :slight_smile:


(Audrius Butkevicius) #16

Well, but I can’t repro this issue:


(Jakob Borg) #17

The issue doesn’t say of course, but I have a feeling the failure is the symlink traversal check


#18

Using the gist from Audrius Butkevicius, I’m able to read the directory with no issues (with the reparse point metadata in place):

My results of the script are at the bottom of the gist.

In the results, the OneDrive folder reports as a symlink, but directories inside the OneDrive folder are not. I’m still a bit confused as to why exactly it is reported as a symlink. If it’s a symlink, then shouldn’t you be able to see where the link is pointing to? Is the golang libraries mis-reporting the status of the folder?


(Audrius Butkevicius) #19

Symlinks is a can of worms on Windows, and syncthing does not support them.


(Jarno) #20

It seems the devs have fixed the issue! I am looking forward for the next release. Thanks devs and geeksquads for you persistence.