Documentation move - again‽

So I’ve been thinking about docs, and how to make things more accessible. Currently the documentation lives in the Github Wiki. I’m proposing moving them to an actual documentation site instead, and I’ve set one up on http://docs.syncthing.net/.

By doing this I hope to gain some advantages.

  • Control over the layout - i.e. get rid of the Github chrome around the wiki which is entirely irrelevant for most users.
  • Improved structure. The Github wiki is essentially one flat namespace, which the sidebar tries to chop into sections for users, developers etc. This doesn’t scale.
  • The possibility of doing other neat tricks. http://docs.syncthing.net/dev/rest.html is for example generated, sorted and indexed based on descriptions of the individual REST endpoints.
  • We can bundle a static copy as documentation into the release packages.

The Github wiki does however advantages in the other direction.

  • Anyone can edit it (as long as they have a Github account). This is not that much worse with the docs site, as https://github.com/syncthing/docs can be edited in almost the same way (i.e. on the Github site), thus generating a pull request.

There may be others?

The docs site linked above isn’t the finished product - some content remains to be moved, and much cleanup needs to happen. I still think it’s better (or will be) than the Github wiki.

Comments?

2 Likes

Personally, I was thinking about proposing migrating the documentation to sphinx. The advantages are:

  • The ability to generate man pages easily. I tried to create some out of the wiki a few weeks ago. I did not enjoy the experience…
  • You can control the layout.
  • Versioned docs. For example, one can use read the docs to render the sphinx documentation. It can generate an endpoint for every git tag automatically.
  • The need of pull requests IMO improves the quality of the documentation (i.e. review). But there may be less contributors.

Despite I don’t know your tool of choice to generate the documentation, I like the move away from the github wiki. It basically includes all the advantages I have listed above.

But what about man pages? Can we generate man pages with this tool? I really want to include traditional documentation in the arch linux packages. With a proper tool that should not be that big deal.

Sphinx looks reasonable too, well worth looking into.

Possibly, I’m not sure the whole “objects in domains” thing fits perfectly to the kind of more user oriented docs we have, and while man page generation is nice that’s only needed for basically one page - syncthing(1) - which doesn’t currently exist?

I think even more are useful:

  • syncthing(1) (note: the exit code page can be merged into this)
  • syncthing-config(5)
  • syncthing-stignore(5)
  • syncthing-rest-api(7)
  • syncthing-event-api(7)
  • syncthing-device-ids(7)
  • syncthing-networking(7)
  • syncthing-security(7)
  • syncthing-file-versioning(7)
  • syncthing-faq(7)

Since these pages are (more or less) already available, we just need some toolchain to convert them into manpage format. As you propose migrating the documentation to something else, I think it is a good moment to think about manpages.

Would you like to set up a proof of concept?

Well, in my mind is something like this (e.g. for syncthing-config(5)):

What needs to be done:

  • The docs need to be in a format that can be converted to man pages:
    • I don’t know how to do this with markdown. If there is a tool available please point me to that location.
    • With a properly configured sphinx I simply need to do make man.
    • For rst (sphinx uses it) there is also rst2man (link).
    • asciidoc/asciidoctor works fine as well
  • The manpages need to be generated and included in release packages/scripts. I can do that for arch linux.
  • One additional page needs to be written -> syncthing(1)
  • IMO it would be better to ship the docs in the same repository. That would make packagers life easier, since they just have to pick one repo.

I am playing around a bit. I’ll post a link here in about half an hour or so.

1 Like

I have quickly converted four pages to sphinx:

  • Intro
  • Getting Started
  • FAQ
  • Ignoring

I am now able to export the docs in the following formats:

  • HTML
  • man
  • pdf
  • txt

The result are (temporarely) here:

3 Likes

Agreed. I’ve moved the docs site to Sphinx and hacked up the docs a bit to work with it. There’s probably lots to do, and I haven’t yet made anything automatic to build pdf:s and manpages etc, but it will at least be possible.

2 Likes

Saw the old version and much prefer the move to Sphinx!

Great work

You can use pandoc to to the basic work. Just use sth like this:

pandoc --from markdown --to rst file.md

It reduces the amount of work a lot.

edit: Ahh I see you are already there… Thanks! I will then look for some potential manpages.

I just pushed some commits which complete the manpage collection. The last milestone is to find a way to include them in releases. :blush:

Nice! I think, on the docs site, the man page should have a more descriptive title like “Command Line Operation” or something?

I’m thinking the man pages can be pulled in by the build process and included in the packages, similar to how the PDFs are currently handled. (That needs updating as well for the new docs.)

We could also check them into the source as the last step before tagging a release, similar to how we handle translation updates. That has the advantage of directly putting them in front of people like district packagers who only pull the source tar balls…

My first idea was something like “Syncthing Daemon”. Not sure if I like it or not…

The best would be to somehow include them in the source. Distributions like Arch, (or maybe even FreeBSD) pull the sources and build the packages themselves. Since we are using two seperate repos, it is quite difficult to ensure that a particular doc revision points to a particular Syncthing release.

Maybe using a git submodule can help? So you just bump the commit hash, and everyone gets the same thing. You also avoid generating and commiting compiled copies all the time.

Yeah. I think just grabbing the latest ones at the pre-release step will do. As I said, there’s already a manual step that must be run to grab the latest translation files from Transifex and rebuild the built in assets etc; it’s no hardship to just download and check in the latest man pages as well at that point. They are still human readable(-ish) and being able to diff them historically in the actual source repo might be useful, possibly, some time.

In the meantime, they are here: http://docs.syncthing.net/man/

I need to install some stuff to get the PDF generation working…

PDF is built, http://docs.syncthing.net/pdf/, but sort of unsurprisingly looks like shit. Some tweaking required…

Yeah. Tweaking that pdf/latex stuff might be interesting. :confused:

Should we also generate the txts? It can be included in packages as well; on linux for example in /usr/share/doc/syncthing. But since we have the manpages…

The new docs look fantastic, thanks so much! :smile: :+1: :beers:

1 Like