Revise and possibly automate offline documentation build process

[dregad]

Offline documentation has not been built for a long time (most recent one goes back to 2020-01-26).

Following recent discussion with @mnewnham the build process is currently quite complex, involving a local copy of the https://adodb.org wiki (updated using Dokuwiki sync plugin), which is converted to static HTML files using the SiteExport plugin. These files are cleaned up using a PHP script, then manually zipped and uploaded to SourceForge.

This could probably be simplified, e.g. by directly exporting the wiki from adodb.org (without the local instance).

I tried with the following commands:

cd /tmp

# wget command copied from Admin | SiteExport page (under Start Process)
wget --max-redirect=191 --output-document=siteexport.zip --post-data="id=index&addParams=1&depth=0&depthType=1.0&do=siteexport&ens=index&exportLinkedPages=1&renderer=&template=dokuwiki" https://adodb.org/dokuwiki/doku.php?id=doku.php

unzip siteexport.zip -d documentation-base

php /path/to/adodb/scripts/fix-static-docs.php

zip -r documentation.zip documentation

Seems to work AFAICT, maybe you want to test and confirm that the output matches what you get with the current process

[dregad]

#882 has a few minor enhancements to the fix-static-docs.php script.

[mnewnham]

I had to add --no-check-certificate to the wget to get it to download straight to my windows box using cygwin bash tools. Also, should be #882 for pull, not #881 (dregad fixed)

[mnewnham]

Can confirm that the wget command plus the fix-static-docs.php can build offline documentation in 2 (and a half steps).

The image "download-big.png" is kept outside of the accessible pages, so we should just keep this as a separate resource and copy it into the final directory tree before zipping it up.

This is so simple we should just automate the whole process. Maybe once a week could be good?

We could also remove the plugins we don't need

[dregad]

Many thanks for looking into it and confirming that it works.

I had to add --no-check-certificate to the wget to get it to download straight to my windows box using cygwin bash tools.

I remember facing this issue on my work laptop. Most likely the certificates authority on your Cygwin is not up-to-date. Did you try installing the ca-certificates package ? It's been a while since I've fixed that issue, and I don't remember how I did it.

If you can't fix it, then I guess that for our purposes it may be acceptable to skip the SSL certificate check.

The image "download-big.png" is kept outside of the accessible pages, so we should just keep this as a separate resource and copy it into the final directory tree before zipping it up.

Weird that this image is not being downloaded, while the Twitter and GitHub icons are.

This is so simple we should just automate the whole process.

Indeed, that was the idea when I tested downloading the site with wget

I'll work on a script to automate the documentation build.

Maybe once a week could be good?

That sounds overkill to me - looking at the history of changes over the past months, it's quite static.
I would go for a build each time we release, and whenever needed (i.e. after significant changes are made)

We could also remove the plugins we don't need

That would be just the sync plugin then ? siteexport we still need.
We can probably disable the XMLRPC (and related adodb-xml user)

[dregad]

The image "download-big.png" is kept outside of the accessible pages, so we should just keep this as a separate resource and copy it into the final directory tree before zipping it up.

Weird that this image is not being downloaded, while the Twitter and GitHub icons are.

I just had a look at this today, and I can see the download-big.png image in the unzipped download directory, as well as the generated documentation directory after running the fix-static-docs.php script.

$ find /tmp/documentation -name 'download-big.png' -ls
 15252082      4 -rw-r--r--   1 dregad   wheel        3247 Mar 12 10:11 /tmp/documentation/download-big.png

The image is correctly rendered when browsing the static html index page. Maybe it was a glitch on your end ?

[dregad]

Comparing the output from the script with the 5.21 documentation zip file, I notice some differences

• page title is different (new version is better)

  • OLD:
  • NEW:

• old version has (non-functional) Edit buttons, which are not present in new version (GOOD)
  

• new version has Dokuwiki toolbar on the right