NEWS for Version 1.17

   Release date: 2020-01-10
   Intended compatibility: WSUS Offline Update 11.8.3

   Bug fixes

    * New method for the calculation of dynamic Office updates

      The existing method for the calculation of Office updates required,
      that the update records in the file package.xml had to be sorted
      in descending order by the RevisionIds.

      This was changed in the WSUS offline scan file wsusscn2.cab from
      December 2019. The result was, that lots of unrelated files were
      downloaded to the directories client/ofc/glb, client/ofc/deu and
      client/ofc/enu.

      The new method does not depend on the correct sort
      order of the file package.xml. It is based on the
      example script extract-office-locations-v2.bash,
      which was suggested in the forum article
      https://forums.wsusoffline.net/viewtopic.php?f=3&t=9954&start=10#p30279

      It is not much tested, though, because I don't have any version
      of Microsoft Office.

      The previous version can be re-enabled again by changing line 47
      of the file download-updates-tasks/60-main-updates.bash from:

         new_method_for_dynamic_office_updates="enabled"

      to:

         new_method_for_dynamic_office_updates="disabled"


NEWS for Version 1.16

   Release date: 2020-01-08
   Intended compatibility: WSUS Offline Update 11.8.3

   Bug fixes

    * The download of wsusscn2.cab indicated success, when the integrity
      check failed

      The integrity of the WSUS offline scan file wsusscn2.cab can be
      tested with cabextract -t. If this test fails, the script would
      still create a file timestamp-wsus-all-glb.txt. This will postpone
      the next download/validation of this task for one day.

      Timestamp files should only be created to indicate successful
      download runs. Then they prevent the repeated evaluation of the
      same task in successive download runs. The next evaluation of the
      WSUS Offline Update configuration files will be postponed for one
      day (21 hours).

      In case of the WSUS offline scan file wsusscn2.cab, a failed
      download should be retried as soon as possible. Therefore, no
      timestamp files should be created for failed download tasks.

    * Static download files in the ../static/custom directory were skipped

      The calculation of static download links ignored custom files
      ../static/custom/StaticDownloadLinks-*.txt, if the original files
      in the ../static directory were all empty.

    * Corrected the spelling of files and directories

      The file 71-make-shapshot.bash was renamed to 71-make-snapshot.bash.

      The noun "licence" is valid British English, but the directory was
      renamed to "license" for consistency with the usage of American
      English in other places.

   User visible changes

    * Revised messages for the update of configuration files

      The messages for the update of the configuration files for WSUS
      Offline Update were revised, to better indicate the different steps.

   Internal changes

    * Better compatibility with FreeBSD 12.1, thanks to TheFlipside

      FreeBSD uses an up to date bash, but there are small differences
      in the core utilities cp, date, mktemp, sed and unzip. Sometimes,
      even the same options don't work the same in GNU/Linux and FreeBSD,
      e.g. sed -i and unzip -u. This has been partially tested with a
      virtual machine from https://www.freebsd.org/where.html .

      Known differences were fixed, but these virtual machines are
      too small and too slow to really run the whole download script,
      so there may be more to be found.

      This was first reported in:

      - modern FreeBSD support
        https://forums.wsusoffline.net/viewtopic.php?f=9&t=9910

   Documentation

    * New file "Differences_between_GNU-Linux_and_FreeBSD.txt"

      This file describes the found differences between the GNU/Linux
      and FreeBSD utilities cp, date, mktemp, sed and unzip. It also
      discusses possible solutions and workarounds.

    * Updated installations guides

      The files Installation_Guide.txt and Installationsanleitung.txt
      were updated to include hints for FreeBSD 12.1.


NEWS for Version 1.15

   Release date: 2019-07-30
   Intended compatibility: WSUS Offline Update 11.8b (r1064)

   New features

    * Automatic update of Sysinternals Utilities

      The archives for the Sysinternals utilities Autologon, Sigcheck and
      Streams are downloaded to the directory wsusoffline/cache. They
      are extracted to the directories wsusoffline/bin and
      wsusoffline/client/bin, when new versions become available.

      This was added in WSUS Offline Update version 11.7.3.

    * The Linux scripts now use sigcheck64.exe on 64-bit hardware

      This was reported by slycordinator:
      https://forums.wsusoffline.net/viewtopic.php?f=9&t=6180&start=50#p28801

      Note however, that running Sigcheck in wine is still flawed: It
      creates too many false results. For example, Sigcheck will report
      a file as "Signed", even if the file has been slightly altered
      by appending a space or changing some bytes with a hexadecimal
      editor. So it seems, that Sigcheck in wine cannot really verify
      digital file signatures. This is most probably a problem with the
      wine library CRYPT32.dll.

    * Support for Windows Server 2019

      Windows Server 2019 was supported by the selection w100-x64 for
      some time. This is now indicated by changing the labels from
      "Windows 10 / Server 2016" to "Windows 10 / Server 2016/2019".

      This was added in WSUS Offline Update version 11.7.3.

    * Support for Internet Explorer 11 on Windows Server 2012

      The installers for Internet Explorer 11 on Windows Server 2012
      are handled similar to the .NET Frameworks: The English installers
      are always downloaded and installed. Other languages are supported
      with language packs.

      This was introduced in WSUS Offline Update version 11.8b,
      changeset 1064:

      http://trac.wsusoffline.net/trac.fcgi/changeset/1064


NEWS for Version 1.14

   Release date: 2019-06-07
   Intended compatibility: WSUS Offline Update 11.7.2+ (r1052)

   New features

    * Support for Trusted Root Certificates and Certificate revocation
      lists

      The support for *.crt and *.crl files was added. Some of these
      files happen to have filenames with spaces. The Linux download
      scripts handle such filenames well, because all variables are
      properly quoted, but in the function cleanup_client_directory,
      filenames must be percent-encoded for comparison with the input
      file StaticDownloadLinks-win-glb.txt.

      These certificate files were introduced in WSUS Offline Update
      11.7.2+ (r1052):

      http://trac.wsusoffline.net/trac.fcgi/changeset/1052

      The option to skip win/glb downloads in the preferences file
      was removed, to make sure that the certificate files will
      be downloaded. If you don't need Silverlight, you could add
      "Silverlight" to the file exclude/custom/ExcludeListForce-all.txt
      instead.

   Internal changes

    * Reordered the positional parameters for the function
      cleanup_client_directory

      Optional parameters should be listed last, to allow them to be
      replaced with standard parameters.

    * Modified the function log_debug_message to better handle multiple
      positional parameters

      The first parameter is considered the debug message. Additional
      parameter are printed on separate lines. This can be used to
      print indexed arrays, without concatenating all elements to a
      single string.

   Documentation

    * Updated files Installation_Guide.txt and Installationsanleitung.txt

      mkisofs/genisoimage are added as recommended packages.


NEWS for Version 1.13

   Release date: 2019-05-14
   Intended compatibility: WSUS Offline Update 11.7

   New features

   New script create-iso-image.bash

      The new script create-iso-image.bash creates ISO images of the
      client directory.

      This script requires either mkisofs from the cdrtools or
      genisoimage from the cdrkit.

        * The cdrtools are the original tools, but they use a
          Solaris-style license, which restricts the distribution of
          binary files. Linux distributions like Gentoo, which provide
          only source files and let the user recompile everything, still
          provide the cdrtools.

          https://en.wikipedia.org/wiki/Cdrtools

        * Most other distributions like Debian and Fedora provide the
          fork cdrkit.

          https://en.wikipedia.org/wiki/Cdrkit

      The usage is:

      ./create-iso-image.bash <update> [<option> ...]

      The first parameter is the profile name. It can be one of:

      all           All Windows and Office updates, 32-bit and 64-bit
      all-x86       All Windows and Office updates, 32-bit
      all-win-x64   All Windows updates, 64-bit
      all-ofc       All Office updates, 32-bit and 64-bit
      wxp           Windows XP, 32-bit                   (ESR version only)
      w2k3          Windows Server 2003, 32-bit          (ESR version only)
      w2k3-x64      Windows XP / Server 2003, 64-bit     (ESR version only)
      w60           Windows Vista / Server 2008, 32-bit
      w60-x64       Windows Vista / Server 2008, 64-bit
      w61           Windows 7, 32-bit
      w61-x64       Windows 7 / Server 2008 R2, 64-bit
      w62           Windows 8, 32-bit                    (ESR version only)
      w62-x64       Windows 8 / Server 2012, 64-bit
      w63           Windows 8.1, 32-bit
      w63-x64       Windows 8.1 / Server 2012 R2, 64-bit
      w100          Windows 10, 32-bit               (current version only)
      w100-x64      Windows 10 / Server 2016, 64-bit (current version only)

      The options of the script are:

      -includesp         Include service packs
      -includecpp        Include Visual C++ Runtime Libraries
      -includedotnet     Include .NET Frameworks
      -includewddefs     Include Windows Defender virus definitions for
                         the built-in Defender of Windows Vista and 7.
      -includemsse       Include Microsoft Security Essentials. The
                         virus definitions are also used for the built-in
                         Defender of Windows 8, 8.1 and 10.
      -output-path <dir> Output directory for the ISO image file
      -create-hashes     Create a hashes file of the ISO image file

      The script create-iso-image.bash is meant to support both the
      current and the ESR version of WSUS Offline Update, but some
      updates are only available in the current version and vice versa.

      The distinction is the presence of the filter files
      ExcludeListISO-*.txt and the download directories: Windows XP is
      not supported by the current version, because neither the filter
      file ExcludeListISO-wxp.txt nor the download directory client/wxp
      can be found. The profile all, which only removes three unneeded
      files, can be used with the ESR version, though.

      There are basically three modes of operation:

       1. The profile all creates one ISO image of the whole client
          directory. It is left to the user to restrict the resulting
          ISO image to a reasonable size.

       2. The profiles all-x86 and all-win-x64 create two cross-product
          ISO images, one per architecture. Originally, these ISO images
          were meant to fit on DVD-5 media with a size of 4.7 GB.

       3. The profiles all-ofc, w60, w60-x64, w61, w61-x64, w62-x64,
          w63, w63-x64, w100 and w100-x64 create one ISO image per
          product. Originally, these ISO images were meant to fit on
          CD-ROM media with a size of 700 MB.

       4. The distinction per language is not used anymore, because all
          Windows versions since Vista use global/multilingual updates,
          and all Office updates are just lumped together anyway.

      Of course, the size restrictions to 700 MB or 4.7 GB don’t work
      anymore. It may explain, though, why large installers are
      sometimes excluded from the created ISO images.

      The Linux script create-iso-image.bash uses its own filter files
      in the sh/exclude directory. These files are based on the files in
      the WSUS Offline Update directory wsusoffline/exclude, but the
      syntax has been reviewed: Many shell patterns seem to be unneeded.
      The syntax for paths seems to be slightly different, although
      basically the same tools are used: mkisofs.exe on Windows, and
      mkisofs or genisoimage on Linux.

      The file About_the_ExcludeListISO-files.txt in the directory
      sh/exclude explains the translation of the ExcludeListISO-*.txt
      files from Windows to Linux.

      Users may create copies of the ExcludeListISO-*.txt files in the
      directory sh/exclude/local. These local copies replace the
      supplied files. This is slightly different from the handling of
      custom files by WSUS Offline Update, but it is more the Linux way,
      and it allows to both add and remove filters.

      The created ISO images have filenames like
      2019-04-14_wsusoffline-11.6.2_all.iso. These are composed of:

       1. The build date from the file wsusoffline/client/builddate.txt,
          which indicates the last run of the download script

       2. The name wsusoffline

       3. The WSUS Offline Update version

       4. The used profile

   New file catalog-creationdate.txt

      The attribute CreationDate is extracted from the WSUS update
      catalog file package.xml and saved to the file
      wsusoffline/client/catalog-creationdate.txt.

        * Forum: https://forums.wsusoffline.net/viewtopic.php?f=3&t=8997

   Internal changes

      Support for .NET Framework 4.8 was added.

      The debug output for the download of virus definition files,
      which was enabled in version 1.12, was removed again.

      Revised function remove_obsolete_files

   Documentation

      New file exclude/About_the_ExcludeListISO-files.txt


NEWS for version 1.12

   Release date: 2019-04-10
   Intended compatibility: WSUS Offline Update 11.6.1+ (r1032)

   Internal changes

   Indirect addressing of virus definition files with "LinkIDs"

      The four virus definition files are now referenced with URLs,
      with refer to LinkIDs first. The filename on the server can only
      be retrieved after several redirections.

      Several functions had to be adjusted to work with these LinkIDs:
      calculate_static_downloads_wddefs8, download_single_file,
      download_single_file_failsafe, and cleanup_client_directory.

      Note: Some debug messages in these functions have been left
      enabled, to see, if anything changes. They will be removed in the
      next versions.

        * Forum: https://forums.wsusoffline.net/viewtopic.php?f=6&t=9098
        * Trac:  http://trac.wsusoffline.net/trac.fcgi/changeset/1032


NEWS for version 1.11

   Release date: 2019-04-04
   Intended compatibility: WSUS Offline Update 11.6.1 and later

   New features

   The script update-generator.bash remembers the last used settings

      The script update-generator.bash writes the current settings to a
      settings file update-generator.ini, if the external utility
      “dialog” is used. On the next run, the settings are reloaded and
      the previously selected options are checked again.

      Note, that this does not work with the internal command “select”
      of the bash.

   Revised method for calculating superseded updates

      An optional, new method for the calculation of superseded updates
      removes one possible cause for missing updates: It automatically
      corrects the list of superseded updates for updates, which are
      only superseded by full quality update rollups, but not by
      security-only updates. This may prevent some rare problems with
      missing updates.

      The current implementation for calculating superseded updates in
      both Windows and Linux is depicted in the forum article:

        * https://forums.wsusoffline.net/viewtopic.php?f=5&t=5676

      One problem with this implementation is, that superseded updates
      may be missing, if the superseding updates are excluded from
      download.

      This is an old problem, which was observed a few times:

      It was first found with Windows XP: The embedded Windows XP
      POSReady was supported longer than the regular desktop versions.
      Updates for the embedded version can supersede older updates for
      the desktop versions. The newer updates for the embedded version
      are not downloaded by WSUS Offline Update, because they cannot be
      installed on the desktop versions. But the older updates for the
      desktop versions are still treated as superseded and not
      downloaded either. This was solved by adding the missing updates
      to the file ExcludeList-superseded-exclude.txt.

      When monthly quality update rollups and security-only updates were
      first introduced, the quality update rollups superseded the
      security-only updates. WSUS Offline Update needed two steps to
      support security-only updates:

        * The quality update rollups had to be excluded from download
          and installation.

        * The security-only updates had to be re-enabled for download.

      At this point, I first suggested a new method, which could
      reschedule superseded updates for download, if the superseding
      updates are excluded from download. The problem with this method
      was, that it needed an initial block list of excluded downloads to
      start with, and this list didn't exist yet. So it was only a
      partial solution.

        * https://forums.wsusoffline.net/viewtopic.php?f=5&t=6141

      The initial release of the Linux download scripts, version
      1.0-beta-1, actually included an implementation of this new method
      in the subdirectory available-tasks.

      The differentiation of quality update rollups and security-only
      updates was finally solved in WSUS Offline Update by creating new
      configuration files in the client/static and client/exclude
      directories.

      In the meantime, Microsoft changed the way, how quality update
      rollups and security-only updates depend on each other, after just
      one month:

        “UPDATED 12/5/2016: Starting in December 2016, monthly rollups
        will not supersede security only updates. The November 2016
        monthly rollup will also be updated to not supersede security
        only updates.”

        https://techcommunity.microsoft.com/t5/Windows-Blog-Archive/More-on-Windows-7-and-Windows-8-1-servicing-changes/ba-p/166783

      A more subtle problem is, that the quality update rollups
      sometimes seem to include older updates, while the security-only
      updates only include new updates for the current month. Then these
      older updates will be missing, if security-only updates are
      selected. Again, this was solved by adding the missing updates to
      a new configuration file
      ExcludeList-superseded-exclude-seconly.txt.

      The problems with the files ExcludeList-superseded-exclude.txt and
      ExcludeList-superseded-exclude-seconly.txt is, that they can only
      be updated, after some updates have been found missing.

      When I retested the new method, it could detect two missing
      updates, before they were reported as missing:

        * https://forums.wsusoffline.net/viewtopic.php?f=4&t=7085
        * https://forums.wsusoffline.net/viewtopic.php?f=2&t=8697

      So, this method may still be useful, and the script
      60-main-updates.bash now includes an optional implementation. It
      uses the file HideList-seconly.txt as an initial block list, to
      automatically correct the list of superseded updates for updates,
      which are only superseded by the monthly quality update rollups,
      but not by security-only updates.

      To select the new file
      ExcludeList-Linux-superseded-seconly-revised.txt for the
      calculation of dynamic updates, both options prefer_seconly and
      revised_method must be set to “enabled” in the preferences file.

   New script open-support-pages.bash

      This script opens the Microsoft support pages for a series of kb
      numbers.

      It tries a series of Linux “open handlers”, to open the URLs with
      the preferred application of the desktop environment. Suitable
      open handlers are:

       Open handler      Package name        Desktop environment
       gio open          libglib2.0-bin      GNOME 3.30 in Debian 10
       gvfs-open         gvfs-bin            GNOME 3.22 in Debian 9
       gnome-open        libgnome2-bin       GNOME 2
       kde-open5         kde-cli-tools       KDE 5 (untested)
       kde-open          kde-runtime         KDE 4 (untested)
       exo-open          exo-utils           Xfce
       xdg-open          xdg-utils           others

       (The package names are for Debian and related distributions.)

      In addition to these open handlers, sensible-browser, firefox-esr
      and firefox are also tried, because this script only needs to
      handle http or https URLs. The script /usr/bin/sensible-browser is
      part of the update-alternatives system in Debian. It uses
      gnome-www-browser, x-www-browser or www-browser, depending on the
      context.

      If none of the above can be found, then the script recommends the
      installation of xdg-open as a general open handler, which is not
      tied to a particular desktop environment.

      Note: Neither gvfs-open nor Firefox can handle multiple URLs on
      the command line. Calling xdg-open repeatedly usually means that
      the application should be launched multiple times, but this will
      fail with Firefox. To have Firefox open the URLs in multiple tabs,
      it should be launched first, before running this script.

   Bug fixes

   Workaround for broken cabextract in Debian 10 Buster/testing

      As of 2019-03-26, the package cabextract is still broken in Debian
      10 Buster/testing, long after two relevant bug reports have been
      marked as fixed and closed.

        * libmspack0: Regression when extracting cabinets using -F
          option fixed upstream, needs to be patched
          https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=912687

        * cabextract: -F option doesn't work correctly
          https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914263

      But so far, the fixed packages only arrived in Debian
      Sid/unstable. They are not yet available in Debian Buster/testing.

      So I added some more tests and a workaround, which does not use
      the cabextract option -F. Then the file wsusscn2.cab must be
      completely unpacked, which may take slightly longer.

        * Forum: https://forums.wsusoffline.net/viewtopic.php?f=9&t=8706

   User visible changes

   Better support for terminal colors

      Version 1.4 of the Linux download scripts introduced text
      formatting of the output, using bold text and terminal colors.
      Such text formatting should only be used, if the output is written
      to a terminal emulator window or to a virtual console. It should
      not be used, if the output is redirected to a file or piped to
      another application, for example if the script is running as a
      cron job.

      The library messages.bash uses two tests, to make sure that text
      formatting can used safely:

        * First, the script tests, if standard output and error output
          are attached to a terminal, using the test -t of POSIX shells.

        * Then all escape sequences are determined with the utility
          tput, rather than hard-coding them. tput checks again, if text
          formatting is safe to use.

      But tput is overly restrictive in the use of terminal colors: It
      only uses colors, if the environment variable TERM is set to
      “xterm-256color”, “rxvt-256color” or “rxvt-unicode-256color”.
      Otherwise, only bold text is used.

      Many terminal emulators simply set TERM to “xterm”, and they don't
      provide any means to change this environment variable. Still, all
      tested terminal emulators support colors, including xterm itself.
      Then it should be safe to change TERM from xterm or xterm-color to
      xterm-256color, to get the expected results.

      The same adjustments could be done with rxvt, but Debian already
      provides different builds with different settings for rxvt. The
      urxvt from the package rxvt-unicode-256color sets the environment
      variable TERM to “rxvt-unicode-256color”, which is recognized as a
      color-capable terminal by tput.

      Notes:

      There are other ways to set or change the environment variable
      TERM. This may benefit other applications as well. For example,
      some themes for the Midnight Commander also require 256 colors.

      Within a shell, environment variables can be defined before the
      script or application to run:

      ~$ TERM=xterm-256color ./update-generator.bash

      Some terminal emulators like the MATE Terminal allow to run a
      custom command instead of the standard shell. This can be used to
      set environment variables with:

      /usr/bin/env TERM=xterm-256color bash

      The environment variable TERM could also be set in files like
      ~/.profile or ~/.bashrc, but this may give unexpected results:
      Using TERM=xterm in the Linux console will mess up the output of
      the external utility “dialog”, because the box drawing characters
      are different.

   Support for ExcludeListForce-all.txt

      The custom file
      wsusoffline/exclude/custom/ExcludeListForce-all.txt was already
      applied to static and dynamic updates for Windows, Office and .NET
      Frameworks. It is now applied to the .NET Framework installation
      files as well.

        * Forum: https://forums.wsusoffline.net/viewtopic.php?f=2&t=8901
        * Trac: http://trac.wsusoffline.net/trac.fcgi/changeset/1015

   Support for ExcludeList-superseded-exclude-seconly.txt

      The custom file
      wsusoffline/exclude/custom/ExcludeList-superseded-exclude-seconly.txt
      will be used for the calculation of superseded updates, if
      security-only updates are selected.

        * Forum: https://forums.wsusoffline.net/viewtopic.php?f=2&t=8697
        * Trac: http://trac.wsusoffline.net/trac.fcgi/changeset/1002

   Virus definition files are tested with cabextract

      The four virus definition files are basically self-extracting
      cabinet files. They can be at least partially tested with
      cabextract -t.

   The scripts compare-integrity-database.bash and
   compare-update-tables.bash are configured with command-line arguments

      These scripts are used for development: They compare the
      directories md and ofc on Windows and Linux. Previously, these
      directories had to be edited directly in the scripts, but they are
      now passed as command-line arguments.

   Internal changes

   Reordered the sections at the top of the scripts update-generator.bash
   and download-updates.bash

      The new order in these files is:

       1. Shell options

       2. Environment variables

       3. Configuration: script version and release date

       4. Global variables: script name and home directory, other
          directories and log file

       5. Preferences: default values for the settings in the optional
          preferences file

       6. Traps

       7. Functions

       8. Commands

   The directories cache, log and timestamps are stored with absolute paths

      The directories cache, log and timestamps used to be defined with
      the relative paths ../cache, ../log and ../timestamps. This does
      not work well with external utilities like hashdeep and curl,
      which require constant changes to the current working directory
      and thereby make relative references invalid.

      After revealing the current working directory with readlink, these
      directories are now stored with absolute paths.

   The function seconly_safety_guard calculates the patch days for two
   months

      The function seconly_safety_guard tries to make sure, that some
      configuration files in the directories client/exclude and
      client/static have been updated after each patch day, before
      downloading security-only updates. Without this configuration,
      WSUS Offline Update would default to download and install the full
      quality update rollups, or it might even download and install both
      sets of updates.

      The first implementation only calculated the patch day of the
      current month, and compared the modification date of the
      configuration files to this date.

      Now the function calculates the official patch days for the last
      two months, and selects the right one for comparison:

        * The last patch day is the second Tuesday of the current month,
          if today is on the same day or later.

        * Otherwise, the last patch day is the second Tuesday of the
          last month.

      These calculations also use an integer value for the day of the
      week, instead of comparing the weekday names literally.

   The function log_message duplicates messages to the terminal and the log
   file unchanged

      The function log_message does not prefix the message with the
      current date anymore. It is now used to duplicate the output of
      hashdeep to the terminal and to the log file.

   The function apply_exclude_lists now skips empty lines in input files

      The function apply_exclude_lists now reads the input files
      line-by-line. Empty lines are ignored. This should prevent errors
      with files, which only consist of one empty line.

   New function name_to_description and changed function
   language_name_to_locale

      The new function name_to_description checks, that the specified
      update, language or option name exists in a table and returns the
      description. This replaces some awkward constructs with grep.

      The function language_name_to_locale was rewritten in the same way
      to return the locale for a language name, e.g. deu → de, enu → en.

   The file sh/exclude/ExcludeListUSB-w60.txt was simplified

      The different filters vcredist* were replaced with a single filter
      *_x64*.

        * Forum: https://forums.wsusoffline.net/viewtopic.php?f=5&t=8258
        * Trac: http://trac.wsusoffline.net/trac.fcgi/changeset/989

   Added Cygwin to known systems and unzip to needed packages, thanks to
   "slycordinator"

      Cygwin is basically a Linux environment on Windows. All needed GNU
      utilities and other packages should be available. trash-cli can be
      installed from the Python repository with "pip install trash-cli".

      unzip is needed for the self-update of the WSUS Offline Update
      installation and to unpack the Sysinternals utilities Autologon and
      Sigcheck. unzip is often installed with graphical archive managers
      like Xarchiver, but it may be missing on a basic command-line
      system.

   Documentation

      The new file compatibility.txt lists the distributions, on which
      the Linux scripts were at least briefly tested, and the
      corresponding Bash versions.

      Some typos were corrected.

      The E-Mail address was replaced.


NEWS for Version 1.10

  Release date: 2018-08-09
  Intended compatibility: WSUS Offline Update 11.4 and later

  Changes in this version

  - Bug fix: USB filters for w61 and newer erroneously excluded ndp46-*
    patches.

    The private copies of the files ExcludeListUSB-*.txt
    were patched according to Changeset 981:
    http://trac.wsusoffline.net/trac.fcgi/changeset/981

    Actually, in the files ExcludeListISO-*.txt, only
    the lower-case filter *ndp46-* was removed, while the
    upper-case filter *NDP46-* is kept. This excludes the two
    installation files NDP46-KB3045557-x86-x64-AllOS-ENU.exe and
    NDP46-KB3045557-x86-x64-AllOS-DEU.exe, but not the dynamic updates
    for .NET Framework 4.6.

    A possible reason for this distinction is to keep the size of ISO
    images below 4.7 GB. Excluding large installers may be needed to
    reach this goal.

    This distinction is not used for the ExcludeListUSB-*.txt files,
    because the filters for xcopy.exe are not case-sensitive, and copying
    to an external USB device does not have the same limitations for
    the overall file size as creating ISO images.

  - The script 10-show-selection-dialogs-with-dialog.bash was made
    more configurable, by moving the variable parts of the dialogs to
    a configuration section.

  - Small corrections to the installation guide.


NEWS for Version 1.9

  Release date: 2018-07-30
  Intended compatibility: WSUS Offline Update 11.4 and later

  Changes in this version

  - The script update-generator.bash now uses the external utility
    "dialog", to display nicely formatted dialogs in the terminal window.

    All three dialogs for updates, languages and optional downloads
    allow multiple selections. This allows to get all needed updates
    with a single call of the download script download-updates.bash.

    The existing script, which uses the internal command "select" of
    the bash, is kept as a fallback, if dialog is not installed. This
    command only allows single selections.

    The new script 10-show-selection-dialogs-with-dialog.bash is based
    on a mockup, which I once created for the now obsolete script
    DownloadUpdates.sh:
    http://forums.wsusoffline.net/viewtopic.php?f=9&t=4061


NEWS for Version 1.8

  Release date: 2018-07-27
  Intended compatibility: WSUS Offline Update 11.4 and later

  Changes in this version

  - The documentation of the Linux download scripts is organized more
    like the Linux documentation in the directory /usr/share/doc.

    The file NEWS.txt replaces the former release_notes_[version].txt. It
    contains detailed information about the last five versions in
    reverse order.

    The file changelog.txt replaces the former version-history.txt. It
    contains a summary of the changes for all versions in reverse order.

  - The file 70-synchronize-with-target.bash was replaced with a more
    elaborate script copy-to-target.bash.

    The usage is:

    ./copy-to-target.bash <update> <destination-directory> [<option> ...]

    The <update> can be one of:

    all           All Windows and Office updates, 32-bit and 64-bit
    all-x86       All Windows and Office updates, 32-bit
    all-win-x64   All Windows updates, 64-bit
    all-ofc       All Office updates, 32-bit and 64-bit
    wxp           Windows XP, 32-bit                    (ESR version only)
    w2k3          Windows Server 2003, 32-bit           (ESR version only)
    w2k3-x64      Windows XP / Server 2003, 64-bit      (ESR version only)
    w60           Windows Vista / Server 2008, 32-bit
    w60-x64       Windows Vista / Server 2008, 64-bit
    w61           Windows 7, 32-bit
    w61-x64       Windows 7 / Server 2008 R2, 64-bit
    w62           Windows 8, 32-bit                     (ESR version only)
    w62-x64       Windows 8 / Server 2012, 64-bit
    w63           Windows 8.1, 32-bit
    w63-x64       Windows 8.1 / Server 2012 R2, 64-bit
    w100          Windows 10, 32-bit                (current version only)
    w100-x64      Windows 10 / Server 2016, 64-bit  (current version only)

    The available options for the update parameter are determined by
    the installed files wsusoffline/exclude/ExcludeListUSB-*.txt, which
    are meant for the Windows script CopyToTarget.cmd. There is a strict
    one-to-one relationship:

    Update        Used exclude list
    ------        -----------------
    all           ExcludeListUSB-all.txt
    all-x86       ExcludeListUSB-all-x86.txt
    all-win-x64   ExcludeListUSB-all-x64.txt
    all-ofc       ExcludeListUSB-ofc.txt
    wxp-x86       ExcludeListUSB-wxp-x86.txt   (ESR version only)
    w2k3          ExcludeListUSB-w2k3-x86.txt  (ESR version only)
    w2k3-x64      ExcludeListUSB-w2k3-x64.txt  (ESR version only)
    w60           ExcludeListUSB-w60-x86.txt
    w60-x64       ExcludeListUSB-w60-x64.txt
    w61           ExcludeListUSB-w61-x86.txt
    w61-x64       ExcludeListUSB-w61-x64.txt
    w62           ExcludeListUSB-w62-x86.txt   (ESR version only)
    w62-x64       ExcludeListUSB-w62-x64.txt
    w63           ExcludeListUSB-w63-x86.txt
    w63-x64       ExcludeListUSB-w63-x64.txt
    w100          ExcludeListUSB-w100-x86.txt  (current version only)
    w100-x64      ExcludeListUSB-w100-x64.txt  (current version only)

    The Windows script CopyToTarget.cmd uses xcopy.exe, and the exclude
    lists had to be edited to work with rsync on Linux. Therefore,
    the Linux script now uses its own set of these files. Some files
    are also renamed to match the names of the command-line parameters.

    The destination directory is the directory, to which files are copied
    or hard-linked. It should be specified without a trailing slash,
    because otherwise rsync may create an additional directory within
    the destination directory.

    The options are:

    -includesp         Include service packs
    -includecpp        Include Visual C++ Runtime Libraries
    -includedotnet     Include .NET Frameworks
    -includewddefs     Include Windows Defender virus definitions for
                       the built-in Defender of Windows Vista and 7.
    -includemsse       Include Microsoft Security Essentials. The virus
                       definitions are also used for the built-in Defender
                       of Windows 8, 8.1 and 10.
    -cleanup           Tell rsync to delete obsolete files from included
                       directories. This does not delete excluded files
                       or directories.
    -delete-excluded   Tell rsync to delete obsolete files from included
                       directories and also all excluded files and
                       directories. Use this option with caution,
                       e.g. try it with the option -dryrun first.
    -hardlink <dir>    Create hard links instead of copying files. The
                       link directory should be specified with an
                       absolute path, otherwise it will be relative to
                       the destination directory. The link directory
                       and the destination directory must be on the same
                       file system.
    -dryrun            Run rsync without copying or deleting
                       anything. This is useful for testing.

    The operation "per language" is not supported, because it is not
    needed anymore. It was useful for Windows XP and Server 2003, because
    these Windows versions had localized updates. But all Windows versions
    since Vista have global/multilingual updates, and Office updates
    are all lumped together, with most updates in the ofc/glb directory.

    There are two known differences in the results between the Windows
    script CopyToTarget.cmd and the new Linux script copy-to-target.bash
    ( see http://forums.wsusoffline.net/viewtopic.php?f=5&t=8258 ):

    1. The original file wsusoffline/exclude/ExcludeListUSB-w60-x86.txt
       misses an entry for vcredist2017_x64.exe. This means, that this
       file is not excluded by the Windows script, if the update "w60"
       is selected.

    2. The file wsusoffline/client/bin/IfAdmin.cpp is only
       excluded by the Windows script CopyToTarget.cmd, if
       the option /includedotnet is NOT used. Then the file
       wsusoffline/exclude/ExcludeListISO-dotnet.txt is appended to the
       filter file. With xcopy.exe, the line "cpp\" matches both the
       directory "cpp" (as expected) and the source file "IfAdmin.cpp".

       But the file IfAdmin.cpp is neither needed for download nor for
       installation, and it should always be excluded. It is only included
       in WSUS Offline Update, because the GPL demands, that the source
       code of all utilities should be made available somewhere.

  Internal changes

  - The definition of the environment variables LINES and COLUMNS, and of
    the terminal colors was moved from the scripts update-generator.bash
    and download-updates.bash to the library messages.bash.

    These variables are only used in the library messages.bash, and then
    they should be defined there. The library messages.bash also provides
    standard values for the global variables logfile and debug, to make
    the library more self-contained and suitable for other scripts.

  - Hashdeep errors while checking the integrity of existing files are
    now reported as errors, not as warnings.

    Before each download run, the integrity of existing files is verified
    with hashdeep. As a forensic tool, hashdeep treats all changes to the
    examined directory as errors, including the manual removal of files.

    There are rarely real problems at this point, and the fix is
    to delete the corresponding hashdeep files in the directory
    wsusoffline/client/md. They will be rebuilt on the next download run.

    But this is actually the normal progress: The hashdeep files will
    be deleted and rebuilt after each download run anyway. Therefore,
    hashdeep errors for the verification of existing files were only
    reported as "warnings".

    Hashdeep errors for existing files still increment an internal counter
    for runtime errors, and for consistency they are now reported as
    "errors".

  - Corrected the copyright year of the files error-counter.bash and
    rebuild-integrity-database.bash to "2018".

    These files were added in 2018, but due to lazy copy-and-past the
    copyright was set to 2016-2018.


Release Notes for Version 1.7

  Release date: 2018-05-25
  Intended compatibility: WSUS Offline Update Version 11.3 and later

  Changes in this version

  - Bug fix: The script download-updates.bash may crash, if running in
    bash version 4.3, for example as of Debian 8 Jessie, and only Office
    Updates are selected.

    While parsing the command-line parameters, the script
    download-updates.bash creates four internal lists for the needed
    updates, architectures, languages and included downloads.

    The list of architectures will be empty, if only Office updates are
    selected. This list determines the architectures of the included
    downloads .NET Frameworks, Windows Defender and Microsoft Security
    Essentials. These downloads should match the specified Windows
    versions, not the Office versions.

    There is an old bug in bash up to version 4.3: Empty arrays are
    treated as "unset", even if the array variables are declared and
    initialized. This bug is solved in bash 4.4, as of Debian 9 Stretch.

    * bash: nounset treats empty array as unset, contrary to man. page
      https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=529627

    For the script this means, that the length of the array must be
    checked, before it is read. This has already been done in most places,
    but it had to be added to the function print_command_line_summary.

    This solves the bug reports
    http://forums.wsusoffline.net/viewtopic.php?f=9&t=8072 and
    http://forums.wsusoffline.net/viewtopic.php?f=9&t=8090 .


Release Notes for Version 1.6

  Release date: 2018-05-04
  Intended compatibility: WSUS Offline Update Version 11.3 and later

  Changes in this version

  - Added support for .NET Framework 4.7.2

  - Bug fix: The function create_integrity_database did not create a
    hashes file, if the hashed directory was a symbolic link.

    To prevent the creation of empty hashdeep files, the function
    create_integrity_database counts the number of files in the hashed
    directory. This was done by using "find" and "wc", but find by
    default does not follows symbolic links. The new implementation lets
    the bash itself calculate the file count.
