Revision control
+ # Dependency Update Utility
+
+ ### What is this for?
+
+ *This utility replaces the script at qt/qtqa/src/qtmoduleupdater*
+
+ Every submodule of qt6 includes a "dependencies.yaml" file which
+ specifies a revision of the other modules it directly depends on.
+ If checked out together, the given module should be guaranteed to
+ build and pass tests. In this model, a "dependency tree" forms,
+ where "leaf" modules depend on "trunk" or "root" modules.
+
+ In order to update a leaf module's dependencies, the tree must be
+ walked down to the root-most module it depends on which itself
+ has no dependencies. Then, starting from the root (often qtbase),
+ each module's dependencies.yaml file is updated with the latest SHA.
+ When a trunk module passes CI after being updated with new root SHAs,
+ leaf modules can be updated with the latest trunk SHAs.
+
+ ### How does it work?
+ This script/bot takes a list of repositories, either explicitly or
+ gathered from qt5.git's .gitmodules file, and discovers the
+ dependency tree by cross-checking every module's dependencies.yaml file.
+ During this process, it collects the latest SHAs for each module, and
+ also checks for modules which need to be updated due to inconsistent
+ SHAs across modules for the same dependency. **Further, if a dependency
+ is discovered that was not explicitly passed to the script, it is
+ added to internal memory and will be updated as well if necessary.**
+
+ The script provides a number of modes of operation. See the script's
+ usage manual below.
+
+ In typical usage, the script will need to be run multiple times to
+ complete a round, as it creates change requests on
+ codereview.qt-project.org which must be integrated. When a tracked
+ change is integrated, the script should be re-run to progress the
+ update round. When all required module changes are successfully
+ integrated, if run with options to do so, the script will perform
+ an update to qt5.git and/or yocto/meta-qt6 with the updated submodule
+ SHAs collected during the round. When a round completes, the state
+ is cleared and a new round will begin when the script is run again.
+
+ Note: It is safe to run the script on a timer/trigger. Since the script
+ keeps track of actions it is taking, it should never duplicate work
+ if a dependency update is in-progress. If there is no work to be done,
+ such a message will be printed and the script will exit.
+
+ ### Usage
+ A number of python modules are required. Usage of a Python virtual
+ environment such as pipenv is strongly suggested.
+
+ ```
+ pipenv install
+ pipenv run python3 main.py [args]
+ ```
+
+ See the script's --help output for a full list of options. The
+ example scenarios below cover some of the most commonly used situations.
+
+ Note that most scenarios require the utility to be run multiple times
+ to complete an update "round" since multiple modules in the dependency
+ tree often need to be updated separately, and in a specific order.
+
+ 1. Clean and clear the utility's state and forget previous runs for a branch.
+ 1. `python3 main.py --branch dev --reset`
+ 2. This action clears state for the specified branch. You should always
+ clear the working state before starting a fresh round/operation.
+ 3. Note that simply deleting the local __state/state.bin_ file will
+ not necessarily clear the state. This utility also stores state
+ data in codereview personal branches if ssh credentials are configured
+ in ~/.ssh/config. Always run the utility with `--reset` to clear state.
+
+ 2. Run the utility to perform a one-time sync of a module with qt/qt5.git
+ submodule SHAs.
+ 1. `python3 main.py --branch dev --noState --repos
+ qt-extensions/qthttpserver`
+ 2. This creates a new change for `qt-extensions/qthttpserver` in
+ codereview which needs to go through the normal review process.
+ The dependencies.yaml file is updated to the latest SHAs in qt5.git
+ 3. This scenario does not usually require more than a single run of
+ the tool. As such, `--noState` can be used to avoid the need to
+ reset the utility before running. This also prevents interference to
+ any ongoing rounds since no state data is written.
+
+ 3. Run the utility to simulate an action
+ 1. `python3 main.py --sim --branch dev --repos qt-extensions/qthttpserver`
+ 2. Performing a dry-run prints to the console which actions the script
+ would take. No actions are actually performed on gerrit, and no
+ changes will be created. Further, using --sim will never update the
+ local persistent state of the tool.
+
+ 4. Perform an update to one or more repos using the latest available
+ SHAs from each dependency's own repo.
+ 1. `python3 main.py --head --branch dev --repos qt-extensions/qthttpserver`
+ 2. This assembles a full dependency map for the given repo(s) and updates
+ each one, starting from the most base module.
+ 3. Performing an update with `--head` should be used with a clean state.
+ 4. When the first updates are merged, run the utility again with the same
+ arguments to continue the "round".
+ 5. Continue to run the utility repeatedly to progress the round until
+ the target module is updated with new SHAs for its dependencies.
+
+ 5. Perform an update for one or more modules, then update qt/qt5.git and
+ yocto/meta-qt6 with the new module SHA(s).
+ 1. `python3 main.py --head --branch dev --qt5Update --yoctoUpdate --repos
+ qt-extensions/qthttpserver`
+ 2. Performs as (4) above, but when the target module(s) have been updated,
+ a further run of the utility will update qt/qt5.git and/or
+ yocto/meta-qt6.git with the merged SHAs of the target modules. Only
+ modules which already exist in the super-repos will be updated.
+
+ 6. Rewind an ongoing round to a specific dependency to pull in additional
+ changes and continue the update round.
+ 1. `python3 main.py --head --branch dev --rewind qt/qtdeclarative --repos
+ qt-extensions/qthttpserver`
+ 2. In this example, if the round has already successfully merged an update
+ for _qt/qtdeclarative_ but _qt/qtwebsockets_ is broken until a further
+ fix can be picked up in _qt/qtdeclarative_, using
+ `--rewind qt/qtdeclarative` will clear the state for modules which
+ depend on _qt/qtdeclarative_ and pull the new head of _qt/qtdeclarative_,
+ then continue from there.
+
+ 7. Remove a dependency from a module
+ 1. `python3 main.py --head --branch dev --dropDependency
+ qt/qtsvg:qt-extensions/qthttpserver --repos qt-extensions/qthttpserver`
+ 2. If a repo no longer needs a dependency, it can be removed in this way.
+ 3. Combine `--dropDependency` with `--qt5Update` to remove a dependency
+ from a module and ensure that qt/qt5.git's .gitmodules reference file
+ also gets updated appropriately.
+ 4. _**Note:**_ This action is destructive! See the `--help` output for
+ important information and detailed usage instructions
+
+ 8. Update all current modules in qt/qt5.git
+ 1. `python3 main.py --default-repos --branch dev --qt5Update --yoctoUpdate`
+ 2. This collects a list of all modules in qt5 marked as 'essential',
+ 'addon', 'deprecated', 'ignore', or 'preview' in the _.gitmodules_
+ file of _qt/qt5.git_ and updates the dependency tree to the latest
+ branch HEAD of each module.
+ 3. When finished, qt/qt5.git and yocto/meta-qt6 are updated with the
+ SHAs of all modules updated in the round
+
+ 9. Include in a round additional repos/modules which should be considered
+ "non-blocking" by the utility.
+ 1. `python3 main.py --default-repos --branch dev --qt5Update --yoctoUpdate
+ --nonBlockingRepos qt-extensions/qthttpserver`
+ 1. This will perform an update round as requested, but if a non-blocking
+ module update fails, it will be ignored for the rest of the round,
+ allowing all other repos to continue normally.
+ 2. If any repos specified by `--repos` or gathered automatically by
+ `--default-repos` require a repo specified as non-blocking, that
+ non-blocking repo will be converted to a blocking-status since
+ failure in it would lead to the failure of a normally blocking
+ module.
+
+ 11. Auto-approve and stage module updates
+ 1. `python3 main.py --stage --default-repos --branch dev --qt5Update
+ --yoctoUpdate`
+ 2. If the user running the tool has provided codereview credentials which
+ have Approver and QtStage rights access, the utility will self-approve
+ created changes and automatically stage them.
+ 3. **_Note:_** Use of this option is generally discouraged outside of
+ automation.
+
+
+ ### Further Notes
+ - The script may take some time to run, depending on internet speed,
+ responsiveness of Gerrit, and how many repositories it is updating.
+
+ - Unless run with `--noState`, a run of the script will attempt to use
+ saved information about repositories and their dependencies/SHAs.
+ If you experience trouble when trying to start a round, try running
+ the script with --reset to clear data for the target branch before
+ continuing.
+
+ - Use of `--noState` is intended to allow the user to perform atomic
+ operations without the need to reset or save state, potentially
+ contaminating an ongoing round for a given branch. Generally,
+ this would be used when syncing a repo's dependencies.yaml
+ with qt5.git's current submodule SHAs, as this usually does not
+ require more than one run of the script.
+
+ - `--sweepChanges` will sweep which have the script's GERRIT_USERNAME
+ as a reviewer on the change. In practice, this option is usually
+ reserved for the Qt Submodule Update bot, but can be enabled ***if you
+ know what you are doing.***
+
+ - Repo prefixes are fuzzy. The script defaults to "qt/", but in theory
+ any prefix can be used. If you set the prefix to "playground/", the
+ script will prefer dependencies which exist there. If a dependency
+ cannot be found in the preferred namespace, a fuzzy-match search is
+ performed and the best match is attempted. The user will be notified
+ of any fuzzy-match repo selections which are made.
+
+ - Running the utility at-will during an ongoing round is a safe operation.
+ The utility will simply update its internal state of ongoing changes
+ in codereview and exit if no further action can be taken.