Skip to content

Revise scap clean implementation

Ahmon Dancy requested to merge master-I0283b0e0e4dd3723d0e2146d4fb3139a12f313ea into master

scap/plugins/clean.py:
No longer rely on the implementation of AbstractSync.main() since
scap clean does not need all of the phases and behaviors it provides.

Replaced several log.Timer() sections with simple one line log
messages since those sections don't take significant time to complete.

Use tasks.get_wikiversions_ondisk() instead of
tasks.get_wikiversions_ondisk_ex(). The latter is now unused and
has been removed.

scap/main.py:
Factor out the code that syncs baremetal proxies and apaches, so
that it can be shared with scap clean.

scap/utils.py:
Added pluralize and should_colorize_output utility functions.

Sample output from the new scap clean:

$ scap clean 1.42.0-wmf.24  
           ___ ____  
         ⎛   ⎛ ,----  
          \  //==--'  
     _//|,.·//==--'    ____________________________  
    _OO≣=-  ︶ ᴹw ⎞_§ ______  ___\ ___\ ,\__ \/ __ \  
   (∞)_, )  (     |  ______/__  \/ /__ / /_/ / /_/ /  
     ¨--¨|| |- (  / ______\____/ \___/ \__^_/  .__/  
         ««_/  «_/ jgs/bd808                /_/  
  
19: 16:46 Cleaning branch: 1.42.0-wmf.24  
19: 16:46 Clean files owned by www-data  
19: 16:46 Clean /srv/mediawiki-staging/wmf-config/ExtensionMessages-1.42.0-wmf.24.php  
19: 16:46 Clean /srv/mediawiki-staging/php-1.42.0-wmf.24  
19: 16:48 Clean /srv/patches/1.42.0-wmf.24  
19: 16:48 Started sync-masters  
19: 16:51 sync-masters: 100% (in-flight: 0; ok: 1; fail: 0; left: 0)  
19: 16:51 Finished sync-masters (duration: 00m 03s)  
19: 16:51 Started sync-apaches  
19: 16:54 sync-apaches: 100% (in-flight: 0; ok: 2; fail: 0; left: 0)  
19: 16:54 Per-host sync duration: average 3.1s, median 3.1s  
19: 16:54 rsync transfer: average 373,339 bytes/host, total 746,678 bytes  
19: 16:54 Finished sync-apaches (duration: 00m 03s)  
19: 16:54 Started clean-remote-caches  
19: 16:54 clean-remote-caches: 100% (in-flight: 0; ok: 2; fail: 0; left: 0)  
19: 16:54 Finished clean-remote-caches (duration: 00m 00s)  
19: 16:54 Pruned MediaWiki: 1.42.0-wmf.24 (duration: 00m 08s)

Related MRs:

Depends-On: !314 (merged)

Edited by Ahmon Dancy

Merge request reports