Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.1 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 12 336 2411090 2375525 2026-05-06T13:25:58Z Filippo Giunchedi 2939 /* Increase quotas for projects */ 2411090 wikitext text/x-wiki nrbqd2hfvbkm9utjeltv2z699w5yfg0 0 4108 2411121 2410922 2026-05-06T13:55:56Z ScheduleDeploymentBot 37566 Add [[gerrit:1283050]] to Wednesday, May 06 UTC afternoon backport window 2411121 wikitext text/x-wiki heq2kfohj9v45j2y9zil4w5jwuyxwwj 2411573 2411121 2026-05-07T07:37:20Z ScheduleDeploymentBot 37566 Add [[gerrit:1269465]] to Thursday, May 07 UTC morning backport window 2411573 wikitext text/x-wiki 02o6wl48dhymynmsagipyy3ft7vykfx 2411620 2411573 2026-05-07T09:17:15Z ScheduleDeploymentBot 37566 Add [[gerrit:1284553]] to Thursday, May 07 UTC afternoon backport window 2411620 wikitext text/x-wiki ceja36qkvjmfaccguw6ge25cwae84l9 2411621 2411620 2026-05-07T09:18:11Z ScheduleDeploymentBot 37566 Add [[gerrit:1284569]] to Thursday, May 07 UTC afternoon backport window 2411621 wikitext text/x-wiki d497fyt85ckpou53tb742brf56gaxvj 2411735 2411621 2026-05-07T11:31:50Z ScheduleDeploymentBot 37566 Add [[gerrit:1284547]] to Thursday, May 07 UTC afternoon backport window 2411735 wikitext text/x-wiki b6hhq7f9o4pz2dikjuddwbpwhx2r4ax 2411737 2411735 2026-05-07T11:35:36Z ScheduleDeploymentBot 37566 Add [[gerrit:1275467]] to Thursday, May 07 UTC afternoon backport window 2411737 wikitext text/x-wiki 15im1k34kf98dkk2pt0qcvwxmttm3d8 0 7919 2411038 2411036 2026-05-06T12:05:16Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host kafka-logging1006.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2411038 wikitext text/x-wiki 18llqqwhimkep09yzvy7cbm8vzxjm95 2411039 2411038 2026-05-06T12:05:24Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1283735|Close Polish Wikinews (T421796)]] 2411039 wikitext text/x-wiki 1ozuhljn23m6vhjfoanair0og2fuarw 2411040 2411039 2026-05-06T12:05:32Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host rdb2012.codfw.wmnet with OS trixie 2411040 wikitext text/x-wiki p43fnmhlz2vhn3onyxmewih0bqdg7ys 2411041 2411040 2026-05-06T12:07:17Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1283735|Close Polish Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411041 wikitext text/x-wiki 6w5chk377h6cobpznwqaj20ba47iqrs 2411042 2411041 2026-05-06T12:07:40Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2411042 wikitext text/x-wiki 5v7vga94mw2pinx5bmcx3edzbc4xnvy 2411043 2411042 2026-05-06T12:07:47Z Stashbot 7414 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host kafka-logging1006.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2411043 wikitext text/x-wiki 58mv73j6xm4ao400gtv4bn7u0nsfq0p 2411045 2411043 2026-05-06T12:11:52Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1283735|Close Polish Wikinews (T421796)]] (duration: 06m 28s) 2411045 wikitext text/x-wiki 8gh9hxfcnqoian0fnyeszqa4kxfcqdl 2411047 2411045 2026-05-06T12:14:50Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host rdb2011.codfw.wmnet with OS trixie 2411047 wikitext text/x-wiki eobhhusnfex7alvt3d9x42r4kycfjd3 2411048 2411047 2026-05-06T12:15:50Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/services/miscweb: apply 2411048 wikitext text/x-wiki bv603uqgnndruai57h3yo3tm6hnazme 2411049 2411048 2026-05-06T12:16:21Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/services/miscweb: apply 2411049 wikitext text/x-wiki flt5v5b25cdwkdljp9cnofa1sccw1zo 2411050 2411049 2026-05-06T12:16:39Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp4050.ulsfo.wmnet with OS trixie 2411050 wikitext text/x-wiki 30lzrbz4t5eaiyrxfyto7tly42ck5m1 2411052 2411050 2026-05-06T12:20:21Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp4048.ulsfo.wmnet with OS trixie 2411052 wikitext text/x-wiki 7f2d3cs2ma1qpdmctat730fdcobsfmj 2411053 2411052 2026-05-06T12:20:52Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1194.eqiad.wmnet with OS trixie 2411053 wikitext text/x-wiki 1nhfv9s2w615jdzz2ezbnsc313vmkhb 2411054 2411053 2026-05-06T12:21:13Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on rdb2012.codfw.wmnet with reason: host reimage 2411054 wikitext text/x-wiki a3cc7ghiclvyzcu02oy9pokzglotrrx 2411055 2411054 2026-05-06T12:24:52Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on rdb2012.codfw.wmnet with reason: host reimage 2411055 wikitext text/x-wiki tl6066kkyd5ogo38oiuc0l2ow793yga 2411058 2411055 2026-05-06T12:35:06Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1194.eqiad.wmnet with reason: host reimage 2411058 wikitext text/x-wiki ajzhs1thvxhahqrje0ujlls7u2huhey 2411059 2411058 2026-05-06T12:36:39Z Stashbot 7414 jynus@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on 8 hosts with reason: update 2411059 wikitext text/x-wiki rydcp3ld3vixbl51ewiyz2u34nzh9ys 2411063 2411059 2026-05-06T12:38:42Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1194.eqiad.wmnet with reason: host reimage 2411063 wikitext text/x-wiki is7ypfvve2tojn39numtd71iabnmz5m 2411064 2411063 2026-05-06T12:39:07Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host kafka-logging1006.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2411064 wikitext text/x-wiki b17ty2uj7dp98rjf8l1nhqzoju11y25 2411065 2411064 2026-05-06T12:43:19Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host rdb2012.codfw.wmnet with OS trixie 2411065 wikitext text/x-wiki tr5pd5g5wuid3yu2qtv00l4mqaaqhhk 2411066 2411065 2026-05-06T12:45:12Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/services/miscweb: apply 2411066 wikitext text/x-wiki sj9aorbdgtqdxbhyuhu6v5gx0i8lon7 2411067 2411066 2026-05-06T12:50:00Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.reimage for host lvs4008.ulsfo.wmnet with OS trixie 2411067 wikitext text/x-wiki eezsnxxxow33vdrpxey9mli1ml3rbos 2411068 2411067 2026-05-06T13:01:17Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1194.eqiad.wmnet with OS trixie 2411068 wikitext text/x-wiki 3d3r10soj8zb0o7v5d561lw5d4rdk9a 2411069 2411068 2026-05-06T13:05:30Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/services/miscweb: apply 2411069 wikitext text/x-wiki a7s74y2wdks6b6fghvshg31udh4zjzw 2411070 2411069 2026-05-06T13:05:59Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1194: after reimage to trixie 2411070 wikitext text/x-wiki fa2xn1o9y76nqzoqz9x81fvfvio1t3b 2411071 2411070 2026-05-06T13:08:32Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on lvs4008.ulsfo.wmnet with reason: host reimage 2411071 wikitext text/x-wiki r1g58aahv75xxp6irv1rm9zs2lksw95 2411072 2411071 2026-05-06T13:11:28Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti4008.mgmt.ulsfo.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411072 wikitext text/x-wiki t48cejzqwj41v9fif6ukdbu2ef9uvva 2411073 2411072 2026-05-06T13:12:40Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.reimage for host lvs4010.ulsfo.wmnet with OS trixie 2411073 wikitext text/x-wiki gugapfpho8vzh3meqhzotnpmxthtmnn 2411074 2411073 2026-05-06T13:13:22Z Stashbot 7414 alexsanford@deploy1003: Started scap sync-world: Backport for [[gerrit:1283028|Add messages related to mandatory 2FA for more groups (T423119)]] 2411074 wikitext text/x-wiki e0gz9hvw7dn1p456vh4qe181tfjc60y 2411075 2411074 2026-05-06T13:14:14Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/services/miscweb: apply 2411075 wikitext text/x-wiki 7hdhwkvxvj88kvse0mv4apfy92aq13z 2411076 2411075 2026-05-06T13:14:19Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on lvs4008.ulsfo.wmnet with reason: host reimage 2411076 wikitext text/x-wiki hox0fllnc447h69no5e1bj3f2vobj29 2411077 2411076 2026-05-06T13:15:15Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.netbox 2411077 wikitext text/x-wiki 0t71x25u96n98f6pt5p19yvncaile25 2411078 2411077 2026-05-06T13:18:53Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add entry for ganeti4008 mgmt - cmooney@cumin1003" 2411078 wikitext text/x-wiki t4clno9wrwdv8l7gh36nbzkcztxiawr 2411079 2411078 2026-05-06T13:18:59Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add entry for ganeti4008 mgmt - cmooney@cumin1003" 2411079 wikitext text/x-wiki sq3n0rxbvghvlevir17xcykn3trhbmo 2411080 2411079 2026-05-06T13:19:00Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411080 wikitext text/x-wiki 4o8ls7z0896qxz8w69k97lgql4kazvd 2411081 2411080 2026-05-06T13:19:18Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.wipe-cache ganeti4008.mgmt.ulsfo.wmnet on all recursors 2411081 wikitext text/x-wiki 0dil3dg2nxoel6gg59lvcw6mokqxfcd 2411082 2411081 2026-05-06T13:19:24Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.reimage for host cp4046.ulsfo.wmnet with OS trixie 2411082 wikitext text/x-wiki 8n53ojq0w0ph31sug1v5ly8sdl7xvf2 2411083 2411082 2026-05-06T13:19:37Z Stashbot 7414 cmooney@cumin1003: END (FAIL) - Cookbook sre.dns.wipe-cache (exit_code=99) ganeti4008.mgmt.ulsfo.wmnet on all recursors 2411083 wikitext text/x-wiki j2jlm6uaj4rgsx3kl1m4a9afmkb8420 2411084 2411083 2026-05-06T13:20:05Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.provision for host ganeti4008.mgmt.ulsfo.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411084 wikitext text/x-wiki ijv1lbakuas3siz1ty3ftjcfc9aotnb 2411086 2411084 2026-05-06T13:21:58Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2411086 wikitext text/x-wiki 5yhrzecbjdha6bze63aa1qrbmptasjg 2411087 2411086 2026-05-06T13:24:27Z Stashbot 7414 pt1979@cumin2002: START - Cookbook sre.dns.netbox 2411087 wikitext text/x-wiki q0k5sx7blczimtw904z1tude5oarnm0 2411089 2411087 2026-05-06T13:24:47Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti4008.mgmt.ulsfo.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411089 wikitext text/x-wiki c7jtp2saur1arglc8k04zj6shblco2r 2411091 2411089 2026-05-06T13:26:37Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/services/miscweb: apply 2411091 wikitext text/x-wiki jzxh8x6fnp31dh4yl9pjg15uts9b4l6 2411092 2411091 2026-05-06T13:27:24Z Stashbot 7414 pt1979@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411092 wikitext text/x-wiki bsxtf3z3nvg4p4alw58mnrpi034f5a9 2411093 2411092 2026-05-06T13:27:30Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ganeti4008.ulsfo.wmnet'] 2411093 wikitext text/x-wiki bc36lcq1o8pfhkgez8nw2ku8g4t6d1e 2411094 2411093 2026-05-06T13:27:49Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['ganeti4008.ulsfo.wmnet'] 2411094 wikitext text/x-wiki eh3in3chku4sb6zl3gcu8q55gyvk87t 2411095 2411094 2026-05-06T13:27:57Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ganeti4008.ulsfo.wmnet'] 2411095 wikitext text/x-wiki rtlxhrgp0845mxw8vua2llqsi507kzp 2411096 2411095 2026-05-06T13:28:10Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['ganeti4008.ulsfo.wmnet'] 2411096 wikitext text/x-wiki 39f0uwh6rn942pg1zjsua6skuu31dl6 2411097 2411096 2026-05-06T13:28:31Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ganeti4008.ulsfo.wmnet'] 2411097 wikitext text/x-wiki pgkd44w0ogu1lzs38laurgg4qfab2y1 2411098 2411097 2026-05-06T13:28:44Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['ganeti4008.ulsfo.wmnet'] 2411098 wikitext text/x-wiki 1d5afwgc1wt3amq24easu7gttn18hb1 2411099 2411098 2026-05-06T13:31:09Z Stashbot 7414 alexsanford@deploy1003: alexsanford: Backport for [[gerrit:1283028|Add messages related to mandatory 2FA for more groups (T423119)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411099 wikitext text/x-wiki pj95fc4s0r5n44rkjlozn4vxvz3odzz 2411100 2411099 2026-05-06T13:31:33Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on lvs4010.ulsfo.wmnet with reason: host reimage 2411100 wikitext text/x-wiki fguatkqk39tdst5wc9ygswtm5qyns7y 2411101 2411100 2026-05-06T13:32:01Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2411101 wikitext text/x-wiki deseeequr1sdipmpctd9c2v3asprwf2 2411102 2411101 2026-05-06T13:32:49Z Stashbot 7414 alexsanford@deploy1003: alexsanford: Continuing with deployment 2411102 wikitext text/x-wiki 59dbk5cd1157txfmkxc9ki6zvs9mi04 2411104 2411102 2026-05-06T13:34:41Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.wipe-cache ganeti4008.ulsfo.wmnet on all recursors 2411104 wikitext text/x-wiki actax5xcs2a74etjp14iepus0c9pj5u 2411105 2411104 2026-05-06T13:35:01Z Stashbot 7414 cmooney@cumin1003: END (FAIL) - Cookbook sre.dns.wipe-cache (exit_code=99) ganeti4008.ulsfo.wmnet on all recursors 2411105 wikitext text/x-wiki b8jfpmrsqmnnyd4a2fomd46g8qg8df4 2411106 2411105 2026-05-06T13:36:15Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host ganeti4008.ulsfo.wmnet with OS bookworm 2411106 wikitext text/x-wiki e0u82jz9a1gu5dn79dv183z2oyb71u9 2411107 2411106 2026-05-06T13:39:31Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on lvs4010.ulsfo.wmnet with reason: host reimage 2411107 wikitext text/x-wiki 876jyaj0dmatcs2ze1yvbp68c81mdko 2411108 2411107 2026-05-06T13:41:22Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cp4046.ulsfo.wmnet with reason: host reimage 2411108 wikitext text/x-wiki iz67udjy56fe4v2q50w9tjh4q1cwgat 2411109 2411108 2026-05-06T13:44:03Z Stashbot 7414 jgreen@dns1004: START - running authdns-update 2411109 wikitext text/x-wiki tqj4nd8bls7yf16dg26gqyrcye7rod1 2411110 2411109 2026-05-06T13:44:16Z Stashbot 7414 alexsanford@deploy1003: Finished scap sync-world: Backport for [[gerrit:1283028|Add messages related to mandatory 2FA for more groups (T423119)]] (duration: 30m 53s) 2411110 wikitext text/x-wiki 891a7jiakm9rp1ivuk2f6zjsuwacpr5 2411112 2411110 2026-05-06T13:45:26Z Stashbot 7414 jgreen@dns1004: END - running authdns-update 2411112 wikitext text/x-wiki mm8kyp2fusiotbpywhj5wffx9gwb3ge 2411113 2411112 2026-05-06T13:45:32Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp4046.ulsfo.wmnet with reason: host reimage 2411113 wikitext text/x-wiki 2e3kab2yj5gcj4rgo5faniiejdus0f1 2411114 2411113 2026-05-06T13:51:24Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1283751|Close French Wikinews (T421796)]] 2411114 wikitext text/x-wiki osg4yihq7cr1nv9c7ytmf8iok96r4c9 2411115 2411114 2026-05-06T13:51:26Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1194: after reimage to trixie 2411115 wikitext text/x-wiki 968hjs351uoklf9aaye06976k14kon7 2411116 2411115 2026-05-06T13:53:08Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: restart to test fixes from T425301 - bking@cumin2002 2411116 wikitext text/x-wiki ca2l8xgre2rlhni2nh6i7abrgs14ttw 2411119 2411116 2026-05-06T13:55:11Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.reimage for host cp4052.ulsfo.wmnet with OS trixie 2411119 wikitext text/x-wiki 5tkvl18mdhe09qwod0tq3nf99jg9fah 2411120 2411119 2026-05-06T13:55:46Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1283751|Close French Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411120 wikitext text/x-wiki 0ua91oji84mroqwpxe3kfs5qrqa2rrc 2411122 2411120 2026-05-06T13:56:04Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti4008.ulsfo.wmnet with reason: host reimage 2411122 wikitext text/x-wiki noumsy9zxrrclem5cx4pa12q65hhmuj 2411123 2411122 2026-05-06T13:56:25Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2411123 wikitext text/x-wiki rjsmrfjsfsd1yhc7coq9oxcfinzl8th 2411125 2411123 2026-05-06T14:00:02Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ganeti4008.ulsfo.wmnet with reason: host reimage 2411125 wikitext text/x-wiki lgv568d7t5b1xokn1vvc408914408s9 2411126 2411125 2026-05-06T14:02:24Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.decommission for hosts durum5001.eqsin.wmnet 2411126 wikitext text/x-wiki 5epz65odhxbs8yyesal9xxf1lwz3n9u 2411127 2411126 2026-05-06T14:02:52Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1283751|Close French Wikinews (T421796)]] (duration: 11m 28s) 2411127 wikitext text/x-wiki ndgrjj3ygvw4w45ciqxdbs5s4b5mb7g 2411128 2411127 2026-05-06T14:08:22Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2411128 wikitext text/x-wiki q9u90brli7l1hx80s5qz3un9ndm07dk 2411129 2411128 2026-05-06T14:08:25Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1283783|Close German Wikinews (T421796)]] 2411129 wikitext text/x-wiki 02cp9cmpy2gkjbbq8ktzodsk5qx3e1d 2411130 2411129 2026-05-06T14:08:34Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/services/miscweb: apply 2411130 wikitext text/x-wiki 1uhgb3x8rrbnify6x2tuhjewsddkrj1 2411131 2411130 2026-05-06T14:09:17Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/services/miscweb: apply 2411131 wikitext text/x-wiki jq6rq75m4dz8l4lqu93kngoxwdk1dlp 2411132 2411131 2026-05-06T14:10:00Z Stashbot 7414 dmartin@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2411132 wikitext text/x-wiki duo98upkzxzcad8pq50bmok4uccofi0 2411133 2411132 2026-05-06T14:10:19Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1283783|Close German Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411133 wikitext text/x-wiki 95o3mo4ra5cyq3cy7wc2xkunipc30as 2411134 2411133 2026-05-06T14:10:38Z Stashbot 7414 dmartin@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2411134 wikitext text/x-wiki 5l5af6mpphrq2ewzcqxsmzdrybyvtxd 2411135 2411134 2026-05-06T14:10:53Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2411135 wikitext text/x-wiki h9kiwo73rlp3fggzxlx9gfsv8nycdvi 2411136 2411135 2026-05-06T14:11:25Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp4046.ulsfo.wmnet with OS trixie 2411136 wikitext text/x-wiki 4rgvcvtgv34op50vqwtnwpbe7twyivv 2411137 2411136 2026-05-06T14:11:51Z Stashbot 7414 dmartin@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2411137 wikitext text/x-wiki l0mc7ewrmgi4zw1bg0ghxxu9ntlk55c 2411138 2411137 2026-05-06T14:12:21Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: durum5001.eqsin.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin2002" 2411138 wikitext text/x-wiki nqgpy0mwxtg5gy574svngl3b1f5vme1 2411139 2411138 2026-05-06T14:12:46Z Stashbot 7414 dmartin@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2411139 wikitext text/x-wiki 3wcgcxgre2d7qlofjon7od6fvg96f06 2411140 2411139 2026-05-06T14:13:08Z Stashbot 7414 dmartin@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2411140 wikitext text/x-wiki bna3wlj6dxwa9opqscnrpx4j3vd96k7 2411141 2411140 2026-05-06T14:13:54Z Stashbot 7414 dmartin@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2411141 wikitext text/x-wiki 85zzsqc8mgedbfffm7ea8ljo89kat84 2411142 2411141 2026-05-06T14:15:06Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1283783|Close German Wikinews (T421796)]] (duration: 06m 40s) 2411142 wikitext text/x-wiki kc5vwis4fgj99yofsll46ee40wky9qu 2411143 2411142 2026-05-06T14:15:26Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: durum5001.eqsin.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin2002" 2411143 wikitext text/x-wiki o5biebo3soda76nfp0tf1ylv32c8bmq 2411144 2411143 2026-05-06T14:15:27Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411144 wikitext text/x-wiki 8kutrhom8rrd13boqh395hdq29jw0td 2411145 2411144 2026-05-06T14:15:28Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts durum5001.eqsin.wmnet 2411145 wikitext text/x-wiki l9feyuno65xiwwq5xozih6dckof1bjy 2411146 2411145 2026-05-06T14:18:33Z Stashbot 7414 dmartin@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2411146 wikitext text/x-wiki dqh485o7th95yilpknjcp44q0viypsm 2411147 2411146 2026-05-06T14:19:29Z Stashbot 7414 dmartin@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2411147 wikitext text/x-wiki 0qhwnhckg82eyhtokbcc94or2d0fjz3 2411148 2411147 2026-05-06T14:19:37Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1283050|Add user_groups to editAttemptStep schema (T424010)]] 2411148 wikitext text/x-wiki fvzomnxekpwws6t2ycglpcx8ujsy0oo 2411149 2411148 2026-05-06T14:20:19Z Stashbot 7414 dmartin@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2411149 wikitext text/x-wiki 8zxd6thjakseyow9t2p73qqzccsm8il 2411150 2411149 2026-05-06T14:20:41Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jmm@cumin2002" 2411150 wikitext text/x-wiki 8opamai93if7mvnq2stnkpwa5lp0xra 2411151 2411150 2026-05-06T14:20:57Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jmm@cumin2002" 2411151 wikitext text/x-wiki 1s7yq376zj150gb6mrofce2hylrrsfo 2411152 2411151 2026-05-06T14:20:58Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti4008.ulsfo.wmnet with OS bookworm 2411152 wikitext text/x-wiki scwq03ks8frbaoqxhbm0wg8qsdei1cf 2411153 2411152 2026-05-06T14:21:05Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.decommission for hosts durum5002.eqsin.wmnet 2411153 wikitext text/x-wiki cwr2kkdzyt4mh0ycfdr9f3v9obc2mt5 2411154 2411153 2026-05-06T14:21:29Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1283050|Add user_groups to editAttemptStep schema (T424010)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411154 wikitext text/x-wiki 8s1lkbmqjknb9bfxxm72ko09eshcqr7 2411155 2411154 2026-05-06T14:21:31Z Stashbot 7414 dmartin@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2411155 wikitext text/x-wiki bse2knj7j9yfxgex21tg4hbey5ag4r5 2411156 2411155 2026-05-06T14:21:54Z Stashbot 7414 dmartin@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2411156 wikitext text/x-wiki 5fwzkwmotz7poft5mbw6ger4b4n775z 2411157 2411156 2026-05-06T14:22:44Z Stashbot 7414 dmartin@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2411157 wikitext text/x-wiki 774dausqguuddbxmxrldlovs4u465l2 2411159 2411157 2026-05-06T14:23:09Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cp4052.ulsfo.wmnet with reason: host reimage 2411159 wikitext text/x-wiki 95w8uj5fqwvhw7eoon37j827c1e339k 2411161 2411159 2026-05-06T14:25:29Z Stashbot 7414 sukhe: sudo cumin "C:bird" "disable-puppet 'merging CR 1282958'" 2411161 wikitext text/x-wiki jlw5wpvbnu7w7ijnekpanwt7n8xknap 2411162 2411161 2026-05-06T14:26:45Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2411162 wikitext text/x-wiki 08zkvnq3abg3toh1lxjcc86fu7zd9qk 2411163 2411162 2026-05-06T14:28:26Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp4052.ulsfo.wmnet with reason: host reimage 2411163 wikitext text/x-wiki mbi8mnl8jzgx6kh732mb3mzxk02ha1q 2411164 2411163 2026-05-06T14:30:53Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1283050|Add user_groups to editAttemptStep schema (T424010)]] (duration: 11m 16s) 2411164 wikitext text/x-wiki n1v0fb0q3sciuueae4fuhxle3w44xz8 2411165 2411164 2026-05-06T14:31:02Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=no; selector: name=dns7001.wikimedia.org [reason: testing bird change] 2411165 wikitext text/x-wiki g759axpzhl46joq5q100rm2j18dx9dz 2411166 2411165 2026-05-06T14:34:48Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=yes; selector: name=dns7001.wikimedia.org [reason: testing bird change] 2411166 wikitext text/x-wiki 0u8w9i1itj9t5dt4krgaqswwzvi3xnd 2411167 2411166 2026-05-06T14:35:38Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2411167 wikitext text/x-wiki 03jdbf9rrx80pjmr72il6jrodp3j1fg 2411168 2411167 2026-05-06T14:36:14Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/services/miscweb: apply 2411168 wikitext text/x-wiki 0a0go8fi87z1danw21zqh9d8b8m8qng 2411169 2411168 2026-05-06T14:36:50Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/services/miscweb: apply 2411169 wikitext text/x-wiki 2011uq9sa8vpkqkljdgfk8upwa57fah 2411170 2411169 2026-05-06T14:41:31Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2411170 wikitext text/x-wiki e6txhkxbufts8x4gbkfuggua6wshrcd 2411171 2411170 2026-05-06T14:45:25Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2411171 wikitext text/x-wiki ry8l9yftvjfe0zeghkyowm7h46jeqjf 2411173 2411171 2026-05-06T14:51:25Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: durum5002.eqsin.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin2002" 2411173 wikitext text/x-wiki 2hrvtqk4l9nwe48h7p32mwtspy4l89s 2411174 2411173 2026-05-06T14:53:46Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp4052.ulsfo.wmnet with OS trixie 2411174 wikitext text/x-wiki 8u0emtz677wwhlimmfj74a838x103cl 2411175 2411174 2026-05-06T14:55:04Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: durum5002.eqsin.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin2002" 2411175 wikitext text/x-wiki 4x5zhfhj144c6nqxflxd24wnx7j88kf 2411176 2411175 2026-05-06T14:55:05Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411176 wikitext text/x-wiki a1j2eakmajc3yor75bvem49xxrruxpj 2411177 2411176 2026-05-06T14:55:06Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts durum5002.eqsin.wmnet 2411177 wikitext text/x-wiki 7fzprbtdms2ebfh38g18xbkvl1xypyi 2411178 2411177 2026-05-06T14:59:27Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1283805|Close Chinese Wikinews (T421796)]] 2411178 wikitext text/x-wiki 9tv4lwmhk9ex4kjzhostymccnbfwfru 2411179 2411178 2026-05-06T15:01:28Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1283805|Close Chinese Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411179 wikitext text/x-wiki a1s5dxw9ljd3io3tbivnry7znlxqqko 2411180 2411179 2026-05-06T15:01:52Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2411180 wikitext text/x-wiki 01h3z7ayaqj1jp6vdcrcvbrs0pdj8wf 2411181 2411180 2026-05-06T15:02:27Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/services/miscweb: apply 2411181 wikitext text/x-wiki 6ifkeojo8fzfycro6x4rwygbut4rk3x 2411182 2411181 2026-05-06T15:02:53Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/services/miscweb: apply 2411182 wikitext text/x-wiki 7aykusce0bkz4e6lz1hytqd7rws7yja 2411184 2411182 2026-05-06T15:06:08Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1283805|Close Chinese Wikinews (T421796)]] (duration: 06m 41s) 2411184 wikitext text/x-wiki d95t17jbtxjteybwchslg2blawixmb2 2411185 2411184 2026-05-06T15:08:32Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.kafka.change-confluent-distro-version Change Confluent distribution for Kafka A:kafka-main-eqiad cluster: Change Confluent distribution. 2411185 wikitext text/x-wiki ln32dskd2mn86xqaooihkoqsyezz04k 2411186 2411185 2026-05-06T15:08:39Z Stashbot 7414 sukhe: sudo cumin -b1 -s5 "C:bird and not dns4004*" "run-puppet-agent --enable 'merging CR 1282958'" 2411186 wikitext text/x-wiki t8rmg4r16sd27xl3b41elc0m7tonzun 2411187 2411186 2026-05-06T15:30:46Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.kafka.change-confluent-distro-version (exit_code=0) Change Confluent distribution for Kafka A:kafka-main-eqiad cluster: Change Confluent distribution. 2411187 wikitext text/x-wiki 3rj3v4osv5qy706vbfqpfg2tn2p3smd 2411188 2411187 2026-05-06T15:35:27Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reimage for host dns4004.wikimedia.org with OS bookworm 2411188 wikitext text/x-wiki gfzyq7rvhaflteo0z3x4xkzehy3tu5r 2411189 2411188 2026-05-06T15:38:23Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reimage for host lvs4008.ulsfo.wmnet with OS bookworm 2411189 wikitext text/x-wiki 0717r6thc5xq80ptclngqdixea3gsd3 2411191 2411189 2026-05-06T15:57:28Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on dns4004.wikimedia.org with reason: host reimage 2411191 wikitext text/x-wiki 9uc40imtu1biwpsv48nib52gh950tfn 2411192 2411191 2026-05-06T15:58:34Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on lvs4008.ulsfo.wmnet with reason: host reimage 2411192 wikitext text/x-wiki 7i2c2r5udrojjpptchtfuqj2a22dh4n 2411193 2411192 2026-05-06T16:04:52Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dns4004.wikimedia.org with reason: host reimage 2411193 wikitext text/x-wiki mhxtib175pkfq5xkvtibxouto9zudqn 2411194 2411193 2026-05-06T16:09:04Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on lvs4008.ulsfo.wmnet with reason: host reimage 2411194 wikitext text/x-wiki ntt24ch1q1ir2r4nojhalss22e5gj22 2411195 2411194 2026-05-06T16:27:01Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply 2411195 wikitext text/x-wiki jxdip0ogi911wl4ae2i7zc9acaej89u 2411196 2411195 2026-05-06T16:28:17Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2411196 wikitext text/x-wiki fbyali8uabi65wgirknhhk902kocfhw 2411197 2411196 2026-05-06T16:28:45Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply 2411197 wikitext text/x-wiki kr7dp4flpuyzgw6mvygn6nyfxg7djud 2411198 2411197 2026-05-06T16:28:47Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lvs4008.ulsfo.wmnet with OS bookworm 2411198 wikitext text/x-wiki exb390x801qyj2zuoiow3y2i007l0bw 2411199 2411198 2026-05-06T16:28:57Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply 2411199 wikitext text/x-wiki ekj39ro01wzvxd03gx49kd44f4lilye 2411200 2411199 2026-05-06T16:29:08Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply 2411200 wikitext text/x-wiki sz56dl08tooxkucyto6gjop40vk3y9v 2411202 2411200 2026-05-06T16:37:06Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dns4004.wikimedia.org with OS bookworm 2411202 wikitext text/x-wiki ch6wrucpokyk9ubugaqh5fokuhmbfc7 2411203 2411202 2026-05-06T16:39:55Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2411203 wikitext text/x-wiki q7ro66valxfb7s3rdj4cdlw97xjatly 2411204 2411203 2026-05-06T16:40:09Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lvs4008.ulsfo.wmnet with OS trixie 2411204 wikitext text/x-wiki tvqlnvuhhm4bvq3aqwplqzdvrrq69cf 2411206 2411204 2026-05-06T16:45:16Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2411206 wikitext text/x-wiki scnl3y6fqr7u8qm73t8b1cgyowqspn3 2411208 2411206 2026-05-06T16:52:39Z Stashbot 7414 topranks: rebooting asw1-22-ulsfo to upgrade SR-Linux OS on switch T408892 2411208 wikitext text/x-wiki a5r76p0h9lox8z33u03h7ot3i5wr9mj 2411209 2411208 2026-05-06T16:53:23Z Stashbot 7414 cmooney@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on asw1-22-ulsfo,asw1-22-ulsfo IPv6 with reason: upgrading sr-linux on asw1-23-ulsfo 2411209 wikitext text/x-wiki 9g6g3m7lrldb554i6fkd9uokoizdpjg 2411210 2411209 2026-05-06T17:02:15Z Stashbot 7414 sukhe@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on 39 hosts with reason: ulsfo depooled for switch work 2411210 wikitext text/x-wiki rbflghvpc8nn1tutrvieepwh2noqgo7 2411211 2411210 2026-05-06T17:06:46Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] START helmfile.d/services/shellbox: apply 2411211 wikitext text/x-wiki p4xje13brobpkedt0lmasi1gf32pshe 2411212 2411211 2026-05-06T17:07:12Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox: apply 2411212 wikitext text/x-wiki l58uh909ucig5hz02n9whbfyr20dh9d 2411213 2411212 2026-05-06T17:07:14Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-constraints: apply 2411213 wikitext text/x-wiki o2uhuksnlv3xkrc5vo9uwbyp2mejj9q 2411214 2411213 2026-05-06T17:07:24Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-constraints: apply 2411214 wikitext text/x-wiki 3dbskv5pk5yt9fvp1ghkefajhhkm14u 2411215 2411214 2026-05-06T17:07:25Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply 2411215 wikitext text/x-wiki n7c4120zxhqja0f8se5ra5ytlw0sscb 2411216 2411215 2026-05-06T17:07:37Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply 2411216 wikitext text/x-wiki e4wx61390qkxxaqvvh59cjrgv3po2gy 2411217 2411216 2026-05-06T17:07:39Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-syntaxhighlight: apply 2411217 wikitext text/x-wiki go9tuz88f61pxypdgfccrasy37coem4 2411218 2411217 2026-05-06T17:07:55Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-syntaxhighlight: apply 2411218 wikitext text/x-wiki 0mje3bqd6dxq6ll6oz6ch9x8c8wqk95 2411219 2411218 2026-05-06T17:07:56Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-timeline: apply 2411219 wikitext text/x-wiki fduto2er03d5ove2sibfpsh335bsqpw 2411220 2411219 2026-05-06T17:08:14Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-timeline: apply 2411220 wikitext text/x-wiki lrjfsa4bhh4mo5u49ajmv986n5pe3i8 2411221 2411220 2026-05-06T17:08:16Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-video: apply 2411221 wikitext text/x-wiki grrf7kmubitvzxvh1if9x1ph363w19k 2411222 2411221 2026-05-06T17:08:38Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-video: apply 2411222 wikitext text/x-wiki ks4glz8mog8474tabssekajhwgugqre 2411224 2411222 2026-05-06T17:14:15Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox: apply 2411224 wikitext text/x-wiki d8ktrypsfu5b5fi7mdu59ih6teyzolf 2411225 2411224 2026-05-06T17:14:51Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox: apply 2411225 wikitext text/x-wiki sgcgdc2pjamctsp4haozrwfrdek11ge 2411226 2411225 2026-05-06T17:15:23Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-constraints: apply 2411226 wikitext text/x-wiki jzuynqa9zy9i70fj2larf4w7hhgulso 2411227 2411226 2026-05-06T17:15:55Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-constraints: apply 2411227 wikitext text/x-wiki 975cunzbiko1jnyp7kwaj7heeql1c7w 2411228 2411227 2026-05-06T17:16:32Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply 2411228 wikitext text/x-wiki obvrx1k2zvywjbpbgiqxc6e3yoh2f0t 2411229 2411228 2026-05-06T17:16:41Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply 2411229 wikitext text/x-wiki m31lnydwakpscyg0cwv9m7porcy69n0 2411230 2411229 2026-05-06T17:17:12Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-syntaxhighlight: apply 2411230 wikitext text/x-wiki demf4fvkl4fcqg7glpt46zglu6pqcom 2411231 2411230 2026-05-06T17:17:31Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-syntaxhighlight: apply 2411231 wikitext text/x-wiki i03h4fltk7ie9mllfqzlzfz1vxpw9b8 2411232 2411231 2026-05-06T17:18:04Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-timeline: apply 2411232 wikitext text/x-wiki dpd71q4ube6y6jtcp3gsb298qf8db0w 2411233 2411232 2026-05-06T17:18:26Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-timeline: apply 2411233 wikitext text/x-wiki ias52emu9wycu1kl4ulguklmrbhs23b 2411235 2411233 2026-05-06T17:18:57Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-video: apply 2411235 wikitext text/x-wiki l84ru9fqkqybs0wsloofugiwxcik1zd 2411236 2411235 2026-05-06T17:20:02Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-video: apply 2411236 wikitext text/x-wiki afdo6dam241kvtljtvbyeqqrrf2gx80 2411238 2411236 2026-05-06T17:27:46Z Stashbot 7414 cmooney@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on asw1-23-ulsfo,asw1-23-ulsfo IPv6 with reason: upgrading sr-linux on asw1-23-ulsfo 2411238 wikitext text/x-wiki ookdyhzf5radil410uukpprepahi3wn 2411239 2411238 2026-05-06T17:28:13Z Stashbot 7414 topranks: rebooting asw1-23-ulsfo to upgrade SR-Linux OS on switch T408892 2411239 wikitext text/x-wiki bs96h63tbjeoval0qa1vwg91ghagbbf 2411240 2411239 2026-05-06T17:31:36Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox: apply 2411240 wikitext text/x-wiki 7tt4nkf12nvf4eykb9j7pjgc39u6u55 2411241 2411240 2026-05-06T17:32:23Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox: apply 2411241 wikitext text/x-wiki gkcy18b1o8m0nmcblvnrv4cu6tf42i5 2411242 2411241 2026-05-06T17:32:54Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-constraints: apply 2411242 wikitext text/x-wiki s023jadpsel8dtl3qzv3cc17h1lxpkd 2411243 2411242 2026-05-06T17:33:36Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-constraints: apply 2411243 wikitext text/x-wiki byljxv966qhzsmd6aszs13mpt3xcc8n 2411244 2411243 2026-05-06T17:34:07Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply 2411244 wikitext text/x-wiki m26qxazrn27pvhx6te93vtmxemi02t6 2411245 2411244 2026-05-06T17:34:23Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply 2411245 wikitext text/x-wiki 1ieu31w1n0l1yq4ri05wbwfb68garhw 2411247 2411245 2026-05-06T17:34:54Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-syntaxhighlight: apply 2411247 wikitext text/x-wiki pjjwvqvgeljmzzasbqq4j6aijpdwab7 2411248 2411247 2026-05-06T17:35:11Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-syntaxhighlight: apply 2411248 wikitext text/x-wiki bya1ep65b9m13hmegywhxpmcbqt9qrx 2411249 2411248 2026-05-06T17:35:43Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-timeline: apply 2411249 wikitext text/x-wiki mr2s5ambqnytu6msyqkntkhk3fpm9e0 2411251 2411249 2026-05-06T17:36:06Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-timeline: apply 2411251 wikitext text/x-wiki lnnzmv1pem9xvbzdaiko6y9ox8ywju8 2411252 2411251 2026-05-06T17:36:37Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-video: apply 2411252 wikitext text/x-wiki 6cepg44qua47m4aulfw9cbx3jr7qeon 2411253 2411252 2026-05-06T17:37:41Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-video: apply 2411253 wikitext text/x-wiki 0afnh51snmu6k9oepk4w4nm1llk1cx7 2411258 2411253 2026-05-06T17:55:30Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.network.tls for network device asw1-23-ulsfo 2411258 wikitext text/x-wiki 2veajkotzcde58bjh1aucbx28hsqx9j 2411259 2411258 2026-05-06T17:55:51Z Stashbot 7414 cmooney@cumin1003: END (FAIL) - Cookbook sre.network.tls (exit_code=99) for network device asw1-23-ulsfo 2411259 wikitext text/x-wiki ahclqmqnio6v11sp2rqsuwrdfxsjk1p 2411263 2411259 2026-05-06T17:59:12Z Stashbot 7414 bking@cumin2002: END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: restart to test fixes from T425301 - bking@cumin2002 2411263 wikitext text/x-wiki g0rifqilvh2enht2d4uhakw7xq603zl 2411264 2411263 2026-05-06T18:01:53Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.network.tls for network device asw1-23-ulsfo 2411264 wikitext text/x-wiki rq056ejcwi610ej63c9h7fij5rainnd 2411265 2411264 2026-05-06T18:02:18Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device asw1-23-ulsfo 2411265 wikitext text/x-wiki 9nyo678561riksa71wph4ibgr33hyt9 2411275 2411265 2026-05-06T18:29:44Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reimage for host lvs4010.ulsfo.wmnet with OS bookworm 2411275 wikitext text/x-wiki cy0agt96woihv2oyrqfhjc9n4tl0cw8 2411276 2411275 2026-05-06T18:31:55Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reimage for host cp4050.ulsfo.wmnet with OS trixie 2411276 wikitext text/x-wiki jhpzcd872zo5jyts1mfta9aoedfeb89 2411277 2411276 2026-05-06T18:33:00Z Stashbot 7414 brennen: 1.47.0-wmf.1 train status (T423910): blockers resolved, rolling to group1 2411277 wikitext text/x-wiki j20unojuu09247u11tvyrc0qdffuajm 2411280 2411277 2026-05-06T18:35:48Z Stashbot 7414 dzahn@dns1005: START - running authdns-update 2411280 wikitext text/x-wiki qtu3si44mpdquo23uw2toi22qgeyyg3 2411281 2411280 2026-05-06T18:37:17Z Stashbot 7414 dzahn@dns1005: END - running authdns-update 2411281 wikitext text/x-wiki mk58rxnzjrzk48edl54g0wg0wyul7fb 2411283 2411281 2026-05-06T18:39:56Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2411283 wikitext text/x-wiki 9nien7cmnh7fm1gev6petots5muekz4 2411284 2411283 2026-05-06T18:40:17Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2411284 wikitext text/x-wiki i4njab8e25aoieg4drg6qos0b0bdzbq 2411285 2411284 2026-05-06T18:40:35Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2411285 wikitext text/x-wiki 7s3kxwzyqv8ltgqoouqkjegwwewntwz 2411286 2411285 2026-05-06T18:41:16Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2411286 wikitext text/x-wiki 9jtv3y98xhd4mlxljw209ncbuttebj1 2411287 2411286 2026-05-06T18:42:14Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2411287 wikitext text/x-wiki s2vd9ql2g29agii7r2mol6gx1mk212d 2411290 2411287 2026-05-06T18:42:52Z Stashbot 7414 brennen@deploy1003: rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.1 refs T423910 2411290 wikitext text/x-wiki tcnhmunxq6uiq7g069hlqpoagduzc3f 2411291 2411290 2026-05-06T18:42:53Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2411291 wikitext text/x-wiki 85g1ia9ymoy7vba59m6z98blt4x356u 2411292 2411291 2026-05-06T18:47:34Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2411292 wikitext text/x-wiki 8ix4gzefl5j87og57xc38ihgzgqutdl 2411293 2411292 2026-05-06T18:47:51Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2411293 wikitext text/x-wiki olj6upyenla5ehjwlvdegcaczppl8cj 2411294 2411293 2026-05-06T18:48:51Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on lvs4010.ulsfo.wmnet with reason: host reimage 2411294 wikitext text/x-wiki s7iu28pp9qb8b6gc2h2it1d6l4mjhyv 2411295 2411294 2026-05-06T18:49:00Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.hosts.reimage for host cloudelastic1012.eqiad.wmnet with OS trixie 2411295 wikitext text/x-wiki msiuybhhij8x83j8z3n6cu7qn7ag7cg 2411296 2411295 2026-05-06T18:53:56Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2411296 wikitext text/x-wiki drzxv7k1em55rxdp77sfzhqlobramnl 2411297 2411296 2026-05-06T18:53:59Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2411297 wikitext text/x-wiki onfl9awq9298jlt1nn0pgad3fse3786 2411298 2411297 2026-05-06T18:54:14Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2411298 wikitext text/x-wiki cy1rkvwkvfu2omixdr46ozadgsp6dbp 2411299 2411298 2026-05-06T18:54:48Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cp4050.ulsfo.wmnet with reason: host reimage 2411299 wikitext text/x-wiki 9fj07w1f6032iojeo6eod0fice9a8kn 2411300 2411299 2026-05-06T18:54:53Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on lvs4010.ulsfo.wmnet with reason: host reimage 2411300 wikitext text/x-wiki 5lr73q685txypig5tmznbteia5a3r54 2411301 2411300 2026-05-06T18:55:08Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2411301 wikitext text/x-wiki 7dd2y6az6g1gfg9ba1hsyptz1tdqntr 2411302 2411301 2026-05-06T18:55:22Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2411302 wikitext text/x-wiki t6fz9y5fx1znv57apz97eesiu3jv0yo 2411303 2411302 2026-05-06T18:55:55Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2411303 wikitext text/x-wiki hqvfehrz4kly8nkg90m9z3zy6mrdo51 2411304 2411303 2026-05-06T18:59:05Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp4050.ulsfo.wmnet with reason: host reimage 2411304 wikitext text/x-wiki oooviafnpdx1k63j0heufowhk09v2pg 2411305 2411304 2026-05-06T18:59:09Z Stashbot 7414 dani@deploy1003: helmfile [staging] START helmfile.d/services/miscweb: apply 2411305 wikitext text/x-wiki f5mdj5ybq1x31hira46pdyyfmtprv4n 2411306 2411305 2026-05-06T18:59:22Z Stashbot 7414 dani@deploy1003: helmfile [staging] DONE helmfile.d/services/miscweb: apply 2411306 wikitext text/x-wiki 6ny2wl9sk7bhw9m10qdryvfe3ebc9ta 2411307 2411306 2026-05-06T18:59:24Z Stashbot 7414 dani@deploy1003: helmfile [eqiad] START helmfile.d/services/miscweb: apply 2411307 wikitext text/x-wiki co5k4serueen6jzljpsdos4lks6h67z 2411308 2411307 2026-05-06T18:59:36Z Stashbot 7414 dani@deploy1003: helmfile [eqiad] DONE helmfile.d/services/miscweb: apply 2411308 wikitext text/x-wiki e03yaf99ij1rhjbf40nux6f1onnlgk4 2411309 2411308 2026-05-06T18:59:37Z Stashbot 7414 dani@deploy1003: helmfile [codfw] START helmfile.d/services/miscweb: apply 2411309 wikitext text/x-wiki aqvceysg20ka10h94lrsjmrbm8vpsva 2411310 2411309 2026-05-06T18:59:52Z Stashbot 7414 dani@deploy1003: helmfile [codfw] DONE helmfile.d/services/miscweb: apply 2411310 wikitext text/x-wiki q7smhwpv34ioj9ariwrw3h5vgyj8s6m 2411311 2411310 2026-05-06T19:01:09Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudelastic1012.eqiad.wmnet with reason: host reimage 2411311 wikitext text/x-wiki fjd920dd5069fjr3odrwdldutnwnxbw 2411312 2411311 2026-05-06T19:05:30Z Stashbot 7414 bking@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudelastic1012.eqiad.wmnet with reason: host reimage 2411312 wikitext text/x-wiki oox18u8s8kza2z97lr0jqwj0024izvr 2411314 2411312 2026-05-06T19:14:53Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lvs4010.ulsfo.wmnet with OS bookworm 2411314 wikitext text/x-wiki 1erzzil9ulz0m9ria5iz47hf451dap0 2411315 2411314 2026-05-06T19:23:25Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp4050.ulsfo.wmnet with OS trixie 2411315 wikitext text/x-wiki 302eulhnsrp3g4ripixxoqz3kyd1gif 2411316 2411315 2026-05-06T19:24:05Z Stashbot 7414 bking@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudelastic1012.eqiad.wmnet with OS trixie 2411316 wikitext text/x-wiki mc6yvzmxyn2qg64rhfxx8p5ecw2pcy3 2411317 2411316 2026-05-06T19:30:56Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lvs4010.ulsfo.wmnet with OS trixie 2411317 wikitext text/x-wiki bwjpn0yglx5dtj5hsdkw5i4t07i47lm 2411319 2411317 2026-05-06T19:57:10Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.hosts.reimage for host cloudelastic1012.eqiad.wmnet with OS trixie 2411319 wikitext text/x-wiki clo11dtezanm0aeyyicmt8ch66z6kao 2411320 2411319 2026-05-06T20:00:08Z Stashbot 7414 bking@cumin2002: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudelastic1012.eqiad.wmnet with OS trixie 2411320 wikitext text/x-wiki lbdlnzr4lwajrsrjf45aq9lzetgqalx 2411321 2411320 2026-05-06T20:14:15Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.hosts.reimage for host cloudelastic1012.eqiad.wmnet with OS trixie 2411321 wikitext text/x-wiki 9efq8hcu2p2tmmkp45djild75l9qvyu 2411323 2411321 2026-05-06T20:19:33Z Stashbot 7414 catrope@deploy1003: Started scap sync-world: Backport for [[gerrit:1281526|Replace use of $wgRequest (T336703)]] 2411323 wikitext text/x-wiki e4pzibv9l96zacxk6cfcil4m8oy7x0w 2411325 2411323 2026-05-06T20:21:33Z Stashbot 7414 catrope@deploy1003: catrope, somerandomdeveloper: Backport for [[gerrit:1281526|Replace use of $wgRequest (T336703)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411325 wikitext text/x-wiki 9qsd4y8js50f4rc1o0plcsbhcnxvydm 2411326 2411325 2026-05-06T20:24:29Z Stashbot 7414 catrope@deploy1003: catrope, somerandomdeveloper: Continuing with deployment 2411326 wikitext text/x-wiki ps0wrma6ia1f4zdz8exzit5quuixexn 2411327 2411326 2026-05-06T20:25:55Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudelastic1012.eqiad.wmnet with reason: host reimage 2411327 wikitext text/x-wiki 32vpnbbf152w7umtzvxiw7zd7gd5xll 2411328 2411327 2026-05-06T20:28:45Z Stashbot 7414 catrope@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281526|Replace use of $wgRequest (T336703)]] (duration: 09m 12s) 2411328 wikitext text/x-wiki a9z9yafk63csinldeycrju6ssurk5vz 2411329 2411328 2026-05-06T20:29:26Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411329 wikitext text/x-wiki 15ik2qy2nkk35oz00501x1j1rwazown 2411330 2411329 2026-05-06T20:29:50Z Stashbot 7414 bking@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudelastic1012.eqiad.wmnet with reason: host reimage 2411330 wikitext text/x-wiki p1ame2qrb00gcur0edb1snxemuulx05 2411338 2411330 2026-05-06T21:04:52Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411338 wikitext text/x-wiki 0n62bltj7gn731el4othlovyjdqxqu7 2411339 2411338 2026-05-06T21:05:55Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host pc1021 2411339 wikitext text/x-wiki b3pnuy282k9xry12mqr4myhra1ab91q 2411340 2411339 2026-05-06T21:06:56Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host pc1021 2411340 wikitext text/x-wiki 2invfeb6k2lg9uffft2gdyp2170def1 2411341 2411340 2026-05-06T21:07:42Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.dns.netbox 2411341 wikitext text/x-wiki qus54kv5c0k2trzzzy7x86pnmd3jpud 2411342 2411341 2026-05-06T21:11:48Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [pc1021] - vriley@cumin1003" 2411342 wikitext text/x-wiki 7zgphvrnhcax35oiz1ew7iacqv5pyxo 2411343 2411342 2026-05-06T21:11:54Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [pc1021] - vriley@cumin1003" 2411343 wikitext text/x-wiki 5qqrqwc4jgjurc2qp0ya3rsnqh1h33i 2411344 2411343 2026-05-06T21:11:55Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411344 wikitext text/x-wiki bk1o6venmqoc83ferk2qblajrtdmlxb 2411345 2411344 2026-05-06T21:12:57Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411345 wikitext text/x-wiki f7gs8tqzpgmgli5gsww9risjmmrfhlj 2411346 2411345 2026-05-06T21:14:16Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411346 wikitext text/x-wiki 6mkv78fseikk7c93xv80hdpfapp8cdb 2411347 2411346 2026-05-06T21:15:11Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411347 wikitext text/x-wiki 54pvc40d8pnkc3ow2zfbbiqoew7l413 2411348 2411347 2026-05-06T21:17:17Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411348 wikitext text/x-wiki ibuc75b815yxe2wnxeit838dbfe9803 2411349 2411348 2026-05-06T21:22:17Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411349 wikitext text/x-wiki kkz8he1s1dp1wvhraflzkymu3ic43ik 2411350 2411349 2026-05-06T21:24:10Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411350 wikitext text/x-wiki iy990t82f9lyfs8yyumf5f7rr8cu212 2411351 2411350 2026-05-06T21:26:59Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411351 wikitext text/x-wiki 45crtkbaraybh5za7jh1pmpnq59ztc4 2411352 2411351 2026-05-06T21:27:58Z Stashbot 7414 bking@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudelastic1012.eqiad.wmnet with OS trixie 2411352 wikitext text/x-wiki qjk2yy0rtk04hec6zuj6hncakyq4o1k 2411353 2411352 2026-05-06T21:28:27Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411353 wikitext text/x-wiki ia08hnljibsqvb50e1yw4e1bfba336j 2411354 2411353 2026-05-06T21:29:32Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411354 wikitext text/x-wiki 8c7nxgjk698p1lzfjt4h9lgjo6y50ht 2411355 2411354 2026-05-06T21:31:36Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411355 wikitext text/x-wiki kqsbnfeh5e7pih0sz5238rs0webl0tk 2411357 2411355 2026-05-06T21:45:40Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1283953|Disable GNSM on dewikinews (T421798)]] 2411357 wikitext text/x-wiki g2c6ul13qxukktuijbimlcb806m9mtz 2411358 2411357 2026-05-06T21:47:36Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1283953|Disable GNSM on dewikinews (T421798)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411358 wikitext text/x-wiki pn60dgloyrwhs9kjikbs1d5lwy7uz8x 2411359 2411358 2026-05-06T21:48:24Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2411359 wikitext text/x-wiki hoxl5zz0kj9recymlg6ezmr3zmwbbtv 2411361 2411359 2026-05-06T21:52:36Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1283953|Disable GNSM on dewikinews (T421798)]] (duration: 06m 56s) 2411361 wikitext text/x-wiki 6sxwxn9ccoc7r2vr74pkeefdsexsne6 2411362 2411361 2026-05-06T22:04:35Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host pc1021 2411362 wikitext text/x-wiki nrd6urk50tnsw0rqepbuyfiy6ab5e8b 2411363 2411362 2026-05-06T22:05:36Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host pc1021 2411363 wikitext text/x-wiki mbvl4mxzc0h3g66arwnnhj7fjnjm0q0 2411364 2411363 2026-05-06T22:06:10Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.dns.netbox 2411364 wikitext text/x-wiki 7hi43omr5lm8r4dzhzhmn9bznku9y1m 2411365 2411364 2026-05-06T22:08:13Z Stashbot 7414 cjming@deploy1003: Started scap sync-world: Backport for [[gerrit:1283972|UBN fix: guard entry.serverTiming before forEach (T425591)]] 2411365 wikitext text/x-wiki bj871qj2ih4bufh0tmb25g859tf31j9 2411366 2411365 2026-05-06T22:08:51Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411366 wikitext text/x-wiki 8gaph2mw3uagxlpf1bcixz9jn1vgd2m 2411367 2411366 2026-05-06T22:09:25Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411367 wikitext text/x-wiki mr30m5uhnu6k6n705jchmsukaqufppq 2411368 2411367 2026-05-06T22:10:06Z Stashbot 7414 cjming@deploy1003: cjming: Backport for [[gerrit:1283972|UBN fix: guard entry.serverTiming before forEach (T425591)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411368 wikitext text/x-wiki 8kiyyvm7itaym8go9l5oqlcrkokd9a5 2411369 2411368 2026-05-06T22:10:28Z Stashbot 7414 cjming@deploy1003: cjming: Continuing with deployment 2411369 wikitext text/x-wiki p48fzjs9giq4fqren5vrlmy0qpyc0mr 2411370 2411369 2026-05-06T22:11:26Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411370 wikitext text/x-wiki e9hqb3pyntgw47uzzo4bxpyengtisoz 2411371 2411370 2026-05-06T22:11:49Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411371 wikitext text/x-wiki badwy6lbemn9616b5xj2ure682s2mrl 2411372 2411371 2026-05-06T22:14:38Z Stashbot 7414 cjming@deploy1003: Finished scap sync-world: Backport for [[gerrit:1283972|UBN fix: guard entry.serverTiming before forEach (T425591)]] (duration: 06m 25s) 2411372 wikitext text/x-wiki n521pboci7zgh12c792qh9vsb108ybt 2411373 2411372 2026-05-06T22:14:41Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host pc1021.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2411373 wikitext text/x-wiki oty65d67dj636i3vanhllwwblidx21q 2411374 2411373 2026-05-06T22:18:29Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.reimage for host pc1021.eqiad.wmnet with OS trixie 2411374 wikitext text/x-wiki sqoahqc7s3ooeqqlvqjeef2bcdart6y 2411375 2411374 2026-05-06T22:26:30Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1283872|Close English Wikinews (T421796)]] 2411375 wikitext text/x-wiki cv3l7jofbhhr609l1kxx2p1gubvh9g9 2411376 2411375 2026-05-06T22:28:26Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1283872|Close English Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411376 wikitext text/x-wiki elyp6f7kn4l7m50j5zqdisdn8ewzhh0 2411377 2411376 2026-05-06T22:28:59Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2411377 wikitext text/x-wiki oypr0q87cyzea9hfdlvbjg71afoue2a 2411378 2411377 2026-05-06T22:33:11Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1283872|Close English Wikinews (T421796)]] (duration: 06m 40s) 2411378 wikitext text/x-wiki ky0uogrula4eafkqy9gy5zyy5fhqbko 2411380 2411378 2026-05-06T22:43:18Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1284004|Close Spanish Wikinews (T421796)]] 2411380 wikitext text/x-wiki 62sij2actxz46kg0oxffm5zylcudikt 2411381 2411380 2026-05-06T22:45:14Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1284004|Close Spanish Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411381 wikitext text/x-wiki 0eq4z2mof7q0y21q88nru1rssqaqz9l 2411382 2411381 2026-05-06T22:46:12Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2411382 wikitext text/x-wiki 6yn5mfvozf4fvwrhg1vohetfdk9mjcq 2411383 2411382 2026-05-06T22:50:26Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1284004|Close Spanish Wikinews (T421796)]] (duration: 07m 08s) 2411383 wikitext text/x-wiki p9snbwc8b9jtgz6qe3q6ucq5n68faty 2411384 2411383 2026-05-06T23:12:25Z Stashbot 7414 ladsgroup@deploy1003: Synchronized portals/wikipedia.org/assets: Sync portals for removal of Wikinews (duration: 06m 12s) 2411384 wikitext text/x-wiki p1xfjllwk6rwv6c76w67llqti014uho 2411385 2411384 2026-05-06T23:14:49Z Stashbot 7414 ladsgroup@deploy1003: Synchronized portals: Sync portals for removal of Wikinews (duration: 02m 22s) 2411385 wikitext text/x-wiki t04jupo351u18cb4xkq0mpjpuu3own0 2411420 2411385 2026-05-06T23:38:46Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host pc1021.eqiad.wmnet with OS trixie 2411420 wikitext text/x-wiki 9p6egmyemhal9bmwbi6kl5lzsh0mi9s 2411426 2411420 2026-05-06T23:41:23Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.reimage for host pc1021.eqiad.wmnet with OS trixie 2411426 wikitext text/x-wiki 2ds1ffocu1tnk767awo0ars584zyll8 2411483 2411426 2026-05-07T00:10:04Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1277783|Undeploy GoogleNewsSitemap (T421798)]] 2411483 wikitext text/x-wiki o37tb9mb4bbpr568eknaztbyvzmm765 2411521 2411483 2026-05-07T00:29:06Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1277783|Undeploy GoogleNewsSitemap (T421798)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411521 wikitext text/x-wiki 0k1i2tl5fzcyn2d7j8blk754z304sv8 2411526 2411521 2026-05-07T00:31:26Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2411526 wikitext text/x-wiki 3wxse3q05grq14yhimtef4f4rfgi3jf 2411535 2411526 2026-05-07T00:43:59Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277783|Undeploy GoogleNewsSitemap (T421798)]] (duration: 33m 54s) 2411535 wikitext text/x-wiki tq7dar88fvkpy0dso6pfm017arsps0t 2411536 2411535 2026-05-07T01:01:39Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host pc1021.eqiad.wmnet with OS trixie 2411536 wikitext text/x-wiki n7s7ky10fctxd6qtwtmo7ga0hokqhq2 2411537 2411536 2026-05-07T01:02:59Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281485|Drop some unneeded wikinews configs (T421796)]] 2411537 wikitext text/x-wiki 0llvr6sb3tpu31r2yer8qh2hnhxycln 2411538 2411537 2026-05-07T01:09:19Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1281485|Drop some unneeded wikinews configs (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411538 wikitext text/x-wiki 4nyjuxzvr91b5x7ygwhvx1kn0u8g84c 2411539 2411538 2026-05-07T01:09:37Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2411539 wikitext text/x-wiki ifdlnxf23dz6ubpvid6uy3kycws1ki7 2411540 2411539 2026-05-07T01:15:57Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281485|Drop some unneeded wikinews configs (T421796)]] (duration: 12m 57s) 2411540 wikitext text/x-wiki 4yibtaxp30jvpq4nn08eiqbloot1luq 2411541 2411540 2026-05-07T02:01:09Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2411541 wikitext text/x-wiki azrv2gfdt1rihd52ifjflu50pv2xmyf 2411542 2411541 2026-05-07T02:07:45Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 35s) 2411542 wikitext text/x-wiki 43k78wvao18bmicpouex31rl8tzvtgj 2411543 2411542 2026-05-07T04:46:04Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2208.codfw.wmnet with OS trixie 2411543 wikitext text/x-wiki arcw8g1epdcqyuxqr6gbyfa35sjvqci 2411544 2411543 2026-05-07T04:46:53Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Set db2204 with weight 0 T424848', diff saved to https://phabricator.wikimedia.org/P92381 and previous config saved to /var/cache/conftool/dbconfig/20260507-044651-marostegui.json 2411544 wikitext text/x-wiki 4z42hfakr8q1e9q3t6vujd83jfc0y0x 2411545 2411544 2026-05-07T04:47:12Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 25 hosts with reason: Primary switchover s2 T424848 2411545 wikitext text/x-wiki 610fbszcj82g2ddj2m5c2prj5je2c3l 2411547 2411545 2026-05-07T04:51:02Z Stashbot 7414 marostegui: Starting s2 codfw failover from db2207 to db2204 - T424848 2411547 wikitext text/x-wiki 5m4qs2rtt3rnaj2vayo8h0736uoso92 2411548 2411547 2026-05-07T04:51:44Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Promote db2204 to s2 primary T424848', diff saved to https://phabricator.wikimedia.org/P92382 and previous config saved to /var/cache/conftool/dbconfig/20260507-045141-marostegui.json 2411548 wikitext text/x-wiki qq0m71ygfx85csulecsn0tssrcbb79g 2411549 2411548 2026-05-07T04:52:21Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool db2207 T424848', diff saved to https://phabricator.wikimedia.org/P92383 and previous config saved to /var/cache/conftool/dbconfig/20260507-045219-marostegui.json 2411549 wikitext text/x-wiki 52o0ssdkc9hj0m8axxyz2vlk0ndal1c 2411550 2411549 2026-05-07T05:01:24Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2207.codfw.wmnet with reason: Reimage to Trixie 2411550 wikitext text/x-wiki 6dc3t64s2yohk8pn05tid5kspi1sfv8 2411551 2411550 2026-05-07T05:01:30Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2207: Reimage to Trixie 2411551 wikitext text/x-wiki 4p2karfrtiffnsijlsroqfiur5is7bd 2411552 2411551 2026-05-07T05:01:38Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2207: Reimage to Trixie 2411552 wikitext text/x-wiki h6mg3n0fh34hothlcj8s7kfstfcouq1 2411553 2411552 2026-05-07T05:03:30Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2207.codfw.wmnet with OS trixie 2411553 wikitext text/x-wiki auyfyze8mlojplid4pw61dd3xhft03u 2411554 2411553 2026-05-07T05:04:45Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2208.codfw.wmnet with reason: host reimage 2411554 wikitext text/x-wiki bikkohpq0pel91817vsjmhmx06ifviv 2411555 2411554 2026-05-07T05:09:46Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2208.codfw.wmnet with reason: host reimage 2411555 wikitext text/x-wiki 5cpyszj1j9lhd3jtvfx23pp5vrgp42i 2411556 2411555 2026-05-07T05:23:02Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2207.codfw.wmnet with reason: host reimage 2411556 wikitext text/x-wiki ov43508tz6gjvxyatrp4i4uen58ehjj 2411557 2411556 2026-05-07T05:28:16Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2207.codfw.wmnet with reason: host reimage 2411557 wikitext text/x-wiki h3zbn3o0vz27ov8hsupp4be5lq6bjc8 2411558 2411557 2026-05-07T05:33:13Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2208.codfw.wmnet with OS trixie 2411558 wikitext text/x-wiki 821204y5sfgu75e2l9jifu378y4f7z2 2411560 2411558 2026-05-07T05:51:12Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2207.codfw.wmnet with OS trixie 2411560 wikitext text/x-wiki g32obgt2xaw5is4edi9zd5tf1i93a3i 2411561 2411560 2026-05-07T05:54:12Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2207: after reimage to trixie 2411561 wikitext text/x-wiki b9dq07uu831y0l4bhq7xzea4d3ps9ri 2411562 2411561 2026-05-07T06:39:37Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2207: after reimage to trixie 2411562 wikitext text/x-wiki 1akfasj1s4a1mkxs5scelpeni5dz93l 2411563 2411562 2026-05-07T06:41:08Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.addnode for new host ganeti4006.ulsfo.wmnet to cluster ulsfo02 and group 01 2411563 wikitext text/x-wiki sfjnx6ij9ruis0ijv9u3y4w34bj6emv 2411564 2411563 2026-05-07T06:42:10Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.ganeti.addnode (exit_code=99) for new host ganeti4006.ulsfo.wmnet to cluster ulsfo02 and group 01 2411564 wikitext text/x-wiki toomz3yerr4l5gxtmubbvfg5e6n9aw0 2411565 2411564 2026-05-07T06:46:08Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.changedisk for changing disk type of ncredir4003.ulsfo.wmnet to drbd 2411565 wikitext text/x-wiki 7c2dfab6m6o3mp3fkbg0lirq3w8a1zr 2411566 2411565 2026-05-07T06:46:30Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.ganeti.changedisk (exit_code=99) for changing disk type of ncredir4003.ulsfo.wmnet to drbd 2411566 wikitext text/x-wiki 5iagop4thx6kvk8017lyvpigrt1v1z8 2411567 2411566 2026-05-07T06:48:09Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.changedisk for changing disk type of ncredir4003.ulsfo.wmnet to drbd 2411567 wikitext text/x-wiki 3cpo83969yfohyhwizk3d1n2ib9s88j 2411568 2411567 2026-05-07T07:02:37Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of ncredir4003.ulsfo.wmnet to drbd 2411568 wikitext text/x-wiki kp1wv67tnnd5gge0bdcw1x2itq3c37o 2411569 2411568 2026-05-07T07:23:04Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.reboot-vm for VM testvm2005.codfw.wmnet 2411569 wikitext text/x-wiki iaogvfl3fjc0dus8tsitac9fcpq1t3u 2411570 2411569 2026-05-07T07:27:01Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.reboot-vm (exit_code=0) for VM testvm2005.codfw.wmnet 2411570 wikitext text/x-wiki gvfzm8bdkif0brt4keqg5z8wlae0c50 2411571 2411570 2026-05-07T07:30:15Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.changedisk for changing disk type of netflow4003.ulsfo.wmnet to drbd 2411571 wikitext text/x-wiki hs5udjcjbc6d09lxvaylj5zf5o4ohe8 2411572 2411571 2026-05-07T07:32:17Z Stashbot 7414 moritzm: installing apache2 security updates 2411572 wikitext text/x-wiki 6bn3ashvybct7k7kzz9un3fdrmqbmma 2411574 2411572 2026-05-07T07:44:22Z Stashbot 7414 dcausse@deploy1003: Started scap sync-world: Backport for [[gerrit:1269465|search: add alt. completion indices to test keyword tokenizer (2/2) (T420427)]] 2411574 wikitext text/x-wiki 3tpcwtujh0gazod26qmuw45vy4wfws1 2411575 2411574 2026-05-07T07:44:47Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of netflow4003.ulsfo.wmnet to drbd 2411575 wikitext text/x-wiki 2cy84s8w1iemuddlyd4ypax63d5p75i 2411576 2411575 2026-05-07T07:46:28Z Stashbot 7414 dcausse@deploy1003: dcausse: Backport for [[gerrit:1269465|search: add alt. completion indices to test keyword tokenizer (2/2) (T420427)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411576 wikitext text/x-wiki e5b18f4t6zra8whf03x7njr8jcxbyav 2411577 2411576 2026-05-07T07:49:53Z Stashbot 7414 dcausse@deploy1003: dcausse: Continuing with deployment 2411577 wikitext text/x-wiki 5k0aetlli8z5qdr8f4otsb22iqthxg4 2411578 2411577 2026-05-07T07:54:08Z Stashbot 7414 dcausse@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269465|search: add alt. completion indices to test keyword tokenizer (2/2) (T420427)]] (duration: 09m 46s) 2411578 wikitext text/x-wiki dp6hk1t5txs8wmaybirnxea7zhdaed0 2411579 2411578 2026-05-07T08:03:36Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.addnode for new host ganeti4008.ulsfo.wmnet to cluster ulsfo02 and group 01 2411579 wikitext text/x-wiki 1z19dy2iisi447mmlelwn2n42e9l4kf 2411580 2411579 2026-05-07T08:03:41Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.ganeti.addnode (exit_code=99) for new host ganeti4008.ulsfo.wmnet to cluster ulsfo02 and group 01 2411580 wikitext text/x-wiki gcjt9jij498me238qw5gb84s7mb5uvu 2411581 2411580 2026-05-07T08:04:28Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti4008.ulsfo.wmnet 2411581 wikitext text/x-wiki 2h5a2i72wsve3qywzq1v5sxj8lrkhdi 2411582 2411581 2026-05-07T08:12:22Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti4008.ulsfo.wmnet 2411582 wikitext text/x-wiki gpsvtbbyofwk0p353fn3n4zv30lsbsb 2411583 2411582 2026-05-07T08:12:40Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.addnode for new host ganeti4008.ulsfo.wmnet to cluster ulsfo02 and group 01 2411583 wikitext text/x-wiki s50v888lr109fr1znonxde0671thm4g 2411584 2411583 2026-05-07T08:12:47Z Stashbot 7414 elukey@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: sync 2411584 wikitext text/x-wiki j4fr65yco4e02jvn9uqbrkw39dr3hck 2411585 2411584 2026-05-07T08:12:54Z Stashbot 7414 elukey@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: sync 2411585 wikitext text/x-wiki g2d5bm1m7chhrgqqdwom9ejlohg44g8 2411586 2411585 2026-05-07T08:13:33Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.addnode (exit_code=0) for new host ganeti4008.ulsfo.wmnet to cluster ulsfo02 and group 01 2411586 wikitext text/x-wiki rk10sg8zbyl3km4nu378byf7aaj96lk 2411587 2411586 2026-05-07T08:14:19Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.changedisk for changing disk type of ncredir4004.ulsfo.wmnet to drbd 2411587 wikitext text/x-wiki jtgtcwi7gs34nxg82y7f7aig5midqs7 2411588 2411587 2026-05-07T08:17:15Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.dns.netbox 2411588 wikitext text/x-wiki nrx3l543ay63yn47qlitk2h1g0d2tgj 2411589 2411588 2026-05-07T08:21:39Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: drmrs v6 gateway IPs change - ayounsi@cumin1003" 2411589 wikitext text/x-wiki 0b0ehf4rkics0yd52lt92o9gzzb3p3u 2411590 2411589 2026-05-07T08:22:18Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2208: After reimage 2411590 wikitext text/x-wiki hh91xwzx0hb6s1yt3gd5iehj2vdr1d6 2411591 2411590 2026-05-07T08:22:28Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: drmrs v6 gateway IPs change - ayounsi@cumin1003" 2411591 wikitext text/x-wiki ddjgxqnlnv5o4t48mnbky558l987as0 2411592 2411591 2026-05-07T08:22:29Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411592 wikitext text/x-wiki tlr5g66m6ka5lpj3o1x5yls3751hbef 2411593 2411592 2026-05-07T08:23:03Z Stashbot 7414 XioNoX: drmrs remove old v6 gateway IP 2411593 wikitext text/x-wiki 9tvswn6wz0u0o9o25hktfndfingqhsp 2411594 2411593 2026-05-07T08:23:08Z Stashbot 7414 marostegui@cumin1003: END (ERROR) - Cookbook sre.mysql.pool (exit_code=97) pool db2208: After reimage 2411594 wikitext text/x-wiki 7i9zfb3ylq40m1etvd7g3yei8pbdj2z 2411595 2411594 2026-05-07T08:23:21Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2208: After reimage 2411595 wikitext text/x-wiki mopc1wudnfov29x5ummlqbu401u41zi 2411596 2411595 2026-05-07T08:28:25Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Remove db2144 T425522', diff saved to https://phabricator.wikimedia.org/P92389 and previous config saved to /var/cache/conftool/dbconfig/20260507-082822-marostegui.json 2411596 wikitext text/x-wiki r5bivgjk2ztdfi3ak8qe7vm6ufxvhxj 2411597 2411596 2026-05-07T08:28:54Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of ncredir4004.ulsfo.wmnet to drbd 2411597 wikitext text/x-wiki tq8hxumniaf0x0gxhxsb8i8aguy6fy2 2411598 2411597 2026-05-07T08:29:31Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.changedisk for changing disk type of prometheus4003.ulsfo.wmnet to drbd 2411598 wikitext text/x-wiki a1celphnabbmh496oxl36rttoaf1zye 2411599 2411598 2026-05-07T08:32:09Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2144.codfw.wmnet 2411599 wikitext text/x-wiki 8s4b1q4kalc85hliyn0xsy6olsitp55 2411600 2411599 2026-05-07T08:37:32Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.dns.netbox 2411600 wikitext text/x-wiki l18ucluuag2o2tauee41bp4nr6v4ixr 2411601 2411600 2026-05-07T08:42:26Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db2144.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003" 2411601 wikitext text/x-wiki jc2ulnneflnau4fwa1bw2zflsk209dw 2411602 2411601 2026-05-07T08:42:44Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db2144.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003" 2411602 wikitext text/x-wiki 4r0yrnw37ncmk1kyu2n9ji7wumddyd5 2411603 2411602 2026-05-07T08:42:45Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411603 wikitext text/x-wiki cbvzcnn4a2j0c3x0nkzfjwf9fcz8zei 2411604 2411603 2026-05-07T08:42:46Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2144.codfw.wmnet 2411604 wikitext text/x-wiki b5hfuc1tja542ea1sze82vkwq1qrpvh 2411605 2411604 2026-05-07T08:50:53Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1202.eqiad.wmnet with reason: Reimage to Trixie 2411605 wikitext text/x-wiki qqr66omcm8nk74u9xfvrm9w9a5bch6f 2411606 2411605 2026-05-07T08:50:58Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1202: Reimage to Trixie 2411606 wikitext text/x-wiki c7j38a9ttti1ennfp7gjkbblj4ywixd 2411607 2411606 2026-05-07T08:51:04Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2182.codfw.wmnet with reason: Reimage to Trixie 2411607 wikitext text/x-wiki 44wpwpwyv7z98ewhdpxevcy5dd71kuy 2411608 2411607 2026-05-07T08:51:10Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2182: Reimage to Trixie 2411608 wikitext text/x-wiki pbkb9kp7rwq543e9g42n5y1kaw95vkp 2411609 2411608 2026-05-07T08:51:26Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2182: Reimage to Trixie 2411609 wikitext text/x-wiki 7jj7uiih24323ltvbabfomqhsw9zlof 2411610 2411609 2026-05-07T08:51:34Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1202: Reimage to Trixie 2411610 wikitext text/x-wiki qlzjltxbepoas3cqcg8tgzc8ksuwmhx 2411611 2411610 2026-05-07T08:52:41Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1202.eqiad.wmnet with OS trixie 2411611 wikitext text/x-wiki 2ycad9uxtfwmgs19ok239v18iq369lk 2411612 2411611 2026-05-07T08:52:47Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2182.codfw.wmnet with OS trixie 2411612 wikitext text/x-wiki 0tc8atlduond2azhfojrk5yiblvzn8w 2411614 2411612 2026-05-07T09:07:02Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1202.eqiad.wmnet with reason: host reimage 2411614 wikitext text/x-wiki ox22bcqqrh2c2xfl2c73yqqxv3b19xd 2411615 2411614 2026-05-07T09:08:43Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2208: After reimage 2411615 wikitext text/x-wiki d6ukvkciz9iqzg7wzc9a2vn5xuyyb23 2411616 2411615 2026-05-07T09:11:17Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.reboot-vm for VM bast4006.wikimedia.org 2411616 wikitext text/x-wiki ionp2575212vgtn0btzg2vef6nuj4bl 2411617 2411616 2026-05-07T09:11:45Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2182.codfw.wmnet with reason: host reimage 2411617 wikitext text/x-wiki ojtkdt650yvdgjywupjlj52ui0hhjld 2411618 2411617 2026-05-07T09:14:19Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1202.eqiad.wmnet with reason: host reimage 2411618 wikitext text/x-wiki c5rkxsj2uo4y0on2p15y21j16wel7yw 2411619 2411618 2026-05-07T09:15:05Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.reboot-vm (exit_code=0) for VM bast4006.wikimedia.org 2411619 wikitext text/x-wiki ovo7va9ye9v39jg8iakshshbgddvrc0 2411622 2411619 2026-05-07T09:18:33Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2182.codfw.wmnet with reason: host reimage 2411622 wikitext text/x-wiki bbnbwo73gdb32qc4ar9kgy00b08qq05 2411623 2411622 2026-05-07T09:19:44Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host bast4006.wikimedia.org with OS trixie 2411623 wikitext text/x-wiki a2z6gzelqlgayq1isb4txjtg4i4p5l2 2411624 2411623 2026-05-07T09:24:33Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts sretest1006.eqiad.wmnet 2411624 wikitext text/x-wiki rk2lbpbrj1h0suju7vcmcyuchzjm97f 2411625 2411624 2026-05-07T09:25:22Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts sretest1006.eqiad.wmnet 2411625 wikitext text/x-wiki hl15csz1xwczsf2g74319g5rpqvg0i4 2411626 2411625 2026-05-07T09:28:20Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of prometheus4003.ulsfo.wmnet to drbd 2411626 wikitext text/x-wiki kaxxaz3x3vs8fis5s0jdhc71mka0nzi 2411627 2411626 2026-05-07T09:32:33Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.changedisk for changing disk type of hcaptcha-proxy4003.wikimedia.org to drbd 2411627 wikitext text/x-wiki f0v2zw8zq8mg65hgxfl464jjlju4e4z 2411628 2411627 2026-05-07T09:35:53Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1245.eqiad.wmnet with reason: Maintenance 2411628 wikitext text/x-wiki e4h4tu7wulz6dw0adqdhuvsjvixsv3d 2411629 2411628 2026-05-07T09:37:06Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1202.eqiad.wmnet with OS trixie 2411629 wikitext text/x-wiki 4xai9dkikxcvwqvrod6es1h0kgsyaz3 2411630 2411629 2026-05-07T09:39:57Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1202: after reimage to trixie 2411630 wikitext text/x-wiki kjd382ym5s0nm34aoz6q3wfx4dqbx3i 2411631 2411630 2026-05-07T09:41:16Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2182.codfw.wmnet with OS trixie 2411631 wikitext text/x-wiki nwx50bx2ee3pyrum2iseb7ge2zz590d 2411632 2411631 2026-05-07T09:42:36Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of hcaptcha-proxy4003.wikimedia.org to drbd 2411632 wikitext text/x-wiki 5yotqvkcfhs7fb6qnqiu7cm4pa7vxyc 2411633 2411632 2026-05-07T09:44:51Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2182: after reimage to trixie 2411633 wikitext text/x-wiki jrj2par0mmuojn423mnohaog8xt4twx 2411634 2411633 2026-05-07T09:49:40Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on bast4006.wikimedia.org with reason: host reimage 2411634 wikitext text/x-wiki 3m1nvulfazhg4tjmvedez3pm0uyghro 2411635 2411634 2026-05-07T09:53:48Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on bast4006.wikimedia.org with reason: host reimage 2411635 wikitext text/x-wiki px8uu40lccherjfpu8nl1j5r8mk8qoo 2411636 2411635 2026-05-07T09:54:15Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.decommission for hosts hcaptcha-proxy5001.wikimedia.org 2411636 wikitext text/x-wiki 21o2gtqtqwppeckhqzurfigvax83rkh 2411637 2411636 2026-05-07T09:59:04Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2411637 wikitext text/x-wiki p1kd7apoozk177bi430lmcohfa6s4hg 2411638 2411637 2026-05-07T10:03:29Z Stashbot 7414 daniel@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2411638 wikitext text/x-wiki skh5h3ui28xgmws59s3awmzpwk4c0ry 2411639 2411638 2026-05-07T10:03:54Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: hcaptcha-proxy5001.wikimedia.org decommissioned, removing all IPs except the asset tag one - jmm@cumin2002" 2411639 wikitext text/x-wiki hflheip43onkasjc5oqr0pvlnm18soi 2411640 2411639 2026-05-07T10:04:51Z Stashbot 7414 daniel@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2411640 wikitext text/x-wiki tw3y2i2mvmf0et0xwhx6k9mkyniy4qo 2411641 2411640 2026-05-07T10:10:50Z Stashbot 7414 daniel@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply 2411641 wikitext text/x-wiki anavag8nrhxcdbeznyl94c107ochjmp 2411642 2411641 2026-05-07T10:10:56Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host bast4006.wikimedia.org with OS trixie 2411642 wikitext text/x-wiki 202zwo3gi4tr6ovavfcow9iufc7ibr0 2411643 2411642 2026-05-07T10:11:23Z Stashbot 7414 daniel@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply 2411643 wikitext text/x-wiki r2ccwcu1xdodq7jopitrj1kpe1kz6rn 2411644 2411643 2026-05-07T10:11:31Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: hcaptcha-proxy5001.wikimedia.org decommissioned, removing all IPs except the asset tag one - jmm@cumin2002" 2411644 wikitext text/x-wiki mx0bo9hh488pciyz63w59bpuu8y244y 2411645 2411644 2026-05-07T10:11:31Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411645 wikitext text/x-wiki 11ayvahaj0xosfde19abr315ydhj4yk 2411646 2411645 2026-05-07T10:11:32Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts hcaptcha-proxy5001.wikimedia.org 2411646 wikitext text/x-wiki 2lqwfd3u51bdw8kg4ongsiy0tm20l9t 2411647 2411646 2026-05-07T10:14:00Z Stashbot 7414 moritzm: rebalance ganti cluster in ulsfo following host reimages T424686 2411647 wikitext text/x-wiki pf6ss6eqe11nrb7cs8w8y69enhqfc5m 2411648 2411647 2026-05-07T10:14:39Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.dns.admin DNS admin: pool ulsfo [reason: New switch configuration, T408892] 2411648 wikitext text/x-wiki a9ucgz0wsj6d4uu5ldio279zai8xn0p 2411649 2411648 2026-05-07T10:14:47Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: pool ulsfo [reason: New switch configuration, T408892] 2411649 wikitext text/x-wiki cijklt3hpt9tet73aw5yb5voq2j79i4 2411650 2411649 2026-05-07T10:16:50Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.decommission for hosts hcaptcha-proxy5002.wikimedia.org 2411650 wikitext text/x-wiki 6qh9q75efemotae5vjrq294zf1r8x6o 2411651 2411650 2026-05-07T10:20:39Z Stashbot 7414 daniel@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply 2411651 wikitext text/x-wiki 2ibr0qup59xy6wt2e73xhnyisszjucn 2411652 2411651 2026-05-07T10:21:03Z Stashbot 7414 daniel@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply 2411652 wikitext text/x-wiki qgjqazchc5uzcjxepcw1cnv9gw5q9em 2411653 2411652 2026-05-07T10:21:39Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2411653 wikitext text/x-wiki gra6pp0ses1ec2aebyi8rxcwtjqi5ue 2411654 2411653 2026-05-07T10:25:21Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1202: after reimage to trixie 2411654 wikitext text/x-wiki njl0uo1wl49qtk5d6sy315wk7nm9e76 2411655 2411654 2026-05-07T10:26:16Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: hcaptcha-proxy5002.wikimedia.org decommissioned, removing all IPs except the asset tag one - jmm@cumin2002" 2411655 wikitext text/x-wiki i26gs2p7au833e6atagez2odzezsz6s 2411656 2411655 2026-05-07T10:26:29Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1227.eqiad.wmnet with reason: Reimage to Trixie 2411656 wikitext text/x-wiki 3ffoq51yyrezsuvwpz22e6uututuyk4 2411657 2411656 2026-05-07T10:26:34Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1227: Reimage to Trixie 2411657 wikitext text/x-wiki 4r18avayq9h80gc29zc09svv5nvid02 2411658 2411657 2026-05-07T10:27:21Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1227: Reimage to Trixie 2411658 wikitext text/x-wiki kj017fg25ogn8f2e971dkpquiwo8mkb 2411659 2411658 2026-05-07T10:28:25Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1227.eqiad.wmnet with OS trixie 2411659 wikitext text/x-wiki mzkr7tc02e0hwa4suxdmg9tuj6lenmd 2411660 2411659 2026-05-07T10:30:16Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2182: after reimage to trixie 2411660 wikitext text/x-wiki c90cmj9z95utco2pjo52fe18uipgchm 2411661 2411660 2026-05-07T10:30:49Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2168.codfw.wmnet with reason: Reimage to Trixie 2411661 wikitext text/x-wiki hig4ducbu2aor2754nrwhkgdwsefgk0 2411662 2411661 2026-05-07T10:30:54Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2168: Reimage to Trixie 2411662 wikitext text/x-wiki kx1rc9cwiafkx8j0ih0i51z3dd5cbty 2411663 2411662 2026-05-07T10:31:23Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2168: Reimage to Trixie 2411663 wikitext text/x-wiki 90qv9q4l4r8oertd6fdzzogmz0rjkvx 2411664 2411663 2026-05-07T10:32:17Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: hcaptcha-proxy5002.wikimedia.org decommissioned, removing all IPs except the asset tag one - jmm@cumin2002" 2411664 wikitext text/x-wiki q6fn1ua3lhipwup3z7v43d60drp66zo 2411665 2411664 2026-05-07T10:32:18Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411665 wikitext text/x-wiki q1vogyf3wkz0ldqoilkwjdme42tfgvx 2411666 2411665 2026-05-07T10:32:19Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts hcaptcha-proxy5002.wikimedia.org 2411666 wikitext text/x-wiki spbhglpdo9egp716912xscboj5stwwi 2411667 2411666 2026-05-07T10:32:35Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2168.codfw.wmnet with OS trixie 2411667 wikitext text/x-wiki ptf6vp5uzdsyw7nc6lvnjyqvbzd1qqn 2411668 2411667 2026-05-07T10:33:42Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on es1048.eqiad.wmnet with reason: Maintenance 2411668 wikitext text/x-wiki lfiflvaj33lzce6dqmquufh899epx8j 2411669 2411668 2026-05-07T10:33:56Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling es1048 (T419961)', diff saved to https://phabricator.wikimedia.org/P92405 and previous config saved to /var/cache/conftool/dbconfig/20260507-103349-fceratto.json 2411669 wikitext text/x-wiki 2fdtvb2r71ifhbr87sl66h2pscjkcz6 2411670 2411669 2026-05-07T10:35:02Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host install5004.wikimedia.org 2411670 wikitext text/x-wiki iw4fvnou0ee9wga1jd9e1lqftg5kezg 2411671 2411670 2026-05-07T10:35:04Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2411671 wikitext text/x-wiki kof5qet3img3qetl1num0ktlerrelaq 2411673 2411671 2026-05-07T10:39:09Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM install5004.wikimedia.org - jmm@cumin2002" 2411673 wikitext text/x-wiki 7id0ijfi69ma8qey9bl0n9qc5alxzds 2411674 2411673 2026-05-07T10:39:17Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM install5004.wikimedia.org - jmm@cumin2002" 2411674 wikitext text/x-wiki cnwmt7wsyvxq07uy74d84ub6nnmpcve 2411675 2411674 2026-05-07T10:39:18Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411675 wikitext text/x-wiki 5buz1f35460h0mt0i0ep1preosslz5w 2411676 2411675 2026-05-07T10:39:19Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache install5004.wikimedia.org on all recursors 2411676 wikitext text/x-wiki 1ya2nvbqpikkzqzrw54tzw8mouuyp2c 2411677 2411676 2026-05-07T10:39:21Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) install5004.wikimedia.org on all recursors 2411677 wikitext text/x-wiki sv8tms4g9kc9qr55yzwnfzeebprr1p5 2411678 2411677 2026-05-07T10:39:41Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2411678 wikitext text/x-wiki lxe0cy7dfjd2uussfp9ygngm7j5sbgr 2411679 2411678 2026-05-07T10:40:19Z Stashbot 7414 daniel@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2411679 wikitext text/x-wiki 38i0r6jmj7dhzt3gs4ll431q2pnep4x 2411680 2411679 2026-05-07T10:40:55Z Stashbot 7414 daniel@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2411680 wikitext text/x-wiki prg1zac3t9c674xc04468es7lntelps 2411681 2411680 2026-05-07T10:40:56Z Stashbot 7414 jynus@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2184.codfw.wmnet with reason: reimage 2411681 wikitext text/x-wiki gb6ne2t9wpxi48wssjo6oyyeuclpbl2 2411682 2411681 2026-05-07T10:42:34Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1227.eqiad.wmnet with reason: host reimage 2411682 wikitext text/x-wiki azw3v0r0cx61ctt5vj5f8rxow3sbu4m 2411683 2411682 2026-05-07T10:44:00Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance es1048 (T419961)', diff saved to https://phabricator.wikimedia.org/P92406 and previous config saved to /var/cache/conftool/dbconfig/20260507-104359-fceratto.json 2411683 wikitext text/x-wiki elj3uemfanahccaxnjyc1knhh8p4xfn 2411685 2411683 2026-05-07T10:47:32Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1284592|Close Russian Wikinews (T421796)]] 2411685 wikitext text/x-wiki tq2wfsdy3frisi48r9ohmb0y3hvps7x 2411686 2411685 2026-05-07T10:47:42Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) 2411686 wikitext text/x-wiki 164duat6vyus1da36shee3u09i5kf25 2411687 2411686 2026-05-07T10:47:43Z Stashbot 7414 daniel@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply 2411687 wikitext text/x-wiki qkf0326xj9a4sn9ndaqlml697bxi3d7 2411688 2411687 2026-05-07T10:48:09Z Stashbot 7414 daniel@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply 2411688 wikitext text/x-wiki stk57vevrritbyfgq7441xwk1qe0ezd 2411689 2411688 2026-05-07T10:48:10Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1227.eqiad.wmnet with reason: host reimage 2411689 wikitext text/x-wiki hpfo8d75er6hjmuuftth05axfy7169k 2411690 2411689 2026-05-07T10:48:54Z Stashbot 7414 root@cumin1003: START - Cookbook sre.hosts.reimage for host db2184.codfw.wmnet with OS trixie 2411690 wikitext text/x-wiki hhels3fbtxynbouay4j9vwa4d6tgo08 2411691 2411690 2026-05-07T10:49:12Z Stashbot 7414 root@cumin1003: START - Cookbook sre.hosts.move-vlan for host db2184 2411691 wikitext text/x-wiki e0z158oos4374w5rw76acez3go15tb6 2411692 2411691 2026-05-07T10:49:19Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2411692 wikitext text/x-wiki iqauormvwcgvpuf3y5bf98u7pigiefz 2411693 2411692 2026-05-07T10:49:25Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1284592|Close Russian Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2411693 wikitext text/x-wiki g85eu6wiwj0fbilvbp7nfsj8z8p1rbj 2411694 2411693 2026-05-07T10:51:34Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2168.codfw.wmnet with reason: host reimage 2411694 wikitext text/x-wiki rqe40e8yv92t3cqnqjy6a52r3vfi3zf 2411695 2411694 2026-05-07T10:51:55Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2411695 wikitext text/x-wiki elfamjap9vfvofgezdsluvi4jyoe3wq 2411696 2411695 2026-05-07T10:54:08Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance es1048', diff saved to https://phabricator.wikimedia.org/P92407 and previous config saved to /var/cache/conftool/dbconfig/20260507-105407-fceratto.json 2411696 wikitext text/x-wiki gl3o0f2tzr8od75jq9ef0w2f93asage 2411697 2411696 2026-05-07T10:54:58Z Stashbot 7414 root@cumin1003: START - Cookbook sre.dns.netbox 2411697 wikitext text/x-wiki 76nj4yv9n668ls72ssavdxu0xe5hpag 2411698 2411697 2026-05-07T10:55:28Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2168.codfw.wmnet with reason: host reimage 2411698 wikitext text/x-wiki 75vmqhj20ps4ys0gmwzbm7m2li2h9vq 2411699 2411698 2026-05-07T10:56:08Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1284592|Close Russian Wikinews (T421796)]] (duration: 08m 40s) 2411699 wikitext text/x-wiki evj35b1b4hgihe6do7tdqlpj7yjsv4k 2411700 2411699 2026-05-07T10:57:21Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Remove records for VM install5004.wikimedia.org - jmm@cumin2002" 2411700 wikitext text/x-wiki l5x6c429gd97c4puimn7xmhaoecqi1q 2411701 2411700 2026-05-07T10:57:26Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Remove records for VM install5004.wikimedia.org - jmm@cumin2002" 2411701 wikitext text/x-wiki 0ksrprvh3vdbdic5046w10nr15iyyw1 2411702 2411701 2026-05-07T10:57:27Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411702 wikitext text/x-wiki fnn0s35333likxudddvmdlr5hftforw 2411703 2411702 2026-05-07T10:57:28Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache install5004.wikimedia.org on all recursors 2411703 wikitext text/x-wiki f9dce2k6kk9i1wb05b7uzytp6bzymqt 2411704 2411703 2026-05-07T10:57:30Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) install5004.wikimedia.org on all recursors 2411704 wikitext text/x-wiki swlvp9r5qxd97f9xszcp8r2y4d5brxj 2411705 2411704 2026-05-07T10:57:36Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host install5004.wikimedia.org 2411705 wikitext text/x-wiki 3ju5vi7tp1l9u6k7rd68japradexrjp 2411706 2411705 2026-05-07T10:57:45Z Stashbot 7414 root@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411706 wikitext text/x-wiki h8hgd15ia6bw8xkzsh9f3vxdntfvh27 2411707 2411706 2026-05-07T10:57:46Z Stashbot 7414 root@cumin1003: START - Cookbook sre.dns.wipe-cache db2184.codfw.wmnet 129.32.192.10.in-addr.arpa 9.2.1.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2411707 wikitext text/x-wiki p87ev521omahewlhcw8uv9qet11h2ns 2411708 2411707 2026-05-07T10:57:49Z Stashbot 7414 root@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) db2184.codfw.wmnet 129.32.192.10.in-addr.arpa 9.2.1.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2411708 wikitext text/x-wiki r0t4h6lllhx6zlii2urgzcuotivjbio 2411709 2411708 2026-05-07T10:57:50Z Stashbot 7414 root@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host db2184 2411709 wikitext text/x-wiki db0yxh8esvjoky3bfy8kt0kdi58fxvu 2411710 2411709 2026-05-07T10:58:07Z Stashbot 7414 root@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host db2184 2411710 wikitext text/x-wiki 1q8nmwdo9x6dt2xg4atdgh1x3q1amgn 2411711 2411710 2026-05-07T10:58:08Z Stashbot 7414 root@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host db2184 2411711 wikitext text/x-wiki 97nvw9036t8p6wjluvndasbpd8c0s0r 2411712 2411711 2026-05-07T10:58:48Z Stashbot 7414 daniel@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply 2411712 wikitext text/x-wiki 0um9t71vl7nboqfuxbmoqrxud5z87kq 2411713 2411712 2026-05-07T10:59:05Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host install5004.wikimedia.org 2411713 wikitext text/x-wiki b7b6f1ew9rdzzzpy53f9iwq3zbmwvkx 2411714 2411713 2026-05-07T10:59:06Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2411714 wikitext text/x-wiki 1jmwk48iqz6c2djl8xa9c4ws15tld0i 2411715 2411714 2026-05-07T10:59:11Z Stashbot 7414 daniel@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply 2411715 wikitext text/x-wiki 7b8p6mjpkyobapwo8z5wpzs9zbubf3v 2411716 2411715 2026-05-07T11:03:26Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM install5004.wikimedia.org - jmm@cumin2002" 2411716 wikitext text/x-wiki r6ya3q6c9gd450un4hqvn2xe6vxb8oa 2411717 2411716 2026-05-07T11:03:31Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM install5004.wikimedia.org - jmm@cumin2002" 2411717 wikitext text/x-wiki h6cbtdwz6968jwg12s8yzh5jn8bul93 2411718 2411717 2026-05-07T11:03:32Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2411718 wikitext text/x-wiki qtdphyhr76cabzyfy8ccxthjmu2wyq1 2411719 2411718 2026-05-07T11:03:33Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache install5004.wikimedia.org on all recursors 2411719 wikitext text/x-wiki a7cmf4ix9kxujs5o5mnvz94d2tkrext 2411720 2411719 2026-05-07T11:03:35Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) install5004.wikimedia.org on all recursors 2411720 wikitext text/x-wiki 882zgt6waxkj52ft0mv6b3egneyzpjv 2411721 2411720 2026-05-07T11:04:02Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM install5004.wikimedia.org - jmm@cumin2002" 2411721 wikitext text/x-wiki qpgorzvm5mgvuk0pzqkdu1ufx9oq97a 2411722 2411721 2026-05-07T11:04:07Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM install5004.wikimedia.org - jmm@cumin2002" 2411722 wikitext text/x-wiki elxtum640hchuotuampshivck3jd2bx 2411723 2411722 2026-05-07T11:04:17Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance es1048', diff saved to https://phabricator.wikimedia.org/P92409 and previous config saved to /var/cache/conftool/dbconfig/20260507-110415-fceratto.json 2411723 wikitext text/x-wiki 0rxikwgfp9zx19tynxgysawztnxm9zk 2411724 2411723 2026-05-07T11:07:57Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host install5004.wikimedia.org with OS bookworm 2411724 wikitext text/x-wiki pn7woehwgyaz1bywg1nk46lf5d841fl 2411725 2411724 2026-05-07T11:10:39Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1227.eqiad.wmnet with OS trixie 2411725 wikitext text/x-wiki r47k27elnz80q4ui9gsg0tvbgesk9mk 2411726 2411725 2026-05-07T11:11:26Z Stashbot 7414 moritzm: instaling modsecurity-apache security updates 2411726 wikitext text/x-wiki abkc1mnkf7tdjgrnd4cuyq1yi4v7i8i 2411727 2411726 2026-05-07T11:13:16Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1227: after reimage to trixie 2411727 wikitext text/x-wiki lyxigl25fpf0kqtvajzc6biyk7f11r2 2411728 2411727 2026-05-07T11:14:26Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance es1048 (T419961)', diff saved to https://phabricator.wikimedia.org/P92412 and previous config saved to /var/cache/conftool/dbconfig/20260507-111424-fceratto.json 2411728 wikitext text/x-wiki t0qzp9o0nvyo2u71e4nkxi1j0lleder 2411729 2411728 2026-05-07T11:15:53Z Stashbot 7414 root@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2184.codfw.wmnet with reason: host reimage 2411729 wikitext text/x-wiki 53owl5vunyygjktqz2zlhnnd6cnjlhe 2411730 2411729 2026-05-07T11:15:54Z Stashbot 7414 daniel@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2411730 wikitext text/x-wiki e8obdk10000gtyohftynnq1qxc1mzki 2411731 2411730 2026-05-07T11:16:15Z Stashbot 7414 daniel@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2411731 wikitext text/x-wiki i4dvrwx7kpzdixk0pd6ayqkelobdg74 2411732 2411731 2026-05-07T11:17:44Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2168.codfw.wmnet with OS trixie 2411732 wikitext text/x-wiki fhdisqclk7z72dvol8sjmruuy5g4ks4 2411733 2411732 2026-05-07T11:19:51Z Stashbot 7414 root@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2184.codfw.wmnet with reason: host reimage 2411733 wikitext text/x-wiki meh3gxw03mzxj5l9iu8z8ymbddjmlg5 2411734 2411733 2026-05-07T11:20:47Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2168: after reimage to trixie 2411734 wikitext text/x-wiki fhuyr54emtelg4yamu8kzn2gnjrh40q 2411736 2411734 2026-05-07T11:32:34Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host install7002.wikimedia.org 2411736 wikitext text/x-wiki 76kz24jit2o9b95bbw053nb3y30lobn 2411738 2411736 2026-05-07T11:36:00Z Stashbot 7414 daniel@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2411738 wikitext text/x-wiki e4b4lof4y6ov0rsgxyh36c0q8pfthwv 2411739 2411738 2026-05-07T11:36:20Z Stashbot 7414 daniel@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2411739 wikitext text/x-wiki tl2xlhou63704ji4syucmpwvvxjkd5n 2411744 2411739 2026-05-07T11:39:03Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host install7002.wikimedia.org 2411744 wikitext text/x-wiki t8m1aeq4y4w2wrvh1b4k54aj8x1q1ot 2411745 2411744 2026-05-07T11:40:46Z Stashbot 7414 root@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2184.codfw.wmnet with OS trixie 2411745 wikitext text/x-wiki qsx8eeas0qc0nmhqswmkx5zygbzwdbl 2411746 2411745 2026-05-07T11:43:16Z Stashbot 7414 daniel@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply 2411746 wikitext text/x-wiki m1o4kacxsu6rdwp80gvn6hj7vlnqld3 2411747 2411746 2026-05-07T11:43:47Z Stashbot 7414 daniel@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply 2411747 wikitext text/x-wiki 91tcioxnd6wjjyh4uchlkwz9atcivm8 2411748 2411747 2026-05-07T11:46:20Z Stashbot 7414 jynus@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1205.eqiad.wmnet with reason: reimage 2411748 wikitext text/x-wiki dq09q6pgpalrn6rdkymirrkjbdgkgoe 2411749 2411748 2026-05-07T11:47:55Z Stashbot 7414 root@cumin1003: START - Cookbook sre.hosts.reimage for host db1205.eqiad.wmnet with OS trixie 2411749 wikitext text/x-wiki jx2chnhltp4z3b920d2uy9uyokm56ir 2411750 2411749 2026-05-07T11:58:41Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1227: after reimage to trixie 2411750 wikitext text/x-wiki onrforpo4rw91zeltdngyma24u8r3pn 0 17290 2411250 2409807 2026-05-06T17:35:56Z Stashbot 7414 bd808: Unblock 109.134.0.0/16 2411250 wikitext text/x-wiki kejobhj6rq4v1p66joco0wsiia490dx 2411254 2411250 2026-05-06T17:41:22Z Stashbot 7414 bd808: Unblock 94.34.0.0/16 2411254 wikitext text/x-wiki 4ptvwzye85sjeq01lejh0ynuygrqqvk 2411262 2411254 2026-05-06T17:56:16Z Stashbot 7414 bd808: Unblock 84.226.0.0/16 2411262 wikitext text/x-wiki dw7fq9f60b1b8uwjqbho1lv6ztfstnx 2411267 2411262 2026-05-06T18:03:39Z Stashbot 7414 bd808: Unblock 94.208.0.0/14 2411267 wikitext text/x-wiki dnshhqay52owrpp0rn0uhzxsgh7zz77 2411269 2411267 2026-05-06T18:13:20Z Stashbot 7414 bd808: Unblock 88.165.192.0/19 2411269 wikitext text/x-wiki 3jj5g5ndeu3tuc48f40ltczhpn9r7ao 2 26362 2411336 2154399 2026-05-06T20:54:58Z Urhixidur 5948 <s> 2411336 wikitext text/x-wiki n6ccxu4htevm2k86mhggqfjhqy8idua 498 30942 2411103 2409419 2026-05-06T13:34:35Z Stashbot 7414 godog: change cloud-vps quota request phab at https://phabricator.wikimedia.org/project/manage/2880/ to mention https://cloudvps-quota.toolforge.org 2411103 wikitext text/x-wiki 4wp7hevkkdu9eyu4pzksscqfq3u9ipu 498 443628 2411279 2408870 2026-05-06T18:35:48Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25453754913 (https://github.com/cluebotng/component-configs/commits/92f164d1ab158aea1f76cd0a787f33ffe4017e85) 2411279 wikitext text/x-wiki b6tyy7wzxmvk5drlcigrdckl8n0c6ia 498 443949 2411313 2409781 2026-05-06T19:06:56Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 50d6c27 (Add SSE helper: QueueHandler-based event generator), 022bb07 (Add streaming update endpoint using Server-Sent Events), 1fe594f (Add phase field to SSE events for visual distinction), 66eca3c (Filter SSE events by thread to isolate concurrent updates) for T425477 2411313 wikitext text/x-wiki ed47oacyhnj5w80zy03iyoar8mfg2iu 2411318 2411313 2026-05-06T19:46:49Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy fa3a82e (Defer WdqsSparqlQueryEngine instantiation to avoid import-time side effects) for T421552 2411318 wikitext text/x-wiki j4hrzm8wedxsg993zf7tch1uezm69r3 2411740 2411318 2026-05-07T11:36:38Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy d2cdb4d (Pass explicit endpoint to SparqlQuery to avoid Site creation) 2411740 wikitext text/x-wiki maxwrd0tkexgqh6swggkco59r5rzlsc 2411741 2411740 2026-05-07T11:36:39Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy a8dd1ed (Remove unused self.repo from PagesProcessor) 2411741 wikitext text/x-wiki s9w7r0q0wrmu2ew45jnbpsocmyusalf 2411742 2411741 2026-05-07T11:36:40Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy c7f3a04 (Make pywikibot.Site creation lazy in PagesProcessor) 2411742 wikitext text/x-wiki s1mu2xh5v470k6h2rd4k4r4mde6y3nj 2411743 2411742 2026-05-07T11:36:40Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 35b1283 (Derive cache key from URL to avoid triggering pywikibot Site creation) 2411743 wikitext text/x-wiki ri6zn0a8q6c9m4h0h7aruap7c4k5u3t 0 446037 2411046 2402244 2026-05-06T12:13:20Z HNowlan (WMF) 15760 /* To become the Incident Coordinator (IC) */ Add incident severity 2411046 wikitext text/x-wiki nywmdx5ddswuzlaftrx8sskew1nzl2v 0 447188 2411332 2323046 2026-05-06T20:40:18Z CWhite (WMF) 12916 /* Deploying an updated schema */ 2411332 wikitext text/x-wiki ld3f1py1j3gz4ryobuo4qsrztrgjqm3 0 447260 2411613 2401354 2026-05-07T08:59:55Z LToscano (WMF) 7191 /* Renewing an existing intermediate */ 2411613 wikitext text/x-wiki qxkhtjtr8yahc7mxwcd9y1lwktwgplh 0 447862 2411037 2411006 2026-05-06T12:03:03Z FCeratto-WMF 39104 Add details 2411037 wikitext text/x-wiki jfylp750nsnwuaeikasouqgk0xmth87 0 447932 2411331 2410929 2026-05-06T20:38:26Z SFrench-WMF 37362 /* Deploying a new version */ Describe replay rebuild procedure 2411331 wikitext text/x-wiki rxo5e413gkttqbsk6j3eh00wpqvtn9h 0 449160 2411467 2410960 2026-05-07T00:02:14Z Dexbot 30554 Bot: Updating the report 2411467 wikitext text/x-wiki 7s4yv479srbdjfa33e0ieib0ffkz1qf 2411546 2411467 2026-05-07T04:47:51Z Dexbot 30554 Bot: Updating the report 2411546 wikitext text/x-wiki ly5kg0ry9xg87w4qf4q1b3qh7vnfh95 2411672 2411546 2026-05-07T10:36:29Z Dexbot 30554 Bot: Updating the report 2411672 wikitext text/x-wiki nd1nejf1qkllq3o94bpavjwa4xfs8oa 0 451448 2411158 2402645 2026-05-06T14:22:50Z Jdforrester (WMF) 468 Collapsed. 2411158 wikitext text/x-wiki fh3w8jry578ztamzg25a9iz7npbsn4l 2411160 2411158 2026-05-06T14:25:26Z Jdforrester (WMF) 468 /* Deploy a change to the production services */ 2411160 wikitext text/x-wiki gpc6dt9m2p266ve2th2mdavfxukd7hi 0 451533 2411111 2409512 2026-05-06T13:45:13Z Taavi-WMF 41365 /* Next meeting agenda */ rm former item 2411111 wikitext text/x-wiki qwxd9rkano7tdhqhfckyhw82p67hcwn 0 452184 2411057 2377435 2026-05-06T12:32:35Z MLechvien-WMF 49197 /* Available tools */ 2411057 wikitext text/x-wiki siddbzmzuosmkwx4ag5be4lh0l8xwlo 498 452890 2411282 2408662 2026-05-06T18:38:12Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25453754970 (https://github.com/cluebotng/component-configs/commits/92f164d1ab158aea1f76cd0a787f33ffe4017e85) 2411282 wikitext text/x-wiki bzb269szfarq9pqb136v7kz6df27bdl 116 453426 2411559 2407733 2026-05-07T05:44:07Z Phabbanbot 37210 Gabor_Kiss_WMSE was disabled by Sebastian_Berlin-WMSE 2411559 wikitext text/x-wiki fcqhzo5dpms7psdbt5xpeih1mf5hrvy 0 458675 2411234 2403231 2026-05-06T17:18:33Z CMooney (WMF) 24288 /* Interface transceiver states */ 2411234 wikitext text/x-wiki m52etuft7h14vcc3rw8s7xn05onytpd 0 459044 2411172 2410048 2026-05-06T14:50:45Z BKing (WMF) 29017 /* I/O spikes */ 2411172 wikitext text/x-wiki iac9t5gvf2y7kechfim57wwwkmeopal 2411190 2411172 2026-05-06T15:46:15Z BKing (WMF) 29017 /* Operations */ 2411190 wikitext text/x-wiki 3vfbszyegdp0vf8q6aiafd2o2xaucxo 2411322 2411190 2026-05-06T20:17:27Z BKing (WMF) 29017 /* Operations */ 2411322 wikitext text/x-wiki pxd0tbjbt8ukiuqhvx2t73ciukzq4ev 2411324 2411322 2026-05-06T20:19:44Z BKing (WMF) 29017 /* Rolling restarts */ 2411324 wikitext text/x-wiki qrr3278ro9psvjfm6ov1qs9rhv8mxt0 0 459076 2411246 2365548 2026-05-06T17:34:43Z CMooney (WMF) 24288 /* Notes */ 2411246 wikitext text/x-wiki e7zg2xovgxeueabanl9ch0twh430v09 2411255 2411246 2026-05-06T17:42:48Z CMooney (WMF) 24288 /* ONIE */ 2411255 wikitext text/x-wiki 2azad8zvxwxkkew4o2y3o1185pcryf4 2411256 2411255 2026-05-06T17:43:11Z CMooney (WMF) 24288 /* ONIE */ 2411256 wikitext text/x-wiki lw3ja2q43jlqctexkoj5kxau11u9fo0 2411271 2411256 2026-05-06T18:15:55Z CMooney (WMF) 24288 /* ONIE */ 2411271 wikitext text/x-wiki 7yycux4gtq3cxkouvm5vdinrm92cw8h 2411272 2411271 2026-05-06T18:17:09Z CMooney (WMF) 24288 /* First login */ 2411272 wikitext text/x-wiki nthxb5oob4dpu8aplq6hqwxo002d8jj 2411273 2411272 2026-05-06T18:23:55Z CMooney (WMF) 24288 /* TLS Certificate */ 2411273 wikitext text/x-wiki mwf0ppllja9seou45jo20e92x45ohem 2411274 2411273 2026-05-06T18:24:27Z CMooney (WMF) 24288 /* First login */ 2411274 wikitext text/x-wiki 593kirmc8pv9wouoy1ge02phwyj9qgh 2411278 2411274 2026-05-06T18:34:33Z CMooney (WMF) 24288 /* Software Version */ 2411278 wikitext text/x-wiki 5glwbs08t1pfn58mhf14ppudq8zgllr 0 459115 2411268 2407088 2026-05-06T18:04:55Z AndrewBogott (WMF) 51047 /* Magnum-api */ 2411268 wikitext text/x-wiki 35ychqq27thh0ca7cidfcf1l5hhnwdw 498 459243 2411205 2405906 2026-05-06T16:44:16Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25448365994 (https://github.com/cluebotng/component-configs/commits/7eaf56e56e3eacfd97455f271b152f369ca2b3f4) 2411205 wikitext text/x-wiki otssp8ugwdobwwll7kbhtp9w69g8i4n 2411207 2411205 2026-05-06T16:48:20Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25448624822 (https://github.com/cluebotng/component-configs/commits/92c52e117c73389fe04c66d53ad083eef60ef387) 2411207 wikitext text/x-wiki arv8uj49fninqrs6tb3tfvwzk67rvyb 2411257 2411207 2026-05-06T17:50:21Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25451592437 (https://github.com/cluebotng/component-configs/commits/88790d34de5a0bca24d8685f3f7df749b052647b) 2411257 wikitext text/x-wiki 6iz711kjuepneo9r2k8iwqqwujzvxpg 2411266 2411257 2026-05-06T18:03:06Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25452220343 (https://github.com/cluebotng/component-configs/commits/2ba80dd76efd2798ab7d1fedddf2a7baeab1118b) 2411266 wikitext text/x-wiki t7m4ebo7mlqhtmylg8ubxlubwzid8ba 2411270 2411266 2026-05-06T18:14:42Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25452753895 (https://github.com/cluebotng/component-configs/commits/6680b6abb8b5b775f79893981bd070c9e17f0358) 2411270 wikitext text/x-wiki cfshto4gfw4s5w6qw1ibjc5inr8ckmh 2411335 2411270 2026-05-06T20:54:01Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25460310332 (https://github.com/cluebotng/component-configs/commits/511feb967af13f77e59803c69bc3447124733524) 2411335 wikitext text/x-wiki gldjmfp4xc6bl7ynk2pev8v9pl10hs3 2411337 2411335 2026-05-06T20:57:15Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25460509016 (https://github.com/cluebotng/component-configs/commits/b9c2dde9382e1a3a5502f5218ecc7eaef6b4f9ec) 2411337 wikitext text/x-wiki 27kun9a2dt5s15b9tkbbjpm5r62qykb 2411356 2411337 2026-05-06T21:38:16Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25462408819 (https://github.com/cluebotng/component-configs/commits/c905103b2bd6129e76da6cf191567f82913c8349) 2411356 wikitext text/x-wiki o7ywpsbw0j71t9gwbmlw6ajx2lz5fvq 2411360 2411356 2026-05-06T21:50:21Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25462948750 (https://github.com/cluebotng/component-configs/commits/78047ed0bf8def9f4ff2b181d1710a274328a5b3) 2411360 wikitext text/x-wiki 0ijezqul8lqujgkzfs94xo63plb4raf 498 459244 2411201 2405886 2026-05-06T16:35:11Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25447977391 (https://github.com/cluebotng/component-configs/commits/bd39eba234686d41e4d889def1ed91fc28e8fdf6) 2411201 wikitext text/x-wiki 0ob0mub28vx9csr2fvegww0kpmh4jep 2411260 2411201 2026-05-06T17:56:02Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25451879961 (https://github.com/cluebotng/component-configs/commits/966f5e00a0a1098153ca8b6af11bc1124a119a1b) 2411260 wikitext text/x-wiki g6c48i35pc2nc1nj8o5x97zxmli6bid 2411333 2411260 2026-05-06T20:40:22Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25459711948 (https://github.com/cluebotng/component-configs/commits/3e0d938cc4bbeb5069a3e0b0d864df7eabf78c23) 2411333 wikitext text/x-wiki af03jq34ra3gcp6xlmahgbm34drbul4 498 459255 2411261 2404373 2026-05-06T17:56:13Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/25451879892 (https://github.com/cluebotng/component-configs/commits/966f5e00a0a1098153ca8b6af11bc1124a119a1b) 2411261 wikitext text/x-wiki tstvovvodvkf2t32gqi6t0gufe4ge31 0 459699 2411056 2411005 2026-05-06T12:29:33Z Stashbot 7414 eileen: config revision changed from 41cfd677 to 00752f91 2411056 wikitext text/x-wiki gvz6deiuo1auq0vdago7d01az11zr9z 12 460003 2411684 2398844 2026-05-07T10:44:38Z MMuhlenhoff (WMF) 5268 Update fingerprint following reimage to Trixie 2411684 wikitext text/x-wiki a3ssvcr79fbavmledaxunl58sqtn7pl 116 460070 2411386 2410897 2026-05-06T23:21:17Z EtherpadBackupBot 54504 things_to_restart_after_rabbit_explosion archived by BryanDavis 2411386 wikitext text/x-wiki icwf1v8ddg56tjo36qpxfjcmiv5w56t 2411387 2411386 2026-05-06T23:21:49Z EtherpadBackupBot 54504 title_value archived by BryanDavis 2411387 wikitext text/x-wiki rno12tqis1ac6sw3m5gu0pxnh0cq1pw 2411388 2411387 2026-05-06T23:22:20Z EtherpadBackupBot 54504 todays-regressions archived by BryanDavis 2411388 wikitext text/x-wiki nd7n5ldntnolx4b4jq2csrauqyvlvf8 2411389 2411388 2026-05-06T23:22:51Z EtherpadBackupBot 54504 tofu archived by BryanDavis 2411389 wikitext text/x-wiki dmuh2b3v10ajc4fav82imr60e97jni6 2411390 2411389 2026-05-06T23:23:22Z EtherpadBackupBot 54504 tonythomas01 archived by BryanDavis 2411390 wikitext text/x-wiki 0ccws9i3jv6klrva74i0f2xfx789pyr 2411391 2411390 2026-05-06T23:23:53Z EtherpadBackupBot 54504 tool_labs-doc archived by BryanDavis 2411391 wikitext text/x-wiki n8t03u34qf87qr4ya8mwlmjoru9ezky 2411392 2411391 2026-05-06T23:24:25Z EtherpadBackupBot 54504 toolforge-2021-12-14 archived by BryanDavis 2411392 wikitext text/x-wiki 8uii3ihfrxowfiqoincznmo78xk1u3l 2411393 2411392 2026-05-06T23:24:56Z EtherpadBackupBot 54504 toolforge-k8s-1.28-to-1.29-upgrade archived by BryanDavis 2411393 wikitext text/x-wiki cpb61k3eyo8nf4r52yt79p0v6n4talj 2411394 2411393 2026-05-06T23:25:27Z EtherpadBackupBot 54504 toolforge-k8s-1.31 archived by BryanDavis 2411394 wikitext text/x-wiki 23k0f62f2zhf0cq0gcm9nhq8flk4ztf 2411395 2411394 2026-05-06T23:25:58Z EtherpadBackupBot 54504 toolforge-k8s-upgrade-1.23 archived by BryanDavis 2411395 wikitext text/x-wiki tgmuazf0pgkmd9o88kuvs9tqi0s92vq 2411396 2411395 2026-05-06T23:26:29Z EtherpadBackupBot 54504 toolforge-php72-webservice archived by BryanDavis 2411396 wikitext text/x-wiki facwnyre3phetcygytbrnqp1i01dgil 2411397 2411396 2026-05-06T23:27:00Z EtherpadBackupBot 54504 toolforge-reboots archived by BryanDavis 2411397 wikitext text/x-wiki d40gw3nnyf1jydsm2lzh8vcwvi0lkhu 2411398 2411397 2026-05-06T23:27:36Z EtherpadBackupBot 54504 toollabs-cdnjs archived by BryanDavis 2411398 wikitext text/x-wiki lz9ovy5ewdy8qs2jhnjs57lgaxd2fy9 2411399 2411398 2026-05-06T23:28:08Z EtherpadBackupBot 54504 toollabs-ideal-getting-started archived by BryanDavis 2411399 wikitext text/x-wiki dh5ks1d00v8gpl5j5sb11xzbeslz6h1 2411400 2411399 2026-05-06T23:28:39Z EtherpadBackupBot 54504 toollabs-impact archived by BryanDavis 2411400 wikitext text/x-wiki tr8h0ffel2u5a0zd2felpld396s7920 2411401 2411400 2026-05-06T23:29:10Z EtherpadBackupBot 54504 toollabs-quarter-goal-q4-2014-15 archived by BryanDavis 2411401 wikitext text/x-wiki etvcv5ny1uwkr66b3p0hmmrib0j4jhi 2411402 2411401 2026-05-06T23:29:41Z EtherpadBackupBot 54504 toollabs-uwsgi archived by BryanDavis 2411402 wikitext text/x-wiki 6adf72612qklx00z6d58c41r5t88wxs 2411403 2411402 2026-05-06T23:30:12Z EtherpadBackupBot 54504 toollabs-webservice-trusty-switch archived by BryanDavis 2411403 wikitext text/x-wiki dxswu7k102ctt2jb5dr6yqpfb9viwii 2411404 2411403 2026-05-06T23:30:43Z EtherpadBackupBot 54504 toollabs20160127 archived by BryanDavis 2411404 wikitext text/x-wiki il1r01lwxojxs6k8d8u9lpbh9i07wa5 2411405 2411404 2026-05-06T23:31:14Z EtherpadBackupBot 54504 tools-k8s-chat-01 archived by BryanDavis 2411405 wikitext text/x-wiki 6n9lkvo50qn74ufxdbi8693xdoj2qfa 2411406 2411405 2026-05-06T23:31:46Z EtherpadBackupBot 54504 tools-kubernetes-scenarios archived by BryanDavis 2411406 wikitext text/x-wiki n3uoef5on568emswltju9hx85ag08fv 2411407 2411406 2026-05-06T23:32:17Z EtherpadBackupBot 54504 tools-migration archived by BryanDavis 2411407 wikitext text/x-wiki 82t46eojjxlrf2uigpqg9l6j0qa371y 2411408 2411407 2026-05-06T23:32:49Z EtherpadBackupBot 54504 tools-migration-nodes archived by BryanDavis 2411408 wikitext text/x-wiki 6339jnimuk2zqdff5sgp09i6zmpzuil 2411409 2411408 2026-05-06T23:33:20Z EtherpadBackupBot 54504 tools-puppet-prefixes archived by BryanDavis 2411409 wikitext text/x-wiki tlfut8fhfdp8yqrg1dyy7mgcxrvito1 2411410 2411409 2026-05-06T23:33:51Z EtherpadBackupBot 54504 tools-reboots archived by BryanDavis 2411410 wikitext text/x-wiki foixkneic70pprzya4bpr1d12am50zp 2411411 2411410 2026-05-06T23:34:22Z EtherpadBackupBot 54504 tools-reboots-cve-0728 archived by BryanDavis 2411411 wikitext text/x-wiki 5nrffvt2it2ca7avk4fipx6olrh3xfm 2411412 2411411 2026-05-06T23:34:54Z EtherpadBackupBot 54504 tools-the-great-recompress archived by BryanDavis 2411412 wikitext text/x-wiki oousvpixfkfsjh7gwhvb2463wblqz44 2411413 2411412 2026-05-06T23:35:25Z EtherpadBackupBot 54504 tools-trusty-move archived by BryanDavis 2411413 wikitext text/x-wiki gtr2xi54hlxtlix89tok1sb9rhmwn00 2411414 2411413 2026-05-06T23:35:56Z EtherpadBackupBot 54504 toolsdb-10.6 archived by BryanDavis 2411414 wikitext text/x-wiki aktnpxssqwu2dn77y61icmsfgg3l1mw 2411415 2411414 2026-05-06T23:36:27Z EtherpadBackupBot 54504 toolsdb-upgrade archived by BryanDavis 2411415 wikitext text/x-wiki hsnkx5mpdym9e52j8cth9me0lg2kv25 2411416 2411415 2026-05-06T23:36:58Z EtherpadBackupBot 54504 toolssupport archived by BryanDavis 2411416 wikitext text/x-wiki 7w59ecum4bw0cfnwv4cya9c7wk6c3kl 2411417 2411416 2026-05-06T23:37:29Z EtherpadBackupBot 54504 toomanythings archived by BryanDavis 2411417 wikitext text/x-wiki 8ze7lbcjodqbjx1b5nubsmf1ql29vpu 2411418 2411417 2026-05-06T23:38:01Z EtherpadBackupBot 54504 trebuchet-propose archived by BryanDavis 2411418 wikitext text/x-wiki truqv53w2w1ts0brl0gugvzcaxp9ya3 2411419 2411418 2026-05-06T23:38:32Z EtherpadBackupBot 54504 trend-prod archived by BryanDavis 2411419 wikitext text/x-wiki cqqgrsvfo8l1htn1qyy5wpk2dz85hr9 2411421 2411419 2026-05-06T23:39:03Z EtherpadBackupBot 54504 trixie-announcement archived by BryanDavis 2411421 wikitext text/x-wiki gv2z5planhv0xmmfqbgv8w0j9ws913d 2411422 2411421 2026-05-06T23:39:34Z EtherpadBackupBot 54504 trusty archived by BryanDavis 2411422 wikitext text/x-wiki mkfe5a3d7ekhy97d22lu16hyo3ya0d9 2411423 2411422 2026-05-06T23:40:05Z EtherpadBackupBot 54504 twiterbot-debuging archived by BryanDavis 2411423 wikitext text/x-wiki o25wqnf5kzm9qsk2emsxow638lczyf9 2411424 2411423 2026-05-06T23:40:36Z EtherpadBackupBot 54504 twocolconflictdeploy archived by BryanDavis 2411424 wikitext text/x-wiki 5q5ss737mi9a3vtogzp0sgdffnvs0pn 2411425 2411424 2026-05-06T23:41:08Z EtherpadBackupBot 54504 unable_to_move_translated_page archived by BryanDavis 2411425 wikitext text/x-wiki e2nduphc6j4bzdatx9ml9p8hxurptxk 2411427 2411425 2026-05-06T23:41:39Z EtherpadBackupBot 54504 unaliased archived by BryanDavis 2411427 wikitext text/x-wiki 11dlgjtbjj3wzq2mer68zg20i6hp363 2411428 2411427 2026-05-06T23:42:10Z EtherpadBackupBot 54504 unified archived by BryanDavis 2411428 wikitext text/x-wiki r78lkuogr21zwnj2vgxvzt3hjf1nghl 2411429 2411428 2026-05-06T23:42:41Z EtherpadBackupBot 54504 unittestingbrownbag archived by BryanDavis 2411429 wikitext text/x-wiki m7mbslerzl82ebe7v2wre6kqjxruto1 2411430 2411429 2026-05-06T23:43:12Z EtherpadBackupBot 54504 unsophabricated archived by BryanDavis 2411430 wikitext text/x-wiki 55qd5k9hxfwzjvx6xg9vywxhlficncf 2411431 2411430 2026-05-06T23:43:43Z EtherpadBackupBot 54504 upload-tool-class-structure archived by BryanDavis 2411431 wikitext text/x-wiki 8l0tw7fthdvyct4wmcjyf3juzar6xb6 2411432 2411431 2026-05-06T23:44:14Z EtherpadBackupBot 54504 uploadcampaigns-squids archived by BryanDavis 2411432 wikitext text/x-wiki 35igkhutr8ktby7adv2abhzdx0pvv8t 2411433 2411432 2026-05-06T23:44:46Z EtherpadBackupBot 54504 uploadwizard-browser-tests archived by BryanDavis 2411433 wikitext text/x-wiki cks139zdzwpo6cxdjduchu10u3r6rte 2411434 2411433 2026-05-06T23:45:17Z EtherpadBackupBot 54504 uploadwizard-refactor-planning archived by BryanDavis 2411434 wikitext text/x-wiki 5usdfhe1hi31y8kxhcjnyduj1ma8n5j 2411435 2411434 2026-05-06T23:45:48Z EtherpadBackupBot 54504 uw-bugday-2014-09-09 archived by BryanDavis 2411435 wikitext text/x-wiki 9d6fwxmi90096ykmdug67d991p8s9rq 2411436 2411435 2026-05-06T23:46:20Z EtherpadBackupBot 54504 uw-cucumber-sketch archived by BryanDavis 2411436 wikitext text/x-wiki 49ki1rvzpouc75ybqjay9pfmojujtua 2411437 2411436 2026-05-06T23:46:51Z EtherpadBackupBot 54504 vandrew archived by BryanDavis 2411437 wikitext text/x-wiki fkinsjk1wv2ree2l8o3cpw7zuome6hx 2411438 2411437 2026-05-06T23:47:22Z EtherpadBackupBot 54504 varnishkafka_troubleshooting archived by BryanDavis 2411438 wikitext text/x-wiki hh7xyhtfzqbm7ghhxcij39hsyjmvfxj 2411439 2411438 2026-05-06T23:47:53Z EtherpadBackupBot 54504 ve archived by BryanDavis 2411439 wikitext text/x-wiki dftl8dicqpzvr9vxkzotznvhe4pu9z2 2411440 2411439 2026-05-06T23:48:25Z EtherpadBackupBot 54504 ve-2016-05-05 archived by BryanDavis 2411440 wikitext text/x-wiki p8e2ta0j6q9w7g4ose8gwotkuf08evz 2411441 2411440 2026-05-06T23:48:56Z EtherpadBackupBot 54504 ve-future-thoughts-2015 archived by BryanDavis 2411441 wikitext text/x-wiki supeic3z96qchzeyc3yp6qt53p0hsw1 2411442 2411441 2026-05-06T23:49:27Z EtherpadBackupBot 54504 vikas archived by BryanDavis 2411442 wikitext text/x-wiki avsd0npl2v8121jf9qdc9go0ffxk053 2411443 2411442 2026-05-06T23:49:59Z EtherpadBackupBot 54504 vikas-gerrit archived by BryanDavis 2411443 wikitext text/x-wiki nng1i17xombtqzsbxh0iir87tg2lrac 2411444 2411443 2026-05-06T23:50:30Z EtherpadBackupBot 54504 vikas-ruby-style archived by BryanDavis 2411444 wikitext text/x-wiki 37j7dsgbk9yul40tkon4w5h807aaces 2411445 2411444 2026-05-06T23:51:01Z EtherpadBackupBot 54504 vikas-upload-error archived by BryanDavis 2411445 wikitext text/x-wiki gzts4szadl7bruqmq40swvesq2aorf5 2411446 2411445 2026-05-06T23:51:32Z EtherpadBackupBot 54504 vikas_vagrant archived by BryanDavis 2411446 wikitext text/x-wiki f1f2kzox32778vws5c2bmdz86w8wwv1 2411447 2411446 2026-05-06T23:52:03Z EtherpadBackupBot 54504 vikas_ve archived by BryanDavis 2411447 wikitext text/x-wiki ootmbf6gmbhnjvrcx4utz45u4iy4hqw 2411448 2411447 2026-05-06T23:52:34Z EtherpadBackupBot 54504 vikassy archived by BryanDavis 2411448 wikitext text/x-wiki hli0454ubaw1gxjskqvel25wxx3v7mg 2411449 2411448 2026-05-06T23:53:05Z EtherpadBackupBot 54504 virt1006migrate archived by BryanDavis 2411449 wikitext text/x-wiki lhyt3hstdvxqs82xoeaxrbdhtcrtyoc 2411450 2411449 2026-05-06T23:53:36Z EtherpadBackupBot 54504 vk-jumbo-cleanup archived by BryanDavis 2411450 wikitext text/x-wiki q15bk4v7sy5crvelughl2w590uznhog 2411451 2411450 2026-05-06T23:54:07Z EtherpadBackupBot 54504 volans-tmp archived by BryanDavis 2411451 wikitext text/x-wiki 4ned8wf8axje8poy0m4la4szf7ch3la 2411452 2411451 2026-05-06T23:54:39Z EtherpadBackupBot 54504 wallaby-dev-magnum-install archived by BryanDavis 2411452 wikitext text/x-wiki hijseuwh99wcjzg8jbstdf7q09zo57r 2411453 2411452 2026-05-06T23:55:10Z EtherpadBackupBot 54504 watchlist_id archived by BryanDavis 2411453 wikitext text/x-wiki 1x01k9lgqzhmdy2k9kl005qdadmab05 2411454 2411453 2026-05-06T23:55:41Z EtherpadBackupBot 54504 wbq_test archived by BryanDavis 2411454 wikitext text/x-wiki srbwnchf5q2dripjfs63ked3trmfdpo 2411455 2411454 2026-05-06T23:56:12Z EtherpadBackupBot 54504 wcna_tools archived by BryanDavis 2411455 wikitext text/x-wiki 0ooom4kvyccbutwqmglhpyk7tpltj2v 2411456 2411455 2026-05-06T23:56:44Z EtherpadBackupBot 54504 wdio archived by BryanDavis 2411456 wikitext text/x-wiki qu5x9g5r6gck7xh5fw8bhl38fdwcx4t 2411457 2411456 2026-05-06T23:57:15Z EtherpadBackupBot 54504 wdqs-2024-08-08 archived by BryanDavis 2411457 wikitext text/x-wiki 2hrt1x6pzoz0390503ip5cjnx9bzs1j 2411458 2411457 2026-05-06T23:57:46Z EtherpadBackupBot 54504 wdqs-split-graph-rollout-plan archived by BryanDavis 2411458 wikitext text/x-wiki 7w8z3m3amiemmh0z5ppsvr642aurmwu 2411459 2411458 2026-05-06T23:58:18Z EtherpadBackupBot 54504 wdqs_graph_split_production_deploy archived by BryanDavis 2411459 wikitext text/x-wiki 5fypmevc5iv57liwpp0r7mgcxrcjl3y 2411460 2411459 2026-05-06T23:58:49Z EtherpadBackupBot 54504 we-really-love-nfs archived by BryanDavis 2411460 wikitext text/x-wiki 42zpjvlnvdzo2aw96zcgbjzv6zm1nlg 2411461 2411460 2026-05-06T23:59:20Z EtherpadBackupBot 54504 webreq_update archived by BryanDavis 2411461 wikitext text/x-wiki pt5hhac7um2oyr41nzx8l7lvahx8lr1 2411462 2411461 2026-05-06T23:59:54Z EtherpadBackupBot 54504 webrequest-2021-Q4-loss archived by BryanDavis 2411462 wikitext text/x-wiki mbdmho8de07bzkdt7jobzhpmdv9i7wi 2411463 2411462 2026-05-07T00:00:25Z EtherpadBackupBot 54504 webservice-k8s-announce archived by BryanDavis 2411463 wikitext text/x-wiki duud1t1ms6at5rio6ts1f61h36ca5p8 2411464 2411463 2026-05-07T00:00:57Z EtherpadBackupBot 54504 what_is_left_in_tampa archived by BryanDavis 2411464 wikitext text/x-wiki 6gidxn3kgyzvw86pfa2s74l930yhccf 2411465 2411464 2026-05-07T00:01:28Z EtherpadBackupBot 54504 whats-left-in-toolforge-eqiad archived by BryanDavis 2411465 wikitext text/x-wiki 5tq5ryb8dvannjg1rg87p6q1xuxscch 2411466 2411465 2026-05-07T00:01:59Z EtherpadBackupBot 54504 whm2023-Overview_of_technical_areas_&_projects archived by BryanDavis 2411466 wikitext text/x-wiki 4n3gxchdsrvn3xd4m6bn13z13bw17wc 2411468 2411466 2026-05-07T00:02:31Z EtherpadBackupBot 54504 widgets-for-uploadwizard archived by BryanDavis 2411468 wikitext text/x-wiki bvhzrt448flqq2mtogekj2uafz9py5j 2411469 2411468 2026-05-07T00:03:02Z EtherpadBackupBot 54504 wiki_technology_notes archived by BryanDavis 2411469 wikitext text/x-wiki dic1ct8rcbukz7fdmwvvvw59kgxrljp 2411470 2411469 2026-05-07T00:03:33Z EtherpadBackupBot 54504 wikibits-undefault archived by BryanDavis 2411470 wikitext text/x-wiki lpsml8w1cqux0xtqu5pbzuwaw993p6t 2411471 2411470 2026-05-07T00:04:04Z EtherpadBackupBot 54504 wikidata_query_service_checkin archived by BryanDavis 2411471 wikitext text/x-wiki ctdwoc99o7696l2g7r17k8h3upa8o94 2411472 2411471 2026-05-07T00:04:35Z EtherpadBackupBot 54504 wikidev17-devwishlist archived by BryanDavis 2411472 wikitext text/x-wiki gh0uklt0t1is3v4e83k6jyfvvx89cx9 2411473 2411472 2026-05-07T00:05:07Z EtherpadBackupBot 54504 wikigrokcollections archived by BryanDavis 2411473 wikitext text/x-wiki eeyk3b07majvv06o91ie97d3jgqda0h 2411474 2411473 2026-05-07T00:05:38Z EtherpadBackupBot 54504 wikigroknextsteps archived by BryanDavis 2411474 wikitext text/x-wiki r0du3vt9ve5t45c9ptahxyqrztzofwn 2411475 2411474 2026-05-07T00:06:09Z EtherpadBackupBot 54504 wikimania_knowledge archived by BryanDavis 2411475 wikitext text/x-wiki qi9fgxtwkhqn1nfp0jkgs48nbjr1usn 2411476 2411475 2026-05-07T00:06:40Z EtherpadBackupBot 54504 wikipedia-app-ios-issues archived by BryanDavis 2411476 wikitext text/x-wiki 81th35hzczhh3va5q5v80go9xl3i92j 2411477 2411476 2026-05-07T00:07:11Z EtherpadBackupBot 54504 wikipedia-app-saved-pages-architecture archived by BryanDavis 2411477 wikitext text/x-wiki cfbpalx9nsc3p703q7cnbu5vii9lq43 2411478 2411477 2026-05-07T00:07:43Z EtherpadBackupBot 54504 wikipedia-ios-4.0.5-release-notes archived by BryanDavis 2411478 wikitext text/x-wiki m2bykk6s0tivbevjs7ijv84scrc8bww 2411479 2411478 2026-05-07T00:08:14Z EtherpadBackupBot 54504 wikipedia-mobile-1-2-beta1-notes archived by BryanDavis 2411479 wikitext text/x-wiki h1l47l06deytoh864hpcyloglkp0zcw 2411480 2411479 2026-05-07T00:08:45Z EtherpadBackupBot 54504 wikireplica-beta archived by BryanDavis 2411480 wikitext text/x-wiki ripag90o8slg37b03nsi6hwd9ah6fky 2411481 2411480 2026-05-07T00:09:16Z EtherpadBackupBot 54504 wikireplica-dns-switch archived by BryanDavis 2411481 wikitext text/x-wiki ri34ewl8amewuo69bl1pfcieob4xmck 2411482 2411481 2026-05-07T00:09:48Z EtherpadBackupBot 54504 wikitech-extensions archived by BryanDavis 2411482 wikitext text/x-wiki q796mnexrh6cuplxoe49syan90akbyf 2411484 2411482 2026-05-07T00:10:19Z EtherpadBackupBot 54504 wikitechrebuild archived by BryanDavis 2411484 wikitext text/x-wiki cpqm8r5dtwgjcq2cg9uzwk5rovies6f 2411485 2411484 2026-05-07T00:10:50Z EtherpadBackupBot 54504 wishathon-march-2024-fotw-session archived by BryanDavis 2411485 wikitext text/x-wiki 72b6bb2tv5w4dzpge4eeygfpae64p4h 2411486 2411485 2026-05-07T00:11:22Z EtherpadBackupBot 54504 wishathon-march-2024-office-hour archived by BryanDavis 2411486 wikitext text/x-wiki bxo8n87ud5a7ylqku9rxcri1s88drpj 2411487 2411486 2026-05-07T00:11:53Z EtherpadBackupBot 54504 wishathon-march-2024-showcase archived by BryanDavis 2411487 wikitext text/x-wiki qhoswhxqbn97xe814p19p9rrv3abp60 2411488 2411487 2026-05-07T00:12:24Z EtherpadBackupBot 54504 wishathon-march-2024-wishlist-overview-session archived by BryanDavis 2411488 wikitext text/x-wiki 5trbztzey5m3dtdrmizep5w2rt0jbh5 2411489 2411488 2026-05-07T00:12:55Z EtherpadBackupBot 54504 wlm-api-admin-tree archived by BryanDavis 2411489 wikitext text/x-wiki 6ac2h0dc3f7dgeg4mfa6z9dcvnn50oj 2411490 2411489 2026-05-07T00:13:27Z EtherpadBackupBot 54504 wlm-on-labs-steps archived by BryanDavis 2411490 wikitext text/x-wiki d4yixderjr2p2e2tel2lty25kxui5gr 2411491 2411490 2026-05-07T00:13:58Z EtherpadBackupBot 54504 wlm-project-retro archived by BryanDavis 2411491 wikitext text/x-wiki b5op0m4dyqti0d8nw4pyqhhkzz99rta 2411492 2411491 2026-05-07T00:14:29Z EtherpadBackupBot 54504 wlm-sprint1-retro archived by BryanDavis 2411492 wikitext text/x-wiki 40yx4oju7dmc63kj41068thn3km5rap 2411493 2411492 2026-05-07T00:15:00Z EtherpadBackupBot 54504 wlm-sprint2-retro archived by BryanDavis 2411493 wikitext text/x-wiki m2t6phpb2sohm20rm3km3g1wh9ccgeg 2411494 2411493 2026-05-07T00:15:31Z EtherpadBackupBot 54504 wlm-sprint3-retro archived by BryanDavis 2411494 wikitext text/x-wiki ohfzrwtm17nev67jegsswjuidg6x11z 2411495 2411494 2026-05-07T00:16:03Z EtherpadBackupBot 54504 wlm-sprint4-retro archived by BryanDavis 2411495 wikitext text/x-wiki qgqviwl2isnhwnhwpecsd23i6qoulhb 2411496 2411495 2026-05-07T00:16:34Z EtherpadBackupBot 54504 wlm-sprint5-retro archived by BryanDavis 2411496 wikitext text/x-wiki juzkkh1qxw198o3ohec1ka6iykwfpvv 2411497 2411496 2026-05-07T00:17:05Z EtherpadBackupBot 54504 wlmapp-iteration1-story-estimation-and-review archived by BryanDavis 2411497 wikitext text/x-wiki 5r03jbbjexctwglnq58jt8hbeeaotc8 2411498 2411497 2026-05-07T00:17:37Z EtherpadBackupBot 54504 wmconf2013-evaluating-programs archived by BryanDavis 2411498 wikitext text/x-wiki rtu3e0nsm185i6xzprdzffn7g7pbzr2 2411499 2411498 2026-05-07T00:18:08Z EtherpadBackupBot 54504 wmconf2013-evaluating_programs archived by BryanDavis 2411499 wikitext text/x-wiki rkvow7d6lkmwarau94xsaxtmqipf3bu 2411500 2411499 2026-05-07T00:18:39Z EtherpadBackupBot 54504 wmcs-2023-06-07 archived by BryanDavis 2411500 wikitext text/x-wiki ai8p0ude6djmotz7mwic868pwe5fsh2 2411501 2411500 2026-05-07T00:19:11Z EtherpadBackupBot 54504 wmcs-contributors archived by BryanDavis 2411501 wikitext text/x-wiki n3byezn8eqjbmfhabi2n248sre9azkg 2411502 2411501 2026-05-07T00:19:42Z EtherpadBackupBot 54504 wmcs-cumin-fail archived by BryanDavis 2411502 wikitext text/x-wiki k4fdoq5fty2ihl1wtbedrfbq3cxafwo 2411503 2411502 2026-05-07T00:20:13Z EtherpadBackupBot 54504 wmcs-fallout archived by BryanDavis 2411503 wikitext text/x-wiki obycwq5o44envj6ox8o520bi9shc7qx 2411504 2411503 2026-05-07T00:20:44Z EtherpadBackupBot 54504 wmcs-goals-q2-17-18 archived by BryanDavis 2411504 wikitext text/x-wiki kujng7u833ieuud7uq3crl6t2yvc83k 2411505 2411504 2026-05-07T00:21:16Z EtherpadBackupBot 54504 wmcs-goals-q4-16-17 archived by BryanDavis 2411505 wikitext text/x-wiki 2n3w49scrmpmvfry7uzdsjwacih4ha4 2411506 2411505 2026-05-07T00:21:47Z EtherpadBackupBot 54504 wmcs-vs-rowb-upgrade archived by BryanDavis 2411506 wikitext text/x-wiki sis6g0dddw5khdxmg4dpfwpyak7dac4 2411507 2411506 2026-05-07T00:22:18Z EtherpadBackupBot 54504 wmcs_nfs_future archived by BryanDavis 2411507 wikitext text/x-wiki 02tgrq1ysn5j0nvg4b20ku6x9kvj2ug 2411508 2411507 2026-05-07T00:22:49Z EtherpadBackupBot 54504 wmcslightning archived by BryanDavis 2411508 wikitext text/x-wiki mwdxn9edmdeylo8rzuucvdjs2pjebgd 2411509 2411508 2026-05-07T00:23:20Z EtherpadBackupBot 54504 wmf11and12 archived by BryanDavis 2411509 wikitext text/x-wiki 45bh067ce8wf2l8su6upbomgb82au2z 2411510 2411509 2026-05-07T00:23:51Z EtherpadBackupBot 54504 wmh2023-Best_practices_&_ideas_for_implementing archived by BryanDavis 2411510 wikitext text/x-wiki 5gbkv5remuym6jhm8vtg1tnzqmnjupz 2411511 2411510 2026-05-07T00:24:22Z EtherpadBackupBot 54504 wmh2023-LLMs,_ChatGPT,_machine_learning_tools,_etc archived by BryanDavis 2411511 wikitext text/x-wiki 5jqb41aw7xp6gnc7zqs2ov550aigqwt 2411512 2411511 2026-05-07T00:24:53Z EtherpadBackupBot 54504 wmh2023-Overview_of_technical_areas_%26_projects archived by BryanDavis 2411512 wikitext text/x-wiki 8zxlxnzmkc9o4x2yzhr3wpchkrf0j80 2411513 2411512 2026-05-07T00:25:25Z EtherpadBackupBot 54504 wmh2023-Past,_Present_and_Future_of_Wikimedia archived by BryanDavis 2411513 wikitext text/x-wiki cgvumbmp0qaxy7mxx6ykgwwm3tu0qa2 2411514 2411513 2026-05-07T00:25:56Z EtherpadBackupBot 54504 wmh2023-Toolhub,_Toolhunt,_and_the_quest archived by BryanDavis 2411514 wikitext text/x-wiki f10c10jpk1yg7s5jsu3svahke16ehxu 2411515 2411514 2026-05-07T00:26:28Z EtherpadBackupBot 54504 wmh2024-leaving-coordination archived by BryanDavis 2411515 wikitext text/x-wiki tiw875rywm6g6wzjb006wknogzf9qe5 2411516 2411515 2026-05-07T00:26:59Z EtherpadBackupBot 54504 wmhack15-wikidata-architecture-session archived by BryanDavis 2411516 wikitext text/x-wiki 6fv3h82wy4h508jrw64ipjn8ldzil8j 2411517 2411516 2026-05-07T00:27:31Z EtherpadBackupBot 54504 wmhack2017-ciw archived by BryanDavis 2411517 wikitext text/x-wiki irha2coejilrp48n36b3d5bbanbmmg1 2411518 2411517 2026-05-07T00:28:02Z EtherpadBackupBot 54504 wmhack2021-phab archived by BryanDavis 2411518 wikitext text/x-wiki qbzn1dldqambkv390xesew7vunsxk2b 2411519 2411518 2026-05-07T00:28:33Z EtherpadBackupBot 54504 wp-app-1-1-release-post archived by BryanDavis 2411519 wikitext text/x-wiki 8idmls13j85bsy4sx6rni9fbiu71doo 2411520 2411519 2026-05-07T00:29:05Z EtherpadBackupBot 54504 wtf archived by BryanDavis 2411520 wikitext text/x-wiki hwfq8vavphlb053muhyojdv6kmoho0i 2411522 2411520 2026-05-07T00:29:36Z EtherpadBackupBot 54504 wtf-is-production archived by BryanDavis 2411522 wikitext text/x-wiki 8zbjv1fsbregew9ke9d0q729mhusmio 2411523 2411522 2026-05-07T00:30:07Z EtherpadBackupBot 54504 x-analytics archived by BryanDavis 2411523 wikitext text/x-wiki 8mr5i0qg8ga76463m8bifjxy8gn5vur 2411524 2411523 2026-05-07T00:30:38Z EtherpadBackupBot 54504 xZLrPEASWV archived by BryanDavis 2411524 wikitext text/x-wiki dnmjtw3tj10svt46g1xcx9abk6oov35 2411525 2411524 2026-05-07T00:31:09Z EtherpadBackupBot 54504 y4gqz6<fe46Q archived by BryanDavis 2411525 wikitext text/x-wiki 0bw6kw1vs82lasiwkx399naomu1tcf6 2411527 2411525 2026-05-07T00:31:40Z EtherpadBackupBot 54504 y4gqz6<fe46Q archived by BryanDavis 2411527 wikitext text/x-wiki 7bhnp7an787s5cydwhqcexw1j1qc1b1 2411528 2411527 2026-05-07T00:32:11Z EtherpadBackupBot 54504 yAqOuwkDKuhwhwjvd44j archived by BryanDavis 2411528 wikitext text/x-wiki s615sjuo4tj649yamp4m1v97gdfd7rg 2411529 2411528 2026-05-07T00:32:43Z EtherpadBackupBot 54504 yeZfmYkMu2 archived by BryanDavis 2411529 wikitext text/x-wiki 1k3ogof008njs3uwh79ikvdsfha8zin 2411530 2411529 2026-05-07T00:33:14Z EtherpadBackupBot 54504 yuvi-knowledge-transfer archived by BryanDavis 2411530 wikitext text/x-wiki f1ui6qxxzthe7a7bitxi3vrjb59szqc 2411531 2411530 2026-05-07T00:33:45Z EtherpadBackupBot 54504 yuvidump archived by BryanDavis 2411531 wikitext text/x-wiki swkpy9s5s24dqs9xmxdgw6k24d5vqlo 2411532 2411531 2026-05-07T00:34:16Z EtherpadBackupBot 54504 zero-mf-sync-up archived by BryanDavis 2411532 wikitext text/x-wiki c4ntq9hlsewj3vez6kuu6p0gwu7jphi 2411533 2411532 2026-05-07T00:34:48Z EtherpadBackupBot 54504 zerocrash archived by BryanDavis 2411533 wikitext text/x-wiki iiljraf1taajdzih86x67qzl1m5daf6 2411534 2411533 2026-05-07T00:35:19Z EtherpadBackupBot 54504 zh-ve-screenshots archived by BryanDavis 2411534 wikitext text/x-wiki nu0cmlrvg4mqdrnx6unbvzn1gb5e484 2 460118 2411117 2408255 2026-05-06T13:54:43Z BKing (WMF) 29017 BKing (WMF) moved page [[Incidents/2026-04-30 cloudelastic unavailable]] to [[User:Incidents/2026-04-30 cloudelastic unavailable]]: does not need to be treated as in incident 2408255 wikitext text/x-wiki hwo0uihiiu1mz69gx2sa4k6yuqefvt9 2411124 2411117 2026-05-06T13:56:26Z BKing (WMF) 29017 /* Summary */ 2411124 wikitext text/x-wiki h40jylxl69sbe2b7d8piw7h74n35q63 498 460126 2411223 2408770 2026-05-06T17:12:35Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: A pull request (https://github.com/wiki-connect/wikimonitor/pull/56) has been submitted by Gerges on GitHub (T425534) 2411223 wikitext text/x-wiki 6vpnist584cef4dcv0hpa7662idc6q0 2411237 2411223 2026-05-06T17:23:12Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: A pull request (https://github.com/wiki-connect/wikimonitor/pull/56) has been submitted by Gerges has been merged (T425534) 2411237 wikitext text/x-wiki hc4i8ex3xtaiajwh8vcr0dqsj1tintn 2411288 2411237 2026-05-06T18:42:28Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: A pull request (https://github.com/wiki-connect/wikimonitor/pull/56) has been submitted by �[200~praffq-dev has been merged (T422194) 2411288 wikitext text/x-wiki h2qk8egdxyel5c18c2ifqjql84nppnx 2411289 2411288 2026-05-06T18:42:50Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: A pull request (https://github.com/wiki-connect/wikimonitor/pull/53) has been submitted by praffq-dev has been merged (T422194) 2411289 wikitext text/x-wiki pd0cr9jqg51bxqo6z6yf1zhb2zjwifa 0 460142 2411044 2026-05-06T12:11:46Z HNowlan (WMF) 15760 Created page with "In order to understand the impact of an incident, SRE classifies incidents with a severity level. There are three severity levels in use by SRE, all prefixed by "Sev" with 1 being the most severe and 3 the least severe. These levels indicate differing levels of impact and different levels of warranted response. When an incident document is opened, the IC should attempt to assign a severity level soon after filling in other basic details. The severity level of an inciden..." 2411044 wikitext text/x-wiki e4ufyndldkvcu34zolxyu3e7lrzzpar 2411060 2411044 2026-05-06T12:36:50Z Taavi-WMF 41365 add navigation 2411060 wikitext text/x-wiki bvfu1uzzmms7h44e1rw1ccfgegnlqbx 2411061 2411060 2026-05-06T12:36:59Z Taavi-WMF 41365 Taavi-WMF moved page [[Incident response/Incident Severities]] to [[Incident response/Incident severities]]: title case 2411060 wikitext text/x-wiki bvfu1uzzmms7h44e1rw1ccfgegnlqbx 498 460143 2411051 2026-05-06T12:20:18Z Labslogbot 55 Auto update of instance info. 2411051 wikitext text/x-wiki 53e58gkhi7lya8g29ijxmkt6hwn3a4y 0 460144 2411062 2026-05-06T12:36:59Z Taavi-WMF 41365 Taavi-WMF moved page [[Incident response/Incident Severities]] to [[Incident response/Incident severities]]: title case 2411062 wikitext text/x-wiki 03stq2w46grvjy7qceiga9noykb7rb6 0 460147 2411118 2026-05-06T13:54:43Z BKing (WMF) 29017 BKing (WMF) moved page [[Incidents/2026-04-30 cloudelastic unavailable]] to [[User:Incidents/2026-04-30 cloudelastic unavailable]]: does not need to be treated as in incident 2411118 wikitext text/x-wiki pihhiy9s42z8wbfe5ijtv8uzylldd5j 12 460148 2411183 2026-05-06T15:06:03Z MMuhlenhoff (WMF) 5268 Add fingerprint for bast5005 2411183 wikitext text/x-wiki t33q6yxwy9abbow7bebhx940x4c8dp6 3 460149 2411334 2026-05-06T20:47:30Z A smart kitten 38172 /* Random thoughts/ideas/comments (on skim-reading this page, having found it from phab:T385007#11896154) */ new section 2411334 wikitext text/x-wiki qdfo7k85zql5kmnxywp12izv42hq9vv 2411379 2411334 2026-05-06T22:41:58Z RLazarus (WMF) 15215 /* Random thoughts/ideas/comments (on skim-reading this page, having found it from phab:T385007#11896154) */ Reply 2411379 wikitext text/x-wiki o497nusxq6sjccgyw8zi4t39b0lqwbh