Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.5 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 3086 2424716 2423702 2026-06-09T11:01:21Z Stashbot 7414 taavi: starting Kyverno upgrade T427928 2424716 wikitext text/x-wiki me7qv4pt2xzegpdns8j99me7ed5lr1n 2424717 2424716 2026-06-09T11:01:38Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component kyverno 2424717 wikitext text/x-wiki lbvva9mfezkhb2h0pg32lrjd3vppbmz 2424722 2424717 2026-06-09T11:23:21Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component kyverno 2424722 wikitext text/x-wiki a60a4ps2s75wl8z7l8y93ghximr0qh8 0 3788 2424485 2379343 2026-06-08T18:31:04Z BCornwall-WMF 31521 /* HOWTO */ Remove recently-added confd reload section as this bug has been fixed 2424485 wikitext text/x-wiki s22n94vhp1h8b13pgk3jh9tpemxbc6g 0 4108 2424262 2424207 2026-06-08T12:37:49Z ScheduleDeploymentBot 37566 Add [[gerrit:1298758]] to Monday, June 08 UTC afternoon backport window 2424262 wikitext text/x-wiki 9fvofooe72beojpac7ok6mck192zm3r 2424263 2424262 2026-06-08T12:38:07Z ScheduleDeploymentBot 37566 Add [[gerrit:1298762]] to Monday, June 08 UTC afternoon backport window 2424263 wikitext text/x-wiki 5tfot5pr1c3mwq2ytont4iz57ubzpl2 2424264 2424263 2026-06-08T12:38:27Z ScheduleDeploymentBot 37566 Add [[gerrit:1298764]] to Monday, June 08 UTC afternoon backport window 2424264 wikitext text/x-wiki g8u0ii6sxksiwyn0l7mq26372rgfaiq 2424265 2424264 2026-06-08T12:38:56Z ScheduleDeploymentBot 37566 Add [[gerrit:1298765]] to Monday, June 08 UTC afternoon backport window 2424265 wikitext text/x-wiki khvrecrgmpo0s557mciucwhxmszhq0o 2424266 2424265 2026-06-08T12:39:02Z ScheduleDeploymentBot 37566 Add [[gerrit:1298766]] to Monday, June 08 UTC afternoon backport window 2424266 wikitext text/x-wiki 09bgthulhwwf7pcvlwpajcto9npb8si 2424299 2424266 2026-06-08T13:02:34Z ScheduleDeploymentBot 37566 Add [[gerrit:1296550]] to Monday, June 08 UTC afternoon backport window 2424299 wikitext text/x-wiki 9fpv36tzyqe5d6v341biwqp5my6xqay 2424426 2424299 2026-06-08T16:14:02Z ScheduleDeploymentBot 37566 Add [[gerrit:1298834]] to Tuesday, June 09 UTC afternoon backport window 2424426 wikitext text/x-wiki g4dm03msgr63lc779ocvfzvfjhlkcog 2424428 2424426 2026-06-08T16:14:23Z ScheduleDeploymentBot 37566 Add [[gerrit:1297162]] to Monday, June 08 UTC late backport window 2424428 wikitext text/x-wiki r62z0e2kxi65ilgi2ty9a28zjkk3d5y 2424455 2424428 2026-06-08T16:49:40Z ScheduleDeploymentBot 37566 Add [[gerrit:1298841]] to Monday, June 08 UTC late backport window 2424455 wikitext text/x-wiki qs2qlkvu2ldu5nyhfllzci001339tnq 2424503 2424455 2026-06-08T18:58:31Z ScheduleDeploymentBot 37566 Add [[gerrit:1298879]] to Monday, June 08 UTC late backport window 2424503 wikitext text/x-wiki 49c0ubp6x9eno74vlogw67v91xqxd1d 2424523 2424503 2026-06-08T19:55:11Z ScheduleDeploymentBot 37566 Add [[gerrit:1298890]] to Thursday, June 11 UTC afternoon backport window 2424523 wikitext text/x-wiki cpw8y3ny41uxk01fyrg2ertdlagw2qg 2424524 2424523 2026-06-08T19:55:37Z ScheduleDeploymentBot 37566 Add [[gerrit:1298891]] to Monday, June 08 UTC late backport window 2424524 wikitext text/x-wiki k8v4x8i79aphpf5rg4pzf0rbqe0ecs9 2424544 2424524 2026-06-08T22:21:40Z ScheduleDeploymentBot 37566 Add [[gerrit:1298929]] to Tuesday, June 09 UTC afternoon backport window 2424544 wikitext text/x-wiki 87iau6d6p0thtkjduzici6xuya736gt 2424545 2424544 2026-06-08T22:22:02Z ScheduleDeploymentBot 37566 Add [[gerrit:1298926]] to Tuesday, June 09 UTC afternoon backport window 2424545 wikitext text/x-wiki jc0v3ajekab0kr7rlz4x1jlxqeodpwm 2424546 2424545 2026-06-08T22:22:12Z ScheduleDeploymentBot 37566 Add [[gerrit:1298927]] to Tuesday, June 09 UTC afternoon backport window 2424546 wikitext text/x-wiki 4kddv16ojck2q53rfeb5fedbnxix6dg 2424547 2424546 2026-06-08T22:22:37Z ScheduleDeploymentBot 37566 Add [[gerrit:1298925]] to Tuesday, June 09 UTC afternoon backport window 2424547 wikitext text/x-wiki h9xptkhu1nm1ca0r6505hs5kruoaufq 2424574 2424547 2026-06-09T06:05:25Z Revi C. 1919 /* {{Deployment_day|date=2026-06-09}} */ puppet request windows [[gerrit:1298381]] 2424574 wikitext text/x-wiki 0g1rifwyk7kh0v2d0tdydagwq50f32v 2424625 2424574 2026-06-09T08:38:00Z WBrown (WMF) 37696 /* {{Deployment_day|date=2026-06-09}} */ + 2424625 wikitext text/x-wiki fiuws69nyyj01bzt9gqeja9vm3rxp1o 2424728 2424625 2026-06-09T11:30:43Z ScheduleDeploymentBot 37566 Add [[gerrit:1299451]] to Thursday, June 11 UTC morning backport window 2424728 wikitext text/x-wiki om6isiyfwo0maf18cchq3c5203g1i8y 0 6479 2424667 2420358 2026-06-09T09:51:03Z Effie Mouzeli (WMF) 12880 scrollbot update 2424667 wikitext text/x-wiki 4i1xbo861lk77uz268z38j8djlkpydo 0 7919 2424237 2424236 2026-06-08T12:00:26Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/ratelimit: apply 2424237 wikitext text/x-wiki qxu1tkvjik5mk8gq63k6wvt718cgp7x 2424238 2424237 2026-06-08T12:00:38Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/ratelimit: apply 2424238 wikitext text/x-wiki 6npvbud5uizx7e6s6nn7gvv04jh1g2v 2424239 2424238 2026-06-08T12:00:40Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.dns.netbox 2424239 wikitext text/x-wiki 6v6rddj37gqgopigfalaq2hgk0od53g 2424240 2424239 2026-06-08T12:00:43Z Stashbot 7414 ayounsi@cumin1003: END (ERROR) - Cookbook sre.dns.netbox (exit_code=97) 2424240 wikitext text/x-wiki l5r0290ny7txchytg5adt3kqxdg4gtk 2424241 2424240 2026-06-08T12:01:18Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.dns.netbox 2424241 wikitext text/x-wiki nzfjxinkfbjojhxct7az1qm95v0cqgr 2424242 2424241 2026-06-08T12:01:45Z Stashbot 7414 joal@deploy1003: Started deploy [analytics/refinery@d67c584] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@d67c584f] 2424242 wikitext text/x-wiki fi37thpc75cj94l8ntv033a0n224oqq 2424243 2424242 2026-06-08T12:03:10Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es2041.codfw.wmnet with reason: host reimage 2424243 wikitext text/x-wiki punm47g7hkik0lb0e6fuj66dvp4zfpb 2424244 2424243 2026-06-08T12:03:46Z Stashbot 7414 joal@deploy1003: Finished deploy [analytics/refinery@d67c584] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@d67c584f] (duration: 02m 00s) 2424244 wikitext text/x-wiki 28k61z6ptvh94m7dzh94m6bvk417y92 2424245 2424244 2026-06-08T12:06:15Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add eqiad e8 public vlans - ayounsi@cumin1003" 2424245 wikitext text/x-wiki jbklizyki9c2f78ok7kdx2rzo3afhic 2424246 2424245 2026-06-08T12:06:19Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add eqiad e8 public vlans - ayounsi@cumin1003" 2424246 wikitext text/x-wiki mm7m22246wpdtvez4aq06buttqpqsd5 2424247 2424246 2026-06-08T12:06:21Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2424247 wikitext text/x-wiki 9cg0rgx8ws40tawair8k9x3m9yoekd9 2424248 2424247 2026-06-08T12:08:27Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be2062.codfw.wmnet with reason: host reimage 2424248 wikitext text/x-wiki 2vvukfluanuivcuh5mzyhushn4nd8w6 2424249 2424248 2026-06-08T12:08:43Z Stashbot 7414 joal@deploy1003: Started deploy [analytics/refinery@d67c584]: Regular analytics weekly train [analytics/refinery@d67c584f] 2424249 wikitext text/x-wiki hj1ipco0q872t5mvq9x0izwqp7i6v9n 2424250 2424249 2026-06-08T12:09:32Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es2041.codfw.wmnet with reason: host reimage 2424250 wikitext text/x-wiki 15wlzk2q6mxrkyxvk8013s9xvpgad8e 2424251 2424250 2026-06-08T12:13:25Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be2062.codfw.wmnet with reason: host reimage 2424251 wikitext text/x-wiki kcgvgx5gbb2k9gu9n7k54xstj9bxtaz 2424253 2424251 2026-06-08T12:15:23Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be2063.codfw.wmnet with reason: host reimage 2424253 wikitext text/x-wiki leevicdihbov7ep8afcbzao1s19ackp 2424254 2424253 2026-06-08T12:16:35Z Stashbot 7414 joal@deploy1003: Finished deploy [analytics/refinery@d67c584]: Regular analytics weekly train [analytics/refinery@d67c584f] (duration: 07m 52s) 2424254 wikitext text/x-wiki 833pdnnx2qagtg0oxleu1iyilq5ue0w 2424255 2424254 2026-06-08T12:17:56Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/ratelimit: apply 2424255 wikitext text/x-wiki 20utihyg4l3ahmbg8h0h5ut4gz6jb12 2424256 2424255 2026-06-08T12:18:03Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/ratelimit: apply 2424256 wikitext text/x-wiki rebjr3ultc8m951ybt3k4vbvzofw4dj 2424257 2424256 2026-06-08T12:19:05Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be2063.codfw.wmnet with reason: host reimage 2424257 wikitext text/x-wiki sfitblnzntpqyzkdd3t9gmsjm5sgzpe 2424258 2424257 2026-06-08T12:19:34Z Stashbot 7414 joal@deploy1003: Started deploy [analytics/refinery@d67c584] (thin): Regular analytics weekly train THIN [analytics/refinery@d67c584f] 2424258 wikitext text/x-wiki 5b8t02yq3b4g6h7whfk3w7c1361flvw 2424259 2424258 2026-06-08T12:21:35Z Stashbot 7414 joal@deploy1003: Finished deploy [analytics/refinery@d67c584] (thin): Regular analytics weekly train THIN [analytics/refinery@d67c584f] (duration: 02m 00s) 2424259 wikitext text/x-wiki pee9mn8vi19gve8myl6oovq6uemm1dj 2424260 2424259 2026-06-08T12:27:07Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es2041.codfw.wmnet with OS trixie 2424260 wikitext text/x-wiki thj772b6oftn1qgnr3tz714lqwj5j7f 2424261 2424260 2026-06-08T12:32:44Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be2062.codfw.wmnet with OS bullseye 2424261 wikitext text/x-wiki o6iwcsi61zu6oqz5g1tlqmdzcl7lquw 2424268 2424261 2026-06-08T12:40:22Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be2063.codfw.wmnet with OS bullseye 2424268 wikitext text/x-wiki 482tosd0sp1z7tf4rutzod9dtwovi32 2424269 2424268 2026-06-08T12:41:39Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2424269 wikitext text/x-wiki bersmhd4qe0woznjikm7jqo692fdfu9 2424271 2424269 2026-06-08T12:43:17Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.major-upgrade (exit_code=99) 2424271 wikitext text/x-wiki ooh4p3rgey5tk1kbu9zc8opuimbs1jx 2424273 2424271 2026-06-08T12:43:31Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2424273 wikitext text/x-wiki i1lyraafck0mpaynh3ocngnqf22z5hx 2424275 2424273 2026-06-08T12:44:27Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2424275 wikitext text/x-wiki g4vpphnyhae16k4s8zb69dc8rqcpdp0 2424277 2424275 2026-06-08T12:46:20Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2424277 wikitext text/x-wiki ezhzhab38isfb0mpvo4pmcqoandeq10 2424278 2424277 2026-06-08T12:47:27Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-serve-codfw] START helmfile.d/admin 'sync'. 2424278 wikitext text/x-wiki 92ryslvpb421996ixrw3l9jx0460ngt 2424280 2424278 2026-06-08T12:49:17Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-serve-codfw] DONE helmfile.d/admin 'sync'. 2424280 wikitext text/x-wiki gh0uko6m7kvgbvsoy8bl20sqbabqasw 2424281 2424280 2026-06-08T12:49:24Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es2041: repool after upgrade 2424281 wikitext text/x-wiki q8q0gubwyqag466vabr1gtoxq8k9jed 2424282 2424281 2026-06-08T12:49:29Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-serve-codfw] START helmfile.d/admin 'sync'. 2424282 wikitext text/x-wiki 9fu67lbfeuf8rxmik5l4s03t8bvowc6 2424284 2424282 2026-06-08T12:51:16Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1298767|Follow-up: Allow CaptchaConsequence to be skipped via hook (T427608)]] 2424284 wikitext text/x-wiki mmd7181v8ard53bymxq9gatglcf6kbh 2424285 2424284 2026-06-08T12:51:22Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-serve-codfw] DONE helmfile.d/admin 'sync'. 2424285 wikitext text/x-wiki bmvtidl4n15rzqcfyq6ut1uer5aulv5 2424286 2424285 2026-06-08T12:53:01Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1298767|Follow-up: Allow CaptchaConsequence to be skipped via hook (T427608)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2424286 wikitext text/x-wiki aan809edyhvad7xk7p2hv7w5wb685vr 2424287 2424286 2026-06-08T12:53:25Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2424287 wikitext text/x-wiki 4h08o07bmfdr5mz3kf50uqqnm08s18k 2424288 2424287 2026-06-08T12:54:40Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. 2424288 wikitext text/x-wiki iz6gksoudsx4r2yf529aufmt29s12g9 2424289 2424288 2026-06-08T12:55:17Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2424289 wikitext text/x-wiki caajskio5vncaixzi4l8t65b9arrudj 2424290 2424289 2026-06-08T12:55:37Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es1041: Upgrading es1041.eqiad.wmnet 2424290 wikitext text/x-wiki b345cj9431saco79nh76k8uucdgot2i 2424291 2424290 2026-06-08T12:55:42Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. 2424291 wikitext text/x-wiki ljd2xcay2yispju67f99rkp0rhbvj8g 2424292 2424291 2026-06-08T12:55:49Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. 2424292 wikitext text/x-wiki tmbh1rj3mdv89jdt0bp72j5z1n90l0t 2424293 2424292 2026-06-08T12:56:08Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool es1041: Upgrading es1041.eqiad.wmnet 2424293 wikitext text/x-wiki bxo0d8ppkgyxk87pr70a74llvi2zlqp 2424294 2424293 2026-06-08T12:56:48Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. 2424294 wikitext text/x-wiki pj0ersb5lflh45tn3lpw0xi3m0nhyp2 2424295 2424294 2026-06-08T12:57:03Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es1041.eqiad.wmnet with OS trixie 2424295 wikitext text/x-wiki 7246k7k1u1tqy4gggl56uz5oycunzhy 2424296 2424295 2026-06-08T12:57:36Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1298767|Follow-up: Allow CaptchaConsequence to be skipped via hook (T427608)]] (duration: 06m 20s) 2424296 wikitext text/x-wiki 0wec7hted5kig6mc2w8qqooyk7iln00 2424300 2424296 2026-06-08T13:10:40Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Started scap sync-world: Backport for [[gerrit:1298418|NewUserMessage: Add $wgNewUserMessageOnAutoCreateFirstEdit (T426206)]], [[gerrit:1298717|Replace NewUserMessageOnAutoCreateFirstEdit with wgNewUserMessageOnFirstEdit (T426206)]], [[gerrit:1298734|Enable wgNewUserMessageOnFirstEdit on incubatorwiki (T426206)]] 2424300 wikitext text/x-wiki cygz67bm7uauyr3pcykiwv9ug2q8quz 2424301 2424300 2026-06-08T13:11:11Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-syntaxhighlight: apply 2424301 wikitext text/x-wiki 2s1kkqcjsiazdv56uuwwyctfyskucd7 2424302 2424301 2026-06-08T13:11:38Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-syntaxhighlight: apply 2424302 wikitext text/x-wiki j2qzyk4wwddc8kj1g89vffr2xepnd4k 2424304 2424302 2026-06-08T13:12:01Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es1041.eqiad.wmnet with reason: host reimage 2424304 wikitext text/x-wiki 101z60f3a2ojj88dzg7kwnxx2ra51k7 2424305 2424304 2026-06-08T13:12:12Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-constraints: apply 2424305 wikitext text/x-wiki ddjvnp0isl5bcqfnc3lmu8v3f9e04g3 2424306 2424305 2026-06-08T13:12:26Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde, neriah: Backport for [[gerrit:1298418|NewUserMessage: Add $wgNewUserMessageOnAutoCreateFirstEdit (T426206)]], [[gerrit:1298717|Replace NewUserMessageOnAutoCreateFirstEdit with wgNewUserMessageOnFirstEdit (T426206)]], [[gerrit:1298734|Enable wgNewUserMessageOnFirstEdit on incubatorwiki (T426206)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki 2424306 wikitext text/x-wiki bwp5k6tyx7xu9u6p075h4olrjbtd3hx 2424307 2424306 2026-06-08T13:12:46Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-constraints: apply 2424307 wikitext text/x-wiki fb07u2czb4w8e4qd4kzqz6hrgnhzt5f 2424310 2424307 2026-06-08T13:17:24Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde, neriah: Continuing with deployment 2424310 wikitext text/x-wiki 9sumfw1pfidl5btsnty0vlw3fwd3218 2424311 2424310 2026-06-08T13:18:42Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es1041.eqiad.wmnet with reason: host reimage 2424311 wikitext text/x-wiki 8irfi1a82d4631ogpvzqs2mp685v4ws 2424312 2424311 2026-06-08T13:21:46Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Finished scap sync-world: Backport for [[gerrit:1298418|NewUserMessage: Add $wgNewUserMessageOnAutoCreateFirstEdit (T426206)]], [[gerrit:1298717|Replace NewUserMessageOnAutoCreateFirstEdit with wgNewUserMessageOnFirstEdit (T426206)]], [[gerrit:1298734|Enable wgNewUserMessageOnFirstEdit on incubatorwiki (T426206)]] (duration: 11m 06s) 2424312 wikitext text/x-wiki 96fffm8z8jz7v9o6cgboxlfmsmcanfz 2424314 2424312 2026-06-08T13:29:05Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Started scap sync-world: Backport for [[gerrit:1298758|feat(V2): toggle experiment features based on custom url override (T424646)]], [[gerrit:1298762|specialCreateAccount: use GECreateAccountExperimentV2 instead of hook (T424646)]], [[gerrit:1298764|fix: correctly read experiments param on Special:UserLogin]], [[gerrit:1298765|signup.js: use JS var instead of TestKitchen to show expe 2424314 wikitext text/x-wiki f818d698zv6xihi0a1779vmdsmj530h 2424316 2424314 2026-06-08T13:30:50Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: migr, lucaswerkmeister-wmde: Backport for [[gerrit:1298758|feat(V2): toggle experiment features based on custom url override (T424646)]], [[gerrit:1298762|specialCreateAccount: use GECreateAccountExperimentV2 instead of hook (T424646)]], [[gerrit:1298764|fix: correctly read experiments param on Special:UserLogin]], [[gerrit:1298765|signup.js: use JS var instead of TestKitchen to show 2424316 wikitext text/x-wiki av4qfon9qko1zwfouaioi2miko5qgoz 2424317 2424316 2026-06-08T13:33:00Z Stashbot 7414 urbanecm@deploy1003: helmfile [staging] START helmfile.d/services/linkrecommendation: apply 2424317 wikitext text/x-wiki 7866m88o5ggba393nn7a4vq57oio5ab 2424318 2424317 2026-06-08T13:34:09Z Stashbot 7414 urbanecm@deploy1003: helmfile [staging] DONE helmfile.d/services/linkrecommendation: apply 2424318 wikitext text/x-wiki cmsdmfi52ei2vmsiqxihzy8iz05aadq 2424319 2424318 2026-06-08T13:34:18Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: migr, lucaswerkmeister-wmde: Continuing with deployment 2424319 wikitext text/x-wiki 0wejdu8p1q269z4ffnxtdk71xvewzo4 2424320 2424319 2026-06-08T13:34:49Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es2041: repool after upgrade 2424320 wikitext text/x-wiki rout069tq9862bxo9gso20v5kapfh0u 2424321 2424320 2026-06-08T13:34:52Z Stashbot 7414 urbanecm@deploy1003: helmfile [eqiad] START helmfile.d/services/linkrecommendation: apply 2424321 wikitext text/x-wiki q4myto23buwn2gd6i1dh3oehdien58r 2424322 2424321 2026-06-08T13:35:38Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es1041.eqiad.wmnet with OS trixie 2424322 wikitext text/x-wiki fcxv3baubmi0v9wg2nmomfqs6j3j28b 2424323 2424322 2026-06-08T13:36:47Z Stashbot 7414 urbanecm@deploy1003: helmfile [eqiad] DONE helmfile.d/services/linkrecommendation: apply 2424323 wikitext text/x-wiki 1ikc43sf2heciqstbgso5o194vnxaye 2424324 2424323 2026-06-08T13:37:12Z Stashbot 7414 urbanecm@deploy1003: helmfile [codfw] START helmfile.d/services/linkrecommendation: apply 2424324 wikitext text/x-wiki 83c6w42ynthz3adxel8mwcjs3c25c1a 2424325 2424324 2026-06-08T13:38:02Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.major-upgrade (exit_code=99) 2424325 wikitext text/x-wiki nzvcciamymufpmml9z7xlok0711usr5 2424326 2424325 2026-06-08T13:38:18Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-staging-codfw] 'sync' command on namespace 'liftwing-openapi-server' for release 'main' . 2424326 wikitext text/x-wiki di3ivh0emn7qhgfj4rlpja87m4ezxkf 2424327 2424326 2026-06-08T13:38:20Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es1041: repool after maintenance 2424327 wikitext text/x-wiki 03uaslzc2ybq7h33p2qqmaulcemzf44 2424328 2424327 2026-06-08T13:38:29Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Finished scap sync-world: Backport for [[gerrit:1298758|feat(V2): toggle experiment features based on custom url override (T424646)]], [[gerrit:1298762|specialCreateAccount: use GECreateAccountExperimentV2 instead of hook (T424646)]], [[gerrit:1298764|fix: correctly read experiments param on Special:UserLogin]], [[gerrit:1298765|signup.js: use JS var instead of TestKitchen to show exp 2424328 wikitext text/x-wiki eruooz31hcafxzdbtkwwywgpid63pd6 2424329 2424328 2026-06-08T13:39:27Z Stashbot 7414 urbanecm@deploy1003: helmfile [codfw] DONE helmfile.d/services/linkrecommendation: apply 2424329 wikitext text/x-wiki fwytn3eg7rfw2sd9gcspohdfca0v9yw 2424330 2424329 2026-06-08T13:41:50Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1296550|hCaptcha: Don't show AbuseFilter CAPTCHA for wbsetclaim API (T427608)]] 2424330 wikitext text/x-wiki 66i4b3y117zj1vwhh795hf4wnrnxhqc 2424331 2424330 2026-06-08T13:42:00Z Stashbot 7414 cgoubert@dns1004: START - running authdns-update 2424331 wikitext text/x-wiki rzzllouzuh9wr7tx70v19de279wjaar 2424332 2424331 2026-06-08T13:43:37Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1296550|hCaptcha: Don't show AbuseFilter CAPTCHA for wbsetclaim API (T427608)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2424332 wikitext text/x-wiki 6owbqak7bi1rqq9mw6e028bnuh8st5v 2424333 2424332 2026-06-08T13:44:00Z Stashbot 7414 cgoubert@dns1004: END - running authdns-update 2424333 wikitext text/x-wiki g6mnbv7w47x4aoqo1zfhm22xzw4fzr0 2424334 2424333 2026-06-08T13:46:05Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2424334 wikitext text/x-wiki k7d8of5jlrviie6155rcsfdc6pqpdyb 2424335 2424334 2026-06-08T13:48:33Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2424335 wikitext text/x-wiki mxvchzruai5jff4dw2azol3a3afumqt 2424336 2424335 2026-06-08T13:50:22Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1296550|hCaptcha: Don't show AbuseFilter CAPTCHA for wbsetclaim API (T427608)]] (duration: 08m 31s) 2424336 wikitext text/x-wiki nou1rkpinr4psrd99rvw8l5izqirykz 2424337 2424336 2026-06-08T13:50:25Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2424337 wikitext text/x-wiki 50h4c2bbmxwmiqc5q0216tokfb7k4qk 2424338 2424337 2026-06-08T13:50:54Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2424338 wikitext text/x-wiki 3d06hiafj00vr4e8vm9rrlv8j897iya 2424339 2424338 2026-06-08T13:52:47Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2424339 wikitext text/x-wiki d6ikyvor5bt6eh4a4axjfhsl7buidgy 2424340 2424339 2026-06-08T13:54:29Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] 'sync' command on namespace 'liftwing-openapi-server' for release 'main' . 2424340 wikitext text/x-wiki 8oj3g3q7qz88543g6e8g8tcvlk3bba0 2424341 2424340 2026-06-08T14:05:33Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] 'sync' command on namespace 'liftwing-openapi-server' for release 'main' . 2424341 wikitext text/x-wiki h7evhjgp54ihrf1gsgwoxjmknv4sqi7 2424342 2424341 2026-06-08T14:05:53Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-codfw] 'sync' command on namespace 'liftwing-openapi-server' for release 'main' . 2424342 wikitext text/x-wiki qagosvk0wdcx403iz61x1r5wnkw3qv8 2424343 2424342 2026-06-08T14:10:27Z Stashbot 7414 cgoubert@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2424343 wikitext text/x-wiki 5x73tkjpe8cxxb36bseflkq8bvbzmo1 2424344 2424343 2026-06-08T14:10:40Z Stashbot 7414 fabfur@cumin1003: conftool action : set/pooled=yes; selector: name=cp6013.* 2424344 wikitext text/x-wiki 6s4000ldmevtiypc4gig8b03nquq44b 2424345 2424344 2026-06-08T14:11:53Z Stashbot 7414 cgoubert@cumin1003: conftool action : set/pooled=true; selector: dnsdisc=liftwing-openapi-server.* 2424345 wikitext text/x-wiki 06kq54o8hs8yweyd0s5sigjklvgdt8h 2424346 2424345 2026-06-08T14:18:10Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde, audreypenven: Backport for [[gerrit:1298709|Add translatable messages for WikiProject names (T427804)]], [[gerrit:1298710|Use translatable messages for WikiProject links (T427804)]], [[gerrit:1297644|WikiProject links - remove 'text' config (T427804)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2424346 wikitext text/x-wiki i1v3h6m7lguvl43esv43nywql2j7s7p 2424347 2424346 2026-06-08T14:19:58Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde, audreypenven: Continuing with deployment 2424347 wikitext text/x-wiki oy75x2u7ys48ukmu2wjbxffdwbvljod 2424349 2424347 2026-06-08T14:23:46Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es1041: repool after maintenance 2424349 wikitext text/x-wiki imnim47f02deswp6g0bwu74z2o9qrga 2424350 2424349 2026-06-08T14:23:57Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . 2424350 wikitext text/x-wiki 11nfq1k20szwopjqbumr9dgubk01msg 2424351 2424350 2026-06-08T14:24:27Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Promote es2043 to es4 codfw primary T428386', diff saved to https://phabricator.wikimedia.org/P93926 and previous config saved to /var/cache/conftool/dbconfig/20260608-142423-marostegui.json 2424351 wikitext text/x-wiki la5ei3u5he9qxt0itr4blsitskis5t6 2424352 2424351 2026-06-08T14:25:03Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2424352 wikitext text/x-wiki 4v1c2yuwqt47xd6183a0ih976nn2yf8 2424353 2424352 2026-06-08T14:25:13Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es2042: Upgrading es2042.codfw.wmnet 2424353 wikitext text/x-wiki pll2sx0gqbcc9bqbhsag6k6be2l8r2a 2424354 2424353 2026-06-08T14:25:45Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool es2042: Upgrading es2042.codfw.wmnet 2424354 wikitext text/x-wiki foaodovew4l9si22q1nionkmoqh2og1 2424355 2424354 2026-06-08T14:26:09Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . 2424355 wikitext text/x-wiki il48ef2h2p7tfcutc04txoeem0plm8z 2424356 2424355 2026-06-08T14:26:43Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es2042.codfw.wmnet with OS trixie 2424356 wikitext text/x-wiki qzg9c1v1e3cm9co9qgei9rfxlvuzqa1 2424357 2424356 2026-06-08T14:27:05Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . 2424357 wikitext text/x-wiki oq2a8gtu57lhxzdpmrd20f5d15iqpif 2424359 2424357 2026-06-08T14:32:30Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Finished scap sync-world: Backport for [[gerrit:1298709|Add translatable messages for WikiProject names (T427804)]], [[gerrit:1298710|Use translatable messages for WikiProject links (T427804)]], [[gerrit:1297644|WikiProject links - remove 'text' config (T427804)]] (duration: 31m 57s) 2424359 wikitext text/x-wiki cvgpgb4ukzxmmqpxv61hsytabh7pyuq 2424360 2424359 2026-06-08T14:32:32Z Stashbot 7414 Lucas_WMDE: UTC afternoon backport+config window done 2424360 wikitext text/x-wiki hib401j79xx3fnf47w2gydqjnzcite5 2424361 2424360 2026-06-08T14:42:39Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es2042.codfw.wmnet with reason: host reimage 2424361 wikitext text/x-wiki bq1zory75y8tzhl2no35onc48otfc33 2424363 2424361 2026-06-08T14:49:52Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es2042.codfw.wmnet with reason: host reimage 2424363 wikitext text/x-wiki 28bmidumviwc6xeslgbm27xal7sl0jm 2424364 2424363 2026-06-08T14:49:57Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2424364 wikitext text/x-wiki rcqg4ext16skrwvazjp8kp9oj7fbelp 2424365 2424364 2026-06-08T14:50:24Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2424365 wikitext text/x-wiki hmmur4k2m3wwo0npvis9db6ecq0m8yb 2424366 2424365 2026-06-08T14:50:26Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2424366 wikitext text/x-wiki a4zhueemi9a88w18qyw1u842korfjan 2424367 2424366 2026-06-08T14:50:52Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2424367 wikitext text/x-wiki eg5noe6efnet72i9mxce7ozovce8d09 2424369 2424367 2026-06-08T14:54:12Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2424369 wikitext text/x-wiki 7epd9q6cihp61flnjht3p0l3uox534y 2424370 2424369 2026-06-08T14:55:10Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2424370 wikitext text/x-wiki 8tt8npvifo3253o4ils93ixrluhu95i 2424371 2424370 2026-06-08T14:55:43Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2424371 wikitext text/x-wiki ekqgwhpauabl9m3kvgxz3p0ewajutvb 2424372 2424371 2026-06-08T14:59:03Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2424372 wikitext text/x-wiki ntjsavubo5k9nqqkmo1kxk6qtbzso54 2424373 2424372 2026-06-08T15:00:55Z Stashbot 7414 eevans@deploy1003: helmfile [staging] START helmfile.d/services/data-gateway: apply 2424373 wikitext text/x-wiki 2w318mi85p8jm396dzkkcs7z5uv2v1o 2424374 2424373 2026-06-08T15:01:30Z Stashbot 7414 eevans@deploy1003: helmfile [staging] DONE helmfile.d/services/data-gateway: apply 2424374 wikitext text/x-wiki di8zqlrvr5s7sxsuu1ve1z8accagkdz 2424375 2424374 2026-06-08T15:02:30Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2424375 wikitext text/x-wiki p198xuab9qbl9iv0q0tvxrq97g7fk0y 2424376 2424375 2026-06-08T15:03:20Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2424376 wikitext text/x-wiki ki0144ee8cgjjxqsiltifuybfvqa3rt 2424377 2424376 2026-06-08T15:03:41Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2424377 wikitext text/x-wiki gqhs13u3ooezniclg1ys62fd0mk65yb 2424378 2424377 2026-06-08T15:03:43Z Stashbot 7414 jynus@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db[2183-2184].codfw.wmnet with reason: Switchover db 2424378 wikitext text/x-wiki 6oveynko0lggk2dxqhnw00tyit4lqg2 2424379 2424378 2026-06-08T15:04:11Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2424379 wikitext text/x-wiki d8fx18f2sfepkk8i1l41wux9tod2vi2 2424380 2424379 2026-06-08T15:04:33Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2424380 wikitext text/x-wiki jizo73hryta1p0blr94cpixwacdhf2c 2424383 2424380 2026-06-08T15:07:15Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es2042.codfw.wmnet with OS trixie 2424383 wikitext text/x-wiki i4kc5dmhnv7qurh894izb9tdmk3niax 2424385 2424383 2026-06-08T15:08:29Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2424385 wikitext text/x-wiki 1aq1cuzo9c3ayb7gfq80o2g4fmesfak 2424386 2424385 2026-06-08T15:08:47Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2424386 wikitext text/x-wiki ov08u7j3u4itzryl9j0ree3ywxiu5bc 2424387 2424386 2026-06-08T15:08:52Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2424387 wikitext text/x-wiki 1075v5n09zxis6m5ezk22ti3stdxjut 2424388 2424387 2026-06-08T15:08:59Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2424388 wikitext text/x-wiki ob9i7o33fgspbjrnzq241hgeuq3lgdc 2424389 2424388 2026-06-08T15:09:03Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2424389 wikitext text/x-wiki 638dfg19jd4al42ev4ffvwqhx0cpcmv 2424390 2424389 2026-06-08T15:09:19Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2424390 wikitext text/x-wiki numijip3y53wdld8o4bexd4zskbrx5h 2424391 2424390 2026-06-08T15:09:24Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2424391 wikitext text/x-wiki eumlwgfetjknjw4txz970zwpqg9w61e 2424394 2424391 2026-06-08T15:12:04Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.major-upgrade (exit_code=99) 2424394 wikitext text/x-wiki 4bn5vchzr9hm6d9usvjo18znqmwiqti 2424395 2424394 2026-06-08T15:12:12Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es2042: repool after upgrade 2424395 wikitext text/x-wiki gxs94kf8xpe1kr8j8ebhobtmm78klov 2424396 2424395 2026-06-08T15:18:20Z Stashbot 7414 jynus: dbmaint on backup1-codfw@codfw (T428467) 2424396 wikitext text/x-wiki fk43gu6mw610k43npcoa4mq1y5y62o3 2424402 2424396 2026-06-08T15:45:16Z Stashbot 7414 jynus@cumin2002: START - Cookbook sre.hosts.remove-downtime for db[2183-2184].codfw.wmnet 2424402 wikitext text/x-wiki 4yhw0sv49f2jntgzpx94v8rpvp7vaaf 2424404 2424402 2026-06-08T15:45:18Z Stashbot 7414 jynus@cumin2002: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for db[2183-2184].codfw.wmnet 2424404 wikitext text/x-wiki ptnzgs7893wlywqvzeluct3mqhwannn 2424407 2424404 2026-06-08T15:57:37Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es2042: repool after upgrade 2424407 wikitext text/x-wiki 5a8111lcblzt6vzf7w1xdqs9fmwra9y 2424409 2424407 2026-06-08T16:06:47Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox: apply 2424409 wikitext text/x-wiki lkzcg5cpyypd3mgtzyv7j5ia0kx8n21 2424410 2424409 2026-06-08T16:07:27Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox: apply 2424410 wikitext text/x-wiki d2q05t1tiztrqzj61zjlao8h9ig64ae 2424411 2424410 2026-06-08T16:08:04Z Stashbot 7414 kamila@deploy1003: helmfile [staging] START helmfile.d/services/shellbox: apply 2424411 wikitext text/x-wiki t97dnzbgvxv9v2vem9c7z8o1ebdlg5z 2424412 2424411 2026-06-08T16:08:33Z Stashbot 7414 kamila@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox: apply 2424412 wikitext text/x-wiki a50j6uolrt9c9qeqrijebv3knzqownk 2424413 2424412 2026-06-08T16:09:25Z Stashbot 7414 kamila@deploy1003: helmfile [staging] START helmfile.d/services/shellbox: apply 2424413 wikitext text/x-wiki shkp00rtevgcnia30ml556yliyuat4w 2424414 2424413 2026-06-08T16:10:05Z Stashbot 7414 kamila@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox: apply 2424414 wikitext text/x-wiki sw6p83tsai8t14ja3qiakx7hgdh932i 2424415 2424414 2026-06-08T16:10:12Z Stashbot 7414 kamila@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-constraints: apply 2424415 wikitext text/x-wiki tl3wul8yrbakuua8d8do6q5gstcvkn0 2424416 2424415 2026-06-08T16:10:19Z Stashbot 7414 kamila@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-constraints: apply 2424416 wikitext text/x-wiki qaohisclt0zkeazukzyax9ps1lm4k4v 2424417 2424416 2026-06-08T16:10:25Z Stashbot 7414 kamila@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply 2424417 wikitext text/x-wiki toji7njkkxtufm3fywpka8mvgix66yc 2424418 2424417 2026-06-08T16:10:39Z Stashbot 7414 kamila@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply 2424418 wikitext text/x-wiki go7yn1vtdgwncig298ov6nbn96rqean 2424419 2424418 2026-06-08T16:10:45Z Stashbot 7414 kamila@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-syntaxhighlight: apply 2424419 wikitext text/x-wiki 3b9c82nsfvv7katd3bm3nom0gqdg621 2424420 2424419 2026-06-08T16:12:37Z Stashbot 7414 kamila@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-syntaxhighlight: apply 2424420 wikitext text/x-wiki l0891gq75n9sf9rkn3y4xendstpshyp 2424421 2424420 2026-06-08T16:12:44Z Stashbot 7414 kamila@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-timeline: apply 2424421 wikitext text/x-wiki 9hyivnqzncdi4y0lh44muf7n89qzphr 2424422 2424421 2026-06-08T16:13:05Z Stashbot 7414 kamila@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-timeline: apply 2424422 wikitext text/x-wiki b8lkodybx3yri8hb0ilqat761rq1g4t 2424423 2424422 2026-06-08T16:13:11Z Stashbot 7414 kamila@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-video: apply 2424423 wikitext text/x-wiki 6sleipun9vl7y7tdzl0w3vpts6ztgy0 2424424 2424423 2026-06-08T16:13:37Z Stashbot 7414 kamila@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-video: apply 2424424 wikitext text/x-wiki d0epindp5rdjoa9ndtc051e6pwm4tla 2424427 2424424 2026-06-08T16:14:23Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox: apply 2424427 wikitext text/x-wiki pi374ln0iiknggp6irnm36uafejpzff 2424429 2424427 2026-06-08T16:14:43Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox: apply 2424429 wikitext text/x-wiki s0194gzt4em6su0vi9c2uypjsvyvlkh 2424430 2424429 2026-06-08T16:14:49Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-constraints: apply 2424430 wikitext text/x-wiki k54kfxux0boct9ohujf7lxm1eya824d 2424431 2424430 2026-06-08T16:14:55Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-constraints: apply 2424431 wikitext text/x-wiki q6i3e9b27v2rh7ol9b661rxa9e67xiq 2424432 2424431 2026-06-08T16:15:02Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply 2424432 wikitext text/x-wiki 8og1nclkvhuvrzd4fij5idbu4439b9s 2424434 2424432 2026-06-08T16:16:13Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply 2424434 wikitext text/x-wiki rkoj8eeh65clac01dzvegqfexoj0s6o 2424435 2424434 2026-06-08T16:16:19Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-syntaxhighlight: apply 2424435 wikitext text/x-wiki by8g7igh26afpez4on3h8i8h8qc8m8u 2424436 2424435 2026-06-08T16:16:31Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-syntaxhighlight: apply 2424436 wikitext text/x-wiki 63fayl7ta6a7n6r6mqiokznjlfdpyxe 2424437 2424436 2026-06-08T16:16:37Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-timeline: apply 2424437 wikitext text/x-wiki 7skt7r7bdkmy2s0fiuqmmpvvxsn83cu 2424438 2424437 2026-06-08T16:17:39Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-timeline: apply 2424438 wikitext text/x-wiki 1ppk8u7kc5dyq404wjvkbdzjgmdkp3a 2424439 2424438 2026-06-08T16:17:45Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-video: apply 2424439 wikitext text/x-wiki ihk011l4y7q8n73n2kkey0h71g4vtrv 2424440 2424439 2026-06-08T16:18:56Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-video: apply 2424440 wikitext text/x-wiki pi3t1xey8k71mrfr43j672rg6jyek5b 2424441 2424440 2026-06-08T16:25:57Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox: apply 2424441 wikitext text/x-wiki hq6efw96kglp94b7btwgim2tbn79qio 2424442 2424441 2026-06-08T16:26:49Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox: apply 2424442 wikitext text/x-wiki imay4eifv36zu1bnx7s5c097p8jvmv3 2424443 2424442 2026-06-08T16:26:55Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-constraints: apply 2424443 wikitext text/x-wiki nwisb9bwb625eyj3dqcurjd0cryycab 2424444 2424443 2026-06-08T16:27:12Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-constraints: apply 2424444 wikitext text/x-wiki jjozoeccrucdre2rms1t03sl5416ixf 2424445 2424444 2026-06-08T16:27:19Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply 2424445 wikitext text/x-wiki 5psax25pz41udxqkt0ojrgstq7ytmzl 2424446 2424445 2026-06-08T16:28:05Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply 2424446 wikitext text/x-wiki ezmltofan4wpv1vs33tsmvjeefikx70 2424447 2424446 2026-06-08T16:28:11Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-syntaxhighlight: apply 2424447 wikitext text/x-wiki nb8855cc6cxk1n019gybdydpfuy2jzj 2424448 2424447 2026-06-08T16:28:22Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-syntaxhighlight: apply 2424448 wikitext text/x-wiki fmpzahzylfcf8ae43ms8leyzpza9lw0 2424449 2424448 2026-06-08T16:28:29Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-timeline: apply 2424449 wikitext text/x-wiki t5kqxlg69fiexapkbhgrrzijh62k68l 2424450 2424449 2026-06-08T16:29:09Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-timeline: apply 2424450 wikitext text/x-wiki f60ybqui3143z6ztjlwoddn7xh09s58 2424451 2424450 2026-06-08T16:29:16Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-video: apply 2424451 wikitext text/x-wiki hscmy2l01xko2d6ywwcuk0baks79ddf 2424452 2424451 2026-06-08T16:30:34Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-video: apply 2424452 wikitext text/x-wiki dxuksjgj5jkju4719iw3b9be6549tdr 2424456 2424452 2026-06-08T16:52:25Z Stashbot 7414 urbanecm@deploy1003: helmfile [staging] START helmfile.d/services/linkrecommendation: apply 2424456 wikitext text/x-wiki dlr66c3qynkcautshx76sexoz55zsn3 2424457 2424456 2026-06-08T16:53:38Z Stashbot 7414 urbanecm@deploy1003: helmfile [staging] DONE helmfile.d/services/linkrecommendation: apply 2424457 wikitext text/x-wiki 60wlm973sl4wmuulk5yileldeadix21 2424458 2424457 2026-06-08T16:53:58Z Stashbot 7414 urbanecm@deploy1003: helmfile [eqiad] START helmfile.d/services/linkrecommendation: apply 2424458 wikitext text/x-wiki 5apoetnhi9lppp33d02mks7mhkh3ux1 2424459 2424458 2026-06-08T16:55:43Z Stashbot 7414 urbanecm@deploy1003: helmfile [eqiad] DONE helmfile.d/services/linkrecommendation: apply 2424459 wikitext text/x-wiki 78va6ohc0p9jmx54qicx5vheenc7wr9 2424460 2424459 2026-06-08T16:57:16Z Stashbot 7414 urbanecm@deploy1003: helmfile [codfw] START helmfile.d/services/linkrecommendation: apply 2424460 wikitext text/x-wiki t8ciozrzrawcjt7migfmgceitm26gqz 2424461 2424460 2026-06-08T16:59:00Z Stashbot 7414 urbanecm@deploy1003: helmfile [codfw] DONE helmfile.d/services/linkrecommendation: apply 2424461 wikitext text/x-wiki fckeajzvmgnoev72z3oa90r5gzhcrok 2424462 2424461 2026-06-08T17:05:30Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply 2424462 wikitext text/x-wiki fkmoze4jf5pi37eq2u6g2pk3i9gkmm2 2424463 2424462 2026-06-08T17:05:53Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply 2424463 wikitext text/x-wiki 99q9xnod34f6080iglhkqvhbs54l37q 2424465 2424463 2026-06-08T17:13:55Z Stashbot 7414 taavi: bounce sirenbot to get it to re-join a channel 2424465 wikitext text/x-wiki ab89lai3ddf7jide6afanijtvwzcsh0 2424466 2424465 2026-06-08T17:21:17Z Stashbot 7414 claime: restarted varnish-frontend service on cp6009 2424466 wikitext text/x-wiki q8273zhav4nxaeiukmb1wd4bay3cjsi 2424467 2424466 2026-06-08T17:21:21Z Stashbot 7414 claime: restarting varnish-frontend service on cp6011 2424467 wikitext text/x-wiki siqtxij5jjl8jepsta5tnnmugza3tgd 2424468 2424467 2026-06-08T17:21:56Z Stashbot 7414 claime: restarting varnish-frontend service on cp6012 2424468 wikitext text/x-wiki d7phh04ospdcdm1ncmjvstvyrouz7kc 2424472 2424468 2026-06-08T17:35:58Z Stashbot 7414 jnuche@deploy1003: Installing scap version "4.268.0" for 2 host(s) 2424472 wikitext text/x-wiki s49bhxzgasjmon7tyhd4opftevbdetm 2424473 2424472 2026-06-08T17:37:50Z Stashbot 7414 jnuche@deploy1003: Installation of scap version "4.268.0" completed for 2 hosts 2424473 wikitext text/x-wiki 2dtoh1qbh083xc6bu8pu4k2eva1ulca 2424475 2424473 2026-06-08T18:02:48Z Stashbot 7414 aokoth@deploy1003: Started deploy [phabricator/deployment@939557b]: deploy phab2003 - T427286 2424475 wikitext text/x-wiki 8s3erzzmtqn7x8erc6nnjdicccnjt6j 2424476 2424475 2026-06-08T18:03:00Z Stashbot 7414 aokoth@deploy1003: Finished deploy [phabricator/deployment@939557b]: deploy phab2003 - T427286 (duration: 00m 12s) 2424476 wikitext text/x-wiki r5xzpvjy72qx8rh1wou783atluxmjnq 2424477 2424476 2026-06-08T18:17:21Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.dns.netbox 2424477 wikitext text/x-wiki hw9zrfgph0onjvueb8jujdl0j477703 2424479 2424477 2026-06-08T18:21:35Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: updating dse-k8s-wdqs2001 to codfw - jhancock@cumin2002" 2424479 wikitext text/x-wiki 5tgniwlfhtzssjwkfjjupuot7r8x2d3 2424480 2424479 2026-06-08T18:21:40Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: updating dse-k8s-wdqs2001 to codfw - jhancock@cumin2002" 2424480 wikitext text/x-wiki fff9mpdho8lcked2vwaxmf3n3rf39ez 2424481 2424480 2026-06-08T18:21:42Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2424481 wikitext text/x-wiki qkwc9qlbojls90ki51obdi8d8jehn6z 2424482 2424481 2026-06-08T18:22:01Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host dse-k8s-wdqs2001 2424482 wikitext text/x-wiki glr36hn1vd6ixcilhob4vmklle3bff1 2424483 2424482 2026-06-08T18:22:16Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host dse-k8s-wdqs2001 2424483 wikitext text/x-wiki tvwyer0i387un69zgcdlqbhw0ebmwax 2424484 2424483 2026-06-08T18:25:48Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.dns.netbox 2424484 wikitext text/x-wiki 3l3xf54b8j7ts242bo1yermvavumnpk 2424486 2424484 2026-06-08T18:31:51Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding dse-k8s-wdqs2002 to codfw - jhancock@cumin2002" 2424486 wikitext text/x-wiki ez1juyrq10j0uz5kh4l9wmhi4h3lu89 2424487 2424486 2026-06-08T18:31:57Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding dse-k8s-wdqs2002 to codfw - jhancock@cumin2002" 2424487 wikitext text/x-wiki c56t2ajuqguksxdq3lx9aumyhrbopot 2424488 2424487 2026-06-08T18:32:00Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2424488 wikitext text/x-wiki 75c1zyiux55285xos32ldndgf73n9ma 2424489 2424488 2026-06-08T18:32:03Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host dse-k8s-wdqs2002 2424489 wikitext text/x-wiki 7m77px0e9zqdpzatwas2gi0bqh140vo 2424490 2424489 2026-06-08T18:33:35Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host dse-k8s-wdqs2002 2424490 wikitext text/x-wiki 7ezdv9ckha8bwzivjevpi9gko87ugzm 2424491 2424490 2026-06-08T18:37:42Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.dns.netbox 2424491 wikitext text/x-wiki kzgcapyy3mp6p0xg8fvmb8buevov2hg 2424492 2424491 2026-06-08T18:42:41Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding wdqs2030 to codfw - jhancock@cumin2002" 2424492 wikitext text/x-wiki 6nbzv7kys6bigrt4re2tv1ahve3z9lw 2424493 2424492 2026-06-08T18:42:46Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding wdqs2030 to codfw - jhancock@cumin2002" 2424493 wikitext text/x-wiki fc3tj7ukqybbfi9ad2c7omf8n5p6mmc 2424494 2424493 2026-06-08T18:42:48Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2424494 wikitext text/x-wiki gpz533cnqyjwgy0mflv7twdhif401ys 2424495 2424494 2026-06-08T18:44:40Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.dns.netbox 2424495 wikitext text/x-wiki 7uor4qfbsjuzhbwifbiqg04z8jw1hiw 2424496 2424495 2026-06-08T18:51:49Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding dse-k8s-wdqs2004 to codfw - jhancock@cumin2002" 2424496 wikitext text/x-wiki 38bdgss479rtms2d705eilhcr6yue1f 2424497 2424496 2026-06-08T18:51:55Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding dse-k8s-wdqs2004 to codfw - jhancock@cumin2002" 2424497 wikitext text/x-wiki 5na0paabnxrl9nti4llghema298r07r 2424498 2424497 2026-06-08T18:51:57Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2424498 wikitext text/x-wiki 3av3nkhcn2l9gp9icezuy69twwugjc0 2424499 2424498 2026-06-08T18:52:18Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host dse-k8s-wdqs2003 2424499 wikitext text/x-wiki tp1efg5qvjdfrpmzxjtfb7x7avsnz29 2424500 2424499 2026-06-08T18:52:29Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host dse-k8s-wdqs2003 2424500 wikitext text/x-wiki b7bkadh0ymqfjlcztkce0ycrus284do 2424501 2424500 2026-06-08T18:52:34Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host dse-k8s-wdqs2004 2424501 wikitext text/x-wiki 92kjzyxlnm62prc3mm94gmdk74o5wsv 2424502 2424501 2026-06-08T18:57:42Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host dse-k8s-wdqs2004 2424502 wikitext text/x-wiki cdjnpfsxpvsza7j55ibhij2mlulbzxo 2424504 2424502 2026-06-08T18:59:21Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host dse-k8s-wdqs2001.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2424504 wikitext text/x-wiki 6qrymo7z3vdl7if6rzfryaov9cce95j 2424505 2424504 2026-06-08T19:06:26Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host dse-k8s-wdqs2001.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2424505 wikitext text/x-wiki 5qmsafny68dx1xuawq01gso37ee21kb 2424506 2424505 2026-06-08T19:07:29Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host dse-k8s-wdqs2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2424506 wikitext text/x-wiki k53fb3ikopeihq7acjjnzr795aksify 2424507 2424506 2026-06-08T19:14:35Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-wdqs2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2424507 wikitext text/x-wiki kgyoa56cox9u14p6mz5ue7fn3b0iqhz 2424508 2424507 2026-06-08T19:16:36Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host dse-k8s-wdqs2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2424508 wikitext text/x-wiki drq5vs1jtei0mg4d10nw0xx11uzyyys 2424509 2424508 2026-06-08T19:19:19Z Stashbot 7414 aokoth@deploy1003: Started deploy [phabricator/deployment@939557b]: deploy phab 2424509 wikitext text/x-wiki 917bqt7gl27n0rftzirm26or6e13yzm 2424510 2424509 2026-06-08T19:21:00Z Stashbot 7414 aokoth@deploy1003: Finished deploy [phabricator/deployment@939557b]: deploy phab (duration: 01m 40s) 2424510 wikitext text/x-wiki 5kmqgh9i349sjtdcq8lfpu420dmw333 2424511 2424510 2026-06-08T19:22:33Z Stashbot 7414 aokoth@deploy1003: Started deploy [phabricator/deployment@939557b]: deploy phab 2424511 wikitext text/x-wiki mweax8vzuilhsx21jq75if55rprv5zo 2424512 2424511 2026-06-08T19:23:48Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-wdqs2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2424512 wikitext text/x-wiki koqvfodx5918pcqdx91s2n2trji6w0g 2424513 2424512 2026-06-08T19:24:05Z Stashbot 7414 aokoth@deploy1003: Finished deploy [phabricator/deployment@939557b]: deploy phab (duration: 01m 32s) 2424513 wikitext text/x-wiki 6zp6cnv9mn3tr1fiye86a1admxdkxco 2424514 2424513 2026-06-08T19:25:44Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1298879|SimpleCaptcha: Re-render captcha when edit form is redisplayed (T428437)]] 2424514 wikitext text/x-wiki dw590fql3amus3r11zqqdsy5ltnouhc 2424515 2424514 2026-06-08T19:27:37Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host dse-k8s-wdqs2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2424515 wikitext text/x-wiki hkiyc7ig52yx0khr4ij1y8cs46pfcp5 2424516 2424515 2026-06-08T19:28:29Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-wdqs2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2424516 wikitext text/x-wiki lmabgn1o4fb3pias1092zex122yqbb3 2424517 2424516 2026-06-08T19:29:43Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1298879|SimpleCaptcha: Re-render captcha when edit form is redisplayed (T428437)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2424517 wikitext text/x-wiki bytajx9m8c0atqt235xqqr57te9dy8t 2424518 2424517 2026-06-08T19:30:01Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host dse-k8s-wdqs2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2424518 wikitext text/x-wiki emrwztv7eijl6xc0kt7z828bv4kl3y1 2424519 2424518 2026-06-08T19:30:26Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-wdqs2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2424519 wikitext text/x-wiki qmbwnyc0izvxmmotqpzous8s6tcz8il 2424520 2424519 2026-06-08T19:31:36Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2424520 wikitext text/x-wiki tavyl1p9lmcmtp2exmb70sr753uoctm 2424521 2424520 2026-06-08T19:43:18Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host dse-k8s-wdqs2004.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2424521 wikitext text/x-wiki to83ivmxqj50drxsc2bs7rpvf88wf4n 2424522 2424521 2026-06-08T19:43:34Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-wdqs2004.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2424522 wikitext text/x-wiki refuznegxkkid8cth1yi8obenzb31wv 2424525 2424522 2026-06-08T20:03:27Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1298879|SimpleCaptcha: Re-render captcha when edit form is redisplayed (T428437)]] (duration: 37m 43s) 2424525 wikitext text/x-wiki 9hbhwcu3jt1cbvu0cpfa7y1k93xo7y3 2424526 2424525 2026-06-08T20:20:40Z Stashbot 7414 mlitn@deploy1003: Started scap sync-world: Backport for [[gerrit:1298390|English Wikibooks: update FlaggedRevs configuration (T428329)]], [[gerrit:1298328|English Wikiversity: Add new user group "autopatrolled" (T428269)]] 2424526 wikitext text/x-wiki r5da1syfjw5ubprtv0f2uluwxvv21uz 2424527 2424526 2026-06-08T20:22:26Z Stashbot 7414 mlitn@deploy1003: mlitn, vadymts1: Backport for [[gerrit:1298390|English Wikibooks: update FlaggedRevs configuration (T428329)]], [[gerrit:1298328|English Wikiversity: Add new user group "autopatrolled" (T428269)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2424527 wikitext text/x-wiki qpsy44pn8nlsckl6ivkad532mwpis15 2424528 2424527 2026-06-08T20:25:18Z Stashbot 7414 mlitn@deploy1003: mlitn, vadymts1: Continuing with deployment 2424528 wikitext text/x-wiki ti9xaejlwhtfx1kb6wn0u2mp5cau549 2424529 2424528 2026-06-08T20:29:38Z Stashbot 7414 mlitn@deploy1003: Finished scap sync-world: Backport for [[gerrit:1298390|English Wikibooks: update FlaggedRevs configuration (T428329)]], [[gerrit:1298328|English Wikiversity: Add new user group "autopatrolled" (T428269)]] (duration: 08m 58s) 2424529 wikitext text/x-wiki 9io5g1xb7ddaa9dbcg4cqvhbqgl9whs 2424530 2424529 2026-06-08T20:36:28Z Stashbot 7414 mlitn@deploy1003: Started scap sync-world: Backport for [[gerrit:1297162|MultimediaViewer: enable image carousel as a beta feature on Wikipedias]], [[gerrit:1298841|Squashed diff to master]] 2424530 wikitext text/x-wiki 796yrih5sofe70qiieepw7nvvah6ji2 2424531 2424530 2026-06-08T20:38:19Z Stashbot 7414 mlitn@deploy1003: mlitn: Backport for [[gerrit:1297162|MultimediaViewer: enable image carousel as a beta feature on Wikipedias]], [[gerrit:1298841|Squashed diff to master]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2424531 wikitext text/x-wiki s5yy7zqm00t2llagkzzezd1z8l6mfxq 2424532 2424531 2026-06-08T20:39:21Z Stashbot 7414 mlitn@deploy1003: mlitn: Continuing with deployment 2424532 wikitext text/x-wiki 33ceau1ok4yw4yu3gbdgisvxaxgzsid 2424533 2424532 2026-06-08T20:43:33Z Stashbot 7414 mlitn@deploy1003: Finished scap sync-world: Backport for [[gerrit:1297162|MultimediaViewer: enable image carousel as a beta feature on Wikipedias]], [[gerrit:1298841|Squashed diff to master]] (duration: 07m 05s) 2424533 wikitext text/x-wiki bl7mjcys9rnxosbbbb6n8yt4k9jg7dv 2424534 2424533 2026-06-08T21:03:52Z Stashbot 7414 mlitn@deploy1003: Started scap sync-world: Backport for [[gerrit:1298891|OOUIHTMLForm: Avoid treating form header as a clickable label (T428359)]] 2424534 wikitext text/x-wiki 0fsvr5oxpxs2feeqa6m6c66hrdo4ns5 2424535 2424534 2026-06-08T21:05:42Z Stashbot 7414 mlitn@deploy1003: mlitn, neriah: Backport for [[gerrit:1298891|OOUIHTMLForm: Avoid treating form header as a clickable label (T428359)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2424535 wikitext text/x-wiki 7s6qtqhapl8d5o4z4ee2sbyw468uu4z 2424536 2424535 2026-06-08T21:07:49Z Stashbot 7414 mlitn@deploy1003: mlitn, neriah: Continuing with deployment 2424536 wikitext text/x-wiki 3q38rnuza4gvl0hgwntkbzwc5898mjm 2424537 2424536 2026-06-08T21:12:02Z Stashbot 7414 mlitn@deploy1003: Finished scap sync-world: Backport for [[gerrit:1298891|OOUIHTMLForm: Avoid treating form header as a clickable label (T428359)]] (duration: 08m 10s) 2424537 wikitext text/x-wiki 0ho1hmv6utjxzoom7co289mdqc9ptpk 2424540 2424537 2026-06-08T21:53:01Z Stashbot 7414 reedy@deploy1003: Started scap sync-world: Backport for [[gerrit:1298915|CommonSettings: Set $wgScoreSafeMode = false (T428484)]] 2424540 wikitext text/x-wiki 3zu4chyaglz64x28t0eikmm0yq672ca 2424541 2424540 2026-06-08T21:54:49Z Stashbot 7414 reedy@deploy1003: reedy: Backport for [[gerrit:1298915|CommonSettings: Set $wgScoreSafeMode = false (T428484)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2424541 wikitext text/x-wiki 18ynx79u0gxzyn8lzd07nf5g08v76ud 2424542 2424541 2026-06-08T21:56:28Z Stashbot 7414 reedy@deploy1003: reedy: Continuing with deployment 2424542 wikitext text/x-wiki khaihtq9wcilp9bcik263tuozbrnyhn 2424543 2424542 2026-06-08T22:00:44Z Stashbot 7414 reedy@deploy1003: Finished scap sync-world: Backport for [[gerrit:1298915|CommonSettings: Set $wgScoreSafeMode = false (T428484)]] (duration: 07m 42s) 2424543 wikitext text/x-wiki fmmm7ewrn0xxztz01omt7tk10rtwmki 2424550 2424543 2026-06-09T02:01:42Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2424550 wikitext text/x-wiki 8b8fabniy371mg3dd01j0a7z5ayb7n9 2424551 2424550 2026-06-09T02:08:21Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 38s) 2424551 wikitext text/x-wiki o146wpayq96tc4bzbky2h7r2w7887d4 2424554 2424551 2026-06-09T03:03:31Z Stashbot 7414 mwpresync@deploy1003: Started scap sync-world: testwikis to 1.47.0-wmf.6 refs T423915 2424554 wikitext text/x-wiki tfr3wgyc1rhponfx1v7lhayae09buq7 2424555 2424554 2026-06-09T03:40:47Z Stashbot 7414 mwpresync@deploy1003: Finished scap sync-world: testwikis to 1.47.0-wmf.6 refs T423915 (duration: 37m 16s) 2424555 wikitext text/x-wiki 5uxd0kvzsgjueefy98b8beb7uosxc7q 2424556 2424555 2026-06-09T04:02:52Z Stashbot 7414 mwpresync@deploy1003: Pruned MediaWiki: 1.47.0-wmf.3 (duration: 02m 43s) 2424556 wikitext text/x-wiki lwbe69axh0u4hcnyk8ce2xy7u5x4qes 2424558 2424556 2026-06-09T05:18:36Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 16 hosts with reason: Primary switchover x1 T428158 2424558 wikitext text/x-wiki dr2jcycq5isv5fi4hx8f70cqo25ta96 2424559 2424558 2026-06-09T05:19:02Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Set db1220 with weight 0 T428158', diff saved to https://phabricator.wikimedia.org/P93932 and previous config saved to /var/cache/conftool/dbconfig/20260609-051859-marostegui.json 2424559 wikitext text/x-wiki 5ifoicgz1798xi9udr858o6wj6j3bes 2424561 2424559 2026-06-09T05:22:14Z Stashbot 7414 marostegui: Starting x1 eqiad failover from db1237 to db1220 - T428158 2424561 wikitext text/x-wiki 1448hlxqj35jxauofff3wnx8xkgi5y8 2424562 2424561 2026-06-09T05:22:56Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Set x1 eqiad as read-only for maintenance - T428158', diff saved to https://phabricator.wikimedia.org/P93933 and previous config saved to /var/cache/conftool/dbconfig/20260609-052253-marostegui.json 2424562 wikitext text/x-wiki t3p9yzhxndue7waqiveb3yhydcyy8ut 2424563 2424562 2026-06-09T05:23:15Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Promote db1220 to x1 primary and set section read-write T428158', diff saved to https://phabricator.wikimedia.org/P93934 and previous config saved to /var/cache/conftool/dbconfig/20260609-052311-marostegui.json 2424563 wikitext text/x-wiki nmcp09qycprz5e7e3djfehh7g2ovgop 2424564 2424563 2026-06-09T05:23:43Z Stashbot 7414 marostegui@dns1004: START - running authdns-update 2424564 wikitext text/x-wiki qdh7ppvaslt3us7z29c66eph3a8rano 2424565 2424564 2026-06-09T05:24:24Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool db1237 T428158', diff saved to https://phabricator.wikimedia.org/P93935 and previous config saved to /var/cache/conftool/dbconfig/20260609-052420-marostegui.json 2424565 wikitext text/x-wiki j0aoqb3v2ks2hwxn7ch0klu0ng63f3q 2424566 2424565 2026-06-09T05:27:15Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2424566 wikitext text/x-wiki f6p7q30thro3530c4itzook8p9sy7pz 2424567 2424566 2026-06-09T05:27:24Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1237: Upgrading db1237.eqiad.wmnet 2424567 wikitext text/x-wiki ky9ioa4pfegpps75acqj1pq2hxfb8bf 2424568 2424567 2026-06-09T05:27:33Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1237: Upgrading db1237.eqiad.wmnet 2424568 wikitext text/x-wiki hq7g1pgwiona24hz4c07pg7p2dnjwxk 2424569 2424568 2026-06-09T05:37:52Z Stashbot 7414 marostegui@dns1004: START - running authdns-update 2424569 wikitext text/x-wiki 0dpemwh12zef3ri4vjijb2h6a3dhjow 2424570 2424569 2026-06-09T05:40:15Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1237.eqiad.wmnet with OS trixie 2424570 wikitext text/x-wiki lc87vskudsl3risrswrukvizyiwttgk 2424571 2424570 2026-06-09T06:01:06Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 40 hosts with reason: Primary switchover s4 T426086 2424571 wikitext text/x-wiki jnx0bdhaoypwjoz5zfn1xne1c0svdui 2424572 2424571 2026-06-09T06:01:26Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Set db1244 with weight 0 T426086', diff saved to https://phabricator.wikimedia.org/P93937 and previous config saved to /var/cache/conftool/dbconfig/20260609-060121-fceratto.json 2424572 wikitext text/x-wiki alwz24fozseyu8e1szhg7lw13hlnqmi 2424575 2424572 2026-06-09T06:10:43Z Stashbot 7414 federico3: Starting s4 eqiad failover from db1160 to db1244 - T426086 2424575 wikitext text/x-wiki qsrj92i667myvspgrftw435cvodemha 2424576 2424575 2026-06-09T06:11:34Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Set s4 eqiad as read-only for maintenance - T426086', diff saved to https://phabricator.wikimedia.org/P93938 and previous config saved to /var/cache/conftool/dbconfig/20260609-061131-fceratto.json 2424576 wikitext text/x-wiki 7muq3jyou5mpx0s412s9m4457l6dymq 2424577 2424576 2026-06-09T06:12:25Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Promote db1244 to s4 primary and set section read-write T426086', diff saved to https://phabricator.wikimedia.org/P93939 and previous config saved to /var/cache/conftool/dbconfig/20260609-061222-fceratto.json 2424577 wikitext text/x-wiki rfv7hlgng4y4hazua6jfpjgs9sw8etl 2424578 2424577 2026-06-09T06:14:44Z Stashbot 7414 cscott@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2424578 wikitext text/x-wiki p0fmjlmu4nar8ceaf09ry6pc9f39bly 2424579 2424578 2026-06-09T06:15:13Z Stashbot 7414 cscott@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2424579 wikitext text/x-wiki lt54v18asfage95frl9b1a8gimae4wo 2424580 2424579 2026-06-09T06:15:15Z Stashbot 7414 cscott@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2424580 wikitext text/x-wiki f9zsx49q3o49huyd4c1ddfd34x72hfr 2424581 2424580 2026-06-09T06:15:45Z Stashbot 7414 cscott@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2424581 wikitext text/x-wiki iphxyxp3wkngn66tze8xa4iq355ue61 2424582 2424581 2026-06-09T06:16:55Z Stashbot 7414 cscott@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2424582 wikitext text/x-wiki qzazbbuvmpumdnld0ancao8bqgtm23e 2424583 2424582 2026-06-09T06:16:57Z Stashbot 7414 cscott@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2424583 wikitext text/x-wiki nfjo8hzzolktrov1nxsft5dsyxiz7oa 2424584 2424583 2026-06-09T06:16:59Z Stashbot 7414 cscott@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2424584 wikitext text/x-wiki sed0zs62d8xas1x2r63zdgvq83vpjir 2424585 2424584 2026-06-09T06:17:02Z Stashbot 7414 cscott@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2424585 wikitext text/x-wiki cjyzdlrtz66qdtzfvk6ojqp6cqtlb92 2424586 2424585 2026-06-09T06:24:16Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depool db1160 T426086', diff saved to https://phabricator.wikimedia.org/P93940 and previous config saved to /var/cache/conftool/dbconfig/20260609-062412-fceratto.json 2424586 wikitext text/x-wiki cskv161ganw2m8vyntt0vq672a7mcmn 2424587 2424586 2026-06-09T07:00:50Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host db1237.eqiad.wmnet with OS trixie 2424587 wikitext text/x-wiki 1iy8twi8wk9l2fawzfoy5j78mg6n3av 2424588 2424587 2026-06-09T07:00:52Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.major-upgrade (exit_code=99) 2424588 wikitext text/x-wiki sk6ebpnlypgpeq3m1miu6ev8xqv4e5g 2424589 2424588 2026-06-09T07:11:14Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.pool pool db1160: Repooling 2424589 wikitext text/x-wiki eycvvr3kedsdep6deh2eyzg18p8m1bw 2424590 2424589 2026-06-09T07:11:22Z Stashbot 7414 fceratto@cumin1003: END (ERROR) - Cookbook sre.mysql.pool (exit_code=97) pool db1160: Repooling 2424590 wikitext text/x-wiki 80l1ex5xminwbudy6vp16ajfm0nofl1 2424591 2424590 2026-06-09T07:11:27Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.pool pool db1160: Repooling 2424591 wikitext text/x-wiki 0peurbe9eseejlvd9xljaev1bwvzfid 2424592 2424591 2026-06-09T07:11:33Z Stashbot 7414 fceratto@cumin1003: END (ERROR) - Cookbook sre.mysql.pool (exit_code=97) pool db1160: Repooling 2424592 wikitext text/x-wiki cdg2ii3opk7rer896yu1ipl1wr41n4f 2424593 2424592 2026-06-09T07:12:44Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.dns.netbox 2424593 wikitext text/x-wiki 34wdxw295qez0gykenlr66tl73w9cjc 2424594 2424593 2026-06-09T07:16:07Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1160.eqiad.wmnet with reason: Maintenance 2424594 wikitext text/x-wiki pl9ztioy1tk2wvftbyi9mls84zp91f3 2424595 2424594 2026-06-09T07:19:25Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Fix dse-k8s-wdqs2002 duplicate ipv6 address - elukey@cumin1003" 2424595 wikitext text/x-wiki 8mtoh1oe242dg0gl7j7n6ftujs6ta9l 2424596 2424595 2026-06-09T07:19:30Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Fix dse-k8s-wdqs2002 duplicate ipv6 address - elukey@cumin1003" 2424596 wikitext text/x-wiki hn2nbda20fp3ayuvepnhdx10bnea1e9 2424597 2424596 2026-06-09T07:19:32Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2424597 wikitext text/x-wiki llxot5sru3z19y18ds1bgzbrtng6ap3 2424598 2424597 2026-06-09T07:21:19Z Stashbot 7414 marostegui@dns1004: START - running authdns-update 2424598 wikitext text/x-wiki hcobajncu3lmtlu32rt5pk3odnxkdt2 2424599 2424598 2026-06-09T07:22:49Z Stashbot 7414 marostegui@dns1004: END - running authdns-update 2424599 wikitext text/x-wiki jjswk41xxakhyhanfiumbio2bs8tvzd 2424600 2424599 2026-06-09T07:24:45Z Stashbot 7414 fceratto@dns1004: START - running authdns-update 2424600 wikitext text/x-wiki b7k3z1fcvuv0wea480uj35d25psaqkc 2424601 2424600 2026-06-09T07:26:20Z Stashbot 7414 fceratto@dns1004: END - running authdns-update 2424601 wikitext text/x-wiki 8ilbffssfh4w4lxuna32ma57ncmhwwr 2424602 2424601 2026-06-09T07:36:02Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2424602 wikitext text/x-wiki mkoyngwt6qftqfs5054qpncpixlogq5 2424603 2424602 2026-06-09T07:36:05Z Stashbot 7414 brouberol@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/admin 'apply'. 2424603 wikitext text/x-wiki 7bjo71xcel3bazrpokgu9n7gfeslm6u 2424604 2424603 2026-06-09T07:36:37Z Stashbot 7414 brouberol@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2424604 wikitext text/x-wiki fwnb7f8b442ajzbje39cuqnjdj4axbm 2424605 2424604 2026-06-09T07:36:39Z Stashbot 7414 marostegui@cumin1003: END (ERROR) - Cookbook sre.mysql.major-upgrade (exit_code=97) 2424605 wikitext text/x-wiki d514lz9qhh3cshskvyc93hnhv9d506x 2424606 2424605 2026-06-09T07:37:34Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1237.eqiad.wmnet with OS trixie 2424606 wikitext text/x-wiki 518mdeqxcd2litp8qllg12t11ete849 2424607 2424606 2026-06-09T07:37:55Z Stashbot 7414 brouberol@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/admin 'apply'. 2424607 wikitext text/x-wiki 6mrry3ltsip9z8i5al4ui39uk241nac 2424608 2424607 2026-06-09T07:38:57Z Stashbot 7414 brouberol@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/admin 'apply'. 2424608 wikitext text/x-wiki t7te6ade9xodfj8j9rya9j7y5buwh88 2424609 2424608 2026-06-09T07:39:14Z Stashbot 7414 brouberol@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/aux-k8s-services/kafka-ui: apply 2424609 wikitext text/x-wiki 97grnc0e8rdxgc7ujb6ae6bzvoyde0a 2424610 2424609 2026-06-09T07:41:32Z Stashbot 7414 brouberol@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/aux-k8s-services/kafka-ui: apply 2424610 wikitext text/x-wiki 9zwujm3r9xcruo5wtcq5kguqq3zsj7x 2424611 2424610 2026-06-09T07:42:54Z Stashbot 7414 brouberol@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/aux-k8s-services/kafka-ui: apply 2424611 wikitext text/x-wiki 6vrhc48554kit7zlbqqaoh289w7bbz1 2424612 2424611 2026-06-09T07:43:33Z Stashbot 7414 brouberol@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/aux-k8s-services/kafka-ui: apply 2424612 wikitext text/x-wiki iij7q8rjudgs0ue2v0x1f8uu04acgx3 2424613 2424612 2026-06-09T07:43:45Z Stashbot 7414 brouberol@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/aux-k8s-services/kafka-ui: apply 2424613 wikitext text/x-wiki 9mc3l9pubwkef54k4x7r3ezipvxicfq 2424614 2424613 2026-06-09T07:44:06Z Stashbot 7414 brouberol@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/aux-k8s-services/kafka-ui: apply 2424614 wikitext text/x-wiki hczgmr8amaugmz3ifoibbqjr0th4xda 2424615 2424614 2026-06-09T07:46:07Z Stashbot 7414 brouberol@dns1004: START - running authdns-update 2424615 wikitext text/x-wiki rt7ok5k1mh4l9fop7hiuhtigig656z2 2424616 2424615 2026-06-09T07:47:42Z Stashbot 7414 brouberol@dns1004: END - running authdns-update 2424616 wikitext text/x-wiki tvdsfhmanpo2dzv64buwbps6xb8pczk 2424617 2424616 2026-06-09T07:52:14Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.dns.netbox 2424617 wikitext text/x-wiki 37hl1vexoba21hw8iyzg1qa86snwtmj 2424618 2424617 2026-06-09T07:59:05Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: fix typoes - ayounsi@cumin1003" 2424618 wikitext text/x-wiki jqe3boxje3whnb5tz0ddhzj4llymi9t 2424619 2424618 2026-06-09T07:59:09Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: fix typoes - ayounsi@cumin1003" 2424619 wikitext text/x-wiki feof7mgkii8jqcmebya3ywjmp5stq17 2424620 2424619 2026-06-09T07:59:11Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2424620 wikitext text/x-wiki 6rlpduqvhttd47cajub7zvblyg60mmq 2424621 2424620 2026-06-09T08:08:06Z Stashbot 7414 jiji@cumin1003: conftool action : set/pooled=false; selector: dnsdisc=docker-registry,name=codfw 2424621 wikitext text/x-wiki cweij3nib93ldvgx4rzui5uc9b2wr7a 2424622 2424621 2026-06-09T08:08:16Z Stashbot 7414 jiji@cumin1003: conftool action : set/pooled=true; selector: dnsdisc=docker-registry,name=eqiad 2424622 wikitext text/x-wiki a2v1sz930yjspavsgluivrx2yc8bfk7 2424623 2424622 2026-06-09T08:22:11Z Stashbot 7414 jiji@cumin1003: conftool action : set/pooled=false; selector: dnsdisc=docker-registry,name=eqiad 2424623 wikitext text/x-wiki 83k0zjyi263405yjj99td0dgga3soze 2424624 2424623 2026-06-09T08:22:19Z Stashbot 7414 jiji@cumin1003: conftool action : set/pooled=true; selector: dnsdisc=docker-registry,name=codfw 2424624 wikitext text/x-wiki fxxhbyaw71ru4azodl9stthdl7xitk3 2424627 2424624 2026-06-09T08:50:24Z Stashbot 7414 jiji@cumin1003: conftool action : set/pooled=no; selector: service=docker-registry,name=registry2004.codfw.wmnet 2424627 wikitext text/x-wiki 2yaeuqpl0a9jt3xeby9u9jixn4gwyfi 2424629 2424627 2026-06-09T08:55:02Z Stashbot 7414 jiji@cumin1003: conftool action : set/pooled=yes; selector: service=docker-registry,name=registry2004.codfw.wmnet 2424629 wikitext text/x-wiki dpey2l3pnx6kyqrc2iblbyr788tqozy 2424630 2424629 2026-06-09T08:55:13Z Stashbot 7414 jiji@cumin1003: conftool action : set/pooled=no; selector: service=docker-registry,name=registry2005.codfw.wmnet 2424630 wikitext text/x-wiki b6gtdbni9htvnhaslwhwtcsaei2wzbc 2424631 2424630 2026-06-09T08:57:44Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host db1237.eqiad.wmnet with OS trixie 2424631 wikitext text/x-wiki g3dsbqqsi8uzivpxe91lebubgyttdto 2424632 2424631 2026-06-09T08:59:07Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2424632 wikitext text/x-wiki j1i6tfk07rjdzs7fh5z6mmfuirmnd7p 2424633 2424632 2026-06-09T08:59:38Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2424633 wikitext text/x-wiki 1nkkm86f779pcuhttsp2bb5mzgzcwel 2424635 2424633 2026-06-09T09:03:13Z Stashbot 7414 jiji@cumin1003: conftool action : set/pooled=yes; selector: service=docker-registry,name=registry2005.codfw.wmnet 2424635 wikitext text/x-wiki ad4quhuxjyzr9qpi1xm609cw7spcylv 2424637 2424635 2026-06-09T09:11:51Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Promote es1043 to es4 eqiad primary T428386', diff saved to https://phabricator.wikimedia.org/P93942 and previous config saved to /var/cache/conftool/dbconfig/20260609-091147-marostegui.json 2424637 wikitext text/x-wiki ivhu95cqnx0cijbfaxe8ttia71zbhu7 2424639 2424637 2026-06-09T09:12:18Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Promote es1042 to es4 eqiad primary T428386', diff saved to https://phabricator.wikimedia.org/P93943 and previous config saved to /var/cache/conftool/dbconfig/20260609-091215-marostegui.json 2424639 wikitext text/x-wiki 4u0ir0seobio3yzivslov539kst5dlh 2424640 2424639 2026-06-09T09:14:15Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2424640 wikitext text/x-wiki ah0mh0xoho68qje769wpfpkqhox3j8u 2424641 2424640 2026-06-09T09:14:34Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es1043: Upgrading es1043.eqiad.wmnet 2424641 wikitext text/x-wiki jr5ymyfeojpyniyhtth26m01pvn5s9l 2424643 2424641 2026-06-09T09:17:16Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.global-read-only 2424643 wikitext text/x-wiki pdg3svgsoguviookry112xfod1gvgtr 2424644 2424643 2026-06-09T09:17:21Z Stashbot 7414 fceratto@cumin1003: MariaDB change: Setting sections s5 as read-write 2424644 wikitext text/x-wiki 5ct80p14ritf5rmsrmc9syxd53fiw28 2424645 2424644 2026-06-09T09:17:29Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.global-read-only (exit_code=0) 2424645 wikitext text/x-wiki 86i7ug1fv3cjpqshh59mkjeiksvjood 2424648 2424645 2026-06-09T09:26:29Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2424648 wikitext text/x-wiki r8nqzt5g1bd3ym6ebmbdyxc2zkh1mv5 2424650 2424648 2026-06-09T09:27:28Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2424650 wikitext text/x-wiki mswixyxgsfo1ax8dl77m2xzumb54q2s 2424655 2424650 2026-06-09T09:34:49Z Stashbot 7414 Dreamy_Jazz: Running `mwscript-k8s extensions/MediaModeration/maintenance/scanFilesInScanTable.php --wiki="commonswiki" --use-jobqueue --poll-sleep=5 --verbose` (after stopping previous scan run) 2424655 wikitext text/x-wiki dqaczrvhrq65dh4rdukkbltobwh5nu4 2424656 2424655 2026-06-09T09:36:49Z Stashbot 7414 Dreamy_Jazz: Running `mwscript-k8s extensions/MediaModeration/maintenance/scanFilesInScanTable.php --wiki="commonswiki" --use-jobqueue --poll-sleep=5 --verbose --last-checked="20260603"` (after stopping previous scan run) 2424656 wikitext text/x-wiki c9256ehg6hnmldkf6hfea67brrpq11j 2424659 2424656 2026-06-09T09:41:58Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2424659 wikitext text/x-wiki qyivhcmh6nkiv0jjej3tojwngbhv03g 2424660 2424659 2026-06-09T09:45:14Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2424660 wikitext text/x-wiki 8i3s2s9oijpcelp196tml0g7ldouf1l 2424661 2424660 2026-06-09T09:47:52Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2424661 wikitext text/x-wiki tjzjqovaocouge9dkj0jhvqe3qydqae 2424662 2424661 2026-06-09T09:48:03Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2424662 wikitext text/x-wiki 33fhofjalibbm8qqq3uoq2e02lxe2bv 2424663 2424662 2026-06-09T09:48:21Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.depool (exit_code=99) depool es1043: Upgrading es1043.eqiad.wmnet 2424663 wikitext text/x-wiki dgqmwrpibo13j53jrns5g2nepc9lnkr 2424664 2424663 2026-06-09T09:49:22Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es1043.eqiad.wmnet with OS trixie 2424664 wikitext text/x-wiki 1rpigkrrtc0v7afcrdza8fpyl9d5mqd 2424665 2424664 2026-06-09T09:50:14Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply 2424665 wikitext text/x-wiki a5xtku237abbp8q2thawe457d2u949m 2424666 2424665 2026-06-09T09:50:32Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply 2424666 wikitext text/x-wiki p4cz6933eikpozvf2h2hbf3cwrxjf2v 2424668 2424666 2026-06-09T09:51:22Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply 2424668 wikitext text/x-wiki 4w7h9fsmpvhhynpmqi1ds7u8ar9xeo3 2424669 2424668 2026-06-09T09:51:38Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply 2424669 wikitext text/x-wiki byyz8nv9y6h11car7ieqik3xuxo21vp 2424671 2424669 2026-06-09T09:57:58Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.pool pool db1160: Repooling 2424671 wikitext text/x-wiki 4lx09b25s0fricc1qhjw0vo2w00jh50 2424672 2424671 2026-06-09T10:04:08Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2424672 wikitext text/x-wiki st0c8ivlp3qmuo0qtw9qwa6mr04zhvm 2424673 2424672 2026-06-09T10:04:12Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply 2424673 wikitext text/x-wiki fglgsn9d6insmrbnq4tvszjjy2n0bav 2424674 2424673 2026-06-09T10:04:14Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply 2424674 wikitext text/x-wiki 2137iy40qo4tt1ve4fvova0s4nnr4vb 2424675 2424674 2026-06-09T10:04:16Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2424675 wikitext text/x-wiki tp4fyjzfdz9fwqnoyszjk20fe3winyu 2424676 2424675 2026-06-09T10:04:42Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply 2424676 wikitext text/x-wiki fkm8mhr3q1dsc9wqr5o9z0g79kjto61 2424677 2424676 2026-06-09T10:04:44Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es1043.eqiad.wmnet with reason: host reimage 2424677 wikitext text/x-wiki 8fgbwjew7w6y0vyezyk986siucs55ip 2424678 2424677 2026-06-09T10:04:52Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply 2424678 wikitext text/x-wiki sldljrf0h6oeowdrofgnla9ilphenvd 2424679 2424678 2026-06-09T10:08:22Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es1043.eqiad.wmnet with reason: host reimage 2424679 wikitext text/x-wiki 2ry83d0wiosdaewts3j4jqugz7dyca5 2424681 2424679 2026-06-09T10:12:53Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2424681 wikitext text/x-wiki 719in76ui99fuztxn4pmfydlwmdi2y8 2424682 2424681 2026-06-09T10:12:55Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2424682 wikitext text/x-wiki p9ix11e87f2tlq2ix6bzbid9zurthhg 2424683 2424682 2026-06-09T10:13:07Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2424683 wikitext text/x-wiki ldvor28b6jlgqvgtp30w1646w5sh5go 2424684 2424683 2026-06-09T10:13:15Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2424684 wikitext text/x-wiki 5om9kbujjuzp3pgo8i6thks533d5f0m 2424685 2424684 2026-06-09T10:13:22Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply 2424685 wikitext text/x-wiki cj8ljn7e7fybt27fe3b08wd2hgslvxe 2424686 2424685 2026-06-09T10:13:34Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply 2424686 wikitext text/x-wiki bxndg0vrlv1sse6b79houit6j0ocdm8 2424687 2424686 2026-06-09T10:13:43Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply 2424687 wikitext text/x-wiki letut3sb3pmmh4bopkbsg8h804quxb0 2424688 2424687 2026-06-09T10:13:48Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply 2424688 wikitext text/x-wiki i1dkpfcc8a2o9c824lbxvste5dgoyq4 2424690 2424688 2026-06-09T10:16:00Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2424690 wikitext text/x-wiki 4a7707sldkn2eib1l7l2um5qwxuy0ii 2424691 2424690 2026-06-09T10:16:06Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2424691 wikitext text/x-wiki f2ocllrjb07o9o1lontawce1c65z344 2424692 2424691 2026-06-09T10:17:04Z Stashbot 7414 elukey: complete rollout of apache2 upgrades 2424692 wikitext text/x-wiki iofclqkq3j0tzauo93d8snf7bxbp75c 2424693 2424692 2026-06-09T10:17:07Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2424693 wikitext text/x-wiki 2lbe56lejrea2od5zxey4gaz58db1va 2424695 2424693 2026-06-09T10:17:20Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2424695 wikitext text/x-wiki hpsabheg5qwkrxocborfoj2uov16zff 2424697 2424695 2026-06-09T10:26:02Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es1043.eqiad.wmnet with OS trixie 2424697 wikitext text/x-wiki sfbe1rqa6mnjxejyuioihkav409mhu7 2424698 2424697 2026-06-09T10:28:19Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1160: Repooling 2424698 wikitext text/x-wiki snbo3a0ioo50rz00a5wvdmi8amzxm05 2424699 2424698 2026-06-09T10:31:41Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.major-upgrade (exit_code=99) 2424699 wikitext text/x-wiki 8b1q2m09ct6jbgjpndctf3vkn4qlxv7 2424700 2424699 2026-06-09T10:32:04Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es1043: repool after upgrade 2424700 wikitext text/x-wiki ge6tcup1k34ymrc97d7nbtk1wot6eet 2424701 2424700 2026-06-09T10:35:08Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2424701 wikitext text/x-wiki 3ilo7rcoete996u42bhq55v0wa0bmk5 2424702 2424701 2026-06-09T10:35:12Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2424702 wikitext text/x-wiki 2kn6fs8uwxfkbi1z2irhejm5ulcolnq 2424703 2424702 2026-06-09T10:35:38Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2424703 wikitext text/x-wiki 2y8r5u36drdtq4xo6f218idrk77c1w5 2424704 2424703 2026-06-09T10:35:47Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2424704 wikitext text/x-wiki qp1kqsem4gn0q75pbpcuo391ie4c3jd 2424706 2424704 2026-06-09T10:51:59Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/api-gateway: apply 2424706 wikitext text/x-wiki c9wpxdwv7em8w0vxtdjgqh6ut9bkkd4 2424707 2424706 2026-06-09T10:52:06Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/api-gateway: apply 2424707 wikitext text/x-wiki 774qgqycvuxivr2o7jsm6ou5qdbpsx3 2424708 2424707 2026-06-09T10:52:09Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/api-gateway: apply 2424708 wikitext text/x-wiki knoahzkpndnebjhyszsed87nive8b4c 2424709 2424708 2026-06-09T10:52:25Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/api-gateway: apply 2424709 wikitext text/x-wiki 5p2dxr3jshmo10gw1mp2u5gv7mu0unu 2424710 2424709 2026-06-09T10:52:39Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/api-gateway: apply 2424710 wikitext text/x-wiki i8bq8389eyhdj29aqlslgptz0nypptx 2424711 2424710 2026-06-09T10:52:46Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2424711 wikitext text/x-wiki 28d0a8u9hslco1cpbjb6lpb6oawxip2 2424712 2424711 2026-06-09T10:52:48Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/api-gateway: apply 2424712 wikitext text/x-wiki 0luuwxa3qm2ekbimfyg73rqf1x2fcad 2424713 2424712 2026-06-09T10:53:07Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es2046: Upgrading es2046.codfw.wmnet 2424713 wikitext text/x-wiki q38lccr3k138ktiek5jxrpn9mjbc5ys 2424714 2424713 2026-06-09T10:53:39Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool es2046: Upgrading es2046.codfw.wmnet 2424714 wikitext text/x-wiki g9yxnu1n10upbmqdnjcoqrly82re7kg 2424715 2424714 2026-06-09T10:55:43Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es2046.codfw.wmnet with OS trixie 2424715 wikitext text/x-wiki r689crtw1k8jdsrdiuyq6y5w84b6lc7 2424718 2424715 2026-06-09T11:12:00Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es2046.codfw.wmnet with reason: host reimage 2424718 wikitext text/x-wiki l5ehg412ifxgde8kcq2wvbhy1ekzylf 2424719 2424718 2026-06-09T11:17:29Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es1043: repool after upgrade 2424719 wikitext text/x-wiki 8sp7wqgzs72f85r2q2yv6pov462371m 2424720 2424719 2026-06-09T11:18:35Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es2046.codfw.wmnet with reason: host reimage 2424720 wikitext text/x-wiki 8px1uf8ygs9ap5z4lsp8uw74njaih1a 2424723 2424720 2026-06-09T11:24:41Z Stashbot 7414 slyngshede@dns1004: START - running authdns-update 2424723 wikitext text/x-wiki 36nj9c34yo35ak4hc6s4e3rcgyvmzlx 2424724 2424723 2026-06-09T11:26:17Z Stashbot 7414 slyngshede@dns1004: END - running authdns-update 2424724 wikitext text/x-wiki o6si95xkarsr6mzuda90w9kfzmaxjq5 2424725 2424724 2026-06-09T11:26:54Z Stashbot 7414 slyngs: CAS-SSO upgrade to version 3.7.3.2 2424725 wikitext text/x-wiki al9tlbri2qj8e3zhdlqtfi50seppbm2 2424726 2424725 2026-06-09T11:27:40Z SLyngshede-WMF 31501 2424726 wikitext text/x-wiki jhrsf7hrocs2q0hxlr9meu6g23u4phf 2424727 2424726 2026-06-09T11:28:05Z Stashbot 7414 root@cumin2002: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging KSiebert out of all services on: 2435 hosts 2424727 wikitext text/x-wiki 5ym7y89u6g31xkn2tsz500e3u3p8cus 2424729 2424727 2026-06-09T11:31:47Z Stashbot 7414 root@cumin2002: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging HMonroy out of all services on: 2435 hosts 2424729 wikitext text/x-wiki n0xm7yrsx5cqbxd6dsh75yugf6qzeg2 2424730 2424729 2026-06-09T11:35:27Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2185.codfw.wmnet with reason: Reimage 2424730 wikitext text/x-wiki m1q13prrggm4ytkz8wsbmqhpbrhcilu 2424731 2424730 2026-06-09T11:36:04Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es2046.codfw.wmnet with OS trixie 2424731 wikitext text/x-wiki h7fn9toqy06epms46eqlji086p76qrv 2424732 2424731 2026-06-09T11:36:47Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.major-upgrade 2424732 wikitext text/x-wiki isp2tq2m7y4oig01v30jiiz4eirq2pn 2424733 2424732 2026-06-09T11:38:16Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.major-upgrade (exit_code=99) 2424733 wikitext text/x-wiki 7ndbrdohsb4lu3bsf6ih6omyjijvque 2424734 2424733 2026-06-09T11:39:00Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es2046: repool after maintenance 2424734 wikitext text/x-wiki cplegqj1stjkbiwj3s5seqwdhzidr5e 2424735 2424734 2026-06-09T11:43:08Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2424735 wikitext text/x-wiki jcm0moaq95bfe7fkmtd8o883nuhngc3 2424736 2424735 2026-06-09T11:43:43Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2424736 wikitext text/x-wiki it54of6mikwuipmlyu4az5jcd9l19do 2424737 2424736 2026-06-09T11:44:23Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2424737 wikitext text/x-wiki 6l71ajqdb87jv59t5p1x04u0vdcgupr 2424738 2424737 2026-06-09T11:45:21Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2424738 wikitext text/x-wiki e4582qdw7ij66pmqglhkdr2xkgs4juk 2424739 2424738 2026-06-09T11:47:27Z Stashbot 7414 atsuko@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'apply'. 2424739 wikitext text/x-wiki egcck5tg2vft9zs35i71kdf8mbdbbcq 2424740 2424739 2026-06-09T11:48:30Z Stashbot 7414 atsuko@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. 2424740 wikitext text/x-wiki 2hq9rfl9m0smy2r6m5dc154x54yw9br 2424741 2424740 2026-06-09T11:49:09Z Stashbot 7414 atsuko@deploy1003: helmfile [staging-codfw] START helmfile.d/admin 'apply'. 2424741 wikitext text/x-wiki bgqwjpqr1vxz6gug9q655z5y9ensdo4 2424742 2424741 2026-06-09T11:51:01Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1237.eqiad.wmnet with OS trixie 2424742 wikitext text/x-wiki p4ryhcp55parxdbqrduz0zpzza8coir 2424743 2424742 2026-06-09T11:51:19Z Stashbot 7414 atsuko@deploy1003: helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. 2424743 wikitext text/x-wiki 0ej4d83d2vxl32nk1992dm990sxudv5 498 22634 2424553 2207212 2026-06-09T02:57:39Z Stashbot 7414 wmbot~tstarling@tools-bastion-15: Stopped the webservice 2424553 wikitext text/x-wiki j8sxovwwx497ecx76aqcoouo1gqv78i 0 33444 2424694 2420554 2026-06-09T10:17:08Z Əkrəm 37923 Undid revision [[Special:Diff/2420554|2420554]] by [[Special:Contributions/Cybercezaryc13|Cybercezaryc13]] ([[User talk:Cybercezaryc13|talk]]) 2424694 wikitext text/x-wiki buhy6o53jy1ol5byjtbnz9rk02s6azs 498 441911 2424696 2421309 2026-06-09T10:25:59Z Stashbot 7414 taavi: hard reboot codesearch9 2424696 wikitext text/x-wiki 9l1dtdtonz52myt7si69vbvng3247lu 0 442952 2424557 2423785 2026-06-09T05:02:37Z SSapaty (WMF) 28955 Fill in skill matrix 2424557 wikitext text/x-wiki sxeiwabfirfhkiv0dtwvk3p1zvcr6i6 498 443628 2424272 2423956 2026-06-08T12:43:18Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27138156804 (https://github.com/cluebotng/component-configs/commits/4677023bc60821948b76a89b2968d9fa3db267d4) 2424272 wikitext text/x-wiki ke11w3v2xza0hffvoyttongs9706irl 3 443682 2424680 2424006 2026-06-09T10:12:32Z Effie Mouzeli (WMF) 12880 /* Dark mode Duck */ Reply 2424680 wikitext text/x-wiki m7u11vjtyysbgathulcy8zv75edahhe 498 443947 2424267 2418798 2026-06-08T12:40:15Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed 1d78cb4fd9 (l10n updates: nl) 2424267 wikitext text/x-wiki i4pi1fj0vypihgewnb8xv6o35rs6u8x 498 445210 2424313 2423856 2026-06-08T13:24:03Z Stashbot 7414 dhinus: webservice python3.9 start (T428139) 2424313 wikitext text/x-wiki dk08nvccxzrngg5aicq4ofjh0kcb1ol 0 447008 2424470 2399145 2026-06-08T17:32:18Z Krinkle 44 2424470 wikitext text/x-wiki loglgzn4vwkfe1t48l5mycbpgednzu0 2424471 2424470 2026-06-08T17:32:31Z Krinkle 44 2424471 wikitext text/x-wiki lrmb9wigt5cvr5l9nkyj43n6jwfp143 0 447862 2424276 2424203 2026-06-08T12:45:12Z FCeratto-WMF 39104 Sanitarium upgrade 2424276 wikitext text/x-wiki depehm08qwnp85algdg9khbavp93f6d 10 448135 2424628 2350648 2026-06-09T08:51:16Z BJensen-WMF 48942 2424628 wikitext text/x-wiki iar5xhcj1nkcimpoye8g2h1vqutuw83 0 449136 2424721 2336811 2026-06-09T11:18:48Z JMeybohm (WMF) 16709 2424721 wikitext text/x-wiki cplb7t5ywastnje64807rl1q6qidksv 0 449160 2424358 2424036 2026-06-08T14:27:27Z Dexbot 30554 Bot: Updating the report 2424358 wikitext text/x-wiki 1l1pef94seyhot7ikqq9r0g2qv2r4ef 2424384 2424358 2026-06-08T15:07:51Z Jynus 38174 /* Today (2026-06-08) */ +backup1-codfw 2424384 wikitext text/x-wiki oitnu8qpd3ixmckmfai9xxh6eqcqtfs 2424392 2424384 2026-06-08T15:09:32Z Dexbot 30554 Bot: Updating the report 2424392 wikitext text/x-wiki 1l1pef94seyhot7ikqq9r0g2qv2r4ef 2424549 2424392 2026-06-09T00:01:38Z Dexbot 30554 Bot: Updating the report 2424549 wikitext text/x-wiki 3kkdyiqpwmtkrbwyj593ki6rlrcz22o 2424560 2424549 2026-06-09T05:20:19Z Dexbot 30554 Bot: Updating the report 2424560 wikitext text/x-wiki eukd4w373wi8euezaq9lmzw958i38pl 2424573 2424560 2026-06-09T06:02:26Z Dexbot 30554 Bot: Updating the report 2424573 wikitext text/x-wiki 47hzji5ytv802mkv5xe7hycm048i39f 2424642 2424573 2026-06-09T09:14:50Z Dexbot 30554 Bot: Updating the report 2424642 wikitext text/x-wiki kzl203emwj6b6g5xh9w04zye2qq87n8 0 450789 2424705 2395167 2026-06-09T10:40:04Z Lucas Werkmeister (WMDE) 10037 /* Benchmarking and load testing in production */ update for k8s (see also #wikimedia-tech Libera Chat logs today) 2424705 wikitext text/x-wiki mbbrpkls1aiyuknflkdi3x0ro8kuyut 0 451448 2424397 2423973 2026-06-08T15:21:25Z Jdforrester (WMF) 468 Specify Z61s by reference. 2424397 wikitext text/x-wiki dusdteyxt2ur6ln9145382fcg3qhf5q 0 451600 2424308 2401065 2026-06-08T13:13:29Z JMeybohm (WMF) 16709 /* Upgrading Charts */ Add CI validation for a new Kubernetes version 2424308 wikitext text/x-wiki 63pnm7ddfkz3i0bare5avnqsdv569hm 2424315 2424308 2026-06-08T13:29:28Z JMeybohm (WMF) 16709 /* Add CI validation for a new Kubernetes version */ 2424315 wikitext text/x-wiki teikm8t43btc3hun6vervon5zow55tj 2424626 2424315 2026-06-09T08:45:20Z BJensen-WMF 48942 /* Add CI validation for a new Kubernetes version */ wording 2424626 wikitext text/x-wiki h02ts5ifimfi4l4qnkse1gq4n57gos6 2424646 2424626 2026-06-09T09:22:31Z JMeybohm (WMF) 16709 /* Add CI validation for a new Kubernetes version */ 2424646 wikitext text/x-wiki kw3k6ikrd65z60votv78bfbpb47y20h 2424647 2424646 2026-06-09T09:24:11Z BJensen-WMF 48942 /* Add CI validation for a new Kubernetes version */ wording 2424647 wikitext text/x-wiki aex2p7bxsqkjd4fdq5aixge6myhjotv 498 452890 2424274 2423955 2026-06-08T12:44:01Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27138156886 (https://github.com/cluebotng/component-configs/commits/4677023bc60821948b76a89b2968d9fa3db267d4) 2424274 wikitext text/x-wiki f1tu5243kqicka44lq8fo09lsqjqxtl 116 453906 2424297 2424029 2026-06-08T12:57:37Z Gitlabaccountapprovalbot 37332 @rainmonger was approved. 2424297 wikitext text/x-wiki fkfqpcmx6omh8mysytyemnurpr3pjdw 2424433 2424297 2026-06-08T16:15:25Z Gitlabaccountapprovalbot 37332 @ahonc was approved. 2424433 wikitext text/x-wiki aiczqukaxxzehx75bveipu6ywsj2k3j 2424474 2424433 2026-06-08T17:48:13Z Gitlabaccountapprovalbot 37332 jmprax was rejected. 2424474 wikitext text/x-wiki gjj4cffpg2qn2u4gzrjlgghgkwy96c6 0 454312 2424649 2230257 2026-06-09T09:27:23Z AYounsi (WMF) 10040 /* Upgrade */ 2424649 wikitext text/x-wiki tt5gmy7gdh2bvi196io1qqnchkr1vqp 2424651 2424649 2026-06-09T09:32:17Z AYounsi (WMF) 10040 /* Provisioning */ 2424651 wikitext text/x-wiki ik1lquyda0nwoymqo0dzsz9mwdy5q71 2424652 2424651 2026-06-09T09:32:35Z AYounsi (WMF) 10040 2424652 wikitext text/x-wiki 8kjmdsm7jmnukantupwkbqcuyyrtvou 0 456310 2424538 2424009 2026-06-08T21:52:49Z BryanDavis 1604 BryanDavis moved page [[Template talk:SRE/LDAP/Groups/Request access]] to [[SRE/LDAP/Groups/Request access]] over redirect: Undo vandal move 2383603 wikitext text/x-wiki 186py06e51sjo9ddtmuu3de0godpqdu 12 457949 2424408 2419098 2026-06-08T15:58:23Z FNegri-WMF 32595 /* Backups */ add --single-transaction to prevent replication lag during backups (T428470) 2424408 wikitext text/x-wiki 0fh6m9waeh4j6aatvrk3yqrgarozb2e 498 459244 2424270 2423131 2026-06-08T12:42:46Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27138156826 (https://github.com/cluebotng/component-configs/commits/4677023bc60821948b76a89b2968d9fa3db267d4) 2424270 wikitext text/x-wiki bm4ciajndl0bkeo1sxdb3vy3rjiofx6 0 459290 2424382 2424172 2026-06-08T15:06:59Z XCollazo-WMF 31846 /* YYYY-MM-DD NEXT TUESDAY TRAIN (REPLACE THIS AFTER DEPLOY) */ 2424382 wikitext text/x-wiki 1905w5mzl9glyp8x8uo0u4bckx7uktw 2424398 2424382 2026-06-08T15:36:01Z JAllemandou (WMF) 37709 /* YYYY-MM-DD NEXT TUESDAY TRAIN (REPLACE THIS AFTER DEPLOY) */ Add pageview project allowlist change 2424398 wikitext text/x-wiki odk0mns6gwri2a3djmlg3soaw47jw6w 498 459586 2424548 2368587 2026-06-08T23:39:47Z BryanDavis 1604 /* See also */ interwiki links 2424548 wikitext text/x-wiki 942waiadtfi57ca6a76rx67d3sjo7ze 0 459920 2424654 2413331 2026-06-09T09:34:28Z Effie Mouzeli (WMF) 12880 2424654 wikitext text/x-wiki kco5d2qjl3zyas7n2bnfrrgxsadfwia 0 460052 2424454 2422027 2026-06-08T16:43:01Z MPopov (WMF) 5891 /* Approver flow */ Simplified approval flow 2424454 wikitext text/x-wiki pyosqd7vy65tv3wug50h9b72sm60lko 498 460173 2424453 2414037 2026-06-08T16:36:33Z ZI Jony 13427 /* Server Access */ 2424453 wikitext text/x-wiki t7n1zvy4u27xvb9cs4ty3wg40vb0e56 0 460178 2424279 2424005 2026-06-08T12:48:44Z Effie Mouzeli (WMF) 12880 2424279 wikitext text/x-wiki jxjs9l9hvs6njh3cyf64ae55rq6wgwq 2424283 2424279 2026-06-08T12:50:35Z Effie Mouzeli (WMF) 12880 2424283 wikitext text/x-wiki blw8uo9gghxb5y9l4qpnx04s2z3wbr9 2424298 2424283 2026-06-08T12:57:43Z Effie Mouzeli (WMF) 12880 2424298 wikitext text/x-wiki msqawq68iivbznsgnceigwv99fsz9ts 2424303 2424298 2026-06-08T13:11:50Z Effie Mouzeli (WMF) 12880 scrollbot update 2424303 wikitext text/x-wiki qas89yt85mfqkvywxsfx15ztvz4tqbj 2424552 2424303 2026-06-09T02:19:53Z RLazarus (WMF) 15215 2424552 wikitext text/x-wiki c537ku9kyh3xrf7v9q7gkvs43t48222 2424670 2424552 2026-06-09T09:52:25Z MLechvien-WMF 49197 Linking to Service Catalog page 2424670 wikitext text/x-wiki iqfh39xea2c8wqr51n5cm04lbs49eat 0 460192 2424381 2420638 2026-06-08T15:05:59Z Effie Mouzeli (WMF) 12880 scrollbot update 2424381 wikitext text/x-wiki 0n5dkybn4b105fr7zd1htadxbdyt9po 2424393 2424381 2026-06-08T15:10:00Z Effie Mouzeli (WMF) 12880 scrollbot update 2424393 wikitext text/x-wiki 70ys0v1qy82fbeiwazng8opt79bdm7y 2424406 2424393 2026-06-08T15:55:44Z Effie Mouzeli (WMF) 12880 2424406 wikitext text/x-wiki qqt3om57nnyehb6ckqt2v50995m503w 0 460197 2424362 2417188 2026-06-08T14:49:12Z Effie Mouzeli (WMF) 12880 scrollbot update 2424362 wikitext text/x-wiki qrmq8j229ww949xbrurqhe5263uwt0i 2424368 2424362 2026-06-08T14:52:18Z Effie Mouzeli (WMF) 12880 scrollbot update 2424368 wikitext text/x-wiki tkn0nx5y6efcu3lf9xlwplpqaep86f8 2424399 2424368 2026-06-08T15:40:55Z Effie Mouzeli (WMF) 12880 2424399 wikitext text/x-wiki 95pqrvntv8qrslohjw4b2lw13ao1eol 2424400 2424399 2026-06-08T15:41:50Z Effie Mouzeli (WMF) 12880 /* Infrastructure */ 2424400 wikitext text/x-wiki nfefbhoo2bmw7ij633pgwzaoz4g9tdx 2424401 2424400 2026-06-08T15:42:51Z Effie Mouzeli (WMF) 12880 /* Traffic and networking */ 2424401 wikitext text/x-wiki 97pozhjcyygfr6eh49f1u0hteb6udyo 0 460216 2424309 2418565 2026-06-08T13:15:27Z JMeybohm (WMF) 16709 2424309 wikitext text/x-wiki opvrnvcre3xpd2hi4x24yzets0sf3he 0 460232 2424469 2423990 2026-06-08T17:32:04Z MPopov (WMF) 5891 Change tense for consistency 2424469 wikitext text/x-wiki ifpm17garqvdefo0zhxlblwnxwim876 2424478 2424469 2026-06-08T18:18:05Z MPopov (WMF) 5891 Added section on organizing info 2424478 wikitext text/x-wiki f6r1ufnkef1ezpyix75sa7s4eww4tzh 0 460233 2424464 2423710 2026-06-08T17:13:24Z MPopov (WMF) 5891 2424464 wikitext text/x-wiki 0iq5sj5h846z1gsekgb9w7ileszybyo 0 460305 2424252 2026-06-08T12:15:20Z Filippo Giunchedi 2939 First draft 2424252 wikitext text/x-wiki cx13l5rensh4ylypx0bd2d0ee37xglc 498 460306 2424348 2026-06-08T14:21:57Z Stashbot 7414 taavi: reset mariadb password T428367 2424348 wikitext text/x-wiki 058kfmahzvitmf037626hsk315bup1s 0 460307 2424403 2026-06-08T15:45:18Z Effie Mouzeli (WMF) 12880 Created page with "LOrem Ipsum" 2424403 wikitext text/x-wiki 6lan3o0020y3c82dfhi2xwt58dl8kw4 2424405 2424403 2026-06-08T15:45:59Z Effie Mouzeli (WMF) 12880 2424405 wikitext text/x-wiki j6ctx2gz55qaaef3so1wdufv3q9384x 0 460308 2424425 2026-06-08T16:14:02Z Taavi-WMF 41365 Redirected page to [[Incident response/Incident severities]] 2424425 wikitext text/x-wiki 03stq2w46grvjy7qceiga9noykb7rb6 11 460309 2424539 2026-06-08T21:52:49Z BryanDavis 1604 BryanDavis moved page [[Template talk:SRE/LDAP/Groups/Request access]] to [[SRE/LDAP/Groups/Request access]] over redirect: Undo vandal move 2424539 wikitext text/x-wiki 6wucij6z21wgcei81f8xbrr40vfk4yb 0 460310 2424634 2026-06-09T09:01:08Z BWojtowicz-WMF 42268 First draft 2424634 wikitext text/x-wiki 32jz7t8w1csyyev9nxclipgi1msqppy 0 460311 2424636 2026-06-09T09:10:06Z BJensen-WMF 48942 Add point upgrades doc 2424636 wikitext text/x-wiki 4c57oiirgcmi1prdhtznixoju7ev58s 2424638 2424636 2026-06-09T09:12:11Z BJensen-WMF 48942 fix point version example 2424638 wikitext text/x-wiki 3crk32lfgvkfuftwburc9rdwcv6jmu5 2424657 2424638 2026-06-09T09:36:57Z BJensen-WMF 48942 deploying to other clusters 2424657 wikitext text/x-wiki ffjz3r9zijl6ekd8kohqe6x4ayj6a5o 2424658 2424657 2026-06-09T09:41:05Z BJensen-WMF 48942 add cluster owners pointer 2424658 wikitext text/x-wiki bhs2ykvtg7wmv7wufe7y4zhhrozbuvm 2424689 2424658 2026-06-09T10:16:00Z BJensen-WMF 48942 recommend batches 2424689 wikitext text/x-wiki rsmj12k47cx4kxq6nsn9owqt34nxir4 0 460312 2424653 2026-06-09T09:34:18Z Effie Mouzeli (WMF) 12880 Redirected page to [[External-services]] 2424653 wikitext text/x-wiki mnb0ngsj4kza7f2mhy3s3zfzw5wosa9