Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.8 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 0 586 2431229 2431182 2026-06-28T22:32:13Z Neil Shah-Quinn (WMF) 36601 Copyedit. 2431229 wikitext text/x-wiki 3pe33fwidvqamxc8z04grbmcp5h4cxg 2431230 2431229 2026-06-28T23:10:06Z Krinkle 44 2431230 wikitext text/x-wiki je5r6jakwhkmeduwu2h9e3hkl6c3tj5 2431231 2431230 2026-06-28T23:10:21Z Krinkle 44 2431231 wikitext text/x-wiki ldsu3iqv7hdhg2jp4cueobzh2nsgp5g 0 4108 2431308 2431180 2026-06-29T10:26:58Z ScheduleDeploymentBot 37566 Add [[gerrit:1306210]] to Tuesday, June 30 UTC late backport window 2431308 wikitext text/x-wiki i2y9bo4ne5wy9lf2zi9mz9f12f3y040 2431320 2431308 2026-06-29T11:05:15Z ScheduleDeploymentBot 37566 Add [[gerrit:1306221]] to Tuesday, June 30 UTC morning backport window 2431320 wikitext text/x-wiki 8xo6i5sb5tf3m21pxok2uztg5r6nhh7 0 7919 2431233 2431197 2026-06-29T02:00:24Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2431233 wikitext text/x-wiki 10utzorn47fzpag3m8nlaostpsxxrts 2431235 2431233 2026-06-29T02:07:30Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 07m 05s) 2431235 wikitext text/x-wiki 384m2wdimkbeb3zhr8nlx14vdzpubbk 2431237 2431235 2026-06-29T05:20:48Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool pc2021: pc1 hw issues 2431237 wikitext text/x-wiki svtkiffjd2vuim9ygwtovllzy74y2ga 2431238 2431237 2026-06-29T05:20:51Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2431238 wikitext text/x-wiki 1ok0mf3trq5a7m0xa64yj3l9ogvuqe2 2431239 2431238 2026-06-29T05:20:56Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) 2431239 wikitext text/x-wiki d9k0fw1e1oa1tqees405zscmpcvgrbk 2431240 2431239 2026-06-29T05:20:58Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool pc2021: pc1 hw issues 2431240 wikitext text/x-wiki e1vukk7xyguo71gnkeaip1c86ypafgb 2431241 2431240 2026-06-29T05:24:09Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on pc2021.codfw.wmnet,pc1021.eqiad.wmnet with reason: Debugging 2431241 wikitext text/x-wiki 963s72q19zt30hlcy8kzveusca1vwdo 2431242 2431241 2026-06-29T06:04:37Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool pc2021: pc1 repool 2431242 wikitext text/x-wiki eohfas4tdb3mkzfz3x0rrr8h58c2j3a 2431243 2431242 2026-06-29T06:04:40Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2431243 wikitext text/x-wiki qemitfrg7j05mphxdgw3035kyanw669 2431244 2431243 2026-06-29T06:04:51Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) 2431244 wikitext text/x-wiki s7fgv99nb4rsoy2xtpsxxz3dinm1n3w 2431245 2431244 2026-06-29T06:04:53Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool pc2021: pc1 repool 2431245 wikitext text/x-wiki avt97z6tay96u707kp5e1x706orp2af 2431247 2431245 2026-06-29T07:02:25Z Stashbot 7414 tappof: bump space for prometheus k8s-dse in eqiad 2431247 wikitext text/x-wiki hu7guyunhdlzu9m6a7iz8y7bjjyw6b3 2431248 2431247 2026-06-29T07:23:10Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.puppet.disable-merges 2431248 wikitext text/x-wiki 4930e23vbdsevl7zr4hfxq88ja85cqx 2431249 2431248 2026-06-29T07:23:13Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.puppet.disable-merges (exit_code=0) 2431249 wikitext text/x-wiki caw8nkeewee92ak1oeguyejiqv3n2oa 2431250 2431249 2026-06-29T07:23:54Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.puppet.disable-merges 2431250 wikitext text/x-wiki oxmjuqwqv9m2mvqohtl44amvsjebys1 2431251 2431250 2026-06-29T07:23:57Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.puppet.disable-merges (exit_code=0) 2431251 wikitext text/x-wiki fq99ljkdwanltfa4rsfc50z2aw9222l 2431252 2431251 2026-06-29T07:45:31Z Stashbot 7414 marostegui@cumin1003: conftool action : set/weight=100; selector: name=clouddb1026.eqiad.wmnet 2431252 wikitext text/x-wiki m6xxf08hzkjeh2wd6hodypauqesb3sy 2431255 2431252 2026-06-29T08:00:47Z Stashbot 7414 jmm@deploy1003: helmfile [staging] START helmfile.d/services/proton: apply 2431255 wikitext text/x-wiki 4y47mnub3ve23vxa0py7djuskxowdv3 2431256 2431255 2026-06-29T08:01:45Z Stashbot 7414 jmm@deploy1003: helmfile [staging] DONE helmfile.d/services/proton: apply 2431256 wikitext text/x-wiki c07g6rs1ar41djnljbdg4dbd3hwd196 2431257 2431256 2026-06-29T08:04:15Z Stashbot 7414 jmm@deploy1003: helmfile [codfw] START helmfile.d/services/proton: apply 2431257 wikitext text/x-wiki ldohspjanetz93fxtzu284cym8ugax0 2431258 2431257 2026-06-29T08:05:30Z Stashbot 7414 jmm@deploy1003: helmfile [codfw] DONE helmfile.d/services/proton: apply 2431258 wikitext text/x-wiki avkt7fvfonxqrpgeqp173qm4j3kr0tg 2431259 2431258 2026-06-29T08:06:09Z Stashbot 7414 jmm@deploy1003: helmfile [eqiad] START helmfile.d/services/proton: apply 2431259 wikitext text/x-wiki qsmh42boczh4kwuopjkstgp2p26kcbi 2431260 2431259 2026-06-29T08:07:40Z Stashbot 7414 jmm@deploy1003: helmfile [eqiad] DONE helmfile.d/services/proton: apply 2431260 wikitext text/x-wiki f6210954ggriip2xn3d94znxdyv725k 2431261 2431260 2026-06-29T08:09:35Z Stashbot 7414 moritzm: installing lcms2 security updates 2431261 wikitext text/x-wiki mt9lpabjdq1uzpye9jvdd1yx36ukz28 2431262 2431261 2026-06-29T08:25:33Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . 2431262 wikitext text/x-wiki pce1b469kt25dw9lkqhoyacjn24112e 2431263 2431262 2026-06-29T08:32:18Z Stashbot 7414 moritzm: installing libxslt bugfix updates 2431263 wikitext text/x-wiki caxwp9dthmhjzzfwta2goleqg1hjawn 2431264 2431263 2026-06-29T08:33:49Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudvirt1078.eqiad.wmnet 2431264 wikitext text/x-wiki e65gch16rx7z9tofp61b85m1vr2tkp2 2431265 2431264 2026-06-29T08:34:06Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudvirt1079.eqiad.wmnet 2431265 wikitext text/x-wiki kwbh048ghpk7coyx02lzi4zl8zjdl2z 2431266 2431265 2026-06-29T08:34:15Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudvirt1080.eqiad.wmnet 2431266 wikitext text/x-wiki oep1wf5o1dvf0pk0vlh2ru45ywdo6yp 2431267 2431266 2026-06-29T08:40:31Z Stashbot 7414 ozge@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' . 2431267 wikitext text/x-wiki qp9hgz65mecm1u46rbldiidd6qsj5b1 2431268 2431267 2026-06-29T08:43:59Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudvirt1078.eqiad.wmnet 2431268 wikitext text/x-wiki lc2s61dt3zkdqjzzipmoxv34pd5vgfz 2431269 2431268 2026-06-29T08:47:18Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudvirt1079.eqiad.wmnet 2431269 wikitext text/x-wiki gmzgne30asrw1ljc32rr752giq7a75w 2431270 2431269 2026-06-29T08:51:37Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2431270 wikitext text/x-wiki r4ib02ls5e8g47bla843ccwshepljr5 2431271 2431270 2026-06-29T08:51:40Z Stashbot 7414 cwilliams@cumin1003: dbmaint on s4@eqiad T429893 2431271 wikitext text/x-wiki 805j3kr5t05lim0lx99jkqiifl0tyzs 2431272 2431271 2026-06-29T08:51:44Z Stashbot 7414 Tran: Deployed patch for T427287 2431272 wikitext text/x-wiki kqxhec3pek41heeci1c7hykj77u2y12 2431273 2431272 2026-06-29T08:51:57Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db1249: Upgrading db1249.eqiad.wmnet 2431273 wikitext text/x-wiki 0v13ulmfofwruf2wiwiyzavv3gcxujj 2431274 2431273 2026-06-29T08:52:47Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1249: Upgrading db1249.eqiad.wmnet 2431274 wikitext text/x-wiki 3iakm5azbh2y53dp2v0k0ndy2xh3waw 2431275 2431274 2026-06-29T08:54:53Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db1249.eqiad.wmnet with OS trixie 2431275 wikitext text/x-wiki sckva4eion9twcxapch94xn1b6jrq1v 2431276 2431275 2026-06-29T08:54:59Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2431276 wikitext text/x-wiki te1dbxuj6f7t5ghz5guizu800djefdt 2431277 2431276 2026-06-29T08:55:04Z Stashbot 7414 cwilliams@cumin1003: dbmaint on s4@codfw T429893 2431277 wikitext text/x-wiki 6ocoyv7xgdblcwy0u80dvzogfnfirpp 2431278 2431277 2026-06-29T08:55:21Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db2247: Upgrading db2247.codfw.wmnet 2431278 wikitext text/x-wiki 5fg1krcflunh3pzng0us5d1h3x2q3mo 2431279 2431278 2026-06-29T08:55:43Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2247: Upgrading db2247.codfw.wmnet 2431279 wikitext text/x-wiki hx5k99kebm91kygoy0efy7fgo5a4uds 2431280 2431279 2026-06-29T08:58:51Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/pooled=no; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-wdqs2003.codfw.wmnet 2431280 wikitext text/x-wiki jvfcpa51rnbrnqbulvx37q8rfx6xb0f 2431281 2431280 2026-06-29T08:59:13Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/pooled=no; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-test-wdqs2001.codfw.wmnet 2431281 wikitext text/x-wiki ne2knydev5fo4m7sekuoa48g1x9cuae 2431282 2431281 2026-06-29T08:59:31Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudvirt1080.eqiad.wmnet 2431282 wikitext text/x-wiki 00l0qmcv9jubhpmermjnktsinfvfkoi 2431283 2431282 2026-06-29T08:59:43Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/pooled=no; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-test-wdqs2001.codfw.wmnet 2431283 wikitext text/x-wiki r2ubv71nhd6nlawtvdydx9hnt76m1vg 2431284 2431283 2026-06-29T09:00:14Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/pooled=no; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-wdqs-test2001.codfw.wmnet 2431284 wikitext text/x-wiki 067yyov52m1xuo34ushws2z2qmmg9q5 2431285 2431284 2026-06-29T09:09:27Z Stashbot 7414 ihurbain@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2431285 wikitext text/x-wiki 1yalnv0p9tnxnncybj9l4gqiucedz52 2431286 2431285 2026-06-29T09:10:03Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db2247.codfw.wmnet with OS trixie 2431286 wikitext text/x-wiki mogk18avtpyfjzb6xpr9w1ubvgkn3nb 2431287 2431286 2026-06-29T09:12:24Z Stashbot 7414 ihurbain@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2431287 wikitext text/x-wiki 84662xv304qsm9uja3jvd931y6xrfiq 2431288 2431287 2026-06-29T09:12:27Z Stashbot 7414 ihurbain@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2431288 wikitext text/x-wiki 9i9vux2khac83jcxs36f27f0f1mkgdf 2431289 2431288 2026-06-29T09:12:45Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1249.eqiad.wmnet with reason: host reimage 2431289 wikitext text/x-wiki rrbxq8kzh3f0g8bbp2egrt0guf18e93 2431290 2431289 2026-06-29T09:13:04Z Stashbot 7414 ihurbain@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2431290 wikitext text/x-wiki k44jdipas3135ml8dif4sbfsdybf1vh 2431292 2431290 2026-06-29T09:19:14Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1249.eqiad.wmnet with reason: host reimage 2431292 wikitext text/x-wiki b9qqx4ydxolqrginkz0fpcu06miamfb 2431293 2431292 2026-06-29T09:19:44Z Stashbot 7414 moritzm: installing librabbitmq security updates 2431293 wikitext text/x-wiki 14ptmaqme650oaex4d2qxxoz9ru22yx 2431294 2431293 2026-06-29T09:22:03Z Stashbot 7414 dcausse: T418494 delete apiportalwiki cirrussearch indices 2431294 wikitext text/x-wiki o0xo2bq5lnllbbu1g0a51ptm6lhvjrz 2431295 2431294 2026-06-29T09:25:42Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2247.codfw.wmnet with reason: host reimage 2431295 wikitext text/x-wiki pixma7burg2pn8ihqug4m3j09gb202n 2431296 2431295 2026-06-29T09:29:30Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2247.codfw.wmnet with reason: host reimage 2431296 wikitext text/x-wiki 76umgygb7t6f3un4xqtzskbcd98lj88 2431298 2431296 2026-06-29T09:35:56Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1249.eqiad.wmnet with OS trixie 2431298 wikitext text/x-wiki st1nwwe1t89xoxl1rkv0sd7al05r0lb 2431299 2431298 2026-06-29T09:36:39Z Stashbot 7414 marostegui: drop database apiportalwiki T418494 2431299 wikitext text/x-wiki s8o9l8hp591urvclfd7stingh3ztdtn 2431300 2431299 2026-06-29T09:39:43Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/pooled=yes; selector: service=kubesvc,cluster=dse-k8s,dc=codfw 2431300 wikitext text/x-wiki 9px0k7y36siwzices6ltue8jx7x5tdz 2431301 2431300 2026-06-29T09:46:29Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2247.codfw.wmnet with OS trixie 2431301 wikitext text/x-wiki 87duy6ds8dduvnacm019n5qls8a4kkd 2431302 2431301 2026-06-29T09:49:25Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1249: Migration of db1249.eqiad.wmnet completed 2431302 wikitext text/x-wiki gmuf75teg0tu5ta1xt4b3cgj60r037u 2431303 2431302 2026-06-29T09:54:24Z Stashbot 7414 moritzm: installing libssh2 security updates 2431303 wikitext text/x-wiki mttiydjm2olpm2xr14h6sm9mglt6v8d 2431304 2431303 2026-06-29T10:02:37Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db2247: Migration of db2247.codfw.wmnet completed 2431304 wikitext text/x-wiki ibrnys9yvo303tnqp2u52cr85fnzkf0 2431306 2431304 2026-06-29T10:17:16Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'article-models' for release 'main' . 2431306 wikitext text/x-wiki oqxjin1a5k7jekla71uiiayl62bn8tn 2431307 2431306 2026-06-29T10:23:23Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . 2431307 wikitext text/x-wiki 64psm3le4peggoiy0sag1s92d5ytuxv 2431309 2431307 2026-06-29T10:28:46Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' . 2431309 wikitext text/x-wiki gpp3nx3tr130o1fmha7emgmz0i4zrz6 2431310 2431309 2026-06-29T10:34:56Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1249: Migration of db1249.eqiad.wmnet completed 2431310 wikitext text/x-wiki 14e5m2j26c385qmd3c6tgrpa16n5jgc 2431311 2431310 2026-06-29T10:34:59Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2431311 wikitext text/x-wiki jgloa9hfirdgypsdff22o0sbssnn5zh 2431312 2431311 2026-06-29T10:35:02Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.hosts.reimage for host sretest1006.eqiad.wmnet with OS trixie 2431312 wikitext text/x-wiki ikon4aeofmhl231qota7z0qr3fwifks 2431313 2431312 2026-06-29T10:41:55Z Stashbot 7414 cmooney@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host sretest1006.eqiad.wmnet with OS trixie 2431313 wikitext text/x-wiki 5lvwk5lv4q8bkvq520hg8m725v4xukw 2431314 2431313 2026-06-29T10:47:35Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . 2431314 wikitext text/x-wiki trnokuj3sl4vdoop8volksudzf5kxbd 2431315 2431314 2026-06-29T10:48:07Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2247: Migration of db2247.codfw.wmnet completed 2431315 wikitext text/x-wiki thbx0ziephehbpxggd7k19l9ndkrgam 2431316 2431315 2026-06-29T10:48:09Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2431316 wikitext text/x-wiki 9wdpedvim220rj6wiakd67wf4u8rm7c 2431317 2431316 2026-06-29T10:48:41Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.hosts.reimage for host sretest1006.eqiad.wmnet with OS trixie 2431317 wikitext text/x-wiki 76vwllhy2cnyc6lods4r4z9jlcd4y2a 2431318 2431317 2026-06-29T10:54:42Z Stashbot 7414 cmooney@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest1006.eqiad.wmnet with OS trixie 2431318 wikitext text/x-wiki kamfixfknpmrad8jlcyro02elchylth 2431319 2431318 2026-06-29T10:55:59Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.hosts.reimage for host sretest1006.eqiad.wmnet with OS trixie 2431319 wikitext text/x-wiki 7b9ssfi71q8a3fw4wdowctxobd56i2t 2431321 2431319 2026-06-29T11:13:46Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on sretest1006.eqiad.wmnet with reason: host reimage 2431321 wikitext text/x-wiki 5b6jasne3hev00k0bnaigkoccz2d4r9 2431322 2431321 2026-06-29T11:19:51Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on sretest1006.eqiad.wmnet with reason: host reimage 2431322 wikitext text/x-wiki du07id14fciaabfxndc1178aqhg1q2w 2431323 2431322 2026-06-29T11:26:49Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'apply'. 2431323 wikitext text/x-wiki 7rahls7bd4kjk3v5hdggsmvnqwbgrzv 2431324 2431323 2026-06-29T11:27:23Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. 2431324 wikitext text/x-wiki h9e76u9717of8cubkrjt7wsihqpuihi 2431325 2431324 2026-06-29T11:27:38Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging-codfw] START helmfile.d/admin 'apply'. 2431325 wikitext text/x-wiki p6yrqb49fr12k0x7npx0dyqyb75ajju 2431326 2431325 2026-06-29T11:28:38Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. 2431326 wikitext text/x-wiki 07k689qpitshwun5dij0s4qxiyg7rvg 2431327 2431326 2026-06-29T11:29:18Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/admin 'apply'. 2431327 wikitext text/x-wiki 422usdjfgzdx1eu63mm981y8owic5x6 2431328 2431327 2026-06-29T11:30:36Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'apply'. 2431328 wikitext text/x-wiki nt1oa12j37f8uszsze8eyz695m103x9 2431329 2431328 2026-06-29T11:30:58Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/admin 'apply'. 2431329 wikitext text/x-wiki pbkgtj5rhlrj4vhs8ss134wilp32o91 2431330 2431329 2026-06-29T11:31:34Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'apply'. 2431330 wikitext text/x-wiki 7h3zdts68omeye333eshadw5ts2frjx 2431331 2431330 2026-06-29T11:32:37Z Stashbot 7414 cgoubert@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/admin 'apply'. 2431331 wikitext text/x-wiki 6fxexp2p3emc77p5ly0cjly29kqispb 2431332 2431331 2026-06-29T11:33:35Z Stashbot 7414 cgoubert@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/admin 'apply'. 2431332 wikitext text/x-wiki d3uw86us0p89hnjicuwuk6o1n7n0nvf 2431333 2431332 2026-06-29T11:34:07Z Stashbot 7414 cgoubert@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/admin 'apply'. 2431333 wikitext text/x-wiki l4pj2ofmup7hscjgevr5l8ywl8xx8p3 2431334 2431333 2026-06-29T11:36:16Z Stashbot 7414 cgoubert@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2431334 wikitext text/x-wiki ipgdw5zodvhh7t3oo90lh4du2xmk1ur 2431335 2431334 2026-06-29T11:36:45Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2431335 wikitext text/x-wiki kb61urbzglraw6s8m2ymtf121zujk57 2431336 2431335 2026-06-29T11:36:49Z Stashbot 7414 cwilliams@cumin1003: dbmaint on s4@eqiad T429893 2431336 wikitext text/x-wiki gke5b8w14l7cs9ihaz219iu6auho620 2431337 2431336 2026-06-29T11:37:00Z Stashbot 7414 cgoubert@deploy1003: helmfile [ml-serve-codfw] START helmfile.d/admin 'apply'. 2431337 wikitext text/x-wiki 8aa8uf0bmsb9ux9vihxmf4ikntfytwl 2431338 2431337 2026-06-29T11:37:05Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db1252: Upgrading db1252.eqiad.wmnet 2431338 wikitext text/x-wiki oqvyw9s4bd22duxm9hxn19dqwiafe17 2431339 2431338 2026-06-29T11:37:36Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1252: Upgrading db1252.eqiad.wmnet 2431339 wikitext text/x-wiki 28unrwor7uqrdjikq303z18madm6rw0 2431340 2431339 2026-06-29T11:38:12Z Stashbot 7414 cgoubert@deploy1003: helmfile [ml-serve-codfw] DONE helmfile.d/admin 'apply'. 2431340 wikitext text/x-wiki 5gt34uh5w9x28pi4hwecaryx4rjzm3y 2431341 2431340 2026-06-29T11:38:18Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host sretest1006.eqiad.wmnet with OS trixie 2431341 wikitext text/x-wiki ix0iw7ripij0eqi0zqtr9m6qej8da6u 2431342 2431341 2026-06-29T11:38:32Z Stashbot 7414 cgoubert@deploy1003: helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. 2431342 wikitext text/x-wiki okfuma1azi9cibqwwdc95qflsta9vbg 2431343 2431342 2026-06-29T11:39:11Z Stashbot 7414 cgoubert@deploy1003: helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. 2431343 wikitext text/x-wiki jdkt8to961wq42c1hc8uzxd6eruqppc 2431344 2431343 2026-06-29T11:39:40Z Stashbot 7414 cgoubert@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/admin 'apply'. 2431344 wikitext text/x-wiki 4lfo0vvtr9wg3f4fza7scygnyo78tbt 2431345 2431344 2026-06-29T11:40:47Z Stashbot 7414 cgoubert@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2431345 wikitext text/x-wiki pik4cr9ytxwbma56rnh1sacm0ppcvaw 2431346 2431345 2026-06-29T11:40:52Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db1252.eqiad.wmnet with OS trixie 2431346 wikitext text/x-wiki 6dip6ivtp6yxwiy4i3jk963jfwf21em 2431347 2431346 2026-06-29T11:41:12Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . 2431347 wikitext text/x-wiki ph5jaz6u9ylpr479n8l1szybwcqj1pe 2431348 2431347 2026-06-29T11:43:07Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.netbox 2431348 wikitext text/x-wiki ch2gsuymc0p0afkgru4qhbxaqpmdpy0 2431349 2431348 2026-06-29T11:46:27Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add new entries for private1-d8test-eqiad vlan - cmooney@cumin1003" 2431349 wikitext text/x-wiki 8t5llqle3u8jqn0ksyew3b6a3gtvq0e 2431350 2431349 2026-06-29T11:47:16Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add new entries for private1-d8test-eqiad vlan - cmooney@cumin1003" 2431350 wikitext text/x-wiki om8l4dewfcs7kxbh0989y3r72d5heix 2431351 2431350 2026-06-29T11:47:18Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2431351 wikitext text/x-wiki 01otxzmz9m6zdqna9lqu8h0ppcgqeev 2431352 2431351 2026-06-29T11:53:04Z Stashbot 7414 cmooney@dns3003: START - running authdns-update 2431352 wikitext text/x-wiki sxr8npdptbhe1zrreziddo6mnvbres8 2431353 2431352 2026-06-29T11:54:40Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s_services/services/datahub: apply 2431353 wikitext text/x-wiki 1bhge33ft0jpvshlzv643c1rgco3pa8 2431354 2431353 2026-06-29T11:55:06Z Stashbot 7414 cmooney@dns3003: END - running authdns-update 2431354 wikitext text/x-wiki h0enrx8x8mx24x3ldks0j1hegkap8c5 2431355 2431354 2026-06-29T11:57:03Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s_services/services/datahub: apply} 2431355 wikitext text/x-wiki op2n9rg4b82ake7a998f6nx96de8y3b 2431356 2431355 2026-06-29T11:57:19Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s_services/services/datahub-next: apply 2431356 wikitext text/x-wiki 1378bgaocv0v2ecz3phjgx070l48yxw 2431357 2431356 2026-06-29T11:57:48Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1252.eqiad.wmnet with reason: host reimage 2431357 wikitext text/x-wiki dgahjotx04ezvviclc6px57kf0zxp0d 2431358 2431357 2026-06-29T11:58:37Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s_services/services/datahub-next: apply 2431358 wikitext text/x-wiki oss5bw4avdm8qz299n449flyb3kbyrf 0 17290 2431206 2431055 2026-06-28T15:30:12Z Stashbot 7414 hashar: Updated integration/zuul-jobs from upstream (c75fe6ef19c..fc4af6d4471), notably to remove `requestsexceptions` in `upload-logs-swift` role # T430458 2431206 wikitext text/x-wiki rhotywdu5xhmjbvhrvtoxgn19ds20qf 2431297 2431206 2026-06-29T09:32:11Z Stashbot 7414 hashar: gerrit: deleted repository phabricator/extensions/BurnDownCharts , created in July 2014, had no commit/changes 2431297 wikitext text/x-wiki pphuqi3c04ntly6ri3mrjsriyqbf67d 0 221439 2431254 2407179 2026-06-29T07:48:29Z AKlapper (WMF) 421 "of late" quickly gets outdated 2431254 wikitext text/x-wiki i1ljeho805aj932q809641zfarpvyy6 0 255597 2431253 2202532 2026-06-29T07:47:44Z AKlapper (WMF) 421 typo fix 2431253 wikitext text/x-wiki pua8d3eyz493wvelayfi2j3lez3bkmd 0 449160 2431232 2431195 2026-06-29T00:00:36Z Dexbot 30554 Bot: Updating the report 2431232 wikitext text/x-wiki n7v7rb9zwpakv2ou8t4dd58863esz4h 116 453426 2431246 2418828 2026-06-29T06:34:32Z Phabbanbot 37210 LuniZunie was disabled by RhinosF1 2431246 wikitext text/x-wiki f8hiqsoaurcamfqzgvjm3c7br7zjhqd 116 453906 2431202 2431198 2026-06-28T13:03:14Z Gitlabaccountapprovalbot 37332 stepmay was rejected. 2431202 wikitext text/x-wiki 8zecq7due314dqkly7vtd2voch88agn 2431203 2431202 2026-06-28T14:03:13Z Gitlabaccountapprovalbot 37332 vaishnavikumbhar was rejected. 2431203 wikitext text/x-wiki 60pvvowu3nfi41wrjanbhid33ufu0j3 2431204 2431203 2026-06-28T14:51:13Z Gitlabaccountapprovalbot 37332 nkwenuinadine was rejected. 2431204 wikitext text/x-wiki njjiy1kakl0l0hu5rgb5vl0ob9xqpff 498 459005 2431226 2398757 2026-06-28T17:34:02Z Stashbot 7414 wmbot~soda@tools-bastion-15: soda built and uploaded a new version 2431226 wikitext text/x-wiki qac49qeiy5y7pfs9xjxuffer6adwn37 2431234 2431226 2026-06-29T02:02:49Z Stashbot 7414 wmbot~soda@tools-bastion-15: soda built and uploaded a new version 2431234 wikitext text/x-wiki 4ue1roptylbbuvbl6wsuppsyki9mv3q 0 459290 2431214 2429803 2026-06-28T16:21:13Z ABaso (WMF) 1044 Add deployer name for 2026-06-30 target deployment, link to additional patch 2431214 wikitext text/x-wiki 87h2djbp0k1y2st6mbnck59gd6fkq12 12 460354 2431236 2431138 2026-06-29T02:18:22Z SSapaty (WMF) 28955 Remove draft template 2431236 wikitext text/x-wiki 07tjdx6d97ljzkt9rp79msfo699m5fd 0 460375 2431227 2431173 2026-06-28T17:34:31Z AWesterinen-WMF 54189 2431227 wikitext text/x-wiki 1pm1s2k7dm7xc3qfzgd4whouiv5gxk3 0 460376 2431217 2429797 2026-06-28T16:49:37Z AWesterinen-WMF 54189 2431217 wikitext text/x-wiki oxzbvztn1l4nmglc8fcqsb38iaqzywl 2431218 2431217 2026-06-28T16:58:33Z AWesterinen-WMF 54189 2431218 wikitext text/x-wiki a8a65rk3qk1cpineelszjgsg24pa4ck 2431219 2431218 2026-06-28T17:14:14Z AWesterinen-WMF 54189 2431219 wikitext text/x-wiki 76qqfzg77i0kekx6h54cq8bewnc43f6 2431220 2431219 2026-06-28T17:22:31Z AWesterinen-WMF 54189 2431220 wikitext text/x-wiki jsqrp65h8k0ra2qkjl4ijgfdqd7okt3 2431221 2431220 2026-06-28T17:25:38Z AWesterinen-WMF 54189 2431221 wikitext text/x-wiki sdcedlbki9rm0muzfslpkm3l4whicg0 2431222 2431221 2026-06-28T17:26:24Z AWesterinen-WMF 54189 /* wikibase:label */ 2431222 wikitext text/x-wiki 2bep203vtke9765xktn6s8g6iu24ver 2431223 2431222 2026-06-28T17:30:11Z AWesterinen-WMF 54189 2431223 wikitext text/x-wiki lhztoeechiotrutc69ka16fh6ni85im 2431224 2431223 2026-06-28T17:31:06Z AWesterinen-WMF 54189 /* wikibase:label */ 2431224 wikitext text/x-wiki 55j5fwp2wrkdcudq4p7buldck0vxf75 2431225 2431224 2026-06-28T17:33:00Z AWesterinen-WMF 54189 /* wikibase:label */ 2431225 wikitext text/x-wiki dlqyfjfgrnc2mujnau7s2dw267lhfvf 0 460377 2431205 2431175 2026-06-28T15:14:53Z AWesterinen-WMF 54189 2431205 wikitext text/x-wiki mi7k9mh9l94s1gq9xkoh9oakinbf714 2431207 2431205 2026-06-28T15:41:26Z AWesterinen-WMF 54189 2431207 wikitext text/x-wiki ottnrlm4sugw4yn1eokbo320swtxfuu 2431208 2431207 2026-06-28T15:43:23Z AWesterinen-WMF 54189 2431208 wikitext text/x-wiki 40ndfon55d7x9f6nl2m1sndbjuyx77r 2431209 2431208 2026-06-28T15:48:21Z AWesterinen-WMF 54189 2431209 wikitext text/x-wiki m9tdkibba09gqyqwbggn39hxoeidwjm 2431210 2431209 2026-06-28T15:50:25Z AWesterinen-WMF 54189 2431210 wikitext text/x-wiki mfhykbl44r7c5iul5sid8e8f6zojhb6 2431211 2431210 2026-06-28T15:57:33Z AWesterinen-WMF 54189 2431211 wikitext text/x-wiki k104j59tn6oseh9hcgn0ps31un4fqon 2431212 2431211 2026-06-28T16:01:43Z AWesterinen-WMF 54189 2431212 wikitext text/x-wiki rh33lqxxeiv0rz4cazlymfefbm7fm4i 2431213 2431212 2026-06-28T16:02:19Z AWesterinen-WMF 54189 2431213 wikitext text/x-wiki 0z4tdut2q7f3ywm0k3wup5he38mn8wc 2431215 2431213 2026-06-28T16:37:43Z AWesterinen-WMF 54189 2431215 wikitext text/x-wiki juoe6t37iledl0nrccswsngpwdu9n5t 2431216 2431215 2026-06-28T16:38:18Z AWesterinen-WMF 54189 /* Use of Python for Pagination, Chunking and Error Handling */ 2431216 wikitext text/x-wiki gcaqz30zimn41av90o51zsgzx6pm8ce 2431228 2431216 2026-06-28T17:35:34Z AWesterinen-WMF 54189 2431228 wikitext text/x-wiki 9qihbkoge6415htafyrzqgfee978let 0 460388 2431291 2431167 2026-06-29T09:17:53Z GGoncalves-WMF 42848 /* Details */ Fix typo and link to ticket tracking one of the issues. 2431291 wikitext text/x-wiki okrf31dyut0309mr2lzheknc0hyuqia 0 460389 2431305 2026-06-29T10:05:52Z FNegri-WMF 32595 Create runbook for new alert ToolsDBHistoryLengthGrowing 2431305 wikitext text/x-wiki o05rzk276nx0n184ce9hlp20c62i7ja