Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.20 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 1380 2459743 2459004 2026-09-23T09:43:18Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase by 20 cores 2459743 wikitext text/x-wiki 1ej8dplzcjifpkwjiri39x7w0h91qtf 2459744 2459743 2026-09-23T09:43:19Z Stashbot 7414 dcaro@cloudcumin1001: END (ERROR) - Cookbook wmcs.openstack.quota_increase (exit_code=97) by 20 cores 2459744 wikitext text/x-wiki a4i04noywryjdm87c7tjps252aa4jq5 2459745 2459744 2026-09-23T09:43:32Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase by 20 cores (T438940) 2459745 wikitext text/x-wiki e3tcj1pf2jahgp5w3e8wwvnriyxtaki 2459746 2459745 2026-09-23T09:43:39Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) by 20 cores (T438940) 2459746 wikitext text/x-wiki c6l26ekz25ne72vrlhacn5nf61nlz75 498 3086 2459433 2459295 2026-09-22T16:18:02Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2459433 wikitext text/x-wiki mn7wb6myskohwbh5l1qco5mxm1szu82 2459435 2459433 2026-09-22T16:23:32Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api 2459435 wikitext text/x-wiki 25kry0g8ayjqpm8cy8lcj7bfkzlat9b 2459724 2459435 2026-09-23T08:56:39Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2459724 wikitext text/x-wiki peilb7o57hzujyo7yvd8lo2bnfjkqqx 2459726 2459724 2026-09-23T08:57:55Z Stashbot 7414 tlepage@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component logs-api 2459726 wikitext text/x-wiki oi53w8972m55c33oytix37pgpoyshsh 2459728 2459726 2026-09-23T08:59:43Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2459728 wikitext text/x-wiki nawr4atz1m4n2h05hrsizeuvoqdkoad 2459734 2459728 2026-09-23T09:11:19Z Stashbot 7414 tlepage@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-api 2459734 wikitext text/x-wiki sihol2gxe8nwmz6sg5siz8lo8ptfb8v 2459778 2459734 2026-09-23T10:10:00Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2459778 wikitext text/x-wiki n96ktzfk7gs9nx0572yhrlj75nic417 2459779 2459778 2026-09-23T10:11:06Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component logs-api 2459779 wikitext text/x-wiki r7iuch11y8t5bo2ce2cuj3nxprdvo75 2459785 2459779 2026-09-23T10:47:18Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2459785 wikitext text/x-wiki 1wfgp16g59wtwzr9pn8j85gfkfnirnt 2459786 2459785 2026-09-23T10:59:16Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-api 2459786 wikitext text/x-wiki 6wntgeoigwa5jqmgcjncjd4rj02mvyv 0 4108 2459330 2459182 2026-09-22T12:26:25Z ScheduleDeploymentBot 37566 Add [[gerrit:1343917]] to Monday, September 28 UTC morning backport window 2459330 wikitext text/x-wiki mi5kzoptjfivfy7ov4u82eotbc3g2ts 2459331 2459330 2026-09-22T12:28:10Z ScheduleDeploymentBot 37566 Add [[gerrit:1343611]] to Tuesday, September 22 UTC late backport window 2459331 wikitext text/x-wiki pjtocjf5xrui39l0hasup5xnlf1lu8z 2459340 2459331 2026-09-22T13:11:18Z MGrosse-WMF 13323 moving my deployment 2459340 wikitext text/x-wiki 5wmc79c844cgwnd572a8t1pgwnqm2g4 2459417 2459340 2026-09-22T15:47:18Z ScheduleDeploymentBot 37566 Add [[gerrit:1342285]] to Tuesday, September 22 UTC late backport window 2459417 wikitext text/x-wiki o9gy4c38bjbuxau1xpuck16grylyuef 2459441 2459417 2026-09-22T16:32:40Z ScheduleDeploymentBot 37566 Add [[gerrit:1344020]] to Tuesday, September 22 UTC late backport window 2459441 wikitext text/x-wiki cyulafr2xcg9i2u1dn3pq8083t2swum 2459525 2459441 2026-09-22T18:06:21Z TCipriani (WMF) 4895 /* Week of September 21 */ 2459525 wikitext text/x-wiki d0ap3518h26540qvclc50at6cwmy76l 2459588 2459525 2026-09-22T19:54:25Z ScheduleDeploymentBot 37566 Add [[gerrit:1344014]] to Tuesday, September 22 UTC late backport window 2459588 wikitext text/x-wiki 4n0fsqeh4j8cc2juvli5owazpwkptzs 2459591 2459588 2026-09-22T20:00:55Z WBrown (WMF) 37696 /* {{Deployment_day|date=2026-09-22}} */ remove, already done 2459591 wikitext text/x-wiki 271dgrsstxbj5s3e3vwnsql7agnow3h 2459731 2459591 2026-09-23T09:06:47Z SLyngshede-WMF 31501 /* Wednesday, September 23 */ Cancelled DC switchover 2459731 wikitext text/x-wiki awiu2qkfxewsyxl4ujcr335lbfyd2i9 2459732 2459731 2026-09-23T09:07:11Z SLyngshede-WMF 31501 /* Thursday, September 24 */ Cancelled deployment server switchover 2459732 wikitext text/x-wiki lopsxtlu4x8vccclygfz0jtt48nlox3 2459733 2459732 2026-09-23T09:10:47Z Mamouri 56893 Add config patch to Thu UTC morning backport window: enable $wgCiteDefaultRefDirAuto on fawiki (T432544) 2459733 wikitext text/x-wiki mzmehubfg2wxqdp6c7agylysh2akup2 498 6726 2459430 2459178 2026-09-22T16:07:33Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2459430 wikitext text/x-wiki nzeu8iqcntpzkh6g3sxfmar3bysuqsz 2459431 2459430 2026-09-22T16:12:11Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api 2459431 wikitext text/x-wiki nvcsm6g58al3k5nl9vbjix3ihpmrwlo 2459437 2459431 2026-09-22T16:26:10Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2459437 wikitext text/x-wiki q1ane9wfyy2i03sfhyantrg67bd2rt6 2459440 2459437 2026-09-22T16:31:21Z Stashbot 7414 tlepage@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component logs-api 2459440 wikitext text/x-wiki n9d9cdbi7lofunxeqj4gs36s185pkn5 2459445 2459440 2026-09-22T16:39:31Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2459445 wikitext text/x-wiki 34erk8wuiorg6353y0yuut6w9edjgbr 2459450 2459445 2026-09-22T16:44:41Z Stashbot 7414 tlepage@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component logs-api 2459450 wikitext text/x-wiki 524hndak59ldmaqmobn395boqukyrba 2459453 2459450 2026-09-22T16:47:40Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2459453 wikitext text/x-wiki g8o7jzo4vft5r0mtcjbocys71vmrr7o 2459457 2459453 2026-09-22T16:52:51Z Stashbot 7414 tlepage@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component logs-api 2459457 wikitext text/x-wiki et1x85oz7txwejwz6jku1h2auan7m7b 2459467 2459457 2026-09-22T17:04:17Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2459467 wikitext text/x-wiki geoxm018c4hosi2ljuan0wgleeggjp5 2459470 2459467 2026-09-22T17:09:27Z Stashbot 7414 tlepage@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component logs-api 2459470 wikitext text/x-wiki ne3111wa2eqix7e5k8ga2iwu4qn2q4m 2459475 2459470 2026-09-22T17:12:32Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2459475 wikitext text/x-wiki jr0oc7vgasndhbrz3oc3gu289fxnblu 2459480 2459475 2026-09-22T17:17:43Z Stashbot 7414 tlepage@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component logs-api 2459480 wikitext text/x-wiki l2z6n9u3dt8ynhhwsqholunq046avdd 2459708 2459480 2026-09-23T08:26:42Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2459708 wikitext text/x-wiki hvqqmeri231htklx8m3eccnv9ibaj97 2459711 2459708 2026-09-23T08:33:57Z Stashbot 7414 tlepage@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component logs-api 2459711 wikitext text/x-wiki iwf6jcv7nwgl6sq9cm9m36ypco06oit 2459712 2459711 2026-09-23T08:34:32Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2459712 wikitext text/x-wiki lrj24xl6vu5two59thgvjt90g62a5cs 2459718 2459712 2026-09-23T08:44:29Z Stashbot 7414 tlepage@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component logs-api 2459718 wikitext text/x-wiki apdnsfx3kwvxspwnths8byc3wx24huo 2459720 2459718 2026-09-23T08:45:45Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2459720 wikitext text/x-wiki m0bee2imnm88n2m9snc1b6dumfqccii 2459723 2459720 2026-09-23T08:56:18Z Stashbot 7414 tlepage@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-api 2459723 wikitext text/x-wiki 7rqgcax7sr58dezij0cmuvi9p3bxl3b 2459742 2459723 2026-09-23T09:40:56Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2459742 wikitext text/x-wiki 37i49cjnmpsl6wl8rshsobi0ixzf0d3 2459758 2459742 2026-09-23T09:53:20Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-api 2459758 wikitext text/x-wiki mwah2u7yx8q0fada0vet3msup00kq56 828 6996 2459557 2396444 2026-09-22T19:09:00Z ADancy (WMF) 17694 Update SpiderPig backport operation URL 2459557 Scribunto text/plain 93mb3u1w46ue7g682j7wdcqxsud858c 0 7919 2459321 2459320 2026-09-22T12:06:48Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1343953|AbuseReview: Hide recently saved revisions from the vandalism queue (T438235)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2459321 wikitext text/x-wiki haax3xmfhbjeqyma8wjvw5i2vf9rugg 2459322 2459321 2026-09-22T12:08:03Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2459322 wikitext text/x-wiki neu1jemmcat4f9z3ds0j0filluujylq 2459323 2459322 2026-09-22T12:15:30Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2459323 wikitext text/x-wiki 90syrhwfb3d0ntg7e09vto2nbwowbmt 2459324 2459323 2026-09-22T12:16:06Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2459324 wikitext text/x-wiki 2fjou2cokufppn6pqex6uutnrmbucg6 2459325 2459324 2026-09-22T12:16:51Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2459325 wikitext text/x-wiki mzwp48gwljz9kpjrxuf5v12feqyzvs1 2459326 2459325 2026-09-22T12:17:47Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2459326 wikitext text/x-wiki 8lmxqkmrb2vuqqhy95vvklq1dwe6gl0 2459327 2459326 2026-09-22T12:19:35Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1343953|AbuseReview: Hide recently saved revisions from the vandalism queue (T438235)]] (duration: 33m 01s) 2459327 wikitext text/x-wiki 37u9v9rznt6423hb46zno3jar8kay0g 2459328 2459327 2026-09-22T12:22:53Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1001.eqiad.wmnet 2459328 wikitext text/x-wiki ee9teq9tatbmhgbpenbmhdklndwcbj7 2459333 2459328 2026-09-22T12:31:05Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1001.eqiad.wmnet 2459333 wikitext text/x-wiki njiz8jxuf78f9aotbu8znezgfyafqwj 2459334 2459333 2026-09-22T12:31:06Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1001.eqiad.wmnet 2459334 wikitext text/x-wiki 4sqdkvdr9driy7in7qjbb5dzn2c6g58 2459335 2459334 2026-09-22T12:31:07Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on P{dse-k8s-worker1001.eqiad.wmnet} and (A:dse-k8s-master-eqiad or A:dse-k8s-worker-eqiad) 2459335 wikitext text/x-wiki 6yoxxv7ykr4nq142kg5969a8la6ckij 2459336 2459335 2026-09-22T12:43:36Z Stashbot 7414 marostegui@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2250.codfw.wmnet with reason: preparations 2459336 wikitext text/x-wiki e4ry5prbm2wd8vthv6sm44cmh2faa3q 2459337 2459336 2026-09-22T12:44:03Z Stashbot 7414 marostegui: Stop mariadb on db2250:s5 T437411 T437279 2459337 wikitext text/x-wiki k9l9cwgonk7p3ur7ktgurindkbt21zj 2459338 2459337 2026-09-22T12:48:54Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reboot-single for host dse-k8s-ctrl1001.eqiad.wmnet 2459338 wikitext text/x-wiki 88466oc9tekuithly8kyxldwgm5q1ja 2459339 2459338 2026-09-22T12:53:55Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-ctrl1001.eqiad.wmnet 2459339 wikitext text/x-wiki e4trpjjby4a0h0ibtyow95ir292m1tm 2459341 2459339 2026-09-22T13:21:19Z Stashbot 7414 tappof@deploy1003: helmfile [eqiad] START helmfile.d/admin 'sync'. 2459341 wikitext text/x-wiki me3rafbci35g4pxn61wmjztkxdip6ko 2459342 2459341 2026-09-22T13:21:33Z Stashbot 7414 tappof@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'sync'. 2459342 wikitext text/x-wiki 7luoubdykpt2ynsdr8dqgtb87nna63h 2459343 2459342 2026-09-22T13:22:47Z Stashbot 7414 tappof@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'. 2459343 wikitext text/x-wiki hm5mdc4q7ynkydfpb1dyxdcivvu6r8o 2459344 2459343 2026-09-22T13:23:23Z Stashbot 7414 tappof@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'. 2459344 wikitext text/x-wiki mwyx1w612diqsi1zieeg56qo4ir40je 2459345 2459344 2026-09-22T13:51:42Z Stashbot 7414 elukey@cumin1004: START - Cookbook sre.hosts.powercycle for host pki1002 2459345 wikitext text/x-wiki ffd13e2nttpcrqpiuxhlp0c6e9zewb8 2459346 2459345 2026-09-22T13:53:01Z Stashbot 7414 elukey@cumin1004: END (PASS) - Cookbook sre.hosts.powercycle (exit_code=0) for host pki1002 2459346 wikitext text/x-wiki 93ezfpbcgjwcbtay9q2tsfg7ts0op51 2459347 2459346 2026-09-22T13:55:58Z Stashbot 7414 tappof@deploy1003: helmfile [eqiad] START helmfile.d/admin 'sync'. 2459347 wikitext text/x-wiki 13gp7m61woqy6f4uyaen4wp9ta1ifn1 2459348 2459347 2026-09-22T13:56:14Z Stashbot 7414 tappof@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'sync'. 2459348 wikitext text/x-wiki gqec1mmqdjmb0rjza47w4rg2e6fier0 2459349 2459348 2026-09-22T13:56:33Z Stashbot 7414 tappof@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'. 2459349 wikitext text/x-wiki q5u8qlbyve33m3z583yifiu7fb4r41z 2459350 2459349 2026-09-22T13:57:07Z Stashbot 7414 tappof@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'. 2459350 wikitext text/x-wiki lxkzpl932nt39eve553e39cakf2482m 2459351 2459350 2026-09-22T13:57:52Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.reboot-nodes rolling reboot on P{dse-k8s-worker10[02-28].eqiad.wmnet} and (A:dse-k8s-master-eqiad or A:dse-k8s-worker-eqiad) 2459351 wikitext text/x-wiki crm6dfjocgn1jw61i6g7jny9mkndlqu 2459352 2459351 2026-09-22T13:57:55Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1002.eqiad.wmnet 2459352 wikitext text/x-wiki hsah2s4grac5k91itxq58spi0cvieyo 2459353 2459352 2026-09-22T13:59:57Z Stashbot 7414 btullis@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host dse-k8s-worker1002.eqiad.wmnet 2459353 wikitext text/x-wiki 5f3usgiafj81akb5l6xz4mppinehlbh 2459356 2459353 2026-09-22T14:15:09Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1002.eqiad.wmnet 2459356 wikitext text/x-wiki 0ywbubw4ovwb2w206hobp74sq5zv7w6 2459357 2459356 2026-09-22T14:15:10Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1002.eqiad.wmnet 2459357 wikitext text/x-wiki otm54lg9aw53o1jc39rw6x4eg3zlpk5 2459358 2459357 2026-09-22T14:15:18Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1003.eqiad.wmnet 2459358 wikitext text/x-wiki pjk5ccnnrd4d9rpj2hs1a0wi9jc44gq 2459359 2459358 2026-09-22T14:17:20Z Stashbot 7414 btullis@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host dse-k8s-worker1003.eqiad.wmnet 2459359 wikitext text/x-wiki 6e1bh6pd7mmypnkyuzalsfnnqr16ubb 2459360 2459359 2026-09-22T14:25:19Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1003.eqiad.wmnet 2459360 wikitext text/x-wiki qx15hnihzjr2f9eqrra4esko0uj9k4x 2459361 2459360 2026-09-22T14:25:21Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1003.eqiad.wmnet 2459361 wikitext text/x-wiki kt3woswjhz4cwsqww3u9jj32tjxbwa8 2459362 2459361 2026-09-22T14:25:26Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1004.eqiad.wmnet 2459362 wikitext text/x-wiki oslnywtz6gysxfvhsmy83pt7kmalry2 2459364 2459362 2026-09-22T14:35:45Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.rename from sretest2013 to cp2059 2459364 wikitext text/x-wiki siuj0myqqtz73n9a949y07a23fzvrtt 2459365 2459364 2026-09-22T14:36:21Z Stashbot 7414 cdobbins@cumin1004: END (FAIL) - Cookbook sre.hosts.rename (exit_code=93) from sretest2013 to cp2059 2459365 wikitext text/x-wiki ni4qdsfmwyx4cv6zjuvbu9o7dll6og7 2459368 2459365 2026-09-22T14:42:34Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.rename from sretest2013 to cp2059 2459368 wikitext text/x-wiki 75ej1zkrjszmr447l0q5ff5xyyh8ygs 2459369 2459368 2026-09-22T14:42:58Z Stashbot 7414 cdobbins@cumin1004: END (FAIL) - Cookbook sre.hosts.rename (exit_code=93) from sretest2013 to cp2059 2459369 wikitext text/x-wiki 0beitqvc6cv0u4heqnhue0l6l117nqp 2459370 2459369 2026-09-22T14:47:16Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.move-vlan for host cirrussearch1120 2459370 wikitext text/x-wiki 9jtn3s1978jpb5rj3bpbevbjcj5y7px 2459371 2459370 2026-09-22T14:47:17Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host cirrussearch1120 2459371 wikitext text/x-wiki eq0x06ulrvn0t3hcpjbdu9igen7bfdq 2459372 2459371 2026-09-22T14:51:43Z Stashbot 7414 bking@cumin2003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on cirrussearch1120.eqiad.wmnet with reason: migrate VLAN T436571 2459372 wikitext text/x-wiki 5j8o4g4aamtsyf2mnrmsz8wzgfj5zqa 2459373 2459372 2026-09-22T14:53:58Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.move-vlan for host cirrussearch1120 2459373 wikitext text/x-wiki bqkv36qwqnexloom3vjf15rmbyka5ue 2459374 2459373 2026-09-22T14:54:07Z Stashbot 7414 slyngshede@cumin1004: START - Cookbook sre.dns.admin DNS admin: depool eqiad [reason: no reason specified, no task ID specified] 2459374 wikitext text/x-wiki app2ym9w6vfp2ai5xa6vdrgyfcl1y9h 2459375 2459374 2026-09-22T14:54:13Z Stashbot 7414 slyngshede@cumin1004: END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: depool eqiad [reason: no reason specified, no task ID specified] 2459375 wikitext text/x-wiki 9mma5j4gcw20816gdgkoomuhh8uyr0r 2459376 2459375 2026-09-22T14:54:26Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.290.0" for 155 host(s) 2459376 wikitext text/x-wiki lnhrzffi2ozwhiuhtqxld8pbs255o6c 2459377 2459376 2026-09-22T14:55:29Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1004.eqiad.wmnet 2459377 wikitext text/x-wiki ej0cf00c1ftog8ntpo6f5hu8khxy0cr 2459378 2459377 2026-09-22T14:55:33Z Stashbot 7414 slyngshede@cumin1004: START - Cookbook sre.discovery.datacenter depool all services in eqiad: Datacenter services switchover - T435443 2459378 wikitext text/x-wiki a6wc2mbthc9v9p0knrrfktzg9augr4j 2459379 2459378 2026-09-22T14:59:29Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.290.0" for 3 host(s) 2459379 wikitext text/x-wiki ahdst6gj5pyci1paz8an02m57quxag5 2459381 2459379 2026-09-22T15:01:18Z Stashbot 7414 dancy@deploy1003: Installation of scap version "4.290.0" completed for 3 hosts 2459381 wikitext text/x-wiki eao88rh7hw8izcxgbpghylh5jls41hb 2459382 2459381 2026-09-22T15:03:40Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1004.eqiad.wmnet 2459382 wikitext text/x-wiki 7lli8nceb7behvb65hbrg5p37kkb1o6 2459383 2459382 2026-09-22T15:03:44Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1004.eqiad.wmnet 2459383 wikitext text/x-wiki 6phu36emu8srcosp7xmn6yubgy86owr 2459384 2459383 2026-09-22T15:03:47Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1005.eqiad.wmnet 2459384 wikitext text/x-wiki nc79k4r6r9z138gug9j8fbyxm1hnq2i 2459385 2459384 2026-09-22T15:04:18Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1005.eqiad.wmnet 2459385 wikitext text/x-wiki 1wmwpfhbdpxvqoy55iiwb4zwy34ee33 2459388 2459385 2026-09-22T15:11:16Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1005.eqiad.wmnet 2459388 wikitext text/x-wiki 4rj87g7pevn54pwzx2j3qf03k3vcagd 2459389 2459388 2026-09-22T15:11:20Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1005.eqiad.wmnet 2459389 wikitext text/x-wiki bo9ia5tm4r4vzn12zptzos61okna452 2459390 2459389 2026-09-22T15:11:23Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1006.eqiad.wmnet 2459390 wikitext text/x-wiki jbnq3gv7mhjyzdvost551a76pj32rtz 2459391 2459390 2026-09-22T15:11:54Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1006.eqiad.wmnet 2459391 wikitext text/x-wiki q7is5yi78gun3z4kmbpv5txhgqwukb7 2459392 2459391 2026-09-22T15:14:21Z Stashbot 7414 bking@cumin2003: END (FAIL) - Cookbook sre.hosts.move-vlan (exit_code=99) for host cirrussearch1120 2459392 wikitext text/x-wiki j5coc6uq9qzhb9577s4fj0nbedr1key 2459393 2459392 2026-09-22T15:16:28Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.network.configure-switch-interfaces for host cirrussearch1120 2459393 wikitext text/x-wiki 6xakhc3tqntob271s4oij8il1cu1ums 2459394 2459393 2026-09-22T15:16:41Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cirrussearch1120 2459394 wikitext text/x-wiki dymmz1f4vynvyhf8uq0enf6id2f1zlz 2459396 2459394 2026-09-22T15:18:57Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1006.eqiad.wmnet 2459396 wikitext text/x-wiki 1t2xostthmuy7texlmsqefcmooj1jnb 2459397 2459396 2026-09-22T15:19:00Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1006.eqiad.wmnet 2459397 wikitext text/x-wiki crac66xvbhfsbb28zbbyrzz9qtuairn 2459398 2459397 2026-09-22T15:19:04Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1007.eqiad.wmnet 2459398 wikitext text/x-wiki bum87vt0yu2b8y2vbs85z9qb61p0z7t 2459400 2459398 2026-09-22T15:22:22Z Stashbot 7414 slyngshede@cumin1004: END (PASS) - Cookbook sre.discovery.datacenter (exit_code=0) depool all services in eqiad: Datacenter services switchover - T435443 2459400 wikitext text/x-wiki 9om3s0x8rcgtm3ehd0ugpi3jc3a2hro 2459403 2459400 2026-09-22T15:27:17Z Stashbot 7414 ayounsi@cumin1004: START - Cookbook sre.dns.netbox 2459403 wikitext text/x-wiki r0dg3d6rnael7mb6wtimgzkn3fruu7v 2459407 2459403 2026-09-22T15:35:03Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.discovery.service-route check mw-web-ro: maintenance 2459407 wikitext text/x-wiki e1s70i74mpcbflhrquehmdlilj6bu3t 2459408 2459407 2026-09-22T15:35:06Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) check mw-web-ro: maintenance 2459408 wikitext text/x-wiki jhowqa77voblpcmsye8xv4rfbyo500n 2459409 2459408 2026-09-22T15:35:34Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.discovery.service-route pool mw-web-ro in eqiad: maintenance 2459409 wikitext text/x-wiki hjs204ev84ikdglf4a327lxejtd4a7h 2459411 2459409 2026-09-22T15:36:22Z Stashbot 7414 ayounsi@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cirrussearch1120 move vlan - ayounsi@cumin1004" 2459411 wikitext text/x-wiki 7j4x2cliemed14x8u2eht7zqomg95id 2459412 2459411 2026-09-22T15:36:25Z Stashbot 7414 ayounsi@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cirrussearch1120 move vlan - ayounsi@cumin1004" 2459412 wikitext text/x-wiki 9c4c5e11rmu8urzsvcbw12ki0w1vnft 2459413 2459412 2026-09-22T15:36:28Z Stashbot 7414 ayounsi@cumin1004: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2459413 wikitext text/x-wiki 8ikucv27h58g93r1t1aifgq5lgmc78q 2459414 2459413 2026-09-22T15:40:39Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) pool mw-web-ro in eqiad: maintenance 2459414 wikitext text/x-wiki my1pao6l05q7dlspiqjgt1p8bewa5yy 2459419 2459414 2026-09-22T15:49:06Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1007.eqiad.wmnet 2459419 wikitext text/x-wiki n1ridw1utya4hbve38vd59ygbli8ykl 2459420 2459419 2026-09-22T15:50:35Z Stashbot 7414 moritzm: installing libhtml-parser-perl security updates 2459420 wikitext text/x-wiki r7m2teepf3d8rvamvvgg8373d5hc7pz 2459422 2459420 2026-09-22T15:55:51Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1007.eqiad.wmnet 2459422 wikitext text/x-wiki 3c52wlcrwi300se069eoi8kkc304jm8 2459423 2459422 2026-09-22T15:55:54Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1007.eqiad.wmnet 2459423 wikitext text/x-wiki l82wjkg1o9kgesuovskfjzitiugonyp 2459424 2459423 2026-09-22T15:55:57Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1008.eqiad.wmnet 2459424 wikitext text/x-wiki lqxgqofu8hajyn3co8ji8nse1x4xyhe 2459425 2459424 2026-09-22T15:57:07Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.mysql.depool depool db2218: optimizer issues 2459425 wikitext text/x-wiki rwg08qxhvfmv9019rwzz0igxt3ypy7h 2459426 2459425 2026-09-22T15:58:13Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2218: optimizer issues 2459426 wikitext text/x-wiki lod5qw8lljhswa23z6ltywkjcfbb23a 2459428 2459426 2026-09-22T16:04:21Z Stashbot 7414 sukhe@cumin1004: START - Cookbook sre.hosts.rename from sretest2013 to cp2059 2459428 wikitext text/x-wiki lh26s4z3lyagi9hycg4gl1sjfil801x 2459429 2459428 2026-09-22T16:04:46Z Stashbot 7414 sukhe@cumin1004: END (FAIL) - Cookbook sre.hosts.rename (exit_code=93) from sretest2013 to cp2059 2459429 wikitext text/x-wiki gg8tvk3wjp36fv1mogb1yxcewnzsgn0 2459432 2459429 2026-09-22T16:13:46Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.discovery.service-route pool 4 services in eqiad: maintenance 2459432 wikitext text/x-wiki d9do300vhmib7d894mpt5rsk6fq8xa8 2459434 2459432 2026-09-22T16:19:03Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) pool 4 services in eqiad: maintenance 2459434 wikitext text/x-wiki 9u5y6zabzax2c9o9ssu5kmrazd0sco7 2459436 2459434 2026-09-22T16:26:00Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1008.eqiad.wmnet 2459436 wikitext text/x-wiki p3lu7hzkpykfesp8qf1ayhc7g07o257 2459438 2459436 2026-09-22T16:26:16Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.rename from sretest2013 to cp2059 2459438 wikitext text/x-wiki guf9pmzfecbmtiim00jxxlqxvra5w7u 2459439 2459438 2026-09-22T16:26:42Z Stashbot 7414 cdobbins@cumin1004: END (FAIL) - Cookbook sre.hosts.rename (exit_code=93) from sretest2013 to cp2059 2459439 wikitext text/x-wiki tof2wbntrrqsb1bfolxxkar7uhlbiyv 2459442 2459439 2026-09-22T16:33:07Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1008.eqiad.wmnet 2459442 wikitext text/x-wiki e9zlv20dxa0lrm5ikwuq9pctb99hgjc 2459443 2459442 2026-09-22T16:33:10Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1008.eqiad.wmnet 2459443 wikitext text/x-wiki e0e9dednppuy4oynimrgc3n5ymdksuj 2459444 2459443 2026-09-22T16:33:14Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1009.eqiad.wmnet 2459444 wikitext text/x-wiki 6tdila5mjsa5ym4zb0z7l2j5sbpvp8o 2459447 2459444 2026-09-22T16:41:43Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.rename from sretest2013 to cp2059 2459447 wikitext text/x-wiki 72sxe5oq4cw8oztlnxls1id398s6gx2 2459448 2459447 2026-09-22T16:42:07Z Stashbot 7414 cdobbins@cumin1004: END (FAIL) - Cookbook sre.hosts.rename (exit_code=93) from sretest2013 to cp2059 2459448 wikitext text/x-wiki bijglir75zhlq8qcd76qdpapdn72llv 2459449 2459448 2026-09-22T16:42:43Z Stashbot 7414 marostegui@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2218.codfw.wmnet with reason: fixing 2459449 wikitext text/x-wiki 4i5iavekbd7lz3zuqi2bvggxsxro0e4 2459451 2459449 2026-09-22T16:45:58Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1344020|Enable AbuseReview on jawiki for likely PII (T438867)]] 2459451 wikitext text/x-wiki h76irt4ewqpw9u9vd8qakhdhi9okpyu 2459452 2459451 2026-09-22T16:46:29Z Stashbot 7414 marostegui@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2218.codfw.wmnet with reason: fixing 2459452 wikitext text/x-wiki 36n5p44e22j41ii0dw3ahjpix8iijxs 2459454 2459452 2026-09-22T16:48:52Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.discovery.service-route pool 2 services in eqiad: maintenance 2459454 wikitext text/x-wiki co6ysevpawjb7rpg3gsewr6n3rqmrzp 2459455 2459454 2026-09-22T16:50:23Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1344020|Enable AbuseReview on jawiki for likely PII (T438867)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2459455 wikitext text/x-wiki 0felab0stkdhfpyjvqf8dljaezwvei0 2459456 2459455 2026-09-22T16:51:52Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2459456 wikitext text/x-wiki 03w8tl8ngs8f1pmh8tpopm0fdjvphxu 2459458 2459456 2026-09-22T16:54:08Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.discovery.service-route (exit_code=99) pool 2 services in eqiad: maintenance 2459458 wikitext text/x-wiki 59txkjfj0jny9wcc4gibudaxdpfcqc9 2459461 2459458 2026-09-22T16:59:11Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344020|Enable AbuseReview on jawiki for likely PII (T438867)]] (duration: 13m 13s) 2459461 wikitext text/x-wiki 2zsal1og3el0ly09w4fnepnjrw2ln7o 2459462 2459461 2026-09-22T17:00:55Z Stashbot 7414 sukhe@cumin1004: START - Cookbook sre.hosts.rename from sretest2013 to cp2059 2459462 wikitext text/x-wiki irbcxoygs1j3fp9mt7k4kswc3s4qtap 2459463 2459462 2026-09-22T17:01:20Z Stashbot 7414 sukhe@cumin1004: END (FAIL) - Cookbook sre.hosts.rename (exit_code=93) from sretest2013 to cp2059 2459463 wikitext text/x-wiki jp3l7oon1gttce5qkl2k6bzkp776knt 2459464 2459463 2026-09-22T17:01:46Z Stashbot 7414 sukhe@cumin1004: START - Cookbook sre.hosts.rename from sretest2013 to cp2059.codfw.wmnet 2459464 wikitext text/x-wiki dp0lyu7jatcem3m7q1qn5sgospgyqnk 2459465 2459464 2026-09-22T17:02:06Z Stashbot 7414 sukhe@cumin1004: END (FAIL) - Cookbook sre.hosts.rename (exit_code=93) from sretest2013 to cp2059.codfw.wmnet 2459465 wikitext text/x-wiki jw5wcwvn8pbo90j3gqovfnujjz53sem 2459466 2459465 2026-09-22T17:03:18Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1009.eqiad.wmnet 2459466 wikitext text/x-wiki n2ps5jxcr0von29p24tfx30o8f81dex 2459468 2459466 2026-09-22T17:06:07Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.mysql.pool pool db2218: Optimizer issues fixed 2459468 wikitext text/x-wiki lm65rhenvh2hqm34zdnxkh5uybkyma5 2459471 2459468 2026-09-22T17:09:34Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1009.eqiad.wmnet 2459471 wikitext text/x-wiki fqr7lecmpxyxqq2nbtti46wussrd1g1 2459472 2459471 2026-09-22T17:09:36Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1009.eqiad.wmnet 2459472 wikitext text/x-wiki o3spjigo30ikytzxvphnjji1abc81l2 2459473 2459472 2026-09-22T17:09:40Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1010.eqiad.wmnet 2459473 wikitext text/x-wiki ooaw89ivr5o7p25ahl9g1oo7bmm76bx 2459474 2459473 2026-09-22T17:10:15Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1010.eqiad.wmnet 2459474 wikitext text/x-wiki crqh97f7ihubytcj3m69lydixqw2jg9 2459476 2459474 2026-09-22T17:15:23Z Stashbot 7414 oblivian@puppetserver1001: conftool action : set/pooled=false; selector: dnsdisc=rest-gateway,name=codfw 2459476 wikitext text/x-wiki ixh1n6ptpebt9jf5xiljw7osx95yhge 2459477 2459476 2026-09-22T17:16:58Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1010.eqiad.wmnet 2459477 wikitext text/x-wiki 6mhijqs4tm5shhvc1q5wz9inrxjqe4a 2459478 2459477 2026-09-22T17:17:00Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1010.eqiad.wmnet 2459478 wikitext text/x-wiki h9awmpns67y5scdr107b0az3erryo9s 2459479 2459478 2026-09-22T17:17:04Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1011.eqiad.wmnet 2459479 wikitext text/x-wiki srrppqd2enpfiex5q0v02u7n73ajfro 2459481 2459479 2026-09-22T17:17:45Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1011.eqiad.wmnet 2459481 wikitext text/x-wiki miqd2nfl8x8kq6qneioz1sy7fgpfa0d 2459482 2459481 2026-09-22T17:22:37Z Stashbot 7414 dzahn@dns1004: START - running authdns-update 2459482 wikitext text/x-wiki 227fblaq1pr7hs5x1ogr12g8y7a5m1o 2459483 2459482 2026-09-22T17:24:11Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1011.eqiad.wmnet 2459483 wikitext text/x-wiki 67jm7bptoktz5cib6kkayfzro47k0l7 2459484 2459483 2026-09-22T17:24:12Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1011.eqiad.wmnet 2459484 wikitext text/x-wiki ovc6gxiaqau5nhavs76h1xqvz3w5x69 2459485 2459484 2026-09-22T17:24:18Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1012.eqiad.wmnet 2459485 wikitext text/x-wiki 8pnzeavei942pri9qkr6w6oguuwoc8o 2459486 2459485 2026-09-22T17:25:00Z Stashbot 7414 dzahn@dns1004: END - running authdns-update 2459486 wikitext text/x-wiki q850yjq57e1xhs9ghfenms8o6nvn16x 2459487 2459486 2026-09-22T17:25:01Z Stashbot 7414 sukhe@cumin1004: START - Cookbook sre.hosts.rename from sretest2013 to cp2059 2459487 wikitext text/x-wiki qpjxgwtneg3mhhp3bjxjgxtllo5ylea 2459488 2459487 2026-09-22T17:25:25Z Stashbot 7414 sukhe@cumin1004: END (FAIL) - Cookbook sre.hosts.rename (exit_code=93) from sretest2013 to cp2059 2459488 wikitext text/x-wiki f0wzdcnbvvx3r8rqfbtzg03e79gb8ax 2459489 2459488 2026-09-22T17:26:20Z Stashbot 7414 btullis@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host dse-k8s-worker1012.eqiad.wmnet 2459489 wikitext text/x-wiki k2qwu968guw6wuaq6ufezxixr7hc5cc 2459490 2459489 2026-09-22T17:31:29Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.move-vlan for host wdqs1029 2459490 wikitext text/x-wiki dghsjtqd91nwjvb5pbv7qf1syggioy5 2459491 2459490 2026-09-22T17:31:46Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.dns.netbox 2459491 wikitext text/x-wiki b7ioz6u2t18xn5wjibr6yl3aww65ktr 2459492 2459491 2026-09-22T17:32:51Z Stashbot 7414 sukhe@cumin1004: START - Cookbook sre.hosts.rename from sretest2013 to cp2059 2459492 wikitext text/x-wiki 45ex7htb17yhfdscz739vkvqi71tvuk 2459493 2459492 2026-09-22T17:33:15Z Stashbot 7414 sukhe@cumin1004: END (FAIL) - Cookbook sre.hosts.rename (exit_code=93) from sretest2013 to cp2059 2459493 wikitext text/x-wiki sbfb9ifou7zb22yafvvp1ub86896p5n 2459494 2459493 2026-09-22T17:36:50Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wdqs1029 - bking@cumin2003" 2459494 wikitext text/x-wiki dico75za7zgq9myx0kr0fso6lkpcyp0 2459495 2459494 2026-09-22T17:36:54Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wdqs1029 - bking@cumin2003" 2459495 wikitext text/x-wiki 28r1hm46xdip1lr4s487zc1xngsfluz 2459496 2459495 2026-09-22T17:36:56Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2459496 wikitext text/x-wiki lqm4rlfe9ymsvv5tcl7iqdg58580gcf 2459497 2459496 2026-09-22T17:36:57Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.dns.wipe-cache wdqs1029.eqiad.wmnet 8.48.64.10.in-addr.arpa 8.0.0.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2459497 wikitext text/x-wiki 5hfetnvd9ysa9tnyic9zfzzhfz8t7xq 2459498 2459497 2026-09-22T17:36:58Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wdqs1029.eqiad.wmnet 8.48.64.10.in-addr.arpa 8.0.0.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2459498 wikitext text/x-wiki 8fptpc70dzovihlnd90ecpat4mr69wn 2459499 2459498 2026-09-22T17:36:59Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.network.configure-switch-interfaces for host wdqs1029 2459499 wikitext text/x-wiki f31xji11e5mtwhtb9cesk0w1dfi95jh 2459500 2459499 2026-09-22T17:37:26Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wdqs1029 2459500 wikitext text/x-wiki ig6um6fcxm0u90xcrmruzkonglkp2ny 2459501 2459500 2026-09-22T17:37:28Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wdqs1029 2459501 wikitext text/x-wiki rjw1ng2dxf6ywhz2hv74s2jjse3l8gx 2459502 2459501 2026-09-22T17:39:32Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1012.eqiad.wmnet 2459502 wikitext text/x-wiki ac0ycxzfre7y62l31svimmsig9f78a9 2459503 2459502 2026-09-22T17:39:34Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1012.eqiad.wmnet 2459503 wikitext text/x-wiki 855xaqc8oghvfewxnk02xy3ym43iuvk 2459504 2459503 2026-09-22T17:39:40Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1013.eqiad.wmnet 2459504 wikitext text/x-wiki 819e5h9a4w8hmd119ec8rwtz2t1zwst 2459505 2459504 2026-09-22T17:43:51Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.move-vlan for host wdqs1030 2459505 wikitext text/x-wiki ia5nx0w8fsvt9cf886l9fq87pk1shsj 2459506 2459505 2026-09-22T17:45:24Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.dns.netbox 2459506 wikitext text/x-wiki kpe6ewmb2irpt54bnzyeokksivnmyqx 2459507 2459506 2026-09-22T17:45:34Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1344040|fix(WikimediaAntiAbuse): use correct endpoint for LiftWing in eqiad]] 2459507 wikitext text/x-wiki 73g8dd9m9925r7ohwpb4tppps1yls0u 2459508 2459507 2026-09-22T17:46:59Z Stashbot 7414 sukhe@cumin1004: START - Cookbook sre.hosts.rename from sretest2013 to cp2059 2459508 wikitext text/x-wiki aqfiiv6exmjeyrova1wx410udn9vq6z 2459509 2459508 2026-09-22T17:47:23Z Stashbot 7414 sukhe@cumin1004: END (FAIL) - Cookbook sre.hosts.rename (exit_code=93) from sretest2013 to cp2059 2459509 wikitext text/x-wiki seyuweq4axwlrjyuniflj17txsskchn 2459510 2459509 2026-09-22T17:50:02Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz, isaranto: Backport for [[gerrit:1344040|fix(WikimediaAntiAbuse): use correct endpoint for LiftWing in eqiad]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2459510 wikitext text/x-wiki ov11zy8f3rsjm5ig1piu772nlgbsf0m 2459511 2459510 2026-09-22T17:50:35Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz, isaranto: Continuing with deployment 2459511 wikitext text/x-wiki 2i2i7cexlbchflkn1l0rs1ptcixp2yb 2459512 2459511 2026-09-22T17:51:19Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2218: Optimizer issues fixed 2459512 wikitext text/x-wiki rc02mp6eokcb2swmagrnclraz35l6r0 2459516 2459512 2026-09-22T17:53:26Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wdqs1030 - bking@cumin2003" 2459516 wikitext text/x-wiki iacp228sph5y5w8xsgdjnavrxiamnlh 2459517 2459516 2026-09-22T17:53:30Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wdqs1030 - bking@cumin2003" 2459517 wikitext text/x-wiki 3wk9bwn06yie8iryhyvgoolnbd73avf 2459518 2459517 2026-09-22T17:53:31Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2459518 wikitext text/x-wiki 281oiblcy8psbhhmiebcr5p9ibvooxu 2459519 2459518 2026-09-22T17:53:32Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.dns.wipe-cache wdqs1030.eqiad.wmnet 8.32.64.10.in-addr.arpa 8.0.0.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2459519 wikitext text/x-wiki qihzpb1xnbdkae5ihse1k9ffnptncrr 2459520 2459519 2026-09-22T17:53:34Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wdqs1030.eqiad.wmnet 8.32.64.10.in-addr.arpa 8.0.0.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2459520 wikitext text/x-wiki fn610gmfttj2lbetu9xz153swzaglr8 2459521 2459520 2026-09-22T17:53:35Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.network.configure-switch-interfaces for host wdqs1030 2459521 wikitext text/x-wiki 5n759e0y6tabixjw1xopogr15t2a85q 2459522 2459521 2026-09-22T17:54:30Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wdqs1030 2459522 wikitext text/x-wiki gy5e0zivzh3154esnofg4troqrefaqz 2459523 2459522 2026-09-22T17:54:31Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wdqs1030 2459523 wikitext text/x-wiki mjlr9alwva5ecdh914wbagnpy3hour4 2459524 2459523 2026-09-22T17:55:43Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344040|fix(WikimediaAntiAbuse): use correct endpoint for LiftWing in eqiad]] (duration: 10m 09s) 2459524 wikitext text/x-wiki dadm10b51y0vq5m6vgdo9je3rxxca8q 2459526 2459524 2026-09-22T18:07:07Z Stashbot 7414 ebernhardson: Rolling restart opensearch-semantic-search in dse-k8s-eqiad to update to opensearch 3.8.0 2459526 wikitext text/x-wiki 27dnmjj2v2pzc0qsngnbx4z6vbt63bj 2459527 2459526 2026-09-22T18:09:44Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1013.eqiad.wmnet 2459527 wikitext text/x-wiki 5glr60wo98344vg1r97m6pjh7qatcbx 2459528 2459527 2026-09-22T18:16:05Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1013.eqiad.wmnet 2459528 wikitext text/x-wiki kw2e20zsl8nabuutzkdl1ahns4rad1e 2459529 2459528 2026-09-22T18:16:06Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1013.eqiad.wmnet 2459529 wikitext text/x-wiki 2nrjndb5g44yx6cs7uajza47iinkka8 2459530 2459529 2026-09-22T18:16:11Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1014.eqiad.wmnet 2459530 wikitext text/x-wiki n08xgzu4ncl00w7eafa0o0eu6l7f9z0 2459532 2459530 2026-09-22T18:18:15Z Stashbot 7414 btullis@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host dse-k8s-worker1014.eqiad.wmnet 2459532 wikitext text/x-wiki npjz6xfrftd5kirxtnjm4bsrmanes9i 2459533 2459532 2026-09-22T18:35:31Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1014.eqiad.wmnet 2459533 wikitext text/x-wiki 9g8mijugcweww25lmnro5xsf53s5orv 2459534 2459533 2026-09-22T18:35:32Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1014.eqiad.wmnet 2459534 wikitext text/x-wiki 05gj6ciw5fe0tzq88vgc73hyd5ydta3 2459535 2459534 2026-09-22T18:35:37Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1018.eqiad.wmnet 2459535 wikitext text/x-wiki 673mk4e8ygqxfzkftewkzwfvjcr7mry 2459536 2459535 2026-09-22T18:36:23Z Stashbot 7414 jhuneidi@deploy1003: rebuilt and synchronized wikiversions files: group0 to 1.47.0-wmf.21 refs T438217 2459536 wikitext text/x-wiki lt9eu90axhd7rtag1rfppzbxsoeburl 2459537 2459536 2026-09-22T18:40:43Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1018.eqiad.wmnet 2459537 wikitext text/x-wiki h58mt5mcg6qq9zfhl2d3b7kvv1p52eg 2459538 2459537 2026-09-22T18:41:01Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.291.0" for 3 host(s) 2459538 wikitext text/x-wiki h2vp12png48cnm4wr2x30hqdlk87pi1 2459539 2459538 2026-09-22T18:41:48Z Stashbot 7414 dancy@deploy1003: install-world aborted: (no justification provided) (duration: 00m 48s) 2459539 wikitext text/x-wiki m1gmeq6ey91bwfqr8gxpwarzoibyobf 2459540 2459539 2026-09-22T18:43:13Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.291.0" for 3 host(s) 2459540 wikitext text/x-wiki bgkl3tl4q7wyfmecdljlo302d1bfm5k 2459541 2459540 2026-09-22T18:45:00Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.291.0" for 3 host(s) 2459541 wikitext text/x-wiki nzskbt65qoliaknka3ttamftol5g6uq 2459542 2459541 2026-09-22T18:47:10Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.291.0" for 3 host(s) 2459542 wikitext text/x-wiki c2qpuaudymk37nqo70w2vcp0lowi9sk 2459543 2459542 2026-09-22T18:49:04Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1018.eqiad.wmnet 2459543 wikitext text/x-wiki cw5uvk0v0u8tdiba4wwas9s0vahe0ju 2459544 2459543 2026-09-22T18:49:06Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1018.eqiad.wmnet 2459544 wikitext text/x-wiki d5ajwybzgk6q4s1w93mrg9b60ezrvrf 2459545 2459544 2026-09-22T18:49:15Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1019.eqiad.wmnet 2459545 wikitext text/x-wiki 270ehljgjczrtblid80t3mbkopvan7y 2459546 2459545 2026-09-22T18:49:46Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1019.eqiad.wmnet 2459546 wikitext text/x-wiki p94x32evqh2fslimsemen8mk46ldtpq 2459547 2459546 2026-09-22T18:51:54Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.291.0" for 3 host(s) 2459547 wikitext text/x-wiki hdsmlbkfxfci4pt92sm1sg4vnq7wg7c 2459548 2459547 2026-09-22T18:53:45Z Stashbot 7414 dancy@deploy1003: Installation of scap version "4.291.0" completed for 3 hosts 2459548 wikitext text/x-wiki lvzgbdcj75pvahng0m7eg0glc0ap3t6 2459549 2459548 2026-09-22T18:53:58Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.291.0" for 2 host(s) 2459549 wikitext text/x-wiki f6ch6s2g8n41d2s48g6ava8xfxzu8mw 2459550 2459549 2026-09-22T18:55:03Z Stashbot 7414 dancy@deploy1003: Installation of scap version "4.291.0" completed for 2 hosts 2459550 wikitext text/x-wiki sl00e2lw4lxav9x0tym0h2j5ozuuons 2459551 2459550 2026-09-22T18:56:37Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1019.eqiad.wmnet 2459551 wikitext text/x-wiki smk0ujez1x151uhs25iwdevuc0idjiq 2459552 2459551 2026-09-22T18:56:38Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1019.eqiad.wmnet 2459552 wikitext text/x-wiki kjhy6wxfwle561cs3ppyh0avl7wy1cn 2459553 2459552 2026-09-22T18:56:41Z Stashbot 7414 jclark@cumin1004: START - Cookbook sre.hosts.reimage for host ml-serve1016.eqiad.wmnet with OS trixie 2459553 wikitext text/x-wiki tahkxa3bx4hfuewpfvl69wssdp0t4bc 2459554 2459553 2026-09-22T18:56:43Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1020.eqiad.wmnet 2459554 wikitext text/x-wiki au6je0z1mzmsvgnhy3nlxl1pja4o1sr 2459555 2459554 2026-09-22T18:58:47Z Stashbot 7414 btullis@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host dse-k8s-worker1020.eqiad.wmnet 2459555 wikitext text/x-wiki k4eg2ac2am5t9u9ueg3hdr458pa4znp 2459556 2459555 2026-09-22T19:01:39Z Stashbot 7414 ebernhardson: Rolling restart opensearch-semantic-search in dse-k8s-codfw to update to opensearch 3.8.0 2459556 wikitext text/x-wiki 0tnct7lieogaqkbeyoozc06rtrfc3a8 2459558 2459556 2026-09-22T19:15:49Z Stashbot 7414 jclark@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ml-serve1016.eqiad.wmnet with reason: host reimage 2459558 wikitext text/x-wiki 750eoo0jjnqb18ggs0a5s9slc3tkvby 2459559 2459558 2026-09-22T19:17:09Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1020.eqiad.wmnet 2459559 wikitext text/x-wiki dric4ysvkb0yw9ynbqt1acgmlibglnp 2459560 2459559 2026-09-22T19:17:10Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1020.eqiad.wmnet 2459560 wikitext text/x-wiki lf18i2tw8m11cetbkqjezb1ztpv91j9 2459561 2459560 2026-09-22T19:17:16Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1021.eqiad.wmnet 2459561 wikitext text/x-wiki tqopwz5fxbyfz4mi7kk3qvfe11zsh62 2459562 2459561 2026-09-22T19:19:04Z Stashbot 7414 jclark@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ml-serve1016.eqiad.wmnet with reason: host reimage 2459562 wikitext text/x-wiki e24pfrv2qxebfgt7udxi05jzc8x64xv 2459563 2459562 2026-09-22T19:19:22Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.reimage for host ncredir2002.codfw.wmnet with OS trixie 2459563 wikitext text/x-wiki 22wmnaj2f69lbrzp5e4oyfu9kxxfk4b 2459566 2459563 2026-09-22T19:21:41Z Stashbot 7414 sukhe@cumin1004: START - Cookbook sre.hosts.rename from sretest2013 to cp2059 2459566 wikitext text/x-wiki 3ogznlgw7enif8db3w53t65frj7zvzs 2459567 2459566 2026-09-22T19:22:06Z Stashbot 7414 sukhe@cumin1004: END (FAIL) - Cookbook sre.hosts.rename (exit_code=93) from sretest2013 to cp2059 2459567 wikitext text/x-wiki 99lcpqhnhulfupk0whrodpqw3uniq4j 2459568 2459567 2026-09-22T19:23:20Z Stashbot 7414 btullis@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host dse-k8s-worker1021.eqiad.wmnet 2459568 wikitext text/x-wiki 6h091fcanjtu33j8z7qoiwczavsn5ns 2459569 2459568 2026-09-22T19:24:57Z Stashbot 7414 sukhe@cumin1004: START - Cookbook sre.hosts.rename from sretest2013 to cp2059 2459569 wikitext text/x-wiki 04jejdu3vxd22i26i7v0ekiq34n0w7x 2459570 2459569 2026-09-22T19:25:22Z Stashbot 7414 sukhe@cumin1004: END (FAIL) - Cookbook sre.hosts.rename (exit_code=93) from sretest2013 to cp2059 2459570 wikitext text/x-wiki r7bfwtvja5sfpj3oq4g52whr5mmoz4e 2459573 2459570 2026-09-22T19:33:48Z Stashbot 7414 jclark@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1004" 2459573 wikitext text/x-wiki 86jy0cmgsuvtzs5qx8km6o5drzxujoa 2459574 2459573 2026-09-22T19:34:03Z Stashbot 7414 jclark@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1004" 2459574 wikitext text/x-wiki 34yuyldrnwkodl6fjkgv9bq2wbvc0c1 2459575 2459574 2026-09-22T19:34:04Z Stashbot 7414 jclark@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ml-serve1016.eqiad.wmnet with OS trixie 2459575 wikitext text/x-wiki kl2y8733x8pfbwxa097rcj4anjgsfl3 2459576 2459575 2026-09-22T19:38:52Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ncredir2002.codfw.wmnet with reason: host reimage 2459576 wikitext text/x-wiki gyhyj5etb9s68njjthwgenpz1nn7qgn 2459577 2459576 2026-09-22T19:42:12Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1021.eqiad.wmnet 2459577 wikitext text/x-wiki qhsixjq4effs7t2kzvotagsovr2r3q8 2459578 2459577 2026-09-22T19:42:13Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1021.eqiad.wmnet 2459578 wikitext text/x-wiki 91vq70wswosbt7tzr9cxu3a62v2hi69 2459579 2459578 2026-09-22T19:42:19Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1022.eqiad.wmnet 2459579 wikitext text/x-wiki c0k5hf4a821rzobnnsw05inh1eq5ouf 2459580 2459579 2026-09-22T19:42:21Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ncredir2002.codfw.wmnet with reason: host reimage 2459580 wikitext text/x-wiki 7crmwgviqdvwegugk4ocd9hq5qwbicn 2459581 2459580 2026-09-22T19:44:22Z Stashbot 7414 btullis@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host dse-k8s-worker1022.eqiad.wmnet 2459581 wikitext text/x-wiki m52pihqx4auxtnquyx20kk30k5uaua4 2459590 2459581 2026-09-22T19:59:35Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ncredir2002.codfw.wmnet with OS trixie 2459590 wikitext text/x-wiki 8s7aqjctx61kun04uwgv96i8vxvjqyj 2459592 2459590 2026-09-22T20:02:05Z Stashbot 7414 sukhe@cumin1004: START - Cookbook sre.hosts.reimage for host sretest2013.codfw.wmnet with OS trixie 2459592 wikitext text/x-wiki 8odooxg4xoink932chl5l6vpnexz09m 2459593 2459592 2026-09-22T20:02:54Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1022.eqiad.wmnet 2459593 wikitext text/x-wiki 6pgdweekytrl0zi9hjsoppe65ybd85l 2459594 2459593 2026-09-22T20:02:56Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1022.eqiad.wmnet 2459594 wikitext text/x-wiki itpxg9pudcex8hp96pn9tzaba55zjpo 2459595 2459594 2026-09-22T20:03:01Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1023.eqiad.wmnet 2459595 wikitext text/x-wiki 7kkwcr3eyozpxiarr034fh7wh6ihmyf 2459597 2459595 2026-09-22T20:06:53Z Stashbot 7414 ebernhardson@deploy1003: Started scap sync-world: Backport for [[gerrit:1344014|cirrus: Send more_like traffic to eqiad]] 2459597 wikitext text/x-wiki 0tnk0qzy7qt3pnc22svqbaib87x0u1t 2459600 2459597 2026-09-22T20:11:16Z Stashbot 7414 ebernhardson@deploy1003: ebernhardson: Backport for [[gerrit:1344014|cirrus: Send more_like traffic to eqiad]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2459600 wikitext text/x-wiki fzpd1il1cfuekrdfgfsa94qvxancxgz 2459601 2459600 2026-09-22T20:12:22Z Stashbot 7414 ebernhardson@deploy1003: ebernhardson: Continuing with deployment 2459601 wikitext text/x-wiki rob0o4vdw76hr2zvme2ash7ja28ux03 2459602 2459601 2026-09-22T20:12:41Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2459602 wikitext text/x-wiki sxroykrqfou4inwl8m64lf2dvfhj8w2 2459603 2459602 2026-09-22T20:13:07Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=yes; selector: name=ncredir2002.* 2459603 wikitext text/x-wiki efbfawny31vevkootkr1kx5wynjihlv 2459604 2459603 2026-09-22T20:15:08Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2459604 wikitext text/x-wiki k6v0n5xwjcngfkspw8t1wnrhgy2u3nm 2459606 2459604 2026-09-22T20:17:22Z Stashbot 7414 ebernhardson@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344014|cirrus: Send more_like traffic to eqiad]] (duration: 10m 29s) 2459606 wikitext text/x-wiki b3rv14yh7n2zgwys3iyu9rykguik294 2459608 2459606 2026-09-22T20:20:01Z Stashbot 7414 ebernhardson@deploy1003: Started scap sync-world: Backport for [[gerrit:1342825|eswiki: Add abusefilter-access-protected-vars to abusefilter user group (T436652)]] 2459608 wikitext text/x-wiki oqzg5xpfoa5qk8eq0ad8y0ipgul3w6m 2459609 2459608 2026-09-22T20:24:23Z Stashbot 7414 ebernhardson@deploy1003: ebernhardson, codenamenoreste: Backport for [[gerrit:1342825|eswiki: Add abusefilter-access-protected-vars to abusefilter user group (T436652)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2459609 wikitext text/x-wiki smgewr8n20xp8ltlif5uq6yngetj0ki 2459610 2459609 2026-09-22T20:28:36Z Stashbot 7414 ebernhardson@deploy1003: ebernhardson, codenamenoreste: Continuing with deployment 2459610 wikitext text/x-wiki e8o9b3qrpdzvrqmthz4lyrabfit0boq 2459612 2459610 2026-09-22T20:33:05Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1023.eqiad.wmnet 2459612 wikitext text/x-wiki 07n4muk4mbjazzdnm6kpnmcn786eb8a 2459613 2459612 2026-09-22T20:33:36Z Stashbot 7414 ebernhardson@deploy1003: Finished scap sync-world: Backport for [[gerrit:1342825|eswiki: Add abusefilter-access-protected-vars to abusefilter user group (T436652)]] (duration: 13m 35s) 2459613 wikitext text/x-wiki dsird9hrt9a2h2zzbdoddh399x2gg38 2459614 2459613 2026-09-22T20:35:52Z Stashbot 7414 sbisson@deploy1003: Started scap sync-world: Backport for [[gerrit:1342285|Keep Article Guidance on where it is on today (T433293)]] 2459614 wikitext text/x-wiki c8p3997ldl99ajlq4qiqhr26t19d3gd 2459616 2459614 2026-09-22T20:40:03Z Stashbot 7414 sbisson@deploy1003: sbisson: Backport for [[gerrit:1342285|Keep Article Guidance on where it is on today (T433293)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2459616 wikitext text/x-wiki rd6o7tt0z54etlgtkf599n1vbtpc1j5 2459617 2459616 2026-09-22T20:40:09Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1023.eqiad.wmnet 2459617 wikitext text/x-wiki 78n0yso4gqlkeoz4yhhr13txz5mbnpd 2459618 2459617 2026-09-22T20:40:10Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1023.eqiad.wmnet 2459618 wikitext text/x-wiki 3jiu4yhrhz2ldmcd1e7oww3a8nq3ex4 2459619 2459618 2026-09-22T20:40:16Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1024.eqiad.wmnet 2459619 wikitext text/x-wiki hlbszsurwea3fd4gu9oscai7vaa9soa 2459620 2459619 2026-09-22T20:40:45Z Stashbot 7414 sbisson@deploy1003: sbisson: Continuing with deployment 2459620 wikitext text/x-wiki 5la8zvv9vmp8n8551tji20f9et10zg2 2459621 2459620 2026-09-22T20:43:29Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2459621 wikitext text/x-wiki 7fopugc1a4axexoxa48xsqcwpxliyzm 2459622 2459621 2026-09-22T20:44:22Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2459622 wikitext text/x-wiki lu4gvfn1mpbrdx5jvh7yoz66vazsn3m 2459623 2459622 2026-09-22T20:44:40Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2459623 wikitext text/x-wiki s5bufpi32lyvxshn9n8dsn9sz4jtgdj 2459624 2459623 2026-09-22T20:45:33Z Stashbot 7414 aqu@deploy1003: Started deploy [analytics/refinery@58c9356] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@58c93566] 2459624 wikitext text/x-wiki ipzog4jmk5ph0d11djo4kmp13fu0zn3 2459625 2459624 2026-09-22T20:45:45Z Stashbot 7414 sbisson@deploy1003: Finished scap sync-world: Backport for [[gerrit:1342285|Keep Article Guidance on where it is on today (T433293)]] (duration: 09m 53s) 2459625 wikitext text/x-wiki qhe37wdp0b1m0pmop4lnbqo4gf45vsj 2459626 2459625 2026-09-22T20:45:58Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2459626 wikitext text/x-wiki ipikfswaw3oh5cqr2v83scn7vrqzhkw 2459627 2459626 2026-09-22T20:46:13Z Stashbot 7414 aqu@deploy1003: Finished deploy [analytics/refinery@58c9356] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@58c93566] (duration: 00m 40s) 2459627 wikitext text/x-wiki g91sfjxy8optj8la88spa4cf6fyjfig 2459628 2459627 2026-09-22T20:48:05Z Stashbot 7414 aqu@deploy1003: Started deploy [analytics/refinery@58c9356]: Regular analytics weekly train [analytics/refinery@58c93566] 2459628 wikitext text/x-wiki gszugfab4l3i0bawr2bxe18e3hsa1hk 2459632 2459628 2026-09-22T20:55:04Z Stashbot 7414 aqu@deploy1003: Finished deploy [analytics/refinery@58c9356]: Regular analytics weekly train [analytics/refinery@58c93566] (duration: 06m 59s) 2459632 wikitext text/x-wiki j0g1rrwpo744dn3ngy7ick7erewg3x8 2459633 2459632 2026-09-22T20:59:11Z Stashbot 7414 aqu@deploy1003: Started deploy [analytics/refinery@58c9356] (thin): Regular analytics weekly train THIN [analytics/refinery@58c93566] 2459633 wikitext text/x-wiki gxdot62x7759r7bceiftebzw57cfcwu 2459634 2459633 2026-09-22T20:59:41Z Stashbot 7414 aqu@deploy1003: Finished deploy [analytics/refinery@58c9356] (thin): Regular analytics weekly train THIN [analytics/refinery@58c93566] (duration: 00m 30s) 2459634 wikitext text/x-wiki mb4opo3by5jhmq49rzc7xyhmzaev1hl 2459635 2459634 2026-09-22T20:59:50Z Stashbot 7414 aqu@deploy1003: Started deploy [analytics/refinery@58c9356] (thin): Regular analytics weekly train THIN [analytics/refinery@58c93566] 2459635 wikitext text/x-wiki mklxugnd9yq1qnk92a76peipo9ukc9n 2459636 2459635 2026-09-22T21:05:11Z Stashbot 7414 sukhe@cumin1004: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2013.codfw.wmnet with OS trixie 2459636 wikitext text/x-wiki aboxiwnuv5lpssqx3xd3zaqlowcs5l6 2459637 2459636 2026-09-22T21:10:20Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1024.eqiad.wmnet 2459637 wikitext text/x-wiki a8bgen6k9u9wza53p7zwarsmdvygrs1 2459638 2459637 2026-09-22T21:18:58Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1024.eqiad.wmnet 2459638 wikitext text/x-wiki bho302v1nequ1hrrr1m4xarrlu4of2g 2459639 2459638 2026-09-22T21:18:59Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1024.eqiad.wmnet 2459639 wikitext text/x-wiki 2yredmnh8ib9csjvchyv7rf5geubq3q 2459640 2459639 2026-09-22T21:19:05Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1025.eqiad.wmnet 2459640 wikitext text/x-wiki mt37jz92pprj9y07wuz2l9zts3hlp94 2459645 2459640 2026-09-22T21:24:10Z Stashbot 7414 aqu@deploy1003: Finished deploy [analytics/refinery@58c9356] (thin): Regular analytics weekly train THIN [analytics/refinery@58c93566] (duration: 24m 20s) 2459645 wikitext text/x-wiki m73zvintuxgovocp1saspmfk2wibcps 2459646 2459645 2026-09-22T21:24:13Z Stashbot 7414 aqu@deploy1003: Started deploy [analytics/refinery@58c9356] (thin): Regular analytics weekly train THIN [analytics/refinery@58c93566] 2459646 wikitext text/x-wiki gmsm6n16xf2hoikirsvgrhzatyopth6 2459647 2459646 2026-09-22T21:25:23Z Stashbot 7414 aqu@deploy1003: Finished deploy [analytics/refinery@58c9356] (thin): Regular analytics weekly train THIN [analytics/refinery@58c93566] (duration: 01m 09s) 2459647 wikitext text/x-wiki ts2grqx0j3nfsktxi68lhte2rqr3n29 2459650 2459647 2026-09-22T21:47:46Z Stashbot 7414 rzl@deploy1003: Started scap sync-world: https://gerrit.wikimedia.org/r/1339694 T437403 2459650 wikitext text/x-wiki l9twrnlt0aurrj677n1vyaxyys7m3y7 2459651 2459650 2026-09-22T21:49:06Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1025.eqiad.wmnet 2459651 wikitext text/x-wiki o04iwoquxgf6zf52vhwiuxlh6xth2g9 2459652 2459651 2026-09-22T21:51:01Z Stashbot 7414 rzl@deploy1003: rzl: https://gerrit.wikimedia.org/r/1339694 T437403 synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2459652 wikitext text/x-wiki 1r9ivz77b7vbp8y2vmnly8px4oj5bpq 2459653 2459652 2026-09-22T21:53:35Z Stashbot 7414 rzl@deploy1003: rzl: Continuing with deployment 2459653 wikitext text/x-wiki 9ozpfvi7gjb1gjckp2my8tkqfno3hdi 2459654 2459653 2026-09-22T21:57:16Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1025.eqiad.wmnet 2459654 wikitext text/x-wiki 4afqwzyb2xpciz8ya5ulmigvurmon7n 2459655 2459654 2026-09-22T21:57:18Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1025.eqiad.wmnet 2459655 wikitext text/x-wiki cn90oysmde253jtguhjt225lyqhrjyu 2459656 2459655 2026-09-22T21:57:23Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1026.eqiad.wmnet 2459656 wikitext text/x-wiki 49ah1i100kxhe0cd1ajxe8prckvf21q 2459657 2459656 2026-09-22T21:57:58Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1026.eqiad.wmnet 2459657 wikitext text/x-wiki 1r86hu8rocwxcu1iv4en0qicq599won 2459658 2459657 2026-09-22T21:58:29Z Stashbot 7414 rzl@deploy1003: Finished scap sync-world: https://gerrit.wikimedia.org/r/1339694 T437403 (duration: 13m 43s) 2459658 wikitext text/x-wiki 74k71rmmk3jxtab4dcq9uu9znkxcnyh 2459659 2459658 2026-09-22T22:06:25Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1026.eqiad.wmnet 2459659 wikitext text/x-wiki snzc1txheriqd4b9prkf87r0pgjlv7a 2459660 2459659 2026-09-22T22:06:27Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1026.eqiad.wmnet 2459660 wikitext text/x-wiki pqtiemp1qvaijowkfwb34ulhwti6dwd 2459661 2459660 2026-09-22T22:06:32Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1027.eqiad.wmnet 2459661 wikitext text/x-wiki f4rug9prhklyogk1zdnvep655xeqwm8 2459663 2459661 2026-09-22T22:30:21Z Stashbot 7414 ryankemper: [WDQS] codfw wdqs-main is struggling under the switchover load, fiddling with some auto-restart knobs to see if it helps or hurts 2459663 wikitext text/x-wiki lp0n1uokxwz9ocs7gtg6om5ltxza1xd 2459664 2459663 2026-09-22T22:36:35Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1027.eqiad.wmnet 2459664 wikitext text/x-wiki i3e8vkuds8nyt2yxrznevusvg6fnt7x 2459669 2459664 2026-09-22T22:45:19Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1027.eqiad.wmnet 2459669 wikitext text/x-wiki kx5c3z35f84xpxh2a8dooosfmxyvy7g 2459670 2459669 2026-09-22T22:45:20Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1027.eqiad.wmnet 2459670 wikitext text/x-wiki co9ptp8pel71pw5yzip354hwdxhyc8h 2459671 2459670 2026-09-22T22:45:26Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1028.eqiad.wmnet 2459671 wikitext text/x-wiki hzrql1a2uooc99qbp2ppa8c42rglmwt 2459672 2459671 2026-09-22T23:15:28Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1028.eqiad.wmnet 2459672 wikitext text/x-wiki p75obc1a5xrmg2v56e83xrc1mm1v5wt 2459673 2459672 2026-09-22T23:23:37Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1028.eqiad.wmnet 2459673 wikitext text/x-wiki dd67rrr2xbyk0hyb1yfoi2hs3ekhgmv 2459674 2459673 2026-09-22T23:23:38Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1028.eqiad.wmnet 2459674 wikitext text/x-wiki 9jgbi7v7kcyfexefqkj5e5m44ap8f2r 2459675 2459674 2026-09-22T23:23:39Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on P{dse-k8s-worker10[02-28].eqiad.wmnet} and (A:dse-k8s-master-eqiad or A:dse-k8s-worker-eqiad) 2459675 wikitext text/x-wiki 7s79e4o4e33gnjamrof0a3tqr72csor 2459677 2459675 2026-09-23T00:45:28Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.discovery.service-route pool wdqs-main in eqiad: maintenance 2459677 wikitext text/x-wiki 3s4obbqig1qi4f4a8gox5s1dt77f1qg 2459678 2459677 2026-09-23T00:46:46Z Stashbot 7414 ryankemper: [WDQS] T435443 Restore eqiad wdqs-main; wdqs was unable to keep up with traffic with only one datacenter. sadly this will continue to be the case until wdqsv2 is ready to switch backend architecture 2459678 wikitext text/x-wiki ral3ruycia95a1x5dbmqvfz0b6sgh9g 2459679 2459678 2026-09-23T00:50:33Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) pool wdqs-main in eqiad: maintenance 2459679 wikitext text/x-wiki ormkpki34yu6ekv4qoqetkjlkof9z34 2459680 2459679 2026-09-23T02:00:43Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2459680 wikitext text/x-wiki m6kpa3rrikintpjmanhhfobkr99igsl 2459681 2459680 2026-09-23T02:08:23Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 07m 39s) 2459681 wikitext text/x-wiki g5pq10ly8wi618jmf5oz12zvc5rnth9 2459696 2459681 2026-09-23T07:29:53Z Stashbot 7414 moritzm: installing python-idna security updates 2459696 wikitext text/x-wiki gblhml90mm7fsqmtxqfve2unm95aea4 2459698 2459696 2026-09-23T07:58:36Z Stashbot 7414 tappof@deploy1003: helmfile [eqiad] START helmfile.d/admin 'sync'. 2459698 wikitext text/x-wiki 03a04w6bie42ln8oqs6uswgdgxeqdip 2459699 2459698 2026-09-23T07:58:51Z Stashbot 7414 tappof@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'sync'. 2459699 wikitext text/x-wiki qoasovlim6n2nk1z6k7s5211ow73178 2459700 2459699 2026-09-23T07:59:26Z Stashbot 7414 moritzm: installing giflib security updates 2459700 wikitext text/x-wiki c15iht1o2ndj82onc1zphgmfkzbsyjb 2459701 2459700 2026-09-23T07:59:32Z Stashbot 7414 tappof@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'. 2459701 wikitext text/x-wiki dwnbto5o1vl3si3y5noac817g1ru8i5 2459702 2459701 2026-09-23T08:00:07Z Stashbot 7414 tappof@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'. 2459702 wikitext text/x-wiki l1liprf301jdgsbjz19afwyftvfyyai 2459703 2459702 2026-09-23T08:04:20Z Stashbot 7414 tappof@deploy1003: Started scap sync-world: T432444 - Provision kafka-logging100[6-8] 2459703 wikitext text/x-wiki qhucn3w590fvlgxjhnw6bcyyu8fzyyw 2459704 2459703 2026-09-23T08:05:48Z Stashbot 7414 moritzm: installing grub2 bugfix updates on Bookworm hosts 2459704 wikitext text/x-wiki o80kynq246rxyep3ae2mf82z4od0415 2459705 2459704 2026-09-23T08:13:56Z Stashbot 7414 tappof@deploy1003: Finished scap sync-world: T432444 - Provision kafka-logging100[6-8] (duration: 12m 52s) 2459705 wikitext text/x-wiki qm21ggcqxq6sroq6q1xf2h7qb94v9nh 2459706 2459705 2026-09-23T08:24:50Z Stashbot 7414 kevinbazira@deploy1003: helmfile [ml-serve-eqiad] 'sync' command on namespace 'tts-section-generator' for release 'main' . 2459706 wikitext text/x-wiki 5gc7lx7eitz9e6cxpkrx8hosh7cgpcd 2459707 2459706 2026-09-23T08:25:31Z Stashbot 7414 kevinbazira@deploy1003: helmfile [ml-serve-codfw] 'sync' command on namespace 'tts-section-generator' for release 'main' . 2459707 wikitext text/x-wiki bn7cbkej1aylnn5fftvngk37n56oxe3 2459709 2459707 2026-09-23T08:27:23Z Stashbot 7414 brouberol@cumin1004: START - Cookbook sre.ceph.remove-osd 2459709 wikitext text/x-wiki om2ghokiuaprvmkl1oatgdge1hzvnhl 2459710 2459709 2026-09-23T08:27:25Z Stashbot 7414 brouberol@cumin1004: END (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459710 wikitext text/x-wiki cayei1k2y30qx0qazugaj1qy599aabc 2459714 2459710 2026-09-23T08:41:19Z Stashbot 7414 brouberol@cumin1004: DONE (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459714 wikitext text/x-wiki si5eba0uar8xfq1vihs85aj7jgc907m 2459716 2459714 2026-09-23T08:44:04Z Stashbot 7414 brouberol@cumin1004: DONE (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459716 wikitext text/x-wiki 5q5173sw0m2j7uebwx50dvlrou2q19q 2459717 2459716 2026-09-23T08:44:24Z Stashbot 7414 brouberol@cumin1004: DONE (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459717 wikitext text/x-wiki m2lwujjncdj4owudjpn8tdxuay4r8d1 2459719 2459717 2026-09-23T08:45:11Z Stashbot 7414 brouberol@cumin1004: DONE (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459719 wikitext text/x-wiki f9k6iwu9attx5n1virfok2rsnnbvh79 2459740 2459719 2026-09-23T09:34:20Z Stashbot 7414 brouberol@cumin1004: DONE (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459740 wikitext text/x-wiki 8itprsy6sz4kef4fcpl34wxaedd7lw2 2459741 2459740 2026-09-23T09:38:36Z Stashbot 7414 brouberol@cumin1004: DONE (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459741 wikitext text/x-wiki smzpcm9565npapxg5js24bg1ovine8l 2459747 2459741 2026-09-23T09:43:55Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.reboot-nodes rolling reboot on P{dse-k8s-wdqs100[1-3].eqiad.wmnet} and (A:dse-k8s-master-eqiad or A:dse-k8s-worker-eqiad) 2459747 wikitext text/x-wiki 4b2z6b32agji1zvbf99v86juiup1e1t 2459748 2459747 2026-09-23T09:43:58Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-wdqs1001.eqiad.wmnet 2459748 wikitext text/x-wiki 17a2ubwr6k41xnfgf8ecb0hcsg6ir19 2459749 2459748 2026-09-23T09:44:33Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-wdqs1001.eqiad.wmnet 2459749 wikitext text/x-wiki nm6na9iq77qgy6jjg65q9nz3houdxx7 2459750 2459749 2026-09-23T09:50:14Z Stashbot 7414 brouberol@cumin1004: DONE (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459750 wikitext text/x-wiki 0befmi2e4q6tculkbeyoqvki560z3sz 2459751 2459750 2026-09-23T09:51:12Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-wdqs1001.eqiad.wmnet 2459751 wikitext text/x-wiki 0eh2t7q4y7atlwgdo5tmi9buop04g4m 2459752 2459751 2026-09-23T09:51:13Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-wdqs1001.eqiad.wmnet 2459752 wikitext text/x-wiki 8lwifvc2jwq3sse9pogngi5o0kq3um8 2459753 2459752 2026-09-23T09:51:18Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-wdqs1002.eqiad.wmnet 2459753 wikitext text/x-wiki pdemuogwtt2oaybl6w3e3sdcuq7ybzw 2459754 2459753 2026-09-23T09:51:53Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-wdqs1002.eqiad.wmnet 2459754 wikitext text/x-wiki l19pgpuo1a2u3zwrlh7sxcwhstda161 2459755 2459754 2026-09-23T09:51:56Z Stashbot 7414 brouberol@cumin1004: DONE (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459755 wikitext text/x-wiki fwv3f50lt8s4rk0p9zvwxs6et3fl2j2 2459756 2459755 2026-09-23T09:52:45Z Stashbot 7414 brouberol@cumin1004: DONE (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459756 wikitext text/x-wiki ripa137z4z5v5d0fmcctjjgnpnpwygg 2459759 2459756 2026-09-23T09:53:51Z Stashbot 7414 brouberol@cumin1004: DONE (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459759 wikitext text/x-wiki hp3fxsrmd0v8try24iosfid1idrrqn8 2459760 2459759 2026-09-23T09:54:05Z Stashbot 7414 brouberol@cumin1004: DONE (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459760 wikitext text/x-wiki 708u76j8gfho6t3j041ozs1zm7p1fpl 2459761 2459760 2026-09-23T09:54:46Z Stashbot 7414 brouberol@cumin1004: DONE (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459761 wikitext text/x-wiki rerrtbb6wgnhwp6p6u5gubal9oag3fl 2459762 2459761 2026-09-23T09:55:28Z Stashbot 7414 brouberol@cumin1004: DONE (PASS) - Cookbook sre.ceph.remove-osd (exit_code=0) 2459762 wikitext text/x-wiki 5dfoxsk6y2erai8e1o4bm7n2ck443sk 2459763 2459762 2026-09-23T09:57:28Z Stashbot 7414 brouberol@cumin1004: DONE (FAIL) - Cookbook sre.ceph.remove-osd (exit_code=99) 2459763 wikitext text/x-wiki mc99qwwryer0f24artxz2g5gm2yzwvq 2459764 2459763 2026-09-23T09:58:48Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-wdqs1002.eqiad.wmnet 2459764 wikitext text/x-wiki 6kgt7qy6d9lgol7q7wxxh5fkkd2ssr4 2459765 2459764 2026-09-23T09:58:49Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-wdqs1002.eqiad.wmnet 2459765 wikitext text/x-wiki 9z4avuiehthxld662fbwxfrs2wwiusg 2459766 2459765 2026-09-23T09:58:55Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-wdqs1003.eqiad.wmnet 2459766 wikitext text/x-wiki f5lsxs7ugdu1dlboo52wdixz9yd05r5 2459767 2459766 2026-09-23T09:59:04Z Stashbot 7414 ayounsi@cumin1004: START - Cookbook sre.deploy.python-code netbox to netbox-dev2003.codfw.wmnet with reason: Add netbox-bgp and update wheelson netbox-next - ayounsi@cumin1004 2459767 wikitext text/x-wiki o43awnk2l03vijeupwz8xwvuzwslia5 2459768 2459767 2026-09-23T09:59:29Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-wdqs1003.eqiad.wmnet 2459768 wikitext text/x-wiki 0dpqikun5rl1bb8lau4ns7bx6val1n7 2459772 2459768 2026-09-23T10:04:29Z Stashbot 7414 ayounsi@cumin1004: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) netbox to netbox-dev2003.codfw.wmnet with reason: Add netbox-bgp and update wheelson netbox-next - ayounsi@cumin1004 2459772 wikitext text/x-wiki abpwxqv3hwk9glypoxkhomj5a82slt3 2459773 2459772 2026-09-23T10:06:26Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-wdqs1003.eqiad.wmnet 2459773 wikitext text/x-wiki qatycnur5uereega0g8ci3f6uyfyxbg 2459774 2459773 2026-09-23T10:06:28Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-wdqs1003.eqiad.wmnet 2459774 wikitext text/x-wiki ifj8sco0zvwytzjn20e944t7w2k7h76 2459775 2459774 2026-09-23T10:06:29Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on P{dse-k8s-wdqs100[1-3].eqiad.wmnet} and (A:dse-k8s-master-eqiad or A:dse-k8s-worker-eqiad) 2459775 wikitext text/x-wiki bndu7nonp1b5canbe278ywee8ylp02a 2459780 2459775 2026-09-23T10:22:10Z Stashbot 7414 blake@deploy1003: Started scap sync-world: Upsize mw-web T438896 2459780 wikitext text/x-wiki sywvl7whjrp8gph4l4d6491yfej5mcj 2459781 2459780 2026-09-23T10:25:43Z Stashbot 7414 blake@deploy1003: Finished scap sync-world: Upsize mw-web T438896 (duration: 04m 20s) 2459781 wikitext text/x-wiki fry39dcq1k86ecq8quuava87fpom02z 2459782 2459781 2026-09-23T10:31:29Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2459782 wikitext text/x-wiki 2irtkk8x4y2fyodedftspi4vbjux3ag 2459783 2459782 2026-09-23T10:31:39Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2459783 wikitext text/x-wiki nzj8s2lz3v45qh4p03b5yi7yny1mn5a 2459784 2459783 2026-09-23T10:38:20Z Stashbot 7414 jayme: sudo confctl --quiet --object-type discovery select 'dnsdisc=mw-web-ro' set/ttl=10 - T438896 2459784 wikitext text/x-wiki e8jayulfsb39v3d71m7zcpxo21ysjw4 2459788 2459784 2026-09-23T11:25:22Z Stashbot 7414 mvolz@deploy1003: helmfile [staging] START helmfile.d/services/citoid: apply 2459788 wikitext text/x-wiki 3jronv6466c42b4z9ydey2ar6vnk2fa 2459789 2459788 2026-09-23T11:25:42Z Stashbot 7414 mvolz@deploy1003: helmfile [staging] DONE helmfile.d/services/citoid: apply 2459789 wikitext text/x-wiki 1lt45ofgm4ju0fwwbi043cgkj43wynz 2459790 2459789 2026-09-23T11:27:08Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] START helmfile.d/services/citoid: apply 2459790 wikitext text/x-wiki l8yqnsmiek09vjucqwbnxna4l5yj4zt 2459791 2459790 2026-09-23T11:27:57Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] DONE helmfile.d/services/citoid: apply 2459791 wikitext text/x-wiki fltwmrmaisp689hqbfl495xoflu35m8 2459792 2459791 2026-09-23T11:28:33Z Stashbot 7414 mvolz@deploy1003: helmfile [eqiad] START helmfile.d/services/citoid: apply 2459792 wikitext text/x-wiki 7yr6pwrv3pqm6q5n3bd6wgawm2zvbon 2459793 2459792 2026-09-23T11:31:11Z Stashbot 7414 mvolz@deploy1003: helmfile [eqiad] DONE helmfile.d/services/citoid: apply 2459793 wikitext text/x-wiki mmyvw7r5ww4m46jo684jmpu214glsgb 2459794 2459793 2026-09-23T11:51:19Z Stashbot 7414 Dreamy_Jazz: Deployed patch for T438729 2459794 wikitext text/x-wiki oty4rg722afwgu8gg8bs8rfxql5jx4w 498 9925 2459599 2458531 2026-09-22T20:09:01Z Stashbot 7414 Southparkfan: add deployment-jobrunner06 to 'jobrunner' dsh group - T438656 2459599 wikitext text/x-wiki 8bamhp0jylxvs0kebd3wq9nu84zuysh 2459631 2459599 2026-09-22T20:53:41Z Stashbot 7414 Southparkfan: remove puppetserver cherry-pick for T428052, running newer version of HAProxy nowadays 2459631 wikitext text/x-wiki 9oo7epr60zpglbxrp7sfylirtdabn03 2459642 2459631 2026-09-22T21:20:34Z Stashbot 7414 Southparkfan: add deployment-deploy06 to scap::dsh::scap_masters and deployment_hosts - T435393 2459642 wikitext text/x-wiki jbygk7zn7uviypqw0rv2uj2htkkt4tb 2459644 2459642 2026-09-22T21:23:01Z Stashbot 7414 Southparkfan: decommission deployment-cumin-3 - T436470 2459644 wikitext text/x-wiki hrcstgx0dmd6b0tyaq1vfq8xgb0upsw 2459666 2459644 2026-09-22T22:40:24Z Stashbot 7414 Southparkfan: create empty /etc/helmfile-defaults/mediawiki/release on -deploy04 and -deploy06 to unbreak Scap sync-masters - T435393 2459666 wikitext text/x-wiki 5924tqngduta82nzr7t7j4m45pag9o2 2459668 2459666 2026-09-22T22:43:51Z Stashbot 7414 Southparkfan: add wmgRedisLockPassword to PrivateSettings.php - T436480 2459668 wikitext text/x-wiki cvr37wa844h0bb2ou7nixbc6bcxqpqj 0 17290 2459531 2459226 2026-09-22T18:17:06Z Stashbot 7414 dancy: Remove deployment-mwmaint03.deployment-prep.eqiad1.wikimedia.cloud from scap::dsh::groups.mediawiki-installation.host in deployment-prep project puppet config (T435393) to stop wmf-beta-update-all spam. 2459531 wikitext text/x-wiki 7i329p994hve0zfaumxbhea1179s6rh 2459598 2459531 2026-09-22T20:08:57Z Stashbot 7414 Southparkfan: add deployment-jobrunner06 to 'jobrunner' dsh group - T438656 2459598 wikitext text/x-wiki h6xikhzkbt65yy8b89rqn3gdyjwcmx7 2459630 2459598 2026-09-22T20:53:38Z Stashbot 7414 Southparkfan: remove puppetserver cherry-pick for T428052, running newer version of HAProxy nowadays 2459630 wikitext text/x-wiki 6hnlph4wvlx7ohdb442lv37bze7q95r 2459641 2459630 2026-09-22T21:20:31Z Stashbot 7414 Southparkfan: add deployment-deploy06 to scap::dsh::scap_masters and deployment_hosts - T435393 2459641 wikitext text/x-wiki 93j4tl3m1aksmu2nh3f9xmkf3fodwcg 2459643 2459641 2026-09-22T21:22:58Z Stashbot 7414 Southparkfan: decommission deployment-cumin-3 - T436470 2459643 wikitext text/x-wiki qx4ysm4zv109yc4s8slc8pkzwc1fhsb 2459665 2459643 2026-09-22T22:40:21Z Stashbot 7414 Southparkfan: create empty /etc/helmfile-defaults/mediawiki/release on -deploy04 and -deploy06 to unbreak Scap sync-masters - T435393 2459665 wikitext text/x-wiki fju5atlq7tfunpv6vp4xt4snmkna7c7 2459667 2459665 2026-09-22T22:43:47Z Stashbot 7414 Southparkfan: add wmgRedisLockPassword to PrivateSettings.php - T436480 2459667 wikitext text/x-wiki a0zhfoff4z0yozm8sri8leunqufmpqm 0 19208 2459329 2455348 2026-09-22T12:23:36Z Hashar 32 /* Mr. Widget (Related Changes in GitLab) */ add link to GitLabPatchesCustomField + new section for Gerrit equivalent 2459329 wikitext text/x-wiki 5q75e5unj5lw10q1zun9wdlpj7in2fl 498 30942 2459354 2459065 2026-09-22T14:03:46Z Stashbot 7414 andrewbogott: destroying ceph osd.334, osd.335, osd.336 and osd.337 for testing described on T429387 2459354 wikitext text/x-wiki 4yobfr9azwad9vpyuot334au8oiy6pf 2459355 2459354 2026-09-22T14:04:39Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2459355 wikitext text/x-wiki lcr4q9lxjf59ihgpfhoe9jqgznkqzac 2459395 2459355 2026-09-22T15:17:16Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2459395 wikitext text/x-wiki psy8vw34p6f42ddwy3fjslz9cul13u4 2459399 2459395 2026-09-22T15:20:58Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2459399 wikitext text/x-wiki gt840feicge5l5jj2cohz742xqv4l96 2459446 2459399 2026-09-22T16:41:36Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2459446 wikitext text/x-wiki hpjsuh2uqteh4e96n6passla9plb5bo 2459513 2459446 2026-09-22T17:51:33Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2459513 wikitext text/x-wiki p8lx7gm1ndgt34m5sh4ha1ily2wvjrf 2459514 2459513 2026-09-22T17:51:41Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2459514 wikitext text/x-wiki a4skbfjvp92grhgbke91ag2elo3eyw3 2459515 2459514 2026-09-22T17:53:07Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2459515 wikitext text/x-wiki l8u7u12t39uv4l7d5ra04hqo0gy0obq 2459564 2459515 2026-09-22T19:19:29Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2459564 wikitext text/x-wiki fuojqbv2szvdelb4iopcflg8iyzno5w 2459565 2459564 2026-09-22T19:19:56Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2459565 wikitext text/x-wiki drcjcf53oxiuw8mkho0j5imgsfc0phl 2459629 2459565 2026-09-22T20:49:40Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2459629 wikitext text/x-wiki jrsxf4lerqbrz2vc96uynhujpj3o0up 2459687 2459629 2026-09-23T06:04:17Z Stashbot 7414 filippo@cloudcumin1001: START - Cookbook wmcs.openstack.cloudvirt.set_maintenance (T437407) 2459687 wikitext text/x-wiki k091pc2sdckqqtic1z9xvrrgmhjtv9b 2459688 2459687 2026-09-23T06:04:24Z Stashbot 7414 filippo@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.cloudvirt.set_maintenance (exit_code=0) (T437407) 2459688 wikitext text/x-wiki 7cmlaijl6nh787o7s6ry8zikws7uiz4 2459737 2459688 2026-09-23T09:30:15Z Stashbot 7414 filippo@cloudcumin1001: START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T437407) 2459737 wikitext text/x-wiki 3wjq6qwrv82rvsnsn9hrl5325sdhkq5 2459738 2459737 2026-09-23T09:30:23Z Stashbot 7414 filippo@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) (T437407) 2459738 wikitext text/x-wiki 1u3jbnxvh5pddiww53y2cr1andit2xq 10 54864 2459648 2444864 2026-09-22T21:29:05Z Wbm1058 2226 logically, [[Portal:Toolforge/About Toolforge|How Toolforge works]] should come before [[Help:Toolforge/Quickstart|Quickstart: set up and get access]]... [[Portal:Toolforge/About Toolforge#Next steps]]... flip the order 2459648 wikitext text/x-wiki e4xiov1u117nkvhm5fnjc9syz85xlmv 0 91251 2459736 2452866 2026-09-23T09:22:01Z Cathalmooney100 58598 2459736 wikitext text/x-wiki pe0udj5o78s4zyb9r5fqaf2gv9pvfv7 498 443947 2459605 2458919 2026-09-22T20:15:26Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed c00bcb6310 (push-to-deploy) and 550f1df480 (upgrade dependencies) 2459605 wikitext text/x-wiki o1klm37xs927kd4pzepm97ig4aji33p 2459607 2459605 2026-09-22T20:19:00Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed b5a93138aa (Python 3.14) 2459607 wikitext text/x-wiki 8oflnuwercelkojv56llta3m7u1b1fz 12 444256 2459662 2444109 2026-09-22T22:12:53Z Wbm1058 2226 /* Set up an SSH client and a key */ link to [[Help:Access to Toolforge instances with PuTTY and WinSCP]] again 2459662 wikitext text/x-wiki kf0j2agt0abpa2sshommx3f5vse9fke 13 446016 2459615 2070053 2026-09-22T20:36:31Z Wbm1058 2226 /* This page and the 'my first X tool' pages */ 2459615 wikitext text/x-wiki evn37kenuxfj9w4plq2y174d3wkjzg8 0 449160 2459676 2459136 2026-09-23T00:01:38Z Dexbot 30554 Bot: Updating the report 2459676 wikitext text/x-wiki k04ir4k4eher4v6b059g73voxozd181 0 451075 2459571 2459138 2026-09-22T19:25:51Z WFan (WMF) 38087 2459571 wikitext text/x-wiki nzg08340mq8x3stb37y4d43mfv4xry5 0 458585 2459776 2302735 2026-09-23T10:07:39Z BRouberol-WMF 39141 /* Identifying the physical slot associated with an OSD or device */ 2459776 wikitext text/x-wiki 4qld4pvdarhpefnqkpwzxiqj1f96npo 2459777 2459776 2026-09-23T10:09:42Z BRouberol-WMF 39141 /* Replacing Failed Disks */ 2459777 wikitext text/x-wiki 0kzzupfmqis8jr1et2tk6a349slj2hk 2459795 2459777 2026-09-23T11:55:28Z BRouberol-WMF 39141 /* Identifying the physical slot associated with an OSD or device */ 2459795 wikitext text/x-wiki hh1feyds5yea0j6u3vdosvybfut1tiq 12 458860 2459721 2458464 2026-09-23T08:47:42Z DCaro (WMF) 18271 /* Triggering a deployment from your CI runner */ added a note about the need of having already created the config 2459721 wikitext text/x-wiki oljqmi9xtc2bw32jduxn9ur8vev7pzd 0 459290 2459572 2459045 2026-09-22T19:32:58Z XCollazo-WMF 31846 /* YYYY-MM-DD NEXT TUESDAY TRAIN (REPLACE THIS AFTER DEPLOY) */ 2459572 wikitext text/x-wiki luku4x0jnjunwsuizok9unebmtmc3z7 2459649 2459572 2026-09-22T21:29:20Z AQuhen (WMF) 29318 2026-09-22 train 2459649 wikitext text/x-wiki kng4aw0w37fgeumjmc86wk6bug9h4pp 0 459699 2459589 2459214 2026-09-22T19:54:59Z Stashbot 7414 eileen: civicrm upgraded from c80b7fb5 to d575ad3e 2459589 wikitext text/x-wiki 64y7eljk7gve77j9nutscn10xe8nuwq 3 459880 2459460 2387968 2026-09-22T16:55:44Z Meno25 1446 fix 2459460 wikitext text/x-wiki pw14t8k72bdfvlrnykeikyw94m5m0k3 0 460147 2459459 2411118 2026-09-22T16:54:50Z Meno25 1446 delete 2459459 wikitext text/x-wiki mwp8wuq7ojsxvos4vfrfokxo0zvuxnb 0 460366 2459427 2459093 2026-09-22T16:01:51Z TBurmeister (WMF) 27417 Stylistic and tool reference updates 2459427 wikitext text/x-wiki jr1axywlcr0rr7ef1rcuvjqnnl57kb6 0 460375 2459416 2458155 2026-09-22T15:47:02Z TBurmeister (WMF) 27417 Update tool references 2459416 wikitext text/x-wiki laz9n45ij6kxcu6hd80obhjw9v4ztxi 2459418 2459416 2026-09-22T15:48:03Z TBurmeister (WMF) 27417 Remove errant gitlab links 2459418 wikitext text/x-wiki 7k1jwcer5tagknuobg447ghmhkn83bx 0 460376 2459406 2459095 2026-09-22T15:33:31Z TBurmeister (WMF) 27417 Remove template and update tool mentions 2459406 wikitext text/x-wiki lvem59iw5jxnux0ffmr33rdwqnfvrnl 0 460377 2459611 2459102 2026-09-22T20:32:28Z TBurmeister (WMF) 27417 Remove template 2459611 wikitext text/x-wiki 36nh9ihyqp3wuxmrdxx3t22oxsiz0zn 0 460417 2459735 2457414 2026-09-23T09:19:56Z Kevin Bazira 14737 /* KServe Predict Endpoint */ add note to use the eqiad endpoint instead of discovery following T438894 2459735 wikitext text/x-wiki m6clrvtmyywvzrad9l422j5tsqk59o5 2459739 2459735 2026-09-23T09:31:00Z ISarantopoulos-WMF 34784 2459739 wikitext text/x-wiki 5pq4ptyrbvo5oj884y5e06vs9a4yvmn 2459757 2459739 2026-09-23T09:53:03Z ISarantopoulos-WMF 34784 2459757 wikitext text/x-wiki l2pffogu1i169bm2k916cc29hq4lz9f 0 460582 2459410 2457176 2026-09-22T15:35:45Z TBurmeister (WMF) 27417 Update tool references 2459410 wikitext text/x-wiki h4u01wdqcv2jm1qt086m6z2awgo73eb 0 460600 2459366 2454667 2026-09-22T14:39:59Z AKhatun (WMF) 50461 Minor updates to scroll 2459366 wikitext text/x-wiki 348iqdx0nungwpwgb5q39gxvqxpzy71 2459367 2459366 2026-09-22T14:40:31Z AKhatun (WMF) 50461 2459367 wikitext text/x-wiki k857ghnj6cbv8u14xf1kq2geqx3a5x0 0 460624 2459332 2458636 2026-09-22T12:30:29Z MLechvien-WMF 49197 Update the state based on Moritz' feedback 2459332 wikitext text/x-wiki 25ti6bi04dv4iltrux8wdxrstsn5k5y 2 460651 2459683 2457633 2026-09-23T04:30:28Z MathXplore 42627 Requesting deletion ([[:m:Special:MyLanguage/User:TenWhile6/XReport|XReport]] v3.1c) 2459683 wikitext text/x-wiki oubsft85x2fs8oo2ba2j7xgctyw3ib4 0 460667 2459582 2459155 2026-09-22T19:49:32Z AWesterinen-WMF 54189 /* Manual mode involving nested scopes */ 2459582 wikitext text/x-wiki gyd4y3wl380j4oliwz589cgy7bjqyba 2459583 2459582 2026-09-22T19:50:17Z AWesterinen-WMF 54189 /* Manual mode retrieving rdfs:label */ 2459583 wikitext text/x-wiki 6jpawioxyjkus0vkl708afm4oo1omon 2459584 2459583 2026-09-22T19:50:35Z AWesterinen-WMF 54189 /* Manual mode retrieving several labels and a schema:description */ 2459584 wikitext text/x-wiki 3fff873sfu7ia2cax170vxc5ceg0o4q 2459585 2459584 2026-09-22T19:50:58Z AWesterinen-WMF 54189 /* Manual mode retrieving rdfs:label and skos:altLabels */ 2459585 wikitext text/x-wiki bsuzq2hvy799je7p01nk2omvibpdx77 2459586 2459585 2026-09-22T19:51:37Z AWesterinen-WMF 54189 /* Automatic mode retrieving rdfs:label, schema:description and skos:altLabel */ 2459586 wikitext text/x-wiki 05u6mjreycstw3wm9v8gcy8snt3cyyk 2459587 2459586 2026-09-22T19:52:01Z AWesterinen-WMF 54189 /* Automatic mode involving nested scopes */ 2459587 wikitext text/x-wiki f9c3h41p3treugg2f4j2fw7ecr3gr6p 2459596 2459587 2026-09-22T20:03:59Z AWesterinen-WMF 54189 /* Manual mode involving nested scopes */ 2459596 wikitext text/x-wiki 01ir1r6ayapwu0im5k93gwqtrsfxrrq 2459690 2459596 2026-09-23T06:05:28Z AWesterinen-WMF 54189 /* Manual mode involving nested scopes */ 2459690 wikitext text/x-wiki 17ulooj8yte98ath7crn61qj079tdi7 12 460669 2459682 2457608 2026-09-23T04:08:20Z SSapaty (WMF) 28955 /* How to create a basic Python Flask webservice */ add 'pipe' in the code sample template 2459682 wikitext text/x-wiki rgdqa86ytjo6tipyki7osblyubo4yo5 2459689 2459682 2026-09-23T06:04:32Z SSapaty (WMF) 28955 Add minimal setup for push-to-deploy configuration 2459689 wikitext text/x-wiki ibd0fbsltxyhtzezly27r7afkqtva7x 2459691 2459689 2026-09-23T06:06:10Z SSapaty (WMF) 28955 /* Test Push-to-Deploy */ 2459691 wikitext text/x-wiki so1waxmff0tct77o9q0p4ufk12o19gz 2459692 2459691 2026-09-23T06:07:37Z SSapaty (WMF) 28955 /* Test Push-to-Deploy */ 2459692 wikitext text/x-wiki r8cjucseb420je2wgz9nzdj9qidtovj 2459693 2459692 2026-09-23T06:08:10Z SSapaty (WMF) 28955 /* Test Push-to-Deploy */ 2459693 wikitext text/x-wiki 4nyn9fky3r99my1mcur2pej6xv122h0 2 460677 2459363 2457931 2026-09-22T14:29:45Z SSastry (WMF) 344 /* PCS */ list item ==> subheading-2 2459363 wikitext text/x-wiki 2jbiz15ft6w4czez47nm09om7n7i6uq 2459415 2459363 2026-09-22T15:42:32Z Cscott 41391 /* Discovery */ Added link to puppet file which defines this 2459415 wikitext text/x-wiki iig5p6app1almhmpexpmx3bjicfa3wi 2 460699 2459684 2459170 2026-09-23T05:32:36Z SSapaty (WMF) 28955 /* Component fails to generate config from a running job */ 2459684 wikitext text/x-wiki 9t1s9exfp6gknyovvnd8en9yttwk0qi 2459685 2459684 2026-09-23T05:32:44Z SSapaty (WMF) 28955 /* Rebuild the source manually */ 2459685 wikitext text/x-wiki 37up6ukir9ay8t24i5yletcs267xz9m 2459686 2459685 2026-09-23T05:52:48Z SSapaty (WMF) 28955 /* Configure Push-to-Deploy */ 2459686 wikitext text/x-wiki aif9yiq1ecfk75vt70jeduxitmzzxil 0 460718 2459380 2026-09-22T15:00:10Z JMonton-WMF 47887 First draft, Relative Trending. 2459380 wikitext text/x-wiki pyvgjhh2ixgl5wginuekeec8svqqcss 2459387 2459380 2026-09-22T15:10:21Z JMonton-WMF 47887 change diagram 2459387 wikitext text/x-wiki 978jvsff7birg8erz0iqkeoxxvy12mx 2459401 2459387 2026-09-22T15:25:18Z JMonton-WMF 47887 fix sql 2459401 wikitext text/x-wiki m6ibx165uwztx2nam49mc02h76k42po 2459402 2459401 2026-09-22T15:26:00Z JMonton-WMF 47887 fix sql 2459402 wikitext text/x-wiki gxa1h84ezmdn0p0dt3u4k9g0hbidvrs 2459404 2459402 2026-09-22T15:27:55Z JMonton-WMF 47887 remove query 2459404 wikitext text/x-wiki 5rjyklw3cwbj6ur6jy4lu4e0kjtj56t 2459405 2459404 2026-09-22T15:32:40Z JMonton-WMF 47887 fix PHP code for relative trending. 2459405 wikitext text/x-wiki m1ef7db9xrwmo0t6fm9naqy5hpwi92u 2459421 2459405 2026-09-22T15:51:00Z JMonton-WMF 47887 typo fix 2459421 wikitext text/x-wiki rorzodfl7r8npmnahm846kt8k978otk 2459697 2459421 2026-09-23T07:46:02Z JMonton-WMF 47887 remove navigation data platform 2459697 wikitext text/x-wiki ii9icw7nrpzih5hsjlrvql69wtnksy2 2459713 2459697 2026-09-23T08:38:12Z JMonton-WMF 47887 minor changes 2459713 wikitext text/x-wiki sdsp5l3g9otfx0c3frpcav1irxspbh6 2459715 2459713 2026-09-23T08:42:24Z JMonton-WMF 47887 minor changes 2459715 wikitext text/x-wiki buostnhc2pqkcy6ui3hnebyznohg7je 2459722 2459715 2026-09-23T08:54:06Z JMonton-WMF 47887 minor changes 2459722 wikitext text/x-wiki 5noqswf7sktrzjcpri9u53pf5c4xxkm 2459725 2459722 2026-09-23T08:57:09Z JMonton-WMF 47887 how to deploy 2459725 wikitext text/x-wiki 94i9idmsnxjbfkukz0gvjg1w5xmvc5q 2459727 2459725 2026-09-23T08:59:20Z JMonton-WMF 47887 minor changes 2459727 wikitext text/x-wiki 385c0pm5kvqwz4zl2y8oprmgvog9rac 2459729 2459727 2026-09-23T08:59:53Z JMonton-WMF 47887 minor changes 2459729 wikitext text/x-wiki bm6ekvxyv6evmcjwdskf995glivwdvb 2459769 2459729 2026-09-23T10:00:32Z JMonton-WMF 47887 link to prep pantry 2459769 wikitext text/x-wiki qat5s5sokde98srcu7dnyzs0l47n6at 2459770 2459769 2026-09-23T10:01:55Z JMonton-WMF 47887 title changes. 2459770 wikitext text/x-wiki 47hvnmkm90vvu6zdxw3g1mzm3al1zvw 2459771 2459770 2026-09-23T10:03:03Z JMonton-WMF 47887 structure change 2459771 wikitext text/x-wiki jqxowhh2k6zm9gebne0a1fh448vif47 2459787 2459771 2026-09-23T11:17:32Z JMonton-WMF 47887 reorganize sections. 2459787 wikitext text/x-wiki pndfafn02g9lg2s39f8tcve3dzokthp 0 460719 2459386 2026-09-22T15:10:16Z Mvolz (WMF) 2813 Created page with "So you'd like to allowlist [[mw:Citoid|citoid]]? That's great! Here's what you need to know: == IPs == WMF has [[IP and AS allocations|allocated IP ranges]]. The ones you'll need to allowlist for citoid are: 208.80.152.0/22 for IPv4 2620:0:860::/46 for IPv6 These resolve to WMF controlled domains (*.wikimedia.org) == User-Agents == We use both [[citoid]] and [[zotero]] to generate citations, so please allowlist both user-agents: Mozilla/5.0 (Macintosh; Intel Mac..." 2459386 wikitext text/x-wiki p3vuxr454bx4yrehdgpv21rnrd7vghg 2 460720 2459469 2026-09-22T17:06:57Z Krinkle 44 Created page with "{{Draft}} {{Image frame|content={{#chart:Wikimedia_pageviews_by_referer_class.chart}}|width=500|caption=Jan 2025.}} {{Image frame|content={{#chart:Wikimedia_pageviews_by_referer_class.chart|arg:eq=2025/01/31|arg:cols=month,external_search_engine,external_media_sites,external_other,none|title=X|subtitle=Y}}|width=500|caption=Jan 2025 (without internal).}} {{Image frame|content={{#chart:Wikimedia_pageviews_by_referer_class.chart|arg:eq=2025/01/31|arg:cols=month,external_..." 2459469 wikitext text/x-wiki hvbx83vcr1ut2ipmdfxeeblyaiitb8t 0 460721 2459694 2026-09-23T06:12:44Z Tobias Gritschacher (WMDE) 226 empty scroll for T438932 2459694 wikitext text/x-wiki 6a45915b8eiwfjjd66ps1f6bopwd4yk 2459695 2459694 2026-09-23T06:26:23Z Tobias Gritschacher (WMDE) 226 provide basic information and references 2459695 wikitext text/x-wiki 2b54liqo2mjynfpwmjk80s47h9hjedh 3 460722 2459730 2026-09-23T09:06:41Z StrikerBot 8475 Welcome to Toolforge! 2459730 wikitext text/x-wiki 2m7bevdkcpxqe53ot7s4mhhxcn1wyqc