Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.2 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 3086 2415553 2414471 2026-05-19T11:17:08Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component istio-gateway 2415553 wikitext text/x-wiki fr1vx69lg2pe1j409llwfb5gzzexp4s 2415554 2415553 2026-05-19T11:17:20Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component istio-gateway 2415554 wikitext text/x-wiki 92kpbss9jmcwnwnw9jd3vhnsd7y8o5q 0 4108 2414622 2414603 2026-05-18T12:11:32Z ScheduleDeploymentBot 37566 Add [[gerrit:1288042]] to Monday, May 18 UTC afternoon backport window 2414622 wikitext text/x-wiki dez7hwo530k8o06bpafuo3ai6zcehra 2414706 2414622 2026-05-18T13:31:46Z ScheduleDeploymentBot 37566 Add [[gerrit:1287026]] to Monday, May 18 UTC afternoon backport window 2414706 wikitext text/x-wiki d1wxg1m7i50dgz6czi8ceoetsd0k6mx 2414783 2414706 2026-05-18T14:31:45Z ScheduleDeploymentBot 37566 Add [[gerrit:1288877]] to Tuesday, May 19 UTC morning backport window 2414783 wikitext text/x-wiki ssr8zxjoqxlb55w9qt03ndvd5kz81p8 2414967 2414783 2026-05-18T17:07:20Z ScheduleDeploymentBot 37566 Add [[gerrit:1288924]] to Monday, May 18 UTC late backport window 2414967 wikitext text/x-wiki 5ziwyxzk0c1neu8xh4n4lr4gqs3tf0c 2415151 2414967 2026-05-18T21:57:58Z Mmullie (WMF) 291 /* {{Deployment_day|date=2026-05-19}} */ 2415151 wikitext text/x-wiki mk2fkgxxnh8mpbz4efe6z6hnu8irbvl 2415211 2415151 2026-05-19T03:09:05Z ScheduleDeploymentBot 37566 Add [[gerrit:1287433]] to Tuesday, May 19 UTC afternoon backport window 2415211 wikitext text/x-wiki 85tf6drri1ucuoz9aqi8im3q285l4uw 2415212 2415211 2026-05-19T03:10:37Z ScheduleDeploymentBot 37566 Add [[gerrit:1281901]] to Tuesday, May 19 UTC afternoon backport window 2415212 wikitext text/x-wiki mot5dci17jiuwka8crtpjzeyoyuju2b 2415215 2415212 2026-05-19T04:05:48Z ScheduleDeploymentBot 37566 Add [[gerrit:1289070]] to Tuesday, May 19 UTC afternoon backport window 2415215 wikitext text/x-wiki gw8uhs86ojcxcgtn27loknpqcm7bf6f 2415216 2415215 2026-05-19T04:05:59Z ScheduleDeploymentBot 37566 Add [[gerrit:1289071]] to Tuesday, May 19 UTC afternoon backport window 2415216 wikitext text/x-wiki 1dv7mf3qnvjowbr12l8qssv29tuvqtq 0 6479 2415333 2413699 2026-05-19T08:37:42Z Effie Mouzeli (WMF) 12880 /* Pair 1 (rdb1013 and rdb2007) */ 2415333 wikitext text/x-wiki hx3e2hcrtd4y4xujf9my58hhjnoi7n6 2415343 2415333 2026-05-19T08:40:49Z Effie Mouzeli (WMF) 12880 /* Pair 1 (rdb2011 - Redis 8 migration) */ 2415343 wikitext text/x-wiki j93g9r78bupjz9odk4du7xbzbt54wq4 2415349 2415343 2026-05-19T08:43:16Z Effie Mouzeli (WMF) 12880 /* Pair 1 (rdb2011 - Redis 8 migration) */ 2415349 wikitext text/x-wiki 9ksumh1kgmi1szfsn21tm0jbt1sutx6 498 6726 2415542 2414414 2026-05-19T11:11:38Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component istio-gateway 2415542 wikitext text/x-wiki 0y0n55cwt9udol4l4x2xgrameo3yo21 2415543 2415542 2026-05-19T11:11:49Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component istio-gateway 2415543 wikitext text/x-wiki 8pytz9fkw73847gdb7prmolltyaj8bp 2415548 2415543 2026-05-19T11:14:37Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component istio-gateway 2415548 wikitext text/x-wiki ockcb20i01xujne2oe3qjvs7bhj6l7a 2415549 2415548 2026-05-19T11:14:48Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component istio-gateway 2415549 wikitext text/x-wiki rnqtbc7niqbhqqflk9nlkxspxzbuqjc 2415550 2415549 2026-05-19T11:15:45Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component istio-gateway 2415550 wikitext text/x-wiki f8ueuqbgsk86yze4i5pvm0liond3ifq 2415551 2415550 2026-05-19T11:15:56Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component istio-gateway 2415551 wikitext text/x-wiki 7xt04do4cnygchgjy7kbwy4gc5svp45 0 7919 2414607 2414604 2026-05-18T12:00:11Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cumin2003.codfw.wmnet 2414607 wikitext text/x-wiki a1tp6cabnxjhossmgidrdfb66lujsau 2414611 2414607 2026-05-18T12:02:19Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1071.eqiad.wmnet with reason: host reimage 2414611 wikitext text/x-wiki 027z9v6pyt7nxbkcr9fbb7oi9fjb0i5 2414612 2414611 2026-05-18T12:02:25Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2056-2058].codfw.wmnet 2414612 wikitext text/x-wiki 8ko6qefaezyijnxytpxd08ln1tzxbc5 2414613 2414612 2026-05-18T12:02:27Z Stashbot 7414 Dreamy_Jazz: Ran `scap remove-patch --message-body 'Dropping patch already made public' /srv/patches/next/extensions/ConfirmEdit/01-T423840.patch` 2414613 wikitext text/x-wiki r63hpyuyifhq4b7m5yucc5kn9ug6hii 2414614 2414613 2026-05-18T12:02:28Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2056-2058].codfw.wmnet 2414614 wikitext text/x-wiki coc53to5cd53nufvs3x6fywnek4fmtn 2414615 2414614 2026-05-18T12:02:36Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2059-2061].codfw.wmnet 2414615 wikitext text/x-wiki fmnecz93zqp4iuhhtcmul4dki38nvm0 2414617 2414615 2026-05-18T12:04:20Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2059-2061].codfw.wmnet 2414617 wikitext text/x-wiki gj4nsl3v0ohhp8qcvnytgwyvtpio0qo 2414618 2414617 2026-05-18T12:07:09Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1069.eqiad.wmnet with reason: host reimage 2414618 wikitext text/x-wiki rpb5kyazm9ux23oqsxvjju7u1yfh68g 2414619 2414618 2026-05-18T12:09:30Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host netbox-dev2003.codfw.wmnet 2414619 wikitext text/x-wiki feqjm33b88mataa3jzuc5kh7tgl85m5 2414620 2414619 2026-05-18T12:10:10Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1072.eqiad.wmnet with OS bookworm 2414620 wikitext text/x-wiki 3ffze00msmgejsrh8rtsqnydmgdo8m2 2414621 2414620 2026-05-18T12:11:32Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2059-2061].codfw.wmnet 2414621 wikitext text/x-wiki n4v2s1uhs06hkwq11nunp20sqxq5j9v 2414623 2414621 2026-05-18T12:11:35Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2059-2061].codfw.wmnet 2414623 wikitext text/x-wiki 1mid7vhug69wf3dohdhg6y7jc36xlq1 2414624 2414623 2026-05-18T12:11:44Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2062,2064-2065].codfw.wmnet 2414624 wikitext text/x-wiki pcdhumbxnrwsckje7cr0mss88v38yee 2414625 2414624 2026-05-18T12:13:29Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2062,2064-2065].codfw.wmnet 2414625 wikitext text/x-wiki t3jwwnju3g0s4uu6t6rl28e0sh0rw3o 2414626 2414625 2026-05-18T12:13:32Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netbox-dev2003.codfw.wmnet 2414626 wikitext text/x-wiki 03thrragfrnjgd4ipgowusriyxaqnkw 2414627 2414626 2026-05-18T12:14:12Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1070.eqiad.wmnet with OS bookworm 2414627 wikitext text/x-wiki j5y0ki425b68kvu9hjnd93obgra88ke 2414630 2414627 2026-05-18T12:17:41Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host debmonitor-dev2001.codfw.wmnet 2414630 wikitext text/x-wiki 9u9cloy2f40hac7nmac2rby929poixl 2414631 2414630 2026-05-18T12:18:46Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1071.eqiad.wmnet with OS bookworm 2414631 wikitext text/x-wiki qobdqzdf3fxy8e85kznk6lgf31tarc7 2414632 2414631 2026-05-18T12:20:17Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2179 (T419635)', diff saved to https://phabricator.wikimedia.org/P92567 and previous config saved to /var/cache/conftool/dbconfig/20260518-122014-fceratto.json 2414632 wikitext text/x-wiki 3ybwabjg6m3k9alhndesjeh77m22f71 2414633 2414632 2026-05-18T12:20:43Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2062,2064-2065].codfw.wmnet 2414633 wikitext text/x-wiki 3ir4xswmyze3m2sxxe04mu23aec8cjk 2414634 2414633 2026-05-18T12:20:46Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2062,2064-2065].codfw.wmnet 2414634 wikitext text/x-wiki lv6tguy727agijh2aav5qyobvas3equ 2414636 2414634 2026-05-18T12:20:55Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2067-2069].codfw.wmnet 2414636 wikitext text/x-wiki 729tbp4ijw9i845b6qsgmysy6yf38r3 2414638 2414636 2026-05-18T12:21:43Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host debmonitor-dev2001.codfw.wmnet 2414638 wikitext text/x-wiki 80dm04d0prgsjup7sri4b7hjbprio0r 2414640 2414638 2026-05-18T12:22:08Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host debmonitor2003.codfw.wmnet 2414640 wikitext text/x-wiki tuy0ft3972b6oy4no4ybnclq075zrfo 2414641 2414640 2026-05-18T12:22:33Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1069.eqiad.wmnet with OS bookworm 2414641 wikitext text/x-wiki tu3m2tf7q5nl22yhrdhuwvuljc4ubm5 2414642 2414641 2026-05-18T12:22:38Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2067-2069].codfw.wmnet 2414642 wikitext text/x-wiki rrgdghqublw4cqpb8k493i26prna5dd 2414644 2414642 2026-05-18T12:26:09Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host debmonitor2003.codfw.wmnet 2414644 wikitext text/x-wiki 9undfey6leknwqvahjzhxnztm2j3r2g 2414645 2414644 2026-05-18T12:27:47Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host debmonitor1003.eqiad.wmnet 2414645 wikitext text/x-wiki i2gxl1tycje1fslck84tmbop1cio9lc 2414646 2414645 2026-05-18T12:30:24Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2179', diff saved to https://phabricator.wikimedia.org/P92568 and previous config saved to /var/cache/conftool/dbconfig/20260518-123022-fceratto.json 2414646 wikitext text/x-wiki l8pt36a32elt706aug1kkbnom22fqrf 2414647 2414646 2026-05-18T12:31:51Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host debmonitor1003.eqiad.wmnet 2414647 wikitext text/x-wiki lekh83ksynchvz8z113w6vua5o7ugcn 2414648 2414647 2026-05-18T12:31:54Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2067-2069].codfw.wmnet 2414648 wikitext text/x-wiki fvgwym6quz8r82ednih6y35hqh9t287 2414649 2414648 2026-05-18T12:31:57Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2067-2069].codfw.wmnet 2414649 wikitext text/x-wiki lu8mxbv89wadot56e3n0jc853nrz1x5 2414650 2414649 2026-05-18T12:32:06Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2070-2072].codfw.wmnet 2414650 wikitext text/x-wiki 51hizrhwly5qugp5u0erdjs6qcvz5qo 2414651 2414650 2026-05-18T12:34:28Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2070-2072].codfw.wmnet 2414651 wikitext text/x-wiki 1aavsyvz0kfnmi38hbp6xbsmzmq9ywn 2414652 2414651 2026-05-18T12:34:51Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast7002.wikimedia.org 2414652 wikitext text/x-wiki 50cjb4z474ru7d6btc9amkbp01pid71 2414653 2414652 2026-05-18T12:40:31Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2179', diff saved to https://phabricator.wikimedia.org/P92569 and previous config saved to /var/cache/conftool/dbconfig/20260518-124030-fceratto.json 2414653 wikitext text/x-wiki au8zqi2c414882w4qf2iz0f1451oo6e 2414654 2414653 2026-05-18T12:40:57Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast7002.wikimedia.org 2414654 wikitext text/x-wiki ng2k4bvlcclbzdf9jg0xzwo78s282dr 2414655 2414654 2026-05-18T12:41:48Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2070-2072].codfw.wmnet 2414655 wikitext text/x-wiki ca4l46lt7qi154a6iwf4qns3u9mgzeb 2414656 2414655 2026-05-18T12:41:51Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2070-2072].codfw.wmnet 2414656 wikitext text/x-wiki g078def1ivcbom7hxe1m3a6hghtb97n 2414657 2414656 2026-05-18T12:41:59Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2073-2075].codfw.wmnet 2414657 wikitext text/x-wiki 00rrrz9xas06g23jm3ac68rkwc240ln 2414658 2414657 2026-05-18T12:43:44Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2073-2075].codfw.wmnet 2414658 wikitext text/x-wiki 8pt117hadye3piihqzdklyh8c1dlclb 2414659 2414658 2026-05-18T12:50:37Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2073-2075].codfw.wmnet 2414659 wikitext text/x-wiki hfdlkh1zlgax7kas9pmthu5bevntlr1 2414660 2414659 2026-05-18T12:50:41Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2179 (T419635)', diff saved to https://phabricator.wikimedia.org/P92570 and previous config saved to /var/cache/conftool/dbconfig/20260518-125038-fceratto.json 2414660 wikitext text/x-wiki ebl29hi6flik3cwzja97utwayhtlbps 2414661 2414660 2026-05-18T12:50:43Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2073-2075].codfw.wmnet 2414661 wikitext text/x-wiki b365xyw6z2suj3dpyjxkwsziy9pkz5h 2414662 2414661 2026-05-18T12:50:56Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2076-2078].codfw.wmnet 2414662 wikitext text/x-wiki kj1n0lvcdzaavdie1iq0d6qwqp60l8i 2414663 2414662 2026-05-18T12:51:51Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host netflow7002.magru.wmnet 2414663 wikitext text/x-wiki rjav0h4yr7nyy4s4t400pedmlgw7lku 2414664 2414663 2026-05-18T12:56:04Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1288494|hCaptcha: Drop addurl trigger and 100% passive mode SiteKey (T426587)]] 2414664 wikitext text/x-wiki hxkcx75fqrn0fsoiiewxpnppwn0nq0o 2414665 2414664 2026-05-18T12:56:11Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netflow7002.magru.wmnet 2414665 wikitext text/x-wiki q8u5kpueexc66saeakiyzo24679noy9 2414666 2414665 2026-05-18T12:57:07Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2076-2078].codfw.wmnet 2414666 wikitext text/x-wiki ivb2md2uy0panxnzxsdlxg3gxfncvyz 2414667 2414666 2026-05-18T12:59:47Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1288494|hCaptcha: Drop addurl trigger and 100% passive mode SiteKey (T426587)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2414667 wikitext text/x-wiki b1fp8u6r0b72xqf0qxikohhyim4aasf 2414668 2414667 2026-05-18T13:00:50Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2414668 wikitext text/x-wiki 5lekkx0celg3d2gjzkxsiwoglq8zl7v 2414669 2414668 2026-05-18T13:03:23Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host netflow6001.drmrs.wmnet 2414669 wikitext text/x-wiki 3dy5chsrt9rs4fi1rp3s5yu63tbfrc8 2414670 2414669 2026-05-18T13:03:45Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2076-2078].codfw.wmnet 2414670 wikitext text/x-wiki alqx2ra1eq12k80o5dhm88zcac0d1qz 2414671 2414670 2026-05-18T13:03:48Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2076-2078].codfw.wmnet 2414671 wikitext text/x-wiki b1li4pl1v72zgqxmamrhqwe1stk0jnp 2414672 2414671 2026-05-18T13:03:57Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2087-2089].codfw.wmnet 2414672 wikitext text/x-wiki ibc3w3b6186cn3mnwk0fr1dfp201jbh 2414673 2414672 2026-05-18T13:05:43Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2087-2089].codfw.wmnet 2414673 wikitext text/x-wiki jjb9s0mah4of5ft78u1jqoko3e6u3ao 2414674 2414673 2026-05-18T13:07:27Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1288494|hCaptcha: Drop addurl trigger and 100% passive mode SiteKey (T426587)]] (duration: 11m 27s) 2414674 wikitext text/x-wiki tqe73j8dvggatyzbn2yroxtt1hxyeui 2414675 2414674 2026-05-18T13:07:31Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netflow6001.drmrs.wmnet 2414675 wikitext text/x-wiki frosmkqm7n21gvc08g7equfhiaiokpr 2414676 2414675 2026-05-18T13:10:09Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Started scap sync-world: Backport for [[gerrit:1288487|fix(signup.js): Do not warn about a username being available (T419401)]] 2414676 wikitext text/x-wiki phmmz3qt2rih9vhl8kyll1u9aanubz4 2414677 2414676 2026-05-18T13:11:53Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde, sgimeno: Backport for [[gerrit:1288487|fix(signup.js): Do not warn about a username being available (T419401)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2414677 wikitext text/x-wiki c5oi86jpxxxzb449s43vtkxh9voqaa5 2414679 2414677 2026-05-18T13:12:53Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2087-2089].codfw.wmnet 2414679 wikitext text/x-wiki mprfuncepzk7p2fa0owidcbffw1vx72 2414680 2414679 2026-05-18T13:12:56Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2087-2089].codfw.wmnet 2414680 wikitext text/x-wiki p4ibfu1r9wpz7e2dnhi818ujdyk1x1a 2414681 2414680 2026-05-18T13:13:06Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2090-2092].codfw.wmnet 2414681 wikitext text/x-wiki 47i6n4bd8h2hclkquzb1amth9898x7v 2414682 2414681 2026-05-18T13:13:38Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc1068.eqiad.wmnet with OS bookworm 2414682 wikitext text/x-wiki 0l64smde926mu65v560gkmd88vo6k64 2414683 2414682 2026-05-18T13:14:48Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2090-2092].codfw.wmnet 2414683 wikitext text/x-wiki 3zmz4ev7juk0ed3rw7wztirqyaz959c 2414684 2414683 2026-05-18T13:15:18Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde, sgimeno: Continuing with deployment 2414684 wikitext text/x-wiki qoq6p9vnfj76qlqgvn99o3ci6ibptat 2414685 2414684 2026-05-18T13:16:09Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc1067.eqiad.wmnet with OS bookworm 2414685 wikitext text/x-wiki 6dtprmkvvrub3qz974yx3c0oq09vjt8 2414686 2414685 2026-05-18T13:18:52Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2414686 wikitext text/x-wiki h30f4l2lglvtg3904xuted4j4h1df7s 2414687 2414686 2026-05-18T13:19:27Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Finished scap sync-world: Backport for [[gerrit:1288487|fix(signup.js): Do not warn about a username being available (T419401)]] (duration: 09m 18s) 2414687 wikitext text/x-wiki si96q6w3srjsqcakcm1n7hspf8s3vej 2414688 2414687 2026-05-18T13:19:59Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2414688 wikitext text/x-wiki brpsog5ephamftno2bcocqa8iq7e88r 2414689 2414688 2026-05-18T13:21:06Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2414689 wikitext text/x-wiki nkx0drkhu3ezgs5axtl1id6lablze6n 2414690 2414689 2026-05-18T13:21:36Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2090-2092].codfw.wmnet 2414690 wikitext text/x-wiki mmrac875jqzwidnvugvzj9hfel17mif 2414691 2414690 2026-05-18T13:21:38Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host netflow5003.eqsin.wmnet 2414691 wikitext text/x-wiki 7vh17ham660mk1nzo7u0es5ow0hfeph 2414692 2414691 2026-05-18T13:21:39Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2090-2092].codfw.wmnet 2414692 wikitext text/x-wiki 1wdoqfn9ycm0rmlrkaxfz86ackq6dm1 2414693 2414692 2026-05-18T13:21:42Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Started scap sync-world: Backport for [[gerrit:1288042|[config] Set Category Collation for arwikisource (T426526)]] 2414693 wikitext text/x-wiki 127zlvbsl1eremm7hklhpphhoflieiy 2414694 2414693 2026-05-18T13:21:47Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2093-2095].codfw.wmnet 2414694 wikitext text/x-wiki luxvmxu3e0y6fg09vws5n6a8bn7nzcc 2414695 2414694 2026-05-18T13:21:58Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2414695 wikitext text/x-wiki dshvy6jheg2wm1thbiocbp7k9xib12k 2414696 2414695 2026-05-18T13:23:24Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: hubaishan, lucaswerkmeister-wmde: Backport for [[gerrit:1288042|[config] Set Category Collation for arwikisource (T426526)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2414696 wikitext text/x-wiki 5tf8str112tm1fzt7ja3o05hazli7g5 2414697 2414696 2026-05-18T13:23:33Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2093-2095].codfw.wmnet 2414697 wikitext text/x-wiki nawlm90dvh4bwgkzaolub7amyezy4is 2414698 2414697 2026-05-18T13:23:49Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netflow5003.eqsin.wmnet 2414698 wikitext text/x-wiki 8zm4ndw8je53gxyuq7oaxy7n3awua0j 2414699 2414698 2026-05-18T13:25:45Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc1068.eqiad.wmnet with reason: host reimage 2414699 wikitext text/x-wiki r0lin775ows7qb8bpyadmolm631sjig 2414700 2414699 2026-05-18T13:27:41Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc1067.eqiad.wmnet with reason: host reimage 2414700 wikitext text/x-wiki t8h1v5i0koiijvuh42wba4hklbvyhdy 2414701 2414700 2026-05-18T13:28:51Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: hubaishan, lucaswerkmeister-wmde: Continuing with deployment 2414701 wikitext text/x-wiki 30tk6wlu60zxpwoa3bssgbcl8w3isp5 2414702 2414701 2026-05-18T13:29:25Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1068.eqiad.wmnet with reason: host reimage 2414702 wikitext text/x-wiki rmg0a77fyf4qwg2jjx36ta522wkimp0 2414703 2414702 2026-05-18T13:30:52Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2093-2095].codfw.wmnet 2414703 wikitext text/x-wiki m9jjhq3s85xec2ycfjyrkytlto0hk8q 2414704 2414703 2026-05-18T13:30:55Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2093-2095].codfw.wmnet 2414704 wikitext text/x-wiki 2y2gh4elufpa1ys8l5p9nzylub9z4d7 2414705 2414704 2026-05-18T13:31:03Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2102-2104].codfw.wmnet 2414705 wikitext text/x-wiki 88u03pym06t5nejk0jycqt75fhee10g 2414707 2414705 2026-05-18T13:33:04Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Finished scap sync-world: Backport for [[gerrit:1288042|[config] Set Category Collation for arwikisource (T426526)]] (duration: 11m 24s) 2414707 wikitext text/x-wiki 1nbcyu54z8zq8ypmfwy9s94z9vibcuq 2414708 2414707 2026-05-18T13:33:11Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1067.eqiad.wmnet with reason: host reimage 2414708 wikitext text/x-wiki dz4xwg706u8yb3q7z8i9ysam1chwk74 2414709 2414708 2026-05-18T13:33:17Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: mwscript-k8s job started: updateCollation arwikisource --previous-collation=uppercase # T426526 2414709 wikitext text/x-wiki lae3dgi7l9t81iz9tgl8dg0hzgxp8o6 2414710 2414709 2026-05-18T13:33:26Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db2218'] 2414710 wikitext text/x-wiki rst2wd5a36axb4g8rgi35lr9kj28dol 2414711 2414710 2026-05-18T13:33:44Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['db2218'] 2414711 wikitext text/x-wiki b5w62rdpm43vzz3bi709nekuiqx9cez 2414712 2414711 2026-05-18T13:34:15Z Stashbot 7414 tchin@deploy1003: Started deploy [analytics/refinery@ba10fca] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@ba10fcad] 2414712 wikitext text/x-wiki 5v8vh5uoowk5kp7pdgbw33wg6gy7vti 2414713 2414712 2026-05-18T13:36:09Z Stashbot 7414 tchin@deploy1003: Finished deploy [analytics/refinery@ba10fca] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@ba10fcad] (duration: 01m 54s) 2414713 wikitext text/x-wiki 5rwiv1peqntzlyo34vf4m65n6u97vhp 2414714 2414713 2026-05-18T13:36:52Z Stashbot 7414 tchin@deploy1003: Started deploy [analytics/refinery@ba10fca]: Regular analytics weekly train [analytics/refinery@ba10fcad] 2414714 wikitext text/x-wiki h4p16qz5x3vytjay2to9vzvxho8jwtr 2414715 2414714 2026-05-18T13:37:18Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2102-2104].codfw.wmnet 2414715 wikitext text/x-wiki iyi2qwbaa3l56m0bk2zc1lx2rw5avms 2414717 2414715 2026-05-18T13:40:06Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2414717 wikitext text/x-wiki j26zzjvewuflqvr5b498g1fgbkcxum3 2414718 2414717 2026-05-18T13:40:37Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2414718 wikitext text/x-wiki kzrvwawfhvrkdzoorqlyhvlaek7bwkb 2414719 2414718 2026-05-18T13:41:15Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2414719 wikitext text/x-wiki i47uifh3n9a8j877kkr97v8th8xxfnp 2414720 2414719 2026-05-18T13:41:40Z Stashbot 7414 Lucas_WMDE: updateCollation arwikisource for T426526 finished 2414720 wikitext text/x-wiki d4o7eok73cciwkid6rrsf7ujqbfbhog 2414721 2414720 2026-05-18T13:41:58Z Stashbot 7414 tchin@deploy1003: Finished deploy [analytics/refinery@ba10fca]: Regular analytics weekly train [analytics/refinery@ba10fcad] (duration: 05m 05s) 2414721 wikitext text/x-wiki hije1zjon3p5swhwc3aeusovy02wusk 2414722 2414721 2026-05-18T13:42:08Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2414722 wikitext text/x-wiki 7psfgvfqw2tu3ma7diqduef2g3k36mv 2414723 2414722 2026-05-18T13:42:10Z Stashbot 7414 tchin@deploy1003: Started deploy [analytics/refinery@ba10fca] (thin): Regular analytics weekly train THIN [analytics/refinery@ba10fcad] 2414723 wikitext text/x-wiki 72wjftttmeaecppxfi44z34po0ok9n8 2414724 2414723 2026-05-18T13:44:06Z Stashbot 7414 tchin@deploy1003: Finished deploy [analytics/refinery@ba10fca] (thin): Regular analytics weekly train THIN [analytics/refinery@ba10fcad] (duration: 01m 55s) 2414724 wikitext text/x-wiki 0lv8976skplbfkjex5gehu89bsx6aiz 2414725 2414724 2026-05-18T13:44:29Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2102-2104].codfw.wmnet 2414725 wikitext text/x-wiki l58lm1ggltg4vgpthhfny6asy8vhsrc 2414726 2414725 2026-05-18T13:44:31Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2102-2104].codfw.wmnet 2414726 wikitext text/x-wiki i3dydqgy68r61cre6idcmugllqz1y0p 2414727 2414726 2026-05-18T13:44:39Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2105-2107].codfw.wmnet 2414727 wikitext text/x-wiki 0kyk1iujdfo6mtxmnayf42dytsdf4gr 2414728 2414727 2026-05-18T13:44:50Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1068.eqiad.wmnet with OS bookworm 2414728 wikitext text/x-wiki htnb6g14qjmrd1d9lplpx0145a0rtve 2414729 2414728 2026-05-18T13:46:06Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Started scap sync-world: Backport for [[gerrit:1287895|Store uncomputed references delta as null, not 0 (T426002)]], [[gerrit:1287026|.gitignore: Add /static/hcaptcha/ (T403829)]] 2414729 wikitext text/x-wiki p15n3ivn5mzj0af4m0ccijy8m0d6t98 2414730 2414729 2026-05-18T13:46:24Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2105-2107].codfw.wmnet 2414730 wikitext text/x-wiki 0tmwvohww4mgdcmlo1x8oleut0g4apn 2414731 2414730 2026-05-18T13:47:55Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: daimona, lucaswerkmeister-wmde, dancy: Backport for [[gerrit:1287895|Store uncomputed references delta as null, not 0 (T426002)]], [[gerrit:1287026|.gitignore: Add /static/hcaptcha/ (T403829)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2414731 wikitext text/x-wiki 6j3my962c55lb6y6hdeax0wv61vv3d8 2414732 2414731 2026-05-18T13:48:40Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1067.eqiad.wmnet with OS bookworm 2414732 wikitext text/x-wiki m5ysjm2lqmaz76h186cjst0fjegtxqw 2414733 2414732 2026-05-18T13:50:11Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.swift.roll-restart-reboot-swift-thanos-proxies rolling reboot on A:thanos-fe 2414733 wikitext text/x-wiki mhb94e2b8v46b93md8kwuw9b6zjw4q3 2414734 2414733 2026-05-18T13:51:45Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: daimona, lucaswerkmeister-wmde, dancy: Continuing with deployment 2414734 wikitext text/x-wiki 08qfg6jfdpz13oyj5r3w1ybzm0mfhi8 2414735 2414734 2026-05-18T13:53:03Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2105-2107].codfw.wmnet 2414735 wikitext text/x-wiki eeurllbyv93mhogeevq43b5p66bjmeh 2414736 2414735 2026-05-18T13:53:06Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2105-2107].codfw.wmnet 2414736 wikitext text/x-wiki 6jcvlv0ldgzzhvawl8l3848l6dfusz3 2414737 2414736 2026-05-18T13:53:15Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2108-2110].codfw.wmnet 2414737 wikitext text/x-wiki hga7lpoconalfhnccyug6y7qedsgk94 2414739 2414737 2026-05-18T13:54:35Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc1066.eqiad.wmnet with OS bookworm 2414739 wikitext text/x-wiki blxqqvd5f67zopc1ktenisvw67zzk5e 2414740 2414739 2026-05-18T13:54:49Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc1065.eqiad.wmnet with OS bookworm 2414740 wikitext text/x-wiki p6ma6shu0hrt4kjy4poqae934vdd8ex 2414741 2414740 2026-05-18T13:55:35Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2108-2110].codfw.wmnet 2414741 wikitext text/x-wiki ngpr1wx72tb1cvmqo1yof7blugk3fx5 2414742 2414741 2026-05-18T13:56:04Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Finished scap sync-world: Backport for [[gerrit:1287895|Store uncomputed references delta as null, not 0 (T426002)]], [[gerrit:1287026|.gitignore: Add /static/hcaptcha/ (T403829)]] (duration: 09m 57s) 2414742 wikitext text/x-wiki 8bkrs1dvemvub0mhqfz7tzpwe34a2ug 2414743 2414742 2026-05-18T13:58:53Z Stashbot 7414 bking@deploy1003: Started deploy [wdqs/wdqs@e8fb00c]: 0.3.163 2414743 wikitext text/x-wiki qkt9o4q8vxqn8tmrcgs89vddh4a2n1y 2414744 2414743 2026-05-18T14:02:16Z Stashbot 7414 mlitn@deploy1003: Started scap sync-world: Backport for [[gerrit:1288504|Squashed diff to master]] 2414744 wikitext text/x-wiki mr81c4uniol4hbk3hel44yn36pm25fo 2414745 2414744 2026-05-18T14:03:21Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2108-2110].codfw.wmnet 2414745 wikitext text/x-wiki cw7rdlg08fdz7fk19rh5ndi5lpwsolm 2414746 2414745 2026-05-18T14:03:24Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2108-2110].codfw.wmnet 2414746 wikitext text/x-wiki dklpm47479stu5b64y9p1fumq2jlcwm 2414747 2414746 2026-05-18T14:03:32Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2111-2113].codfw.wmnet 2414747 wikitext text/x-wiki on3nisueurpwcvxpnqycvvap4b60a5j 2414748 2414747 2026-05-18T14:05:32Z Stashbot 7414 mlitn@deploy1003: mlitn: Backport for [[gerrit:1288504|Squashed diff to master]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2414748 wikitext text/x-wiki 4is75ihaxuf9kroi7ci9bkkqsiyw36v 2414749 2414748 2026-05-18T14:06:17Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc1066.eqiad.wmnet with reason: host reimage 2414749 wikitext text/x-wiki cvguubog13kpxi7koatwh14run2xx9h 2414750 2414749 2026-05-18T14:06:23Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc1065.eqiad.wmnet with reason: host reimage 2414750 wikitext text/x-wiki 8zjdpj9695fx9walzpdfi4tgj1dqhwp 2414751 2414750 2026-05-18T14:08:25Z Stashbot 7414 mlitn@deploy1003: mlitn: Continuing with deployment 2414751 wikitext text/x-wiki iocxds5x8u3fm30z3fq56xgqyefv5a4 2414752 2414751 2026-05-18T14:09:22Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1066.eqiad.wmnet with reason: host reimage 2414752 wikitext text/x-wiki deycxttj729kiz8paonk8vslkxqqjmo 2414753 2414752 2026-05-18T14:09:46Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2111-2113].codfw.wmnet 2414753 wikitext text/x-wiki nkep51kii4uk5l4wzqqd1d17wo78e4i 2414754 2414753 2026-05-18T14:10:03Z Stashbot 7414 Amir1: mapping thumbsize of 0 to 2 in all group1 wikis (T376152) 2414754 wikitext text/x-wiki ck2iqec2bx61iiv6wdg9ri6rla7j43w 2414755 2414754 2026-05-18T14:11:32Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.cassandra.roll-restart for nodes matching A:sessionstore: Restart for upgrade to JVM 11.0.31 - eevans@cumin1003 2414755 wikitext text/x-wiki q9vky2mgyj5lqyt2nztua3dfh8jg0fj 2414756 2414755 2026-05-18T14:13:28Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1065.eqiad.wmnet with reason: host reimage 2414756 wikitext text/x-wiki 877jh93lwl4h2bvbfaom64u1hgggpyc 2414757 2414756 2026-05-18T14:13:59Z Stashbot 7414 mlitn@deploy1003: Rolling back deployment 2414757 wikitext text/x-wiki p4x59x5r54beul2yf2hd5pqaslm3gwo 2414758 2414757 2026-05-18T14:16:06Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.pool pool db2179: Repooling after switchover 2414758 wikitext text/x-wiki j4tr2a6ex5uexagsrdznx8wqfs2vlqc 2414759 2414758 2026-05-18T14:16:12Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2179: Repooling after switchover 2414759 wikitext text/x-wiki fgh5jk1m3gl4hy78nhefmq0k04wf0f9 2414760 2414759 2026-05-18T14:16:41Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2111-2113].codfw.wmnet 2414760 wikitext text/x-wiki 00u7mh6trlz6hndcp65vyrnoalu79h8 2414761 2414760 2026-05-18T14:16:44Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2111-2113].codfw.wmnet 2414761 wikitext text/x-wiki 61bkd9kkgav8ptjpwldrywiihw7etnj 2414762 2414761 2026-05-18T14:16:53Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling reboot on A:swift-fe 2414762 wikitext text/x-wiki bv4lhiu2y1w7nnnjuqxj5bn0rd46kl4 2414763 2414762 2026-05-18T14:16:55Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2114-2115,2124].codfw.wmnet 2414763 wikitext text/x-wiki c182ddui4iu7cu4jd5y17lvhwabl72k 2414764 2414763 2026-05-18T14:18:12Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.pool pool db2192: Repooling after switchover 2414764 wikitext text/x-wiki 9mcgg5b5gneb1w7ahifhfzagy166p7z 2414765 2414764 2026-05-18T14:18:17Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2192: Repooling after switchover 2414765 wikitext text/x-wiki p28u23e8yac23318vzz5epk0q533hv5 2414766 2414765 2026-05-18T14:18:37Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2114-2115,2124].codfw.wmnet 2414766 wikitext text/x-wiki pdiiky0tidqtouz1979zey3ywj0qpix 2414767 2414766 2026-05-18T14:22:21Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2414767 wikitext text/x-wiki 7pnw2871sd81fjdnw583h93suenjx1c 2414768 2414767 2026-05-18T14:22:23Z Stashbot 7414 mlitn@deploy1003: Finished scap sync-world: Backport for [[gerrit:1288504|Squashed diff to master]] (duration: 20m 05s) 2414768 wikitext text/x-wiki l2bgjkoh3le1l42rzvna8zqfdkx87gf 2414769 2414768 2026-05-18T14:22:52Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2414769 wikitext text/x-wiki e70ry7zxajs1y1gwmecgnl09w12rox9 2414770 2414769 2026-05-18T14:23:15Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2414770 wikitext text/x-wiki 867oo1danbn4ekjziakdki9jd4m5lin 2414771 2414770 2026-05-18T14:24:08Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2414771 wikitext text/x-wiki o98ngny4mhl2nn80yocrnra07di9bvt 2414772 2414771 2026-05-18T14:25:02Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1066.eqiad.wmnet with OS bookworm 2414772 wikitext text/x-wiki 2j277ch2co9nuh8aw2akusq3tltcvdq 2414773 2414772 2026-05-18T14:25:39Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2114-2115,2124].codfw.wmnet 2414773 wikitext text/x-wiki 5ovf6zex9qqpuvmd1t8dn5muvp14lf1 2414774 2414773 2026-05-18T14:25:42Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2114-2115,2124].codfw.wmnet 2414774 wikitext text/x-wiki eevpgt4tkca5lqzda8376t8f1jkfndp 2414775 2414774 2026-05-18T14:25:55Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2125-2127].codfw.wmnet 2414775 wikitext text/x-wiki 2xb0p1dw1hp4ud6wwu25jq8tedau96b 2414778 2414775 2026-05-18T14:27:46Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2125-2127].codfw.wmnet 2414778 wikitext text/x-wiki tg1lzq6g1xokwxszjh9uvbp84c4s833 2414779 2414778 2026-05-18T14:29:48Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1065.eqiad.wmnet with OS bookworm 2414779 wikitext text/x-wiki 1yureobwix6wypqc6qwbqjpx1aveoqd 2414780 2414779 2026-05-18T14:30:41Z Stashbot 7414 bking@deploy1003: Finished deploy [wdqs/wdqs@e8fb00c]: 0.3.163 (duration: 31m 47s) 2414780 wikitext text/x-wiki ab28it9au1t1clm0zcc45nggdo6m44l 2414781 2414780 2026-05-18T14:31:35Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.reboot-nodes rolling reboot on P{wikikube-worker[1328-1384].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) 2414781 wikitext text/x-wiki jppzva615unvfzd5r7nxkkrw977x5qo 2414782 2414781 2026-05-18T14:31:42Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1328-1330].eqiad.wmnet 2414782 wikitext text/x-wiki jok54tuymw6vofowhmoqm7gpsgqlwv8 2414784 2414782 2026-05-18T14:32:23Z Stashbot 7414 cwilliams@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on db2143.codfw.wmnet with reason: Depooled host, will be decommissioned 2414784 wikitext text/x-wiki clfe2ctx8hsry5mn3wo5bjhflw1nxn8 2414785 2414784 2026-05-18T14:32:39Z Stashbot 7414 cwilliams@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on db2149.codfw.wmnet with reason: Depooled host, will be decommissioned 2414785 wikitext text/x-wiki q6rozfpntrvh54l0hxk7q9qohcvjkbn 2414786 2414785 2026-05-18T14:33:22Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Set db2192 weight', diff saved to https://phabricator.wikimedia.org/P92573 and previous config saved to /var/cache/conftool/dbconfig/20260518-143320-fceratto.json 2414786 wikitext text/x-wiki 1gmq7vdeob7rr0rml52oxdi8ctrm0l3 2414787 2414786 2026-05-18T14:33:25Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1328-1330].eqiad.wmnet 2414787 wikitext text/x-wiki 3v6wxkhw1t9skmvxj3d5mkhgbbhvc0k 2414788 2414787 2026-05-18T14:34:16Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2125-2127].codfw.wmnet 2414788 wikitext text/x-wiki aodrglgn2x0i1hslyk74k4x9d1hccjk 2414789 2414788 2026-05-18T14:34:19Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2125-2127].codfw.wmnet 2414789 wikitext text/x-wiki lwi3noxz8adyhn6tcj2ks1b4jewwuu3 2414790 2414789 2026-05-18T14:34:28Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2128-2130].codfw.wmnet 2414790 wikitext text/x-wiki net1gybjxctuw7tqwoonxp8m89pevlp 2414791 2414790 2026-05-18T14:36:13Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2128-2130].codfw.wmnet 2414791 wikitext text/x-wiki 0tq3zu1bsf43ecf4i0nanuv8uqw5hkg 2414792 2414791 2026-05-18T14:38:28Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1328-1330].eqiad.wmnet 2414792 wikitext text/x-wiki 9ilit7ehhe3s8aj6zwf930logtxr7da 2414793 2414792 2026-05-18T14:38:30Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1328-1330].eqiad.wmnet 2414793 wikitext text/x-wiki awmum87bm2l3zy013xsxy2fhflpl48r 2414794 2414793 2026-05-18T14:38:38Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1331-1333].eqiad.wmnet 2414794 wikitext text/x-wiki fsufhxpjyewf8vj8uq4vhpvfyish8yg 2414795 2414794 2026-05-18T14:40:20Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1331-1333].eqiad.wmnet 2414795 wikitext text/x-wiki jbjmbw0ku5hdcz504uiytckrg2g947h 2414796 2414795 2026-05-18T14:42:13Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-thanos-proxies (exit_code=0) rolling reboot on A:thanos-fe 2414796 wikitext text/x-wiki 3anycjuiub5a92222knxuqvgjdeqlsq 2414797 2414796 2026-05-18T14:43:13Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2128-2130].codfw.wmnet 2414797 wikitext text/x-wiki 7qqjg9jd26pslh9fm1jbqmcql2zw4yj 2414798 2414797 2026-05-18T14:43:18Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2128-2130].codfw.wmnet 2414798 wikitext text/x-wiki iaibampunxwtzl4hhygk6qykgltoh0i 2414799 2414798 2026-05-18T14:43:25Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2131-2133].codfw.wmnet 2414799 wikitext text/x-wiki be0rrk0ithm7p6ruj2gy6k9tuty9kan 2414800 2414799 2026-05-18T14:43:46Z Stashbot 7414 Daimona: Running queries to fixup data for T426002 2414800 wikitext text/x-wiki kh8ilpwanj7oytxqxghuf3emt65b5d1 2414801 2414800 2026-05-18T14:45:10Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2131-2133].codfw.wmnet 2414801 wikitext text/x-wiki d3pk7lib5hswvmbit25e0eo043lstr4 2414802 2414801 2026-05-18T14:47:07Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1331-1333].eqiad.wmnet 2414802 wikitext text/x-wiki 4t9r2xmr8xido5vivvg6s7yxwca5y63 2414803 2414802 2026-05-18T14:47:09Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1331-1333].eqiad.wmnet 2414803 wikitext text/x-wiki 5d7cpufey2l2yrz2vpreaju68mneepd 2414804 2414803 2026-05-18T14:47:14Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching A:sessionstore: Restart for upgrade to JVM 11.0.31 - eevans@cumin1003 2414804 wikitext text/x-wiki 4c9axk32cko8ebnw75ak5ex2ixzqrql 2414805 2414804 2026-05-18T14:47:18Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1334-1336].eqiad.wmnet 2414805 wikitext text/x-wiki r2wi7ovu0ud0wythfcjf785ylyap71a 2414806 2414805 2026-05-18T14:48:06Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.kafka.roll-restart-reboot-brokers rolling reboot on A:kafka-logging-codfw 2414806 wikitext text/x-wiki 48ok2xmq71wwp3rg33ue0p8omhondqf 2414807 2414806 2026-05-18T14:48:58Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1334-1336].eqiad.wmnet 2414807 wikitext text/x-wiki 1gsob8zh4cbe440oxe2wfi9dw0h7kx8 2414808 2414807 2026-05-18T14:52:25Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2131-2133].codfw.wmnet 2414808 wikitext text/x-wiki 1058osylao0gho9n94wy7wuabsj795h 2414809 2414808 2026-05-18T14:52:27Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2131-2133].codfw.wmnet 2414809 wikitext text/x-wiki iuu34p3jki29rkfx3xeqpf83sgoyj7o 2414810 2414809 2026-05-18T14:52:36Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2134-2136].codfw.wmnet 2414810 wikitext text/x-wiki hczqoktrk6z85rwurb4esc3spqzq49w 2414811 2414810 2026-05-18T14:54:21Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2134-2136].codfw.wmnet 2414811 wikitext text/x-wiki 0ddxwx0nafxhc7b3qv22iqudw4bd8c1 2414812 2414811 2026-05-18T14:55:45Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1334-1336].eqiad.wmnet 2414812 wikitext text/x-wiki b2lni133jy7wpt8lbdkinsc3hw1dr8w 2414813 2414812 2026-05-18T14:55:49Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1334-1336].eqiad.wmnet 2414813 wikitext text/x-wiki dvjy2b3tnjpsmv9sh3ma5lpraio7sda 2414814 2414813 2026-05-18T14:55:56Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1337-1339].eqiad.wmnet 2414814 wikitext text/x-wiki 47ekloc4ntsg96lcbifgexz3l9xtzsg 2414815 2414814 2026-05-18T14:57:35Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1337-1339].eqiad.wmnet 2414815 wikitext text/x-wiki 7fo3w5wme4dy6d6qvi9bh3kcco935tq 2414816 2414815 2026-05-18T15:01:39Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2134-2136].codfw.wmnet 2414816 wikitext text/x-wiki 20bm8nx78bqwqhkz0yx0pdytm30guns 2414817 2414816 2026-05-18T15:01:41Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2134-2136].codfw.wmnet 2414817 wikitext text/x-wiki r5lqchz9smmmfhodmw6o85hl89u1a7a 2414818 2414817 2026-05-18T15:01:51Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2137-2139].codfw.wmnet 2414818 wikitext text/x-wiki 9thls5xfmut4ijpwx7vyynkokl1su01 2414819 2414818 2026-05-18T15:01:53Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host thanos-be1005.eqiad.wmnet 2414819 wikitext text/x-wiki h9snk16b4jsynz49vufjyxlmswcpf21 2414820 2414819 2026-05-18T15:03:36Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2137-2139].codfw.wmnet 2414820 wikitext text/x-wiki hpwx6wh6zl3jaab8do4bj8srf2wdpcc 2414821 2414820 2026-05-18T15:04:09Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1337-1339].eqiad.wmnet 2414821 wikitext text/x-wiki e92o3w8esnqt9jjff2dax1qfudlsuo9 2414822 2414821 2026-05-18T15:04:10Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1337-1339].eqiad.wmnet 2414822 wikitext text/x-wiki dcm2egl07big33lcpvjehmbioxassw2 2414823 2414822 2026-05-18T15:04:19Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1340-1342].eqiad.wmnet 2414823 wikitext text/x-wiki tizl1733km1hm5molacar02qnhxsisp 2414824 2414823 2026-05-18T15:05:58Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1340-1342].eqiad.wmnet 2414824 wikitext text/x-wiki nz9hyiaz0052auohgp9kku761my4ljv 2414825 2414824 2026-05-18T15:08:00Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host netflow4003.ulsfo.wmnet 2414825 wikitext text/x-wiki af8vajdbwqvqhnmqxocd0rjt9781fzr 2414826 2414825 2026-05-18T15:08:14Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be1005.eqiad.wmnet 2414826 wikitext text/x-wiki rj38o2831791kl4c5hglocd02e71e35 2414827 2414826 2026-05-18T15:08:18Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host thanos-be2005.codfw.wmnet 2414827 wikitext text/x-wiki jtqmo2k1zoqn9lezoduu3q01wihjwxr 2414828 2414827 2026-05-18T15:10:50Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2137-2139].codfw.wmnet 2414828 wikitext text/x-wiki e1qm35w9wv2lyc9sbjeddbt9s6fadt2 2414829 2414828 2026-05-18T15:10:53Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2137-2139].codfw.wmnet 2414829 wikitext text/x-wiki r8xprgqyx79crqfbg8cjzzny9v48bjf 2414830 2414829 2026-05-18T15:11:00Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1340-1342].eqiad.wmnet 2414830 wikitext text/x-wiki 2dd5uyuysx0lib7h0g0k1f0tewvtawp 2414831 2414830 2026-05-18T15:11:02Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1340-1342].eqiad.wmnet 2414831 wikitext text/x-wiki oquab0n8ty6qb453saq1v53gszrezj6 2414832 2414831 2026-05-18T15:11:03Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2140-2142].codfw.wmnet 2414832 wikitext text/x-wiki gnmwn4xcl9s2o0g2mylbi9827yt9zmt 2414833 2414832 2026-05-18T15:11:05Z Stashbot 7414 herron@cumin1003: END (FAIL) - Cookbook sre.kafka.roll-restart-reboot-brokers (exit_code=99) rolling reboot on A:kafka-logging-codfw 2414833 wikitext text/x-wiki rbo2vcrs6re7r0ub4743dzlugpfli5x 2414834 2414833 2026-05-18T15:11:10Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1343-1345].eqiad.wmnet 2414834 wikitext text/x-wiki qfw92fvy0flfffiqynlbg76b5qrnxxa 2414835 2414834 2026-05-18T15:11:52Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netflow4003.ulsfo.wmnet 2414835 wikitext text/x-wiki bgisikutzs9vmthnaxr11tof1h2lbya 2414836 2414835 2026-05-18T15:12:47Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2140-2142].codfw.wmnet 2414836 wikitext text/x-wiki t6mbiee79vyti68bxnfdqvanszdnn5y 2414837 2414836 2026-05-18T15:12:49Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1343-1345].eqiad.wmnet 2414837 wikitext text/x-wiki ee9aj3v88dqaprm3oqnzz0l9sqjcu88 2414838 2414837 2026-05-18T15:16:33Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be2005.codfw.wmnet 2414838 wikitext text/x-wiki t6t2b4y7470hpbmjp33ddfo2chvzo9k 2414839 2414838 2026-05-18T15:16:38Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host thanos-be1006.eqiad.wmnet 2414839 wikitext text/x-wiki ajmye56a5lpqgia3ls9w9ck81dfhk3u 2414840 2414839 2026-05-18T15:17:50Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1343-1345].eqiad.wmnet 2414840 wikitext text/x-wiki m7slc1kucqzswqvvk6ql9lfs6s4gu6k 2414841 2414840 2026-05-18T15:17:52Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1343-1345].eqiad.wmnet 2414841 wikitext text/x-wiki 1trulpdn6ujbp3vp0w65up8a59ri2op 2414842 2414841 2026-05-18T15:18:01Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1346-1348].eqiad.wmnet 2414842 wikitext text/x-wiki lcmx71y8pq97e4ne420nomca3w1qgjn 2414843 2414842 2026-05-18T15:19:07Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host netflow3004.esams.wmnet 2414843 wikitext text/x-wiki hrq60tsrs6pj4twaasaby9m3q36j286 2414844 2414843 2026-05-18T15:19:41Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1346-1348].eqiad.wmnet 2414844 wikitext text/x-wiki pfdhtegkt4llq7yak6gauuukitd7gm5 2414845 2414844 2026-05-18T15:20:05Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2140-2142].codfw.wmnet 2414845 wikitext text/x-wiki bo5nx2pmip001wohfpikjqsshuuyigb 2414846 2414845 2026-05-18T15:20:07Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2140-2142].codfw.wmnet 2414846 wikitext text/x-wiki 9ye0zdoty0vqg564ogyxfq5oqcrwnfw 2414847 2414846 2026-05-18T15:20:16Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2143-2145].codfw.wmnet 2414847 wikitext text/x-wiki 3d83fjht03onvte35mmyucj9jv2cb44 2414848 2414847 2026-05-18T15:22:02Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2143-2145].codfw.wmnet 2414848 wikitext text/x-wiki soqxi26aic09br6ge8x169d28sas3ti 2414849 2414848 2026-05-18T15:23:22Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netflow3004.esams.wmnet 2414849 wikitext text/x-wiki lssyao9qvon6cf8c8szvewpxh7dh30e 2414850 2414849 2026-05-18T15:25:10Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be1006.eqiad.wmnet 2414850 wikitext text/x-wiki cq1j2oiekeppu5cjjtfrtzmycbzj3l1 2414851 2414850 2026-05-18T15:25:14Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host thanos-be2006.codfw.wmnet 2414851 wikitext text/x-wiki 3u09kqeirmytjt18jtrflzzmnky8on7 2414852 2414851 2026-05-18T15:26:17Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1346-1348].eqiad.wmnet 2414852 wikitext text/x-wiki 8blqawm018oh8jvq67sn6gworzlb6dg 2414853 2414852 2026-05-18T15:26:19Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1346-1348].eqiad.wmnet 2414853 wikitext text/x-wiki pkrhplwrrs8rrxq60yv9cgm3d52nray 2414854 2414853 2026-05-18T15:26:30Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1349-1351].eqiad.wmnet 2414854 wikitext text/x-wiki 4hjli9zvx15v1sdlxonmnxnx0q1v8go 2414855 2414854 2026-05-18T15:28:10Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1349-1351].eqiad.wmnet 2414855 wikitext text/x-wiki iogyan9htkgd2vg2e3g2uohkp8wo7xu 2414856 2414855 2026-05-18T15:29:15Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2143-2145].codfw.wmnet 2414856 wikitext text/x-wiki 3x709m79j6wyh4iedfa8sdw1qbi7lua 2414857 2414856 2026-05-18T15:29:18Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2143-2145].codfw.wmnet 2414857 wikitext text/x-wiki n1206dqjxihkb98s5ge02616x9uiwjb 2414858 2414857 2026-05-18T15:29:27Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2146-2148].codfw.wmnet 2414858 wikitext text/x-wiki hffpp40q9j7j89xt4nsxrybc4kvsnry 2414859 2414858 2026-05-18T15:31:12Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2146-2148].codfw.wmnet 2414859 wikitext text/x-wiki p9pgabljxckmj2lh9xsmltdzr8nlm5k 2414861 2414859 2026-05-18T15:32:38Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be2006.codfw.wmnet 2414861 wikitext text/x-wiki bn01xtylxvk1d50lyjzx6ukllt4ndp4 2414862 2414861 2026-05-18T15:32:43Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host thanos-be1007.eqiad.wmnet 2414862 wikitext text/x-wiki tl479j76n9q2uyuctobk5gl3xe2vncg 2414863 2414862 2026-05-18T15:33:44Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host netflow2003.codfw.wmnet 2414863 wikitext text/x-wiki eld8awk5jhsa0w8nayqn7fhm51pu3b8 2414864 2414863 2026-05-18T15:34:47Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1349-1351].eqiad.wmnet 2414864 wikitext text/x-wiki gm4o7dtbqmlp3egaqh8uwkx0o2gr7qw 2414865 2414864 2026-05-18T15:34:49Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1349-1351].eqiad.wmnet 2414865 wikitext text/x-wiki 2qxsq9d56qvdq5hfw2qkvng2f9zjl6n 2414866 2414865 2026-05-18T15:34:58Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1352-1354].eqiad.wmnet 2414866 wikitext text/x-wiki br2lhsxl6i0kybrzwfdiool9idi4ded 2414867 2414866 2026-05-18T15:36:40Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1352-1354].eqiad.wmnet 2414867 wikitext text/x-wiki faekuyh6k1n82yyijtqge6x0qql8pvd 2414868 2414867 2026-05-18T15:37:36Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netflow2003.codfw.wmnet 2414868 wikitext text/x-wiki c0iy2gvamuuajebmovzxfh94s767697 2414869 2414868 2026-05-18T15:37:55Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc1063.eqiad.wmnet with OS bookworm 2414869 wikitext text/x-wiki 67tqya7nv2hl1kbjerrgkuhmt27lpfv 2414870 2414869 2026-05-18T15:37:57Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc1064.eqiad.wmnet with OS bookworm 2414870 wikitext text/x-wiki 3vycwh93kubwfoz2zpizh16imfndhpb 2414871 2414870 2026-05-18T15:38:22Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2146-2148].codfw.wmnet 2414871 wikitext text/x-wiki f0c7daak9rx7a4xf1xbxsjwbkhaob6x 2414872 2414871 2026-05-18T15:38:24Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2146-2148].codfw.wmnet 2414872 wikitext text/x-wiki 0bw6ujaemdlf4tqsqli72wq6zj0g1wr 2414873 2414872 2026-05-18T15:38:33Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2149-2151].codfw.wmnet 2414873 wikitext text/x-wiki 82sfvmmfh29rfx9gycznktcwnrar8je 2414874 2414873 2026-05-18T15:38:51Z Stashbot 7414 Amir1: re-mapping thumbsize of 1 to 2 in all group0 wikis (T376152) 2414874 wikitext text/x-wiki ruooi65k4txfneykrunrvt7wejrk4zv 2414875 2414874 2026-05-18T15:40:17Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2149-2151].codfw.wmnet 2414875 wikitext text/x-wiki b5xninhlidhpj2akzald7p0ww3lw8og 2414876 2414875 2026-05-18T15:41:02Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be1007.eqiad.wmnet 2414876 wikitext text/x-wiki asw5yhwg8brx2gdj7d7glfr47yzkx42 2414877 2414876 2026-05-18T15:41:06Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host thanos-be2007.codfw.wmnet 2414877 wikitext text/x-wiki r1ffpw6cmjcjceyu0oocwozy1wdjfjm 2414878 2414877 2026-05-18T15:43:14Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1352-1354].eqiad.wmnet 2414878 wikitext text/x-wiki hzhz1fs1mcwjyrjb9p1xbqcujrzta1p 2414879 2414878 2026-05-18T15:43:16Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1352-1354].eqiad.wmnet 2414879 wikitext text/x-wiki bg4f3y3z88f4ice19niz7s55vjnym3j 2414880 2414879 2026-05-18T15:43:24Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1355-1357].eqiad.wmnet 2414880 wikitext text/x-wiki gumv162eru7uh6piggw5ade909j1j07 2414881 2414880 2026-05-18T15:45:06Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1355-1357].eqiad.wmnet 2414881 wikitext text/x-wiki hsu7ti97hsdbouxx92ufw2isaoonxud 2414882 2414881 2026-05-18T15:47:03Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host netflow2004.codfw.wmnet 2414882 wikitext text/x-wiki paemui2t4u5f9ufjc0uqolpb6sk5rhy 2414883 2414882 2026-05-18T15:47:33Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2149-2151].codfw.wmnet 2414883 wikitext text/x-wiki 67k74dujz1ouuhtx4zvysdby9m63fx5 2414884 2414883 2026-05-18T15:48:28Z Stashbot 7414 blake@cumin1003: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) pool for host wikikube-worker[2149-2151].codfw.wmnet 2414884 wikitext text/x-wiki 8ey5yclfonaot1hn3atkac6we3q2rii 2414885 2414884 2026-05-18T15:48:43Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2152-2154].codfw.wmnet 2414885 wikitext text/x-wiki pzh9rlpus7tvfefaoqgnh87vz212uu3 2414886 2414885 2026-05-18T15:48:46Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be2007.codfw.wmnet 2414886 wikitext text/x-wiki cyx9zm8hte1b2ys61c573asi50q3zwg 2414887 2414886 2026-05-18T15:48:51Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host thanos-be1008.eqiad.wmnet 2414887 wikitext text/x-wiki 0celi3wbfj2paciu8o51cvgr4b3nqka 2414888 2414887 2026-05-18T15:49:40Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc1063.eqiad.wmnet with reason: host reimage 2414888 wikitext text/x-wiki aci2xfem7s201cbivnu8f91egjf0562 2414889 2414888 2026-05-18T15:49:54Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc1064.eqiad.wmnet with reason: host reimage 2414889 wikitext text/x-wiki ic093gb51vz8rcfypo9md8jdr939j42 2414890 2414889 2026-05-18T15:50:23Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2152-2154].codfw.wmnet 2414890 wikitext text/x-wiki a3age4ir1edb39oso9tihieumjyyc4j 2414891 2414890 2026-05-18T15:50:47Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.kafka.roll-restart-reboot-brokers rolling reboot on A:kafka-logging-codfw 2414891 wikitext text/x-wiki 27dgtaze0ac00wc8w7tz2wrj5of92l3 2414892 2414891 2026-05-18T15:51:54Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1355-1357].eqiad.wmnet 2414892 wikitext text/x-wiki nrfmwsyxrkbng4j9ph4k66uvuxyzk5x 2414893 2414892 2026-05-18T15:51:56Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1355-1357].eqiad.wmnet 2414893 wikitext text/x-wiki pup3m9lu2tnrqens056odgpwh4p3xmu 2414894 2414893 2026-05-18T15:52:04Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1358-1360].eqiad.wmnet 2414894 wikitext text/x-wiki 4awa1rst7rjianl47n4cyn7e0uz4lxo 2414895 2414894 2026-05-18T15:52:24Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netflow2004.codfw.wmnet 2414895 wikitext text/x-wiki h8zfw1tbb6o4bw1q5jr296r406nclkt 2414897 2414895 2026-05-18T15:53:38Z Stashbot 7414 blake@cumin1003: END (ERROR) - Cookbook sre.k8s.reboot-nodes (exit_code=97) rolling reboot on P{wikikube-worker[2001-2331].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2414897 wikitext text/x-wiki 3u88tsajitfglgls4eo92gbtyuyj0mm 2414898 2414897 2026-05-18T15:53:45Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1358-1360].eqiad.wmnet 2414898 wikitext text/x-wiki b754rptvfiaonrr96g15sov4ppp4xqz 2414899 2414898 2026-05-18T15:54:19Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1063.eqiad.wmnet with reason: host reimage 2414899 wikitext text/x-wiki jhchm2q98gkqik121j313mdp81a2gqw 2414900 2414899 2026-05-18T15:54:41Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.reboot-nodes rolling reboot on P{wikikube-worker[2155-2331].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2414900 wikitext text/x-wiki c397latdiuw67yd80iepsi5qcs3th8d 2414901 2414900 2026-05-18T15:55:08Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2155-2169].codfw.wmnet 2414901 wikitext text/x-wiki ktc353hd43u2wotasjoh3cz8rdiw0i4 2414902 2414901 2026-05-18T15:56:03Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be1008.eqiad.wmnet 2414902 wikitext text/x-wiki k97g5jfbhvqjsqzfwpm1juhnqh1v9gt 2414903 2414902 2026-05-18T15:56:07Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host thanos-be2008.codfw.wmnet 2414903 wikitext text/x-wiki lh8v5ayce3kviw05pqpj2k60jtkg65y 2414904 2414903 2026-05-18T15:57:17Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1064.eqiad.wmnet with reason: host reimage 2414904 wikitext text/x-wiki scukhywc5phav0ofa9tf98pl0yzpm43 2414905 2414904 2026-05-18T16:00:53Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1358-1360].eqiad.wmnet 2414905 wikitext text/x-wiki nm3zdehv5rvn0ovdmh3ejwgk90rfya7 2414906 2414905 2026-05-18T16:00:54Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1358-1360].eqiad.wmnet 2414906 wikitext text/x-wiki c4yudf9rkyh0cigpqa1tatvymi24je9 2414907 2414906 2026-05-18T16:01:03Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1361-1363].eqiad.wmnet 2414907 wikitext text/x-wiki 29n6umm5d41q8jx4bx1bcid6i615vlr 2414908 2414907 2026-05-18T16:02:44Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1361-1363].eqiad.wmnet 2414908 wikitext text/x-wiki rog3z9teos3wq2mfz05wilwizooj3od 2414910 2414908 2026-05-18T16:04:28Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2155-2169].codfw.wmnet 2414910 wikitext text/x-wiki l8na189mkhugibwhqla0k47rb4w5nxa 2414916 2414910 2026-05-18T16:05:51Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be2008.codfw.wmnet 2414916 wikitext text/x-wiki d6zwxtnqzfczifjz31s8282jwpb1fcv 2414918 2414916 2026-05-18T16:05:56Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host thanos-be1009.eqiad.wmnet 2414918 wikitext text/x-wiki 7ejbndxqcv9hvwk0yrfguim90ootyyc 2414923 2414918 2026-05-18T16:09:19Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1361-1363].eqiad.wmnet 2414923 wikitext text/x-wiki juq4wuf71mzsc81qggla791m26hdwan 2414924 2414923 2026-05-18T16:09:21Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1361-1363].eqiad.wmnet 2414924 wikitext text/x-wiki ft0c7fk38i5a9v3qtx5kqomi9ozn02h 2414925 2414924 2026-05-18T16:09:29Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1364-1366].eqiad.wmnet 2414925 wikitext text/x-wiki jkbupmsmm9wlhs8kqt9pm4tp365notq 2414926 2414925 2026-05-18T16:10:14Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1063.eqiad.wmnet with OS bookworm 2414926 wikitext text/x-wiki h61l91n4n1ogynzwklr4mhijvlrz4h7 2414927 2414926 2026-05-18T16:11:14Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1364-1366].eqiad.wmnet 2414927 wikitext text/x-wiki 7pgnquo31942a2udlhcqqky3kkfsakc 2414928 2414927 2026-05-18T16:12:07Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1064.eqiad.wmnet with OS bookworm 2414928 wikitext text/x-wiki 65hecsp62o6yk4thqrgdte6eymyyzce 2414929 2414928 2026-05-18T16:13:03Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2155-2169].codfw.wmnet 2414929 wikitext text/x-wiki 42fctfasugai8okkhhzsgc5pk9hvpod 2414930 2414929 2026-05-18T16:13:12Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2155-2169].codfw.wmnet 2414930 wikitext text/x-wiki dwweil576mzuf2d8wbuph30nynhptoz 2414931 2414930 2026-05-18T16:13:43Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2170-2179,2184-2188].codfw.wmnet 2414931 wikitext text/x-wiki 5ca0okibjwj3gyvz25kn7nexozidjjj 2414932 2414931 2026-05-18T16:14:19Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be1009.eqiad.wmnet 2414932 wikitext text/x-wiki p4jh5cgf8qslsxdrs0q06idnmusrkv1 2414933 2414932 2026-05-18T16:14:23Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host thanos-be2009.codfw.wmnet 2414933 wikitext text/x-wiki 8kjqe9099kkzs5xc1aexjrbvvc6jokq 2414934 2414933 2026-05-18T16:18:57Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1364-1366].eqiad.wmnet 2414934 wikitext text/x-wiki 9jpu0zn6xngocc2hniwqjug2ia7u4jt 2414935 2414934 2026-05-18T16:18:59Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1364-1366].eqiad.wmnet 2414935 wikitext text/x-wiki 2pc6623qerqhycwv2zm8g5ogk4skxuu 2414936 2414935 2026-05-18T16:19:07Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1367-1369].eqiad.wmnet 2414936 wikitext text/x-wiki laymfpp2thnwztplsvgyqb61k4hnopl 2414937 2414936 2026-05-18T16:20:49Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1367-1369].eqiad.wmnet 2414937 wikitext text/x-wiki 9ws2lu2exp1o0olj14g4pv8zwnnad9e 2414938 2414937 2026-05-18T16:22:26Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2170-2179,2184-2188].codfw.wmnet 2414938 wikitext text/x-wiki 047kettp5jg7b7fhflpgn642dl2wwdf 2414939 2414938 2026-05-18T16:22:31Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be2009.codfw.wmnet 2414939 wikitext text/x-wiki 1q61jjndmturacw8avzjjhkuj58aall 2414940 2414939 2026-05-18T16:23:00Z Stashbot 7414 mutante: contint1003 - rebooting 2414940 wikitext text/x-wiki h435z64cm70rngdpwc23dy7b8qdxe11 2414941 2414940 2026-05-18T16:27:18Z Stashbot 7414 mutante: people.wikimedia.org backend - rebooting 2414941 wikitext text/x-wiki 7l9p81n5dddokx647m37ipa79yvas9g 2414942 2414941 2026-05-18T16:28:25Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1367-1369].eqiad.wmnet 2414942 wikitext text/x-wiki mtck25j6v50ttpcfxyae9jnde1h1qug 2414943 2414942 2026-05-18T16:28:26Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1367-1369].eqiad.wmnet 2414943 wikitext text/x-wiki f9qo64nhkplosi6966kpmheujtylw1o 2414944 2414943 2026-05-18T16:28:33Z Stashbot 7414 mutante: contint2003 - new jenkins - reboot for kernel upgrade 2414944 wikitext text/x-wiki d9zn5km9gbyah00jjnxo2t42f05a5pg 2414945 2414944 2026-05-18T16:28:35Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1370-1372].eqiad.wmnet 2414945 wikitext text/x-wiki j6f4k6p8aibrkt7ild2kdgkw3rfihze 2414946 2414945 2026-05-18T16:31:26Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2170-2179,2184-2188].codfw.wmnet 2414946 wikitext text/x-wiki 0bqdkexryxhw8wzivimf6omxngsvra2 2414947 2414946 2026-05-18T16:31:35Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2170-2179,2184-2188].codfw.wmnet 2414947 wikitext text/x-wiki qrcbq19cwctw9is838m665e8f6id6hv 2414948 2414947 2026-05-18T16:31:58Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2189-2202].codfw.wmnet 2414948 wikitext text/x-wiki t6yyq20tam1kuyuv74r972ryj759rne 2414949 2414948 2026-05-18T16:32:53Z Stashbot 7414 mutante: zuul[12]00[123] / zuul* - rebooting 2414949 wikitext text/x-wiki nxc3vo2vcn1s49yyn0w7j4t2hvn7k4h 2414950 2414949 2026-05-18T16:34:48Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1370-1372].eqiad.wmnet 2414950 wikitext text/x-wiki 4yderyicp0oucegdrd73nsgnpwkzmfb 2414952 2414950 2026-05-18T16:37:37Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.kafka.roll-restart-reboot-brokers (exit_code=0) rolling reboot on A:kafka-logging-codfw 2414952 wikitext text/x-wiki 58mjm2c4xhqb6vpux9biqfrr8cre7xv 2414953 2414952 2026-05-18T16:40:42Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2189-2202].codfw.wmnet 2414953 wikitext text/x-wiki hl4r9j16wnu3d7hpjaen3rpkc3bob9q 2414954 2414953 2026-05-18T16:41:22Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1370-1372].eqiad.wmnet 2414954 wikitext text/x-wiki bm68ic4ds92hh8rhb6aajhn7yi12dm5 2414955 2414954 2026-05-18T16:41:24Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1370-1372].eqiad.wmnet 2414955 wikitext text/x-wiki kc50mz83zs1ok0hx4esylhm2sh5nu58 2414956 2414955 2026-05-18T16:41:32Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1373-1374].eqiad.wmnet 2414956 wikitext text/x-wiki dii42ce5zibapysz1r3pf1mfz492qk5 2414957 2414956 2026-05-18T16:42:40Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1373-1374].eqiad.wmnet 2414957 wikitext text/x-wiki ogl72rojvx4r30ufx557pxx7gvf1sav 2414960 2414957 2026-05-18T16:49:10Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1373-1374].eqiad.wmnet 2414960 wikitext text/x-wiki ckpa4wffxadfb70dzn96w6f18gkgwvj 2414961 2414960 2026-05-18T16:49:12Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1373-1374].eqiad.wmnet 2414961 wikitext text/x-wiki 09vtiswurr49ndaw7dejdhcvrcbwodk 2414962 2414961 2026-05-18T16:49:14Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on P{wikikube-worker[1328-1384].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) 2414962 wikitext text/x-wiki 13xuoq3fzjut2w1165j98u9qn4x2k1i 2414965 2414962 2026-05-18T17:04:51Z Stashbot 7414 mutante: contint2002, phab2002 - rebooting 2414965 wikitext text/x-wiki 7x7ahvm71h25utrah7vkld3xo4tl3jl 2414966 2414965 2026-05-18T17:05:33Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.kafka.roll-restart-reboot-brokers rolling reboot on A:kafka-logging-eqiad 2414966 wikitext text/x-wiki mgqsoqf91u1mg1bn0xwlvq865s0xw87 2414968 2414966 2026-05-18T17:07:43Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host o11ytest1001.eqiad.wmnet 2414968 wikitext text/x-wiki 9tu0gu25243xzje46t5eqhamswqli24 2414969 2414968 2026-05-18T17:07:54Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host o11ytest2001.codfw.wmnet 2414969 wikitext text/x-wiki axv9ae0te2j33ocwnh1gl4dcpayqq3w 2414970 2414969 2026-05-18T17:08:15Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host mwlog2003.codfw.wmnet 2414970 wikitext text/x-wiki l2pqzgb2obd23btyr5wvxul2q5x9t9a 2414971 2414970 2026-05-18T17:10:08Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host webperf2003.codfw.wmnet 2414971 wikitext text/x-wiki mby3symepbrk0ry0z3nfb65ujx8jsem 2414972 2414971 2026-05-18T17:10:29Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host arclamp2001.codfw.wmnet 2414972 wikitext text/x-wiki imgyjewn29193ojyb3296j4zwy1rbt6 2414973 2414972 2026-05-18T17:10:48Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on etherpad1004.eqiad.wmnet with reason: T426563 2414973 wikitext text/x-wiki dqw2w4qmb2jcz9228y001g362l8915m 2414975 2414973 2026-05-18T17:11:10Z Stashbot 7414 mutante: etherpad - rebooting backends 2414975 wikitext text/x-wiki ilo4wezewscojj6qifg4uub2jxib7tr 2414976 2414975 2026-05-18T17:11:36Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc1062.eqiad.wmnet with OS bookworm 2414976 wikitext text/x-wiki akxmbsm2ib6blf0c8va71k3ts7qqcg5 2414977 2414976 2026-05-18T17:11:41Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc1061.eqiad.wmnet with OS bookworm 2414977 wikitext text/x-wiki 6ly1mha2qqoxx7r7ixustuy6ji5gu3m 2414978 2414977 2026-05-18T17:12:01Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host o11ytest1001.eqiad.wmnet 2414978 wikitext text/x-wiki fi5gvnqat9zx3qeodid6z3e5avqx048 2414981 2414978 2026-05-18T17:13:10Z Stashbot 7414 topranks: restarted gnmic on netflow3004 as series missing for cr2-esams 2414981 wikitext text/x-wiki 65qew7o2rney6wmu3woc2d2hz0rm6li 2414982 2414981 2026-05-18T17:13:49Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on doc1004.eqiad.wmnet with reason: T426563 2414982 wikitext text/x-wiki j2w6psw1u3ggzhfvfk4hslzcb4v01z3 2414983 2414982 2026-05-18T17:13:51Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host o11ytest2001.codfw.wmnet 2414983 wikitext text/x-wiki 7p2jabnp92b41k22hoeako96gzrxlrw 2414984 2414983 2026-05-18T17:13:55Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf2003.codfw.wmnet 2414984 wikitext text/x-wiki 2k5auozil4meud28jwhy05b1tinyyb9 2414985 2414984 2026-05-18T17:14:17Z Stashbot 7414 mutante: doc.wikimedia.org - rebooting backends 2414985 wikitext text/x-wiki 5uyxqu5vlc9bglruvdzg6dpuwtjrsze 2414986 2414985 2026-05-18T17:14:21Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on doc2003.codfw.wmnet with reason: T426563 2414986 wikitext text/x-wiki f4gx0h7d07ila22gm6xc8n5h2ftd575 2414988 2414986 2026-05-18T17:15:20Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mwlog2003.codfw.wmnet 2414988 wikitext text/x-wiki jweh1toymp5bd8fsre0csdjr4znzof4 2414989 2414988 2026-05-18T17:16:16Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host arclamp2001.codfw.wmnet 2414989 wikitext text/x-wiki aoz4xupzrxyog70ut5p72wg2cjy0uuy 2414991 2414989 2026-05-18T17:16:46Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet 2414991 wikitext text/x-wiki 7sq8ky9m217fj3d4vh3dxg1ul2w4wti 2414992 2414991 2026-05-18T17:16:55Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host arclamp1001.eqiad.wmnet 2414992 wikitext text/x-wiki ba60ou5asvld0bcli82hexwh6vfymtt 2414994 2414992 2026-05-18T17:18:33Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host mwlog1003.eqiad.wmnet 2414994 wikitext text/x-wiki lm35azb8katzdy0focq90ii9288557s 2414996 2414994 2026-05-18T17:21:01Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet 2414996 wikitext text/x-wiki 0vom66g63xidstm4jdaux2ungjqh07i 2414997 2414996 2026-05-18T17:22:24Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host arclamp1001.eqiad.wmnet 2414997 wikitext text/x-wiki 4knk6nluh6vemr2e9jk6m8l2uc5qn8t 2414998 2414997 2026-05-18T17:23:17Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host graphite2004.codfw.wmnet 2414998 wikitext text/x-wiki 7w3uzmuc57wvydjytpfzoz5dqssyscj 2414999 2414998 2026-05-18T17:23:25Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc1062.eqiad.wmnet with reason: host reimage 2414999 wikitext text/x-wiki ep36zslbs2swtdnry2y8uav514i5hn3 2415000 2414999 2026-05-18T17:23:32Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc1061.eqiad.wmnet with reason: host reimage 2415000 wikitext text/x-wiki 6et895ob29v7iad0mdqq8cf7h477e3b 2415001 2415000 2026-05-18T17:25:12Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mwlog1003.eqiad.wmnet 2415001 wikitext text/x-wiki fezlagvpel1hku1bkbi3tiflanm7aa9 2415002 2415001 2026-05-18T17:28:39Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1062.eqiad.wmnet with reason: host reimage 2415002 wikitext text/x-wiki 1w0yimvjvr2m0iujgo21ci4lqqtabgx 2415003 2415002 2026-05-18T17:30:35Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host graphite2004.codfw.wmnet 2415003 wikitext text/x-wiki gh2huvklfedz8g3to1p04srhx5d42od 2415004 2415003 2026-05-18T17:31:33Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host graphite1005.eqiad.wmnet 2415004 wikitext text/x-wiki hj6s0c94bnyam13aojw6fqk4626ry0s 2415005 2415004 2026-05-18T17:32:08Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host grafana2001.codfw.wmnet 2415005 wikitext text/x-wiki sqzf240ob1x8264g4a6f9b0r5wmgerm 2415006 2415005 2026-05-18T17:33:03Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1061.eqiad.wmnet with reason: host reimage 2415006 wikitext text/x-wiki n6zucxm4qp6ru0o3f2nv3l49pzpl87d 2415018 2415006 2026-05-18T17:36:10Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host grafana2001.codfw.wmnet 2415018 wikitext text/x-wiki fw4pm3pdf9gsdmmq3avpi0i21p9zwcg 2415026 2415018 2026-05-18T17:37:52Z Stashbot 7414 mutante: stewards* - rebooting 2415026 wikitext text/x-wiki 2c7govyex1wiai0xgn6836n6hqaha58 2415027 2415026 2026-05-18T17:38:26Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host graphite1005.eqiad.wmnet 2415027 wikitext text/x-wiki 6qkb4uwhisq8zzfhiiyq40bg6po5y71 2415028 2415027 2026-05-18T17:40:02Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host grafana1002.eqiad.wmnet 2415028 wikitext text/x-wiki 7i19ycm6m82lwrs4me9f6pgvqz93nm2 2415029 2415028 2026-05-18T17:43:50Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1062.eqiad.wmnet with OS bookworm 2415029 wikitext text/x-wiki 6318p3temzhz1498tcc5eylj3kbs2et 2415030 2415029 2026-05-18T17:44:13Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host centrallog2002.codfw.wmnet 2415030 wikitext text/x-wiki 2ow39di95ue1n07mf8gt8ziox6anco2 2415031 2415030 2026-05-18T17:44:43Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host alert2002.wikimedia.org 2415031 wikitext text/x-wiki jucsb0dfp2s9hk20331jjk59lux7rzg 2415032 2415031 2026-05-18T17:44:44Z Stashbot 7414 herron@cumin1003: END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host alert2002.wikimedia.org 2415032 wikitext text/x-wiki o311aimz37gz0lgjcsfqjy8vq8rj5l8 2415033 2415032 2026-05-18T17:44:48Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host grafana1002.eqiad.wmnet 2415033 wikitext text/x-wiki 9635ltp2kman647zix3do5xof2ttmyu 2415034 2415033 2026-05-18T17:45:29Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host alert2002.wikimedia.org 2415034 wikitext text/x-wiki aapoe8ul5vm5vdkvp5d0ij8x8hwlfed 2415035 2415034 2026-05-18T17:45:30Z Stashbot 7414 herron@cumin1003: END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host alert2002.wikimedia.org 2415035 wikitext text/x-wiki sdja05j90udchgx85cawfayo7nsh7ia 2415036 2415035 2026-05-18T17:45:59Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on phab2003.codfw.wmnet with reason: T426563 2415036 wikitext text/x-wiki 04ztlwxi854p8w0og8pj7ubi04m4ggo 2415037 2415036 2026-05-18T17:46:18Z Stashbot 7414 herron: rebooting alert2002 2415037 wikitext text/x-wiki iowbcquwc1cuonjbt9i3nos84406dhb 2415038 2415037 2026-05-18T17:46:25Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on phab1005.eqiad.wmnet with reason: T426563 2415038 wikitext text/x-wiki pebw3dvk0ykj7p39xcc1i1tcfnq2mky 2415039 2415038 2026-05-18T17:48:58Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1061.eqiad.wmnet with OS bookworm 2415039 wikitext text/x-wiki 67fbkjp6qmhygbnnrn4xakf2ebijfpc 2415040 2415039 2026-05-18T17:50:38Z Stashbot 7414 dzahn@cumin2002: START - Cookbook sre.hosts.reboot-single for host gitlab-runner1002.eqiad.wmnet 2415040 wikitext text/x-wiki 4fcq3i02bcw0e1jz38q7ma5vf8x890c 2415041 2415040 2026-05-18T17:50:46Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host centrallog2002.codfw.wmnet 2415041 wikitext text/x-wiki duu2z47qmgj2gjdx78p8su7o3sfhadr 2415042 2415041 2026-05-18T17:56:42Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.kafka.roll-restart-reboot-brokers (exit_code=0) rolling reboot on A:kafka-logging-eqiad 2415042 wikitext text/x-wiki dt4pafvur3qtsdba3m90yuk2rv2omg4 2415043 2415042 2026-05-18T17:56:58Z Stashbot 7414 dzahn@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host gitlab-runner1002.eqiad.wmnet 2415043 wikitext text/x-wiki l8jaszbf5ynlbkzasqn9ybzxj9aohhf 2415044 2415043 2026-05-18T17:59:30Z Stashbot 7414 dzahn@cumin2002: START - Cookbook sre.hosts.reboot-single for host gitlab-runner1003.eqiad.wmnet 2415044 wikitext text/x-wiki 2w1r9k4xdujp9im7qvhjam3ed61hv0w 2415045 2415044 2026-05-18T18:02:32Z Stashbot 7414 Reedy: Deployed patch for T426631 2415045 wikitext text/x-wiki 1pv1ab5b877g5y6pkfenagnfxt8weph 2415046 2415045 2026-05-18T18:04:18Z Stashbot 7414 blake@cumin1003: END (FAIL) - Cookbook sre.k8s.reboot-nodes (exit_code=1) rolling reboot on P{wikikube-worker[2155-2331].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2415046 wikitext text/x-wiki 1awgo8hnpb6nci4ki7mbyzuqkbma3tt 2415047 2415046 2026-05-18T18:06:03Z Stashbot 7414 dzahn@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host gitlab-runner1003.eqiad.wmnet 2415047 wikitext text/x-wiki 6hw16t4iqxbzwt9c28b85msoln6iufe 2415048 2415047 2026-05-18T18:06:40Z Stashbot 7414 dzahn@cumin2002: START - Cookbook sre.hosts.reboot-single for host gitlab-runner1004.eqiad.wmnet 2415048 wikitext text/x-wiki 0uv4fencp5eyf57v04h2e7fokbuqbgw 2415049 2415048 2026-05-18T18:07:15Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host kafkamon2003.codfw.wmnet 2415049 wikitext text/x-wiki 3hel0ibaw5t4d9bpjur8u17q9p6w27b 2415050 2415049 2026-05-18T18:11:16Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kafkamon2003.codfw.wmnet 2415050 wikitext text/x-wiki edc8n8evwqxy7jvs30wp7nixcob9f6n 2415052 2415050 2026-05-18T18:11:32Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host kafkamon1003.eqiad.wmnet 2415052 wikitext text/x-wiki m1srkxsvioj4ynluvuqrl1oxeabreg5 2415054 2415052 2026-05-18T18:13:26Z Stashbot 7414 dzahn@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host gitlab-runner1004.eqiad.wmnet 2415054 wikitext text/x-wiki kluweyzmpu74r0q4rx0sghz6k315i2w 2415055 2415054 2026-05-18T18:13:54Z Stashbot 7414 dzahn@cumin2002: START - Cookbook sre.hosts.reboot-single for host gitlab-runner2002.codfw.wmnet 2415055 wikitext text/x-wiki p3munrvmovj4gf49uobkzvhi0z7dc6c 2415056 2415055 2026-05-18T18:13:56Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kafkamon1003.eqiad.wmnet 2415056 wikitext text/x-wiki kg25t3lqdlr1vrwacgyqzdgkpb1rj3d 2415058 2415056 2026-05-18T18:16:55Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on releases2003.codfw.wmnet with reason: T426563 2415058 wikitext text/x-wiki 72j6q3bkybudn3sxdxecmuq9axepog4 2415059 2415058 2026-05-18T18:16:59Z Stashbot 7414 mutante: releases.wikimedia.org - rebooting backends 2415059 wikitext text/x-wiki oq7g9hww3wbqaul8cjg1m2hegixmw4k 2415061 2415059 2026-05-18T18:19:03Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2189.codfw.wmnet 2415061 wikitext text/x-wiki 2nk5wb4vn4yp534qwb0985ae51xhkuk 2415062 2415061 2026-05-18T18:19:05Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2189.codfw.wmnet 2415062 wikitext text/x-wiki mre3hflrsffyk56yz03vkr1ae40zvxs 2415063 2415062 2026-05-18T18:20:19Z Stashbot 7414 dzahn@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host gitlab-runner2002.codfw.wmnet 2415063 wikitext text/x-wiki sr2nbj1griue54clkq3sc0x22i058uc 2415064 2415063 2026-05-18T18:20:35Z Stashbot 7414 dzahn@cumin2002: START - Cookbook sre.hosts.reboot-single for host gitlab-runner2003.codfw.wmnet 2415064 wikitext text/x-wiki plhs9lp06qauza9hajm6supg6r41b9y 2415065 2415064 2026-05-18T18:22:35Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2191-2202].codfw.wmnet 2415065 wikitext text/x-wiki b7v3khqklez3xvbw4adq39xww0d9ame 2415066 2415065 2026-05-18T18:22:48Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2191-2202].codfw.wmnet 2415066 wikitext text/x-wiki 3zsumelnwjd37sqn9lraj9187ijat70 2415067 2415066 2026-05-18T18:26:35Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.reboot-nodes rolling reboot on P{wikikube-worker[2203-2331].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2415067 wikitext text/x-wiki mmx9l279nwvphoc3xrpd2rpyvbealeh 2415068 2415067 2026-05-18T18:26:48Z Stashbot 7414 herron: rebooting alert1002 2415068 wikitext text/x-wiki 6br854pfkqxtwaedazs6qtfbbg5ycbh 2415069 2415068 2026-05-18T18:30:58Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reboot-single for host centrallog1002.eqiad.wmnet 2415069 wikitext text/x-wiki f1asv02sh1bj433lhkmyq5vjg00mf3s 2415070 2415069 2026-05-18T18:31:00Z Stashbot 7414 herron@cumin1003: END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host centrallog1002.eqiad.wmnet 2415070 wikitext text/x-wiki 23h6alk7fvvwr3qx9e7hmwl747ko3ij 2415071 2415070 2026-05-18T18:31:35Z Stashbot 7414 dzahn@cumin2002: START - Cookbook sre.hosts.reboot-single for host gitlab-runner2004.codfw.wmnet 2415071 wikitext text/x-wiki decq5qcgqgnrd3evmh47truc07xbg34 2415072 2415071 2026-05-18T18:31:37Z Stashbot 7414 dzahn@cumin2002: END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host gitlab-runner2004.codfw.wmnet 2415072 wikitext text/x-wiki l0g8yligq033orrcfjy1v9bw6f4uzgi 2415073 2415072 2026-05-18T18:32:50Z Stashbot 7414 dzahn@cumin2002: START - Cookbook sre.hosts.reboot-single for host gitlab-runner2004.codfw.wmnet 2415073 wikitext text/x-wiki bmvlarhjs82hcf745i8nd7hh523dk8u 2415074 2415073 2026-05-18T18:32:52Z Stashbot 7414 dzahn@cumin2002: END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host gitlab-runner2004.codfw.wmnet 2415074 wikitext text/x-wiki fp3he273r9tsx4kg4tz3hfblbtx1h7a 2415075 2415074 2026-05-18T18:35:52Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2203-2215,2242].codfw.wmnet 2415075 wikitext text/x-wiki dxc9y6rsccj1jgkbs4isfl633lh2j77 2415076 2415075 2026-05-18T18:38:56Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on gitlab2003.wikimedia.org with reason: T426563 2415076 wikitext text/x-wiki kh62pfwkbksc1fmtzlfu1mw9azf7jzm 2415077 2415076 2026-05-18T18:44:57Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2203-2215,2242].codfw.wmnet 2415077 wikitext text/x-wiki i1ike7jidtybyeryl42rnqvjg0sojx7 2415078 2415077 2026-05-18T18:45:06Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2203-2215,2242].codfw.wmnet 2415078 wikitext text/x-wiki iohcrcbgkk00h8yv60mcpohsqvjqiu2 2415079 2415078 2026-05-18T18:45:30Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2243,2248-2260].codfw.wmnet 2415079 wikitext text/x-wiki ml1auuj3bx42z5jx6sfhn6yd2hjlzxe 2415080 2415079 2026-05-18T18:46:43Z Stashbot 7414 jhathaway@dns1004: START - running authdns-update 2415080 wikitext text/x-wiki dfshv5iw5zthihrk4i2rm81cg0x048b 2415081 2415080 2026-05-18T18:48:23Z Stashbot 7414 jhathaway@dns1004: END - running authdns-update 2415081 wikitext text/x-wiki lzspygf1asl7q92bizkw8n8atiunymm 2415082 2415081 2026-05-18T18:54:17Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2243,2248-2260].codfw.wmnet 2415082 wikitext text/x-wiki 1y2lg8sr6k6035jdvtwkvc8sxoauxwa 2415088 2415082 2026-05-18T19:02:25Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2243,2248-2260].codfw.wmnet 2415088 wikitext text/x-wiki 293yqzcidteufjxq4o1o4h447l2d87p 2415089 2415088 2026-05-18T19:02:34Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2243,2248-2260].codfw.wmnet 2415089 wikitext text/x-wiki 15c5lkcv9z8oaaee1p2p3jbk8vqj93y 2415091 2415089 2026-05-18T19:03:05Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2261-2274].codfw.wmnet 2415091 wikitext text/x-wiki t4b3wtvy5esj141brxhx4mrb4yubblr 2415095 2415091 2026-05-18T19:12:23Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2261-2274].codfw.wmnet 2415095 wikitext text/x-wiki j0jfarg40pmam25rd0nsw045kv89l5o 2415097 2415095 2026-05-18T19:20:41Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2261-2274].codfw.wmnet 2415097 wikitext text/x-wiki 3ypy2pjek03b2g0d2h5g23is3hsjew2 2415098 2415097 2026-05-18T19:20:50Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2261-2274].codfw.wmnet 2415098 wikitext text/x-wiki h5559tapelford3jv5rjrf5v2cua99x 2415099 2415098 2026-05-18T19:21:14Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2275-2288].codfw.wmnet 2415099 wikitext text/x-wiki 5nddgb3jfdgustxcrtbgzm10v44fabp 2415100 2415099 2026-05-18T19:29:55Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2275-2288].codfw.wmnet 2415100 wikitext text/x-wiki atutfulv5siy64dheq6nnky0zb7xbaa 2415101 2415100 2026-05-18T19:39:43Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2275-2288].codfw.wmnet 2415101 wikitext text/x-wiki 3bmn6egay6v6iq4xrtxqr7gjvpi9yyu 2415102 2415101 2026-05-18T19:39:52Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2275-2288].codfw.wmnet 2415102 wikitext text/x-wiki lgrmsq745guymyv311akr6z84k6tkz2 2415103 2415102 2026-05-18T19:40:17Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2289-2302].codfw.wmnet 2415103 wikitext text/x-wiki f7lwj5n4w7kgi4eqyeat25sjbm8dyk8 2415104 2415103 2026-05-18T19:48:28Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2289-2302].codfw.wmnet 2415104 wikitext text/x-wiki sm7afa5ht54dbnkc5824zgo5zr8j0kj 2415105 2415104 2026-05-18T19:56:03Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2289-2302].codfw.wmnet 2415105 wikitext text/x-wiki sact8q2y9uzcagl88lbvpoow6r39f4d 2415106 2415105 2026-05-18T19:56:11Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2289-2302].codfw.wmnet 2415106 wikitext text/x-wiki 8e673snjgcikj4do8vmg9aw4nb4pbqs 2415107 2415106 2026-05-18T19:56:38Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2303-2316].codfw.wmnet 2415107 wikitext text/x-wiki aid4tbnzi7nbifyy4lhiptha8t20tel 2415110 2415107 2026-05-18T20:05:21Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2303-2316].codfw.wmnet 2415110 wikitext text/x-wiki senhe8u9x71la82zfomxbu9m6i9ulmc 2415111 2415110 2026-05-18T20:13:26Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2303-2316].codfw.wmnet 2415111 wikitext text/x-wiki j4aj7r1mhtn04nyzhhy6qosegyp7w7e 2415112 2415111 2026-05-18T20:13:35Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2303-2316].codfw.wmnet 2415112 wikitext text/x-wiki 0kz3t71pzu9jfl126f5fso47dtbxtc0 2415113 2415112 2026-05-18T20:14:01Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2317-2330].codfw.wmnet 2415113 wikitext text/x-wiki 8sh2e9qd6iih6b7siojzlsmx681ftmj 2415117 2415113 2026-05-18T20:22:51Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2317-2330].codfw.wmnet 2415117 wikitext text/x-wiki 2fc4glbrphtuclvp89eyi2fjfg88hqe 2415118 2415117 2026-05-18T20:30:17Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2317-2330].codfw.wmnet 2415118 wikitext text/x-wiki 2wiv03qcxaqs7ner17nhxox2kef0tqs 2415119 2415118 2026-05-18T20:30:26Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2317-2330].codfw.wmnet 2415119 wikitext text/x-wiki sb6fnfsnpr8jgwr4pi8glt3m0mvlyzk 2415120 2415119 2026-05-18T20:30:28Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on P{wikikube-worker[2203-2331].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2415120 wikitext text/x-wiki 1klzn0ap96t0htzb8bosrgoi2fitbhs 2415129 2415120 2026-05-18T21:10:00Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on gerrit1003.wikimedia.org with reason: T426563 2415129 wikitext text/x-wiki a2jwzi1dralzxcycfb58tfdayj0zpfa 2415130 2415129 2026-05-18T21:10:25Z Stashbot 7414 mutante: gerrit-replica.wikimedia.org, gerrit-spare.wikimedia.org - rebooting backends 2415130 wikitext text/x-wiki cuhq1tz7043a2hle2m2ci0bcb3cm63f 2415131 2415130 2026-05-18T21:10:39Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on gerrit2002.wikimedia.org with reason: T426563 2415131 wikitext text/x-wiki 6l37j6xsqnm4joftjr4vffjwb1l1mr2 2415134 2415131 2026-05-18T21:14:19Z Stashbot 7414 dzahn@cumin2002: DONE (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 0:15:00 on gerrit-replica.wikimedia.org with reason: T426563 2415134 wikitext text/x-wiki p3fs36vp2gpkdo8ti954o3yoa6nvelk 2415135 2415134 2026-05-18T21:16:32Z Stashbot 7414 mutante: gerrit-replica.wikimedia.org back online 2415135 wikitext text/x-wiki ivr1l0n0uude7e4ruhftytguq0rfviu 2415136 2415135 2026-05-18T21:28:26Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2190.codfw.wmnet 2415136 wikitext text/x-wiki 5vqtz73zw283k0g8lr813disqehdalv 2415137 2415136 2026-05-18T21:28:28Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2190.codfw.wmnet 2415137 wikitext text/x-wiki kcvq4z0pyc0bqu9xi4fu2ve880ix8qg 2415138 2415137 2026-05-18T21:31:13Z Stashbot 7414 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1288925|Revert "Enable wgTrackMediaRequestProvenance on Commons" (T414338 T425580)]] 2415138 wikitext text/x-wiki 6yup3d7qxc5j2m225gz1zfwas5kfiyq 2415139 2415138 2026-05-18T21:33:00Z Stashbot 7414 krinkle@deploy1003: seddon, krinkle: Backport for [[gerrit:1288925|Revert "Enable wgTrackMediaRequestProvenance on Commons" (T414338 T425580)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2415139 wikitext text/x-wiki f240fjy3x5lnfho4t7m7crmjjesq473 2415140 2415139 2026-05-18T21:38:19Z Stashbot 7414 krinkle@deploy1003: seddon, krinkle: Continuing with deployment 2415140 wikitext text/x-wiki 96g3fmwc9a5lti51p2kcyqiy1sadrcc 2415141 2415140 2026-05-18T21:42:43Z Stashbot 7414 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1288925|Revert "Enable wgTrackMediaRequestProvenance on Commons" (T414338 T425580)]] (duration: 11m 29s) 2415141 wikitext text/x-wiki b7c7yes4thzlp0xdy0kfku41yxkhwam 2415143 2415141 2026-05-18T21:43:24Z Stashbot 7414 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mx-in2001.wikimedia.org with reason: T426563 2415143 wikitext text/x-wiki 6unwvaxpnglc5opud6bmfe3n0utdtna 2415144 2415143 2026-05-18T21:43:51Z Stashbot 7414 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mx-out2001.wikimedia.org with reason: T426563 2415144 wikitext text/x-wiki lwiuz6wo2kfa2z3rjw7329k8y76wrv6 2415145 2415144 2026-05-18T21:44:59Z Stashbot 7414 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mx-in1001.wikimedia.org with reason: T426563 2415145 wikitext text/x-wiki i8q8odxu528jpp9ooifqwbrjczo03qw 2415146 2415145 2026-05-18T21:45:23Z Stashbot 7414 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mx-out1001.wikimedia.org with reason: T426563 2415146 wikitext text/x-wiki 3ttc23edv866rfjhxi2fg32bagzfx6h 2415148 2415146 2026-05-18T21:54:59Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2149-2154].codfw.wmnet 2415148 wikitext text/x-wiki q3fqluw2oiah29m9p81oiduwhb4rfy1 2415149 2415148 2026-05-18T21:55:03Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2149-2154].codfw.wmnet 2415149 wikitext text/x-wiki c6nqx5x9vgi1q7ka7rf8s1ohwccsksy 2415153 2415149 2026-05-18T22:06:08Z Stashbot 7414 swfrench@cumin1003: START - Cookbook sre.hosts.reboot-single for host wikikube-worker2331.codfw.wmnet 2415153 wikitext text/x-wiki i5enjg6ui5jcl3swnx7q4mvlrpx5o0a 2415155 2415153 2026-05-18T22:11:35Z Stashbot 7414 swfrench@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host wikikube-worker2331.codfw.wmnet 2415155 wikitext text/x-wiki hl2jb48z33tj0b7ami95a8l6xyp1cs3 2415156 2415155 2026-05-18T22:48:22Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc1059.eqiad.wmnet with OS bookworm 2415156 wikitext text/x-wiki gr2esnxub4iqfgq3vb7vizcqmpeahi9 2415157 2415156 2026-05-18T22:48:43Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc1060.eqiad.wmnet with OS bookworm 2415157 wikitext text/x-wiki jewlq9ohm27u0u7zmvgg0zs90623yiq 2415158 2415157 2026-05-18T22:59:57Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc1059.eqiad.wmnet with reason: host reimage 2415158 wikitext text/x-wiki p05rnzyrox9h3r9t3oe08y9gh9o39zc 2415159 2415158 2026-05-18T23:00:25Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc1060.eqiad.wmnet with reason: host reimage 2415159 wikitext text/x-wiki cpkxznv3zkd7ldkvc0377pudrovyvkm 2415160 2415159 2026-05-18T23:03:26Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1059.eqiad.wmnet with reason: host reimage 2415160 wikitext text/x-wiki gv6tjxctz698supj54wfpehjcybnvh4 2415161 2415160 2026-05-18T23:07:21Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1060.eqiad.wmnet with reason: host reimage 2415161 wikitext text/x-wiki jijgrnr2bx4g9080i6l26wq5thl07m0 2415162 2415161 2026-05-18T23:11:11Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1289000|Remove wgThumbnailStepsRatio]] 2415162 wikitext text/x-wiki rqzxgtdk5yfbw8mqbtp5tt6c7qurdly 2415163 2415162 2026-05-18T23:12:57Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1289000|Remove wgThumbnailStepsRatio]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2415163 wikitext text/x-wiki si677ygab0otmj2gn8pbkqn2x5flyka 2415164 2415163 2026-05-18T23:13:49Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2415164 wikitext text/x-wiki s06y4yjr27wqv5724fpmtc974eavb7y 2415165 2415164 2026-05-18T23:18:03Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1289000|Remove wgThumbnailStepsRatio]] (duration: 06m 52s) 2415165 wikitext text/x-wiki kn2z59suz15osoqkw49ji5xv7nsknth 2415168 2415165 2026-05-18T23:19:06Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1059.eqiad.wmnet with OS bookworm 2415168 wikitext text/x-wiki j43uxp5zukf72v05yerdz9qif1voy7g 2415169 2415168 2026-05-18T23:23:18Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1060.eqiad.wmnet with OS bookworm 2415169 wikitext text/x-wiki kmt39s6cpt168ma2xwtw5v2qyl3ejpp 2415171 2415169 2026-05-18T23:37:58Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on lists2001.wikimedia.org with reason: T426563 2415171 wikitext text/x-wiki 8khurxcoc6mfl484rntbj7igp5hhviq 2415174 2415171 2026-05-18T23:44:07Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on vrts1004.eqiad.wmnet with reason: T426563 2415174 wikitext text/x-wiki efy0o5tgiuh9tprgmd7js05vlpb5acn 2415175 2415174 2026-05-18T23:44:39Z Stashbot 7414 dzahn@cumin2002: START - Cookbook sre.hosts.reboot-single for host vrts1004.eqiad.wmnet 2415175 wikitext text/x-wiki muysvy9husrnfivlknudsoqieghu9i5 2415176 2415175 2026-05-18T23:48:26Z Stashbot 7414 dzahn@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host vrts1004.eqiad.wmnet 2415176 wikitext text/x-wiki 1b66jwveezmq5lygn9xld03toffn0t6 2415183 2415176 2026-05-19T00:23:29Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1289005|IS: Drop wgGraphDefaultVegaVer, never used any more]], [[gerrit:1289006|IS: Drop wgEnableSpecialMute, ignored since MW 1.46]], [[gerrit:1289007|IS: Drop wgDiscussionTools_visualenhancements_*, ignored since 2025]] 2415183 wikitext text/x-wiki 8o5v4lztfpf9tet4x7f7mchkkafox5w 2415184 2415183 2026-05-19T00:24:11Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on gitlab2002.wikimedia.org with reason: T426563 2415184 wikitext text/x-wiki 1ysk8b0galxm5xmhf26de94bf8mjm43 2415185 2415184 2026-05-19T00:25:13Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup, jforrester: Backport for [[gerrit:1289005|IS: Drop wgGraphDefaultVegaVer, never used any more]], [[gerrit:1289006|IS: Drop wgEnableSpecialMute, ignored since MW 1.46]], [[gerrit:1289007|IS: Drop wgDiscussionTools_visualenhancements_*, ignored since 2025]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2415185 wikitext text/x-wiki 1chuotfl61t5jrxf53c97ssdvg8rygb 2415186 2415185 2026-05-19T00:25:40Z Stashbot 7414 dzahn@cumin2002: START - Cookbook sre.hosts.reboot-single for host gitlab2002.wikimedia.org 2415186 wikitext text/x-wiki ad7np1xtaurncxulbl8jwrgi00rfbbz 2415187 2415186 2026-05-19T00:26:25Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup, jforrester: Continuing with deployment 2415187 wikitext text/x-wiki nhyqxao5w9p0ey9my5pe3u6l2ppdf1k 2415188 2415187 2026-05-19T00:30:37Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1289005|IS: Drop wgGraphDefaultVegaVer, never used any more]], [[gerrit:1289006|IS: Drop wgEnableSpecialMute, ignored since MW 1.46]], [[gerrit:1289007|IS: Drop wgDiscussionTools_visualenhancements_*, ignored since 2025]] (duration: 07m 08s) 2415188 wikitext text/x-wiki 95rrchbnfiru6adrcki8mjy4fqelk0y 2415189 2415188 2026-05-19T00:32:06Z Stashbot 7414 dzahn@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host gitlab2002.wikimedia.org 2415189 wikitext text/x-wiki 6f7gdsfxpu7r3uiepkvtvxxezlf7app 2415190 2415189 2026-05-19T00:52:13Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1289008|ThumbLimits: Harmonize svwiki large size with the rest of wikis (T376152)]] 2415190 wikitext text/x-wiki 3vcwycrnx0debknjctodh0hwctjgd2o 2415191 2415190 2026-05-19T00:54:14Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1289008|ThumbLimits: Harmonize svwiki large size with the rest of wikis (T376152)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2415191 wikitext text/x-wiki hbk7fs62jixz1iybdq5dpi61on4gwxv 2415192 2415191 2026-05-19T00:54:36Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2415192 wikitext text/x-wiki fqnriivvc5vol5hr73u987n88xzo2pc 2415193 2415192 2026-05-19T00:58:49Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1289008|ThumbLimits: Harmonize svwiki large size with the rest of wikis (T376152)]] (duration: 06m 36s) 2415193 wikitext text/x-wiki a5isyb3e8f2srsmejfvkrpbmdds55fe 2415197 2415193 2026-05-19T02:00:10Z Stashbot 7414 dani@deploy1003: helmfile [staging] START helmfile.d/services/miscweb: apply 2415197 wikitext text/x-wiki qrj884ctx9hy51kljxs060mgxbu0a38 2415198 2415197 2026-05-19T02:00:23Z Stashbot 7414 dani@deploy1003: helmfile [staging] DONE helmfile.d/services/miscweb: apply 2415198 wikitext text/x-wiki qnn9u9f89i9q5e3hfgemhn3nyxnhver 2415199 2415198 2026-05-19T02:00:25Z Stashbot 7414 dani@deploy1003: helmfile [eqiad] START helmfile.d/services/miscweb: apply 2415199 wikitext text/x-wiki 8ouinjyk4k1ycz1m3b2g4zocc1zc8s3 2415200 2415199 2026-05-19T02:00:37Z Stashbot 7414 dani@deploy1003: helmfile [eqiad] DONE helmfile.d/services/miscweb: apply 2415200 wikitext text/x-wiki fcjnng8uwm6cku24hm5qz7tg24ioje9 2415201 2415200 2026-05-19T02:00:38Z Stashbot 7414 dani@deploy1003: helmfile [codfw] START helmfile.d/services/miscweb: apply 2415201 wikitext text/x-wiki 6zy9xz5cidk5569d2bfd5nsduz0ngco 2415202 2415201 2026-05-19T02:00:53Z Stashbot 7414 dani@deploy1003: helmfile [codfw] DONE helmfile.d/services/miscweb: apply 2415202 wikitext text/x-wiki ickhk67z6q59qj9m5vlejxijtucq8ut 2415203 2415202 2026-05-19T02:01:24Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2415203 wikitext text/x-wiki lhqj91xhq45nso5pcloday48y5hr9j3 2415204 2415203 2026-05-19T02:08:03Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 39s) 2415204 wikitext text/x-wiki ifmz1zu6frsuqhxi5e65v4f54jrzi7l 2415210 2415204 2026-05-19T03:03:20Z Stashbot 7414 mwpresync@deploy1003: Started scap sync-world: testwikis to 1.47.0-wmf.3 refs T423912 2415210 wikitext text/x-wiki 5dehgrfi7saib7tveyt9v5p9nwkgzcf 2415213 2415210 2026-05-19T03:41:45Z Stashbot 7414 mwpresync@deploy1003: Finished scap sync-world: testwikis to 1.47.0-wmf.3 refs T423912 (duration: 38m 23s) 2415213 wikitext text/x-wiki imgknbmpdu0xo0cllswjcwq4cr03cfn 2415214 2415213 2026-05-19T04:02:43Z Stashbot 7414 mwpresync@deploy1003: Pruned MediaWiki: 1.46.0-wmf.26 (duration: 02m 40s) 2415214 wikitext text/x-wiki ixoj6mttisikep50m7htj95gku1tv59 2415217 2415214 2026-05-19T05:11:55Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on pc2014.codfw.wmnet with reason: Maintenance on pc4 2415217 wikitext text/x-wiki 57kltmwgn9wmaytsjy5x74l6o6hyyle 2415218 2415217 2026-05-19T06:08:43Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 23 hosts with reason: Primary switchover s5 T426087 2415218 wikitext text/x-wiki qkcfbgg74uvzcezinlgqad2m7mncia8 2415219 2415218 2026-05-19T06:09:32Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Set db1230 with weight 0 T426087', diff saved to https://phabricator.wikimedia.org/P92574 and previous config saved to /var/cache/conftool/dbconfig/20260519-060929-fceratto.json 2415219 wikitext text/x-wiki kvsw6458yd9vsuyx9orpw6wbnufulwm 2415221 2415219 2026-05-19T06:14:03Z Stashbot 7414 federico3: Starting s5 eqiad failover from db1210 to db1230 - T426087 2415221 wikitext text/x-wiki 89stxiiyy3r9tkger142vp0ogor8pgq 2415222 2415221 2026-05-19T06:14:38Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Set s5 eqiad as read-only for maintenance - T426087', diff saved to https://phabricator.wikimedia.org/P92575 and previous config saved to /var/cache/conftool/dbconfig/20260519-061435-fceratto.json 2415222 wikitext text/x-wiki mev0y0hxibz1x1ooqd6n1u249jewwz0 2415223 2415222 2026-05-19T06:15:27Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Promote db1230 to s5 primary and set section read-write T426087', diff saved to https://phabricator.wikimedia.org/P92576 and previous config saved to /var/cache/conftool/dbconfig/20260519-061524-fceratto.json 2415223 wikitext text/x-wiki c9dyd5hkaf58lllb0r48j8gl49h3oa1 2415224 2415223 2026-05-19T06:18:20Z Stashbot 7414 fceratto@dns1005: START - running authdns-update 2415224 wikitext text/x-wiki son3gsz8nt39azltiogxu0wd99ct118 2415225 2415224 2026-05-19T06:19:57Z Stashbot 7414 fceratto@dns1005: END - running authdns-update 2415225 wikitext text/x-wiki j4c7z9rp5hc4p0f808rg67tb82zytjc 2415226 2415225 2026-05-19T06:20:59Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depool db1210 T426087', diff saved to https://phabricator.wikimedia.org/P92577 and previous config saved to /var/cache/conftool/dbconfig/20260519-062056-fceratto.json 2415226 wikitext text/x-wiki 9y0pr16dsll4oo3jvcoogxerzb6lyxx 2415227 2415226 2026-05-19T06:22:03Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1210.eqiad.wmnet with reason: Maintenance 2415227 wikitext text/x-wiki 90pjs2t4vg3xjz4ollhu8uz6nm42hqu 2415228 2415227 2026-05-19T06:22:30Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Remove pc2014 from dbctl T426595', diff saved to https://phabricator.wikimedia.org/P92578 and previous config saved to /var/cache/conftool/dbconfig/20260519-062227-marostegui.json 2415228 wikitext text/x-wiki 8uhgsuz9nyn40vzj6lks702nowtmul0 2415229 2415228 2026-05-19T06:24:45Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts pc2014.codfw.wmnet 2415229 wikitext text/x-wiki q35hz6jpr1dc43mqj29zth5zg1mkasb 2415230 2415229 2026-05-19T06:28:12Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc1057.eqiad.wmnet with OS bookworm 2415230 wikitext text/x-wiki t9ewn7dbsj03w560kq7n6hy9b38449z 2415231 2415230 2026-05-19T06:28:35Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.pool pool db1210: Repooling after switchover 2415231 wikitext text/x-wiki led8uqdvdq4j8sfqxe9hg8xu5znf2sm 2415232 2415231 2026-05-19T06:28:49Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc1058.eqiad.wmnet with OS bookworm 2415232 wikitext text/x-wiki lj4q0x44gwsx8yd3v88gyke4jolm3c1 2415233 2415232 2026-05-19T06:29:14Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.dns.netbox 2415233 wikitext text/x-wiki ajeikittrgw4cp601a83mnrqvttr9w1 2415234 2415233 2026-05-19T06:33:10Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: pc2014.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003" 2415234 wikitext text/x-wiki 16oa92bmn511bbli5dkyzfivqjwmy0p 2415235 2415234 2026-05-19T06:33:27Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: pc2014.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003" 2415235 wikitext text/x-wiki 3f8t45blxaqustxu0q264iuda68s829 2415236 2415235 2026-05-19T06:33:31Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2415236 wikitext text/x-wiki nw319s0l7pow9hztp05zkdbzcruvql3 2415237 2415236 2026-05-19T06:33:37Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts pc2014.codfw.wmnet 2415237 wikitext text/x-wiki 3uehmy1au8l3d1xbds8k5agimk3yc9u 2415238 2415237 2026-05-19T06:39:52Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc1057.eqiad.wmnet with reason: host reimage 2415238 wikitext text/x-wiki kh63mjjm0atc9vbo1r3980zb3c0dikm 2415239 2415238 2026-05-19T06:40:41Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc1058.eqiad.wmnet with reason: host reimage 2415239 wikitext text/x-wiki h3hsmaimrwb8re5agbtfpkakc61xjku 2415240 2415239 2026-05-19T06:41:49Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host netflow1002.eqiad.wmnet 2415240 wikitext text/x-wiki lb2mcsecs3it65g1yyi4575ik265qya 2415241 2415240 2026-05-19T06:44:25Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1057.eqiad.wmnet with reason: host reimage 2415241 wikitext text/x-wiki bx92vd7jm96iucamhjllpz6wiw0citw 2415242 2415241 2026-05-19T06:44:46Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 30 hosts with reason: Primary switchover s1 T426703 2415242 wikitext text/x-wiki 3nghxw86vchn77fhkilrt4nz7dryidu 2415243 2415242 2026-05-19T06:45:03Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Set db2203 with weight 0 T426703', diff saved to https://phabricator.wikimedia.org/P92581 and previous config saved to /var/cache/conftool/dbconfig/20260519-064500-fceratto.json 2415243 wikitext text/x-wiki h99vrrkdr4o2adyhym29cfxl2ot36mj 2415244 2415243 2026-05-19T06:45:40Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netflow1002.eqiad.wmnet 2415244 wikitext text/x-wiki 4aywogji1vkqljzy88rkis9fv952q2h 2415246 2415244 2026-05-19T06:48:19Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1058.eqiad.wmnet with reason: host reimage 2415246 wikitext text/x-wiki hgtlvmcvr8tp2ttm4jkmoi1h6024g5j 2415247 2415246 2026-05-19T06:50:22Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2415247 wikitext text/x-wiki 4x6hwgfslp9suv6cnesqqqkum0hp4l4 2415248 2415247 2026-05-19T06:50:27Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.parsercache (exit_code=99) 2415248 wikitext text/x-wiki 6lmx6j3gvmnusmylpn2ic9af5x0vqwo 2415249 2415248 2026-05-19T06:51:22Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool pc1011.eqiad.wmnet: Maintenance on pc1 2415249 wikitext text/x-wiki 5t5n7xjtfxq4r6fqoioze5temd5skqv 2415250 2415249 2026-05-19T06:51:29Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2415250 wikitext text/x-wiki 5lfga804kuhxmey8bnpwav62ujogudw 2415251 2415250 2026-05-19T06:51:37Z Stashbot 7414 federico3: Starting s1 codfw failover from db2212 to db2203 - T426703 2415251 wikitext text/x-wiki r7zvh3bx83v4sxxddy1ccmi7ua0ysgd 2415252 2415251 2026-05-19T06:51:42Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) 2415252 wikitext text/x-wiki 125cq98qm3ri2l09j8c6vprx0r2ge89 2415253 2415252 2026-05-19T06:51:47Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool pc1011.eqiad.wmnet: Maintenance on pc1 2415253 wikitext text/x-wiki brkjmzksirho6pd3cs00jt21g96ygxb 2415254 2415253 2026-05-19T06:52:02Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on pc2021.codfw.wmnet,pc[1011,1021].eqiad.wmnet with reason: Maintenance on pc1 2415254 wikitext text/x-wiki eelrrzexz79aya41b1yb1ika67b66n1 2415255 2415254 2026-05-19T06:52:27Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Promote db2203 to s1 primary T426703', diff saved to https://phabricator.wikimedia.org/P92583 and previous config saved to /var/cache/conftool/dbconfig/20260519-065224-fceratto.json 2415255 wikitext text/x-wiki 8fw3vy71zghyc2gxm7nvucw3bzbyav6 2415256 2415255 2026-05-19T06:54:40Z Stashbot 7414 moritzm: installing qemu security updates 2415256 wikitext text/x-wiki tnlvqbgxy8i126qyvv73fv4zl8slghj 2415257 2415256 2026-05-19T06:56:40Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depool db2212 T426703', diff saved to https://phabricator.wikimedia.org/P92584 and previous config saved to /var/cache/conftool/dbconfig/20260519-065637-fceratto.json 2415257 wikitext text/x-wiki e4fmoptfcofw69jhx5g505mksvwflvv 2415258 2415257 2026-05-19T06:56:48Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host netflow1003.eqiad.wmnet 2415258 wikitext text/x-wiki m480m9z56khj570mtm9w47a7oy4ziqy 2415259 2415258 2026-05-19T06:57:40Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2212.codfw.wmnet with reason: Maintenance 2415259 wikitext text/x-wiki meguouj50grnoeit7thtczsfmb6o3oj 2415260 2415259 2026-05-19T06:59:39Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1057.eqiad.wmnet with OS bookworm 2415260 wikitext text/x-wiki ms031bt74jntkz8yz67vg1defmnkya1 2415261 2415260 2026-05-19T07:02:13Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netflow1003.eqiad.wmnet 2415261 wikitext text/x-wiki 38u066slm1ly0oeuvfds45iwwon0i00 2415262 2415261 2026-05-19T07:02:38Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc1056.eqiad.wmnet with OS bookworm 2415262 wikitext text/x-wiki 9nzt1zzuaov6r1q2n01ge24zc9beak3 2415263 2415262 2026-05-19T07:03:27Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ldap-maint1001.eqiad.wmnet 2415263 wikitext text/x-wiki 3svxxk4r69tnuwy8itqc7d2wy1b8u6x 2415264 2415263 2026-05-19T07:04:25Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1058.eqiad.wmnet with OS bookworm 2415264 wikitext text/x-wiki 2wxibfkmds67lxojhc2vi7hgp9xd407 2415265 2415264 2026-05-19T07:07:18Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-maint1001.eqiad.wmnet 2415265 wikitext text/x-wiki s727gj4tdtrhv521lyfh3kle6yzlzxf 2415266 2415265 2026-05-19T07:07:25Z Stashbot 7414 mlitn@deploy1003: Started scap sync-world: Backport for [[gerrit:1288994|Squashed diff to master]] 2415266 wikitext text/x-wiki bi3zypget3fqk59xabill96e9yceh3d 2415267 2415266 2026-05-19T07:07:41Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet 2415267 wikitext text/x-wiki q1pfzxa5twh1pjgy8vwel3039iq5ias 2415268 2415267 2026-05-19T07:13:13Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet 2415268 wikitext text/x-wiki 25wmfymd3ef86oku1jfub4wq00zolx6 2415269 2415268 2026-05-19T07:13:24Z Stashbot 7414 mlitn@deploy1003: mlitn: Backport for [[gerrit:1288994|Squashed diff to master]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2415269 wikitext text/x-wiki pt9tx3lhueddv1wv1cwv36ina6r1inv 2415270 2415269 2026-05-19T07:14:04Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1210: Repooling after switchover 2415270 wikitext text/x-wiki 7htuu6h8y197pw50dg1hajmaoqrzkst 2415271 2415270 2026-05-19T07:14:24Z Stashbot 7414 mlitn@deploy1003: mlitn: Continuing with deployment 2415271 wikitext text/x-wiki dbr0698fdzt8g0b4trprjcxdxe2plrn 2415272 2415271 2026-05-19T07:14:30Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc1056.eqiad.wmnet with reason: host reimage 2415272 wikitext text/x-wiki dj2onaknqumsjxcsitaeq95z05yid5m 2415273 2415272 2026-05-19T07:17:01Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.pool pool db2212: Repooling after switchover 2415273 wikitext text/x-wiki ihwlnx0mjc0zq9colyz9us25kczzhx0 2415274 2415273 2026-05-19T07:19:00Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1056.eqiad.wmnet with reason: host reimage 2415274 wikitext text/x-wiki 38yps00p3jtutef3wes6itzpgteuhxm 2415275 2415274 2026-05-19T07:20:43Z Stashbot 7414 mlitn@deploy1003: Finished scap sync-world: Backport for [[gerrit:1288994|Squashed diff to master]] (duration: 13m 17s) 2415275 wikitext text/x-wiki 839lniv6vayvbh2zblcsm6ipk84jze4 2415276 2415275 2026-05-19T07:33:59Z Stashbot 7414 XioNoX: add gnmic 0.46.0 to reprepro 2415276 wikitext text/x-wiki 2m8vyvm6ifqkhya8vv3kbyg61jpeqos 2415277 2415276 2026-05-19T07:34:28Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1056.eqiad.wmnet with OS bookworm 2415277 wikitext text/x-wiki tosz7cjn7n11ymxq2cugz1cl33qkt2w 2415278 2415277 2026-05-19T07:39:10Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2150.codfw.wmnet 2415278 wikitext text/x-wiki loknz0vnxdq1jjzh3lgqrk0adapgvej 2415279 2415278 2026-05-19T07:39:29Z Stashbot 7414 volans@cumin2002: START - Cookbook sre.hosts.reboot-single for host cloudcumin2001.codfw.wmnet 2415279 wikitext text/x-wiki jlsd5l5x21qntc7jjqnrkfbzwcavxun 2415280 2415279 2026-05-19T07:43:47Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.dns.netbox 2415280 wikitext text/x-wiki di1m33v6ja23zdlzf9y6qfgqp84zcix 2415281 2415280 2026-05-19T07:44:29Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host install2005.wikimedia.org 2415281 wikitext text/x-wiki o76kkc1m309rk51zg5k9mtc7jvtogcl 2415282 2415281 2026-05-19T07:45:21Z Stashbot 7414 volans@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcumin2001.codfw.wmnet 2415282 wikitext text/x-wiki spa66954s7zezk6v8azotk5hbxv633l 2415283 2415282 2026-05-19T07:46:12Z Stashbot 7414 volans@cumin2002: START - Cookbook sre.hosts.reboot-single for host cloudcumin1001.eqiad.wmnet 2415283 wikitext text/x-wiki an0suz5mwetkxpfa37929wpfxjufksf 2415284 2415283 2026-05-19T07:48:13Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db2150.codfw.wmnet decommissioned, removing all IPs except the asset tag one - cwilliams@cumin1003" 2415284 wikitext text/x-wiki k7zib735hult38rl811eyn952z93mgn 2415285 2415284 2026-05-19T07:48:39Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db2150.codfw.wmnet decommissioned, removing all IPs except the asset tag one - cwilliams@cumin1003" 2415285 wikitext text/x-wiki 01mjqrzynvhl6836j6a8yoheygwme1d 2415286 2415285 2026-05-19T07:48:41Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2415286 wikitext text/x-wiki d8vy5itp9yh641zkz95dxf5y13a3pya 2415287 2415286 2026-05-19T07:48:42Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2150.codfw.wmnet 2415287 wikitext text/x-wiki cewxl8lx95v3f9yhk1pdsheiazjesy4 2415288 2415287 2026-05-19T07:50:05Z Stashbot 7414 cezmunsta: Removing db2150 from zarcillo T424342 2415288 wikitext text/x-wiki k52lfzf4ilsidnw053uast5a8odv3x7 2415289 2415288 2026-05-19T07:50:52Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host install2005.wikimedia.org 2415289 wikitext text/x-wiki 10bnosrv9dsm6dgb5459ugw09t0xe8e 2415290 2415289 2026-05-19T07:52:16Z Stashbot 7414 volans@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcumin1001.eqiad.wmnet 2415290 wikitext text/x-wiki 063xtxh3nbn4b24ebd5326kt5zafsne 2415291 2415290 2026-05-19T07:57:57Z Stashbot 7414 Emperor: reboot apus eqiad frontends (May reboots) 2415291 wikitext text/x-wiki 7sg8rhuse7tel7y527up8xylts4potx 2415292 2415291 2026-05-19T07:58:15Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-cluster 2415292 wikitext text/x-wiki 1dtaoif93j12bl4rl6288yze0ig1emt 2415293 2415292 2026-05-19T07:58:37Z Stashbot 7414 cezmunsta: Removing db2150 from orchestrator T424342 2415293 wikitext text/x-wiki qvso06bq809m1oh95m3bsb9p74dzen7 2415294 2415293 2026-05-19T08:02:31Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2212: Repooling after switchover 2415294 wikitext text/x-wiki b6af94djkyinesezoqszrgur2g2ynft 2415295 2415294 2026-05-19T08:10:54Z Stashbot 7414 volans@cumin2002: START - Cookbook sre.hosts.reboot-single for host cloudrabbit2003-dev.codfw.wmnet 2415295 wikitext text/x-wiki rnfmxtae1u9agyxc718bpmo6b5c3gtk 2415296 2415295 2026-05-19T08:13:17Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4008.ulsfo.wmnet 2415296 wikitext text/x-wiki 7lgf99lrimef2c3y0346xwx922n0ruu 2415297 2415296 2026-05-19T08:16:48Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti4008.ulsfo.wmnet 2415297 wikitext text/x-wiki rrjr5z3jhx7ebyvlgsrxgxanoa8z8l0 2415298 2415297 2026-05-19T08:17:28Z Stashbot 7414 volans@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudrabbit2003-dev.codfw.wmnet 2415298 wikitext text/x-wiki qnoniwwh6dejt3j8swtu4jmb3hkjogp 2415299 2415298 2026-05-19T08:17:36Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.memcached.roll-reboot-restart rolling reboot on A:memcached-gutter-eqiad 2415299 wikitext text/x-wiki 2n5u7m8g2nxxwm6vq7bcjm1l24mbbcr 2415300 2415299 2026-05-19T08:17:54Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-cluster (exit_code=0) 2415300 wikitext text/x-wiki fww5g7xaiovtdeji4lnmh4dt5zv25f5 2415301 2415300 2026-05-19T08:19:04Z Stashbot 7414 volans@cumin2002: START - Cookbook sre.hosts.reboot-single for host cloudrabbit2002-dev.codfw.wmnet 2415301 wikitext text/x-wiki 3wt0dibemm1p54tzw12qfcvd6k4yr9b 2415302 2415301 2026-05-19T08:19:14Z Stashbot 7414 hashar@deploy1003: rebuilt and synchronized wikiversions files: group0 to 1.47.0-wmf.3 refs T423912 2415302 wikitext text/x-wiki 7t03m645l1t6fd2a0zujcraydr0v25z 2415303 2415302 2026-05-19T08:22:53Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2151.codfw.wmnet 2415303 wikitext text/x-wiki p6iezwejamk65pggjasxcu6xc4p0flo 2415304 2415303 2026-05-19T08:23:22Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti4008.ulsfo.wmnet 2415304 wikitext text/x-wiki hs99qaaphl015cx68oc8icce3ire9y2 2415305 2415304 2026-05-19T08:23:27Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4008.ulsfo.wmnet 2415305 wikitext text/x-wiki jagz8syzyfkbgzroypgc5a1yj21cptg 2415306 2415305 2026-05-19T08:24:18Z Stashbot 7414 Emperor: reboot apus codfw frontends (May reboots) 2415306 wikitext text/x-wiki hgjd8pdavpwoomwe0w482t6akns5v3k 2415307 2415306 2026-05-19T08:24:22Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-cluster 2415307 wikitext text/x-wiki alv5demv3t2tdantw2adugx8qmpxquz 2415308 2415307 2026-05-19T08:26:01Z Stashbot 7414 volans@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudrabbit2002-dev.codfw.wmnet 2415308 wikitext text/x-wiki enz4fxyl35k1fhuabjlo6yaarhqmes6 2415309 2415308 2026-05-19T08:26:03Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4007.ulsfo.wmnet 2415309 wikitext text/x-wiki 0jwt3ns69x6uix0t989g3a9jof33su7 2415310 2415309 2026-05-19T08:27:26Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.dns.netbox 2415310 wikitext text/x-wiki s3x7030otgkjx2641zbc2usjp2s05vt 2415311 2415310 2026-05-19T08:28:22Z Stashbot 7414 volans@cumin2002: START - Cookbook sre.hosts.reboot-single for host cloudrabbit2001-dev.codfw.wmnet 2415311 wikitext text/x-wiki jmezo80tq5r1q37fzdq9fkep54kko14 2415312 2415311 2026-05-19T08:28:56Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti4007.ulsfo.wmnet 2415312 wikitext text/x-wiki eb7oonqy3oursxcsnc4ta6xncbcx854 2415313 2415312 2026-05-19T08:29:32Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ldap-maint2001.codfw.wmnet 2415313 wikitext text/x-wiki 1g5v6vte1bn10hhy4kbsi9ktgqqmkwy 2415314 2415313 2026-05-19T08:31:27Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db2151.codfw.wmnet decommissioned, removing all IPs except the asset tag one - cwilliams@cumin1003" 2415314 wikitext text/x-wiki enboezim0b4brq4588f1j3issg2zw4q 2415315 2415314 2026-05-19T08:31:48Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db2151.codfw.wmnet decommissioned, removing all IPs except the asset tag one - cwilliams@cumin1003" 2415315 wikitext text/x-wiki l3vh8rloplg05v7fawjm6f5wmvyh3hh 2415316 2415315 2026-05-19T08:31:49Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2415316 wikitext text/x-wiki 0hbf5qk6lc1011i00a4fjog45tvgvcd 2415317 2415316 2026-05-19T08:31:51Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2151.codfw.wmnet 2415317 wikitext text/x-wiki k6duh6wuxvlt3067jvbap41sl6o12hi 2415318 2415317 2026-05-19T08:32:33Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reboot-single for host mc1070.eqiad.wmnet 2415318 wikitext text/x-wiki 3kb3n1i5jwmryw3pv6hdfu8faqitaap 2415319 2415318 2026-05-19T08:32:51Z Stashbot 7414 cezmunsta: Removing db2151 from zarcillo T424343 2415319 wikitext text/x-wiki km3b3oyo59ds10aht6sjps94an07mpa 2415320 2415319 2026-05-19T08:33:23Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-maint2001.codfw.wmnet 2415320 wikitext text/x-wiki ej3snnfk0yp6a63hcgg7fq2dphwroe2 2415321 2415320 2026-05-19T08:33:33Z Stashbot 7414 cezmunsta: Removing db2151 from orchestrator T424343 2415321 wikitext text/x-wiki 7nxmpjxxhbp682lhwup9ziwzfjeeq5k 2415322 2415321 2026-05-19T08:33:44Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host cuminunpriv1001.eqiad.wmnet 2415322 wikitext text/x-wiki 4zuess65fosxik6sl6210omgziwtk4g 2415323 2415322 2026-05-19T08:34:00Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ping1004.eqiad.wmnet 2415323 wikitext text/x-wiki dxj0p20i0i4rztp6pg7il5i3caraim8 2415325 2415323 2026-05-19T08:35:12Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti4007.ulsfo.wmnet 2415325 wikitext text/x-wiki mw49j3nf240lalq192l1zivzz4ztd43 2415326 2415325 2026-05-19T08:35:15Z Stashbot 7414 volans@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudrabbit2001-dev.codfw.wmnet 2415326 wikitext text/x-wiki 2ljf2nwj8mcnv8zhrepd8abtgm7c6b3 2415327 2415326 2026-05-19T08:35:27Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4007.ulsfo.wmnet 2415327 wikitext text/x-wiki 5gybryz94fagpehhw91ttvbam9s5a6d 2415328 2415327 2026-05-19T08:35:44Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1286804|IPReputation: Route opensearch_ipoid through envoy service mesh (T421293)]] 2415328 wikitext text/x-wiki re61t4qfzwi9ghr0qx5a8wufuxabrmv 2415329 2415328 2026-05-19T08:35:55Z Stashbot 7414 volans@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudrabbit1003.eqiad.wmnet 2415329 wikitext text/x-wiki nabnkb1si1z7o869swd4gr20ccmx40g 2415330 2415329 2026-05-19T08:36:31Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4006.ulsfo.wmnet 2415330 wikitext text/x-wiki j828xw5d41un3j64ay1o773weyrp92n 2415331 2415330 2026-05-19T08:37:21Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.memcached.roll-reboot-restart (exit_code=0) rolling reboot on A:memcached-gutter-eqiad 2415331 wikitext text/x-wiki apuzizgkb74b0nkjn4nns6368q2sucj 2415332 2415331 2026-05-19T08:37:41Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cuminunpriv1001.eqiad.wmnet 2415332 wikitext text/x-wiki pshescjfdq8vbcpma4tdww8jpqnvc3s 2415334 2415332 2026-05-19T08:37:48Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1286804|IPReputation: Route opensearch_ipoid through envoy service mesh (T421293)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2415334 wikitext text/x-wiki mob8s5qjrql8t9mttx50sc4frojaym8 2415335 2415334 2026-05-19T08:37:50Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc1070.eqiad.wmnet 2415335 wikitext text/x-wiki q9d1gn1cab689ftph3qi9qbktlcu2at 2415336 2415335 2026-05-19T08:37:52Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ping1004.eqiad.wmnet 2415336 wikitext text/x-wiki anyhtaa28dd0apivug4vkjbvoxkca7h 2415337 2415336 2026-05-19T08:37:54Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reboot-single for host mc1071.eqiad.wmnet 2415337 wikitext text/x-wiki dcuwcpagilizs40bd9f9uisw772lmfx 2415338 2415337 2026-05-19T08:39:35Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2415338 wikitext text/x-wiki p13avpwejcfeufrymofuuszz92o2032 2415339 2415338 2026-05-19T08:39:46Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti4006.ulsfo.wmnet 2415339 wikitext text/x-wiki m3s5zdtqu13wy72b6vt1d9jysbdf5q2 2415340 2415339 2026-05-19T08:39:59Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2244: Upgrading db2244.codfw.wmnet 2415340 wikitext text/x-wiki 9ctgm72empryvuj927wafinv4sef5bv 2415341 2415340 2026-05-19T08:40:28Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2244: Upgrading db2244.codfw.wmnet 2415341 wikitext text/x-wiki kgvhauyb3fxv6ajr94n5l9j5zgriiva 2415342 2415341 2026-05-19T08:40:38Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2415342 wikitext text/x-wiki ohcyatydcomr2hlgkxfztcvzdj6i7dh 2415344 2415342 2026-05-19T08:41:53Z Stashbot 7414 volans@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudrabbit1003.eqiad.wmnet 2415344 wikitext text/x-wiki htp5mebdgu2ylpxdvaoons057x8dv24 2415345 2415344 2026-05-19T08:42:04Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ping2004.codfw.wmnet 2415345 wikitext text/x-wiki eyns4bll6v1qcbstge7vxkpmld65y9e 2415346 2415345 2026-05-19T08:42:41Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudnet2007-dev.codfw.wmnet 2415346 wikitext text/x-wiki ctdrshu4dkn4bmzylahhlycydgpdb8e 2415347 2415346 2026-05-19T08:43:06Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc1071.eqiad.wmnet 2415347 wikitext text/x-wiki 20uy4t7ax56k3jvhewv78o34y92piv1 2415348 2415347 2026-05-19T08:43:10Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reboot-single for host mc1072.eqiad.wmnet 2415348 wikitext text/x-wiki j42rmcadfc43qkmi6rumif8tl4lvbxy 2415350 2415348 2026-05-19T08:43:54Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-cluster (exit_code=0) 2415350 wikitext text/x-wiki srmj86axzl02abrauwkl52hs7y303vy 2415351 2415350 2026-05-19T08:44:12Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2244.codfw.wmnet with OS trixie 2415351 wikitext text/x-wiki mcjwebgkcsxl9yp288u1s1z4rfsa052 2415352 2415351 2026-05-19T08:44:31Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.memcached.roll-reboot-restart rolling reboot on A:memcached-gutter-codfw 2415352 wikitext text/x-wiki nd8bwlfc1k2tws1l998i2deb7rb5cla 2415353 2415352 2026-05-19T08:44:52Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1286804|IPReputation: Route opensearch_ipoid through envoy service mesh (T421293)]] (duration: 09m 08s) 2415353 wikitext text/x-wiki pl0jy187szqaimsmlzluxda56goyxkz 2415354 2415353 2026-05-19T08:45:50Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti4006.ulsfo.wmnet 2415354 wikitext text/x-wiki rupg5xuhea3p1fjn2qkmwtytwl1omf3 2415355 2415354 2026-05-19T08:45:55Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4006.ulsfo.wmnet 2415355 wikitext text/x-wiki hfsajjih1b67ivtycx9t49tuniisihw 2415356 2415355 2026-05-19T08:45:57Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ping2004.codfw.wmnet 2415356 wikitext text/x-wiki 0tngdxv2jgwu5e1zu5t9voic9n8fluo 2415357 2415356 2026-05-19T08:46:34Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ldap-rw1001.wikimedia.org 2415357 wikitext text/x-wiki 2lavdp6ka9c2fog6ja4xj1yo1kfiv4n 2415358 2415357 2026-05-19T08:47:02Z Stashbot 7414 moritzm: failover Ganeti cluster in ulsfo to ganeti4008 2415358 wikitext text/x-wiki g28goec5u93imihhqdd9t15y5xnehiz 2415359 2415358 2026-05-19T08:48:12Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1289275|Remove unused $wgEnableUserEmailMuteList config (T413867)]] 2415359 wikitext text/x-wiki 4ilxtpoauuxhsmvlj5tjo41hwnz51sd 2415360 2415359 2026-05-19T08:48:24Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc1072.eqiad.wmnet 2415360 wikitext text/x-wiki 1j9b4fmnysyxkpghrhu4ao0cmcuohpz 2415361 2415360 2026-05-19T08:49:05Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.reboot-nodes rolling reboot on A:aux-worker-eqiad 2415361 wikitext text/x-wiki opo6qx51ylbum1q2t2c8hb8pykedi4m 2415362 2415361 2026-05-19T08:49:22Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudnet2007-dev.codfw.wmnet 2415362 wikitext text/x-wiki bjv4mslqvqa3yxiyt23do1qim3ae3m7 2415363 2415362 2026-05-19T08:49:26Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudnet2005-dev.codfw.wmnet 2415363 wikitext text/x-wiki qc7v0ovdpqnoqxyqj52phxncajdsyk9 2415364 2415363 2026-05-19T08:50:12Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1289275|Remove unused $wgEnableUserEmailMuteList config (T413867)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2415364 wikitext text/x-wiki a12nkk99jbd9sbtcdn0mynh0hs9xpve 2415365 2415364 2026-05-19T08:50:30Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-rw1001.wikimedia.org 2415365 wikitext text/x-wiki n73jjnbeiufh7ar5r6322haf4qddv8p 2415367 2415365 2026-05-19T08:51:15Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2415367 wikitext text/x-wiki g48dakt7q1es5dx7rxds9bvrrlgp20i 2415368 2415367 2026-05-19T08:52:30Z Stashbot 7414 volans@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudrabbit1002.eqiad.wmnet 2415368 wikitext text/x-wiki j7dqbo5am248rhkpjwoo3lvbehdmgt6 2415370 2415368 2026-05-19T08:55:27Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1289275|Remove unused $wgEnableUserEmailMuteList config (T413867)]] (duration: 07m 15s) 2415370 wikitext text/x-wiki odezlqd2l62fbx7ka3zkwwcvgt16jil 2415372 2415370 2026-05-19T08:57:39Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudnet2005-dev.codfw.wmnet 2415372 wikitext text/x-wiki 9ap5qza657kxebo99pb5otfqjzqig3q 2415373 2415372 2026-05-19T08:57:43Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudnet2006-dev.codfw.wmnet 2415373 wikitext text/x-wiki eswk5jo6fxlhgyflv8wjx3alcdrn9uv 2415374 2415373 2026-05-19T08:58:20Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ldap-rw2001.wikimedia.org 2415374 wikitext text/x-wiki j7v6dclxky1le89pcepmjhicu13vqgr 2415375 2415374 2026-05-19T08:58:28Z Stashbot 7414 volans@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudrabbit1002.eqiad.wmnet 2415375 wikitext text/x-wiki q2hkpmripn78g7hebn67n1nlmscmg2t 2415376 2415375 2026-05-19T08:59:21Z Stashbot 7414 volans@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudrabbit1001.eqiad.wmnet 2415376 wikitext text/x-wiki qn1tfmqcywtfkuoqgae8b4gj9esmoqe 2415377 2415376 2026-05-19T09:00:48Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2244.codfw.wmnet with reason: host reimage 2415377 wikitext text/x-wiki je3adb2hfd0gtc94v9sejq6dpv34nx7 2415378 2415377 2026-05-19T09:02:24Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-rw2001.wikimedia.org 2415378 wikitext text/x-wiki 7vbq5pdcig9rj4t757c3cld93fho7qv 2415379 2415378 2026-05-19T09:03:41Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2244.codfw.wmnet with reason: host reimage 2415379 wikitext text/x-wiki liyjd4wwskq5qozmcn2w93tfv3mdijx 2415380 2415379 2026-05-19T09:04:02Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.memcached.roll-reboot-restart (exit_code=0) rolling reboot on A:memcached-gutter-codfw 2415380 wikitext text/x-wiki t5rpnfeh6ts71116hncl3z8kwu5r6rf 2415381 2415380 2026-05-19T09:05:32Z Stashbot 7414 volans@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudrabbit1001.eqiad.wmnet 2415381 wikitext text/x-wiki ao6dsygy4oxzqkm4no5h7nbek28iphm 2415382 2415381 2026-05-19T09:05:34Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudnet2006-dev.codfw.wmnet 2415382 wikitext text/x-wiki hby8h04stizsjp80lsrnr3sh6r01xcz 2415383 2415382 2026-05-19T09:07:55Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-worker1006.eqiad.wmnet 2415383 wikitext text/x-wiki 3isbhfgi0ee92t9zit3twxtqa2t03g4 2415384 2415383 2026-05-19T09:08:30Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-worker1006.eqiad.wmnet 2415384 wikitext text/x-wiki 7e1kdyxj71pc747oczitx21ffabsorn 2415385 2415384 2026-05-19T09:13:17Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudnet1006.eqiad.wmnet 2415385 wikitext text/x-wiki d1ollkakrgsa9b8jerhdxcgrfp7yjf1 2415386 2415385 2026-05-19T09:13:23Z Stashbot 7414 filippo@cumin1003: END (ERROR) - Cookbook sre.hosts.reboot-single (exit_code=97) for host cloudnet1006.eqiad.wmnet 2415386 wikitext text/x-wiki a7wo5rvnxwlxr43iib8i4zub6yb5vpl 2415387 2415386 2026-05-19T09:13:33Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudnet1005.eqiad.wmnet 2415387 wikitext text/x-wiki dg74l43itis1ngha4ruxeltsgg6661l 2415388 2415387 2026-05-19T09:13:38Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-worker1006.eqiad.wmnet 2415388 wikitext text/x-wiki bqfj2y56xz2011kx8krdvu7l1epjhws 2415389 2415388 2026-05-19T09:13:40Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-worker1006.eqiad.wmnet 2415389 wikitext text/x-wiki 6fppei8d4invdc3cu118htve9lh6ddb 2415390 2415389 2026-05-19T09:13:44Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-worker1007.eqiad.wmnet 2415390 wikitext text/x-wiki t5kqv4ib4n85w5oh7hqkqzv2i1pf7c9 2415391 2415390 2026-05-19T09:14:20Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-worker1007.eqiad.wmnet 2415391 wikitext text/x-wiki 8iwsr52p53q0yvmo4yra3jrgtrgfkq1 2415392 2415391 2026-05-19T09:18:00Z Stashbot 7414 jiji@deploy1003: helmfile [codfw] START helmfile.d/services/mw-mcrouter: apply 2415392 wikitext text/x-wiki hzw6gzz4p09c48jwfugsxer81jdgpl4 2415393 2415392 2026-05-19T09:18:13Z Stashbot 7414 jiji@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-mcrouter: apply 2415393 wikitext text/x-wiki mfwajc9p2fewfbe4k4rro76cnpn1pn2 2415394 2415393 2026-05-19T09:18:17Z Stashbot 7414 jiji@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-mcrouter: apply 2415394 wikitext text/x-wiki 0cv1m1bts7z4k7mxnzpo9jygvvfajfh 2415395 2415394 2026-05-19T09:18:22Z Stashbot 7414 jiji@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-mcrouter: apply 2415395 wikitext text/x-wiki ttnut9hzay2mptfsedxir24g9j3hdoh 2415396 2415395 2026-05-19T09:19:29Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-worker1007.eqiad.wmnet 2415396 wikitext text/x-wiki fvvfoo5jugbuyf1o95pb9slxt3wflos 2415397 2415396 2026-05-19T09:19:31Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-worker1007.eqiad.wmnet 2415397 wikitext text/x-wiki gpwmog3lbp1x0z0nxlum3na12q78s7b 2415398 2415397 2026-05-19T09:19:36Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-worker1008.eqiad.wmnet 2415398 wikitext text/x-wiki mdjk4tshn5ji0cd7tafs7wpe73olmsm 2415399 2415398 2026-05-19T09:20:10Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4005.ulsfo.wmnet 2415399 wikitext text/x-wiki 566ohalmur8g2vy2nbmjbvija7s00b6 2415400 2415399 2026-05-19T09:20:12Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-worker1008.eqiad.wmnet 2415400 wikitext text/x-wiki qtepogmpz6q4g6j9zh7b6u3bfp8x51h 2415401 2415400 2026-05-19T09:20:47Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2244.codfw.wmnet with OS trixie 2415401 wikitext text/x-wiki lo1ujy56yy1ne4zojwyi0w45plafd4r 2415402 2415401 2026-05-19T09:20:51Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudnet1005.eqiad.wmnet 2415402 wikitext text/x-wiki 4i8zntnkdq6l77n20f6onffh946pp8w 2415403 2415402 2026-05-19T09:22:55Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2244: Migration of db2244.codfw.wmnet completed 2415403 wikitext text/x-wiki bqcyh15fnjn8cpt6i92c6jztn80gocw 2415404 2415403 2026-05-19T09:23:24Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudnet1006.eqiad.wmnet 2415404 wikitext text/x-wiki t6fkbokp4b5ot4ir22fx89fgu5h6sle 2415405 2415404 2026-05-19T09:23:59Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti4005.ulsfo.wmnet 2415405 wikitext text/x-wiki 2r9w7qoguc32hseo1p3ap28y3x4m8aa 2415406 2415405 2026-05-19T09:25:19Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-worker1008.eqiad.wmnet 2415406 wikitext text/x-wiki lci3764m0gjeqb84l89mi88rwpnofd7 2415407 2415406 2026-05-19T09:25:21Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-worker1008.eqiad.wmnet 2415407 wikitext text/x-wiki 7gi50f5anxllximgb913ey79aejq2n1 2415408 2415407 2026-05-19T09:25:26Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-worker1009.eqiad.wmnet 2415408 wikitext text/x-wiki gbmxte3r9kohs2q7h28p0of90j4terl 2415409 2415408 2026-05-19T09:26:01Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-worker1009.eqiad.wmnet 2415409 wikitext text/x-wiki ontvf4tpqi25tx6o684g4jy77b3svsl 2415410 2415409 2026-05-19T09:27:17Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host maps-test2001.codfw.wmnet 2415410 wikitext text/x-wiki 6zm5b76gbcjie0tisbi1al54exhohy2 2415411 2415410 2026-05-19T09:30:20Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti4005.ulsfo.wmnet 2415411 wikitext text/x-wiki 4rcvbd7txsqxm034fze2byg7cl6wgwm 2415412 2415411 2026-05-19T09:30:26Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4005.ulsfo.wmnet 2415412 wikitext text/x-wiki b35e23ksevo86tuhkdnudbvxg813kxn 2415414 2415412 2026-05-19T09:31:19Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-worker1009.eqiad.wmnet 2415414 wikitext text/x-wiki rhw6oh0auy4neba9h8djhwmxo1yovq3 2415416 2415414 2026-05-19T09:31:21Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-worker1009.eqiad.wmnet 2415416 wikitext text/x-wiki n30yqj3ze0hzpp8mu4n3wrj9evg43ls 2415417 2415416 2026-05-19T09:31:23Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on A:aux-worker-eqiad 2415417 wikitext text/x-wiki k13r7ulwowfemr60y8fssce6to2ljf9 2415418 2415417 2026-05-19T09:31:24Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudnet1006.eqiad.wmnet 2415418 wikitext text/x-wiki hp083eotug0txucrhlz5tw9tksrx425 2415420 2415418 2026-05-19T09:34:50Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudcontrol2005-dev.codfw.wmnet 2415420 wikitext text/x-wiki 78um6b8blwmqu1cs4ydif36cx3oc85r 2415421 2415420 2026-05-19T09:35:51Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host maps-test2001.codfw.wmnet 2415421 wikitext text/x-wiki 19g6i4gdnq5jf72lm15e3bsrfdpz1vb 2415423 2415421 2026-05-19T09:38:10Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.network.peering with action 'email' for AS: 40401 2415423 wikitext text/x-wiki ciqtvn4dt3mo9y0vk1xww0ehv5hr4cg 2415424 2415423 2026-05-19T09:38:56Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 40401 2415424 wikitext text/x-wiki b6imen4z6roqmnq4bkwo7rw8nl1h4yc 2415426 2415424 2026-05-19T09:42:52Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcontrol2005-dev.codfw.wmnet 2415426 wikitext text/x-wiki nwagoovrfjipcz6t2729dak4vy77q9f 2415427 2415426 2026-05-19T09:42:56Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudcontrol2006-dev.codfw.wmnet 2415427 wikitext text/x-wiki blqmz1txws37c6rju1b25siq7jsvcjj 2415428 2415427 2026-05-19T09:43:31Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reboot-single for host rdb2012.codfw.wmnet 2415428 wikitext text/x-wiki 6e9o3i4nuuvtc0kq4u6yazmc6wnl444 2415429 2415428 2026-05-19T09:47:07Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revise-tone-task-generator' for release 'main' . 2415429 wikitext text/x-wiki idbydntsu3115o2lwtcthxl73gtzvg3 2415430 2415429 2026-05-19T09:48:58Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host rdb2012.codfw.wmnet 2415430 wikitext text/x-wiki e31rd4ri85xtgaznocbakeepu40ofls 2415431 2415430 2026-05-19T09:51:19Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcontrol2006-dev.codfw.wmnet 2415431 wikitext text/x-wiki a2bta3vgnfed8hsm5n5ekx94na2cijz 2415432 2415431 2026-05-19T09:51:22Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudcontrol2010-dev.codfw.wmnet 2415432 wikitext text/x-wiki qwqu9si5vvwm7fapcn0gwenxu4dpnu8 2415433 2415432 2026-05-19T09:53:18Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reboot-single for host rdb2011.codfw.wmnet 2415433 wikitext text/x-wiki p2tnt8gqrxo1o8jowkublb3g6bk3ptr 2415435 2415433 2026-05-19T09:55:22Z Stashbot 7414 cgoubert@cumin1003: START - Cookbook sre.kafka.roll-restart-reboot-brokers rolling reboot on A:kafka-main-codfw 2415435 wikitext text/x-wiki 70hvflvubcfju3763ye71pvl6ztmf3d 2415436 2415435 2026-05-19T09:58:22Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.hosts.reboot-single for host titan1001.eqiad.wmnet 2415436 wikitext text/x-wiki azdvdsgpwy1lenx372msyjiux9siodz 2415437 2415436 2026-05-19T09:58:46Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host rdb2011.codfw.wmnet 2415437 wikitext text/x-wiki 541lk1u7ib7nk8nxhtb7f9sy1xrjzry 2415438 2415437 2026-05-19T10:00:26Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcontrol2010-dev.codfw.wmnet 2415438 wikitext text/x-wiki pc43i42lixfpqyrdp9zq3my9j4sc3io 2415439 2415438 2026-05-19T10:00:30Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudcontrol1008-dev.eqiad.wmnet 2415439 wikitext text/x-wiki hi5xs4xrztk3dsuopfn9i3kqj27cf2u 2415440 2415439 2026-05-19T10:05:52Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host titan1001.eqiad.wmnet 2415440 wikitext text/x-wiki h79fc3113syxc7qxshux0ggpij58cbw 2415442 2415440 2026-05-19T10:07:08Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcontrol1008-dev.eqiad.wmnet 2415442 wikitext text/x-wiki pia05esd6ts6k01svg0oinkuu8ggtvm 2415443 2415442 2026-05-19T10:08:24Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2244: Migration of db2244.codfw.wmnet completed 2415443 wikitext text/x-wiki g5q5rqqmivi54kpojwwall87df1ind6 2415444 2415443 2026-05-19T10:08:26Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2415444 wikitext text/x-wiki 812caggth3rhswfyiumd5o7h6jul5gg 2415445 2415444 2026-05-19T10:10:59Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.hosts.reboot-single for host titan2002.codfw.wmnet 2415445 wikitext text/x-wiki a87nx0wj1xo8864xjzm83261op9zy3p 2415446 2415445 2026-05-19T10:15:36Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1266.eqiad.wmnet with OS trixie 2415446 wikitext text/x-wiki bowesmx5hnbzx5c63bn118xsatgy7yp 2415447 2415446 2026-05-19T10:18:50Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host titan2002.codfw.wmnet 2415447 wikitext text/x-wiki 4ka2vq60nq1ryvi1l5d3vjexrzwdiuv 2415448 2415447 2026-05-19T10:19:18Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1267.eqiad.wmnet with OS trixie 2415448 wikitext text/x-wiki 06hjtrbajjabkib9cgm2xt1b2mlvuvl 2415449 2415448 2026-05-19T10:19:43Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1268.eqiad.wmnet with OS trixie 2415449 wikitext text/x-wiki jfo5wvvx37q4klqobvbtj2y1adg3x5g 2415450 2415449 2026-05-19T10:20:04Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1269.eqiad.wmnet with OS trixie 2415450 wikitext text/x-wiki ar5dvi32pi66vaovtodh84dwfwke6km 2415451 2415450 2026-05-19T10:21:18Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1270.eqiad.wmnet with OS trixie 2415451 wikitext text/x-wiki b771ump1sw3n2p0yl0jm9fbru43w5o7 2415453 2415451 2026-05-19T10:22:22Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1271.eqiad.wmnet with OS trixie 2415453 wikitext text/x-wiki lhrjrsamn7j65zbkkg157g0y5srwqo9 2415455 2415453 2026-05-19T10:22:47Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1272.eqiad.wmnet with OS trixie 2415455 wikitext text/x-wiki eg7h3os1tv51xgcbwnzhekjpbxtdhvw 2415456 2415455 2026-05-19T10:23:14Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1273.eqiad.wmnet with OS trixie 2415456 wikitext text/x-wiki n6i9ils1n0p91kq5k0ovd1u10425ho6 2415457 2415456 2026-05-19T10:23:33Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1274.eqiad.wmnet with OS trixie 2415457 wikitext text/x-wiki gtjqlb6fpm245dzkt47r97dj7s9wojl 2415458 2415457 2026-05-19T10:24:06Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1275.eqiad.wmnet with OS trixie 2415458 wikitext text/x-wiki lcaqg52xsmbsmnbrtvsaynztg9ur1sy 2415459 2415458 2026-05-19T10:24:27Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1276.eqiad.wmnet with OS trixie 2415459 wikitext text/x-wiki 038rn9k51dbfl9ra96yup7isywn2lwz 2415460 2415459 2026-05-19T10:24:30Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host moss-be1003.eqiad.wmnet 2415460 wikitext text/x-wiki g2dl9ar2jqwjrqlvybwpyoekw0prvfr 2415461 2415460 2026-05-19T10:26:43Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.reboot-nodes rolling reboot on D{aux-k8s-worker100[2-5].eqiad.wmnet} and (A:aux-master-eqiad or A:aux-worker-eqiad) 2415461 wikitext text/x-wiki f2fu0wkia612ejqwx9yj7eyynz9fv8y 2415462 2415461 2026-05-19T10:26:47Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-worker1002.eqiad.wmnet 2415462 wikitext text/x-wiki t7qjiz70pzga0gmu2q8opgfbonqvlwb 2415463 2415462 2026-05-19T10:29:01Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1266.eqiad.wmnet with reason: host reimage 2415463 wikitext text/x-wiki l41afegptljm2wvc40la0yrp1dh0rdv 2415464 2415463 2026-05-19T10:31:12Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host moss-be1003.eqiad.wmnet 2415464 wikitext text/x-wiki 5e5ogp0e4kiwey1ylz7bakbo62oxj3u 2415465 2415464 2026-05-19T10:32:02Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host apus-be1006.eqiad.wmnet 2415465 wikitext text/x-wiki riv3b6kp76fbmiwy5as5m4t74d146zk 2415466 2415465 2026-05-19T10:32:03Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-worker1002.eqiad.wmnet 2415466 wikitext text/x-wiki f22ad7ru8vgfgldh8aur04zcbjc11d5 2415467 2415466 2026-05-19T10:32:50Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1267.eqiad.wmnet with reason: host reimage 2415467 wikitext text/x-wiki e7icbn6rpy939gowkno8ymtzh5jvujr 2415468 2415467 2026-05-19T10:33:15Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1268.eqiad.wmnet with reason: host reimage 2415468 wikitext text/x-wiki q5yfal595vm6mqnqvexuvwbfrb14f8x 2415469 2415468 2026-05-19T10:33:17Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1269.eqiad.wmnet with reason: host reimage 2415469 wikitext text/x-wiki 8r0n9qdlv9i21fctxgbz1nibvblixz0 2415470 2415469 2026-05-19T10:34:11Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1266.eqiad.wmnet with reason: host reimage 2415470 wikitext text/x-wiki dp107f0msubwpl2xz88u2b5btdyng4w 2415471 2415470 2026-05-19T10:34:14Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1270.eqiad.wmnet with reason: host reimage 2415471 wikitext text/x-wiki aezcmjcsvl0lrtoyfx8hhexzaop948x 2415472 2415471 2026-05-19T10:35:40Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1271.eqiad.wmnet with reason: host reimage 2415472 wikitext text/x-wiki 0lsjq8m904s4enm1aowqvaygkf67cvw 2415473 2415472 2026-05-19T10:36:04Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-worker1002.eqiad.wmnet 2415473 wikitext text/x-wiki 0bgsxofkanepdznto8dbdn9tid0vcce 2415474 2415473 2026-05-19T10:36:05Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-worker1002.eqiad.wmnet 2415474 wikitext text/x-wiki m2kka2l13szrjrq0hlloz4go50oeezy 2415475 2415474 2026-05-19T10:36:11Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-worker1003.eqiad.wmnet 2415475 wikitext text/x-wiki bu4tn2tus31gpxu2vjw8n2u96s6gdn4 2415476 2415475 2026-05-19T10:36:19Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1274.eqiad.wmnet with reason: host reimage 2415476 wikitext text/x-wiki g67i4qrnud68oyh5iceo6pdyjiu8xwo 2415477 2415476 2026-05-19T10:36:21Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1272.eqiad.wmnet with reason: host reimage 2415477 wikitext text/x-wiki 1dgpmjkvd0k8h7mkunlxhkwoqrrrgcj 2415478 2415477 2026-05-19T10:36:23Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1273.eqiad.wmnet with reason: host reimage 2415478 wikitext text/x-wiki q3np8i6dxg1knrj3iqjcfbj2owbrw6s 2415479 2415478 2026-05-19T10:36:42Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-worker1003.eqiad.wmnet 2415479 wikitext text/x-wiki 3p0qjk3ki6d4qmnsn84o84wvkwu621i 2415480 2415479 2026-05-19T10:36:48Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on pc2021.codfw.wmnet,pc[1011,1021].eqiad.wmnet with reason: Maintenance on pc1 2415480 wikitext text/x-wiki obloizkmbxs4jooh3mi3h7ddwctg0vr 2415481 2415480 2026-05-19T10:37:32Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1275.eqiad.wmnet with reason: host reimage 2415481 wikitext text/x-wiki qjvt4tf7azbbfuj4vd3xxb78pqg1gqu 2415482 2415481 2026-05-19T10:37:34Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1267.eqiad.wmnet with reason: host reimage 2415482 wikitext text/x-wiki au6yxtfbt58rcwanp6wgzjrpjpe43yg 2415483 2415482 2026-05-19T10:37:52Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1276.eqiad.wmnet with reason: host reimage 2415483 wikitext text/x-wiki hcqxs14g3092y3wljznyfds049mfly5 2415484 2415483 2026-05-19T10:38:05Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host apus-be1006.eqiad.wmnet 2415484 wikitext text/x-wiki 1udxdz7eu9tplqe7f93u8bo5j9uxfme 2415485 2415484 2026-05-19T10:38:40Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host apus-be1004.eqiad.wmnet 2415485 wikitext text/x-wiki r6b75tjjriaefn2tegwsjb4bi42psqr 2415486 2415485 2026-05-19T10:40:46Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-worker1003.eqiad.wmnet 2415486 wikitext text/x-wiki gfpladzmfiix7w5jx5096czqv2plaou 2415487 2415486 2026-05-19T10:40:47Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-worker1003.eqiad.wmnet 2415487 wikitext text/x-wiki kbv1v4zky9rkuyn1bd3550iwtj3x37h 2415488 2415487 2026-05-19T10:40:53Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-worker1004.eqiad.wmnet 2415488 wikitext text/x-wiki 2otmn14je2y9qjfdk4fcgr3nmlf8ze7 2415489 2415488 2026-05-19T10:41:02Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1271.eqiad.wmnet with reason: host reimage 2415489 wikitext text/x-wiki mpc96sjxktko3n43cosmujddc734sd0 2415490 2415489 2026-05-19T10:41:24Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-worker1004.eqiad.wmnet 2415490 wikitext text/x-wiki 6p3etdd9574ppvl6kntwc7s7wxz2408 2415492 2415490 2026-05-19T10:43:46Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1268.eqiad.wmnet with reason: host reimage 2415492 wikitext text/x-wiki 2zm4zdblec3hc6pdh2b497b0qpcsqwy 2415493 2415492 2026-05-19T10:44:43Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host apus-be1004.eqiad.wmnet 2415493 wikitext text/x-wiki if9w8hmbe3bibrnza5q58c7j13kr08k 2415494 2415493 2026-05-19T10:45:15Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host apus-be1005.eqiad.wmnet 2415494 wikitext text/x-wiki jvzde7fuabqdlzu52osmrjtd0mqwu27 2415495 2415494 2026-05-19T10:45:24Z Stashbot 7414 cgoubert@cumin1003: END (PASS) - Cookbook sre.kafka.roll-restart-reboot-brokers (exit_code=0) rolling reboot on A:kafka-main-codfw 2415495 wikitext text/x-wiki m9i0kkfk3k7wxib9ic0vvu4km8eshqx 2415496 2415495 2026-05-19T10:45:26Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-worker1004.eqiad.wmnet 2415496 wikitext text/x-wiki i0g370l48zvygo895zhdbafrnl80uxt 2415497 2415496 2026-05-19T10:45:28Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-worker1004.eqiad.wmnet 2415497 wikitext text/x-wiki 4fcj1ed6ch2v0sz28vel09fntt1a6f3 2415498 2415497 2026-05-19T10:45:33Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-worker1005.eqiad.wmnet 2415498 wikitext text/x-wiki 3h3r06vinban5xem0xoxzy6siuqnr7j 2415499 2415498 2026-05-19T10:46:04Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-worker1005.eqiad.wmnet 2415499 wikitext text/x-wiki kxlp09z6s0g9t1z5y0ee72fijyjn58j 2415500 2415499 2026-05-19T10:47:35Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1274.eqiad.wmnet with reason: host reimage 2415500 wikitext text/x-wiki q2sqfyrh5644lntqc5y3e0zqrqf8a1x 2415501 2415500 2026-05-19T10:49:07Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1266.eqiad.wmnet with OS trixie 2415501 wikitext text/x-wiki du0g1jqjfhw183aimn5a24y00plfg3l 2415502 2415501 2026-05-19T10:49:37Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.hosts.reboot-single for host prometheus1005.eqiad.wmnet 2415502 wikitext text/x-wiki cpzk33wibrpc31d3bvark1e9x0iwzsh 2415503 2415502 2026-05-19T10:50:07Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-worker1005.eqiad.wmnet 2415503 wikitext text/x-wiki i3k29v89o5ce1ulbah9w2rjrr6fammc 2415504 2415503 2026-05-19T10:50:09Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-worker1005.eqiad.wmnet 2415504 wikitext text/x-wiki b4cvbhrxwapi4jhsuxcsgjzi9yyvyh3 2415505 2415504 2026-05-19T10:50:10Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on D{aux-k8s-worker100[2-5].eqiad.wmnet} and (A:aux-master-eqiad or A:aux-worker-eqiad) 2415505 wikitext text/x-wiki dbw12jtk9qm8djd6dre4j93zd7exziu 2415506 2415505 2026-05-19T10:50:43Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host apus-be1005.eqiad.wmnet 2415506 wikitext text/x-wiki qsd40znhsn97efs5p6g2vfygv53ipxm 2415507 2415506 2026-05-19T10:51:20Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1275.eqiad.wmnet with reason: host reimage 2415507 wikitext text/x-wiki ayntkhj3p8afi25xz2ro4ru2y7jui5o 2415508 2415507 2026-05-19T10:51:28Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5004.eqsin.wmnet 2415508 wikitext text/x-wiki 11mh323csjvrikrrulj56pioogyv3af 2415509 2415508 2026-05-19T10:51:50Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1267.eqiad.wmnet with OS trixie 2415509 wikitext text/x-wiki qyg17xwygtwwgwxn5segglnw41jzt28 2415510 2415509 2026-05-19T10:52:30Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7001.magru.wmnet 2415510 wikitext text/x-wiki 78bst0i6jj78o66nm2t0qj2x2z5vve9 2415511 2415510 2026-05-19T10:53:20Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.hosts.reboot-single for host titan1002.eqiad.wmnet 2415511 wikitext text/x-wiki ryox08txn1pu426indtazluz2e8e6us 2415512 2415511 2026-05-19T10:54:58Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti5004.eqsin.wmnet 2415512 wikitext text/x-wiki nes96zom48t11ofgvm6ruv8d1c1ob81 2415513 2415512 2026-05-19T10:55:01Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti7001.magru.wmnet 2415513 wikitext text/x-wiki 41kcz3b16f0qne94bevx4id0sxhbo0p 2415514 2415513 2026-05-19T10:55:17Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1271.eqiad.wmnet with OS trixie 2415514 wikitext text/x-wiki qs286oqxfu6lkp5qim2v7bz8ifi1p5p 2415515 2415514 2026-05-19T10:55:45Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1270.eqiad.wmnet with reason: host reimage 2415515 wikitext text/x-wiki btl1ajql2ylm68avwl0h5wol37ixkrm 2415517 2415515 2026-05-19T10:57:49Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1268.eqiad.wmnet with OS trixie 2415517 wikitext text/x-wiki m73rice6foa1kju8l7s1z4kpprrs8t7 2415518 2415517 2026-05-19T10:59:31Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus1005.eqiad.wmnet 2415518 wikitext text/x-wiki 00lgspxx2nsqdve45t834i1ohaa9ekl 2415519 2415518 2026-05-19T10:59:35Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.hosts.reboot-single for host prometheus1007.eqiad.wmnet 2415519 wikitext text/x-wiki cmzo43wvlbz6gj9vweqfov5rwd518ky 2415520 2415519 2026-05-19T10:59:37Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1272.eqiad.wmnet with reason: host reimage 2415520 wikitext text/x-wiki 4eanavtz488477hl9qhrzev1942oynv 2415521 2415520 2026-05-19T11:00:34Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host titan1002.eqiad.wmnet 2415521 wikitext text/x-wiki o4y9suzx6eujr4xsqomwc35gcz7t69j 2415522 2415521 2026-05-19T11:02:12Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1274.eqiad.wmnet with OS trixie 2415522 wikitext text/x-wiki hj8j2o6y44wnrjipnza4poesuo1o88y 2415523 2415522 2026-05-19T11:02:21Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1273.eqiad.wmnet with reason: host reimage 2415523 wikitext text/x-wiki rnco57tnn8a9aogseq76sj4sj7f0nmj 2415524 2415523 2026-05-19T11:03:11Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti5004.eqsin.wmnet 2415524 wikitext text/x-wiki sz1rxvohfuqj4hpauabd2nyzun9cbwb 2415525 2415524 2026-05-19T11:03:13Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti7001.magru.wmnet 2415525 wikitext text/x-wiki qtv53shu8bioks4lx90thv72d5w5o33 2415526 2415525 2026-05-19T11:03:17Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7001.magru.wmnet 2415526 wikitext text/x-wiki g99z2v78vng5cx4isynt09kgv8qam0t 2415527 2415526 2026-05-19T11:03:19Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti5004.eqsin.wmnet 2415527 wikitext text/x-wiki k9q2hvadlbasi51l4cmjph064gaqqwr 2415528 2415527 2026-05-19T11:05:07Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1275.eqiad.wmnet with OS trixie 2415528 wikitext text/x-wiki 5at3pkpml4z6x1qv2rhsyk5fww1ik8a 2415529 2415528 2026-05-19T11:05:24Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5005.eqsin.wmnet 2415529 wikitext text/x-wiki 5re4amlns1uvwx0r9uxj53ykzfggwwp 2415530 2415529 2026-05-19T11:05:30Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7002.magru.wmnet 2415530 wikitext text/x-wiki 36j78g6i62pd1ucbol9nq2kd106wac8 2415531 2415530 2026-05-19T11:06:02Z Stashbot 7414 cgoubert@cumin1003: START - Cookbook sre.kafka.roll-restart-reboot-brokers rolling reboot on A:kafka-main-eqiad 2415531 wikitext text/x-wiki 7eed0pky4da0jyjpp8088yb2alu3hex 2415532 2415531 2026-05-19T11:06:44Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1269.eqiad.wmnet with reason: host reimage 2415532 wikitext text/x-wiki 7r7ngv1amnnsbn28olaro0pvqcr2gqu 2415533 2415532 2026-05-19T11:07:25Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus1007.eqiad.wmnet 2415533 wikitext text/x-wiki 9tf1mg282ccjhhoirlk0562wbrfiahy 2415534 2415533 2026-05-19T11:07:30Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.hosts.reboot-single for host prometheus3004.esams.wmnet 2415534 wikitext text/x-wiki 7wia6t9kt9ttj9jaw8huwzgalczehvg 2415536 2415534 2026-05-19T11:09:56Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1270.eqiad.wmnet with OS trixie 2415536 wikitext text/x-wiki bose3aakfoh4wdbnhgqu2w0w482fmtm 2415537 2415536 2026-05-19T11:10:01Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti5005.eqsin.wmnet 2415537 wikitext text/x-wiki 4j9q0a8a2qwyx2ht9y74ifj9yhxozd3 2415538 2415537 2026-05-19T11:10:03Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti7002.magru.wmnet 2415538 wikitext text/x-wiki 1j42f5huq9detlvxk62ojw1i94e00ws 2415539 2415538 2026-05-19T11:10:20Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.update-replication 2415539 wikitext text/x-wiki nhjg611iq78soc2s54ksbn4ykufxoin 2415540 2415539 2026-05-19T11:10:34Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.update-replication (exit_code=0) 2415540 wikitext text/x-wiki tet0ikvnyzr23wwtu9rbu1neqpaxg35 2415541 2415540 2026-05-19T11:10:44Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1276.eqiad.wmnet with reason: host reimage 2415541 wikitext text/x-wiki pievkeiiikvqwi9t4k3k4r9c70v0afn 2415545 2415541 2026-05-19T11:13:33Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus3004.esams.wmnet 2415545 wikitext text/x-wiki cnrvota66easczv081jv0sorp1jn63o 2415546 2415545 2026-05-19T11:13:38Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.hosts.reboot-single for host prometheus4003.ulsfo.wmnet 2415546 wikitext text/x-wiki t3b19sm9ikglmbw4jodv33yg74h4qo9 2415547 2415546 2026-05-19T11:14:12Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1272.eqiad.wmnet with OS trixie 2415547 wikitext text/x-wiki d6ui8baydjxzcroy3lo4e22tr8rx0yf 2415552 2415547 2026-05-19T11:16:16Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1273.eqiad.wmnet with OS trixie 2415552 wikitext text/x-wiki 0zp0ef3nc5d6uqjg8x36p46bbkbr0l4 2415555 2415552 2026-05-19T11:18:13Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti5005.eqsin.wmnet 2415555 wikitext text/x-wiki f74s44ga6pb0g9u5uxk9zzlkxrv1l4o 2415556 2415555 2026-05-19T11:18:15Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti7002.magru.wmnet 2415556 wikitext text/x-wiki b3cr2ofayapozj01neqwl9veyqt8ek5 2415557 2415556 2026-05-19T11:18:20Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti5005.eqsin.wmnet 2415557 wikitext text/x-wiki 423ti93aw3hbyl3wcnnluhp66uaoj76 2415558 2415557 2026-05-19T11:18:22Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7002.magru.wmnet 2415558 wikitext text/x-wiki onn2ciyd9mczosstg8bcb2wkv3f0gp6 2415559 2415558 2026-05-19T11:19:33Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus4003.ulsfo.wmnet 2415559 wikitext text/x-wiki 1ii8uc1h5bglvh1d1vt6qftnhjusz3b 2415560 2415559 2026-05-19T11:19:37Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.hosts.reboot-single for host prometheus1006.eqiad.wmnet 2415560 wikitext text/x-wiki jn1oeemlag6ms7b95owvmdivkjwssdd 2415561 2415560 2026-05-19T11:20:35Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5006.eqsin.wmnet 2415561 wikitext text/x-wiki qlj95bqp7ym9d9sr142ej7wahkghts5 2415562 2415561 2026-05-19T11:20:42Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7003.magru.wmnet 2415562 wikitext text/x-wiki m1alzkbu3b5t3v09rijdyy82r52bxbv 2415563 2415562 2026-05-19T11:21:09Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1269.eqiad.wmnet with OS trixie 2415563 wikitext text/x-wiki q545t8m7j6kp8m1ekcgaq5coqffwnxt 2415564 2415563 2026-05-19T11:24:46Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1276.eqiad.wmnet with OS trixie 2415564 wikitext text/x-wiki atfnzc07ge0n0xdayeazadv9et2m2s9 2415565 2415564 2026-05-19T11:24:58Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti5006.eqsin.wmnet 2415565 wikitext text/x-wiki g3pf8idck3v4d407g8nfwnq9xikzuwm 2415566 2415565 2026-05-19T11:25:00Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti7003.magru.wmnet 2415566 wikitext text/x-wiki j63ede2as747faacbekbyakuzg012sk 2415569 2415566 2026-05-19T11:29:34Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus1006.eqiad.wmnet 2415569 wikitext text/x-wiki eon6ykfdf4tygsdepsvslqnfv1gsid3 2415570 2415569 2026-05-19T11:29:38Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.hosts.reboot-single for host prometheus1008.eqiad.wmnet 2415570 wikitext text/x-wiki di1baboixxnmsvrcq1x6rzu7f4kmfx9 2415571 2415570 2026-05-19T11:31:14Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1277.eqiad.wmnet with OS trixie 2415571 wikitext text/x-wiki 95swiii6lao7kn0k4wqgcj0nkc28fn0 2415572 2415571 2026-05-19T11:31:33Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1278.eqiad.wmnet with OS trixie 2415572 wikitext text/x-wiki o6e31j1tx7526hz7sd16cuwo453upeo 2415573 2415572 2026-05-19T11:31:47Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1279.eqiad.wmnet with OS trixie 2415573 wikitext text/x-wiki k3gqd04pyznvyen08d6pf1cnomc2ygl 2415574 2415573 2026-05-19T11:32:10Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1280.eqiad.wmnet with OS trixie 2415574 wikitext text/x-wiki 7nlg4bluocn0j4bdbso4y5n8j4897cf 2415575 2415574 2026-05-19T11:32:31Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1281.eqiad.wmnet with OS trixie 2415575 wikitext text/x-wiki 21sa9j8xwsfm4dw5s1z2qal1c0cnadz 2415576 2415575 2026-05-19T11:33:23Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti5006.eqsin.wmnet 2415576 wikitext text/x-wiki faov67rzsx1rqbftb2me24z3v7szpfm 2415577 2415576 2026-05-19T11:33:25Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti7003.magru.wmnet 2415577 wikitext text/x-wiki ogudr1n9af7qrbnyypveumf5nvpur9d 2415578 2415577 2026-05-19T11:33:30Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7003.magru.wmnet 2415578 wikitext text/x-wiki 9sz0wd9dardig6azxor4ke0g9cajid6 2415579 2415578 2026-05-19T11:33:32Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti5006.eqsin.wmnet 2415579 wikitext text/x-wiki ffj3xz3q7mv6h2qmoojipd6mk4ee3hj 2415580 2415579 2026-05-19T11:34:20Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1282.eqiad.wmnet with OS trixie 2415580 wikitext text/x-wiki pp66j4pck8rmisdew6itiwq1b8pdmx1 2415581 2415580 2026-05-19T11:34:49Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1283.eqiad.wmnet with OS trixie 2415581 wikitext text/x-wiki 9z6295guwhakestky9sq2loxo7zbbj9 2415582 2415581 2026-05-19T11:34:58Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host moss-be2003.codfw.wmnet 2415582 wikitext text/x-wiki q7qdl84fl2h3iklsd07a2ccmaeftdy8 2415583 2415582 2026-05-19T11:35:09Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1284.eqiad.wmnet with OS trixie 2415583 wikitext text/x-wiki cdinbculqat2z8j8x9mbpjva2wh2y25 2415584 2415583 2026-05-19T11:35:34Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1286.eqiad.wmnet with OS trixie 2415584 wikitext text/x-wiki penxyaebp62aq0cu3wyyl8lwnyznkw1 2415585 2415584 2026-05-19T11:35:57Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1287.eqiad.wmnet with OS trixie 2415585 wikitext text/x-wiki 90k5o6u8hfr1qm9gw7y1npzx0q0vnt0 2415586 2415585 2026-05-19T11:36:17Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1288.eqiad.wmnet with OS trixie 2415586 wikitext text/x-wiki 7hep6jgjxt12emaybj6e2lq09drd2da 2415587 2415586 2026-05-19T11:37:35Z Stashbot 7414 moritzm: failover Ganeti cluster in magru to ganeti7001 2415587 wikitext text/x-wiki p0oh0v3064wllprhnd6ks909hzo7f9e 2415588 2415587 2026-05-19T11:37:45Z Stashbot 7414 moritzm: failover Ganeti cluster in eqsin to ganeti5004 2415588 wikitext text/x-wiki paraxksi39qnvq9oa0js4x4h91d526o 2415589 2415588 2026-05-19T11:39:40Z Stashbot 7414 jynus@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on ms-backup[1003-1004].eqiad.wmnet with reason: restart 2415589 wikitext text/x-wiki l5ffymsgcsqaegrlkequmk0rxlpruw3 2415590 2415589 2026-05-19T11:39:44Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus1008.eqiad.wmnet 2415590 wikitext text/x-wiki g1jxni7dl5c61h9og211ssqnaktxhrf 2415591 2415590 2026-05-19T11:39:50Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.hosts.reboot-single for host prometheus5003.eqsin.wmnet 2415591 wikitext text/x-wiki n95om76m8jgkd9pvj8xitjhqtdbrriy 2415592 2415591 2026-05-19T11:42:01Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host moss-be2003.codfw.wmnet 2415592 wikitext text/x-wiki ssjdpda9t916t64p83g08wl61od7pj0 2415593 2415592 2026-05-19T11:44:12Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1277.eqiad.wmnet with reason: host reimage 2415593 wikitext text/x-wiki 1hrqotp7894l6ylbyzxsz1a39shs2co 2415594 2415593 2026-05-19T11:44:43Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1278.eqiad.wmnet with reason: host reimage 2415594 wikitext text/x-wiki mxlxkn35m4l6j0brp5zng3jst3neg8w 2415595 2415594 2026-05-19T11:44:54Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1279.eqiad.wmnet with reason: host reimage 2415595 wikitext text/x-wiki 96fh70d0i8n6wj6nvlnwaxzz2zdfgwv 2415596 2415595 2026-05-19T11:45:38Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1280.eqiad.wmnet with reason: host reimage 2415596 wikitext text/x-wiki oi63uh4gjq1p9670cepddeqcw1ipzgz 2415597 2415596 2026-05-19T11:45:44Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1281.eqiad.wmnet with reason: host reimage 2415597 wikitext text/x-wiki 6d30wd48veiz6j2hc9ag5tjekvwuiap 2415598 2415597 2026-05-19T11:46:08Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus5003.eqsin.wmnet 2415598 wikitext text/x-wiki drq03k9nqblex8egqlqqfk2sotb7y29 2415599 2415598 2026-05-19T11:46:13Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.hosts.reboot-single for host prometheus2005.codfw.wmnet 2415599 wikitext text/x-wiki 7dmdz3djn3lynzqn85a6vhnit1qyam8 2415600 2415599 2026-05-19T11:47:24Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1282.eqiad.wmnet with reason: host reimage 2415600 wikitext text/x-wiki oojxagkki4pp3x5feq468n2myqw1ep5 2415601 2415600 2026-05-19T11:47:46Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1283.eqiad.wmnet with reason: host reimage 2415601 wikitext text/x-wiki pthyaqs2n5ewx8ppsrjy8uq5p32h5yg 2415602 2415601 2026-05-19T11:48:13Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1284.eqiad.wmnet with reason: host reimage 2415602 wikitext text/x-wiki 555qbyqo8kr5guo30xkhopnyi3j78ml 2415603 2415602 2026-05-19T11:48:32Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1286.eqiad.wmnet with reason: host reimage 2415603 wikitext text/x-wiki 2m4acxr79nc3wtdzccyrnd2vn6uxkdg 2415605 2415603 2026-05-19T11:49:25Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1288.eqiad.wmnet with reason: host reimage 2415605 wikitext text/x-wiki 120kabdzcmgjbvhnwifwy4ukgk963qt 2415606 2415605 2026-05-19T11:49:30Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1287.eqiad.wmnet with reason: host reimage 2415606 wikitext text/x-wiki q9cr8erwba5j0yediewevlffea6fes0 2415607 2415606 2026-05-19T11:49:42Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1277.eqiad.wmnet with reason: host reimage 2415607 wikitext text/x-wiki 0fzat6zbiyyxd51embrfwjvsn1l0rlh 2415608 2415607 2026-05-19T11:50:59Z Stashbot 7414 jynus@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on 18 hosts with reason: restart 2415608 wikitext text/x-wiki i4w1clxh0elsfhmhia9qplaw40zzkpj 2415611 2415608 2026-05-19T11:52:52Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1284.eqiad.wmnet with reason: host reimage 2415611 wikitext text/x-wiki 2ak2avg20um2qzx6kk66sjlp6t1a4v7 2415612 2415611 2026-05-19T11:53:23Z Stashbot 7414 taavi@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudidp2001-dev.codfw.wmnet 2415612 wikitext text/x-wiki olphveyi15pf92ewe96uc4c2i2hkxqx 2415613 2415612 2026-05-19T11:56:09Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus2005.codfw.wmnet 2415613 wikitext text/x-wiki nee5l10gxykmkmjhetdmrlp5cv2575f 2415614 2415613 2026-05-19T11:56:13Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.hosts.reboot-single for host prometheus2007.codfw.wmnet 2415614 wikitext text/x-wiki 3amt68erhdb8185q66o3mwt85mncx9f 2415615 2415614 2026-05-19T11:56:19Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reboot-single for host apus-be2006.codfw.wmnet 2415615 wikitext text/x-wiki lrkwwku6jnevrx61ru1g1hupqk3fy89 2415616 2415615 2026-05-19T11:56:36Z Stashbot 7414 cgoubert@cumin1003: END (PASS) - Cookbook sre.kafka.roll-restart-reboot-brokers (exit_code=0) rolling reboot on A:kafka-main-eqiad 2415616 wikitext text/x-wiki n14uymoh033ntvcieehncq4rhjh3uod 2415617 2415616 2026-05-19T11:56:38Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1282.eqiad.wmnet with reason: host reimage 2415617 wikitext text/x-wiki 5w1m74292m2n1c4pt5zbvcaequjy8ct 2415618 2415617 2026-05-19T11:57:13Z Stashbot 7414 taavi@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudidp2001-dev.codfw.wmnet 2415618 wikitext text/x-wiki 87yjohiidjl9ytc6ga0f2u2vgrxds7j 2415620 2415618 2026-05-19T11:58:45Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5007.eqsin.wmnet 2415620 wikitext text/x-wiki kqjndnc48s6oy3w9ky8cm414hbgi288 2415621 2415620 2026-05-19T11:58:47Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7004.magru.wmnet 2415621 wikitext text/x-wiki kjlgrz98puo6ji61i3kypss3jr7n1nc 0 17290 2414777 2414431 2026-05-18T14:27:39Z Stashbot 7414 hashar: Upgrading Quibble jobs to 1.18.0 2414777 wikitext text/x-wiki 66hsuho9hvgrxhsgswhruiey9djkhca 2414896 2414777 2026-05-18T15:52:56Z Stashbot 7414 dancy: gitlab-cloud-runners are in a broken state. I'm investigating 2414896 wikitext text/x-wiki kwn9enxzhaf2y68qgusk683tvfqvrak 2415051 2414896 2026-05-18T18:11:29Z Stashbot 7414 James_F: Zuul: [design/codex] Switch CI to Node 24 2415051 wikitext text/x-wiki 7kn049i9cjxejtgatxfa4syfx50xaq7 2415053 2415051 2026-05-18T18:12:52Z Stashbot 7414 dancy: gitlab-cloud-runners have been revived. 2415053 wikitext text/x-wiki 2l2cj5la902vg3p6rbj3hlix41dstg4 2415057 2415053 2026-05-18T18:14:50Z Stashbot 7414 mutante: rebooting production gitlab-runners 2415057 wikitext text/x-wiki n3sstgbkpznyffos5mq62uopkt0ljrg 2415060 2415057 2026-05-18T18:17:03Z Stashbot 7414 mutante: releases.wikimedia.org - rebooting backends 2415060 wikitext text/x-wiki ctpf6wcy11axx1oa56jw9apo1we9yy9 2415108 2415060 2026-05-18T19:58:01Z Stashbot 7414 James_F: Zuul: Migrate various production/CI things to Node 24 2415108 wikitext text/x-wiki rp0re7w93i3ui5kiz1oomz8bd49drnd 2415109 2415108 2026-05-18T20:02:44Z Stashbot 7414 James_F: Docker: [ajv, sonar-scanner] Migrate to Node 24 2415109 wikitext text/x-wiki nlta9j7o6yj1trldtgyy4tasxdxdfn4 2415115 2415109 2026-05-18T20:18:06Z Stashbot 7414 James_F: Zuul: Migrate various labs things to Node 24 2415115 wikitext text/x-wiki r2kopnledprchmf8yajfq0g9kchnvus 2415116 2415115 2026-05-18T20:18:15Z Stashbot 7414 James_F: Zuul: [wikimedia/fundraising/dash] Replace Node 20 testing with Node 24 2415116 wikitext text/x-wiki 8botj5g0l86q9kz7w7upy8b3pe6iqip 2415127 2415116 2026-05-18T20:56:15Z Stashbot 7414 James_F: Zuul: [analytics/wmde/scripts] Test in PHP 8.3+, dropping 8. 2415127 wikitext text/x-wiki 3emg1g5kpiczzq2x8f1r3pluwe9a0e7 2415128 2415127 2026-05-18T20:57:06Z Stashbot 7414 James_F: Zuul: [integration/docroot] Test in PHP 8.3+, dropping 8.2 2415128 wikitext text/x-wiki ldcmvw34z6hq8ljvdaffkq1dfunngkt 2415132 2415128 2026-05-18T21:10:56Z Stashbot 7414 mutante: gerrit-replica.wikimedia.org, gerrit-spare.wikimedia.org - rebooting backends 2415132 wikitext text/x-wiki scmg7lpnxy73yihylfhcqn99zpofwyd 2415133 2415132 2026-05-18T21:12:43Z Stashbot 7414 James_F: Zuul: [operations/software/gerrit] Add Node26 as experimental 2415133 wikitext text/x-wiki 1142rb4q5ecceuq9c01w2uhz7v74388 0 26946 2415152 2393221 2026-05-18T22:00:25Z Star walker 40724 Tidied up article 2415152 wikitext text/x-wiki im79a3vhlb99l9lfbx6v8uga0hpw2kp 10 26999 2414628 2404341 2026-05-18T12:14:19Z SLyngshede-WMF 31501 2414628 wikitext text/x-wiki intkxr835vftbc4b9g8rqtzflarxe4v 498 30942 2414605 2414505 2026-05-18T11:59:13Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch 2414605 wikitext text/x-wiki hhw4uu6xv9yfjo92tprcuniukrnfqsp 2414606 2414605 2026-05-18T12:00:00Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch 2414606 wikitext text/x-wiki nvhe2f8b1y6qwnpctpbsfufbnfgfdyq 2414608 2414606 2026-05-18T12:01:08Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/313 2414608 wikitext text/x-wiki i53uy9em58md07q4xqntsu0dq103nef 2414609 2414608 2026-05-18T12:01:35Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/313 2414609 wikitext text/x-wiki 7ok4pjzuujvo4ego8hh8x4kve9lrv16 2415121 2414609 2026-05-18T20:33:52Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.roll_reboot_osds (T426563) 2415121 wikitext text/x-wiki h6ba1hjvyqgdrie12fy6ite9xcxbwfs 2415122 2415121 2026-05-18T20:33:56Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.roll_reboot_osds (exit_code=99) (T426563) 2415122 wikitext text/x-wiki ibjvkcg18wa387zlc1c2brfpgxgpk9w 2415123 2415122 2026-05-18T20:34:33Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.roll_reboot_osds (T426563) 2415123 wikitext text/x-wiki j78nxz4rd8tvoqou04r6qvuekw5bb3v 2415124 2415123 2026-05-18T20:34:38Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.roll_reboot_osds (exit_code=99) (T426563) 2415124 wikitext text/x-wiki lprmoqp1ee0e4rk5v5ogptvu3gmjeuo 2415125 2415124 2026-05-18T20:35:08Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.roll_reboot_osds (T426563) 2415125 wikitext text/x-wiki 05jur77ti7u1h07if234m9i1dr0fhv0 2415172 2415125 2026-05-18T23:39:13Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.roll_reboot_osds (exit_code=99) (T426563) 2415172 wikitext text/x-wiki m3wfnk7dn0trtvok77o0cj0sww0zmda 2415178 2415172 2026-05-19T00:11:44Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.roll_reboot_osds (T426563) 2415178 wikitext text/x-wiki 8lyfns9vsnyeths5iz7495vc6s3wz0p 2415179 2415178 2026-05-19T00:11:48Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.roll_reboot_osds (exit_code=99) (T426563) 2415179 wikitext text/x-wiki t6ixqyujts1suphbrsb71z5rd44if56 2415180 2415179 2026-05-19T00:11:56Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.roll_reboot_osds (T426563) 2415180 wikitext text/x-wiki atva6qgt6fzxxowodyufu3tj0yh1md2 2415206 2415180 2026-05-19T02:28:34Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.reboot_node 2415206 wikitext text/x-wiki 2scmbscq6317k6rtlxai8ms7332zwem 2415207 2415206 2026-05-19T02:28:38Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.reboot_node (exit_code=99) 2415207 wikitext text/x-wiki tcvft6bgawykrtx6fu094n82r5ijopp 2415208 2415207 2026-05-19T02:29:00Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.roll_reboot_osds (exit_code=0) (T426563) 2415208 wikitext text/x-wiki qvstlzpsl7asp7yowd4lgy24w7wtsuy 2415422 2415208 2026-05-19T09:37:27Z Stashbot 7414 volans@cloudcumin1001: START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1040.eqiad.wmnet}' 2415422 wikitext text/x-wiki 5ofagdj7c4orar4e9udm4imfcgm0o0n 2415434 2415422 2026-05-19T09:55:18Z Stashbot 7414 volans@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1040.eqiad.wmnet}' 2415434 wikitext text/x-wiki cgbk5kt1iexpmqb2m5u07z9cd5twpgz 2415441 2415434 2026-05-19T10:06:56Z Stashbot 7414 volans@cloudcumin1001: START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1041.eqiad.wmnet}' 2415441 wikitext text/x-wiki psunmp1ncmge1qypw9df9v31miru94f 2415452 2415441 2026-05-19T10:21:30Z Stashbot 7414 volans@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1041.eqiad.wmnet}' 2415452 wikitext text/x-wiki aauxx73trsf06g12c0kwbq0gy50zap5 2415454 2415452 2026-05-19T10:22:42Z Stashbot 7414 volans@cloudcumin1001: START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1042.eqiad.wmnet}' 2415454 wikitext text/x-wiki g0uqt5tzp8tl0ui3dork31t3y9xqcvb 2415491 2415454 2026-05-19T10:43:07Z Stashbot 7414 volans@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1042.eqiad.wmnet}' 2415491 wikitext text/x-wiki t1tiuzgctv56xnyacga3f9mwr91g73i 2415516 2415491 2026-05-19T10:57:21Z Stashbot 7414 volans@cloudcumin1001: START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1043.eqiad.wmnet}' 2415516 wikitext text/x-wiki 1vtwodwsj3djen3i91rh7znp4oyy667 2415544 2415516 2026-05-19T11:13:16Z Stashbot 7414 volans@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1043.eqiad.wmnet}' 2415544 wikitext text/x-wiki l87ro9ipogdp0k80tfewbmmloxbck6q 2415604 2415544 2026-05-19T11:48:55Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.cloudweb.safe_reboot on hosts matched by 'P{O:wmcs::openstack::codfw1dev::cloudweb}' 2415604 wikitext text/x-wiki 96kiq86vl9xechpaekaftpetredqbqy 2415609 2415604 2026-05-19T11:51:50Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.cloudweb.safe_reboot (exit_code=0) on hosts matched by 'P{O:wmcs::openstack::codfw1dev::cloudweb}' 2415609 wikitext text/x-wiki 2392oduk2xew0yr53v73ztnxqhw6o5q 2415610 2415609 2026-05-19T11:52:46Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.cloudweb.safe_reboot on hosts matched by 'P{O:wmcs::openstack::eqiad1::cloudweb}' 2415610 wikitext text/x-wiki 7owxqu0oiac1wysyk3ifwpxlervtk95 2415619 2415610 2026-05-19T11:57:17Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.cloudweb.safe_reboot (exit_code=0) on hosts matched by 'P{O:wmcs::openstack::eqiad1::cloudweb}' 2415619 wikitext text/x-wiki osz7dbycgrt6fyuns7oulkrvurd3edv 0 32278 2414610 2414565 2026-05-18T12:02:14Z LSobanski (WMF) 18783 /* Incident Response */ 2414610 wikitext text/x-wiki 2yc3duj0xoysmtuhy4iv56qy0couj9t 2414616 2414610 2026-05-18T12:02:49Z LSobanski (WMF) 18783 /* Incident Response */ 2414616 wikitext text/x-wiki oytpw9kndbmbg05zmqj1kqntdzf4d8i 0 441820 2414629 2389760 2026-05-18T12:15:47Z GModena (WMF) 20194 /* Further required preparation */ 2414629 wikitext text/x-wiki mwygzks9n1is1331vgyq1bzli59mek9 2414643 2414629 2026-05-18T12:23:30Z GModena (WMF) 20194 /* Further required preparation */ 2414643 wikitext text/x-wiki ad5shh6qf3hmmjhk948fp3yhr9p10ul 2414738 2414643 2026-05-18T13:54:12Z BKing (WMF) 29017 /* The actual code deploy */ 2414738 wikitext text/x-wiki d918unfzsyeesn89wd1cek10f13i361 498 443628 2415181 2414341 2026-05-19T00:17:55Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/26068076424 (https://github.com/cluebotng/component-configs/commits/f7db7f6fff0d4d6dd451b5f92e75ba755a74129c) 2415181 wikitext text/x-wiki o3m20yb6qio1ce83wrf2cgaf78h51f8 0 444533 2415154 2390799 2026-05-18T22:10:32Z Staraction 50753 redundant 2415154 wikitext text/x-wiki 2yq6k2viq3lztejt5ievbduhuep44k1 2415173 2415154 2026-05-18T23:39:57Z JJMC89 7474 unneacessary 2415173 wikitext text/x-wiki bpo1mprs1rc48a17opbl0bs4m6mi6fb 0 446140 2415413 2404048 2026-05-19T09:30:53Z Tiziano Fogli 37623 /* Alerts */ 2415413 wikitext text/x-wiki ktdch3esne0ofqshaj1tqxzu5famhhi 2415415 2415413 2026-05-19T09:31:20Z Tiziano Fogli 37623 /* Service thanos-query:443 has failed probes */ 2415415 wikitext text/x-wiki 4xgdh3oarrsjamqbaud1tgpifa4g82m 2415419 2415415 2026-05-19T09:32:40Z Tiziano Fogli 37623 /* Service thanos-query:443 has failed probes */ 2415419 wikitext text/x-wiki cbdvuszgaclfcw0u7dy51l9yojorgu6 0 446795 2415425 2414433 2026-05-19T09:39:26Z CWilliams-WMF 55316 Removed wiki_willy from final steps 2415425 wikitext text/x-wiki 46dae191n9ofk3i0byetj85mxdj59h9 0 447862 2415366 2412738 2026-05-19T08:51:03Z FCeratto-WMF 39104 Put order on top 2415366 wikitext text/x-wiki lk99f8mrpewr84cp9v9spzvvd6h2qm3 2415369 2415366 2026-05-19T08:54:02Z FCeratto-WMF 39104 Undo moving order on top 2415369 wikitext text/x-wiki 0jtl63hn7bucmfjicutunqovjuuqyr9 2415371 2415369 2026-05-19T08:57:11Z MArostegui (WMF) 8620 /* Upgrading MariaDB major version on a single host */ Added a reimage + upgrade 2415371 wikitext text/x-wiki ahip8bq61ivw9qmza2666omn6rk7pg7 0 448500 2415147 2359681 2026-05-18T21:54:14Z Star walker 40724 Tidied up article 2415147 wikitext text/x-wiki n0eo7diltm0cq48v3ejfqi7njdth7q4 2415150 2415147 2026-05-18T21:55:46Z Star walker 40724 oops 2415150 wikitext text/x-wiki mxml8zq5cmlb9z4m065dpuzqgeysn98 0 449160 2415177 2414589 2026-05-19T00:02:11Z Dexbot 30554 Bot: Updating the report 2415177 wikitext text/x-wiki 7mn69p7q8mms9bynarmasnqe44031dr 2415220 2415177 2026-05-19T06:11:55Z Dexbot 30554 Bot: Updating the report 2415220 wikitext text/x-wiki 0re5g7i4bmrrltb48fesxjedrx4owqx 2415245 2415220 2026-05-19T06:48:01Z Dexbot 30554 Bot: Updating the report 2415245 wikitext text/x-wiki fgyggho3a75g1iioz05o6amrtmx46rx 0 450435 2415114 2397160 2026-05-18T20:16:19Z RLazarus (WMF) 15215 /* Monitor the Service */ update the template to a new link after it was edited in-place 2415114 wikitext text/x-wiki 9ty8zjeof46s5fhhhwret4rs93rl0ny 498 452890 2415182 2414342 2026-05-19T00:18:33Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/26068076396 (https://github.com/cluebotng/component-configs/commits/f7db7f6fff0d4d6dd451b5f92e75ba755a74129c) 2415182 wikitext text/x-wiki caqlhdoc7alzwy69fxx4rhdhgih15ul 0 453073 2415142 2399628 2026-05-18T21:43:15Z Star walker 40724 Tidied up portal 2415142 wikitext text/x-wiki 3w3cy223wpat86wa27omtyynqsiri55 0 453463 2415324 2404856 2026-05-19T08:35:03Z Effie Mouzeli (WMF) 12880 /* Service restart */ 2415324 wikitext text/x-wiki ovogvz38eidnq5i5ohq8e6k0oifs5c6 116 453906 2414951 2414072 2026-05-18T16:36:29Z Gitlabaccountapprovalbot 37332 @effeietsanders was approved. 2414951 wikitext text/x-wiki 8r0lv7xmym8x1lj52ssmhuttptv7bbl 2415167 2414951 2026-05-18T23:18:18Z Gitlabaccountapprovalbot 37332 wladek92 was rejected. 2415167 wikitext text/x-wiki 2xdnb08xek8pmozrbrx2zhtbwm0v5il 498 459243 2415166 2414340 2026-05-18T23:18:15Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/26065893095 (https://github.com/cluebotng/component-configs/commits/0bf6e1847388fd90f2c03acdc1b80d388cb437e7) 2415166 wikitext text/x-wiki 9ahw6y5knu04ezmy33jwcewb6jmy0i7 2415170 2415166 2026-05-18T23:34:02Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/26066535621 (https://github.com/cluebotng/component-configs/commits/2b0b0bbd4e0a727a468343823d4343fe7a082ef9) 2415170 wikitext text/x-wiki 3b1jg9a11cxxux8emhxkylndp5ruy58 2415194 2415170 2026-05-19T01:33:53Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/26070598819 (https://github.com/cluebotng/component-configs/commits/c5378a5646858d02f07ababdb2db2955000ab187) 2415194 wikitext text/x-wiki o2zrvm9c1tf5gskao5dlzztik15z6kz 2415205 2415194 2026-05-19T02:08:49Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/26071765151 (https://github.com/cluebotng/component-configs/commits/5184d69b34b8d2442df1fbbd652535f5890b4804) 2415205 wikitext text/x-wiki 1fofm29zymep6xsklnpc1kt243e31is 2415209 2415205 2026-05-19T02:37:28Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/26072649347 (https://github.com/cluebotng/component-configs/commits/94b7da488f5c23df52fa2e750438a3a80f589f3c) 2415209 wikitext text/x-wiki 67odmdw85mh1v3v8dfwbd9bpzwk5cu8 498 459244 2415195 2414339 2026-05-19T01:47:39Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/26071084518 (https://github.com/cluebotng/component-configs/commits/6831fe17857c84db1afb1023198b7b11d444f69f) 2415195 wikitext text/x-wiki c2mdj5w6tsf5i5q6cgg3fbb1skz2grs 0 459290 2414716 2414280 2026-05-18T13:39:43Z TChin (WMF) 25137 Monday, May 18, 2026 Train 2414716 wikitext text/x-wiki raeouw6rdpnpfvho1ztct4231c6g362 0 459699 2414776 2414569 2026-05-18T14:26:24Z Stashbot 7414 ejegg: payments-wiki upgraded from 40a24102 to e31ac2a9 2414776 wikitext text/x-wiki fjp2w7aen22qvqub0go1n5i7p6qki9l 2415126 2414776 2026-05-18T20:45:45Z Stashbot 7414 eileen: civicrm upgraded from 150d484e to bfa13f7f 2415126 wikitext text/x-wiki 9s1mctykohl6x1brkvt7tmc30nrv792 2415196 2415126 2026-05-19T01:56:29Z Stashbot 7414 eileen: civicrm upgraded from bfa13f7f to 6cff86c1 2415196 wikitext text/x-wiki 80smvzx2gtgvlrow6p3iwoo6kdj7942 0 459719 2414860 2407614 2026-05-18T15:32:15Z BTullis (WMF) 25295 Added a note about excluding the freshness SLO, for now. 2414860 wikitext text/x-wiki tc2pttt0ggrzik2hh3ic5bvcl3zv0dh 828 460036 2414678 2400480 2026-05-18T13:12:06Z Toolforge Image Bot 35765 Bot: Updating available container images 2414678 json application/json 5i6kp68nh48fvzay7a3hrx9y7i30dnd 116 460070 2414909 2412211 2026-05-18T16:04:15Z EtherpadBackupBot 54504 cuny-wikiwed-2025-04 archived by Pharos 2414909 wikitext text/x-wiki jooycezqyfhwp0w4y7iexp82fx5q3n4 2414911 2414909 2026-05-18T16:04:33Z EtherpadBackupBot 54504 cuny-wikiwed-2025-05 archived by Pharos 2414911 wikitext text/x-wiki p3w1hbmfutqowsamuomc53bmqgj07lh 2414912 2414911 2026-05-18T16:04:50Z EtherpadBackupBot 54504 cuny-wikiwed-2025-06 archived by Pharos 2414912 wikitext text/x-wiki gjikxsl4m5milb3c93oatcla5fxbup2 2414913 2414912 2026-05-18T16:05:01Z EtherpadBackupBot 54504 cuny-wikiwed-2025-07 archived by Pharos 2414913 wikitext text/x-wiki 6wulx7fsuomo4tnvzsjfi3c7tyyir1g 2414914 2414913 2026-05-18T16:05:17Z EtherpadBackupBot 54504 cuny-wikiwed-2025-08 archived by Pharos 2414914 wikitext text/x-wiki 4nyquanmxw34tn3v8yob2z7iqmdnkzj 2414915 2414914 2026-05-18T16:05:34Z EtherpadBackupBot 54504 cuny-wikiwed-2025-09 archived by Pharos 2414915 wikitext text/x-wiki t1pnacgtub8k0pzbyhpsmig1f4izo1f 2414917 2414915 2026-05-18T16:05:54Z EtherpadBackupBot 54504 cuny-wikiwed-2025-10 archived by Pharos 2414917 wikitext text/x-wiki 8j2jo7b0lgoqk6oarjk8t4gdudf3boo 2414919 2414917 2026-05-18T16:06:07Z EtherpadBackupBot 54504 cuny-wikiwed-2025-11 archived by Pharos 2414919 wikitext text/x-wiki 88ez5jxe1xoo51vffuw9vejc73wbbpg 2414920 2414919 2026-05-18T16:06:21Z EtherpadBackupBot 54504 cuny-wikiwed-2026-01 archived by Pharos 2414920 wikitext text/x-wiki r2r5tswfgkcy1qp9srx80aw0cpvs0e3 2414921 2414920 2026-05-18T16:06:39Z EtherpadBackupBot 54504 cuny-wikiwed-2026-03 archived by Pharos 2414921 wikitext text/x-wiki 36x2d5yum0t2azf8icoqyztpussxqaa 2414922 2414921 2026-05-18T16:06:54Z EtherpadBackupBot 54504 cuny-wikiwed-2026-04 archived by Pharos 2414922 wikitext text/x-wiki q9yo0ns08w23wm210o1ludkty4b9pfp 2414958 2414922 2026-05-18T16:43:16Z EtherpadBackupBot 54504 multimedia-weekly-meeting-2014-05-14 archived by Sj 2414958 wikitext text/x-wiki f801vaodc7kf8yfa704hxo840p4jldp 2414959 2414958 2026-05-18T16:49:07Z EtherpadBackupBot 54504 editcontentmodel archived by Sj 2414959 wikitext text/x-wiki rptud84rtf9sfl8ijno60hs1vvg03s4 2414963 2414959 2026-05-18T16:54:00Z EtherpadBackupBot 54504 GLAM-Wikipedia-fr archived by Sj 2414963 wikitext text/x-wiki s0qh8gjsz9vl9ph2ac1fpev1qh6yjq2 2414964 2414963 2026-05-18T16:54:39Z EtherpadBackupBot 54504 Templates,_Page_Components_and_editing archived by Sj 2414964 wikitext text/x-wiki aff99eyegj5ryp6an9b8628x98a5886 2414974 2414964 2026-05-18T17:11:03Z EtherpadBackupBot 54504 ceem14-pr archived by Sj 2414974 wikitext text/x-wiki f9trr0wjbphbsgwsklc0us8o37yx3fq 2414979 2414974 2026-05-18T17:12:33Z EtherpadBackupBot 54504 MWF20160226 archived by Sj 2414979 wikitext text/x-wiki g3k3jzp7zs6hdks2zl0rx6l1wypb28z 2414980 2414979 2026-05-18T17:12:43Z EtherpadBackupBot 54504 MWF20160318 archived by Sj 2414980 wikitext text/x-wiki 73ppi3cp9ny0jwxmza2dxrvy0itvit6 2414987 2414980 2026-05-18T17:14:31Z EtherpadBackupBot 54504 whatsnext archived by Sj 2414987 wikitext text/x-wiki 24vw1k4ynmluiyhm6czq52mgjli3j5l 2414990 2414987 2026-05-18T17:16:35Z EtherpadBackupBot 54504 community_health archived by Sj 2414990 wikitext text/x-wiki 4slui0bp0zuop6bhmgyoil4fw8xfmkd 2414993 2414990 2026-05-18T17:17:01Z EtherpadBackupBot 54504 medical_hotspots archived by Sj 2414993 wikitext text/x-wiki q3kr9gwshyubsmm22l6ewbc8vip3qax 2414995 2414993 2026-05-18T17:19:56Z EtherpadBackupBot 54504 15.320 archived by Sj 2414995 wikitext text/x-wiki 028s3hbh7u655oczjcj7cnjnsl0ex5w 2415007 2414995 2026-05-18T17:33:27Z EtherpadBackupBot 54504 JROSTS18-WFC.1 archived by Sj 2415007 wikitext text/x-wiki fde4hxxuizjambq4ynawolmuu2p9zay 2415008 2415007 2026-05-18T17:33:32Z EtherpadBackupBot 54504 JROST-Master archived by Sj 2415008 wikitext text/x-wiki lcgbaf50upw1l2nzk7f8x6wp894vern 2415009 2415008 2026-05-18T17:33:47Z EtherpadBackupBot 54504 JROSTS18-CROSSCUTTING archived by Sj 2415009 wikitext text/x-wiki puf14q4zy3ezyhdp68d1smm7xyodpp8 2415010 2415009 2026-05-18T17:34:20Z EtherpadBackupBot 54504 JROSTS18-WFA.1 archived by Sj 2415010 wikitext text/x-wiki 9g9p0uotwxd6cqvylhlatsd36lnllum 2415011 2415010 2026-05-18T17:34:28Z EtherpadBackupBot 54504 JROSTS18-WFA.2 archived by Sj 2415011 wikitext text/x-wiki 3qbycu24as925c9ad2w8i1o4opalwvi 2415012 2415011 2026-05-18T17:35:01Z EtherpadBackupBot 54504 JROSTS18-WFC.2 archived by Sj 2415012 wikitext text/x-wiki q5g5fiq6rs5b6gr25zmxkip9ry2f3fj 2415013 2415012 2026-05-18T17:35:09Z EtherpadBackupBot 54504 JROSTS18-WFC.3 archived by Sj 2415013 wikitext text/x-wiki j1gvtj0v205xro8whucf4sya0sm3yqp 2415014 2415013 2026-05-18T17:35:15Z EtherpadBackupBot 54504 JROSTS18-EOA archived by Sj 2415014 wikitext text/x-wiki dcmhihu7wao8115fte71glfvqg3tbsj 2415015 2415014 2026-05-18T17:35:52Z EtherpadBackupBot 54504 JROSTS18-OB.1 archived by Sj 2415015 wikitext text/x-wiki h3oedq2diqp83mlipfmqwvsmr3jnyd6 2415016 2415015 2026-05-18T17:36:01Z EtherpadBackupBot 54504 JROSTS18-JROST archived by Sj 2415016 wikitext text/x-wiki 4n5294aeovi5sspv0kbp55rd7vtig5w 2415017 2415016 2026-05-18T17:36:06Z EtherpadBackupBot 54504 JROSTS18-UNCONFERENCING archived by Sj 2415017 wikitext text/x-wiki gbu8c7zgd48v5q7fyfrcsrdqklizmk9 2415019 2415017 2026-05-18T17:36:17Z EtherpadBackupBot 54504 JROSTS18-OA.2 archived by Sj 2415019 wikitext text/x-wiki ictgk23or0jrsmbauftyzj1kta72ozp 2415020 2415019 2026-05-18T17:36:23Z EtherpadBackupBot 54504 JROSTS18-UNCFB archived by Sj 2415020 wikitext text/x-wiki 1i3gmj2uk7tgdrq6m8oi2kaapu5fbox 2415021 2415020 2026-05-18T17:36:29Z EtherpadBackupBot 54504 JROSTS18-UNCFC archived by Sj 2415021 wikitext text/x-wiki r1mxs77bqgeiu47rezqmvs5usxvsze3 2415022 2415021 2026-05-18T17:36:36Z EtherpadBackupBot 54504 JROSTS18-UNCFD archived by Sj 2415022 wikitext text/x-wiki 9u967e5jjb75fxe7owqnfw68qoq8ky9 2415023 2415022 2026-05-18T17:36:43Z EtherpadBackupBot 54504 JROSTS18-UNCFE archived by Sj 2415023 wikitext text/x-wiki bsvkz7l5arm2jmdtsdbaeou4tneb9jz 2415024 2415023 2026-05-18T17:36:49Z EtherpadBackupBot 54504 JROSTS18-UNCFF archived by Sj 2415024 wikitext text/x-wiki tsfi5ik9s18c02qavhtpmerbhtu55tv 2415025 2415024 2026-05-18T17:37:03Z EtherpadBackupBot 54504 JROSTS18-HACKDAY archived by Sj 2415025 wikitext text/x-wiki c6o105ild68agsxmpj1m94lk0qch816 2415083 2415025 2026-05-18T18:54:34Z EtherpadBackupBot 54504 Covering_George_Floyd archived by Sj 2415083 wikitext text/x-wiki pz0sjkrldqjbtejheansg3vura2ovzz 2415084 2415083 2026-05-18T18:55:44Z EtherpadBackupBot 54504 covid19 archived by Sj 2415084 wikitext text/x-wiki lwnxa5whv7efoaf2x4klgq3tdvld8gw 2415085 2415084 2026-05-18T18:59:15Z EtherpadBackupBot 54504 Wikimania_oral_history archived by Sj 2415085 wikitext text/x-wiki 3fuwu3fv8woahjtli9l7q72n9kx5cor 2415086 2415085 2026-05-18T19:01:01Z EtherpadBackupBot 54504 ctp-letter archived by Sj 2415086 wikitext text/x-wiki 5ticnldonqixk53ht7dbuj857588x0b 2415087 2415086 2026-05-18T19:02:16Z EtherpadBackupBot 54504 shexcg_minutes archived by Sj 2415087 wikitext text/x-wiki oqriw7z44fm1b7b8zkxb4az5rcn3x1u 2415090 2415087 2026-05-18T19:02:52Z EtherpadBackupBot 54504 WLM-2021-April archived by Sj 2415090 wikitext text/x-wiki d204yflliurpwght4ew9737qv4tt95h 2415092 2415090 2026-05-18T19:03:15Z EtherpadBackupBot 54504 WLM-2021-May archived by Sj 2415092 wikitext text/x-wiki guctlgqk0096j1oi3j858hv2i5ndk00 2415093 2415092 2026-05-18T19:07:39Z EtherpadBackupBot 54504 WM-structural-problems-overview archived by Sj 2415093 wikitext text/x-wiki 9c6kqt3z0pac5rezeqlcuf8nrly8yjm 2415094 2415093 2026-05-18T19:08:49Z EtherpadBackupBot 54504 city-homelessness-okn archived by Sj 2415094 wikitext text/x-wiki citmi5qntq62wqqze0r9e0ozmjszefe 2415096 2415094 2026-05-18T19:20:30Z EtherpadBackupBot 54504 Chartres archived by Sj 2415096 wikitext text/x-wiki obig63j197ps2ok6ae03zebr4r5hmmx 2 460100 2415535 2414392 2026-05-19T11:09:50Z Effie Mouzeli (WMF) 12880 2415535 wikitext text/x-wiki 225d84n72mc6dq1chj7sasxdeae37wm 2 460101 2415568 2414214 2026-05-19T11:26:16Z Effie Mouzeli (WMF) 12880 2415568 sanitized-css text/css 7c1aqs6x7lvs1k61bn0fyuilnjjidr4 3 460186 2414635 2026-05-18T12:20:53Z StrikerBot 8475 Welcome to Toolforge! 2414635 wikitext text/x-wiki ph2dzny6sr8k1dw3k6u6rfcudhodr0a 3 460187 2414637 2026-05-18T12:21:32Z StrikerBot 8475 Welcome to Toolforge! 2414637 wikitext text/x-wiki mjtp9tja546ayrqg8e3cufugk3k2ste 3 460188 2414639 2026-05-18T12:22:06Z StrikerBot 8475 Welcome to Toolforge! 2414639 wikitext text/x-wiki pi6mxc87r8bqgcy21kaelp6apnl5011 2 460189 2415567 2026-05-19T11:25:23Z Effie Mouzeli (WMF) 12880 Created page with "<includeonly>{{#switch: {{lc:{{{1|}}}}} | done = 🟢 | not done | not-done | notdone = 🔴 | n/a | na = '''N/A''' | #default = <span class="scroll-placeholder">—</span> }}</includeonly><noinclude> == Description == Renders a status marker for the Status column of a SCROLL checklist row: a green dot for done, a red dot for not done, or a bold "N/A". Accepted values (case-insensitive): <code>done</code>, <code>not done</code> (or <code>not-done</code>), <code>n/a</co..." 2415567 wikitext text/x-wiki 21g5qvobsuh7mvj25ynetpoulbm7k0c