Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.24 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 0 4108 2467823 2467622 2026-10-14T12:26:25Z ScheduleDeploymentBot 37566 Add [[gerrit:1355039]] to Wednesday, October 14 UTC afternoon backport window 2467823 wikitext text/x-wiki kb3yxk4s0181rq7tc3p5nuybylugee4 2468071 2467823 2026-10-14T15:37:31Z TCipriani (WMF) 4895 /* {{Deployment_day|date=2026-10-13}} */ 2468071 wikitext text/x-wiki rewzvwfuzw4woqbtbw4l41uafqnw7ub 2468072 2468071 2026-10-14T15:38:09Z TCipriani (WMF) 4895 /* {{Deployment_day|date=2026-10-14}} */ 2468072 wikitext text/x-wiki oa082fsy2vahm8ex51f9qy7em7oszpw 2468148 2468072 2026-10-14T16:27:32Z Brennen 54136 /* {{Deployment_day|date=2026-10-14}} */ Add self to UTC-7 train window. 2468148 wikitext text/x-wiki a3wrtw5nvtahj5iqg86nn30k0kdb7g0 2468242 2468148 2026-10-14T18:09:27Z ScheduleDeploymentBot 37566 Add [[gerrit:1352018]] to Wednesday, October 14 UTC late backport window 2468242 wikitext text/x-wiki jcd7dor6zms960uh9j5ep4ezj9x3lyp 2468243 2468242 2026-10-14T18:09:43Z ScheduleDeploymentBot 37566 Add [[gerrit:1352019]] to Wednesday, October 14 UTC late backport window 2468243 wikitext text/x-wiki rfrz6rbzu5pc788g5vym0pntofk6x1k 2468246 2468243 2026-10-14T18:12:16Z DHardy-WMF 39108 Fixed schedule for 1352018 and 1352019 2468246 wikitext text/x-wiki 8xtumjfkbwn7yv6duw3dmyb5wg6dcfi 2468259 2468246 2026-10-14T18:54:09Z ScheduleDeploymentBot 37566 Add [[gerrit:1355147]] to Wednesday, October 14 UTC late backport window 2468259 wikitext text/x-wiki 3dijw4ijolierdu91sm8q5w3ex3ktct 2468280 2468259 2026-10-14T19:30:01Z ScheduleDeploymentBot 37566 Add [[gerrit:1355168]] to Wednesday, October 14 UTC late backport window 2468280 wikitext text/x-wiki gsjdi8ky70teo2d629zem5lzmw3p3gb 2468281 2468280 2026-10-14T19:34:49Z ScheduleDeploymentBot 37566 Add [[gerrit:1355169]] to Wednesday, October 14 UTC late backport window 2468281 wikitext text/x-wiki lfzhgtz45k6qe8udxhzzyqxc7tarh0b 2468284 2468281 2026-10-14T19:37:18Z ScheduleDeploymentBot 37566 Add [[gerrit:1355170]] to Thursday, October 15 UTC morning backport window 2468284 wikitext text/x-wiki ax3knshlv71w41g2cxuo8i0ibnzpdiw 2468302 2468284 2026-10-14T20:26:10Z Christoph Jauera (WMDE) 3635 /* {{Deployment_day|date=2026-10-14}} */ Adding 1355170 2468302 wikitext text/x-wiki stszqsyp3n1x4nh5itin8d3l24uk7ql 2468316 2468302 2026-10-14T20:45:07Z Christoph Jauera (WMDE) 3635 Not needed anymore I could backport 1355170 on Wednesday 14th 2468316 wikitext text/x-wiki 3mdgchezy9nz7hin5q2jd0xkfeos5tt 2468349 2468316 2026-10-14T21:29:28Z ScheduleDeploymentBot 37566 Add [[gerrit:1355188]] to Thursday, October 15 UTC late backport window 2468349 wikitext text/x-wiki r0ty2yhpwo85w1orkfbuvou78mg9nyx 2468356 2468349 2026-10-14T22:03:30Z ScheduleDeploymentBot 37566 Add [[gerrit:1354488]] to Thursday, October 15 UTC afternoon backport window 2468356 wikitext text/x-wiki cuzs7jsqueb9cvt8ojnn9w5ih0szqxd 2468364 2468356 2026-10-14T22:52:43Z ScheduleDeploymentBot 37566 Add [[gerrit:1355198]] to Thursday, October 15 UTC afternoon backport window 2468364 wikitext text/x-wiki 73exc3r59fijqfc9rbh0uut7esuycjc 2468365 2468364 2026-10-14T22:53:03Z ScheduleDeploymentBot 37566 Add [[gerrit:1355197]] to Thursday, October 15 UTC afternoon backport window 2468365 wikitext text/x-wiki 1ttsytvcge57kwty87ykqgga4yeb3rt 2468533 2468365 2026-10-15T06:59:18Z Tgr (WMF) 39134 /* {{Deployment_day|date=2026-10-15}} */ 2468533 wikitext text/x-wiki mbfyrcgbnk60oj8eb76d0m67gb3zxnc 2468534 2468533 2026-10-15T07:00:30Z Tgr (WMF) 39134 /* {{Deployment_day|date=2026-10-15}} */ 2468534 wikitext text/x-wiki m84eq7iu0bfkwer9picya0s6xlxc9b2 498 6726 2468689 2466284 2026-10-15T08:50:31Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component replica-cnf 2468689 wikitext text/x-wiki s1q0vm6ruvv1lwbzpu7b2itm3eioi1s 2468691 2468689 2026-10-15T08:50:49Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component replica-cnf 2468691 wikitext text/x-wiki ndh8d5qievg18pmofo096fs1kgnjuik 2468726 2468691 2026-10-15T09:42:04Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component replica-cnf 2468726 wikitext text/x-wiki qdzdvvw2klrguoq2mpjbn5q6hngtzjk 2468727 2468726 2026-10-15T09:42:20Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component replica-cnf 2468727 wikitext text/x-wiki 3r9r8emgj9gzjnqthaisfn5peljlekz 2468728 2468727 2026-10-15T09:42:34Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component replica-cnf 2468728 wikitext text/x-wiki bcyqkh7nn7qv86fenv4gncrgxbitrp9 2468729 2468728 2026-10-15T09:42:49Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component replica-cnf 2468729 wikitext text/x-wiki kkbldlsczi7iks7umtb0b4k60mo3i2u 2468731 2468729 2026-10-15T09:48:43Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component replica-cnf 2468731 wikitext text/x-wiki rgpt781yk370ebtc7xesvwbc5p64obb 2468732 2468731 2026-10-15T09:49:04Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component replica-cnf 2468732 wikitext text/x-wiki nfcbw4scu4cp4nndt2g6kgxsdc1x9wv 0 7195 2468192 2458623 2026-10-14T17:04:22Z BKing (WMF) 29017 2468192 wikitext text/x-wiki agl7nc6yw8a3qw2anhm1cnrrhqwguju 0 7919 2467810 2467809 2026-10-14T12:01:05Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1297.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467810 wikitext text/x-wiki bvt2qapee9pjo0egaa0mk045hk5roii 2467811 2467810 2026-10-14T12:01:24Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1297.eqiad.wmnet 2467811 wikitext text/x-wiki jxuygls3kxq5y9v53ji74agax7r06xx 2467812 2467811 2026-10-14T12:03:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1297.eqiad.wmnet 2467812 wikitext text/x-wiki rzfh08cbw5f2k11pbz5486fyr5gn5ny 2467813 2467812 2026-10-14T12:04:07Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1298.eqiad.wmnet 2467813 wikitext text/x-wiki ot0rgkl33psa45h04ff30fyztltbrjz 2467814 2467813 2026-10-14T12:04:38Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1298.eqiad.wmnet 2467814 wikitext text/x-wiki 7hamw0rxqdx31j7v2elx859v22ovzst 2467815 2467814 2026-10-14T12:04:58Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1298.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467815 wikitext text/x-wiki g8d4x5vyvhnpr1jku2m2rcphlzi5s66 2467816 2467815 2026-10-14T12:14:16Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.memcached.roll-reboot-restart rolling restart_daemons on A:memcached-codfw 2467816 wikitext text/x-wiki mbbkh2yf86x18x7q25vg3jdcowjrhyk 2467817 2467816 2026-10-14T12:19:05Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2467817 wikitext text/x-wiki 7w3fs27ho7ic7epkakw9ojzty3evz5g 2467818 2467817 2026-10-14T12:20:18Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2467818 wikitext text/x-wiki 6g99ku6aaucslpx0q3b87oxey8ugxlv 2467820 2467818 2026-10-14T12:23:56Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2256.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467820 wikitext text/x-wiki 5gxp7ns0pml6f968dfi9fj1caqlhyh7 2467821 2467820 2026-10-14T12:24:45Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2467821 wikitext text/x-wiki ggzx7r9xonwqohf6xs7lsiaobbbzaeh 2467822 2467821 2026-10-14T12:25:40Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2467822 wikitext text/x-wiki dlpzqi3104qr1oi7uwhi9btbiyi3vja 2467824 2467822 2026-10-14T12:28:07Z Stashbot 7414 kamila@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-constraints: apply 2467824 wikitext text/x-wiki 8r3wnbffpo8q4amcqjbisrdc02joddg 2467825 2467824 2026-10-14T12:28:21Z Stashbot 7414 kamila@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-constraints: apply 2467825 wikitext text/x-wiki o22xi76kjabimjvfaurru2rdcblw0at 2467826 2467825 2026-10-14T12:29:12Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2256.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467826 wikitext text/x-wiki em4x20r3gqj5fo4rk85tpei8bmy8k8c 2467827 2467826 2026-10-14T12:32:19Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-constraints: apply 2467827 wikitext text/x-wiki 14en6l3fswf4kfepza13ydal1fysg0e 2467828 2467827 2026-10-14T12:32:36Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-constraints: apply 2467828 wikitext text/x-wiki qxbzrxkgjb2j71i5xw6k7r3hs7gknqw 2467829 2467828 2026-10-14T12:32:42Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-constraints: apply 2467829 wikitext text/x-wiki o1onuos0kc13cbaffd6zp4fuzzf3k00 2467830 2467829 2026-10-14T12:32:48Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2256.codfw.wmnet 2467830 wikitext text/x-wiki nh39cr59x1zfg63j7u3ld5umke6bpmn 2467831 2467830 2026-10-14T12:32:49Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2256.codfw.wmnet 2467831 wikitext text/x-wiki ruxsk7o4j2fa0iwkcz6erla23iqiqhe 2467832 2467831 2026-10-14T12:32:56Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-constraints: apply 2467832 wikitext text/x-wiki 4rttjt55jyos5kzxdmntr1aui2bs470 2467833 2467832 2026-10-14T12:33:03Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2257.codfw.wmnet 2467833 wikitext text/x-wiki 7hdubdjaec8fhe0tspcymoozxz2jkuq 2467834 2467833 2026-10-14T12:33:36Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2257.codfw.wmnet 2467834 wikitext text/x-wiki 93dgevt60q3fujxinhuv5mdswh7vg25 2467835 2467834 2026-10-14T12:35:19Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1298.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467835 wikitext text/x-wiki ah68kivvcol23bnymbw4gr8ep8znclq 2467836 2467835 2026-10-14T12:36:59Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.enable_virtualization (exit_code=1) rolling provision on D{wikikube-worker[2088,2203,2242-2243,2248-2260].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2467836 wikitext text/x-wiki 9hb4flxtl9w5ia27nttb51899x971r0 2467837 2467836 2026-10-14T12:39:03Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2257.codfw.wmnet 2467837 wikitext text/x-wiki 2g7iu1nbif4rjzc08o0sedc6dvdviym 2467838 2467837 2026-10-14T12:39:05Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2257.codfw.wmnet 2467838 wikitext text/x-wiki s3smzd8u0spamti1bis1i5f4ya39f0v 2467839 2467838 2026-10-14T12:42:30Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.enable_virtualization rolling provision on D{wikikube-worker[2088,2203,2242-2243,2248-2260].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2467839 wikitext text/x-wiki obnl1og93qily7qowkl4rlo96wzd9fp 2467840 2467839 2026-10-14T12:42:34Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2088.codfw.wmnet 2467840 wikitext text/x-wiki kvfvhm7m3hl60fw0m1vyz7w27sm8up4 2467841 2467840 2026-10-14T12:43:08Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2088.codfw.wmnet 2467841 wikitext text/x-wiki 35w3b4ennoetrbxp4uiwew6whga5amc 2467842 2467841 2026-10-14T12:45:32Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2088.codfw.wmnet 2467842 wikitext text/x-wiki 45wp8bqktuokhnvk6xwe75b8dzqi8n4 2467843 2467842 2026-10-14T12:45:33Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2088.codfw.wmnet 2467843 wikitext text/x-wiki kir7k6q969brrpeooaicmuk8tz5sjpu 2467844 2467843 2026-10-14T12:45:47Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2203.codfw.wmnet 2467844 wikitext text/x-wiki os62dczdb24yzwi1gn8e6ponvjhbciq 2467845 2467844 2026-10-14T12:46:20Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2203.codfw.wmnet 2467845 wikitext text/x-wiki jmhm11gjff7walkg06l961nkm09s0rh 2467846 2467845 2026-10-14T12:46:40Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2203.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467846 wikitext text/x-wiki 6l151gz4pvohf7mif9t08hadvtd7uok 2467847 2467846 2026-10-14T12:47:55Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2203.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467847 wikitext text/x-wiki iro2jmvcm7gl35ku4vkmn8dby4k0g75 2467848 2467847 2026-10-14T12:48:01Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2203.codfw.wmnet 2467848 wikitext text/x-wiki 2w5kt51vn9bm3lobpa2ys9qfys299qv 2467849 2467848 2026-10-14T12:48:03Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2203.codfw.wmnet 2467849 wikitext text/x-wiki j3ew3g6ymu6jk5va09xrxsxm0fos2i1 2467850 2467849 2026-10-14T12:48:16Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2257.codfw.wmnet 2467850 wikitext text/x-wiki 35jgg1319034cgi01886f32u0h5a9xc 2467851 2467850 2026-10-14T12:48:49Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2257.codfw.wmnet 2467851 wikitext text/x-wiki 6iuhuth806uqj00o3ve9l25sryekhjb 2467852 2467851 2026-10-14T12:49:00Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2257.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467852 wikitext text/x-wiki 7esrk4kocrgqwdh453d32tjcl3jqthn 2467853 2467852 2026-10-14T12:54:20Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2257.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467853 wikitext text/x-wiki si954cwghdyzl7m692hibxv308ie58q 2467854 2467853 2026-10-14T12:54:38Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2257.codfw.wmnet 2467854 wikitext text/x-wiki 0birnlvm32nni411k60fkni81jtw4ju 2467855 2467854 2026-10-14T12:54:39Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2257.codfw.wmnet 2467855 wikitext text/x-wiki 8haabmu02f2fjtoh1omsaw565jd30h2 2467856 2467855 2026-10-14T12:54:53Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2258.codfw.wmnet 2467856 wikitext text/x-wiki h4p0paw4tzziawcfqhxh2q0up1ew4bw 2467857 2467856 2026-10-14T12:55:27Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2258.codfw.wmnet 2467857 wikitext text/x-wiki knqvkr6iax4s1bjalkm5rju47t48l7p 2467858 2467857 2026-10-14T12:55:48Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2258.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467858 wikitext text/x-wiki fipcpmndzbfszx7oq7esswbldfkcx39 2467859 2467858 2026-10-14T12:56:58Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.enable_virtualization (exit_code=1) rolling provision on D{wikikube-worker[1291-1300].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) 2467859 wikitext text/x-wiki 0iml8tw0wrxzd1k3oncug2x755dk90s 2467860 2467859 2026-10-14T13:01:05Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2258.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467860 wikitext text/x-wiki 6riw1je67ygzx3kk1px4815goeuj23n 2467861 2467860 2026-10-14T13:01:23Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2258.codfw.wmnet 2467861 wikitext text/x-wiki 2g7st05rcctcva3yreb27y9xw7458qo 2467862 2467861 2026-10-14T13:01:24Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2258.codfw.wmnet 2467862 wikitext text/x-wiki slecajrwz9uvuck2znw9qgh58t4grue 2467863 2467862 2026-10-14T13:01:38Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2259.codfw.wmnet 2467863 wikitext text/x-wiki 8x6vmdk5mkhhvz97nfywwus4sg2a4og 2467864 2467863 2026-10-14T13:02:16Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2259.codfw.wmnet 2467864 wikitext text/x-wiki i2m8s9y8odqh4hafjhb2sca9y67cyy3 2467865 2467864 2026-10-14T13:02:37Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2259.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467865 wikitext text/x-wiki s4bb3m8rjylx6arxvkz93fn39dlm6ph 2467866 2467865 2026-10-14T13:02:59Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.enable_virtualization rolling provision on D{wikikube-worker[1299-1300].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) 2467866 wikitext text/x-wiki 7kp6kj6rutk7c5fsvebbon9dph9ln8x 2467867 2467866 2026-10-14T13:03:03Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1299.eqiad.wmnet 2467867 wikitext text/x-wiki b5ysovkmor87foxdtt8gu1viuwsdxnu 2467868 2467867 2026-10-14T13:03:36Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1299.eqiad.wmnet 2467868 wikitext text/x-wiki mw6o3yehp4l95g8p33vhf4cyaaftri4 2467869 2467868 2026-10-14T13:03:56Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1299.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467869 wikitext text/x-wiki cpfky1lal2xm428b9mz4782iazc0kpf 2467870 2467869 2026-10-14T13:07:02Z Stashbot 7414 ozge@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . 2467870 wikitext text/x-wiki k61htw3foxv3xelc8kqcasllwls7hzb 2467871 2467870 2026-10-14T13:07:55Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2259.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467871 wikitext text/x-wiki pyyfv188deeyf3goeyx3155n85f0ael 2467872 2467871 2026-10-14T13:08:13Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2259.codfw.wmnet 2467872 wikitext text/x-wiki 3gnlljd3wzwceborajprrabpmus6n9w 2467873 2467872 2026-10-14T13:09:13Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2259.codfw.wmnet 2467873 wikitext text/x-wiki neue98p036g37bqfspibrecybeblp8i 2467874 2467873 2026-10-14T13:09:27Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2260.codfw.wmnet 2467874 wikitext text/x-wiki 3fnrj4w3h8n4g0w5346f66xdo7ir5me 2467875 2467874 2026-10-14T13:10:00Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2260.codfw.wmnet 2467875 wikitext text/x-wiki 0v2rxsykruily08267ezh2av6k7l40l 2467876 2467875 2026-10-14T13:10:21Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2260.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467876 wikitext text/x-wiki rki84bp5h4t2mrzp2qiasj5yaf5p7o8 2467877 2467876 2026-10-14T13:11:48Z Stashbot 7414 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1347373|Enable mobile editing for all Wikidata editors (T437054)]] 2467877 wikitext text/x-wiki d2p2k676w54u5hi20m3lfok6jcsq1gx 2467878 2467877 2026-10-14T13:13:45Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1299.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467878 wikitext text/x-wiki 6sjzpnm86n6nyduyg86s65f5vvfonbt 2467879 2467878 2026-10-14T13:14:04Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1299.eqiad.wmnet 2467879 wikitext text/x-wiki 2hrzm8qfofsfxos0dc8ot5xdinhpb28 2467880 2467879 2026-10-14T13:14:06Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1299.eqiad.wmnet 2467880 wikitext text/x-wiki dl2vuxp24thq4e70coi389gx0jk99k5 2467881 2467880 2026-10-14T13:14:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1300.eqiad.wmnet 2467881 wikitext text/x-wiki a05idw6a4ekstw5dy62jfsuj8ebzgkd 2467882 2467881 2026-10-14T13:14:50Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1300.eqiad.wmnet 2467882 wikitext text/x-wiki edbanezbp0r37a43fxqyew2ydmijnle 2467883 2467882 2026-10-14T13:15:11Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1300.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467883 wikitext text/x-wiki d0c9dvy2354ds5u909i3zzzqlq1kjjf 2467885 2467883 2026-10-14T13:15:40Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2260.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467885 wikitext text/x-wiki 2micafvj6vw2ukrkdk00dpwapgp6sxs 2467886 2467885 2026-10-14T13:15:46Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2260.codfw.wmnet 2467886 wikitext text/x-wiki 81hl69f38f811am8zz10v64hlusbna1 2467887 2467886 2026-10-14T13:16:01Z Stashbot 7414 krinkle@deploy1003: krinkle, sadiyamohammed13: Backport for [[gerrit:1347373|Enable mobile editing for all Wikidata editors (T437054)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2467887 wikitext text/x-wiki i8r8vgh4k4puvyalra5kyh1ei6kijlu 2467888 2467887 2026-10-14T13:17:10Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2260.codfw.wmnet 2467888 wikitext text/x-wiki stuyhcc4b3h285xdrcrvab3sxd30eko 2467889 2467888 2026-10-14T13:17:11Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.enable_virtualization (exit_code=0) rolling provision on D{wikikube-worker[2088,2203,2242-2243,2248-2260].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2467889 wikitext text/x-wiki 2ssu1xdf6irsw1e6f793p7926rz3928 2467890 2467889 2026-10-14T13:19:38Z Stashbot 7414 krinkle@deploy1003: krinkle, sadiyamohammed13: Continuing with deployment 2467890 wikitext text/x-wiki 3h7mubs2np27m4uw8vlf066rk89uelo 2467891 2467890 2026-10-14T13:20:39Z Stashbot 7414 root@deploy1003: helmfile [staging] START helmfile.d/services/shellbox: sync 2467891 wikitext text/x-wiki 8i807w6undcidwjir79mebre3qhsvb8 2467892 2467891 2026-10-14T13:20:42Z Stashbot 7414 root@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox: sync 2467892 wikitext text/x-wiki 7mlb9ra82z4a9nn738xmmarg1f9nfvq 2467893 2467892 2026-10-14T13:20:46Z Stashbot 7414 root@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-timeline: sync 2467893 wikitext text/x-wiki 1gxe9x6szpe1p52n20zizjzf0brsco5 2467894 2467893 2026-10-14T13:20:59Z Stashbot 7414 root@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-timeline: sync 2467894 wikitext text/x-wiki onyffp68ooz8kn9e2uyyellcqg1myrh 2467896 2467894 2026-10-14T13:25:02Z Stashbot 7414 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347373|Enable mobile editing for all Wikidata editors (T437054)]] (duration: 13m 14s) 2467896 wikitext text/x-wiki 5zbdy7639ybqjmu3thhsgasjvi2ro8l 2467897 2467896 2026-10-14T13:25:05Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1300.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467897 wikitext text/x-wiki ppl9nib4f0yxqdjib96gkgfcjxjxviq 2467898 2467897 2026-10-14T13:25:30Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1300.eqiad.wmnet 2467898 wikitext text/x-wiki lk259qzz7jvqsh14ss90wckaqgi64rg 2467899 2467898 2026-10-14T13:26:03Z Stashbot 7414 daniel@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2467899 wikitext text/x-wiki r1uxu3sklhhxetio6i2chxkbsgpeb0k 2467900 2467899 2026-10-14T13:27:34Z Stashbot 7414 daniel@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2467900 wikitext text/x-wiki 9x5vesm6egl0vpgxk5445uzygi31qdq 2467901 2467900 2026-10-14T13:27:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1300.eqiad.wmnet 2467901 wikitext text/x-wiki ihhcl3wmwe31oskxo15d9q8c3wl2caq 2467902 2467901 2026-10-14T13:27:55Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.enable_virtualization (exit_code=0) rolling provision on D{wikikube-worker[1299-1300].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) 2467902 wikitext text/x-wiki pdiz6yiymut79ot0s2qhsxdhhnxrp5u 2467903 2467902 2026-10-14T13:29:55Z Stashbot 7414 JavierMonton: Deploying Refinery at 5faed811 for changes 1352228, 1344135, 1354754 2467903 wikitext text/x-wiki aeysz1cwudxd9fgabpaasidyj2kos5x 2467905 2467903 2026-10-14T13:30:49Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.enable_virtualization rolling provision on D{wikikube-worker[1300-1330].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) 2467905 wikitext text/x-wiki 6sribs3atorlec0qmzszyhiwytoevsr 2467906 2467905 2026-10-14T13:30:53Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1301.eqiad.wmnet 2467906 wikitext text/x-wiki s7nvhxpunq571058a294dysa3w708vu 2467907 2467906 2026-10-14T13:31:21Z Stashbot 7414 javiermonton@deploy1003: Started deploy [analytics/refinery@5faed81] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@5faed811] 2467907 wikitext text/x-wiki 7ec8toquy5z1tgrwy8tt4w99zd5rc79 2467908 2467907 2026-10-14T13:31:25Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1301.eqiad.wmnet 2467908 wikitext text/x-wiki g8jxnwk3mdvt68stigxtci51nknvh43 2467909 2467908 2026-10-14T13:31:45Z Stashbot 7414 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1355039|WikiMap: Let $wgConf take precedence over "current wiki" (T440997 T428296)]] (1.47.0-wmf.24) 2467909 wikitext text/x-wiki o8gww74lgkr5oy95p735pkpm4ky318s 2467911 2467909 2026-10-14T13:31:48Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1301.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467911 wikitext text/x-wiki bf1xldnf4z6i5zz25e3a7mjtgrzvi0e 2467912 2467911 2026-10-14T13:32:06Z Stashbot 7414 javiermonton@deploy1003: Finished deploy [analytics/refinery@5faed81] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@5faed811] (duration: 00m 45s) 2467912 wikitext text/x-wiki ca3n8963igkd3kfayinepafz0yu6p3s 2467913 2467912 2026-10-14T13:32:18Z Stashbot 7414 javiermonton@deploy1003: Started deploy [analytics/refinery@5faed81] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@5faed811] 2467913 wikitext text/x-wiki ep6zz0sxks8qdf4nchl0puncqxeyi0i 2467914 2467913 2026-10-14T13:32:25Z Stashbot 7414 javiermonton@deploy1003: Finished deploy [analytics/refinery@5faed81] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@5faed811] (duration: 00m 07s) 2467914 wikitext text/x-wiki gdctg84m375tak8y9k1v1g3jbju007j 2467915 2467914 2026-10-14T13:33:45Z Stashbot 7414 javiermonton@deploy1003: Started deploy [analytics/refinery@5faed81]: Regular analytics weekly train [analytics/refinery@5faed811] 2467915 wikitext text/x-wiki 37tp60cc9uym3e72ud74kinsvkt8dv1 2467916 2467915 2026-10-14T13:35:20Z Stashbot 7414 daniel@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply 2467916 wikitext text/x-wiki 9z1qjupnjncww5mshyi6ki4oqr8wl6k 2467917 2467916 2026-10-14T13:35:54Z Stashbot 7414 daniel@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply 2467917 wikitext text/x-wiki 99ibfha95l16oidqtir4spxckfxg0mz 2467918 2467917 2026-10-14T13:35:59Z Stashbot 7414 krinkle@deploy1003: krinkle: Backport for [[gerrit:1355039|WikiMap: Let $wgConf take precedence over "current wiki" (T440997 T428296)]] (1.47.0-wmf.24) synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2467918 wikitext text/x-wiki hf4pupykwtrpunpyzfq94l4ikve96ly 2467919 2467918 2026-10-14T13:37:34Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.enable_virtualization rolling provision on D{wikikube-worker[2261-2290].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2467919 wikitext text/x-wiki q8dmjtsourb5p91f5j2hwgptcenlecx 2467920 2467919 2026-10-14T13:37:38Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2261.codfw.wmnet 2467920 wikitext text/x-wiki iqqhmwoute6ev2lm5yl4f2mslsfun7l 2467921 2467920 2026-10-14T13:38:12Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2261.codfw.wmnet 2467921 wikitext text/x-wiki 6mcwcl4n5870xc66m6mswd5epj9l8yp 2467922 2467921 2026-10-14T13:38:16Z Stashbot 7414 javiermonton@deploy1003: Finished deploy [analytics/refinery@5faed81]: Regular analytics weekly train [analytics/refinery@5faed811] (duration: 04m 31s) 2467922 wikitext text/x-wiki ro0xdxx0y92f5450hjziixulz6rlu87 2467923 2467922 2026-10-14T13:38:18Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.memcached.roll-reboot-restart (exit_code=0) rolling restart_daemons on A:memcached-codfw 2467923 wikitext text/x-wiki 3bucuewlhvq43sl20ks9grpgcs6rkyc 2467924 2467923 2026-10-14T13:38:34Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2261.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467924 wikitext text/x-wiki pzl9g8zc9t0u0nykbokxoe7ea1becw0 2467925 2467924 2026-10-14T13:39:02Z Stashbot 7414 oblivian@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 14 days, 0:00:00 on wikikube-worker2195.codfw.wmnet with reason: Hw failure 2467925 wikitext text/x-wiki 645356cs5dossx02t8je51i7a2w2url 2467926 2467925 2026-10-14T13:39:31Z Stashbot 7414 krinkle@deploy1003: krinkle: Continuing with deployment 2467926 wikitext text/x-wiki 1v4p56pos7cts53oo877l28jg8tn4wr 2467927 2467926 2026-10-14T13:40:45Z Stashbot 7414 oblivian@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 14 days, 0:00:00 on wikikube-worker2192.codfw.wmnet with reason: Hw failure 2467927 wikitext text/x-wiki f7tnyifhzbhwmgc42iy9v9nb2yy8s9l 2467928 2467927 2026-10-14T13:41:35Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1301.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467928 wikitext text/x-wiki 43cp1666o59bxf97osid2suqzvrzwgo 2467930 2467928 2026-10-14T13:41:54Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1301.eqiad.wmnet 2467930 wikitext text/x-wiki 9lqmu8bj43kmq9w9sczzvr2mp53xa0s 2467931 2467930 2026-10-14T13:42:00Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1301.eqiad.wmnet 2467931 wikitext text/x-wiki su874kwxoubm1f87eyw60gletdsy4wu 2467932 2467931 2026-10-14T13:42:14Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1302.eqiad.wmnet 2467932 wikitext text/x-wiki oqxqlx5q2gixf4w7te65wfhgztsvfph 2467933 2467932 2026-10-14T13:42:42Z Stashbot 7414 oblivian@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 14 days, 0:00:00 on wikikube-worker1298.eqiad.wmnet with reason: Hw failure 2467933 wikitext text/x-wiki c3jsimruny6h390mk5475jvjteozwe1 2467934 2467933 2026-10-14T13:42:58Z Stashbot 7414 daniel@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply 2467934 wikitext text/x-wiki rdbxz45sk1grpw44ks40eali53ijdkh 2467935 2467934 2026-10-14T13:43:48Z Stashbot 7414 daniel@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply 2467935 wikitext text/x-wiki 3c64a3bdvb3pnz13p2vd34hougu2o8c 2467936 2467935 2026-10-14T13:43:52Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2261.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467936 wikitext text/x-wiki baxzirray374sp4rpvmuzsnzh4nyxku 2467937 2467936 2026-10-14T13:43:59Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2261.codfw.wmnet 2467937 wikitext text/x-wiki dd0uhs0ei8j3c0ns63fjiyw5cwojhhq 2467938 2467937 2026-10-14T13:44:41Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2261.codfw.wmnet 2467938 wikitext text/x-wiki 8uieapgkykgijkcxneyui47t8rnyz9x 2467939 2467938 2026-10-14T13:44:55Z Stashbot 7414 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1355039|WikiMap: Let $wgConf take precedence over "current wiki" (T440997 T428296)]] (1.47.0-wmf.24) (duration: 13m 08s) 2467939 wikitext text/x-wiki r1ssg2jkndlzoh9m19rgul2tvh23ryz 2467940 2467939 2026-10-14T13:44:58Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2262.codfw.wmnet 2467940 wikitext text/x-wiki nnoud3g39ar6n5uvs25052ahjmlcnq3 2467941 2467940 2026-10-14T13:45:29Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2262.codfw.wmnet 2467941 wikitext text/x-wiki m944ix7ytu6l85r8l3bgyv4dk9yx7q7 2467942 2467941 2026-10-14T13:45:50Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2262.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467942 wikitext text/x-wiki plomsau8724ffvom8cy6c57ibbjoh0b 2467943 2467942 2026-10-14T13:45:57Z Stashbot 7414 kamila@deploy1003: Started scap deploy-service shellbox: pilot PHP 8.5 on shellbox-media 2467943 wikitext text/x-wiki 3omqgpx4t7im0y2ydfwfsyoxdfxiykn 2467944 2467943 2026-10-14T13:45:59Z Stashbot 7414 kamila@deploy1003: scap deploy-service shellbox: [STAGING/DEFAULT] staging-eqiad Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2467944 wikitext text/x-wiki otdr2aqls07xik6yik9us2m1gcoiwka 2467945 2467944 2026-10-14T13:46:22Z Stashbot 7414 kamila@deploy1003: scap deploy-service shellbox: [PRODUCTION/DEFAULT] codfw Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2467945 wikitext text/x-wiki fw0lsmtz8la88otnb4o2h8b23uqx6im 2467946 2467945 2026-10-14T13:47:17Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1302.eqiad.wmnet 2467946 wikitext text/x-wiki ljb7svid7fh4sn4hvhlek0icnxnvc79 2467947 2467946 2026-10-14T13:47:37Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1302.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467947 wikitext text/x-wiki b0qau68nme9kuzjr91122o5qvf3zwwy 2467948 2467947 2026-10-14T13:47:52Z Stashbot 7414 kamila@deploy1003: scap deploy-service shellbox: [PRODUCTION/DEFAULT] eqiad Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2467948 wikitext text/x-wiki 68s18yuk1200jsb1e6t9lnfjr3a678g 2467949 2467948 2026-10-14T13:48:26Z Stashbot 7414 kamila@deploy1003: Finished scap deploy-service shellbox: pilot PHP 8.5 on shellbox-media (duration: 02m 29s) 2467949 wikitext text/x-wiki h2b8yz3fil4zyk5qzazycsme61ur6g0 2467950 2467949 2026-10-14T13:50:10Z Stashbot 7414 papaul: start maintenance on mr1-eqiad 2467950 wikitext text/x-wiki g30ijm3d2rago13ejfupn94o0np82dg 2467951 2467950 2026-10-14T13:50:51Z Stashbot 7414 pt1979@cumin1004: DONE (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on 8 hosts with reason: Junos upgrade 2467951 wikitext text/x-wiki mtaivopg8wory63cdrgbowv6cc2zab9 2467952 2467951 2026-10-14T13:51:07Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2262.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467952 wikitext text/x-wiki ilf1y4shejtkrxyn8x4u7t3wdxyxto5 2467953 2467952 2026-10-14T13:51:25Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2262.codfw.wmnet 2467953 wikitext text/x-wiki tb3pqz4t14wlclef08r13gzk74oxlr9 2467954 2467953 2026-10-14T13:51:46Z Stashbot 7414 pt1979@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on 8 hosts with reason: Junos upgrade 2467954 wikitext text/x-wiki dkf02d6qcswlskq9zurj2g6dj2ulkgv 2467955 2467954 2026-10-14T13:52:20Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2262.codfw.wmnet 2467955 wikitext text/x-wiki qdrwcy6a9fpukmvwdjxmu7ogtzv4bnn 2467956 2467955 2026-10-14T13:52:33Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2263.codfw.wmnet 2467956 wikitext text/x-wiki gv59b76zvpq4ktndhr5irzilcm0s5td 2467957 2467956 2026-10-14T13:53:08Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2263.codfw.wmnet 2467957 wikitext text/x-wiki m0tn21zxcgrjrnfaxm9zttsizxl49d2 2467958 2467957 2026-10-14T13:53:29Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2263.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467958 wikitext text/x-wiki s8lo7l3rlma3ehqiplqd7a3ldg561wu 2467959 2467958 2026-10-14T13:58:44Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2263.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467959 wikitext text/x-wiki 5n6nf87caghtfpmlwiahutfkk2ckzvp 2467960 2467959 2026-10-14T13:59:01Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2263.codfw.wmnet 2467960 wikitext text/x-wiki 1snbjkx13wd856xcadnou6f978loeqt 2467961 2467960 2026-10-14T14:00:02Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2263.codfw.wmnet 2467961 wikitext text/x-wiki mj8x1jz94su0q6pdbbnzg9wb3f480c4 2467962 2467961 2026-10-14T14:00:13Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker1302.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467962 wikitext text/x-wiki s0xl5zp7zcwjw8fi7dnlj476tjam1ig 2467963 2467962 2026-10-14T14:00:16Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2264.codfw.wmnet 2467963 wikitext text/x-wiki s6ro9yi62fponpwwfeeei6mehb0j3d1 2467966 2467963 2026-10-14T14:00:50Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2264.codfw.wmnet 2467966 wikitext text/x-wiki 9rpjv5mmtxrus0w8ksi1a22u2hwp9ew 2467967 2467966 2026-10-14T14:01:10Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2264.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467967 wikitext text/x-wiki riccgzojijqmz2xst5z0f8ozx7nh6ki 2467968 2467967 2026-10-14T14:01:18Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2264.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467968 wikitext text/x-wiki rljvhe9elpgi9hqwqhrpw70tm6hpl3u 2467969 2467968 2026-10-14T14:04:47Z Stashbot 7414 jforrester@deploy1003: Started scap sync-world: Backport for [[gerrit:1354857|dagwiki & igwiki: Enable Abstract Wikimedia client mode]], [[gerrit:1352573|wikifunctions: Drop CodeMirror enablement, defailt on (T400015)]] 2467969 wikitext text/x-wiki 7dm9r4gmmckr4wu0lxi43p4dra5c8kw 2467970 2467969 2026-10-14T14:09:06Z Stashbot 7414 jforrester@deploy1003: jforrester: Backport for [[gerrit:1354857|dagwiki & igwiki: Enable Abstract Wikimedia client mode]], [[gerrit:1352573|wikifunctions: Drop CodeMirror enablement, defailt on (T400015)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2467970 wikitext text/x-wiki fnj26z5btg59pe2bczi9zw7c4jxc444 2467971 2467970 2026-10-14T14:09:28Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/mediawiki-dumps-legacy: apply 2467971 wikitext text/x-wiki gllnmh49i5kxp8uhbpyzryg14p4d84z 2467972 2467971 2026-10-14T14:09:38Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/mediawiki-dumps-legacy: apply 2467972 wikitext text/x-wiki aqdohgcxou2mtlyybsej1rbocbanq3p 2467973 2467972 2026-10-14T14:09:40Z Stashbot 7414 jforrester@deploy1003: jforrester: Continuing with deployment 2467973 wikitext text/x-wiki 4uf77g97o15t91xlvrcg1tog2bxtbue 2467974 2467973 2026-10-14T14:15:01Z Stashbot 7414 jforrester@deploy1003: Finished scap sync-world: Backport for [[gerrit:1354857|dagwiki & igwiki: Enable Abstract Wikimedia client mode]], [[gerrit:1352573|wikifunctions: Drop CodeMirror enablement, defailt on (T400015)]] (duration: 10m 14s) 2467974 wikitext text/x-wiki s1ez8vlj3exyordviq17c9954ou2u6v 2467980 2467974 2026-10-14T14:28:58Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=yes; selector: name=cp2043.* 2467980 wikitext text/x-wiki b6z6fgzkwxqw5mt11ipv8o640cfjryc 2467981 2467980 2026-10-14T14:30:05Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1302.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467981 wikitext text/x-wiki 2u4mjg8quvdvmxykmikwyyhftdn1g4v 2467982 2467981 2026-10-14T14:33:51Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker1302.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467982 wikitext text/x-wiki 4oumb8pjfzgraraxhvv5csppcy4mb5e 2467983 2467982 2026-10-14T14:33:57Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.enable_virtualization (exit_code=1) rolling provision on D{wikikube-worker[1300-1330].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) 2467983 wikitext text/x-wiki eycgmzlw9jn7ke4w738u61g94l1krjo 2467984 2467983 2026-10-14T14:38:03Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.enable_virtualization rolling provision on D{wikikube-worker[1303-1330].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) 2467984 wikitext text/x-wiki 5ogrgwwrn8ux78qwptcn8d0w7rx2hgu 2467985 2467984 2026-10-14T14:38:06Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1303.eqiad.wmnet 2467985 wikitext text/x-wiki duoujyjqilibgjetjm8jof33ad61c1q 2467986 2467985 2026-10-14T14:38:40Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1303.eqiad.wmnet 2467986 wikitext text/x-wiki mfu720nkogn4gwztbh21m51gvmc1qj8 2467987 2467986 2026-10-14T14:39:01Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1303.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467987 wikitext text/x-wiki qa0ebipvac9fliu1e1wo4aq5uyfnbqn 2467988 2467987 2026-10-14T14:46:10Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2264.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467988 wikitext text/x-wiki i1ffdp5u2y5apusztem7idpy1jjcwvq 2467989 2467988 2026-10-14T14:46:18Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2264.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467989 wikitext text/x-wiki fmoxm036ehhdho5420wdjd80ea0ehzh 2467990 2467989 2026-10-14T14:46:29Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2264.codfw.wmnet 2467990 wikitext text/x-wiki 6p9gsxmu1t25wlk1bshrsch7m7vgdz0 2467991 2467990 2026-10-14T14:46:31Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2264.codfw.wmnet 2467991 wikitext text/x-wiki 2xrtu4qqpbrmbgwj4g0za5qq9p169n6 2467992 2467991 2026-10-14T14:46:44Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2265.codfw.wmnet 2467992 wikitext text/x-wiki kcu8pcm1evm9j3x8bv85wsn5qwdbaxp 2467993 2467992 2026-10-14T14:47:54Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2265.codfw.wmnet 2467993 wikitext text/x-wiki s831sfkjxoy3hlh4raxjge41a6780gg 2467994 2467993 2026-10-14T14:48:03Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2265.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2467994 wikitext text/x-wiki 43ob78z05e3yr7g2wuxa1rwbvedqllb 2467995 2467994 2026-10-14T14:48:35Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.298.0" for 158 host(s) 2467995 wikitext text/x-wiki e7mjahldc5mj3kbrhsrvi5e7egyau14 2467996 2467995 2026-10-14T14:48:51Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1303.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2467996 wikitext text/x-wiki 9dsxlr532cjs7fbf07hicmnrdzted1p 2467997 2467996 2026-10-14T14:49:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1303.eqiad.wmnet 2467997 wikitext text/x-wiki 5cm4a0bm58dvning10byyhszvyryudf 2467998 2467997 2026-10-14T14:49:21Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1303.eqiad.wmnet 2467998 wikitext text/x-wiki 1w92cwncivtv45quxzebu4xp3w2i2od 2467999 2467998 2026-10-14T14:49:34Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1304.eqiad.wmnet 2467999 wikitext text/x-wiki pgyio0tp745cavcyzpzlkwiylw4dj5s 2468001 2467999 2026-10-14T14:50:11Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply 2468001 wikitext text/x-wiki 670180iozcv1nmmoe3miv0jty9rxs0x 2468002 2468001 2026-10-14T14:50:27Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.remove-downtime for cp2043.codfw.wmnet 2468002 wikitext text/x-wiki 0ieap46siz8lwra9ugwe8vpisygsvzk 2468003 2468002 2026-10-14T14:50:29Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cp2043.codfw.wmnet 2468003 wikitext text/x-wiki gbggk7kxwurcuoogcbi50zlx2a5onha 2468004 2468003 2026-10-14T14:50:37Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply 2468004 wikitext text/x-wiki 66k5w57ybww4nwzuj7dilvph4b86lid 2468005 2468004 2026-10-14T14:50:41Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1304.eqiad.wmnet 2468005 wikitext text/x-wiki hqs8d12qa1rawrma43e34bz5ak6n7cw 2468006 2468005 2026-10-14T14:51:00Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1304.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468006 wikitext text/x-wiki cgjjpnv8mvhmi7rn98q5osq3mpr02h9 2468007 2468006 2026-10-14T14:52:41Z Stashbot 7414 dancy@deploy1003: Installation of scap version "4.298.0" completed for 158 hosts 2468007 wikitext text/x-wiki 5qqzmvi0tw9txzwdiamhs4yj9nop2e6 2468008 2468007 2026-10-14T14:53:19Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2265.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468008 wikitext text/x-wiki 39h393lu1mhvl229yafw5ajw1070u49 2468009 2468008 2026-10-14T14:53:25Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2265.codfw.wmnet 2468009 wikitext text/x-wiki iksmxutwuvlos51ckxwzwprjoj3p6s0 2468010 2468009 2026-10-14T14:54:25Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2265.codfw.wmnet 2468010 wikitext text/x-wiki c4sc7q3sibbyl2hmdn8x3pqdtnbixzo 2468011 2468010 2026-10-14T14:54:39Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2266.codfw.wmnet 2468011 wikitext text/x-wiki 5pm5yfq1luz75gjuwkbpc6xnd9trq3o 2468012 2468011 2026-10-14T14:55:13Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2266.codfw.wmnet 2468012 wikitext text/x-wiki sl4tzj5incq13i65nm0usr70xhc8rfe 2468013 2468012 2026-10-14T14:55:33Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2266.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468013 wikitext text/x-wiki 8myexicjq7dj0hsuwh6mpndl7bsn796 2468014 2468013 2026-10-14T15:00:51Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1304.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468014 wikitext text/x-wiki mkjbdw7xtdddp54pxyua03rd2r3gtok 2468015 2468014 2026-10-14T15:00:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2266.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468015 wikitext text/x-wiki odtj82tjpa6n4yyvcdw2tbj1lmm3rei 2468016 2468015 2026-10-14T15:00:57Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1304.eqiad.wmnet 2468016 wikitext text/x-wiki 4jmpu13l0odsojr6mom03h0iiyvbwgb 2468017 2468016 2026-10-14T15:00:58Z Stashbot 7414 elukey: powercycle pki1002, unreachable via ssh, continuous stack dumps on the tty via mgmg console 2468017 wikitext text/x-wiki d5mvy6uvoyflxw3whbd016c3i5t2cua 2468018 2468017 2026-10-14T15:01:03Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2266.codfw.wmnet 2468018 wikitext text/x-wiki 43bksax2fqblc9ud0d9ry2g9sz1dq4d 2468019 2468018 2026-10-14T15:01:52Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2266.codfw.wmnet 2468019 wikitext text/x-wiki g9c3nfrzf5ay2d8meeo81345cefiu5q 2468020 2468019 2026-10-14T15:02:05Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2267.codfw.wmnet 2468020 wikitext text/x-wiki j2mcu5f8y0tf0kbqwpfjlufs4r3fidr 2468021 2468020 2026-10-14T15:02:39Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2267.codfw.wmnet 2468021 wikitext text/x-wiki c4vmoih27qewm6aorqgoci5r34hhtao 2468022 2468021 2026-10-14T15:02:59Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2267.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468022 wikitext text/x-wiki 3lvdz3vt1g748qi9p2yysn2g3byi8no 2468023 2468022 2026-10-14T15:03:21Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1304.eqiad.wmnet 2468023 wikitext text/x-wiki 4eun0arq016weh0akuafrlkjh89tnmy 2468024 2468023 2026-10-14T15:03:35Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1305.eqiad.wmnet 2468024 wikitext text/x-wiki kqj8ajd495bqviash2wx0cig73zoxz7 2468025 2468024 2026-10-14T15:04:06Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1305.eqiad.wmnet 2468025 wikitext text/x-wiki 3dsl8emynrrkavnvwq5ot0vstew9pjt 2468026 2468025 2026-10-14T15:04:26Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1305.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468026 wikitext text/x-wiki fhteaakveiqu1hv1jirhhygd736up5i 2468027 2468026 2026-10-14T15:08:18Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2267.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468027 wikitext text/x-wiki s7u2alt3vxm4rze6de2baea5b7o0279 2468028 2468027 2026-10-14T15:08:24Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2267.codfw.wmnet 2468028 wikitext text/x-wiki gt9g74jovu829ugxk34ecm78tr7bjbp 2468030 2468028 2026-10-14T15:09:06Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2267.codfw.wmnet 2468030 wikitext text/x-wiki 8kethuyrjjrwwkibtz0nuq3czytj49r 2468031 2468030 2026-10-14T15:09:20Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2268.codfw.wmnet 2468031 wikitext text/x-wiki qibu0wf86oixoc14m2yme014lbdsvgu 2468032 2468031 2026-10-14T15:09:55Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2268.codfw.wmnet 2468032 wikitext text/x-wiki dyzorvpiy5lqw2ktbtqtmjraq6s5uvx 2468033 2468032 2026-10-14T15:10:14Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2268.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468033 wikitext text/x-wiki irvbv2gfonx6rmgx50uwfk6sdfk7miq 2468034 2468033 2026-10-14T15:12:46Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1305.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468034 wikitext text/x-wiki cp5tk9mmnkbtl1k2a7du05c6040ejpl 2468035 2468034 2026-10-14T15:13:04Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1305.eqiad.wmnet 2468035 wikitext text/x-wiki cj3bzm8rgzkuqzu5wxt4nex21gcv6tx 2468036 2468035 2026-10-14T15:13:07Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1305.eqiad.wmnet 2468036 wikitext text/x-wiki 9wb82delze63dls6l5ynevcgsnfl5sd 2468037 2468036 2026-10-14T15:13:20Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1306.eqiad.wmnet 2468037 wikitext text/x-wiki de7dc3gj242vkt2ve7c4to0h7hom5nt 2468038 2468037 2026-10-14T15:13:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1306.eqiad.wmnet 2468038 wikitext text/x-wiki 9ym4mhrfzpnxgrul1knazhbhm20bvnd 2468039 2468038 2026-10-14T15:14:12Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1306.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468039 wikitext text/x-wiki a04gn44mo3robhd3na86bbfl84tf3pf 2468040 2468039 2026-10-14T15:15:35Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2268.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468040 wikitext text/x-wiki 730v3wfx0san2a25que0qvpewlgb0s0 2468041 2468040 2026-10-14T15:15:41Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2268.codfw.wmnet 2468041 wikitext text/x-wiki sgo96xyswsqosp93fhhs9yynfeb9nv9 2468042 2468041 2026-10-14T15:16:47Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2268.codfw.wmnet 2468042 wikitext text/x-wiki loil5p9g49oq70pany2947xrd3xeve5 2468043 2468042 2026-10-14T15:17:00Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2269.codfw.wmnet 2468043 wikitext text/x-wiki 20y97tvcg1a4690mr5thpu03n0rz945 2468044 2468043 2026-10-14T15:17:35Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2269.codfw.wmnet 2468044 wikitext text/x-wiki fju875hkyc5a91fz1tggthcvhvu5izq 2468045 2468044 2026-10-14T15:17:55Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2269.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468045 wikitext text/x-wiki bosnogwzdxrmg731rexpp7wkcfh8h7q 2468046 2468045 2026-10-14T15:18:34Z Stashbot 7414 papaul: maintenance on mr1-eqiad complete 2468046 wikitext text/x-wiki 71p12r0crgllgo441aqz722pbhmuza3 2468047 2468046 2026-10-14T15:21:05Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . 2468047 wikitext text/x-wiki 2sgtta7me34orxiojt1bgm9idev586r 2468048 2468047 2026-10-14T15:23:03Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1306.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468048 wikitext text/x-wiki i4j4ut7jmg9hri2lwo3cv3p2nhlmmih 2468049 2468048 2026-10-14T15:23:13Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2269.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468049 wikitext text/x-wiki 3wivj8ncenkp1b985m515ykreizwpgw 2468050 2468049 2026-10-14T15:23:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2269.codfw.wmnet 2468050 wikitext text/x-wiki 5xhdysle6q9ksw27p4wf60m01s5bns6 2468051 2468050 2026-10-14T15:23:21Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1306.eqiad.wmnet 2468051 wikitext text/x-wiki r848y02yh1undofvlqiaqm9nmo0t0kj 2468052 2468051 2026-10-14T15:23:23Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1306.eqiad.wmnet 2468052 wikitext text/x-wiki gb4xtrjvejahi1hfjcvf7uklucvb715 2468053 2468052 2026-10-14T15:23:36Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1307.eqiad.wmnet 2468053 wikitext text/x-wiki 77qegvj8lxnvcguajc13qpmjhorqbmf 2468054 2468053 2026-10-14T15:24:08Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1307.eqiad.wmnet 2468054 wikitext text/x-wiki 53aq9sj7pu0y8ny0nwhzbokna8zowif 2468055 2468054 2026-10-14T15:24:27Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1307.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468055 wikitext text/x-wiki 6jun2lyvc44gtm4zxk823tidyrb1ehj 2468056 2468055 2026-10-14T15:24:37Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2269.codfw.wmnet 2468056 wikitext text/x-wiki dt7v9zqfcft7wuqaeih5kryutx2xkfk 2468057 2468056 2026-10-14T15:24:50Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2270.codfw.wmnet 2468057 wikitext text/x-wiki lrj0hh94avmodotj9ddi3c9bhhoymf7 2468058 2468057 2026-10-14T15:25:24Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2270.codfw.wmnet 2468058 wikitext text/x-wiki 5mr80ayuo0raj7132v3rgyvzklbh8fi 2468059 2468058 2026-10-14T15:25:44Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2270.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468059 wikitext text/x-wiki q3gzx8zyd757qyqbv8yxwz2mmmfmexn 2468060 2468059 2026-10-14T15:26:22Z Stashbot 7414 elukey@cumin1004: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts sretest2003.codfw.wmnet 2468060 wikitext text/x-wiki ni71e9snj7gb8l5svlrh4z36sliddh2 2468061 2468060 2026-10-14T15:26:28Z Stashbot 7414 elukey@cumin1004: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts sretest2003.codfw.wmnet 2468061 wikitext text/x-wiki s3hl1plxq3wcyxtjurkouwu5npfwu3q 2468064 2468061 2026-10-14T15:31:00Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2270.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468064 wikitext text/x-wiki 4icq971kwepyjnesnlmvdj4962x49ui 2468065 2468064 2026-10-14T15:31:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2270.codfw.wmnet 2468065 wikitext text/x-wiki 4wk1pczrn7zg7d5yaw03hnn71wjzhij 2468066 2468065 2026-10-14T15:32:08Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2270.codfw.wmnet 2468066 wikitext text/x-wiki ma29gr19789a8hr77qb3uo2bg58mqv1 2468067 2468066 2026-10-14T15:32:21Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2271.codfw.wmnet 2468067 wikitext text/x-wiki 3nc7e96hqnq0h5pahlosvjzgtm6zfag 2468068 2468067 2026-10-14T15:32:56Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2271.codfw.wmnet 2468068 wikitext text/x-wiki p0bczabmxoh5vwnryw3ucyqkcsak0pc 2468069 2468068 2026-10-14T15:33:17Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2271.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468069 wikitext text/x-wiki 1cn0jaubj6k0whblzbjtexb4dwvlfv5 2468073 2468069 2026-10-14T15:38:36Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2271.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468073 wikitext text/x-wiki ls6asb04boo7xxa2vz5z733e9qlf15b 2468074 2468073 2026-10-14T15:38:52Z Stashbot 7414 elukey@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on pki1002.eqiad.wmnet with reason: T434268 2468074 wikitext text/x-wiki nf6qob4oa8leinhlog776eoqfi7jpu9 2468075 2468074 2026-10-14T15:38:54Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2271.codfw.wmnet 2468075 wikitext text/x-wiki k56ti3iuaf9suwvv2pnbnlko9ronqn0 2468076 2468075 2026-10-14T15:39:13Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2271.codfw.wmnet 2468076 wikitext text/x-wiki e3t8x2vyt576m5x1rcw6wxymtwf62r1 2468077 2468076 2026-10-14T15:39:18Z Stashbot 7414 kamila@deploy1003: Started scap deploy-service shellbox: roll out PHP 8.5 on shellbox-constraints 2468077 wikitext text/x-wiki m3zrf5jns906wbsyb0omjgbzqumqfwx 2468078 2468077 2026-10-14T15:39:20Z Stashbot 7414 kamila@deploy1003: scap deploy-service shellbox: [STAGING/DEFAULT] staging-eqiad Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2468078 wikitext text/x-wiki 9b95vil5emhkqqxqw4uuxhwnjiobqg3 2468079 2468078 2026-10-14T15:39:27Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2272.codfw.wmnet 2468079 wikitext text/x-wiki 84ukgf3vpdf662pxrnyz2sznagt1zct 2468080 2468079 2026-10-14T15:39:34Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] START helmfile.d/services/lac-lambdas: apply 2468080 wikitext text/x-wiki moy248l99wtb8ldwpgsjl2vmrs7it8l 2468081 2468080 2026-10-14T15:39:41Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] DONE helmfile.d/services/lac-lambdas: apply 2468081 wikitext text/x-wiki 1j2c22wgvy5wvzlfx5qgys8ky3n1fbu 2468082 2468081 2026-10-14T15:39:43Z Stashbot 7414 kamila@deploy1003: scap deploy-service shellbox: [PRODUCTION/DEFAULT] codfw Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2468082 wikitext text/x-wiki mi473vgqvdhauwh080mow8zrncvs0gz 2468083 2468082 2026-10-14T15:40:02Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2272.codfw.wmnet 2468083 wikitext text/x-wiki 3rcknwy6aev5oxa80d5doig3ieiza0k 2468084 2468083 2026-10-14T15:40:22Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2272.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468084 wikitext text/x-wiki tff8i5nkk4d0e32twj1jvuopivc0f37 2468085 2468084 2026-10-14T15:41:05Z Stashbot 7414 kamila@deploy1003: scap deploy-service shellbox: [PRODUCTION/DEFAULT] eqiad Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2468085 wikitext text/x-wiki 3x4vl0m2436sszo8c0vjmnvpv5aibvl 2468086 2468085 2026-10-14T15:45:41Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2272.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468086 wikitext text/x-wiki f66o3kjlwft7rzgxitoom1hfm747pp3 2468087 2468086 2026-10-14T15:45:47Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2272.codfw.wmnet 2468087 wikitext text/x-wiki 69x7t14pmd16ammr2y7gceslar6maho 2468088 2468087 2026-10-14T15:45:50Z Stashbot 7414 kamila@deploy1003: Finished scap deploy-service shellbox: roll out PHP 8.5 on shellbox-constraints (duration: 06m 32s) 2468088 wikitext text/x-wiki 0k6lyb6pl94xunmmg4hayj9vsxwdbdb 2468089 2468088 2026-10-14T15:46:30Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2272.codfw.wmnet 2468089 wikitext text/x-wiki ese7il8qeksyuv0uf878qwpf4tmms40 2468090 2468089 2026-10-14T15:46:44Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2273.codfw.wmnet 2468090 wikitext text/x-wiki o6724ea2v76mlt00avewpz72tfcapky 2468091 2468090 2026-10-14T15:47:17Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2273.codfw.wmnet 2468091 wikitext text/x-wiki k92fsmtdp9edekpf4rsgyfki2u7tibo 2468092 2468091 2026-10-14T15:47:38Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2273.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468092 wikitext text/x-wiki 6g229vzyg0pijuqyt809qp8jqjwuzio 2468093 2468092 2026-10-14T15:52:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2273.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468093 wikitext text/x-wiki a0fcxcxco3fidsj7f4ly9yz5np2311g 2468094 2468093 2026-10-14T15:52:59Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2273.codfw.wmnet 2468094 wikitext text/x-wiki cpb8mqfr64zilymlbt6syyhjy5jws65 2468095 2468094 2026-10-14T15:53:54Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2273.codfw.wmnet 2468095 wikitext text/x-wiki 054vvgeey70tljyushbcppqwfisxkgq 2468096 2468095 2026-10-14T15:54:08Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2274.codfw.wmnet 2468096 wikitext text/x-wiki e4xczb6v78vro9ggk61qir43kfgpnkn 2468097 2468096 2026-10-14T15:54:42Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2274.codfw.wmnet 2468097 wikitext text/x-wiki 6emmnb8jm8o9fl1tm7lxdi9p1ut4i6c 2468098 2468097 2026-10-14T15:55:02Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2274.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468098 wikitext text/x-wiki k1fyy74mk6t18v2gbtjrb9rrfy2t8hh 2468099 2468098 2026-10-14T15:57:57Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=no; selector: name=cp2044.* 2468099 wikitext text/x-wiki t0odaxnoubvmnsyjving6sxp4u8rmjt 2468100 2468099 2026-10-14T16:00:20Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2274.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468100 wikitext text/x-wiki o9soq8c8v9xmmvmc7hbrhlfzw2x7fi5 2468101 2468100 2026-10-14T16:00:38Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2274.codfw.wmnet 2468101 wikitext text/x-wiki niyoh9x0484uz12vo5sewsyxnex960i 2468102 2468101 2026-10-14T16:00:52Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp2044.codfw.wmnet 2468102 wikitext text/x-wiki 4wpefnohxtgad0t5pzezsd4h52120i4 2468103 2468102 2026-10-14T16:01:02Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts cp2044.codfw.wmnet 2468103 wikitext text/x-wiki tmyqjho46d8az12yfovcnxja2u9jssa 2468104 2468103 2026-10-14T16:01:51Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2274.codfw.wmnet 2468104 wikitext text/x-wiki bg7czr0cl5z4zcp39vlrspbl8u84oh2 2468105 2468104 2026-10-14T16:02:04Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2275.codfw.wmnet 2468105 wikitext text/x-wiki r8cgun5xorgyx3xqm7w34wbn9dwa8za 2468106 2468105 2026-10-14T16:02:39Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2275.codfw.wmnet 2468106 wikitext text/x-wiki 2088kammfrgfgsf2kgo0ww1z97ehme4 2468107 2468106 2026-10-14T16:02:59Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2275.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468107 wikitext text/x-wiki tlli41w4m3vmu3090z9d7k0co3oqekq 2468108 2468107 2026-10-14T16:03:29Z Stashbot 7414 ladsgroup@dns1004: START - running authdns-update 2468108 wikitext text/x-wiki pqmjffmpegwzjlgpjjfev1sy7hez4o7 2468110 2468108 2026-10-14T16:05:47Z Stashbot 7414 ladsgroup@dns1004: END - running authdns-update 2468110 wikitext text/x-wiki itxpddsw4smi98yldmqhlkalzflz516 2468111 2468110 2026-10-14T16:08:18Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2275.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468111 wikitext text/x-wiki 1rd1s3qtjjhszgc1h8a46setcg7vr4b 2468112 2468111 2026-10-14T16:08:24Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2275.codfw.wmnet 2468112 wikitext text/x-wiki e81proo4echqc7s896bum3s8k7bi815 2468113 2468112 2026-10-14T16:09:00Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1307.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468113 wikitext text/x-wiki dgipstan9z6wycmwvil95lpj0767fht 2468114 2468113 2026-10-14T16:09:20Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1307.eqiad.wmnet 2468114 wikitext text/x-wiki 18zvagdqmb3jnb6jdsoscv7ehjt1ugq 2468115 2468114 2026-10-14T16:09:48Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2275.codfw.wmnet 2468115 wikitext text/x-wiki p0skzgno7d59bw9yix5h1apc04n04ts 2468116 2468115 2026-10-14T16:09:59Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1307.eqiad.wmnet 2468116 wikitext text/x-wiki dceodd4q0vpxbz8ivattwxgi52d7mw4 2468117 2468116 2026-10-14T16:10:01Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2276.codfw.wmnet 2468117 wikitext text/x-wiki 50lb5y9vxroubgcl4ll9fmoo1khsjv7 2468118 2468117 2026-10-14T16:10:13Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1308.eqiad.wmnet 2468118 wikitext text/x-wiki i3plrmqas1brhyrto78lt81i2mliqjr 2468119 2468118 2026-10-14T16:10:35Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2276.codfw.wmnet 2468119 wikitext text/x-wiki 9f0asemhumuucb5p1c7p1tgia305tq6 2468120 2468119 2026-10-14T16:10:45Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1308.eqiad.wmnet 2468120 wikitext text/x-wiki kuzaa3vdh80x7mgklydtc2uy7113cil 2468121 2468120 2026-10-14T16:10:56Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2276.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468121 wikitext text/x-wiki s67x1k92ys7og5rimoxa63r1c60n2ws 2468122 2468121 2026-10-14T16:11:07Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1308.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468122 wikitext text/x-wiki q1pgkgkwx511t4l1rnzj8rxxzrcp3dq 2468123 2468122 2026-10-14T16:16:12Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2276.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468123 wikitext text/x-wiki bp49ezk2bnaglh54sxtpkmi7t3bs2v9 2468125 2468123 2026-10-14T16:16:18Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2276.codfw.wmnet 2468125 wikitext text/x-wiki o64jt4gml37sv5rt1uuzlwpx6agxqs0 2468126 2468125 2026-10-14T16:17:30Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2276.codfw.wmnet 2468126 wikitext text/x-wiki de4eeucwgefmmj3698w6wa2j04qt430 2468127 2468126 2026-10-14T16:17:43Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2277.codfw.wmnet 2468127 wikitext text/x-wiki 22u0lopaef6r1s3gsxh3psfgodton3o 2468128 2468127 2026-10-14T16:18:18Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2277.codfw.wmnet 2468128 wikitext text/x-wiki cwnulrgrqorwljaba7wx24luoifgsuj 2468129 2468128 2026-10-14T16:18:39Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2277.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468129 wikitext text/x-wiki 36hr12lxdk9slxyje4tgo6gs74r5spm 2468130 2468129 2026-10-14T16:20:27Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen: apply 2468130 wikitext text/x-wiki lx26x08qjtwb8fkhfcxltrg7m848xff 2468131 2468130 2026-10-14T16:20:37Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen: apply 2468131 wikitext text/x-wiki qyvt0r1gaoc2xex9j10ibavmzu96ha3 2468132 2468131 2026-10-14T16:20:55Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1308.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468132 wikitext text/x-wiki cgn6yvnvvnos6krsbiyaj1mso4zfz8g 2468133 2468132 2026-10-14T16:21:14Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1308.eqiad.wmnet 2468133 wikitext text/x-wiki cwttp6rtwwsx1yd30sw7i4f8v6r9im1 2468134 2468133 2026-10-14T16:22:18Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp2044.codfw.wmnet 2468134 wikitext text/x-wiki 0nlorhbhj97dhn9m47wy9eo75m2d9x6 2468135 2468134 2026-10-14T16:22:23Z Stashbot 7414 cdobbins@cumin1004: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts cp2044.codfw.wmnet 2468135 wikitext text/x-wiki jkpdhxv9bigsh4mas4uek7fmllvfojc 2468136 2468135 2026-10-14T16:22:44Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp2044.codfw.wmnet 2468136 wikitext text/x-wiki 0qhiyj1o5c027seje0l4xo0acm78vwt 2468137 2468136 2026-10-14T16:22:48Z Stashbot 7414 cdobbins@cumin1004: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts cp2044.codfw.wmnet 2468137 wikitext text/x-wiki 9l7yy6sx7litzh50213227ntpb14bs1 2468138 2468137 2026-10-14T16:23:33Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1308.eqiad.wmnet 2468138 wikitext text/x-wiki 9fwx8lm23cxg62rvttat6zscnjqx0lm 2468139 2468138 2026-10-14T16:23:47Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1309.eqiad.wmnet 2468139 wikitext text/x-wiki 00wku7oc9y0b940c9op225evrc2gtmr 2468140 2468139 2026-10-14T16:23:57Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2277.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468140 wikitext text/x-wiki 1w6iqubqlkt3nyp0fs40hvuhbx1ne2p 2468141 2468140 2026-10-14T16:24:03Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2277.codfw.wmnet 2468141 wikitext text/x-wiki efxkshzrqu802eb97abn9ywix8i5ly8 2468142 2468141 2026-10-14T16:24:19Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1309.eqiad.wmnet 2468142 wikitext text/x-wiki 3l15uq4gccy5klrtujczorl1l8onx49 2468143 2468142 2026-10-14T16:24:41Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1309.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468143 wikitext text/x-wiki 55g36lqct0o1szct0rm33en9vbs22h2 2468144 2468143 2026-10-14T16:25:21Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2277.codfw.wmnet 2468144 wikitext text/x-wiki nv9co4i7570blmi8ha1jn98853k9r5o 2468145 2468144 2026-10-14T16:25:35Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2278.codfw.wmnet 2468145 wikitext text/x-wiki rk1yw9a4zsdozh5lobm40qm8yhvr8x7 2468146 2468145 2026-10-14T16:26:08Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2278.codfw.wmnet 2468146 wikitext text/x-wiki g0m3k4yvhguxub33h12vubbr407kdje 2468147 2468146 2026-10-14T16:26:29Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2278.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468147 wikitext text/x-wiki 6wm387yd54lkpxu7aj1bxxh5bf1kmv4 2468149 2468147 2026-10-14T16:31:48Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2278.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468149 wikitext text/x-wiki o45irfv65vytyb4aaum95ng8qlkpein 2468150 2468149 2026-10-14T16:31:54Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2278.codfw.wmnet 2468150 wikitext text/x-wiki sf1ga43sb7yjwtvf9ztbjlzt8c42rm4 2468151 2468150 2026-10-14T16:31:56Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2278.codfw.wmnet 2468151 wikitext text/x-wiki kdh3449y840ocfl0bguzj25fcibtkez 2468152 2468151 2026-10-14T16:32:09Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2279.codfw.wmnet 2468152 wikitext text/x-wiki ts88pwvjkwuw1seb0gdexe6cot9nroi 2468153 2468152 2026-10-14T16:32:29Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp2044.codfw.wmnet 2468153 wikitext text/x-wiki tve0ldi1hehav6j97fhrvkq8muyvy09 2468154 2468153 2026-10-14T16:32:43Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2279.codfw.wmnet 2468154 wikitext text/x-wiki ow1h0a2d2y5cix1fnq0lmc5cj17g1tb 2468155 2468154 2026-10-14T16:33:02Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2279.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468155 wikitext text/x-wiki mg6v7zibpnozx6sj9r41lor1idq2bjx 2468156 2468155 2026-10-14T16:34:10Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.reboot-single for host cp2044.codfw.wmnet 2468156 wikitext text/x-wiki a80e6bjenb0yahpkdq7au6h19smlzd2 2468157 2468156 2026-10-14T16:38:19Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2279.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468157 wikitext text/x-wiki j2lx0zz4cia1xrntoyf0g7berocknmx 2468158 2468157 2026-10-14T16:38:47Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2279.codfw.wmnet 2468158 wikitext text/x-wiki gh493a6t13pvqqsy2iytzghn8adxi7o 2468159 2468158 2026-10-14T16:39:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2279.codfw.wmnet 2468159 wikitext text/x-wiki rvtdz0bhq2t4lbp4qazlrhyync5l85l 2468160 2468159 2026-10-14T16:40:07Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2280.codfw.wmnet 2468160 wikitext text/x-wiki i1qrmas7shaxqtkojmkxcemmq70e8o4 2468161 2468160 2026-10-14T16:40:40Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2280.codfw.wmnet 2468161 wikitext text/x-wiki 1tdhnckfz223tw2mzaeclgo66ln7266 2468162 2468161 2026-10-14T16:41:01Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2280.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468162 wikitext text/x-wiki k88hoxjdktfv7ohvxbyi8tebkxd2esu 2468163 2468162 2026-10-14T16:46:17Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2280.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468163 wikitext text/x-wiki goxc77b3szxdmbldsraa7fhp5cjbh4u 2468164 2468163 2026-10-14T16:46:28Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2280.codfw.wmnet 2468164 wikitext text/x-wiki 3nuxr8781dy48b1pdvb7t7dhf90w3ek 2468165 2468164 2026-10-14T16:46:30Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2280.codfw.wmnet 2468165 wikitext text/x-wiki 3fcpa96meaodqgdej8sa5vml7zkthx1 2468166 2468165 2026-10-14T16:46:44Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2281.codfw.wmnet 2468166 wikitext text/x-wiki dsqlq8gr59vtmoum6juk9zxe8u5jmjy 2468167 2468166 2026-10-14T16:47:19Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2281.codfw.wmnet 2468167 wikitext text/x-wiki 03s6itrl0j1wpcxtmzkbmq89e3c1ygc 2468168 2468167 2026-10-14T16:47:39Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2281.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468168 wikitext text/x-wiki o39ccebnihp8hxxkizf33i4wogg6zoi 2468170 2468168 2026-10-14T16:52:57Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2281.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468170 wikitext text/x-wiki 1h3ij85kkd3c2nthhov6w4ftbrwj44j 2468171 2468170 2026-10-14T16:53:03Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2281.codfw.wmnet 2468171 wikitext text/x-wiki 3xhpoi0iqddovw011zw0gv4s9h6pqyk 2468172 2468171 2026-10-14T16:53:05Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2281.codfw.wmnet 2468172 wikitext text/x-wiki pyfeas2irrlvucaurcnitw7uexatgum 2468173 2468172 2026-10-14T16:53:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2282.codfw.wmnet 2468173 wikitext text/x-wiki 1v9kt84zv38c0mtcs2uidvfca1d6dq7 2468175 2468173 2026-10-14T16:53:47Z Stashbot 7414 kamila@deploy1003: Started scap deploy-service shellbox: roll out php 8.5 on shellbox-media 2468175 wikitext text/x-wiki befmuw6b554lazu14pa8z6atuivr81s 2468176 2468175 2026-10-14T16:53:49Z Stashbot 7414 kamila@deploy1003: scap deploy-service shellbox: [STAGING/DEFAULT] staging-eqiad Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2468176 wikitext text/x-wiki pbov0nwv5fdru785vq3wxsevzyuo0an 2468177 2468176 2026-10-14T16:53:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2282.codfw.wmnet 2468177 wikitext text/x-wiki 1wgtf0h67gtyj3wkmxnewtfb28w0ygg 2468178 2468177 2026-10-14T16:54:14Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2282.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468178 wikitext text/x-wiki eklhckm6jq2mmebm4vp3ulvk4bp5k5e 2468179 2468178 2026-10-14T16:55:23Z Stashbot 7414 kamila@deploy1003: scap deploy-service shellbox: [PRODUCTION/DEFAULT] codfw Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2468179 wikitext text/x-wiki 9pg9sa8irh3k2nk8op9zef2zltu2kcp 2468181 2468179 2026-10-14T16:56:15Z Stashbot 7414 kamila@deploy1003: scap deploy-service shellbox: [PRODUCTION/DEFAULT] eqiad Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2468181 wikitext text/x-wiki qjtrxf1g2s13nevl84lyghw6tk62e3r 2468182 2468181 2026-10-14T16:56:21Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cp2044.codfw.wmnet 2468182 wikitext text/x-wiki g4ltkgp4n6uz0cj3w6znxlomh62t67a 2468183 2468182 2026-10-14T16:56:24Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts cp2044.codfw.wmnet 2468183 wikitext text/x-wiki n6iilmt7gusn0ppm8759se6y8reudrk 2468184 2468183 2026-10-14T16:56:52Z Stashbot 7414 kamila@deploy1003: Finished scap deploy-service shellbox: roll out php 8.5 on shellbox-media (duration: 03m 05s) 2468184 wikitext text/x-wiki 6xkv9mibgsx9nbx1y4bz4i2ggs6yxo6 2468185 2468184 2026-10-14T16:59:33Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2282.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468185 wikitext text/x-wiki jn991aqv3f9kd5rwdryoq4o38axqcx6 2468186 2468185 2026-10-14T16:59:50Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2282.codfw.wmnet 2468186 wikitext text/x-wiki g2jjf8l2xbd0rniggidn0xr12238bmm 2468187 2468186 2026-10-14T17:01:03Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2282.codfw.wmnet 2468187 wikitext text/x-wiki 7t2k5baizmijztwanfz53p0j5k5vwr9 2468188 2468187 2026-10-14T17:01:16Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2283.codfw.wmnet 2468188 wikitext text/x-wiki ehu2z7fk9fh7eshxgdko0j4s8jk6e90 2468189 2468188 2026-10-14T17:01:50Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2283.codfw.wmnet 2468189 wikitext text/x-wiki fieg2c9xosu0uqmyqukpdvl2uvi893k 2468190 2468189 2026-10-14T17:02:12Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2283.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468190 wikitext text/x-wiki 63dv1njyoav1949s2jtov6rgj9tok8p 2468191 2468190 2026-10-14T17:02:49Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.reboot-single for host cp2044.codfw.wmnet 2468191 wikitext text/x-wiki o1h5vf0k8nte7z0aj0gmhrobpk0ohiz 2468193 2468191 2026-10-14T17:07:31Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2283.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468193 wikitext text/x-wiki g9mpm2hjer1zjjqg4baj8xc7btk0hrd 2468194 2468193 2026-10-14T17:07:37Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2283.codfw.wmnet 2468194 wikitext text/x-wiki ckjppj1ea4q07hkuo0xo9z6n5iayrgu 2468195 2468194 2026-10-14T17:08:55Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2283.codfw.wmnet 2468195 wikitext text/x-wiki bbq5xm90peavtkvt2d40jvp3cyl7djw 2468196 2468195 2026-10-14T17:09:09Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2284.codfw.wmnet 2468196 wikitext text/x-wiki 62jp993mkcomvarjo2spdjx5os60777 2468197 2468196 2026-10-14T17:09:38Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cp2044.codfw.wmnet 2468197 wikitext text/x-wiki bx6x2x17domsdxd0r7i68fly7n4emup 2468198 2468197 2026-10-14T17:09:41Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2284.codfw.wmnet 2468198 wikitext text/x-wiki m1oc4qb74p6jllihw5704t7bra6ky6g 2468199 2468198 2026-10-14T17:09:51Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2284.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468199 wikitext text/x-wiki 0jh4ycu4yipt0t3nk5d99kaad2b8pv2 2468200 2468199 2026-10-14T17:11:31Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.provision for host cp2044.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468200 wikitext text/x-wiki 4t68qm9gyfztfevmmuqbnj8djyhyfvo 2468201 2468200 2026-10-14T17:15:07Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2284.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468201 wikitext text/x-wiki oh19zoersx1j1alhylo6zspjp3owiq3 2468202 2468201 2026-10-14T17:15:35Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2284.codfw.wmnet 2468202 wikitext text/x-wiki tmp1r2vidcqfnwvkumz7r8t6i0b4nap 2468203 2468202 2026-10-14T17:18:09Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2284.codfw.wmnet 2468203 wikitext text/x-wiki 8ubpk4tcgvju8y4z5n5e9cmto70gg4n 2468204 2468203 2026-10-14T17:18:23Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2285.codfw.wmnet 2468204 wikitext text/x-wiki 09oc73emo1o8mhrcqbajusn3k3s68t9 2468205 2468204 2026-10-14T17:18:56Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2285.codfw.wmnet 2468205 wikitext text/x-wiki s9967tmij9touatnqbvdmvfgfbhwm80 2468206 2468205 2026-10-14T17:19:16Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2285.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468206 wikitext text/x-wiki k8bd4fj78v6elozvdsz99bm3cwlbr0p 2468207 2468206 2026-10-14T17:24:33Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2285.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468207 wikitext text/x-wiki lqgdpo0exr1om6p0618s21aezmjv67z 2468208 2468207 2026-10-14T17:25:14Z Stashbot 7414 sukhe@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on cp2044.codfw.wmnet with reason: host is depooled, rebooting 2468208 wikitext text/x-wiki m0pub1of433wbmh5ao74xqan187kfpi 2468209 2468208 2026-10-14T17:27:47Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cp2044.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468209 wikitext text/x-wiki 2mdasfhxu3u1caxea8qgkbhv7s9iskf 2468210 2468209 2026-10-14T17:35:29Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2285.codfw.wmnet 2468210 wikitext text/x-wiki eykhfv0t2tx47ifzwhdrmz71h87x0kx 2468211 2468210 2026-10-14T17:35:31Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2285.codfw.wmnet 2468211 wikitext text/x-wiki 08tfahtfxn2is1it6m5euf6vbewjob9 2468212 2468211 2026-10-14T17:35:44Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2286.codfw.wmnet 2468212 wikitext text/x-wiki 96w23z1o2vbe481nymxw8vlphnxyjux 2468213 2468212 2026-10-14T17:35:50Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=yes; selector: name=cp2044.* 2468213 wikitext text/x-wiki 2jqkv7caxhsbxcc0whgjrnzks8pupe6 2468214 2468213 2026-10-14T17:36:17Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2286.codfw.wmnet 2468214 wikitext text/x-wiki b3a9a0s0bddmlfnteiod4ln90d1e0kj 2468216 2468214 2026-10-14T17:36:37Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2286.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468216 wikitext text/x-wiki 9vfu89tymg1l0mirxtsxa9vij2nwuap 2468217 2468216 2026-10-14T17:41:56Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2286.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468217 wikitext text/x-wiki ealw89zuz85fgo8rafaskz6doeaoh5p 2468218 2468217 2026-10-14T17:42:14Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2286.codfw.wmnet 2468218 wikitext text/x-wiki b80a3tl4wghy696b028my5ouq4lfape 2468219 2468218 2026-10-14T17:42:16Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2286.codfw.wmnet 2468219 wikitext text/x-wiki blysdkxu19mo1pthyyhlxp5u1872aae 2468220 2468219 2026-10-14T17:42:29Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2287.codfw.wmnet 2468220 wikitext text/x-wiki 54rh82qwpmlxhv1zb7gvycp4b6apdgi 2468221 2468220 2026-10-14T17:43:03Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2287.codfw.wmnet 2468221 wikitext text/x-wiki gqz321xcaf3odcsq99605h3fujiiva9 2468222 2468221 2026-10-14T17:43:23Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2287.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468222 wikitext text/x-wiki i823x46ns4sbglhmbck498cd9dlys9b 2468223 2468222 2026-10-14T17:48:43Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2287.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468223 wikitext text/x-wiki 3qpiqgjfhd07okuddwybf7ji6ijt4mx 2468224 2468223 2026-10-14T17:49:01Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2287.codfw.wmnet 2468224 wikitext text/x-wiki 6ywm69eyfe4hrm1l26j0eomek6rz6py 2468225 2468224 2026-10-14T17:50:13Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2287.codfw.wmnet 2468225 wikitext text/x-wiki 2qdv7i95vhp5hj9nj04rcf95leelicy 2468226 2468225 2026-10-14T17:50:27Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2288.codfw.wmnet 2468226 wikitext text/x-wiki 2cz28xa90ftsdhox77r0e0s3zgjrpcp 2468227 2468226 2026-10-14T17:51:01Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2288.codfw.wmnet 2468227 wikitext text/x-wiki 68u6wdltrvzvqbl3a7owelpms68kxis 2468228 2468227 2026-10-14T17:51:10Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2288.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468228 wikitext text/x-wiki f6w2cpr36kf7rcmi0ao8daar1d080x9 2468229 2468228 2026-10-14T17:56:29Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2288.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468229 wikitext text/x-wiki sh7ooxezd7vv2n14zhth1mdi6ese8se 2468230 2468229 2026-10-14T17:56:55Z Stashbot 7414 isaranto@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . 2468230 wikitext text/x-wiki cem8ug5633myn7nxw9xr688bvw6qn51 2468231 2468230 2026-10-14T17:56:57Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2288.codfw.wmnet 2468231 wikitext text/x-wiki aieufn8i8rldtcyic75wybpdui8x2do 2468232 2468231 2026-10-14T17:57:39Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2288.codfw.wmnet 2468232 wikitext text/x-wiki 4ssuw7a6ajbtwzc2wpfooz6rw3jep3n 2468233 2468232 2026-10-14T17:57:52Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2289.codfw.wmnet 2468233 wikitext text/x-wiki 5s4hno7692kb6ox2kikkfoeo3rj63zw 2468234 2468233 2026-10-14T17:58:26Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2289.codfw.wmnet 2468234 wikitext text/x-wiki ic3x2gwzqn71pyvkrd1xcnjwzdr8o3u 2468235 2468234 2026-10-14T17:58:46Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2289.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468235 wikitext text/x-wiki o0fu0q6g6j7shm8wqznl02xwid3sjxi 2468236 2468235 2026-10-14T18:04:06Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2289.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468236 wikitext text/x-wiki j107a48inj83x0wm4qrzl279ugk2p99 2468237 2468236 2026-10-14T18:04:12Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2289.codfw.wmnet 2468237 wikitext text/x-wiki 7kfazknh6bs8ti6tzbj7hf5fb17cbvb 2468238 2468237 2026-10-14T18:04:49Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2289.codfw.wmnet 2468238 wikitext text/x-wiki 866knkqiny3in3vw0o7rv0b7of2skmv 2468239 2468238 2026-10-14T18:05:02Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2290.codfw.wmnet 2468239 wikitext text/x-wiki 6413dgclm48jj65ropyxon5ryzbwxh7 2468240 2468239 2026-10-14T18:05:40Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2290.codfw.wmnet 2468240 wikitext text/x-wiki nfiaf7i87ejawuvyivd6myvxcob3nlh 2468241 2468240 2026-10-14T18:06:01Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2290.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468241 wikitext text/x-wiki lbkkom36l9131r2xwavwqcmjznknlyu 2468244 2468241 2026-10-14T18:11:20Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2290.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468244 wikitext text/x-wiki j90ofywosysv3tyq2bca2xmcrbge198 2468245 2468244 2026-10-14T18:11:26Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2290.codfw.wmnet 2468245 wikitext text/x-wiki kwacgp8zsqp4xmlnul2jz3r5gy5r64n 2468247 2468245 2026-10-14T18:12:38Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2290.codfw.wmnet 2468247 wikitext text/x-wiki h1aapj195ik6ji2v7c5ya5x9tqnj5dg 2468248 2468247 2026-10-14T18:12:40Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.enable_virtualization (exit_code=0) rolling provision on D{wikikube-worker[2261-2290].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2468248 wikitext text/x-wiki qz2lvgfo0wb097eb7qtoqc5iu70ilka 2468249 2468248 2026-10-14T18:19:48Z Stashbot 7414 brennen@deploy1003: rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.24 refs T438220 2468249 wikitext text/x-wiki s4aiwv4at669vfgt0zir6vpnobp4x4v 2468250 2468249 2026-10-14T18:22:49Z Stashbot 7414 brett@cumin2003: START - Cookbook sre.hosts.remove-downtime for cp2044.codfw.wmnet 2468250 wikitext text/x-wiki en6roh36cvgutxmy5va42zxf0raa8dr 2468251 2468250 2026-10-14T18:22:58Z Stashbot 7414 brett@cumin2003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cp2044.codfw.wmnet 2468251 wikitext text/x-wiki 1oconxbklrwidv4gnehhxaf7e2x7n3k 2468253 2468251 2026-10-14T18:47:33Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=no; selector: name=cp2045.* 2468253 wikitext text/x-wiki kxrm3zr6e3gd2752unhjqrvu7biu3kp 2468254 2468253 2026-10-14T18:47:57Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp2045.codfw.wmnet 2468254 wikitext text/x-wiki ahrljrthz4zv5zx55qe7dd3qb50uqqs 2468255 2468254 2026-10-14T18:49:25Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.reboot-single for host cp2045.codfw.wmnet 2468255 wikitext text/x-wiki ftwq50x8123kzk0pm0el5qxxhkkl9xb 2468256 2468255 2026-10-14T18:50:24Z Stashbot 7414 dzahn@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on codesearch2001.codfw.wmnet with reason: in setup, frontend image pending 2468256 wikitext text/x-wiki 2r2uqs73sy5mall5pd2lq2jl60xey46 2468257 2468256 2026-10-14T18:50:44Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] START helmfile.d/services/lac-lambdas: apply 2468257 wikitext text/x-wiki gc486ydbzq3nabbganynh9etnuyobro 2468258 2468257 2026-10-14T18:50:55Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] DONE helmfile.d/services/lac-lambdas: apply 2468258 wikitext text/x-wiki 99e82rv749m0hv8g7o1j5qlo1jzqtmh 2468260 2468258 2026-10-14T18:54:58Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] START helmfile.d/services/lac-lambdas: apply 2468260 wikitext text/x-wiki nl05dex13wcq4bvo0wb02abykfi4oi5 2468261 2468260 2026-10-14T18:55:08Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] DONE helmfile.d/services/lac-lambdas: apply 2468261 wikitext text/x-wiki n4ixm0lhfea2qvkr851ex0nxaqoxwwy 2468262 2468261 2026-10-14T18:55:22Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] START helmfile.d/services/lac-lambdas: apply 2468262 wikitext text/x-wiki 7d8u8ybylaq30yueabmy5wxn06ndgs9 2468263 2468262 2026-10-14T18:55:29Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] DONE helmfile.d/services/lac-lambdas: apply 2468263 wikitext text/x-wiki app8x7tsk4zr0em5fo8jfiqjp801rcv 2468264 2468263 2026-10-14T19:11:37Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cp2045.codfw.wmnet 2468264 wikitext text/x-wiki 4lio2uptj2wnpmg2bltzplaf5xbnwd2 2468265 2468264 2026-10-14T19:11:42Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts cp2045.codfw.wmnet 2468265 wikitext text/x-wiki iuu70b1by7zf8wuyh8lhfpiy9qmbab0 2468266 2468265 2026-10-14T19:12:24Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.reboot-single for host cp2045.codfw.wmnet 2468266 wikitext text/x-wiki nf3c5kld1i19fjhdtyvlhd3rl4ste2s 2468267 2468266 2026-10-14T19:14:08Z Stashbot 7414 kemayo@deploy1003: helmfile [eqiad] START helmfile.d/services/editcheck-headless: apply 2468267 wikitext text/x-wiki 4a0atxue03pojvf2nbyu96vsqm94gag 2468268 2468267 2026-10-14T19:14:46Z Stashbot 7414 kemayo@deploy1003: helmfile [eqiad] DONE helmfile.d/services/editcheck-headless: apply 2468268 wikitext text/x-wiki qqdqmjn7cjxkyqm3c0j21fhie5cxn8j 2468269 2468268 2026-10-14T19:14:56Z Stashbot 7414 kemayo@deploy1003: helmfile [codfw] START helmfile.d/services/editcheck-headless: apply 2468269 wikitext text/x-wiki io1h9qo4nhjxki1v7qr2pvj4dgsfu60 2468270 2468269 2026-10-14T19:15:34Z Stashbot 7414 kemayo@deploy1003: helmfile [codfw] DONE helmfile.d/services/editcheck-headless: apply 2468270 wikitext text/x-wiki h6vfh5z6xgtsyynr1r5l7zz85xkka7s 2468271 2468270 2026-10-14T19:16:35Z Stashbot 7414 kemayo@deploy1003: helmfile [staging] START helmfile.d/services/changeprop-jobqueue: apply 2468271 wikitext text/x-wiki 2krgd0yy1r3wyuqr1b7wzv06ze7kgak 2468272 2468271 2026-10-14T19:17:48Z Stashbot 7414 kemayo@deploy1003: helmfile [staging] DONE helmfile.d/services/changeprop-jobqueue: apply 2468272 wikitext text/x-wiki 3tqfdxemyvwvtawzj64veiuue0i6jws 2468273 2468272 2026-10-14T19:19:01Z Stashbot 7414 kemayo@deploy1003: helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply 2468273 wikitext text/x-wiki 5jivcabxjoo0ikzp930u8gnqg4pwz67 2468274 2468273 2026-10-14T19:19:13Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cp2045.codfw.wmnet 2468274 wikitext text/x-wiki chc9031wei6wy6jit6nimlcp67ovavw 2468275 2468274 2026-10-14T19:20:05Z Stashbot 7414 kemayo@deploy1003: helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply 2468275 wikitext text/x-wiki o85kv3d3fa5uw887wdolvbiuld6o4l1 2468276 2468275 2026-10-14T19:20:48Z Stashbot 7414 kemayo@deploy1003: helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: apply 2468276 wikitext text/x-wiki mbltkn1zdkh9ixwf9nyzetjz0upgim6 2468277 2468276 2026-10-14T19:21:53Z Stashbot 7414 kemayo@deploy1003: helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: apply 2468277 wikitext text/x-wiki e1axp3xrl2740nodjat4jl82lvlethb 2468278 2468277 2026-10-14T19:22:43Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.provision for host cp2045.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468278 wikitext text/x-wiki 1uim524dq71iv75vst6uggj1gr84j8z 2468285 2468278 2026-10-14T19:37:36Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cp2045.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468285 wikitext text/x-wiki ent7x76n631xrmlkpcbvswgkrcb53ec 2468287 2468285 2026-10-14T19:40:15Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.remove-downtime for cp2045.codfw.wmnet 2468287 wikitext text/x-wiki du4yacxl5p4iqx0zl5ln8odih9hanuv 2468288 2468287 2026-10-14T19:40:26Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cp2045.codfw.wmnet 2468288 wikitext text/x-wiki nkhpla570lrx5u1777tvckyt6nc98xj 2468289 2468288 2026-10-14T19:40:31Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=yes; selector: name=cp2045.* 2468289 wikitext text/x-wiki s2je9efo8hpui90w9613jttsa2w06di 2468291 2468289 2026-10-14T19:52:12Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=no; selector: name=cp2046.* 2468291 wikitext text/x-wiki 9kgqc7dm7a3ahrmv77tv5itxnovovv5 2468292 2468291 2026-10-14T19:52:38Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp2046.codfw.wmnet 2468292 wikitext text/x-wiki 9pmjlltqk5fhjiva5cp6ayx3u1zwu5k 2468293 2468292 2026-10-14T19:56:15Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.reboot-single for host cp2046.codfw.wmnet 2468293 wikitext text/x-wiki 4zw13zemzjjkbg71tqpryb250ip6d0v 2468294 2468293 2026-10-14T20:08:01Z Stashbot 7414 kemayo@deploy1003: Started scap sync-world: Backport for [[gerrit:1355168|VisualEditor: Precompute edit suggestion counts on 25% of enwiki (T433024)]], [[gerrit:1355147|ExtensionDistributor: Mark 1.47 as beta (T441077)]], [[gerrit:1352018|InitialiseSettings: Enable ModeratorToolkit on four group2 wikis (T437617)]], [[gerrit:1352019|InitialiseSettings: Enable ModeratorToolkit on enwiki (T437617)]] 2468294 wikitext text/x-wiki okf1d8ys1jiohp7sfeyviwmxqw0x3p6 2468295 2468294 2026-10-14T20:12:09Z Stashbot 7414 kemayo@deploy1003: ariel, kemayo, dhardy: Backport for [[gerrit:1355168|VisualEditor: Precompute edit suggestion counts on 25% of enwiki (T433024)]], [[gerrit:1355147|ExtensionDistributor: Mark 1.47 as beta (T441077)]], [[gerrit:1352018|InitialiseSettings: Enable ModeratorToolkit on four group2 wikis (T437617)]], [[gerrit:1352019|InitialiseSettings: Enable ModeratorToolkit on enwiki (T437617)]] synced to the testserve 2468295 wikitext text/x-wiki pzzi303pyob5obess5grbq400efvqb5 2468296 2468295 2026-10-14T20:16:56Z Stashbot 7414 kemayo@deploy1003: ariel, kemayo, dhardy: Continuing with deployment 2468296 wikitext text/x-wiki 8rkjt8pigyrcg95lvvs0ch9xo8pmjns 2468297 2468296 2026-10-14T20:17:47Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cp2046.codfw.wmnet 2468297 wikitext text/x-wiki fp9uqy2g2ry4y6ensrfwypr85oei7u4 2468298 2468297 2026-10-14T20:17:52Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts cp2046.codfw.wmnet 2468298 wikitext text/x-wiki g75gjn0znmkiz50s5z8te8982lcg7x9 2468299 2468298 2026-10-14T20:19:47Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.provision for host cp2046.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468299 wikitext text/x-wiki rhaa5jad2kkrhfh5njmsj360dcpd3or 2468300 2468299 2026-10-14T20:22:27Z Stashbot 7414 kemayo@deploy1003: Finished scap sync-world: Backport for [[gerrit:1355168|VisualEditor: Precompute edit suggestion counts on 25% of enwiki (T433024)]], [[gerrit:1355147|ExtensionDistributor: Mark 1.47 as beta (T441077)]], [[gerrit:1352018|InitialiseSettings: Enable ModeratorToolkit on four group2 wikis (T437617)]], [[gerrit:1352019|InitialiseSettings: Enable ModeratorToolkit on enwiki (T437617)]] (duration: 14m 26s) 2468300 wikitext text/x-wiki n5fdk510b1trkxujmgmstrouotc5lbc 2468301 2468300 2026-10-14T20:23:06Z Stashbot 7414 wmde-fisch@deploy1003: Started scap sync-world: Backport for [[gerrit:1355169|Check for undefined when using mainListIndex (T441193)]] (1.47.0-wmf.23) 2468301 wikitext text/x-wiki rt1d8ornw7uexxc4prqxto246y84juh 2468303 2468301 2026-10-14T20:27:23Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch: Backport for [[gerrit:1355169|Check for undefined when using mainListIndex (T441193)]] (1.47.0-wmf.23) synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2468303 wikitext text/x-wiki 5kvk3hxe6o1fqeaes8ecboq9tn2av72 2468304 2468303 2026-10-14T20:27:57Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch: Continuing with deployment 2468304 wikitext text/x-wiki r2uis397i101osutdaabg4xu94naqrm 2468305 2468304 2026-10-14T20:31:07Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cp2046.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468305 wikitext text/x-wiki tw86tl341hety4iycc4427e72ge9s5b 2468306 2468305 2026-10-14T20:32:55Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.remove-downtime for cp2046.codfw.wmnet 2468306 wikitext text/x-wiki m6vxzmltizs43bbkbvbji35g0825kso 2468307 2468306 2026-10-14T20:33:01Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cp2046.codfw.wmnet 2468307 wikitext text/x-wiki qstpmy63mbgzpp0kys1r0966tk4493k 2468308 2468307 2026-10-14T20:33:06Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=yes; selector: name=cp2046.* 2468308 wikitext text/x-wiki huge45shruv8taql9nue3z0hl2l7izz 2468309 2468308 2026-10-14T20:33:16Z Stashbot 7414 wmde-fisch@deploy1003: Finished scap sync-world: Backport for [[gerrit:1355169|Check for undefined when using mainListIndex (T441193)]] (1.47.0-wmf.23) (duration: 10m 10s) 2468309 wikitext text/x-wiki qy9myepuw5x8ev7w0mm9r9x6owz2s4i 2468310 2468309 2026-10-14T20:35:09Z Stashbot 7414 wmde-fisch@deploy1003: Started scap sync-world: Backport for [[gerrit:1355170|Check for undefined when using mainListIndex (T441193)]] (1.47.0-wmf.24) 2468310 wikitext text/x-wiki oe4g6fy461li91husep8isrtw2saqrh 2468314 2468310 2026-10-14T20:39:19Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch: Backport for [[gerrit:1355170|Check for undefined when using mainListIndex (T441193)]] (1.47.0-wmf.24) synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2468314 wikitext text/x-wiki ebdshuqvs28fhy0wswbpgrclbop0bfx 2468315 2468314 2026-10-14T20:39:48Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch: Continuing with deployment 2468315 wikitext text/x-wiki 7idlq5dc7y29jdf2ud04w5jzibamjwb 2468317 2468315 2026-10-14T20:45:08Z Stashbot 7414 wmde-fisch@deploy1003: Finished scap sync-world: Backport for [[gerrit:1355170|Check for undefined when using mainListIndex (T441193)]] (1.47.0-wmf.24) (duration: 09m 59s) 2468317 wikitext text/x-wiki gyoukpoum5hnu9jflt78ovzd02mg4mo 2468318 2468317 2026-10-14T20:55:18Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] START helmfile.d/services/lac-lambdas: apply 2468318 wikitext text/x-wiki 5s5ltzmclten3lezrx0bt0kmgl3aycq 2468319 2468318 2026-10-14T20:55:23Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] DONE helmfile.d/services/lac-lambdas: apply 2468319 wikitext text/x-wiki ivmk6mp553pxusbn7sz9arhdatlayv3 2468320 2468319 2026-10-14T20:56:24Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] START helmfile.d/services/lac-lambdas: apply 2468320 wikitext text/x-wiki fikin2l9lfkhha45o4f342limutuqve 2468321 2468320 2026-10-14T20:56:29Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] DONE helmfile.d/services/lac-lambdas: apply 2468321 wikitext text/x-wiki bs31eb8gyvwj59e7xo1nwa1hc7u3cvu 2468322 2468321 2026-10-14T20:57:21Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] START helmfile.d/services/lac-lambdas: apply 2468322 wikitext text/x-wiki 8pbp7pow83fapin63tck187kuasayuw 2468323 2468322 2026-10-14T20:57:28Z Stashbot 7414 akhatun@deploy1003: helmfile [staging] DONE helmfile.d/services/lac-lambdas: apply 2468323 wikitext text/x-wiki 5xe4zb37o9fnvyn96nmh1b2iqhh93c8 2468324 2468323 2026-10-14T21:04:29Z Stashbot 7414 jforrester@deploy1003: Started scap sync-world: Backport for [[gerrit:1355148|abstractwiki: Add four more year articles to generate]] 2468324 wikitext text/x-wiki lwt7td3jqwffti8xj7xzwq57p58v8xr 2468325 2468324 2026-10-14T21:08:50Z Stashbot 7414 jforrester@deploy1003: jforrester: Backport for [[gerrit:1355148|abstractwiki: Add four more year articles to generate]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2468325 wikitext text/x-wiki gan09veovhai2eqw1a1emvpzldyhmby 2468326 2468325 2026-10-14T21:09:18Z Stashbot 7414 jforrester@deploy1003: jforrester: Continuing with deployment 2468326 wikitext text/x-wiki fpw4cq5gwt6m9ubxntaokf4jdt5cs7v 2468327 2468326 2026-10-14T21:10:25Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1309.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468327 wikitext text/x-wiki h1tdy1twnumhp02fy4gjbdvs5nsogn8 2468328 2468327 2026-10-14T21:10:33Z Stashbot 7414 dmartin@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2468328 wikitext text/x-wiki drs0mccnc2hiil5h0ns3ngoysmt048l 2468329 2468328 2026-10-14T21:10:46Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1309.eqiad.wmnet 2468329 wikitext text/x-wiki knz2w1w9uhm5oj4ywwhq459veugxejo 2468330 2468329 2026-10-14T21:11:31Z Stashbot 7414 dmartin@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2468330 wikitext text/x-wiki l21o34feq46eczhiy1nxompichcxx12 2468331 2468330 2026-10-14T21:12:48Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1309.eqiad.wmnet 2468331 wikitext text/x-wiki 43iect1bow2mil1qjwnwnipm2zqdzo8 2468332 2468331 2026-10-14T21:12:58Z Stashbot 7414 dmartin@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2468332 wikitext text/x-wiki 2wwyi4novmazm098vu4zuvy38lm5yu1 2468333 2468332 2026-10-14T21:13:03Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1310.eqiad.wmnet 2468333 wikitext text/x-wiki 5fqhflsuz51g3b2dioqs8zov8u6totu 2468334 2468333 2026-10-14T21:13:34Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1310.eqiad.wmnet 2468334 wikitext text/x-wiki 0pwie6tjlrof2uzik94zsbatlcihf0r 2468335 2468334 2026-10-14T21:13:46Z Stashbot 7414 dmartin@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2468335 wikitext text/x-wiki o17evaj6i0zjq70553dl266xmoh3y6y 2468336 2468335 2026-10-14T21:13:54Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1310.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468336 wikitext text/x-wiki rcot02ax2qpazkv2nzomtcq9a4mi1dw 2468337 2468336 2026-10-14T21:14:23Z Stashbot 7414 dmartin@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2468337 wikitext text/x-wiki 80dgfe95pvr6tj0cj0hovad9z8w1wpd 2468338 2468337 2026-10-14T21:14:39Z Stashbot 7414 jforrester@deploy1003: Finished scap sync-world: Backport for [[gerrit:1355148|abstractwiki: Add four more year articles to generate]] (duration: 10m 10s) 2468338 wikitext text/x-wiki 23gfa08n9pcx1ehw8mjtvk6v5f3nzaf 2468339 2468338 2026-10-14T21:14:56Z Stashbot 7414 dmartin@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2468339 wikitext text/x-wiki ehehkev6sarlbqgi537dqhzjocnufpg 2468340 2468339 2026-10-14T21:18:41Z Stashbot 7414 dmartin@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2468340 wikitext text/x-wiki 0cyuxdx7kvejkao7flo2qdnip1fxrej 2468341 2468340 2026-10-14T21:19:16Z Stashbot 7414 dmartin@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2468341 wikitext text/x-wiki 4dml93o6ec534em1b0aop05wro0fayw 2468342 2468341 2026-10-14T21:20:05Z Stashbot 7414 dmartin@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2468342 wikitext text/x-wiki 5imeq4etxatexug6ik1dw72tuwyjan5 2468343 2468342 2026-10-14T21:20:45Z Stashbot 7414 dmartin@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2468343 wikitext text/x-wiki p9qiwl2how5fkd8m1lrxjie271htt8w 2468344 2468343 2026-10-14T21:21:04Z Stashbot 7414 dmartin@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2468344 wikitext text/x-wiki 4v100ia5lq5077qqsbwmexsqhniwy8p 2468345 2468344 2026-10-14T21:21:38Z Stashbot 7414 dmartin@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2468345 wikitext text/x-wiki nail8q06n4ckm4onpjuhs529hjzg3af 2468346 2468345 2026-10-14T21:26:39Z Stashbot 7414 dmartin@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2468346 wikitext text/x-wiki 01kwu65qj6y1v4c1tbxxo15i31bfjy7 2468347 2468346 2026-10-14T21:27:03Z Stashbot 7414 dmartin@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2468347 wikitext text/x-wiki ihigk18vdqio50pogj8tznutjsjdtkh 2468348 2468347 2026-10-14T21:29:10Z Stashbot 7414 dmartin@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2468348 wikitext text/x-wiki d2xnj4vzq11mnxrafpweg486yu0uv6v 2468350 2468348 2026-10-14T21:29:53Z Stashbot 7414 dmartin@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2468350 wikitext text/x-wiki 0es48autmq2sql8lygu97jmc89e4n4q 2468351 2468350 2026-10-14T21:30:13Z Stashbot 7414 dmartin@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2468351 wikitext text/x-wiki ib80btbke6jd89v6xgpeoyy7lzlaa03 2468352 2468351 2026-10-14T21:30:40Z Stashbot 7414 dmartin@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2468352 wikitext text/x-wiki fdduj2fqj94gali7sqh4tvvmkporlga 2468353 2468352 2026-10-14T21:55:09Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1355189|Prepare mlwikivoyage (T441169)]] 2468353 wikitext text/x-wiki bv0ox7e70ezwtks409kcgwcpt27ed8f 2468354 2468353 2026-10-14T21:59:20Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1355189|Prepare mlwikivoyage (T441169)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2468354 wikitext text/x-wiki l1qi9peq5m04ju0aweghysl8uw7onty 2468355 2468354 2026-10-14T22:00:01Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2468355 wikitext text/x-wiki 0754cca5z87rrgc0pegjaaxtzp7tis3 2468357 2468355 2026-10-14T22:05:45Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1355189|Prepare mlwikivoyage (T441169)]] (duration: 10m 44s) 2468357 wikitext text/x-wiki n3ait1xezavevn5nlodkx212j8uvl1z 2468358 2468357 2026-10-14T22:20:23Z Stashbot 7414 zabe: create Wikivoyage Malayalam # T441169 2468358 wikitext text/x-wiki py5w58oi099ehrujkaiibge0bbjw0dk 2468360 2468358 2026-10-14T22:27:43Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1355190|Activate mlwikivoyage (T441169)]] 2468360 wikitext text/x-wiki oao7x95n0wdd9cnkx84hmsqhujd3y41 2468362 2468360 2026-10-14T22:32:28Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1355190|Activate mlwikivoyage (T441169)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2468362 wikitext text/x-wiki 921w0e04itii74xaibamdzja89shy28 2468363 2468362 2026-10-14T22:48:11Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2468363 wikitext text/x-wiki l8h0jgjgh6roqigdrx4f5qljxwi5d70 2468366 2468363 2026-10-14T22:53:38Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1355190|Activate mlwikivoyage (T441169)]] (duration: 26m 06s) 2468366 wikitext text/x-wiki oyjjri1kbt7mpmsoa9oir1omsiy8lfg 2468367 2468366 2026-10-14T23:01:17Z Stashbot 7414 zabe@deploy1003: mwscript-k8s job started: foreachwikiindblist wikidataclient extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https # T441177 2468367 wikitext text/x-wiki 0qcsxr4kz0tjbqguqkgwi0ffgnl7uwb 2468368 2468367 2026-10-14T23:30:43Z Stashbot 7414 RoanKattouw: Deployed security patches for T441041, T441090, T441030, T437940 2468368 wikitext text/x-wiki gg0qw75qy8my3m8gyi67q2j898n3dh4 2468369 2468368 2026-10-15T00:00:04Z Stashbot 7414 root@deploy1003: helmfile [staging] START helmfile.d/services/shellbox: sync 2468369 wikitext text/x-wiki 8mafya5ji2ktos6ipu21z7vqzp7nn64 2468370 2468369 2026-10-15T00:00:26Z Stashbot 7414 root@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox: sync 2468370 wikitext text/x-wiki 6ky8j1fdzpp9tmh69pjopse86276r6w 2468372 2468370 2026-10-15T01:51:17Z Stashbot 7414 andrewbogott: live-hacking nftables max connection throttle from 32 to 16 on gitlab1004 -- T441247 2468372 wikitext text/x-wiki 08pv1y9fksunmmbmj7uaparoo2mx6dq 2468373 2468372 2026-10-15T02:00:35Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2468373 wikitext text/x-wiki fexzfrvlv89e4o2ybjutknl8dkqk0cn 2468374 2468373 2026-10-15T02:07:59Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 07m 23s) 2468374 wikitext text/x-wiki 5xmyq4938d9gfmy2dls449n9grmthx3 2468375 2468374 2026-10-15T02:39:19Z Stashbot 7414 andrewbogott: reverted live-hacking nftables max connection throttle from 32 to 16 on gitlab1004; didn't help. T441247 2468375 wikitext text/x-wiki 17nmn6h2z6kbkwidok54rwm5i549hah 2468376 2468375 2026-10-15T02:43:59Z Stashbot 7414 tstarling@deploy1003: Started scap sync-world: Backport for Fix {{#invoke:}} erroring when a function returns nil before other values (T439809) ([[gerrit:1355237|1.47.0-wmf.23]], [[gerrit:1355238|1.47.0-wmf.24]]) 2468376 wikitext text/x-wiki 6t8egbhn5armb7p5313j4tozc78dkbf 2468377 2468376 2026-10-15T02:48:24Z Stashbot 7414 tstarling@deploy1003: tstarling: Backport for Fix {{#invoke:}} erroring when a function returns nil before other values (T439809) ([[gerrit:1355237|1.47.0-wmf.23]], [[gerrit:1355238|1.47.0-wmf.24]]) synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2468377 wikitext text/x-wiki 34rwv8dassxtx763bmfpjol6clojhpa 2468378 2468377 2026-10-15T02:49:24Z Stashbot 7414 tstarling@deploy1003: tstarling: Continuing with deployment 2468378 wikitext text/x-wiki kvyprwgd9k1rtbk7wvr3wjlc54jn14o 2468379 2468378 2026-10-15T02:54:50Z Stashbot 7414 tstarling@deploy1003: Finished scap sync-world: Backport for Fix {{#invoke:}} erroring when a function returns nil before other values (T439809) ([[gerrit:1355237|1.47.0-wmf.23]], [[gerrit:1355238|1.47.0-wmf.24]]) (duration: 10m 51s) 2468379 wikitext text/x-wiki 7sx2gjod8g0pcazn2o5hiav96jt2n3v 2468380 2468379 2026-10-15T04:11:25Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.remove-downtime for wikikube-worker2192.codfw.wmnet 2468380 wikitext text/x-wiki nrep8drs6x51tiuagtu5usxfmyf8fpa 2468381 2468380 2026-10-15T04:11:46Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for wikikube-worker2192.codfw.wmnet 2468381 wikitext text/x-wiki 04d8savzv396z6bipabjrbp157abx6v 2468382 2468381 2026-10-15T04:11:51Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.remove-downtime for wikikube-worker2195.codfw.wmnet 2468382 wikitext text/x-wiki i5m4vx1gd7pdwr5fly3lvt3lv3wr6sl 2468383 2468382 2026-10-15T04:11:58Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for wikikube-worker2195.codfw.wmnet 2468383 wikitext text/x-wiki kyjjn7shmkgq9ap5tiut2afoxx0fcze 2468384 2468383 2026-10-15T04:12:05Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2192.codfw.wmnet 2468384 wikitext text/x-wiki da05966c5f422v2j2wcfck0adc2m7pp 2468385 2468384 2026-10-15T04:12:13Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2192.codfw.wmnet 2468385 wikitext text/x-wiki 7gbw6jn1gyt3fx7vzz6i85av7nxfmj7 2468386 2468385 2026-10-15T04:12:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2195.codfw.wmnet 2468386 wikitext text/x-wiki e14ar80x8f7sxvi0xvgc4ltfrs1d8ex 2468387 2468386 2026-10-15T04:12:26Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2195.codfw.wmnet 2468387 wikitext text/x-wiki mk37g070tx0ziesrlr533ksgiytf3e3 2468388 2468387 2026-10-15T04:16:00Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1310.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468388 wikitext text/x-wiki 1kajrrxg34ligz2wnxbqsqq05900ad4 2468389 2468388 2026-10-15T04:16:11Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1310.eqiad.wmnet 2468389 wikitext text/x-wiki 5jwa164b6yg3zmaa3179kkz8bnnyvtx 2468390 2468389 2026-10-15T04:16:19Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1310.eqiad.wmnet 2468390 wikitext text/x-wiki 8dqipoi70jwjxwu4ja4y3vyqylv6a6p 2468391 2468390 2026-10-15T04:16:26Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1311.eqiad.wmnet 2468391 wikitext text/x-wiki sjma8or7ei2kvkf44niry8az7g2apqp 2468392 2468391 2026-10-15T04:16:57Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1311.eqiad.wmnet 2468392 wikitext text/x-wiki b5wb605v2mayt6t778343b2t2auf5cr 2468393 2468392 2026-10-15T04:17:16Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1311.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468393 wikitext text/x-wiki nmtvkjpv7c273nuaea2khjcvq44j3vd 2468394 2468393 2026-10-15T04:27:06Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1311.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468394 wikitext text/x-wiki mlax0audrl6kzn8v23rwkt7eif9yxcj 2468395 2468394 2026-10-15T04:27:25Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1311.eqiad.wmnet 2468395 wikitext text/x-wiki 69psifjvf9ttyzehfcz78v5o81gmkx7 2468396 2468395 2026-10-15T04:29:05Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1311.eqiad.wmnet 2468396 wikitext text/x-wiki mjx449mir64feyutvrkd87nuxnn63qz 2468397 2468396 2026-10-15T04:29:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1312.eqiad.wmnet 2468397 wikitext text/x-wiki jo2ehviiiqnlyabhurzd1r1h52iv70r 2468398 2468397 2026-10-15T04:29:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1312.eqiad.wmnet 2468398 wikitext text/x-wiki oex8q86v1xqfoaykhjyggiga17mssaz 2468399 2468398 2026-10-15T04:30:02Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1312.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468399 wikitext text/x-wiki cpg3whxwts0v1derg2mdgy8yz2zmmsp 2468400 2468399 2026-10-15T04:39:52Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1312.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468400 wikitext text/x-wiki 48x7z8fc0dvwamn6eswahitulcgqsf3 2468401 2468400 2026-10-15T04:40:20Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1312.eqiad.wmnet 2468401 wikitext text/x-wiki bbrsn2rvkfd7my754jahl89hirz8cst 2468402 2468401 2026-10-15T04:42:05Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1312.eqiad.wmnet 2468402 wikitext text/x-wiki h4qzeffob7to6uti6a5ydbw5at9kfpt 2468403 2468402 2026-10-15T04:42:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1313.eqiad.wmnet 2468403 wikitext text/x-wiki ew52ma0biglajvqjzn0vfhsf94n70zx 2468404 2468403 2026-10-15T04:43:28Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1313.eqiad.wmnet 2468404 wikitext text/x-wiki ao98ufs2fl8uqnjselsipon63m4u4wn 2468405 2468404 2026-10-15T04:43:48Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1313.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468405 wikitext text/x-wiki a45nde8txvbha5fopoba1uvxl5zgv88 2468406 2468405 2026-10-15T04:52:06Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1313.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468406 wikitext text/x-wiki 4lomf1fm0fjn2oritq563bu9e5gbs0i 2468407 2468406 2026-10-15T04:52:24Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1313.eqiad.wmnet 2468407 wikitext text/x-wiki d6qt30q5bwuzbjonx22vhzvyekaebiy 2468408 2468407 2026-10-15T04:52:34Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1313.eqiad.wmnet 2468408 wikitext text/x-wiki 7cydr4bg5e817yoigl72uf0t5ql79we 2468409 2468408 2026-10-15T04:52:42Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1314.eqiad.wmnet 2468409 wikitext text/x-wiki i9we8wu0fcxq82p4c0k62g2k7dhvch9 2468410 2468409 2026-10-15T04:53:47Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1314.eqiad.wmnet 2468410 wikitext text/x-wiki q1man611ofl4lx1h645jslk4aba355a 2468411 2468410 2026-10-15T04:54:06Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1314.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468411 wikitext text/x-wiki pq9r4dn2lpug2sgjb350sq0ho1ur7jg 2468412 2468411 2026-10-15T04:55:54Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.mysql.sanitize-wiki Managing sanitization for wikis mlwikivoyage in section s5 2468412 wikitext text/x-wiki 28wvdgosw635otiow297hytcfhca9bt 2468413 2468412 2026-10-15T05:02:11Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.enable_virtualization rolling provision on D{wikikube-worker[2291-2320].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2468413 wikitext text/x-wiki p07vmrr9qa0rzhs31m1r5i7n74rx93p 2468414 2468413 2026-10-15T05:02:20Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2291.codfw.wmnet 2468414 wikitext text/x-wiki hce9jf89u8kv6tv3w0kzdlf5w2cd0aw 2468415 2468414 2026-10-15T05:02:29Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.mysql.sanitize-wiki (exit_code=0) Managing sanitization for wikis mlwikivoyage in section s5 2468415 wikitext text/x-wiki j81wt1uxn2xvun9q68fc1vknihagtsm 2468416 2468415 2026-10-15T05:03:23Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2291.codfw.wmnet 2468416 wikitext text/x-wiki 0wteshvhskf3qlii9cwk8u4fkqijodx 2468417 2468416 2026-10-15T05:03:44Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2291.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468417 wikitext text/x-wiki nv9kemixgjucdr1b4r0gin73cj2v4ht 2468418 2468417 2026-10-15T05:03:56Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1314.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468418 wikitext text/x-wiki mui6gnvlhu7vyhdmw9zilooq0u91ke0 2468419 2468418 2026-10-15T05:04:16Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1314.eqiad.wmnet 2468419 wikitext text/x-wiki 7qaemu15l5yyoymjurk2x4bgl676y1w 2468420 2468419 2026-10-15T05:04:23Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1314.eqiad.wmnet 2468420 wikitext text/x-wiki e939f5wa83utia384z0mngc9o6qikxs 2468421 2468420 2026-10-15T05:04:30Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1315.eqiad.wmnet 2468421 wikitext text/x-wiki n55owsi0prjr909rsnpnlcsmd2usat2 2468422 2468421 2026-10-15T05:05:02Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1315.eqiad.wmnet 2468422 wikitext text/x-wiki m0xrgsm3py16kv3gepxyc6l5dyon17h 2468423 2468422 2026-10-15T05:05:21Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1315.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468423 wikitext text/x-wiki e9wvqvvanzmu495asxleg6agkltwqps 2468424 2468423 2026-10-15T05:08:12Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1300.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468424 wikitext text/x-wiki kjzv82i7rvy4991xhu8x4c3jgxmbp3m 2468425 2468424 2026-10-15T05:08:59Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2291.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468425 wikitext text/x-wiki fddzcb99725htpt6zbzgngdpc64h6lc 2468426 2468425 2026-10-15T05:09:10Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2291.codfw.wmnet 2468426 wikitext text/x-wiki 1ntrhi04qbez0pa3pgnuqyu741mdpkg 2468427 2468426 2026-10-15T05:09:15Z Stashbot 7414 oblivian@cumin1004: END (ERROR) - Cookbook sre.hosts.provision (exit_code=97) for host wikikube-worker1300.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468427 wikitext text/x-wiki a506zemxfvoq3425a5a84lyv3cmavo9 2468428 2468427 2026-10-15T05:09:23Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2291.codfw.wmnet 2468428 wikitext text/x-wiki q9ktsf7gkmwb17ql2ccnexpxgfgb9eu 2468429 2468428 2026-10-15T05:09:30Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2292.codfw.wmnet 2468429 wikitext text/x-wiki t5x3x5idxkajnwq3bsqzvmx84yo84e3 2468430 2468429 2026-10-15T05:09:35Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1300.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468430 wikitext text/x-wiki 6fzfyd9g4h0bckypcpgkvhbu089u1o0 2468431 2468430 2026-10-15T05:09:41Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1300.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468431 wikitext text/x-wiki je0ohqrac3mgfrovkuvezsjnns3s45q 2468432 2468431 2026-10-15T05:10:02Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2292.codfw.wmnet 2468432 wikitext text/x-wiki 45qqicfxqo5fuww0i2yjtmqznrbcv5e 2468433 2468432 2026-10-15T05:10:23Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2292.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468433 wikitext text/x-wiki 7ywylk55g6gkbrn1d23fm8fsmearrz8 2468434 2468433 2026-10-15T05:11:00Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1300.eqiad.wmnet 2468434 wikitext text/x-wiki s1hwlrvlon3yib9lerk48oyautkacbn 2468435 2468434 2026-10-15T05:13:36Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host wikikube-worker1300.eqiad.wmnet 2468435 wikitext text/x-wiki mi0e7lij50jtr4fq5gb91dpfxioi9j2 2468436 2468435 2026-10-15T05:13:50Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1300.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468436 wikitext text/x-wiki 2or0pfszyz1tlu13goruhomkvv5oai9 2468437 2468436 2026-10-15T05:14:30Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker1300.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468437 wikitext text/x-wiki c9d7a1405u3agpxqqkeuxdbvt8bnqe6 2468438 2468437 2026-10-15T05:15:12Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1315.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468438 wikitext text/x-wiki gwlxqm2f2xptwgks1ii2bs6o1dq0gcc 2468439 2468438 2026-10-15T05:15:32Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1315.eqiad.wmnet 2468439 wikitext text/x-wiki n24hukjs3n6wc2988yimg96q3mgo3qf 2468440 2468439 2026-10-15T05:15:38Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2292.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468440 wikitext text/x-wiki dvng2pc6nifscoarxmlmp1rvzy0cvry 2468441 2468440 2026-10-15T05:17:55Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1315.eqiad.wmnet 2468441 wikitext text/x-wiki htyd49twgjuid4lxtentbwvzgndupxc 2468442 2468441 2026-10-15T05:18:09Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1316.eqiad.wmnet 2468442 wikitext text/x-wiki 3zeaskfkly48nvejguwhfkllpqkm0qh 2468443 2468442 2026-10-15T05:18:42Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1316.eqiad.wmnet 2468443 wikitext text/x-wiki 6urzhhzf7wcb6t0swmt4ne3b21t6p1o 2468444 2468443 2026-10-15T05:19:01Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1316.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468444 wikitext text/x-wiki cnnvv11qboeriszx3wgxxwpilpxpypw 2468445 2468444 2026-10-15T05:28:51Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1316.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468445 wikitext text/x-wiki e43avrc3oeop7zwr2cyj79kvj2zicj4 2468446 2468445 2026-10-15T05:29:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1316.eqiad.wmnet 2468446 wikitext text/x-wiki k7nepkbwbpgz85xjg6kvzeln16dtpsy 2468447 2468446 2026-10-15T05:29:24Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1316.eqiad.wmnet 2468447 wikitext text/x-wiki 9sz491evdu8v69jyeeswkcxtwxo5z6m 2468448 2468447 2026-10-15T05:29:34Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1317.eqiad.wmnet 2468448 wikitext text/x-wiki c0i5egxxdn5w3c3sbzx07daowa53uj6 2468449 2468448 2026-10-15T05:30:06Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1317.eqiad.wmnet 2468449 wikitext text/x-wiki 2h78io25fa3pshhf5rf7wvr76mbgszi 2468450 2468449 2026-10-15T05:30:23Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1300.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468450 wikitext text/x-wiki jcx7hp7k30joq0a8zoo1mcpm4gbjf0p 2468451 2468450 2026-10-15T05:30:30Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1317.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468451 wikitext text/x-wiki 50iqijnel9dkot4hha52itamnfehgom 2468452 2468451 2026-10-15T05:39:45Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1317.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468452 wikitext text/x-wiki 4h50dz2wb3xb7jtvavoput6xn08bmbi 2468453 2468452 2026-10-15T05:40:03Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1317.eqiad.wmnet 2468453 wikitext text/x-wiki akk5epqgykehi6t1kp5fyqxzp09eoz4 2468454 2468453 2026-10-15T05:41:29Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1300.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468454 wikitext text/x-wiki cwnwhp5pn745hdp9qdglsi71ezyenah 2468455 2468454 2026-10-15T05:42:43Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1317.eqiad.wmnet 2468455 wikitext text/x-wiki rlbok6ihy4e9g979nlmdd4vdet49cdb 2468456 2468455 2026-10-15T05:42:57Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1318.eqiad.wmnet 2468456 wikitext text/x-wiki 5zlk86gobwv7cvyx69z8ha0373d4l9o 2468457 2468456 2026-10-15T05:43:30Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1318.eqiad.wmnet 2468457 wikitext text/x-wiki b8u2u9ebmjrpz6k4yiq8jblcu9wecst 2468458 2468457 2026-10-15T05:43:50Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1318.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468458 wikitext text/x-wiki hjhhuf1am0i0un4v1tij2xh5nc2ttjb 2468459 2468458 2026-10-15T05:52:21Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.enable_virtualization (exit_code=1) rolling provision on D{wikikube-worker[2291-2320].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2468459 wikitext text/x-wiki gv76rzef00jjdqezdgwbtexddyd3icv 2468460 2468459 2026-10-15T05:53:39Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1318.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468460 wikitext text/x-wiki t8og02ed5qwtkt30ogvl62w0ontw4xv 2468461 2468460 2026-10-15T05:53:51Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1318.eqiad.wmnet 2468461 wikitext text/x-wiki q7mn7hhj8ydceuh5qt7qoxam9snbooh 2468462 2468461 2026-10-15T05:55:04Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1318.eqiad.wmnet 2468462 wikitext text/x-wiki nkurq7g5czbm3vnw43vclxgm1dfy1sj 2468463 2468462 2026-10-15T05:55:18Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1319.eqiad.wmnet 2468463 wikitext text/x-wiki sc5sha0esucgc5v6cv57oe97hg505od 2468464 2468463 2026-10-15T05:55:49Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1319.eqiad.wmnet 2468464 wikitext text/x-wiki im7gtqnwxi0lr32mwsctqjft4cxs2t2 2468465 2468464 2026-10-15T05:56:09Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1319.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468465 wikitext text/x-wiki ngblpf2r660lhmpdoltji15aowuqoic 2468466 2468465 2026-10-15T05:59:07Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.enable_virtualization rolling provision on D{wikikube-worker[2293-2320].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2468466 wikitext text/x-wiki nkva2x4520fpi293nfifggkgorvvcl7 2468467 2468466 2026-10-15T05:59:12Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2293.codfw.wmnet 2468467 wikitext text/x-wiki 5rg6iw43b8h4d53e1pwtikx0tmhfcjc 2468468 2468467 2026-10-15T05:59:44Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2293.codfw.wmnet 2468468 wikitext text/x-wiki 64esvzk07qfsoyby6wuttbmjyw4tiok 2468469 2468468 2026-10-15T06:00:05Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2293.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468469 wikitext text/x-wiki 6m79w6fn05ypgr4cmqonot1b7vbg1bw 2468470 2468469 2026-10-15T06:05:24Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2293.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468470 wikitext text/x-wiki d46ese7684j951d03zx1zsrjiq8iznn 2468471 2468470 2026-10-15T06:05:41Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2293.codfw.wmnet 2468471 wikitext text/x-wiki 912vyqputrjb3wdab5nsbxzbw48ehjp 2468472 2468471 2026-10-15T06:06:00Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1319.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468472 wikitext text/x-wiki r7i7xlmvd9tlpkdfgon2w8tc6qenbfx 2468473 2468472 2026-10-15T06:06:11Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1319.eqiad.wmnet 2468473 wikitext text/x-wiki 36vvcr42ktv8o109ccmkno1w72grx7i 2468474 2468473 2026-10-15T06:07:05Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2293.codfw.wmnet 2468474 wikitext text/x-wiki d103du5v7nyha42fmoeib64kq29jam6 2468475 2468474 2026-10-15T06:07:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2294.codfw.wmnet 2468475 wikitext text/x-wiki nlya3owwloj6ivatdvty2a7na70p0vy 2468476 2468475 2026-10-15T06:07:52Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2294.codfw.wmnet 2468476 wikitext text/x-wiki 89of4v0fqecr9ae9epy24f95ufcbvry 2468477 2468476 2026-10-15T06:08:09Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1319.eqiad.wmnet 2468477 wikitext text/x-wiki cp5zwemwvqs9fopufx9g6k7q627495j 2468478 2468477 2026-10-15T06:08:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2294.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468478 wikitext text/x-wiki gx5p7jzj4y6meeokk7jhymwzf4975hq 2468479 2468478 2026-10-15T06:08:23Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1320.eqiad.wmnet 2468479 wikitext text/x-wiki ai0jtce1bribcfji5yrstacbdpzvlsf 2468480 2468479 2026-10-15T06:08:46Z Stashbot 7414 oblivian@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 14 days, 0:00:00 on wikikube-worker2292.codfw.wmnet with reason: Hw failure 2468480 wikitext text/x-wiki 57pv0ieqlwv9x9c2q7fkkfzba0oe5u1 2468481 2468480 2026-10-15T06:08:59Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1320.eqiad.wmnet 2468481 wikitext text/x-wiki lsdxnijo6nbxh3w8k4mjpjnmsiuzix2 2468482 2468481 2026-10-15T06:09:18Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1320.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468482 wikitext text/x-wiki mbegcc28tetsvme9qyfx3q7u8tvs0k4 2468483 2468482 2026-10-15T06:13:33Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2294.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468483 wikitext text/x-wiki 4jf1wp885q3v1iqwito9yis2o109wat 2468484 2468483 2026-10-15T06:13:40Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2294.codfw.wmnet 2468484 wikitext text/x-wiki ephrphpsy441batsnykmdps6cxjwttb 2468485 2468484 2026-10-15T06:14:50Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2294.codfw.wmnet 2468485 wikitext text/x-wiki o8i163ss43qna78zde4r7lmxofps2sk 2468486 2468485 2026-10-15T06:15:04Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2295.codfw.wmnet 2468486 wikitext text/x-wiki d5vuns8flw37glp7uxezgqb4l0x4j9i 2468487 2468486 2026-10-15T06:15:38Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2295.codfw.wmnet 2468487 wikitext text/x-wiki fey9ze5lfjsw78gumn9vn8xtl7sbgqg 2468488 2468487 2026-10-15T06:15:59Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2295.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468488 wikitext text/x-wiki qjgc7c61l5dqbs5bhwlrfgrqmlx09ni 2468489 2468488 2026-10-15T06:19:08Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1320.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468489 wikitext text/x-wiki cu982n2w8flx3p6ybzzc81eq21rap7u 2468490 2468489 2026-10-15T06:19:26Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1320.eqiad.wmnet 2468490 wikitext text/x-wiki 2kaoip316kbva1vm394029bppexrzi9 2468491 2468490 2026-10-15T06:21:19Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2295.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468491 wikitext text/x-wiki k382u03qx78j819o8bhnbyo3zathol8 2468492 2468491 2026-10-15T06:21:34Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1320.eqiad.wmnet 2468492 wikitext text/x-wiki p9j0mtxqki5fl941itj2gce2rrty1w8 2468493 2468492 2026-10-15T06:21:39Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2295.codfw.wmnet 2468493 wikitext text/x-wiki e2i3a8jkteo0439ztvzub3juuwq4i11 2468494 2468493 2026-10-15T06:21:49Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1321.eqiad.wmnet 2468494 wikitext text/x-wiki 343x9jnbxyxfg0w9cvc97bzgx6m4c9u 2468495 2468494 2026-10-15T06:21:56Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2295.codfw.wmnet 2468495 wikitext text/x-wiki cplm1lijdy4zrawabg1mwemjp1lp1h3 2468496 2468495 2026-10-15T06:22:09Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2296.codfw.wmnet 2468496 wikitext text/x-wiki thzixymfe2sg6zmmygjvhpoli098c4g 2468497 2468496 2026-10-15T06:22:19Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1321.eqiad.wmnet 2468497 wikitext text/x-wiki cuj8b6as2cur9nutg3uny4tzusgmlj1 2468498 2468497 2026-10-15T06:22:39Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1321.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468498 wikitext text/x-wiki iet8jfd8ayupxff6s1jw99u9ohjmsnp 2468499 2468498 2026-10-15T06:22:46Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2296.codfw.wmnet 2468499 wikitext text/x-wiki hu8r8muojzsf4fzvo39lpvahskaaroh 2468500 2468499 2026-10-15T06:23:05Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2296.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468500 wikitext text/x-wiki 4tke0057by2xtljopy9kcljjb53jqur 2468501 2468500 2026-10-15T06:28:24Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2296.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468501 wikitext text/x-wiki sqp643spel3u7okm4605kojopwrs7p5 2468502 2468501 2026-10-15T06:28:30Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2296.codfw.wmnet 2468502 wikitext text/x-wiki 2zcjj0umwna80kz9ojk9scccizmw0dt 2468503 2468502 2026-10-15T06:30:06Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2296.codfw.wmnet 2468503 wikitext text/x-wiki iruebvy7yko9v6sucacl6t0xk2rk5st 2468504 2468503 2026-10-15T06:30:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2297.codfw.wmnet 2468504 wikitext text/x-wiki o3j2mf745xcd2qotagekkg9bsfjl3cl 2468505 2468504 2026-10-15T06:30:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2297.codfw.wmnet 2468505 wikitext text/x-wiki ielfxb4xyr6h43m99bif9xlmyep37n5 2468506 2468505 2026-10-15T06:31:13Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2297.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468506 wikitext text/x-wiki 1s6fizw0hpd3lpxl1mkadapkdr18j9k 2468507 2468506 2026-10-15T06:32:43Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1321.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468507 wikitext text/x-wiki hdtcnrys8b430e3x3iqm8j35tstyo9q 2468508 2468507 2026-10-15T06:33:02Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1321.eqiad.wmnet 2468508 wikitext text/x-wiki n6jpwtdu9kcix3zc4mj6mx0jciwa67y 2468509 2468508 2026-10-15T06:35:31Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1321.eqiad.wmnet 2468509 wikitext text/x-wiki 3n14f3hpx4kgar8sd28ex8ooy23rkgn 2468510 2468509 2026-10-15T06:35:45Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1322.eqiad.wmnet 2468510 wikitext text/x-wiki 7da89kv4kpa86wisygox3qg9bw6s94z 2468511 2468510 2026-10-15T06:36:17Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1322.eqiad.wmnet 2468511 wikitext text/x-wiki 6upjjyrxu87mny1yxhtjximeqzy0qzc 2468512 2468511 2026-10-15T06:36:31Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2297.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468512 wikitext text/x-wiki ts24x1n288mkpashfazpbv3f45qr64p 2468513 2468512 2026-10-15T06:36:36Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1322.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468513 wikitext text/x-wiki qeedo3lugoce9czu0p1919fdq0y8z0a 2468514 2468513 2026-10-15T06:36:58Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2297.codfw.wmnet 2468514 wikitext text/x-wiki b0miaqn0erbll63boc4cobo6gwl5k1x 2468515 2468514 2026-10-15T06:38:16Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2297.codfw.wmnet 2468515 wikitext text/x-wiki cofnfvtz8u4rju0g5d8624p85mmbjuh 2468516 2468515 2026-10-15T06:38:30Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2298.codfw.wmnet 2468516 wikitext text/x-wiki mj79m0a00l0q17a68z0mavbgnhzzf0g 2468517 2468516 2026-10-15T06:39:04Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2298.codfw.wmnet 2468517 wikitext text/x-wiki c9ku6dmysnphchhb9tiz92gmgmvlalr 2468518 2468517 2026-10-15T06:39:25Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2298.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468518 wikitext text/x-wiki g0ybygrqr3r9623opanc7cw64ppcyuk 2468519 2468518 2026-10-15T06:44:13Z Stashbot 7414 moritzm: installing curl security updats 2468519 wikitext text/x-wiki coekspqdrudo18k3vqjvxbpy6bynlxo 2468520 2468519 2026-10-15T06:44:44Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2298.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468520 wikitext text/x-wiki 2l5zu9wg7gyzc57p6lnll6xomvqqodh 2468521 2468520 2026-10-15T06:45:02Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2298.codfw.wmnet 2468521 wikitext text/x-wiki bl7yjnvxpbkds64h2wzenrhkum8znd7 2468522 2468521 2026-10-15T06:46:14Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2298.codfw.wmnet 2468522 wikitext text/x-wiki kcplxm4y0c1xkf9jci8ozdtasnxrs76 2468523 2468522 2026-10-15T06:46:27Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2299.codfw.wmnet 2468523 wikitext text/x-wiki t936ug5bjnoxtwyyjw4w67e8hy82cfm 2468524 2468523 2026-10-15T06:47:01Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2299.codfw.wmnet 2468524 wikitext text/x-wiki 4h4ckdz4r2ws8ik9k23lfkdd6rbogut 2468525 2468524 2026-10-15T06:47:22Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2299.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468525 wikitext text/x-wiki mmewh6s3oz6lak6ct53p7rr4lhmjfc8 2468526 2468525 2026-10-15T06:49:25Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1322.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468526 wikitext text/x-wiki bej8rtlflmu0sghrcscrs93etm25289 2468527 2468526 2026-10-15T06:49:43Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1322.eqiad.wmnet 2468527 wikitext text/x-wiki g2jy32guhuvb4xhur63ei8m056o3vpr 2468528 2468527 2026-10-15T06:50:56Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1322.eqiad.wmnet 2468528 wikitext text/x-wiki cklae77p9t54ye7ep4lc9q5n38zqe7t 2468529 2468528 2026-10-15T06:51:10Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1323.eqiad.wmnet 2468529 wikitext text/x-wiki p04ugju35ukqyiywsd4nlqmmfkvf2gi 2468530 2468529 2026-10-15T06:51:42Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1323.eqiad.wmnet 2468530 wikitext text/x-wiki nv48w5hm445wjzq43wm92cyrhavy929 2468531 2468530 2026-10-15T06:52:02Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1323.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468531 wikitext text/x-wiki j9d1m4jvhffrnyl9w1ybp1ghj0fjybi 2468532 2468531 2026-10-15T06:52:43Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2299.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468532 wikitext text/x-wiki qvvj4owbgqecke18q0v67nsigmv1swn 2468535 2468532 2026-10-15T07:05:36Z Stashbot 7414 tgr@deploy1003: Started scap sync-world: Backport for [[gerrit:1354488|Enable $wgOAuth2ClientCredentialsOwnerAuthCutover (T440873)]] 2468535 wikitext text/x-wiki 0tj4du1qtmumk4mzyoxg8n7tlqndpye 2468536 2468535 2026-10-15T07:06:44Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1323.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468536 wikitext text/x-wiki t23f3o9eyuefkmwa077y25mssc1sdas 2468537 2468536 2026-10-15T07:07:00Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2299.codfw.wmnet 2468537 wikitext text/x-wiki 6c0mpfw5i7e45rr9rx0mcx3fu2nakpj 2468538 2468537 2026-10-15T07:07:05Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1323.eqiad.wmnet 2468538 wikitext text/x-wiki maa7htwhx38i0sw2dnfyssrq9xmm6xh 2468539 2468538 2026-10-15T07:08:54Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1323.eqiad.wmnet 2468539 wikitext text/x-wiki 9bfi37it2ezrvgp2b03zyvbxcn8d729 2468540 2468539 2026-10-15T07:09:07Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1324.eqiad.wmnet 2468540 wikitext text/x-wiki ibyp2qbe55msx3du7gy49idvbu2qghn 2468541 2468540 2026-10-15T07:09:22Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2299.codfw.wmnet 2468541 wikitext text/x-wiki ak4koersckxlmh16uqapg1wp0s8diyc 2468542 2468541 2026-10-15T07:09:36Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2300.codfw.wmnet 2468542 wikitext text/x-wiki qozs5cgh0fjg5p5gs2yl88bxufzn28g 2468543 2468542 2026-10-15T07:09:46Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1324.eqiad.wmnet 2468543 wikitext text/x-wiki gam27n71tqbiepn9okkkdasswkvi29r 2468544 2468543 2026-10-15T07:09:59Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1324.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468544 wikitext text/x-wiki ndov8e3w3s5y5i93bpvnk9h0tulf76n 2468545 2468544 2026-10-15T07:10:09Z Stashbot 7414 tgr@deploy1003: tgr, coderanimeshpathak04092007: Backport for [[gerrit:1354488|Enable $wgOAuth2ClientCredentialsOwnerAuthCutover (T440873)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2468545 wikitext text/x-wiki c1nm4ena7ifi8b57wfdd6o2jy59unrm 2468546 2468545 2026-10-15T07:10:18Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2300.codfw.wmnet 2468546 wikitext text/x-wiki s9ku863a984q857atbik6elm6y76qx2 2468547 2468546 2026-10-15T07:10:30Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2300.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468547 wikitext text/x-wiki 5zjqp2jv742nejsrmdqm15kwx17895s 2468548 2468547 2026-10-15T07:13:40Z Stashbot 7414 tgr@deploy1003: tgr, coderanimeshpathak04092007: Continuing with deployment 2468548 wikitext text/x-wiki 2fwz54zf6j5c8psgcg8n5u8gq9rrh8y 2468549 2468548 2026-10-15T07:15:50Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2300.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468549 wikitext text/x-wiki 7zna6ex3hdv2ego88us6d7d8rl15x5a 2468550 2468549 2026-10-15T07:16:00Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2300.codfw.wmnet 2468550 wikitext text/x-wiki o2enekyjzh4dkhonrw0k1hg75heepht 2468551 2468550 2026-10-15T07:16:43Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2300.codfw.wmnet 2468551 wikitext text/x-wiki h8r5yqyb5n0ic9mutnth2y4knhojkfw 2468552 2468551 2026-10-15T07:16:56Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2301.codfw.wmnet 2468552 wikitext text/x-wiki ois26el14cuc8vldv8izqbcecvmjq50 2468553 2468552 2026-10-15T07:17:18Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.remove-downtime for wikikube-worker2292.codfw.wmnet 2468553 wikitext text/x-wiki 9hfo280xu6vugwj5nvalc5txzx2ntef 2468554 2468553 2026-10-15T07:17:22Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for wikikube-worker2292.codfw.wmnet 2468554 wikitext text/x-wiki ey741lfo1wgx56tx34015dwerizem5p 2468555 2468554 2026-10-15T07:17:30Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2301.codfw.wmnet 2468555 wikitext text/x-wiki aucv0o8sf1kda8kji6gyt3zm3av6w9m 2468556 2468555 2026-10-15T07:17:41Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2292.codfw.wmnet 2468556 wikitext text/x-wiki ckc8v0rj6dsfctmffz4a799k6gap8gb 2468557 2468556 2026-10-15T07:17:47Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2292.codfw.wmnet 2468557 wikitext text/x-wiki ebhvt3za157dl7ibbrhqqwd56e1evwc 2468558 2468557 2026-10-15T07:17:54Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2301.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468558 wikitext text/x-wiki 3osyf5knqaa1479sps5fnglsgpkoyyd 2468559 2468558 2026-10-15T07:19:05Z Stashbot 7414 tgr@deploy1003: Finished scap sync-world: Backport for [[gerrit:1354488|Enable $wgOAuth2ClientCredentialsOwnerAuthCutover (T440873)]] (duration: 13m 29s) 2468559 wikitext text/x-wiki q2rjomqzh0u2nmwt75wcijl9bt881kl 2468560 2468559 2026-10-15T07:20:20Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1324.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468560 wikitext text/x-wiki c8husdxippzrekdfbbct4l842t0u2jo 2468561 2468560 2026-10-15T07:20:31Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1324.eqiad.wmnet 2468561 wikitext text/x-wiki 6hhk80fdcfzfx0xwmd25ix61sikyyjy 2468562 2468561 2026-10-15T07:22:00Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1324.eqiad.wmnet 2468562 wikitext text/x-wiki 5q3xcb7q8rczgu9mze0qh3jc86olvfu 2468563 2468562 2026-10-15T07:22:14Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1325.eqiad.wmnet 2468563 wikitext text/x-wiki 62c8ds3b8lr6jhafh0ob0qmqj0a6eab 2468564 2468563 2026-10-15T07:22:45Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1325.eqiad.wmnet 2468564 wikitext text/x-wiki s3q5zj5a84lzrkjqqr77k6gxcqxkxez 2468565 2468564 2026-10-15T07:23:05Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1325.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468565 wikitext text/x-wiki pmxicu7wdgk1a5uazk3k57fpmhfui4t 2468566 2468565 2026-10-15T07:23:14Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2301.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468566 wikitext text/x-wiki ccq355ad3cizxfh0puce07s59q5goin 2468567 2468566 2026-10-15T07:23:23Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2301.codfw.wmnet 2468567 wikitext text/x-wiki py36947a9lbtj33dw2xnjbsrxgs39m3 2468568 2468567 2026-10-15T07:24:20Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2301.codfw.wmnet 2468568 wikitext text/x-wiki exak7anewvb5bgua9i9yyjjhdj2ny41 2468569 2468568 2026-10-15T07:24:33Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2302.codfw.wmnet 2468569 wikitext text/x-wiki est0tnbfqf4uu2uh1xbw9221yab9273 2468570 2468569 2026-10-15T07:25:07Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2302.codfw.wmnet 2468570 wikitext text/x-wiki 8ggj6xx8lpp0wz6e4urtjo56tzpy4in 2468571 2468570 2026-10-15T07:25:28Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2302.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468571 wikitext text/x-wiki pfro6b3vlbr6vxey19uul5dc8u0a1zm 2468572 2468571 2026-10-15T07:29:59Z Stashbot 7414 root@deploy1003: helmfile [staging] START helmfile.d/services/shellbox: sync 2468572 wikitext text/x-wiki 10gg65946ab9q6fyqs3iaijri1yxoe2 2468573 2468572 2026-10-15T07:30:20Z Stashbot 7414 root@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox: sync 2468573 wikitext text/x-wiki e75uyoxucv0eo8mt5s5ypwbl5b07yml 2468574 2468573 2026-10-15T07:30:25Z Stashbot 7414 root@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-timeline: sync 2468574 wikitext text/x-wiki j6nt3f670klfdl99wjo447eio5qkw2k 2468575 2468574 2026-10-15T07:30:36Z Stashbot 7414 root@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-timeline: sync 2468575 wikitext text/x-wiki fozjatpb681k6in4x9janx0ayj9b9l5 2468576 2468575 2026-10-15T07:30:47Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2302.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468576 wikitext text/x-wiki ongoc4n6y5lkhp5szvij6aqlkk9c875 2468577 2468576 2026-10-15T07:31:04Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2302.codfw.wmnet 2468577 wikitext text/x-wiki hy9psvrkg0y0s1xlx0fd74pppr24gku 2468578 2468577 2026-10-15T07:31:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2302.codfw.wmnet 2468578 wikitext text/x-wiki 1t9mxn0338skm353eizwucgzwggqpwi 2468579 2468578 2026-10-15T07:32:07Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2303.codfw.wmnet 2468579 wikitext text/x-wiki 9dz8r06mzy55zd0c6k201v0056ge561 2468580 2468579 2026-10-15T07:32:15Z Stashbot 7414 tgr_: UTC morning deployments done 2468580 wikitext text/x-wiki 5itbsg6saunzm8s6tn3fey86hayjmxv 2468581 2468580 2026-10-15T07:32:40Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2303.codfw.wmnet 2468581 wikitext text/x-wiki h01yqmxo9xkyk996sqajppnhs370f7m 2468582 2468581 2026-10-15T07:32:55Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1325.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468582 wikitext text/x-wiki 3mxcfuzrbs0sr7ceqfaijxpwloudh5g 2468583 2468582 2026-10-15T07:33:01Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2303.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468583 wikitext text/x-wiki 6q2l2ld58hgqezfyq5wbpkzhghcoaud 2468584 2468583 2026-10-15T07:33:13Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1325.eqiad.wmnet 2468584 wikitext text/x-wiki ay4namj0i9su7rsqw0i70nv08u609uj 2468585 2468584 2026-10-15T07:34:48Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1325.eqiad.wmnet 2468585 wikitext text/x-wiki 3tqsjtuf4yxzp6qw9700bgdki10ry6a 2468586 2468585 2026-10-15T07:35:01Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1326.eqiad.wmnet 2468586 wikitext text/x-wiki 6o2h2aqsc5hiqi5typ84hlimb84deyz 2468587 2468586 2026-10-15T07:35:33Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1326.eqiad.wmnet 2468587 wikitext text/x-wiki 4ybjpqwd5au3zh1ujz7nlxc6c1raoae 2468588 2468587 2026-10-15T07:35:42Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1326.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468588 wikitext text/x-wiki jm5r96v6ncfuu9jq1j4e6u830gn5u06 2468589 2468588 2026-10-15T07:38:18Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2303.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468589 wikitext text/x-wiki e80nvv77438ykxlq99k331r3yihvuwt 2468590 2468589 2026-10-15T07:38:24Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2303.codfw.wmnet 2468590 wikitext text/x-wiki 7056lt47cpz98zmo1b0j84jvp4yp1az 2468591 2468590 2026-10-15T07:39:48Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2303.codfw.wmnet 2468591 wikitext text/x-wiki bvpr30a6pxltrfng8h8rz72gs3aacik 2468592 2468591 2026-10-15T07:40:01Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2304.codfw.wmnet 2468592 wikitext text/x-wiki ith4hc5ex2uhsksmuodtps77qmm1kss 2468593 2468592 2026-10-15T07:40:36Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2304.codfw.wmnet 2468593 wikitext text/x-wiki 8mvvdf8gl07n485m4tebljbu6eorngb 2468594 2468593 2026-10-15T07:40:56Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2304.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468594 wikitext text/x-wiki 2zx3cfedhifqy3hqv1s9wuqrjfhm7xk 2468595 2468594 2026-10-15T07:45:32Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1326.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468595 wikitext text/x-wiki ehghpcacgpdeur3n25w2qp5s02ibz9q 2468596 2468595 2026-10-15T07:45:41Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1326.eqiad.wmnet 2468596 wikitext text/x-wiki 3zqlu3zw4hryhlat5l9wnb1w0gcn439 2468597 2468596 2026-10-15T07:46:01Z Stashbot 7414 moritzm: installing chromium security updates 2468597 wikitext text/x-wiki sp1k7pg2dfmvgfkxne2yxra75a8544s 2468598 2468597 2026-10-15T07:46:12Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2304.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468598 wikitext text/x-wiki 2cyt71skcqrta8wdwh3oax243xer09y 2468599 2468598 2026-10-15T07:46:18Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2304.codfw.wmnet 2468599 wikitext text/x-wiki 1qgldthaj6gom4z0pxiga7t4hwk9j6m 2468600 2468599 2026-10-15T07:47:12Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1326.eqiad.wmnet 2468600 wikitext text/x-wiki rrnlxq5akespxueoy43yj18zt68evx7 2468601 2468600 2026-10-15T07:47:26Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1327.eqiad.wmnet 2468601 wikitext text/x-wiki g9xua38oq6jv08ajrfpiyfbfx5kacne 2468602 2468601 2026-10-15T07:47:36Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2304.codfw.wmnet 2468602 wikitext text/x-wiki m630140g34s5u1kia9fbg5khajzh2oq 2468603 2468602 2026-10-15T07:47:50Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2305.codfw.wmnet 2468603 wikitext text/x-wiki 4etqil7jqzpuymr0e7d02p68ttkcd2w 2468604 2468603 2026-10-15T07:47:58Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1327.eqiad.wmnet 2468604 wikitext text/x-wiki tk87rp1np3zvcg10ed6l3gk7nxxl42e 2468605 2468604 2026-10-15T07:48:18Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1327.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468605 wikitext text/x-wiki hqpo9aks4k6y2h9slsi7iabdyjviz9z 2468606 2468605 2026-10-15T07:48:27Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2305.codfw.wmnet 2468606 wikitext text/x-wiki hj7cf7tq92xqa63fidmnh2ydy41na1x 2468607 2468606 2026-10-15T07:48:32Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2305.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468607 wikitext text/x-wiki 3rfl3braifjnohe7lphb7ev5mlxoau9 2468608 2468607 2026-10-15T07:52:15Z Stashbot 7414 logmsgbot: mszwarc Deployed security patch for T440126 2468608 wikitext text/x-wiki hscfmtbfi2h0d9bpofiy88e04ivn99k 2468609 2468608 2026-10-15T07:53:50Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2305.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468609 wikitext text/x-wiki g5y7aex2wjai5u6xs22slwpodfktdvf 2468610 2468609 2026-10-15T07:53:59Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2305.codfw.wmnet 2468610 wikitext text/x-wiki de6jkz1tdzi14ssvy89lq15bxs70442 2468611 2468610 2026-10-15T07:55:25Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2305.codfw.wmnet 2468611 wikitext text/x-wiki 2xltn5h2etk8izstqsqensr67pcye8w 2468612 2468611 2026-10-15T07:55:39Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2306.codfw.wmnet 2468612 wikitext text/x-wiki ab8cyscun0lkhqjgdyj8qlu3w04c5a0 2468613 2468612 2026-10-15T07:56:13Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2306.codfw.wmnet 2468613 wikitext text/x-wiki sjhjx3xkeozbqlj0zx26rc7bji7b8iq 2468614 2468613 2026-10-15T07:56:34Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2306.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468614 wikitext text/x-wiki f2x16yod55s03xiazzgznhki9hldf1s 2468615 2468614 2026-10-15T07:58:08Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1327.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2468615 wikitext text/x-wiki j7rb50152mvzbkwlrtjukn2hifmsvpe 2468616 2468615 2026-10-15T07:58:27Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1327.eqiad.wmnet 2468616 wikitext text/x-wiki e2e6a0z16da2hw0swuho56x67335znl 2468617 2468616 2026-10-15T07:58:32Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1327.eqiad.wmnet 2468617 wikitext text/x-wiki 92eurmuuhg6vs4hawealyn601fs3riv 2468618 2468617 2026-10-15T07:58:42Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1328.eqiad.wmnet 2468618 wikitext text/x-wiki n7r0txusi2ukk0oep1jijdrx56m5mp9 2468619 2468618 2026-10-15T07:59:15Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1328.eqiad.wmnet 2468619 wikitext text/x-wiki nscqz2984u2njvcrn6kie596vzu58m7 2468620 2468619 2026-10-15T07:59:35Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1328.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2468620 wikitext text/x-wiki d22jaq87dyudf81dyjqw07mfjn1nj2c 2468621 2468620 2026-10-15T08:00:52Z Stashbot 7414 isaranto@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . 2468621 wikitext text/x-wiki 2y8039jwam511m5ctfalkq6xwut2dha 2468622 2468621 2026-10-15T08:01:47Z Stashbot 7414 logmsgbot: mszwarc Deployed security patch for T440126 2468622 wikitext text/x-wiki 4mcz38z81jof2xiltwwp8uxwgvub1wb 2468623 2468622 2026-10-15T08:01:55Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2306.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468623 wikitext text/x-wiki 61rzagksqaqnjzz7uxuqlj788ek8cu5 2468624 2468623 2026-10-15T08:02:20Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2306.codfw.wmnet 2468624 wikitext text/x-wiki mi1s7z8z9mkkswyemykukvwaxql7v57 2468625 2468624 2026-10-15T08:02:29Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2306.codfw.wmnet 2468625 wikitext text/x-wiki jkgmahe4d1c3pieyc64hf0fel76zm0d 2468626 2468625 2026-10-15T08:02:41Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2307.codfw.wmnet 2468626 wikitext text/x-wiki 0godz1q6d3z0gzqxk0k9ycn9hk6ddmg 2468627 2468626 2026-10-15T08:03:15Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2307.codfw.wmnet 2468627 wikitext text/x-wiki ngm5ivxdic1ut3g9q66szfdnmhkmlwa 2468628 2468627 2026-10-15T08:03:38Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2307.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468628 wikitext text/x-wiki t6zwgg6ruqy89u3u4n0dx08iiwex2n5 2468629 2468628 2026-10-15T08:06:14Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1328.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2468629 wikitext text/x-wiki aeylgx8ar91zu8aupf47mce30i37psu 2468630 2468629 2026-10-15T08:07:52Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1328.eqiad.wmnet 2468630 wikitext text/x-wiki 90qgbfrntap62qxnsoh27outukz4ldb 2468631 2468630 2026-10-15T08:08:01Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1328.eqiad.wmnet 2468631 wikitext text/x-wiki pk58fl8b55sojq6ihk37x7hvb41fbwv 2468632 2468631 2026-10-15T08:08:10Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1329.eqiad.wmnet 2468632 wikitext text/x-wiki fztzrmt5grl2oj3apv4v1wry8tmnzru 2468633 2468632 2026-10-15T08:08:40Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1329.eqiad.wmnet 2468633 wikitext text/x-wiki onzw8h1vbx4qcd3fmejevzpnqf7zwjd 2468634 2468633 2026-10-15T08:08:56Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2307.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468634 wikitext text/x-wiki 2xgjav0t0o113wve0nn5d6qhwny9yxj 2468635 2468634 2026-10-15T08:09:04Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1329.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2468635 wikitext text/x-wiki jpwd3mocw9zpwkfz90o9u1po0squ90o 2468636 2468635 2026-10-15T08:09:10Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2307.codfw.wmnet 2468636 wikitext text/x-wiki amy1tdmsxo7cym99d8pvmfzjr8kt237 2468637 2468636 2026-10-15T08:09:39Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2307.codfw.wmnet 2468637 wikitext text/x-wiki 7fatodj6npllwacs7xorcmy7479jtj1 2468638 2468637 2026-10-15T08:09:53Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2308.codfw.wmnet 2468638 wikitext text/x-wiki 6oynucw5fozh2rtwujw7oy5wu8wbcq3 2468639 2468638 2026-10-15T08:11:02Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2308.codfw.wmnet 2468639 wikitext text/x-wiki jexv1fxnvgbiwkilq9frsvtc2mnovpz 2468640 2468639 2026-10-15T08:11:23Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2308.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468640 wikitext text/x-wiki ftdfqv918h392ur1352efe2vltjixb3 2468641 2468640 2026-10-15T08:13:17Z Stashbot 7414 hashar@deploy1003: rebuilt and synchronized wikiversions files: group2 to 1.47.0-wmf.24 refs T438220 2468641 wikitext text/x-wiki dr6dj6j6h7m03ztbed396my5uucm5hh 2468642 2468641 2026-10-15T08:15:35Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1329.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2468642 wikitext text/x-wiki ienavb61wvxk8pqzrwd2t51he90t978 2468643 2468642 2026-10-15T08:16:41Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2308.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468643 wikitext text/x-wiki 7xkf53qnycg283f2vd6uc0gdumb8xm1 2468644 2468643 2026-10-15T08:16:49Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2308.codfw.wmnet 2468644 wikitext text/x-wiki 6sxuge53osyrmdqsqiewbjedlj8fbxd 2468645 2468644 2026-10-15T08:16:56Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1329.eqiad.wmnet 2468645 wikitext text/x-wiki 0x3rqrg6btxq0jzug3bs7yw5t8q6xzg 2468646 2468645 2026-10-15T08:17:02Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1329.eqiad.wmnet 2468646 wikitext text/x-wiki 72mkgcy8goove8jkm3ehh8x028md1mn 2468647 2468646 2026-10-15T08:17:09Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.enable_virtualization (exit_code=0) rolling provision on D{wikikube-worker[1303-1330].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) 2468647 wikitext text/x-wiki oggqzqsmfyut17h8w8f9mcqu88vqwuc 2468648 2468647 2026-10-15T08:17:36Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2308.codfw.wmnet 2468648 wikitext text/x-wiki qu0p4wclj01zin7777oh0d1pyirepzm 2468649 2468648 2026-10-15T08:17:49Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2309.codfw.wmnet 2468649 wikitext text/x-wiki svw7n7oxtjztlrv96dkxj9qn7elt82d 2468650 2468649 2026-10-15T08:18:24Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2309.codfw.wmnet 2468650 wikitext text/x-wiki o952kk1jourk4c6i11sjczsi2blzy18 2468651 2468650 2026-10-15T08:18:44Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2309.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468651 wikitext text/x-wiki dvh6jift0rcg796w2jofa35585ug5qw 2468652 2468651 2026-10-15T08:24:00Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2309.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468652 wikitext text/x-wiki pr72hc2rkm5o0sy2tpv5j1rqxde3qvj 2468653 2468652 2026-10-15T08:24:11Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2309.codfw.wmnet 2468653 wikitext text/x-wiki abvn5m8ns2lmddirburjba59bqmrwfo 2468654 2468653 2026-10-15T08:24:16Z Stashbot 7414 oblivian@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/aux-k8s-services/jaeger: sync 2468654 wikitext text/x-wiki g7dt4bj9k89j5bt7xjfzuulbhxyql8r 2468655 2468654 2026-10-15T08:24:29Z Stashbot 7414 oblivian@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/aux-k8s-services/jaeger: sync 2468655 wikitext text/x-wiki pfy6rvv4rb3dcr4m65hcsjzvusdjblx 2468656 2468655 2026-10-15T08:24:39Z Stashbot 7414 oblivian@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/aux-k8s-services/jaeger: sync 2468656 wikitext text/x-wiki leu138agrro4lnwr3vkzzkkh9d8n7te 2468657 2468656 2026-10-15T08:24:45Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2468657 wikitext text/x-wiki 3jpx68z665j7w4fxjj5byijxnmxsjrm 2468658 2468657 2026-10-15T08:24:54Z Stashbot 7414 oblivian@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/aux-k8s-services/jaeger: sync 2468658 wikitext text/x-wiki de7trboaqmcum8sefbkg3yfv5pysks8 2468659 2468658 2026-10-15T08:25:04Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2468659 wikitext text/x-wiki cq4cgk88wwjexj9c3nmrkdue5p3kkpv 2468660 2468659 2026-10-15T08:25:18Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2468660 wikitext text/x-wiki mwzato4tjv619z00iabhoe767oji2yu 2468661 2468660 2026-10-15T08:25:24Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2309.codfw.wmnet 2468661 wikitext text/x-wiki 81shmgr44cqhw8drcpqve43c1nu15fq 2468662 2468661 2026-10-15T08:25:38Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2310.codfw.wmnet 2468662 wikitext text/x-wiki e7phqe1rid2j2pgt2o193r2nc4qekye 2468663 2468662 2026-10-15T08:25:52Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2468663 wikitext text/x-wiki ctqjsiwy8f203n36glb4o1lvvc7883e 2468664 2468663 2026-10-15T08:26:13Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2310.codfw.wmnet 2468664 wikitext text/x-wiki s2q6ujulxqm66u9nnw75zcj2r7cgraf 2468665 2468664 2026-10-15T08:26:23Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2310.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468665 wikitext text/x-wiki 0fnn37vsz5romwgbr2jiuafvejxviwu 2468667 2468665 2026-10-15T08:29:08Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] START helmfile.d/services/thumbor: apply 2468667 wikitext text/x-wiki gzj0lrim2op6raskc3dz30n80d7tyw6 2468668 2468667 2026-10-15T08:31:16Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] DONE helmfile.d/services/thumbor: apply 2468668 wikitext text/x-wiki rdqcbvhrfb09374z49ls86vtwus88u7 2468669 2468668 2026-10-15T08:31:39Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2310.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468669 wikitext text/x-wiki sit4x8qdf0ld1sbtk467rth821mzfly 2468670 2468669 2026-10-15T08:31:49Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2310.codfw.wmnet 2468670 wikitext text/x-wiki p1oscltoooatw4ia0ez0a0aqh9g3fpj 2468671 2468670 2026-10-15T08:33:08Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2310.codfw.wmnet 2468671 wikitext text/x-wiki 6reqe3ntck4yvio0u25j9ssq93hmckn 2468672 2468671 2026-10-15T08:33:21Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2311.codfw.wmnet 2468672 wikitext text/x-wiki lk3r2hd1f8csnik6gkeo525ij4pu3zw 2468673 2468672 2026-10-15T08:33:55Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2311.codfw.wmnet 2468673 wikitext text/x-wiki cyskzff65ke2yup2gs1uoy397cwxzbi 2468674 2468673 2026-10-15T08:34:15Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2311.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468674 wikitext text/x-wiki ktf4ow43a9ct3pwfc9t0f5p8a3aw7os 2468675 2468674 2026-10-15T08:39:33Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2311.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468675 wikitext text/x-wiki ajvujq7lgbm7je0p4hi0qywe8ta1qxv 2468676 2468675 2026-10-15T08:39:51Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2311.codfw.wmnet 2468676 wikitext text/x-wiki py65bw8q56bsxvdk1qft4g9zoujzl8t 2468677 2468676 2026-10-15T08:40:17Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] START helmfile.d/services/thumbor: apply 2468677 wikitext text/x-wiki phvr8w4uj9ssa45gb1fl2yxvcy1ijpn 2468678 2468677 2026-10-15T08:40:26Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2311.codfw.wmnet 2468678 wikitext text/x-wiki jvs7yw7ys2cfoou77cgyjk8bcbp9yhm 2468679 2468678 2026-10-15T08:40:36Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2312.codfw.wmnet 2468679 wikitext text/x-wiki qgkcljmqei48n1pdoc9hw6ek6zu7850 2468680 2468679 2026-10-15T08:41:28Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] DONE helmfile.d/services/thumbor: apply 2468680 wikitext text/x-wiki is1ksv5vs3khua8t9odv5b51pt8kivp 2468681 2468680 2026-10-15T08:41:46Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2312.codfw.wmnet 2468681 wikitext text/x-wiki 2a36u3v8nwlcsp6xjaga1ev2lrc43bv 2468682 2468681 2026-10-15T08:42:05Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2312.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468682 wikitext text/x-wiki 3p9knb0y4isy5awxx9sf0vcc7kyt8qz 2468684 2468682 2026-10-15T08:47:25Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2312.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468684 wikitext text/x-wiki 2ew3rohxic1cdpta9qgjztszo8qiz0i 2468685 2468684 2026-10-15T08:47:42Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2312.codfw.wmnet 2468685 wikitext text/x-wiki 5mjivi0da9eb02ly6vp2k57ql7p6eti 2468686 2468685 2026-10-15T08:49:29Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2312.codfw.wmnet 2468686 wikitext text/x-wiki a11n9mkq89fiytnohfkmbq99i6ta6ah 2468687 2468686 2026-10-15T08:49:43Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2313.codfw.wmnet 2468687 wikitext text/x-wiki 0cybr29jahez29rbzubzzysb1v971nt 2468688 2468687 2026-10-15T08:50:18Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2313.codfw.wmnet 2468688 wikitext text/x-wiki nsv9x6ffmsrul7xw4fhiqqw5xyrlupl 2468690 2468688 2026-10-15T08:50:39Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2313.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468690 wikitext text/x-wiki 7v0b4jtra4g50lja5pf6wz0j4qv3zbv 2468692 2468690 2026-10-15T08:55:55Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2313.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468692 wikitext text/x-wiki bd7sz8onwj5274p0fej0qrwpoplllst 2468693 2468692 2026-10-15T08:56:12Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2313.codfw.wmnet 2468693 wikitext text/x-wiki dazatijv7vo79xve285jw3tqsfey801 2468694 2468693 2026-10-15T08:58:00Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2313.codfw.wmnet 2468694 wikitext text/x-wiki o07j0mqj59u652kmr5n86webgx0ptfk 2468695 2468694 2026-10-15T08:58:13Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2314.codfw.wmnet 2468695 wikitext text/x-wiki 75asw9zqmytqi853jnl540ricg82yo9 2468696 2468695 2026-10-15T08:58:47Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2314.codfw.wmnet 2468696 wikitext text/x-wiki odwynxpqgz1zpcxd219s1yuqinde0az 2468697 2468696 2026-10-15T08:59:07Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2314.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468697 wikitext text/x-wiki 7sv7leo4tjl3bqtud757qp9qwek4utv 2468698 2468697 2026-10-15T09:03:38Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2468698 wikitext text/x-wiki 25y9t7w7hfo7apkjo8s0vh1s8fq3yn4 2468699 2468698 2026-10-15T09:04:27Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2314.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468699 wikitext text/x-wiki 9w1219iy3ass7ipbibttl8pkfxvldk0 2468700 2468699 2026-10-15T09:04:34Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2314.codfw.wmnet 2468700 wikitext text/x-wiki eztxauort4ii3s22jxvx9pvliqsuvgy 2468701 2468700 2026-10-15T09:05:33Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2468701 wikitext text/x-wiki 33lg0nxzhlmcztl1wcno4jwmlyaunrf 2468702 2468701 2026-10-15T09:06:32Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2314.codfw.wmnet 2468702 wikitext text/x-wiki g935wqw309nkdmoqwlcpzl6x7k8aicc 2468703 2468702 2026-10-15T09:06:46Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2315.codfw.wmnet 2468703 wikitext text/x-wiki lg1ghucw09mhmrmz3smxbcyklahzwcs 2468704 2468703 2026-10-15T09:06:57Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2468704 wikitext text/x-wiki ff9n2nze0vvd7m09vluetpg42pwrinz 2468705 2468704 2026-10-15T09:07:21Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2315.codfw.wmnet 2468705 wikitext text/x-wiki it0f9t3kxqx1101m9vrv613lc6sk8l8 2468706 2468705 2026-10-15T09:07:41Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2315.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468706 wikitext text/x-wiki lt17ojb3tvvf943p7nnvipyhwwstqq3 2468707 2468706 2026-10-15T09:10:12Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2468707 wikitext text/x-wiki owhgmwuwdeehla5kxzxy3ao26oi7efn 2468708 2468707 2026-10-15T09:13:00Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2315.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468708 wikitext text/x-wiki 2ffol43cgmljcrwhr2y7j4tb1kuo5qf 2468709 2468708 2026-10-15T09:13:06Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2315.codfw.wmnet 2468709 wikitext text/x-wiki qlt7lzjklbx54oqz8ijcxn8qjdddmd0 2468710 2468709 2026-10-15T09:14:30Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2315.codfw.wmnet 2468710 wikitext text/x-wiki 36ie2j6v3zg4hk8p2qpgq66xfskld2b 2468711 2468710 2026-10-15T09:14:43Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2316.codfw.wmnet 2468711 wikitext text/x-wiki 5hu2bcxurz0hxa9w0s4y8v5ewo5u8kw 2468712 2468711 2026-10-15T09:15:16Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2316.codfw.wmnet 2468712 wikitext text/x-wiki 5kjlybrnxiaki5la40ummzclbb1sci5 2468713 2468712 2026-10-15T09:15:36Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2316.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468713 wikitext text/x-wiki j1hagmto6lhtkn01ery5zbqx0cc54j9 2468714 2468713 2026-10-15T09:19:28Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/growthbook: apply 2468714 wikitext text/x-wiki px9n7s9juumcj1kdvtzydwyq2g12vgi 2468715 2468714 2026-10-15T09:19:59Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/growthbook: apply 2468715 wikitext text/x-wiki 4niqaamoskyb5jkxk3i2tup4wt5t2m4 2468716 2468715 2026-10-15T09:20:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2316.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2468716 wikitext text/x-wiki 23plqgpzwte6x3s7j7j8uubfixjebuc 2468718 2468716 2026-10-15T09:25:12Z Stashbot 7414 hashar@deploy1003: Started scap sync-world: Backport for [[gerrit:1354793|Discord: Pass array options to ParserOutputAccess::getParserOutput (T440951)]] (1.47.0-wmf.24) 2468718 wikitext text/x-wiki tlxqjpmfp4u4v6vsrsat55dbf6hrhg2 2468720 2468718 2026-10-15T09:28:12Z Stashbot 7414 moritzm: installing expat security updates 2468720 wikitext text/x-wiki 40ztzp96bqovositox3bdwuwlc86l92 2468721 2468720 2026-10-15T09:29:09Z Stashbot 7414 hashar@deploy1003: jforrester, hashar: Backport for [[gerrit:1354793|Discord: Pass array options to ParserOutputAccess::getParserOutput (T440951)]] (1.47.0-wmf.24) synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2468721 wikitext text/x-wiki fqhhenl7wpcj41hixt94ucsuapwwk2c 2468722 2468721 2026-10-15T09:29:40Z Stashbot 7414 hashar@deploy1003: jforrester, hashar: Continuing with deployment 2468722 wikitext text/x-wiki rt4jam21n1x9mhy8zqu5m1g6dt7nj8x 2468723 2468722 2026-10-15T09:35:03Z Stashbot 7414 hashar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1354793|Discord: Pass array options to ParserOutputAccess::getParserOutput (T440951)]] (1.47.0-wmf.24) (duration: 09m 50s) 2468723 wikitext text/x-wiki bcozeopcyzfiyu57gswwhdrfzlkoc0v 2468725 2468723 2026-10-15T09:40:39Z Stashbot 7414 root@cumin2003: START - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors rolling restart_daemons on A:logstash-collector 2468725 wikitext text/x-wiki 4qfqcgps4mtg4uudyqt1z6cmzlng34g 2468730 2468725 2026-10-15T09:48:36Z Stashbot 7414 root@cumin2003: END (PASS) - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors (exit_code=0) rolling restart_daemons on A:logstash-collector 2468730 wikitext text/x-wiki bwkkr7bz59w1fxwyxkjamm6xom48dvr 2468741 2468730 2026-10-15T10:03:10Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2468741 wikitext text/x-wiki pe1d75ygo7c8f401r3gkohfvslxfe34 2468742 2468741 2026-10-15T10:04:14Z Stashbot 7414 marostegui: deploy schema change on s8 with replication T441197 2468742 wikitext text/x-wiki 4lnwel0ali4zusw0o1krqjuj64g6z88 2468745 2468742 2026-10-15T10:08:29Z Stashbot 7414 marostegui@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on db2158.codfw.wmnet with reason: Maintenance 2468745 wikitext text/x-wiki g6nwb4f50n71uvdp3m13ipk2plocg9d 2468746 2468745 2026-10-15T10:08:41Z Stashbot 7414 marostegui@cumin1004: dbctl commit (dc=all): 'Depooling db2158 (T441197)', diff saved to https://phabricator.wikimedia.org/P96684 and previous config saved to /var/cache/conftool/dbconfig/20261015-100829-marostegui.json 2468746 wikitext text/x-wiki tfdi7dcof4a29wojfltbrhjg92n55b4 2468748 2468746 2026-10-15T10:10:40Z Stashbot 7414 marostegui@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db2158 (T441197)', diff saved to https://phabricator.wikimedia.org/P96685 and previous config saved to /var/cache/conftool/dbconfig/20261015-101037-marostegui.json 2468748 wikitext text/x-wiki bi2z070oz7uehmf9qgtt2wyz56bb4h9 2468749 2468748 2026-10-15T10:10:50Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply 2468749 wikitext text/x-wiki hno0u3l8opl1mllb0dnbi2n2uaseuzd 2468750 2468749 2026-10-15T10:11:17Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2468750 wikitext text/x-wiki 2mtwk8xltsrpwuqknask90v2zljki0g 2468753 2468750 2026-10-15T10:23:13Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply 2468753 wikitext text/x-wiki lcwlqmtetpm6a3pdxq3crchhunog79x 2468754 2468753 2026-10-15T10:25:41Z Stashbot 7414 marostegui@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db2158', diff saved to https://phabricator.wikimedia.org/P96686 and previous config saved to /var/cache/conftool/dbconfig/20261015-102539-marostegui.json 2468754 wikitext text/x-wiki eobzugrabrfluh96yw3ia7hfr4burah 2468755 2468754 2026-10-15T10:40:42Z Stashbot 7414 marostegui@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db2158', diff saved to https://phabricator.wikimedia.org/P96687 and previous config saved to /var/cache/conftool/dbconfig/20261015-104041-marostegui.json 2468755 wikitext text/x-wiki iqrskw30gko9ni8op0w5i1kxl4c64or 2468757 2468755 2026-10-15T10:55:46Z Stashbot 7414 marostegui@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db2158 (T441197)', diff saved to https://phabricator.wikimedia.org/P96688 and previous config saved to /var/cache/conftool/dbconfig/20261015-105543-marostegui.json 2468757 wikitext text/x-wiki othrf1wf3fzrih0s87vfnjak8zgzyli 2468758 2468757 2026-10-15T10:56:00Z Stashbot 7414 marostegui@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on db2180.codfw.wmnet with reason: Maintenance 2468758 wikitext text/x-wiki 0boy0q4s0q5f2c5tz7vvo0d9r6db592 2468759 2468758 2026-10-15T10:56:08Z Stashbot 7414 marostegui@cumin1004: dbctl commit (dc=all): 'Depooling db2180 (T441197)', diff saved to https://phabricator.wikimedia.org/P96689 and previous config saved to /var/cache/conftool/dbconfig/20261015-105601-marostegui.json 2468759 wikitext text/x-wiki tnarwwo8go7kgecjy0v8yafxozuxw8e 2468760 2468759 2026-10-15T10:58:14Z Stashbot 7414 marostegui@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db2180 (T441197)', diff saved to https://phabricator.wikimedia.org/P96690 and previous config saved to /var/cache/conftool/dbconfig/20261015-105811-marostegui.json 2468760 wikitext text/x-wiki eorrqunxpmpup5zmdm5dh4kk21u0ziw 2468762 2468760 2026-10-15T11:07:59Z Stashbot 7414 root@cumin2003: START - Cookbook sre.misc-clusters.roll-restart-reboot-eventschemas rolling restart_daemons on A:schema 2468762 wikitext text/x-wiki 6qslq7hj8pbb0adqyqg37g61eeiksln 2468763 2468762 2026-10-15T11:10:12Z Stashbot 7414 root@cumin2003: END (PASS) - Cookbook sre.misc-clusters.roll-restart-reboot-eventschemas (exit_code=0) rolling restart_daemons on A:schema 2468763 wikitext text/x-wiki 2621b0uz4gakvhy2mc1x1nzzsjqm0f1 2468764 2468763 2026-10-15T11:13:14Z Stashbot 7414 marostegui@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db2180', diff saved to https://phabricator.wikimedia.org/P96691 and previous config saved to /var/cache/conftool/dbconfig/20261015-111313-marostegui.json 2468764 wikitext text/x-wiki 8kpplozuzwtdq6acr2utmhqlf9vmqyg 2468765 2468764 2026-10-15T11:28:16Z Stashbot 7414 marostegui@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db2180', diff saved to https://phabricator.wikimedia.org/P96692 and previous config saved to /var/cache/conftool/dbconfig/20261015-112815-marostegui.json 2468765 wikitext text/x-wiki 42g45lgwt29rhxzg7qkic0cugo1e83d 2468766 2468765 2026-10-15T11:43:20Z Stashbot 7414 marostegui@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db2180 (T441197)', diff saved to https://phabricator.wikimedia.org/P96693 and previous config saved to /var/cache/conftool/dbconfig/20261015-114317-marostegui.json 2468766 wikitext text/x-wiki jmbcxemoe8kfjxdtubjyt4kosul135r 2468767 2468766 2026-10-15T11:43:34Z Stashbot 7414 marostegui@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on db2193.codfw.wmnet with reason: Maintenance 2468767 wikitext text/x-wiki d7dnpdlpthk4aif8h6cc6leygm327d2 2468768 2468767 2026-10-15T11:43:41Z Stashbot 7414 marostegui@cumin1004: dbctl commit (dc=all): 'Depooling db2193 (T441197)', diff saved to https://phabricator.wikimedia.org/P96694 and previous config saved to /var/cache/conftool/dbconfig/20261015-114335-marostegui.json 2468768 wikitext text/x-wiki k1t7ehuyh8iiddio0m2w4lnvwqbpwea 2468769 2468768 2026-10-15T11:44:42Z Stashbot 7414 kartik@deploy1003: helmfile [ml-staging-codfw] 'sync' command on namespace 'recommendation-api-ng' for release 'main' . 2468769 wikitext text/x-wiki hgcd3limyc7k2ipy6hp1k4gqap66ct4 2468770 2468769 2026-10-15T11:45:46Z Stashbot 7414 marostegui@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db2193 (T441197)', diff saved to https://phabricator.wikimedia.org/P96695 and previous config saved to /var/cache/conftool/dbconfig/20261015-114543-marostegui.json 2468770 wikitext text/x-wiki fubf11pw5ai8olprzfv2qbstmbyjtxy 2468771 2468770 2026-10-15T11:47:14Z Stashbot 7414 kart_: Updated Recommendation API to 2026-10-07-145851-production (staging only) (T437213) 2468771 wikitext text/x-wiki 48vybd1wizt52jbyl0x58p85n1vlojy 2468772 2468771 2026-10-15T11:56:55Z Stashbot 7414 topranks: drain traffic from ssw1-d8-eqiad in advance of SR-Linux upgrade T431745 2468772 wikitext text/x-wiki ebh06jk195bwopxqeepfu6ap26uo8qe 0 7942 2467975 2435778 2026-10-14T14:26:02Z LToscano (WMF) 7191 /* Maintain the maintenance calendar */ 2467975 wikitext text/x-wiki a9ntetyxzwhl3d99jyt4dlib5tnefji 0 17290 2467895 2467399 2026-10-14T13:23:27Z Stashbot 7414 James_F: Zuul: Configure CI for REL1_47 branch, for T441072 2467895 wikitext text/x-wiki 5z9oa7gyvhk7vvbrmkuwdff5ky4aa0c 2468286 2467895 2026-10-14T19:39:17Z Stashbot 7414 Southparkfan: Assign missing security groups to instances: web to deployment-deploy06, citoid to deployment-docker-citoid03, wikifeeds to deployment-docker-wikifeeds01 2468286 wikitext text/x-wiki im9di5d16o54y54um5farbxd6r05uq8 2468290 2468286 2026-10-14T19:46:24Z Stashbot 7414 Southparkfan: Decommission deployment-deploy04 - T435393, T441229 2468290 wikitext text/x-wiki j4lj08xmgxnh7v01n30ewnkoek1usqm 2468359 2468290 2026-10-14T22:23:09Z Stashbot 7414 dduvall: bringing integration-agent-docker-1095 back online after applying default-ulimit workaround (T440670) 2468359 wikitext text/x-wiki 5jku6tlvyqdppb9yrqtbhft638jnrgz 498 55577 2468737 2461335 2026-10-15T10:00:02Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component replica-cnf 2468737 wikitext text/x-wiki eegpbe09mmzhc2h2znbyobl9tvtbchd 2468738 2468737 2026-10-15T10:00:03Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component replica-cnf 2468738 wikitext text/x-wiki 55bbjgjvib2f9hks56ugin4nxl221se 2468739 2468738 2026-10-15T10:02:16Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component replica-cnf 2468739 wikitext text/x-wiki bhw3olz9kg01tk9uu80gwyqyzjcl71u 2468740 2468739 2026-10-15T10:02:21Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component replica-cnf 2468740 wikitext text/x-wiki fcvrewfu5yaqo7zgpn9xd9bhiprz80f 2468743 2468740 2026-10-15T10:04:25Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component replica-cnf 2468743 wikitext text/x-wiki l6w91zrfdiuub964231iflme8icz9mh 2468744 2468743 2026-10-15T10:04:39Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component replica-cnf 2468744 wikitext text/x-wiki 9nxc12u8hlzqgu56cxx5hsgved6dzf8 2468751 2468744 2026-10-15T10:18:45Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component replica-cnf 2468751 wikitext text/x-wiki 0e38xaeush00q2sg7gnnp69nbn6zhao 2468752 2468751 2026-10-15T10:19:01Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component replica-cnf 2468752 wikitext text/x-wiki nt9nnuzve72iyj8mhb71bql7v8k9ffx 0 63011 2467910 2434476 2026-10-14T13:31:48Z KSoučková-WMF 35993 note to update both infobox and first paragraph on wikitech 2467910 wikitext text/x-wiki 0kaks0ax79owvt44tm8wqpi8apv786a 498 442283 2468180 2439731 2026-10-14T16:55:53Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: refreshed the GiHub token in cdnjs-index/tokenfile (now expires 2027-01-12) 2468180 wikitext text/x-wiki f3f54xmc0ub8whz44zodc1j964pxxed 498 443261 2467976 2458411 2026-10-14T14:27:03Z Stashbot 7414 filippo@cloudcumin1001: START - Cookbook wmcs.openstack.cloudvirt.vm_console 2467976 wikitext text/x-wiki slnprmzv750mea4yeqwtngvap92lvlp 2467977 2467976 2026-10-14T14:27:04Z Stashbot 7414 filippo@cloudcumin1001: END (FAIL) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=99) 2467977 wikitext text/x-wiki ef9qsyyq0mdb2zpr7mxku44idfpmacu 2467978 2467977 2026-10-14T14:27:14Z Stashbot 7414 filippo@cloudcumin1001: START - Cookbook wmcs.openstack.cloudvirt.vm_console 2467978 wikitext text/x-wiki jk1kib99znn8qfvwrhx3fivjb5tkegb 2467979 2467978 2026-10-14T14:27:16Z Stashbot 7414 filippo@cloudcumin1001: END (FAIL) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=99) 2467979 wikitext text/x-wiki 1nicwnhjzjtr8fj7dslv0ufr8xc3aht 0 443904 2467904 2440053 2026-10-14T13:30:30Z KSoučková-WMF 35993 update infobox 2467904 wikitext text/x-wiki 3dsxbqkesww3aihvjav9ec5dgvjqxhf 0 446874 2468174 2401015 2026-10-14T16:53:40Z Meno25 1446 update redirected link 2468174 wikitext text/x-wiki mxg9rfj1gnvuq899v1w2973fm0b085n 0 447718 2467819 2467711 2026-10-14T12:21:42Z Hashar 32 /* Deployment */ delete unwanted generated ssh key pairs see {{PhabT|441113}} 2467819 wikitext text/x-wiki cwjopb09aemscnw3q66smfgqejz4y5b 0 447761 2468733 2466803 2026-10-15T09:52:35Z LSobanski (WMF) 18783 /* Failover (in case primary host is unavailable) */ Add a step to verify Zuul is working 2468733 wikitext text/x-wiki 7id66l9lvn5omhjvqkdqrc3v27ec20t 2468735 2468733 2026-10-15T09:53:56Z LSobanski (WMF) 18783 /* Finalize Puppet Update */ 2468735 wikitext text/x-wiki 3rb3ps5zr59to7ave6cgd8smdevrq3z 0 449160 2468371 2467408 2026-10-15T00:02:19Z Dexbot 30554 Bot: Updating the report 2468371 wikitext text/x-wiki g04awkttl9nesubjok3wq262zpdmvd6 2468747 2468371 2026-10-15T10:09:33Z Dexbot 30554 Bot: Updating the report 2468747 wikitext text/x-wiki iv54kghezzizjjyv5sd52msvk0b6h25 0 450971 2468719 2434895 2026-10-15T09:27:19Z FGiunchedi (WMF) 39205 Remove 24h notice to resolve the page, no longer true 2468719 wikitext text/x-wiki 7pw51vgxvuivbbjcgeu58ruvt49dxpn 0 452006 2468109 2466499 2026-10-14T16:04:08Z BTullis (WMF) 25295 Add information about creating accounts 2468109 wikitext text/x-wiki g64bsoc2lvykfyuf8j05o8v2e5jouar 0 453863 2468756 2135095 2026-10-15T10:48:59Z FGiunchedi (WMF) 39205 Expand on common issues and alert description 2468756 wikitext text/x-wiki cvgo0676gv6goc83m4mw17xxpms1dtm 2468761 2468756 2026-10-15T10:58:32Z FGiunchedi (WMF) 39205 Add debugging commands 2468761 wikitext text/x-wiki poawmt8tb3aqjvt0siibmr6ajht2b65 116 453906 2468215 2466924 2026-10-14T17:36:26Z Gitlabaccountapprovalbot 37332 ayushtripathi was rejected. 2468215 wikitext text/x-wiki 3d3iqbfgek4lvk6w28glugwjry4e4pi 498 455134 2467964 2445079 2026-10-14T14:00:35Z Stashbot 7414 filippo@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase by 12 volumes 2467964 wikitext text/x-wiki 0ud46h58h63x3r851i9zsoe5o5cat3u 2467965 2467964 2026-10-14T14:00:39Z Stashbot 7414 filippo@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) by 12 volumes 2467965 wikitext text/x-wiki ht2lvcikx41f4v5qfunkaza7n2msm6a 0 455989 2468029 2465909 2026-10-14T15:08:48Z Phuedx (WMF) 47850 Add a short section on sending analytics events for users other than the current user (T436960) 2468029 wikitext text/x-wiki f0x5dxt717bjemrylsli2hwmhkhp64b 2468361 2468029 2026-10-14T22:29:37Z Roan Kattouw (WMF) 8 /* Sending events for users other than the current user */ 2468361 wikitext text/x-wiki d2syi5ilm7uwqytmg4ic9icskkgdw0h 0 457961 2468724 2419686 2026-10-15T09:35:50Z HNowlan (WMF) 15760 add restart 2468724 wikitext text/x-wiki f6001mmc57paumt2ofdhshvf4q82mc0 4 458615 2467929 2451581 2026-10-14T13:41:52Z BPiovesan-WMF 56466 /* Community consultation on Wikimedia Hackathon */ new section 2467929 wikitext text/x-wiki 2cs861boxl2g4hgculspre05y1bo6pq 0 458675 2468070 2420813 2026-10-14T15:34:50Z CMooney (WMF) 24288 /* Upgrade software */ 2468070 wikitext text/x-wiki 38m8uxrbey3dwm3guf39avu5su3y2ry 0 459156 2468252 2447958 2026-10-14T18:36:10Z Filippo Giunchedi 2939 Expand on debugging and error/incident 2468252 wikitext text/x-wiki jr5n78tq30q9eb8hlcvsccd01n0bjry 0 459290 2467884 2466929 2026-10-14T13:15:32Z JMonton-WMF 47887 /* YYYY-MM-DD NEXT TUESDAY TRAIN (REPLACE THIS AFTER DEPLOY) */ 2467884 wikitext text/x-wiki et6spwdknvdnitccegwtp805n24arxz 2468666 2467884 2026-10-15T08:28:23Z APizzata-WMF 47941 2468666 wikitext text/x-wiki ahev0ww1mrni2w817dq2yq0km350339 0 459340 2468169 2388312 2026-10-14T16:49:07Z XCollazo-WMF 31846 /* Project rationale */ Rephrase rationale now that we are formally sunsetting the legacy path. 2468169 wikitext text/x-wiki 7nkz8fdw4vebrhb7dbvmpum4mp9wem4 0 459699 2468124 2467359 2026-10-14T16:16:13Z Stashbot 7414 ejegg: SmashPig upgraded from 7e3426cf to 7b7586f2 2468124 wikitext text/x-wiki 1qoq0ue1slnvd2icbfx3ugjyw6cxb29 2468311 2468124 2026-10-14T20:36:33Z Stashbot 7414 ejegg: donorwiki upgraded from 60a7b7af to da1d88b6 2468311 wikitext text/x-wiki 9qmlmkmxf2fgduz2m0o2g3uz0qyc090 2468312 2468311 2026-10-14T20:37:32Z Stashbot 7414 ejegg: donorwiki upgraded from da1d88b6 to 9e8d0c0b 2468312 wikitext text/x-wiki 1t5ast7l5h5fyd69crxz3w9qklysqan 2468313 2468312 2026-10-14T20:39:14Z Stashbot 7414 ejegg: payments-wiki upgraded from da1d88b6 to 9e8d0c0b 2468313 wikitext text/x-wiki 0qghhdf7j09gbikhztfbc57bkx6squ2 0 459895 2468283 2448966 2026-10-14T19:36:58Z DKinzler (WMF) 156 /* Deployment to production */ The "rate limit policy" selector should be set to an actively used policy, otherwise the part of the panel related to rate limits will show "no data" or bogus. 2468283 wikitext text/x-wiki h4prp5egfsecwm100k2fkcyi9v2jim7 10 460153 2468000 2462789 2026-10-14T14:49:51Z TBurmeister (WMF) 27417 add Compare section 2468000 wikitext text/x-wiki 10vjub0wfs36fy2vwz6mny51dxj26sq 0 460791 2468062 2466716 2026-10-14T15:27:13Z GGoncalves-WMF 42848 Added a couple more FAQ based on talk page discussion. 2468062 wikitext text/x-wiki nfbadq1ijnokjuilxf9qjxmgke9nocm 2468734 2468062 2026-10-15T09:53:46Z GGoncalves-WMF 42848 /* FAQ */ Fix dump name and add link to the Summary in FAQ entry. 2468734 wikitext text/x-wiki q83gm6edwszc5qbtawf83b5aaniru09 0 460803 2468279 2466707 2026-10-14T19:27:07Z DLynch (WMF) 6438 2468279 wikitext text/x-wiki rzhbovwqhp27y9npl10y2apkkxvcrmo 2468282 2468279 2026-10-14T19:36:43Z DLynch (WMF) 6438 2468282 wikitext text/x-wiki 4xfvc2lzjge0pgqorweaujj9s8keywu 1 460805 2468063 2467387 2026-10-14T15:28:04Z GGoncalves-WMF 42848 /* Add to FAQ? */ Reply 2468063 wikitext text/x-wiki sxke4p2sa16z72lens13ptq74tce5v9 2468717 2468063 2026-10-15T09:21:35Z This, that and the other 921 /* Add to FAQ? */ Reply 2468717 wikitext text/x-wiki g4arfovxj1kfsuw8gobc8j86yz18ogo 2468736 2468717 2026-10-15T09:54:40Z GGoncalves-WMF 42848 /* Add to FAQ? */ Reply 2468736 wikitext text/x-wiki 9bnpdtle83qf6ff89phav6304zre8yi 3 460820 2468683 2026-10-15T08:42:47Z StrikerBot 8475 Welcome to Toolforge! 2468683 wikitext text/x-wiki toyd8so1ug2xjktbpvxaty6i31mxs3p