Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.22 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 3086 2462250 2462053 2026-09-30T14:06:11Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2462250 wikitext text/x-wiki 7fkqfew90ytvj0staysnazpsjq6c1x0 2462276 2462250 2026-09-30T14:18:12Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-api 2462276 wikitext text/x-wiki mpszkpcqrr9htmt024uic7to5r0m64u 0 4108 2462162 2462058 2026-09-30T12:50:13Z ScheduleDeploymentBot 37566 Add [[gerrit:697110]] to Wednesday, September 30 UTC afternoon backport window 2462162 wikitext text/x-wiki 1ew70hix1yuztz7frxy1ubmcdx2mfbf 2462467 2462162 2026-09-30T19:00:24Z ScheduleDeploymentBot 37566 Add [[gerrit:1346327]] to Wednesday, September 30 UTC late backport window 2462467 wikitext text/x-wiki cg2fot95hfic30u59v1ocq98q63u5j9 2462468 2462467 2026-09-30T19:00:41Z ScheduleDeploymentBot 37566 Add [[gerrit:1346328]] to Wednesday, September 30 UTC late backport window 2462468 wikitext text/x-wiki i9dwqqw23a9io073q416dzo9oc08p30 2462469 2462468 2026-09-30T19:00:53Z ScheduleDeploymentBot 37566 Add [[gerrit:1346329]] to Wednesday, September 30 UTC late backport window 2462469 wikitext text/x-wiki pzzvuvjct3gij07boleu3nf94zh87r0 2462470 2462469 2026-09-30T19:19:15Z ScheduleDeploymentBot 37566 Add [[gerrit:1346763]] to Wednesday, September 30 UTC late backport window 2462470 wikitext text/x-wiki 47i0l6id0xgdsuu80ok108ql49jm2vr 2462491 2462470 2026-09-30T20:53:54Z ScheduleDeploymentBot 37566 Add [[gerrit:1346053]] to Monday, October 05 UTC late backport window 2462491 wikitext text/x-wiki jdlxwgyq43g1wpuqgdihg0s1lb7443i 2462494 2462491 2026-09-30T20:59:41Z ScheduleDeploymentBot 37566 Add [[gerrit:1346793]] to Monday, October 05 UTC late backport window 2462494 wikitext text/x-wiki k9w2xgw3vz6r0dhcyb6nh3a5z8he90f 2462538 2462494 2026-10-01T06:46:46Z ScheduleDeploymentBot 37566 Add [[gerrit:1346542]] to Thursday, October 01 UTC morning backport window 2462538 wikitext text/x-wiki f6pvodsvi36c0067b7lfp51ip68038i 2462558 2462538 2026-10-01T07:41:26Z SLyngshede-WMF 31501 /* Week of October 26 */ Schedule DC switch over 2462558 wikitext text/x-wiki b6vf8tswmtpesz97rk6zrwmakbztea0 2462573 2462558 2026-10-01T08:52:39Z ScheduleDeploymentBot 37566 Add [[gerrit:1347422]] to Thursday, October 01 UTC afternoon backport window 2462573 wikitext text/x-wiki 7huqtcy18d2wp1mxiy43vasxtqfo3fa 2462574 2462573 2026-10-01T08:52:45Z ScheduleDeploymentBot 37566 Add [[gerrit:1347423]] to Thursday, October 01 UTC afternoon backport window 2462574 wikitext text/x-wiki 1q2jfe7upoletn4j2vqtlswuev3ngrj 2462575 2462574 2026-10-01T08:52:52Z ScheduleDeploymentBot 37566 Add [[gerrit:1347424]] to Thursday, October 01 UTC afternoon backport window 2462575 wikitext text/x-wiki qckwbun8ywlxpwygg5vvvjx1hcpv284 2462592 2462575 2026-10-01T10:33:09Z ScheduleDeploymentBot 37566 Add [[gerrit:1347483]] to Thursday, October 01 UTC afternoon backport window 2462592 wikitext text/x-wiki costraw7axbbf3mcte4x475atnudzsm 498 6726 2462235 2462041 2026-09-30T13:52:44Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2462235 wikitext text/x-wiki pb09us35t0oomoiqbk7ub8h2fme0j1h 2462245 2462235 2026-09-30T14:04:14Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-api 2462245 wikitext text/x-wiki 0ztglcvsyhp4mmjlewczieyn0n55gqk 0 7919 2462107 2462106 2026-09-30T12:00:36Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-worker1044.eqiad.wmnet 2462107 wikitext text/x-wiki ewgsn7r2fwqviewlm0pl7819upgqms2 2462108 2462107 2026-09-30T12:02:11Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-worker1043.eqiad.wmnet 2462108 wikitext text/x-wiki beca9p0psvfqqlqpcepuhhqk43yxhim 2462109 2462108 2026-09-30T12:12:09Z Stashbot 7414 tappof@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5 days, 0:00:00 on kafka-logging[1001-1003].eqiad.wmnet with reason: stop service on kafka-logging100[1-3] 2462109 wikitext text/x-wiki 7eqha4o7lu81w2sb4nw0fyp4zn2s2oe 2462110 2462109 2026-09-30T12:13:55Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1380.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462110 wikitext text/x-wiki nvtujfdl94w2dx0oq8ljbz8t6sey1zt 2462111 2462110 2026-09-30T12:14:04Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1380.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462111 wikitext text/x-wiki bm0ithrmjsbd9dl6qi6gnblr21jw8mz 2462112 2462111 2026-09-30T12:14:12Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1381.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462112 wikitext text/x-wiki 7vykxzh8c3utlfwkzkxkwmmfmmbj4ng 2462113 2462112 2026-09-30T12:14:20Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1381.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462113 wikitext text/x-wiki 3n9m1j6lkexg74g5x3hh65tt9whcvcf 2462114 2462113 2026-09-30T12:14:35Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1382.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462114 wikitext text/x-wiki 6mq26to833cpqxegh1vfqyn82tx9161 2462115 2462114 2026-09-30T12:14:46Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1382.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462115 wikitext text/x-wiki 6c5z752ozknrxrbo88pa3xzvta0knrn 2462116 2462115 2026-09-30T12:14:54Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1383.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462116 wikitext text/x-wiki 5svs7hatfn1qhpqdga2rx7yo3wkj0lk 2462117 2462116 2026-09-30T12:15:04Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1383.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462117 wikitext text/x-wiki ji2ksqtosq3zmc9lwe2mslp7kptq2ll 2462118 2462117 2026-09-30T12:16:09Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.reboot-nodes rolling reboot on D{wikikube-worker138[0-3].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) 2462118 wikitext text/x-wiki q7qpgxuufihabhne65h25rm97hos3cd 2462119 2462118 2026-09-30T12:16:13Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1380.eqiad.wmnet 2462119 wikitext text/x-wiki fb6m3zo6djivo5dn2jl9x8v5vgcam8q 2462120 2462119 2026-09-30T12:16:46Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1380.eqiad.wmnet 2462120 wikitext text/x-wiki 0y4c8mu5s6v270kykw5w7l61glyp6fk 2462121 2462120 2026-09-30T12:23:45Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1380.eqiad.wmnet 2462121 wikitext text/x-wiki rnssjkc8ujdme5jshpw92imd5v26ffq 2462122 2462121 2026-09-30T12:23:47Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1380.eqiad.wmnet 2462122 wikitext text/x-wiki 6iqgvxf7nj0up7n2xfw7uk5bm3be8ph 2462123 2462122 2026-09-30T12:23:52Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1381.eqiad.wmnet 2462123 wikitext text/x-wiki j7iwcyxc0idwvsoylm3pvrtxg8dbc7q 2462124 2462123 2026-09-30T12:24:23Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1381.eqiad.wmnet 2462124 wikitext text/x-wiki auwm18atljc6abncizv9u0x6gbf110k 2462125 2462124 2026-09-30T12:26:49Z Stashbot 7414 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1328191|Profiler: Fix excimer component regex skipping the last frame in each stack]] 2462125 wikitext text/x-wiki a8krn2mk6kgeg5um0dx01yit60qr899 2462126 2462125 2026-09-30T12:30:59Z Stashbot 7414 krinkle@deploy1003: somerandomdeveloper, krinkle: Backport for [[gerrit:1328191|Profiler: Fix excimer component regex skipping the last frame in each stack]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462126 wikitext text/x-wiki lwitzqi9uf89eudf8qbvqnsxk02k7ew 2462127 2462126 2026-09-30T12:31:23Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1381.eqiad.wmnet 2462127 wikitext text/x-wiki qi22jf5v2au9xbrybejjj0zpj7snc5b 2462128 2462127 2026-09-30T12:31:24Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1381.eqiad.wmnet 2462128 wikitext text/x-wiki ry9bcs41eswi3b21z8hqux277s1ekpg 2462131 2462128 2026-09-30T12:31:32Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1382.eqiad.wmnet 2462131 wikitext text/x-wiki bi1xl73i29oqlfexzhkd3c709b2z6r1 2462132 2462131 2026-09-30T12:32:01Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1382.eqiad.wmnet 2462132 wikitext text/x-wiki 89fntb94qj5zinh5zzc8u08e9ztl8yq 2462133 2462132 2026-09-30T12:32:30Z Stashbot 7414 krinkle@deploy1003: somerandomdeveloper, krinkle: Continuing with deployment 2462133 wikitext text/x-wiki 7o1a0hz5bd7a6twc9r4pum1td1qfswb 2462134 2462133 2026-09-30T12:33:10Z Stashbot 7414 dkertesz: fetching new haproxy packages on reprepro (3.2.23-1~bpo13+1 -> 3.2.25-1~bpo13+1) - T439667 2462134 wikitext text/x-wiki 1xo1ws20z06o8vkky09ba2vh6lo33w0 2462137 2462134 2026-09-30T12:37:46Z Stashbot 7414 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1328191|Profiler: Fix excimer component regex skipping the last frame in each stack]] (duration: 10m 56s) 2462137 wikitext text/x-wiki m6bffg0pw5g52dz4loatmnr1hejnafc 2462138 2462137 2026-09-30T12:39:00Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1382.eqiad.wmnet 2462138 wikitext text/x-wiki qh5so5djdhy4bnrmgo7j8olh2qrwalg 2462139 2462138 2026-09-30T12:39:08Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1382.eqiad.wmnet 2462139 wikitext text/x-wiki n5jte335a7isyxcpx0330lm5jjyijvb 2462140 2462139 2026-09-30T12:39:09Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1383.eqiad.wmnet 2462140 wikitext text/x-wiki l7v9b2i0gbgao6z3xw6gc86uds9mthx 2462141 2462140 2026-09-30T12:39:40Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1383.eqiad.wmnet 2462141 wikitext text/x-wiki om6zdv511y1g3fq3hl9ak29b0rhp2wk 2462142 2462141 2026-09-30T12:41:47Z Stashbot 7414 dkertesz@cumin1004: START - Cookbook sre.cdn.roll-upgrade-haproxy rolling upgrade of HAProxy on P{cp7001.*} and A:cp - upgrade haproxy to 3.2.25-1 (T439667) 2462142 wikitext text/x-wiki ts0xym1o33ptg15q5t49wyjatpnx8l6 2462143 2462142 2026-09-30T12:42:24Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462143 wikitext text/x-wiki bdv3fe9cz082se48k82055wrirvo1k0 2462146 2462143 2026-09-30T12:43:09Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . 2462146 wikitext text/x-wiki di412tma4o4a2c102vp5h3l6l3mr348 2462147 2462146 2026-09-30T12:43:28Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462147 wikitext text/x-wiki 3g74529c3p2u02m9iyphgoekdnz4jnw 2462148 2462147 2026-09-30T12:43:56Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462148 wikitext text/x-wiki rwwkoaxxuggqo17l8slhde6jgdksa12 2462149 2462148 2026-09-30T12:44:55Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462149 wikitext text/x-wiki r0si7qluyb6e5a28owrxqb81pq93jcf 2462152 2462149 2026-09-30T12:45:49Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462152 wikitext text/x-wiki 34zv4cl9xk1v0nqudpaqtim8aqxmfef 2462155 2462152 2026-09-30T12:46:50Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1383.eqiad.wmnet 2462155 wikitext text/x-wiki 1l9bkemz25zpf5s604i6ee8d0pkhpsr 2462156 2462155 2026-09-30T12:46:51Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1383.eqiad.wmnet 2462156 wikitext text/x-wiki 60tsb62bhbzf1qwwozifqvm43zzcv27 2462157 2462156 2026-09-30T12:46:54Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on D{wikikube-worker138[0-3].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) 2462157 wikitext text/x-wiki d6uos8anw2xfd9orv93l5n6itjx8di1 2462158 2462157 2026-09-30T12:47:05Z Stashbot 7414 dkertesz@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-haproxy (exit_code=0) rolling upgrade of HAProxy on P{cp7001.*} and A:cp - upgrade haproxy to 3.2.25-1 (T439667) 2462158 wikitext text/x-wiki cyolfm4i1tldy07nfkef3ish92s9b6q 2462161 2462158 2026-09-30T12:48:34Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462161 wikitext text/x-wiki d75eexmfs7s2gv7se2vgdlxtkpzehfu 2462163 2462161 2026-09-30T12:53:04Z Stashbot 7414 tappof@deploy1003: helmfile [eqiad] START helmfile.d/admin 'sync'. 2462163 wikitext text/x-wiki n27jzlj83yy6e6shyegextk2rw90n4x 2462164 2462163 2026-09-30T12:53:19Z Stashbot 7414 tappof@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'sync'. 2462164 wikitext text/x-wiki 9wpqeuqr432f3wrqycnk6xn1h4kg5cr 2462165 2462164 2026-09-30T12:53:39Z Stashbot 7414 tappof@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'. 2462165 wikitext text/x-wiki 401m877mj9vt0bbvau7t7vvunfdx7qp 2462166 2462165 2026-09-30T12:54:16Z Stashbot 7414 tappof@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'. 2462166 wikitext text/x-wiki kgbvoqvsihhhraplvcg3rusp0ziwd3i 2462167 2462166 2026-09-30T12:54:32Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462167 wikitext text/x-wiki ptesk94msjobkevj28dxhlh9w7y783s 2462168 2462167 2026-09-30T12:54:33Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462168 wikitext text/x-wiki s3x7zdm0ean9tdpptlp17akola81kkh 2462169 2462168 2026-09-30T12:56:44Z Stashbot 7414 tappof@deploy1003: Started scap sync-world: T432444 - remove kafka-logging100[12] 2462169 wikitext text/x-wiki gsg2dv8ydqsung4jlb33u29ruu12hy1 2462170 2462169 2026-09-30T12:56:54Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462170 wikitext text/x-wiki dr5a922qdr26fnreg6ty8s67ufcc29a 2462171 2462170 2026-09-30T13:02:39Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462171 wikitext text/x-wiki iv3y5sndyun00w7oz8znd1t4dqr22jx 2462172 2462171 2026-09-30T13:04:10Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462172 wikitext text/x-wiki e5p7vzqny8cptxhg9xv7gu3b3s1i9qh 2462173 2462172 2026-09-30T13:04:20Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462173 wikitext text/x-wiki duvnm5yqgjbil770bjwu8g0jo66swsp 2462174 2462173 2026-09-30T13:04:29Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462174 wikitext text/x-wiki 2nvhlviszeld7dkxwduojsedj4v47s4 2462175 2462174 2026-09-30T13:04:39Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462175 wikitext text/x-wiki gxnpbg2z005tquia6jvoa69c9oxbnro 2462176 2462175 2026-09-30T13:04:48Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462176 wikitext text/x-wiki ov1p2u8aw7ykfd6eq8np3mtmilhoag1 2462177 2462176 2026-09-30T13:04:58Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462177 wikitext text/x-wiki ood25go3hd5xxjnebdnnkeby2t3ockh 2462178 2462177 2026-09-30T13:05:20Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462178 wikitext text/x-wiki 27f1hzxlz637hluxjyhd9lwu49tkgnq 2462179 2462178 2026-09-30T13:05:30Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462179 wikitext text/x-wiki s6pv2jl3gt7zn97th7a785vik89tuu8 2462180 2462179 2026-09-30T13:05:50Z Stashbot 7414 tappof@deploy1003: Finished scap sync-world: T432444 - remove kafka-logging100[12] (duration: 10m 41s) 2462180 wikitext text/x-wiki g0koeyadpz7e2fdkoolvcvp123bvuwb 2462181 2462180 2026-09-30T13:06:07Z Stashbot 7414 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1346060|fix(AccountSetup): add missing i18n key to extension.json (T438908 T436872)]] 2462181 wikitext text/x-wiki kxtmblmwydi035ka6272p5glkhn34rt 2462182 2462181 2026-09-30T13:06:15Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462182 wikitext text/x-wiki eik0b6uwdnhj9e1i84yof4ea9zyit2q 2462183 2462182 2026-09-30T13:06:24Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462183 wikitext text/x-wiki opxoele0jb08uo8vzkjdyn0le07y3lf 2462184 2462183 2026-09-30T13:07:14Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.reboot-nodes rolling reboot on D{wikikube-worker237[0-5].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2462184 wikitext text/x-wiki 5e1z7t0rcdhmb1qospcevitkeknbf56 2462185 2462184 2026-09-30T13:07:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2370.codfw.wmnet 2462185 wikitext text/x-wiki nqogmq0jmv5yiqdxiim7sr0625ci002 2462187 2462185 2026-09-30T13:07:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2370.codfw.wmnet 2462187 wikitext text/x-wiki kkwj2owhymp1f2buy3bwrvnpl6nb85v 2462188 2462187 2026-09-30T13:10:18Z Stashbot 7414 samtar@deploy1003: migr, samtar: Backport for [[gerrit:1346060|fix(AccountSetup): add missing i18n key to extension.json (T438908 T436872)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462188 wikitext text/x-wiki im8qhhzvfptb6v4klop171df45kr53y 2462189 2462188 2026-09-30T13:10:39Z Stashbot 7414 dkertesz@cumin1004: START - Cookbook sre.cdn.roll-upgrade-haproxy rolling upgrade of HAProxy on A:cp-text_magru and not P{cp7001.*} and A:cp - upgrade haproxy to 3.2.25-1 (T439667) 2462189 wikitext text/x-wiki lqfirwvgiws3jx7zyduezg5dgoxoxxs 2462190 2462189 2026-09-30T13:12:48Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462190 wikitext text/x-wiki qlnu4pocjukj1j9k7whaezh9s1ycu75 2462191 2462190 2026-09-30T13:13:11Z Stashbot 7414 samtar@deploy1003: migr, samtar: Continuing with deployment 2462191 wikitext text/x-wiki p255cuj02h94s30bo988w7yxixpnz1t 2462192 2462191 2026-09-30T13:15:01Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2370.codfw.wmnet 2462192 wikitext text/x-wiki sqvvcs888g64lthskski0rt4y3dtq1i 2462193 2462192 2026-09-30T13:15:04Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2370.codfw.wmnet 2462193 wikitext text/x-wiki 56iwgipbmwv87bg2wvbnjjf0pjgnger 2462194 2462193 2026-09-30T13:15:09Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2371.codfw.wmnet 2462194 wikitext text/x-wiki r180qvsfqtvj4jgclyw0njxpj03gsr1 2462195 2462194 2026-09-30T13:15:44Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2371.codfw.wmnet 2462195 wikitext text/x-wiki ron7lhr9suh5dfio0anrr5qtkjmjrts 2462196 2462195 2026-09-30T13:18:30Z Stashbot 7414 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1346060|fix(AccountSetup): add missing i18n key to extension.json (T438908 T436872)]] (duration: 12m 23s) 2462196 wikitext text/x-wiki l1osn0re249256bs6wqrl0crlhgxf0m 2462199 2462196 2026-09-30T13:20:42Z Stashbot 7414 tappof@cumin1004: START - Cookbook sre.hosts.decommission for hosts kafka-logging1003.eqiad.wmnet 2462199 wikitext text/x-wiki j8t26swsayq8hij5srstxmte7dw00tc 2462200 2462199 2026-09-30T13:21:45Z Stashbot 7414 jgiannelos@deploy1003: Started scap sync-world: Backport for [[gerrit:1345584|prv: Enable parsoid rendering for 5 wikis (T439444)]] 2462200 wikitext text/x-wiki alp3gtkk0bjygjbb6xn0yqglwnsbptg 2462201 2462200 2026-09-30T13:22:44Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2371.codfw.wmnet 2462201 wikitext text/x-wiki 4n1rrmconfojn0htc7rdk3mkinpp1mk 2462202 2462201 2026-09-30T13:22:45Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2371.codfw.wmnet 2462202 wikitext text/x-wiki d49fi06iwuit6dt91c98skf80duhfny 2462203 2462202 2026-09-30T13:22:53Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2372.codfw.wmnet 2462203 wikitext text/x-wiki tlop0m1al1hhmaij1ehiv5t43v1ob9g 2462204 2462203 2026-09-30T13:23:27Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2372.codfw.wmnet 2462204 wikitext text/x-wiki gxg827vdkr1jis4jg7t72rln1hv4upa 2462205 2462204 2026-09-30T13:23:33Z Stashbot 7414 andrew@cumin2003: START - Cookbook sre.hosts.reimage for host cloudcephosd1043.eqiad.wmnet with OS bookworm 2462205 wikitext text/x-wiki bc21wjzznwbvsnr66aqbz1yepoum7yr 2462206 2462205 2026-09-30T13:23:50Z Stashbot 7414 kevinbazira@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . 2462206 wikitext text/x-wiki oyibi53x58ykvxk9j0nsyz5s5raf9dy 2462207 2462206 2026-09-30T13:23:54Z Stashbot 7414 kevinbazira@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'llm' . 2462207 wikitext text/x-wiki f2phak2fr3mwjw4euj4i81hyiz25eq9 2462208 2462207 2026-09-30T13:25:58Z Stashbot 7414 jgiannelos@deploy1003: jgiannelos: Backport for [[gerrit:1345584|prv: Enable parsoid rendering for 5 wikis (T439444)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462208 wikitext text/x-wiki qkwg0zwby0cfhw2miv8l7moqg6gv7a9 2462209 2462208 2026-09-30T13:26:42Z Stashbot 7414 tappof@cumin1004: START - Cookbook sre.dns.netbox 2462209 wikitext text/x-wiki cnl1s0zza9t861bxxxteypdmm6wt4zs 2462210 2462209 2026-09-30T13:27:41Z Stashbot 7414 jgiannelos@deploy1003: jgiannelos: Continuing with deployment 2462210 wikitext text/x-wiki 4be0e40b9au8gnmbsin84u2blbcqlwg 2462212 2462210 2026-09-30T13:30:29Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2372.codfw.wmnet 2462212 wikitext text/x-wiki mva6xu56ncwf69ypl5t3lt3dw7f71oy 2462213 2462212 2026-09-30T13:30:33Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2372.codfw.wmnet 2462213 wikitext text/x-wiki 56zqa2ibacm7u0tm7bw60n8exujyopq 2462214 2462213 2026-09-30T13:30:37Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2373.codfw.wmnet 2462214 wikitext text/x-wiki tnjbjkb1i8oel0xctxsjwzxd1xtbvqn 2462215 2462214 2026-09-30T13:31:11Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2373.codfw.wmnet 2462215 wikitext text/x-wiki 4lgkzcifua4g0mandaa4658gxbt8qgh 2462216 2462215 2026-09-30T13:31:50Z Stashbot 7414 tappof@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: kafka-logging1003.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - tappof@cumin1004" 2462216 wikitext text/x-wiki n455tzm55oeqhgi9vw91f41xff4cmwo 2462217 2462216 2026-09-30T13:32:23Z Stashbot 7414 tappof@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: kafka-logging1003.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - tappof@cumin1004" 2462217 wikitext text/x-wiki eugk26ptynxcfvp0g8ew1wvfy51ybup 2462218 2462217 2026-09-30T13:32:24Z Stashbot 7414 tappof@cumin1004: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2462218 wikitext text/x-wiki 3f3ankijcklilanl717wcer9fk4h6es 2462219 2462218 2026-09-30T13:32:25Z Stashbot 7414 tappof@cumin1004: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts kafka-logging1003.eqiad.wmnet 2462219 wikitext text/x-wiki 7wo4rqlcdt6p3yu102f8hmjkm3idwoj 2462220 2462219 2026-09-30T13:32:58Z Stashbot 7414 jgiannelos@deploy1003: Finished scap sync-world: Backport for [[gerrit:1345584|prv: Enable parsoid rendering for 5 wikis (T439444)]] (duration: 11m 13s) 2462220 wikitext text/x-wiki c9mjx2xhz8hf7h65narduzs6yla7y2y 2462221 2462220 2026-09-30T13:38:12Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2373.codfw.wmnet 2462221 wikitext text/x-wiki sps8na9evknoibe6jvi39nyujsv2n84 2462222 2462221 2026-09-30T13:38:13Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2373.codfw.wmnet 2462222 wikitext text/x-wiki k2k9o6taefq7tb0unudwyxi4ptkhz07 2462223 2462222 2026-09-30T13:38:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2374.codfw.wmnet 2462223 wikitext text/x-wiki maav5ix9d2xme4iywsir8pysj1qsfu2 2462224 2462223 2026-09-30T13:38:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2374.codfw.wmnet 2462224 wikitext text/x-wiki 56yoookgqkah0z4s8oosflvzjqlwfir 2462225 2462224 2026-09-30T13:39:50Z Stashbot 7414 seanleong-wmde@deploy1003: Started scap sync-world: Backport for [[gerrit:697110|Create new http://www.mediawiki.org/xml/sitelist-1.1/ to reference sitelist-1.1.xsd (T222516 T439093)]] 2462225 wikitext text/x-wiki oimq4byw1njhvj65kfbz69kwroa52fa 2462226 2462225 2026-09-30T13:41:45Z Stashbot 7414 andrew@cumin2003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcephosd1043.eqiad.wmnet with OS bookworm 2462226 wikitext text/x-wiki eumbzzvmwewypodpociwghrk3vp5gz9 2462227 2462226 2026-09-30T13:43:47Z Stashbot 7414 seanleong-wmde@deploy1003: seanleong-wmde, lucamauri: Backport for [[gerrit:697110|Create new http://www.mediawiki.org/xml/sitelist-1.1/ to reference sitelist-1.1.xsd (T222516 T439093)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462227 wikitext text/x-wiki 8e0k38uh0tf07kqlol9q7f43xzio6m5 2462228 2462227 2026-09-30T13:46:13Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2374.codfw.wmnet 2462228 wikitext text/x-wiki pypsrrp9g9i8hjwo64re935peu16zji 2462229 2462228 2026-09-30T13:46:15Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2374.codfw.wmnet 2462229 wikitext text/x-wiki 1cudwzjdwy2m6mzaexq7zok6xxxrdlz 2462230 2462229 2026-09-30T13:46:16Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on D{wikikube-worker237[0-5].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2462230 wikitext text/x-wiki 0x4hpyg358htojsud2fm22wu8gq5bdx 2462231 2462230 2026-09-30T13:46:33Z Stashbot 7414 andrew@cumin2003: START - Cookbook sre.hosts.reimage for host cloudcephosd1043.eqiad.wmnet with OS bookworm 2462231 wikitext text/x-wiki hd8fqhvrhu8s2hjeb54s26ou9600mqy 2462232 2462231 2026-09-30T13:46:35Z Stashbot 7414 seanleong-wmde@deploy1003: seanleong-wmde, lucamauri: Continuing with deployment 2462232 wikitext text/x-wiki mfmhlz77iz9fm6gsye8bxkthub01ukn 2462233 2462232 2026-09-30T13:48:49Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462233 wikitext text/x-wiki 9l8tb2p3un6eun58kjr2vq03aa0pf9p 2462234 2462233 2026-09-30T13:51:51Z Stashbot 7414 seanleong-wmde@deploy1003: Finished scap sync-world: Backport for [[gerrit:697110|Create new http://www.mediawiki.org/xml/sitelist-1.1/ to reference sitelist-1.1.xsd (T222516 T439093)]] (duration: 12m 01s) 2462234 wikitext text/x-wiki i8slxz43mbicvutczx0io4fd6bjeutp 2462236 2462234 2026-09-30T13:53:10Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462236 wikitext text/x-wiki 8sp6h7xdka35vqu95hb6r398dkcu4vk 2462237 2462236 2026-09-30T13:54:34Z Stashbot 7414 seanleong-wmde@deploy1003: Started scap sync-world: Backport for [[gerrit:1346611|Fix xsd xml version (T439093)]] 2462237 wikitext text/x-wiki b63w6wljcvnr03970pqulwnb4mhzsvy 2462238 2462237 2026-09-30T13:56:10Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462238 wikitext text/x-wiki 3ibtvm8znt29q021x6dftgz9a30c9l4 2462239 2462238 2026-09-30T13:58:50Z Stashbot 7414 seanleong-wmde@deploy1003: awight, seanleong-wmde: Backport for [[gerrit:1346611|Fix xsd xml version (T439093)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462239 wikitext text/x-wiki rs69ftdqm5m91seq3kcs8nnc4unky61 2462240 2462239 2026-09-30T14:00:39Z Stashbot 7414 seanleong-wmde@deploy1003: awight, seanleong-wmde: Continuing with deployment 2462240 wikitext text/x-wiki ilz5pvax0ebb0nqt0y7fnsk4p6la3w1 2462241 2462240 2026-09-30T14:00:42Z Stashbot 7414 inflatador: bking@krb2002 disable wmf_auto_restart krb admin timers/services as this host is no longer primary T437937 2462241 wikitext text/x-wiki abkms79ypm5wbfum4z094j9ml1pjfl3 2462242 2462241 2026-09-30T14:00:43Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462242 wikitext text/x-wiki 7x8nx29ogtsjy7vv5m8n8mlatrso3z4 2462243 2462242 2026-09-30T14:01:44Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462243 wikitext text/x-wiki roi50v6q2a9f1apjd3unshzylhyihli 2462244 2462243 2026-09-30T14:04:10Z Stashbot 7414 dkertesz@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-haproxy (exit_code=0) rolling upgrade of HAProxy on A:cp-text_magru and not P{cp7001.*} and A:cp - upgrade haproxy to 3.2.25-1 (T439667) 2462244 wikitext text/x-wiki m8q4q651qjz9wydsphfjyw8xntgyppp 2462246 2462244 2026-09-30T14:04:48Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462246 wikitext text/x-wiki 86dbzgjl6uj8pm1utnyinoonzqo3qub 2462247 2462246 2026-09-30T14:04:53Z Stashbot 7414 Amir1: mwscript-k8s --comment="T358266" --local_dblist=<(expanddblist all | grep -v commons) -- extensions/TimedMediaHandler/maintenance/requeueTranscodes.php --key '360p.mpeg4.mov' --throttle --video --missing (T358266) 2462247 wikitext text/x-wiki n64bdbj58pfhwx6tb7ih6hriuyi7lxk 2462248 2462247 2026-09-30T14:05:34Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462248 wikitext text/x-wiki l9m40tmzlp6eclx6jy3b5oyngchg7h0 2462249 2462248 2026-09-30T14:06:07Z Stashbot 7414 seanleong-wmde@deploy1003: Finished scap sync-world: Backport for [[gerrit:1346611|Fix xsd xml version (T439093)]] (duration: 11m 33s) 2462249 wikitext text/x-wiki 00gd969noei826tjta58u15utolccxf 2462251 2462249 2026-09-30T14:06:13Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462251 wikitext text/x-wiki 428g1881g98tzyrz2qslov38m1qe9is 2462252 2462251 2026-09-30T14:07:13Z Stashbot 7414 ecarg@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2462252 wikitext text/x-wiki jiccb328te4f7b2kq7y679ixk146xnk 2462253 2462252 2026-09-30T14:08:02Z Stashbot 7414 ecarg@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2462253 wikitext text/x-wiki 251iwp2vz7m5jsen7tbkfp6fxa6vyng 2462254 2462253 2026-09-30T14:09:25Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462254 wikitext text/x-wiki ss9ttqds7mconj30zklnqa6a6lviya4 2462255 2462254 2026-09-30T14:09:39Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2462255 wikitext text/x-wiki 9pcxfnxe6kkjbxj18c3p3yklfxo821k 2462256 2462255 2026-09-30T14:11:40Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462256 wikitext text/x-wiki rke8hkitp3wtdbjpsnu47jrllb7cr98 2462257 2462256 2026-09-30T14:12:36Z Stashbot 7414 jforrester@deploy1003: Started scap sync-world: Backport for [[gerrit:1346619|mlwiki: Enable Abstract Wikimedia client mode]] 2462257 wikitext text/x-wiki ezg3immbjvsj9ajznqfm6ed9otot5i3 2462258 2462257 2026-09-30T14:13:44Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2462258 wikitext text/x-wiki ib63whookd1887o765k24az0a673g0s 2462259 2462258 2026-09-30T14:14:08Z Stashbot 7414 ecarg@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2462259 wikitext text/x-wiki s11ev2vlg988my4bjg9pxyzeju25kra 2462260 2462259 2026-09-30T14:14:11Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1330.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462260 wikitext text/x-wiki 40zomsyo2a7o72rhzlul18o2vw097df 2462261 2462260 2026-09-30T14:14:20Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1330.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462261 wikitext text/x-wiki 66a9495j0vig2hadzbsuj27zqhvqcrt 2462262 2462261 2026-09-30T14:14:27Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1331.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462262 wikitext text/x-wiki dezo667tp0ayq9l8cv3pfohs1x6vfdg 2462263 2462262 2026-09-30T14:14:36Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1331.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462263 wikitext text/x-wiki f13ef2vf1b9ubz0i8jnru6ndbtql42x 2462264 2462263 2026-09-30T14:14:45Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1332.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462264 wikitext text/x-wiki s3snealzs202dbo9fm05ctnw51svgtb 2462265 2462264 2026-09-30T14:14:54Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1332.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462265 wikitext text/x-wiki mnlam2r8bkcdxsj24z3i5fz0f5fr5kp 2462266 2462265 2026-09-30T14:15:03Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1333.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462266 wikitext text/x-wiki n16a2h4mppaha2btfq6mrah2vw3gkft 2462267 2462266 2026-09-30T14:15:12Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1333.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462267 wikitext text/x-wiki nccfsc5j1f0q32deqvv9s34674knffl 2462268 2462267 2026-09-30T14:15:21Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1334.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462268 wikitext text/x-wiki mb5c05snzg8i0v6fg4p7ruvikocrtw9 2462269 2462268 2026-09-30T14:15:29Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1334.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2462269 wikitext text/x-wiki 6mxiiz5y4zyyu1vsd9rh6damxgz85kf 2462270 2462269 2026-09-30T14:15:38Z Stashbot 7414 dkertesz@cumin1004: START - Cookbook sre.cdn.roll-upgrade-haproxy rolling upgrade of HAProxy on A:cp-upload_magru and A:cp - upgrade haproxy to 3.2.25-1 (T439667) 2462270 wikitext text/x-wiki gt3cf4w0rfgkcrepqyo32ti0n9u0ku4 2462271 2462270 2026-09-30T14:16:44Z Stashbot 7414 jforrester@deploy1003: jforrester: Backport for [[gerrit:1346619|mlwiki: Enable Abstract Wikimedia client mode]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462271 wikitext text/x-wiki gh7667u5pbdtgvdea00c9sd5xav1z1o 2462272 2462271 2026-09-30T14:16:49Z Stashbot 7414 ecarg@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2462272 wikitext text/x-wiki 0h0jqnmwvzs691lb6u118wupe0i6ipp 2462273 2462272 2026-09-30T14:17:07Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.reboot-nodes rolling reboot on D{wikikube-worker133[0-4].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) 2462273 wikitext text/x-wiki f4101omwkuk0skh7gdlxx10zh24mwnn 2462274 2462273 2026-09-30T14:17:11Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1330.eqiad.wmnet 2462274 wikitext text/x-wiki cnpltg2ehdpxh3w2i6fa7h2hs6prq0t 2462275 2462274 2026-09-30T14:17:12Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host wikikube-worker1330.eqiad.wmnet 2462275 wikitext text/x-wiki ady17srlyug2zll92sbdpy9f9xp55vc 2462278 2462275 2026-09-30T14:18:49Z Stashbot 7414 jforrester@deploy1003: jforrester: Continuing with deployment 2462278 wikitext text/x-wiki r6j17v5voxu6l1grpike78snviipjju 2462280 2462278 2026-09-30T14:20:45Z Stashbot 7414 ecarg@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2462280 wikitext text/x-wiki 3w5hr78i9ay4t4fbuhp0ehg3pt96a63 2462281 2462280 2026-09-30T14:21:03Z Stashbot 7414 ecarg@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2462281 wikitext text/x-wiki migv25tbheeepumgopt6ah38286uvkg 2462283 2462281 2026-09-30T14:21:26Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply 2462283 wikitext text/x-wiki glvdqfh2lhawbcw6d6k2jnyo8fny6gi 2462284 2462283 2026-09-30T14:21:29Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply 2462284 wikitext text/x-wiki t4ayhv059i65c9uut71ti6rua6ikw1r 2462285 2462284 2026-09-30T14:21:32Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2462285 wikitext text/x-wiki as62q2vkr4so30z6kg2jjn1s1bgglme 2462286 2462285 2026-09-30T14:21:44Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462286 wikitext text/x-wiki lsfnpjyvs4p9fabevf18focknbmgrg4 2462287 2462286 2026-09-30T14:21:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462287 wikitext text/x-wiki 6y53cc5zthecs6n74qu08ypakqyiblo 2462288 2462287 2026-09-30T14:22:07Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply 2462288 wikitext text/x-wiki rultrktlozm936edkrxyg15la7tqonc 2462289 2462288 2026-09-30T14:22:16Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2462289 wikitext text/x-wiki t8l5fqwrujd947qj4zjqy6i47v7twat 2462290 2462289 2026-09-30T14:22:23Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply 2462290 wikitext text/x-wiki meqvp8mseo9vt3bff97goo8papaulaq 2462291 2462290 2026-09-30T14:22:33Z Stashbot 7414 ecarg@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2462291 wikitext text/x-wiki 41jpk1av5ad8cbxr47lsaflx9bq7c0n 2462292 2462291 2026-09-30T14:23:12Z Stashbot 7414 ecarg@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2462292 wikitext text/x-wiki 6uwth0spojhddl92jjx46p6dhidtydu 2462294 2462292 2026-09-30T14:24:07Z Stashbot 7414 jforrester@deploy1003: Finished scap sync-world: Backport for [[gerrit:1346619|mlwiki: Enable Abstract Wikimedia client mode]] (duration: 11m 31s) 2462294 wikitext text/x-wiki 8et01xhhhr22spruamf7ruq9q0xnjqo 2462295 2462294 2026-09-30T14:24:25Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462295 wikitext text/x-wiki 57as7a28quo4kj91xgjy2ohy4nbfcqz 2462296 2462295 2026-09-30T14:24:33Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply 2462296 wikitext text/x-wiki fnqtwt2oqbi4nvno83wqo8aqhjv43w4 2462297 2462296 2026-09-30T14:24:34Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462297 wikitext text/x-wiki l4z6yac6vorqevct9vjnqn33wbhltw8 2462298 2462297 2026-09-30T14:24:43Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462298 wikitext text/x-wiki mubxxnz7k3xht5n5raa63fbui2k0yxk 2462299 2462298 2026-09-30T14:24:53Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462299 wikitext text/x-wiki 94xkntwggypqc4w99hvlsqoycll4vyd 2462300 2462299 2026-09-30T14:25:12Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462300 wikitext text/x-wiki 1fkwpl12mv6p9o46il9lhurl69ijnjw 2462301 2462300 2026-09-30T14:25:22Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462301 wikitext text/x-wiki fycwazk2xpt68bafufhf5vc9kd5m3nw 2462302 2462301 2026-09-30T14:25:25Z Stashbot 7414 jforrester@deploy1003: Started scap sync-world: Backport for [[gerrit:1346622|mlwiki: Enable Wikifunctions client mode]] 2462302 wikitext text/x-wiki 3k4e8cmfvkrqobvznh5ormslaeyhxl4 2462303 2462302 2026-09-30T14:25:30Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply 2462303 wikitext text/x-wiki b0et10bn1d1rbchnb261lq2azct1683 2462304 2462303 2026-09-30T14:25:31Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462304 wikitext text/x-wiki rrbn86sn1m9x5bfw5k50ludqsecfr2p 2462305 2462304 2026-09-30T14:25:40Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART 2462305 wikitext text/x-wiki 9w3gjb13i0cgcjkl525xy4luq7kfqk4 2462307 2462305 2026-09-30T14:26:38Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply 2462307 wikitext text/x-wiki 74tbxg77cxq5b9ep2xy4ez6wup0louc 2462308 2462307 2026-09-30T14:26:48Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply 2462308 wikitext text/x-wiki sh1vc8ei0xr706v74dhqs8nerza40la 2462309 2462308 2026-09-30T14:27:19Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1330.eqiad.wmnet 2462309 wikitext text/x-wiki 5b1jjhoy7b8teso1hd8z41nwcq5xwtd 2462310 2462309 2026-09-30T14:27:20Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) pool for host wikikube-worker1330.eqiad.wmnet 2462310 wikitext text/x-wiki c7qgxo18zpw0uwkv7olzbkmug6bf9uf 2462311 2462310 2026-09-30T14:27:26Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1331.eqiad.wmnet 2462311 wikitext text/x-wiki mwepgq8lobb6wct7180ofes27ls8n9b 2462312 2462311 2026-09-30T14:27:27Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host wikikube-worker1331.eqiad.wmnet 2462312 wikitext text/x-wiki 5ahma7o9nzdyudev406r9yr64sij5na 2462313 2462312 2026-09-30T14:29:37Z Stashbot 7414 jforrester@deploy1003: jforrester: Backport for [[gerrit:1346622|mlwiki: Enable Wikifunctions client mode]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462313 wikitext text/x-wiki alooo104dejl94cvtna33xauy8rzz7y 2462314 2462313 2026-09-30T14:29:48Z Stashbot 7414 ecarg@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2462314 wikitext text/x-wiki f32o4i60l8qpmyc6m00x2fq2smcjixr 2462315 2462314 2026-09-30T14:29:59Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.reboot-nodes rolling reboot on D{wikikube-worker236[5-9].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2462315 wikitext text/x-wiki qz3c7ie2870vzoc9uare1j04nptdw7m 2462316 2462315 2026-09-30T14:30:01Z Stashbot 7414 ecarg@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2462316 wikitext text/x-wiki 6kgq5vnhplaqtodewuaqrqyh39eat71 2462317 2462316 2026-09-30T14:30:02Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2365.codfw.wmnet 2462317 wikitext text/x-wiki mu19jax1i0jqha9eb6iucnlwrryb35t 2462318 2462317 2026-09-30T14:30:03Z Stashbot 7414 jforrester@deploy1003: jforrester: Continuing with deployment 2462318 wikitext text/x-wiki 3d89hgq4za991cynpbcywdi13hiv2yj 2462319 2462318 2026-09-30T14:30:26Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2462319 wikitext text/x-wiki 3qid9p4n3jeglu6as0eril1x90eh684 2462320 2462319 2026-09-30T14:30:38Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2365.codfw.wmnet 2462320 wikitext text/x-wiki iewdt7rgrp9nps6xhdjelw8lrg2ovwq 2462322 2462320 2026-09-30T14:31:09Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2462322 wikitext text/x-wiki dxf0jzbwfn3p0psfu8dy1jhgwhujswq 2462323 2462322 2026-09-30T14:31:21Z Stashbot 7414 ecarg@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2462323 wikitext text/x-wiki kg5msg5mogw3maru3gwz896mz51eyyc 2462324 2462323 2026-09-30T14:32:00Z Stashbot 7414 ecarg@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2462324 wikitext text/x-wiki frpp9jia64q69w7tcjtkj8madfsq2p3 2462325 2462324 2026-09-30T14:35:17Z Stashbot 7414 jforrester@deploy1003: Finished scap sync-world: Backport for [[gerrit:1346622|mlwiki: Enable Wikifunctions client mode]] (duration: 09m 51s) 2462325 wikitext text/x-wiki nb2fggosv10dfwperne619l64ie6dwo 2462326 2462325 2026-09-30T14:36:14Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1331.eqiad.wmnet 2462326 wikitext text/x-wiki h0annx6kuhjvpjpg4qw8zcuututbhuj 2462327 2462326 2026-09-30T14:36:15Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) pool for host wikikube-worker1331.eqiad.wmnet 2462327 wikitext text/x-wiki sipm4h620rc0ok9vfs0dira99919xrj 2462328 2462327 2026-09-30T14:36:21Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1332.eqiad.wmnet 2462328 wikitext text/x-wiki lhh74luopuytdnahn5i1xoqlkl9vqg8 2462329 2462328 2026-09-30T14:36:22Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host wikikube-worker1332.eqiad.wmnet 2462329 wikitext text/x-wiki lguwlbgqdeckfud507cv8yyndi9gyqd 2462330 2462329 2026-09-30T14:36:28Z Stashbot 7414 ecarg@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2462330 wikitext text/x-wiki jd1v33uxfc8mu75h0zojdbqyiyz1ime 2462331 2462330 2026-09-30T14:37:01Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462331 wikitext text/x-wiki ge6ja9798rdvjcymyiygy75y0tnssib 2462332 2462331 2026-09-30T14:37:06Z Stashbot 7414 ecarg@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2462332 wikitext text/x-wiki 1yv5bjzvsroj7dsnp52zpqzh0kzr5ch 2462333 2462332 2026-09-30T14:37:27Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462333 wikitext text/x-wiki 42mr3zcufzqb5sw6x14cz6xfz8wut7p 2462334 2462333 2026-09-30T14:37:38Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2365.codfw.wmnet 2462334 wikitext text/x-wiki ceqc9w1k6j7ckpwultt9hj7l2pfhms9 2462335 2462334 2026-09-30T14:37:39Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2365.codfw.wmnet 2462335 wikitext text/x-wiki mfm099hcme84gzvx11nyjgb10x0807u 2462336 2462335 2026-09-30T14:37:42Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2462336 wikitext text/x-wiki ogt1wvahs32lc5flao20ft2vfdpz81l 2462337 2462336 2026-09-30T14:37:46Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2366.codfw.wmnet 2462337 wikitext text/x-wiki 53xpcn36gw0t6ffmi1xb7nqwwrtr8xv 2462338 2462337 2026-09-30T14:38:04Z Stashbot 7414 andrew@cumin2003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcephosd1043.eqiad.wmnet with OS bookworm 2462338 wikitext text/x-wiki aifn1t160ko2bt2b0efa9e1gq3vl4ii 2462339 2462338 2026-09-30T14:38:21Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2366.codfw.wmnet 2462339 wikitext text/x-wiki 3cnb24b211km2nf0j74viqpq9k4tm33 2462340 2462339 2026-09-30T14:43:16Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462340 wikitext text/x-wiki f8pxl8qosvkhl8kymxmlkz2k9lu9olh 2462341 2462340 2026-09-30T14:43:26Z Stashbot 7414 oblivian@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'apply'. 2462341 wikitext text/x-wiki ad2nrqbpq8w6a6mxfk5u2zol32vxuyf 2462342 2462341 2026-09-30T14:43:29Z Stashbot 7414 oblivian@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. 2462342 wikitext text/x-wiki cedb1i0zf34djde7ulsyuhsxsxbbh52 2462343 2462342 2026-09-30T14:44:06Z Stashbot 7414 oblivian@deploy1003: helmfile [staging-codfw] START helmfile.d/admin 'apply'. 2462343 wikitext text/x-wiki i92r0d46w95kb95hc48dcgh632xt0lo 2462344 2462343 2026-09-30T14:44:10Z Stashbot 7414 oblivian@deploy1003: helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. 2462344 wikitext text/x-wiki lakzzjuh505lebr801ufoibodojzm49 2462345 2462344 2026-09-30T14:44:39Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1346537|Add script to drop duplicate oldimage/filerevision rows (T67264 T428406)]], [[gerrit:1346538|Add script to drop duplicate oldimage/filerevision rows (T67264 T428406)]] 2462345 wikitext text/x-wiki 5b5hwabmm2sj7afq8q9xg9nwm6tkz7f 2462346 2462345 2026-09-30T14:44:41Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1332.eqiad.wmnet 2462346 wikitext text/x-wiki ayjlot33hsblskoceqccuwfly55axhp 2462348 2462346 2026-09-30T14:44:48Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) pool for host wikikube-worker1332.eqiad.wmnet 2462348 wikitext text/x-wiki ov6csi41p0x24255gta8ax5tx3wy8ox 2462349 2462348 2026-09-30T14:44:49Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1333.eqiad.wmnet 2462349 wikitext text/x-wiki qi7gh1th6xmpppj142rw3xe7p344h0w 2462350 2462349 2026-09-30T14:44:50Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host wikikube-worker1333.eqiad.wmnet 2462350 wikitext text/x-wiki 0cnxndigr8i2w4jdxgblr4pm98qwq7g 2462351 2462350 2026-09-30T14:45:00Z Stashbot 7414 oblivian@deploy1003: helmfile [codfw] START helmfile.d/admin 'apply'. 2462351 wikitext text/x-wiki 77ppfe5w3q97vxv5zanta9kjb6tqcak 2462352 2462351 2026-09-30T14:45:04Z Stashbot 7414 oblivian@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'apply'. 2462352 wikitext text/x-wiki kkgq81npdesw9ziypzfkktv8hq7spki 2462353 2462352 2026-09-30T14:45:22Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2366.codfw.wmnet 2462353 wikitext text/x-wiki 0j1b4isv0p2y5pff3v3z9x84456nveg 2462354 2462353 2026-09-30T14:45:23Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2366.codfw.wmnet 2462354 wikitext text/x-wiki o1lhr6ezjgoxbo1zbc297paijdx4ofk 2462355 2462354 2026-09-30T14:45:24Z Stashbot 7414 oblivian@deploy1003: helmfile [eqiad] START helmfile.d/admin 'apply'. 2462355 wikitext text/x-wiki cmddr2fzv2m8sg5ho15hr8zxjjy97wc 2462356 2462355 2026-09-30T14:45:26Z Stashbot 7414 oblivian@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'apply'. 2462356 wikitext text/x-wiki b5q26v0s1ve6o5cujt1cxxgc9rez8jw 2462357 2462356 2026-09-30T14:45:29Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2367.codfw.wmnet 2462357 wikitext text/x-wiki eks9hje1zijnm33csv8zezkw07lxm2r 2462358 2462357 2026-09-30T14:46:04Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2367.codfw.wmnet 2462358 wikitext text/x-wiki kxgooclt3gt0ktra8lt67aa3iapy7ie 2462359 2462358 2026-09-30T14:47:59Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2462359 wikitext text/x-wiki mim66ogy6vl7obbxmqk6gecoxhvmdji 2462360 2462359 2026-09-30T14:48:33Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2462360 wikitext text/x-wiki ll47fso5yc0ss37nu3b9622b9cgkxmw 2462361 2462360 2026-09-30T14:48:45Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1346537|Add script to drop duplicate oldimage/filerevision rows (T67264 T428406)]], [[gerrit:1346538|Add script to drop duplicate oldimage/filerevision rows (T67264 T428406)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462361 wikitext text/x-wiki ftsw0xqkqagq9y1lq8kupdwe1mqnn8b 2462362 2462361 2026-09-30T14:48:50Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2462362 wikitext text/x-wiki 6c9w5e6s9a4d8pfqbgrgie90zahafem 2462363 2462362 2026-09-30T14:49:10Z Stashbot 7414 ecarg@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2462363 wikitext text/x-wiki jyx1mupmp2xj1yc3kltkwtzg09mn3m2 2462364 2462363 2026-09-30T14:49:14Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2462364 wikitext text/x-wiki tiwbebjwjld60f56jv4s0apl310w2b3 2462365 2462364 2026-09-30T14:49:40Z Stashbot 7414 ecarg@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2462365 wikitext text/x-wiki 2ipt41uekvjv0df1z38rpoujyacr8z3 2462366 2462365 2026-09-30T14:51:29Z Stashbot 7414 dkertesz@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-haproxy (exit_code=0) rolling upgrade of HAProxy on A:cp-upload_magru and A:cp - upgrade haproxy to 3.2.25-1 (T439667) 2462366 wikitext text/x-wiki ho4bnii82btzqj1j92z1kbbc6qosrzw 2462367 2462366 2026-09-30T14:53:17Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2367.codfw.wmnet 2462367 wikitext text/x-wiki nbdkjom29ddaajgpfxyyxqbtkb0nqmx 2462368 2462367 2026-09-30T14:53:18Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2367.codfw.wmnet 2462368 wikitext text/x-wiki iwt32shixlh88r8bjpvh7gc3sl683q3 2462369 2462368 2026-09-30T14:53:25Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2368.codfw.wmnet 2462369 wikitext text/x-wiki sndi3a2binh2myc7g1j28aj4732sfhh 2462370 2462369 2026-09-30T14:53:26Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1333.eqiad.wmnet 2462370 wikitext text/x-wiki 07j54gu6btdieeq6retpmsuz25g3jf8 2462371 2462370 2026-09-30T14:53:27Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) pool for host wikikube-worker1333.eqiad.wmnet 2462371 wikitext text/x-wiki oc4r4xx4zmpxgmvg06a4loz4kncevvm 2462372 2462371 2026-09-30T14:53:32Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1334.eqiad.wmnet 2462372 wikitext text/x-wiki 8o9ut8d2aupsamrut12dqctphkmfovx 2462373 2462372 2026-09-30T14:53:33Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host wikikube-worker1334.eqiad.wmnet 2462373 wikitext text/x-wiki 6pw6r157g2cjnwdoedi8yyr2go5dfvq 2462374 2462373 2026-09-30T14:53:58Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2368.codfw.wmnet 2462374 wikitext text/x-wiki bokteusp8r8cijpvqhj76929bqvk73n 2462375 2462374 2026-09-30T14:54:33Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1346537|Add script to drop duplicate oldimage/filerevision rows (T67264 T428406)]], [[gerrit:1346538|Add script to drop duplicate oldimage/filerevision rows (T67264 T428406)]] (duration: 09m 54s) 2462375 wikitext text/x-wiki 02rd9u5xa5bp6rxmvulxjqav7jqqiyd 2462376 2462375 2026-09-30T14:58:51Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.misc-clusters.roll-restart-reboot-docker-registry rolling restart_daemons on A:docker-registry 2462376 wikitext text/x-wiki 7yid9mghuy67mv1dy77bhzxnpu7xlvy 2462377 2462376 2026-09-30T15:01:09Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.misc-clusters.roll-restart-reboot-docker-registry (exit_code=0) rolling restart_daemons on A:docker-registry 2462377 wikitext text/x-wiki 017h60tdsvcca3agkfy891dht342vag 2462378 2462377 2026-09-30T15:01:18Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2368.codfw.wmnet 2462378 wikitext text/x-wiki enc4417v5zs5htpe1nl5do36g0y2caz 2462379 2462378 2026-09-30T15:01:19Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2368.codfw.wmnet 2462379 wikitext text/x-wiki 4w6pbbitr08849dhm7o41f3ipjy36nz 2462380 2462379 2026-09-30T15:01:25Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2369.codfw.wmnet 2462380 wikitext text/x-wiki 7pe3ytpq8tp6dia550jhz8aafd6vpct 2462381 2462380 2026-09-30T15:02:02Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1334.eqiad.wmnet 2462381 wikitext text/x-wiki pgeqm15a8duuo57gne5yw81q3vwcj2p 2462382 2462381 2026-09-30T15:02:03Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) pool for host wikikube-worker1334.eqiad.wmnet 2462382 wikitext text/x-wiki ntarii2hhjvdkbqxv9kqjf18almglci 2462383 2462382 2026-09-30T15:02:04Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on D{wikikube-worker133[0-4].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) 2462383 wikitext text/x-wiki jixf69yxdvbyhe3ocux3lwh5g03p468 2462384 2462383 2026-09-30T15:02:34Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2369.codfw.wmnet 2462384 wikitext text/x-wiki 3sx8n6mtmj01ej3dpgr9eu0bfma8fwi 2462385 2462384 2026-09-30T15:03:08Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.cdn.roll-restart-reboot-ncredir rolling restart_daemons on A:ncredir 2462385 wikitext text/x-wiki pvz9z3tg10tf8ey2mjqrhbl8u65vkps 2462386 2462385 2026-09-30T15:09:01Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462386 wikitext text/x-wiki 889hz7z2oy793vmfhbggpnyd8q2lfyb 2462387 2462386 2026-09-30T15:09:11Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2462387 wikitext text/x-wiki 2ygqj4aedxjf7i7bmihh3uhxu4sc4go 2462388 2462387 2026-09-30T15:09:35Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2369.codfw.wmnet 2462388 wikitext text/x-wiki lnlejwsb364jj8phbsa4hszsjmnti6o 2462389 2462388 2026-09-30T15:09:36Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2369.codfw.wmnet 2462389 wikitext text/x-wiki tdfvwwpu33jddzxyqy0sbkumtt75b9c 2462390 2462389 2026-09-30T15:09:38Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on D{wikikube-worker236[5-9].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) 2462390 wikitext text/x-wiki 4lqxfjz1wiz6yyycuv4kj7pnz917mga 2462391 2462390 2026-09-30T15:11:54Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.cdn.roll-restart-reboot-ncredir (exit_code=0) rolling restart_daemons on A:ncredir 2462391 wikitext text/x-wiki 7d9hp46auwxtbmh3xsbt6zzhray5g39 2462392 2462391 2026-09-30T15:15:15Z Stashbot 7414 andrew@cumin2003: START - Cookbook sre.hosts.reimage for host cloudcephosd1043.eqiad.wmnet with OS bookworm 2462392 wikitext text/x-wiki i8c4k1aaw2wb5sw9hvrzg0om8uj7znd 2462393 2462392 2026-09-30T15:25:43Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462393 wikitext text/x-wiki bf2dp6n29xun84cytpehswluu1i4ch9 2462394 2462393 2026-09-30T15:25:45Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462394 wikitext text/x-wiki tm8r4iuz4qmg27gcnj1oo79epsd5w8i 2462395 2462394 2026-09-30T15:41:07Z Stashbot 7414 oblivian@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-timeline: apply 2462395 wikitext text/x-wiki io140p4atpgn64t84a6pzli2red9bgs 2462396 2462395 2026-09-30T15:41:26Z Stashbot 7414 oblivian@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-timeline: apply 2462396 wikitext text/x-wiki 93473gupor2syz3k9rq08h0tykuhqpo 2462397 2462396 2026-09-30T15:42:13Z Stashbot 7414 andrew@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1043.eqiad.wmnet with reason: host reimage 2462397 wikitext text/x-wiki qmin7o2kqr4rlvw7beqt3tkdaeju2za 2462398 2462397 2026-09-30T15:47:26Z Stashbot 7414 andrew@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1043.eqiad.wmnet with reason: host reimage 2462398 wikitext text/x-wiki lotc0hexjoq0a6jidnqhwlkw8no5t6y 2462399 2462398 2026-09-30T15:47:49Z Stashbot 7414 andrew@cumin2003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcephosd1043.eqiad.wmnet with OS bookworm 2462399 wikitext text/x-wiki snv1ni18o41kc04r0m0mx51z0plckgy 2462400 2462399 2026-09-30T15:49:54Z Stashbot 7414 andrew@cumin2003: START - Cookbook sre.hosts.reimage for host cloudcephosd1043.eqiad.wmnet with OS bookworm 2462400 wikitext text/x-wiki eo0dzs34ozj06bmcgglmnji0vkl411o 2462402 2462400 2026-09-30T16:08:15Z Stashbot 7414 andrew@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1043.eqiad.wmnet with reason: host reimage 2462402 wikitext text/x-wiki 3rvh3gnrbyouc9s4qqm0i9l913fnjt4 2462403 2462402 2026-09-30T16:11:15Z Stashbot 7414 aaron@deploy1003: helmfile [staging] START helmfile.d/services/developer-portal: apply 2462403 wikitext text/x-wiki f2qpx163enmfrzd3tfkx2qwhu3bc3wk 2462404 2462403 2026-09-30T16:11:31Z Stashbot 7414 aaron@deploy1003: helmfile [staging] DONE helmfile.d/services/developer-portal: apply 2462404 wikitext text/x-wiki 4ncz07rsk4j6lm4v7w7gt9ysne10nju 2462406 2462404 2026-09-30T16:12:39Z Stashbot 7414 andrew@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1043.eqiad.wmnet with reason: host reimage 2462406 wikitext text/x-wiki 95308cr97m33m264k3kcsq1aupdi91v 2462407 2462406 2026-09-30T16:13:18Z Stashbot 7414 aaron@deploy1003: helmfile [codfw] START helmfile.d/services/developer-portal: apply 2462407 wikitext text/x-wiki 11ouhvqgib6szavpbqfupaw9mk9trfa 2462408 2462407 2026-09-30T16:13:45Z Stashbot 7414 aaron@deploy1003: helmfile [codfw] DONE helmfile.d/services/developer-portal: apply 2462408 wikitext text/x-wiki p9lf7h4s6d4nfmurtd60gm4u0esqb8k 2462409 2462408 2026-09-30T16:14:38Z Stashbot 7414 aaron@deploy1003: helmfile [eqiad] START helmfile.d/services/developer-portal: apply 2462409 wikitext text/x-wiki 8wkuarhtrgxbwv4ju3ztxzy6ard8ba0 2462410 2462409 2026-09-30T16:14:51Z Stashbot 7414 aaron@deploy1003: helmfile [eqiad] DONE helmfile.d/services/developer-portal: apply 2462410 wikitext text/x-wiki 6bqwr2bsgni0p52lt3skmpf8om38nil 2462413 2462410 2026-09-30T16:38:34Z Stashbot 7414 andrew@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudcephosd1043.eqiad.wmnet with OS bookworm 2462413 wikitext text/x-wiki tsa9opymcururh5f9yr8fue0xyev2cn 2462414 2462413 2026-09-30T16:51:51Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2462414 wikitext text/x-wiki lisfktkvowi9iktgfcldv8vdy7uiwmn 2462415 2462414 2026-09-30T16:52:21Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2462415 wikitext text/x-wiki kxy7o2mmd2xlflir0jhgzkgume9bnnh 2462418 2462415 2026-09-30T17:08:57Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2462418 wikitext text/x-wiki t187ef6nnr24m2j1334czwxlwf85ejt 2462419 2462418 2026-09-30T17:09:29Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2462419 wikitext text/x-wiki cksh0o1us5iy8svb0mx7xyxmrxtlha4 2462420 2462419 2026-09-30T17:13:17Z Stashbot 7414 bking@dns2004: START - running authdns-update 2462420 wikitext text/x-wiki cjf0paxs6g0z1vb736vky1m092ulohk 2462421 2462420 2026-09-30T17:15:50Z Stashbot 7414 bking@dns2004: END - running authdns-update 2462421 wikitext text/x-wiki k6vd8kvmshphbvdlxt904agcieson9c 2462422 2462421 2026-09-30T17:18:20Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.wdqs.restart 2462422 wikitext text/x-wiki csb9u3kijb24fjuhkdfsm05bsrk6p8x 2462423 2462422 2026-09-30T17:18:33Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.wdqs.restart 2462423 wikitext text/x-wiki dvptfyd3wrov5a5r61sg4bxqmegym8q 2462424 2462423 2026-09-30T17:21:29Z Stashbot 7414 brett@cumin1004: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on A:cp-text_codfw - 7.1.1-2~bpo13+wmf3 () 2462424 wikitext text/x-wiki kizq56wnlqhocv5gvkh9hv6x7ghep6f 2462425 2462424 2026-09-30T17:21:34Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462425 wikitext text/x-wiki 04mud8eqy6crav8cfybou6jqn3nxljm 2462426 2462425 2026-09-30T17:21:38Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462426 wikitext text/x-wiki i2496846holwj17fnogxkz81zwnxcvx 2462427 2462426 2026-09-30T17:21:38Z Stashbot 7414 brett@cumin1004: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on A:cp-upload_codfw - 7.1.1-2~bpo13+wmf3 () 2462427 wikitext text/x-wiki 84mqbr7tbzv5so1n5brhmnr3o1msguc 2462429 2462427 2026-09-30T17:28:06Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) 2462429 wikitext text/x-wiki js7bjja4pcsn5i5ow4ljsv8xq4qokyq 2462430 2462429 2026-09-30T17:33:41Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462430 wikitext text/x-wiki dutyyjl4h1up8ott5kpg19m0hlpnh0v 2462431 2462430 2026-09-30T17:33:43Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462431 wikitext text/x-wiki cua3ytypk0oxy5rhr4wx8bd9w68y3av 2462432 2462431 2026-09-30T17:34:36Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462432 wikitext text/x-wiki bbeux2godg3dkm0r8s8zpwy5m7wlglf 2462433 2462432 2026-09-30T17:34:39Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462433 wikitext text/x-wiki 6v83zcfh6cbp5wwnbe6rk3yzmn8cxm8 2462435 2462433 2026-09-30T17:47:22Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) 2462435 wikitext text/x-wiki e4ewamxuj644zyj4vth9ld9h2cgk7k3 2462442 2462435 2026-09-30T18:10:38Z Stashbot 7414 brennen: 1.47.0-wmf.22 train status (T438218): no current blockers, rolling to group1 2462442 wikitext text/x-wiki b2mm89yumfkvascamxlmrn0jojxj26p 2462450 2462442 2026-09-30T18:22:26Z Stashbot 7414 brennen@deploy1003: Rolling back deployment 2462450 wikitext text/x-wiki mledy11rswpeonv3uszedh7flxfhw84 2462451 2462450 2026-09-30T18:24:48Z Stashbot 7414 brennen@deploy1003: rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.22 refs T438218 2462451 wikitext text/x-wiki 73ayda31knbfs32jxp2qd8qhihlk8m8 2462455 2462451 2026-09-30T18:37:09Z Stashbot 7414 jhathaway: flipping flag to disable legacy facts in puppet prod 2462455 wikitext text/x-wiki rrdm3dzw8qtebntnel7hwe2vjf0ab2w 2462456 2462455 2026-09-30T18:40:21Z Stashbot 7414 brennen@deploy1003: rebuilt and synchronized wikiversions files: group0 to 1.47.0-wmf.22 refs T438218 2462456 wikitext text/x-wiki n7agmasgdr0sqsdhco4cn0h8k8zvtuh 2462460 2462456 2026-09-30T18:52:05Z Stashbot 7414 brennen@deploy1003: rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.22 refs T438218 2462460 wikitext text/x-wiki 2b6rhsbsytsnq9h4mxwrcf8b2wci7bh 2462461 2462460 2026-09-30T18:55:09Z Stashbot 7414 dzahn@deploy1003: helmfile [staging] START helmfile.d/services/miscweb: apply 2462461 wikitext text/x-wiki 3y2z98mjtrj1dsrqf286pux09a54qiz 2462462 2462461 2026-09-30T18:55:24Z Stashbot 7414 dzahn@deploy1003: helmfile [staging] DONE helmfile.d/services/miscweb: apply 2462462 wikitext text/x-wiki jniy2j1ao23icvbvcfj3z7wvezj0vrd 2462463 2462462 2026-09-30T18:55:35Z Stashbot 7414 dzahn@deploy1003: helmfile [codfw] START helmfile.d/services/miscweb: apply 2462463 wikitext text/x-wiki q30sss8mhg59xyt9hwi387klf2chpqu 2462464 2462463 2026-09-30T18:56:08Z Stashbot 7414 dzahn@deploy1003: helmfile [codfw] DONE helmfile.d/services/miscweb: apply 2462464 wikitext text/x-wiki kpf8tya8xaunjrxt95n2y2f7mlfwk1g 2462465 2462464 2026-09-30T18:56:27Z Stashbot 7414 dzahn@deploy1003: helmfile [eqiad] START helmfile.d/services/miscweb: apply 2462465 wikitext text/x-wiki 5mbvi3sh9qctprk7r74x3s2gouwxywe 2462466 2462465 2026-09-30T18:56:38Z Stashbot 7414 dzahn@deploy1003: helmfile [eqiad] DONE helmfile.d/services/miscweb: apply 2462466 wikitext text/x-wiki 888kk2gj2kd6nn5xl2r3t7kzh0bg467 2462471 2462466 2026-09-30T19:45:48Z Stashbot 7414 jhathaway: reverting legacy facts disablement, discovered some remaining usage 2462471 wikitext text/x-wiki ay05vwtjiu2q267xt8gjp85pnnrdjmu 2462472 2462471 2026-09-30T19:50:39Z Stashbot 7414 brett@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on A:cp-upload_codfw - 7.1.1-2~bpo13+wmf3 () 2462472 wikitext text/x-wiki 8lvow330h81rh0ygfsb12mqhxlkw56k 2462473 2462472 2026-09-30T20:05:40Z Stashbot 7414 kemayo@deploy1003: helmfile [staging] START helmfile.d/services/editcheck-headless: apply 2462473 wikitext text/x-wiki ix3bc0qrp4urtbybt81z41wm2q7n7me 2462474 2462473 2026-09-30T20:06:26Z Stashbot 7414 kemayo@deploy1003: helmfile [staging] DONE helmfile.d/services/editcheck-headless: apply 2462474 wikitext text/x-wiki puylbgab51dql0teddko6zid7ju35dt 2462475 2462474 2026-09-30T20:11:36Z Stashbot 7414 mstyles@deploy1003: Started scap sync-world: Backport for [[gerrit:1346327|Email confirmation: implement delayed enforcement experiment (T439283)]], [[gerrit:1346328|Email confirmation: Add server-side enforcement (T439283)]], [[gerrit:1346329|Email confirmation: Prepare for next experiment (T439283)]] 2462475 wikitext text/x-wiki rkshno26w1nb3nzd0q9v5wzab0v90fn 2462476 2462475 2026-09-30T20:11:56Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply 2462476 wikitext text/x-wiki qhp6iz5f1tpd274e16w7flsjg6ilitp 2462477 2462476 2026-09-30T20:20:05Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply 2462477 wikitext text/x-wiki of6tcbbtjetmz6sben694lq21o5qozx 2462478 2462477 2026-09-30T20:20:09Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply 2462478 wikitext text/x-wiki sas2wh9yw5vl3c3aw4or3eun319tyie 2462479 2462478 2026-09-30T20:20:42Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply 2462479 wikitext text/x-wiki t9dmxa693idjc94u88fakxhqd6xpfdv 2462480 2462479 2026-09-30T20:31:25Z Stashbot 7414 mstyles@deploy1003: mstyles, catrope: Backport for [[gerrit:1346327|Email confirmation: implement delayed enforcement experiment (T439283)]], [[gerrit:1346328|Email confirmation: Add server-side enforcement (T439283)]], [[gerrit:1346329|Email confirmation: Prepare for next experiment (T439283)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462480 wikitext text/x-wiki 0zcxob5q6rdn34avqt51lw6mh4yds86 2462481 2462480 2026-09-30T20:32:20Z Stashbot 7414 mstyles@deploy1003: mstyles, catrope: Continuing with deployment 2462481 wikitext text/x-wiki kncy4k75ie0q4b76j7bf9atngr6qw7h 2462482 2462481 2026-09-30T20:38:37Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply 2462482 wikitext text/x-wiki ifz5ninr84n14quthjxkg9ttt7uapkt 2462483 2462482 2026-09-30T20:38:56Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply 2462483 wikitext text/x-wiki 9j0mf3qi05dkpa8cmsjkz27qs6fqhoa 2462485 2462483 2026-09-30T20:44:24Z Stashbot 7414 mstyles@deploy1003: Finished scap sync-world: Backport for [[gerrit:1346327|Email confirmation: implement delayed enforcement experiment (T439283)]], [[gerrit:1346328|Email confirmation: Add server-side enforcement (T439283)]], [[gerrit:1346329|Email confirmation: Prepare for next experiment (T439283)]] (duration: 32m 47s) 2462485 wikitext text/x-wiki d55u4mth30t3nm8wxpb44j2gtv57b3i 2462488 2462485 2026-09-30T20:47:42Z Stashbot 7414 arlolra@deploy1003: Started scap sync-world: Backport for [[gerrit:1346763|Set a default for wgSplitParsoidParserCache in prod (T432760)]] 2462488 wikitext text/x-wiki gnqvlolgrujz3v3kww791gn6677lg4w 2462489 2462488 2026-09-30T20:52:03Z Stashbot 7414 arlolra@deploy1003: arlolra: Backport for [[gerrit:1346763|Set a default for wgSplitParsoidParserCache in prod (T432760)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462489 wikitext text/x-wiki k7fba63q2tz6b4cwqxrjwadrmkv3axi 2462490 2462489 2026-09-30T20:52:47Z Stashbot 7414 arlolra@deploy1003: arlolra: Continuing with deployment 2462490 wikitext text/x-wiki drijvhuykeki6bhfbdgn77ccbarkm1i 2462492 2462490 2026-09-30T20:57:55Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply 2462492 wikitext text/x-wiki 2h7tlg50n4wvda8ktrwze1kqq7vjze4 2462493 2462492 2026-09-30T20:58:01Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply 2462493 wikitext text/x-wiki sordsvtt8j28guzzbxew7xkvsq1l13p 2462495 2462493 2026-09-30T21:00:07Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply 2462495 wikitext text/x-wiki 4kqhphk3xw6ux294nnonte72i5cauph 2462496 2462495 2026-09-30T21:00:12Z Stashbot 7414 arlolra@deploy1003: Finished scap sync-world: Backport for [[gerrit:1346763|Set a default for wgSplitParsoidParserCache in prod (T432760)]] (duration: 12m 28s) 2462496 wikitext text/x-wiki tw0v02fc7js2945a3j5caa7silmvjkc 2462497 2462496 2026-09-30T21:00:16Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply 2462497 wikitext text/x-wiki 0xd71uo1csuk5dnjn9xtjreptsgrjd4 2462498 2462497 2026-09-30T21:09:02Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply 2462498 wikitext text/x-wiki q1gmg0aa3p2e7d56sj68t7403wq53i5 2462499 2462498 2026-09-30T21:09:12Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply 2462499 wikitext text/x-wiki ca3xkljt0azyf8255jd2ffwcrhsg60k 2462500 2462499 2026-09-30T21:22:10Z Stashbot 7414 sbassett: Deployed updated mitigation for T435455 2462500 wikitext text/x-wiki j13a5ab5tf8a0fr8yqzh8vney22fux6 2462503 2462500 2026-09-30T22:23:05Z Stashbot 7414 brett@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on A:cp-text_codfw - 7.1.1-2~bpo13+wmf3 () 2462503 wikitext text/x-wiki bozkamzvwv74irf1rqowmnvmqduwt0v 2462527 2462503 2026-10-01T06:18:41Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1347210|AbuseReview: Send the abuse review tag in interaction events (T439238)]], [[gerrit:1347211|AbuseReview: Instrument when content policy match adds tags (T439242)]] 2462527 wikitext text/x-wiki slyxip1iqv3uwxumx39oxragvqcw6s0 2462528 2462527 2026-10-01T06:22:56Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1347210|AbuseReview: Send the abuse review tag in interaction events (T439238)]], [[gerrit:1347211|AbuseReview: Instrument when content policy match adds tags (T439242)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462528 wikitext text/x-wiki hny5g4ybnlz2y4l0ftytr44k0zw3hgs 2462529 2462528 2026-10-01T06:23:51Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2462529 wikitext text/x-wiki grb30c544owblrfpzkho8psgl1fgl71 2462530 2462529 2026-10-01T06:29:13Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347210|AbuseReview: Send the abuse review tag in interaction events (T439238)]], [[gerrit:1347211|AbuseReview: Instrument when content policy match adds tags (T439242)]] (duration: 10m 33s) 2462530 wikitext text/x-wiki jh7qhwfqho5ztok994idj947cxcd49j 2462531 2462530 2026-10-01T06:33:13Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1347223|Add extra UI elements to Special:AbuseReview following user feedback (T439694)]] 2462531 wikitext text/x-wiki mbr2lm5olvdt00xlfp5gxdz5p330cm6 2462532 2462531 2026-10-01T06:35:16Z Stashbot 7414 tappof@cumin1004: START - Cookbook sre.hosts.decommission for hosts kafka-logging1001.eqiad.wmnet 2462532 wikitext text/x-wiki ot37al42ipoivmyb6sp2bx1cpgefrch 2462533 2462532 2026-10-01T06:41:37Z Stashbot 7414 tappof@cumin1004: START - Cookbook sre.dns.netbox 2462533 wikitext text/x-wiki ddk1sv1eudf913u6ytk0j6bqg7y2r73 2462534 2462533 2026-10-01T06:45:58Z Stashbot 7414 tappof@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: kafka-logging1001.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - tappof@cumin1004" 2462534 wikitext text/x-wiki 6s6x6qbmtcj0oj8ig9x5b5gzcx516hy 2462535 2462534 2026-10-01T06:46:12Z Stashbot 7414 tappof@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: kafka-logging1001.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - tappof@cumin1004" 2462535 wikitext text/x-wiki qslsuhvqt4i5e4drcg3yb2lxbvw3kc1 2462536 2462535 2026-10-01T06:46:14Z Stashbot 7414 tappof@cumin1004: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2462536 wikitext text/x-wiki 5wjmqskqyvw0zu626awak7wc7c6hhhw 2462537 2462536 2026-10-01T06:46:16Z Stashbot 7414 tappof@cumin1004: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts kafka-logging1001.eqiad.wmnet 2462537 wikitext text/x-wiki 0exif57zuxj8m2dfwdl5c16a9e21fw0 2462539 2462537 2026-10-01T06:47:13Z Stashbot 7414 tappof@cumin1004: START - Cookbook sre.hosts.decommission for hosts kafka-logging1002.eqiad.wmnet 2462539 wikitext text/x-wiki m6yommyvjfs1e0pwnfz9wfcuncfbiju 2462540 2462539 2026-10-01T06:53:15Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1347223|Add extra UI elements to Special:AbuseReview following user feedback (T439694)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462540 wikitext text/x-wiki 5g0kvzfbcen6lrwn2tzytup1hzlg94i 2462541 2462540 2026-10-01T06:53:24Z Stashbot 7414 tappof@cumin1004: START - Cookbook sre.dns.netbox 2462541 wikitext text/x-wiki noeotrqf21cww15aian3uziswq7frof 2462542 2462541 2026-10-01T06:54:31Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2462542 wikitext text/x-wiki jwtloo3uwvcllbyorou3mslp3mh0vbv 2462543 2462542 2026-10-01T06:57:14Z Stashbot 7414 tappof@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: kafka-logging1002.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - tappof@cumin1004" 2462543 wikitext text/x-wiki drzgsjiysaowy8mllq318efkrqvo4yd 2462544 2462543 2026-10-01T06:57:26Z Stashbot 7414 tappof@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: kafka-logging1002.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - tappof@cumin1004" 2462544 wikitext text/x-wiki bvadxwkh20hmciesc9wpnjerorm9uo4 2462545 2462544 2026-10-01T06:57:27Z Stashbot 7414 tappof@cumin1004: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2462545 wikitext text/x-wiki 2wppfq07ge3v8x7tzdsj3np3pk23pl9 2462546 2462545 2026-10-01T06:57:28Z Stashbot 7414 tappof@cumin1004: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts kafka-logging1002.eqiad.wmnet 2462546 wikitext text/x-wiki k6q39ovjwqwvyc3jgx3mrr3nh3e4qj9 2462547 2462546 2026-10-01T07:06:31Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347223|Add extra UI elements to Special:AbuseReview following user feedback (T439694)]] (duration: 33m 18s) 2462547 wikitext text/x-wiki r70wvxkf2n2asolqx7tyzam866o1bkq 2462548 2462547 2026-10-01T07:07:27Z Stashbot 7414 jmm@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2462548 wikitext text/x-wiki s5zcjnu66rzo6i4stn1kbl5fuqqhn6a 2462549 2462548 2026-10-01T07:09:34Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1346542|CheckUser: Enable ClientHintsInEditRequest on all wikis (T401808)]], [[gerrit:1346541|CheckUser: Enable ClientHintsInEditRequest on group1 (T401808)]] 2462549 wikitext text/x-wiki hbxmefs0p4317lgdiz4out6kt768rg6 2462550 2462549 2026-10-01T07:11:56Z Stashbot 7414 jmm@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2462550 wikitext text/x-wiki 316x5s5w2xkyz5vlsjixtefwmy97t8n 2462551 2462550 2026-10-01T07:15:39Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1346542|CheckUser: Enable ClientHintsInEditRequest on all wikis (T401808)]], [[gerrit:1346541|CheckUser: Enable ClientHintsInEditRequest on group1 (T401808)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462551 wikitext text/x-wiki sqk7r24dxjq7fc32ney2z9be24iczuq 2462552 2462551 2026-10-01T07:16:30Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2462552 wikitext text/x-wiki k529auuh1zf9m6z4we9xtovmrjv1ivh 2462553 2462552 2026-10-01T07:23:50Z Stashbot 7414 jmm@deploy1003: helmfile [codfw] START helmfile.d/services/thumbor: apply 2462553 wikitext text/x-wiki 9jwzcdirz5tqcwr5javsyiu17ufhhsq 2462554 2462553 2026-10-01T07:24:11Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1346542|CheckUser: Enable ClientHintsInEditRequest on all wikis (T401808)]], [[gerrit:1346541|CheckUser: Enable ClientHintsInEditRequest on group1 (T401808)]] (duration: 14m 37s) 2462554 wikitext text/x-wiki qlt30yo0evzrbnoztrj4p4rh99w6tbm 2462555 2462554 2026-10-01T07:28:49Z Stashbot 7414 jmm@deploy1003: helmfile [codfw] DONE helmfile.d/services/thumbor: apply 2462555 wikitext text/x-wiki e1a3ds19sa6qw95tlam188x3f62gwsl 2462556 2462555 2026-10-01T07:29:05Z Stashbot 7414 jmm@deploy1003: helmfile [eqiad] START helmfile.d/services/thumbor: apply 2462556 wikitext text/x-wiki cpe8zb6edeqxaher0ufi0g849ss0rft 2462557 2462556 2026-10-01T07:31:29Z Stashbot 7414 jmm@deploy1003: helmfile [eqiad] DONE helmfile.d/services/thumbor: apply 2462557 wikitext text/x-wiki imz6a9s98upks9x8rvkqel39tak9a1v 2462560 2462557 2026-10-01T07:54:36Z Stashbot 7414 vgutierrez: upload purged 0.25 to trixie-wikimedia (apt.wm.o) 2462560 wikitext text/x-wiki 4xnbyuh47ki2woqz5qx39ex2l6c8l67 2462561 2462560 2026-10-01T08:05:12Z Stashbot 7414 moritzm: installing pcre2/openssl security updates 2462561 wikitext text/x-wiki smxtrgbgvytmk50pnnvdddx27n4rv84 2462562 2462561 2026-10-01T08:22:16Z Stashbot 7414 moritzm: restart FPM on phab1005 to pick up openssl/PCRE security updates 2462562 wikitext text/x-wiki pvmffgxhm77vfkwk4pw1rho22dhu9a6 2462566 2462562 2026-10-01T08:35:01Z Stashbot 7414 dkertesz@cumin1004: START - Cookbook sre.cdn.roll-upgrade-haproxy rolling upgrade of HAProxy on A:cp-ulsfo and A:cp - upgrade haproxy to 3.2.25-1 (T439667) 2462566 wikitext text/x-wiki 3b1zr0jmdkqdjxsyd5lei0x50esvgwv 2462571 2462566 2026-10-01T08:38:58Z Stashbot 7414 hashar@deploy1003: Started deploy [integration/docroot@5cf09c8]: build: Updating brace-expansion to 1.1.21, 2.1.7 2462571 wikitext text/x-wiki gev647k1u0d6qeyapuo2udtt34jhx9i 2462572 2462571 2026-10-01T08:39:09Z Stashbot 7414 hashar@deploy1003: Finished deploy [integration/docroot@5cf09c8]: build: Updating brace-expansion to 1.1.21, 2.1.7 (duration: 00m 10s) 2462572 wikitext text/x-wiki lst1x1g6ptv00e5774fpax713a4w030 2462577 2462572 2026-10-01T09:35:23Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/services/miscweb: apply 2462577 wikitext text/x-wiki lctti7eex6kmnwsbbtghm4upue8r3r4 2462578 2462577 2026-10-01T09:35:45Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/services/miscweb: apply 2462578 wikitext text/x-wiki pniaayl56ryqo2ko5cnz74jsey4n5v8 2462582 2462578 2026-10-01T09:47:04Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/services/miscweb: apply 2462582 wikitext text/x-wiki 4cvodev7fknry8zfjjei3n9zat5a7sz 2462583 2462582 2026-10-01T09:47:50Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/services/miscweb: apply 2462583 wikitext text/x-wiki r0zxu315k6x99a5mm1bmqyrgbffa4ex 2462584 2462583 2026-10-01T09:59:38Z Stashbot 7414 dkertesz@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-haproxy (exit_code=0) rolling upgrade of HAProxy on A:cp-ulsfo and A:cp - upgrade haproxy to 3.2.25-1 (T439667) 2462584 wikitext text/x-wiki 4s20rofi7inmremj6uw6rey7nh0x2uy 2462586 2462584 2026-10-01T10:22:33Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply 2462586 wikitext text/x-wiki k9i3t9ija6a0hyprdo6314x6ppycjmr 2462587 2462586 2026-10-01T10:22:54Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply 2462587 wikitext text/x-wiki oytxq4kgonpdjsckdhkdtc09pdwieev 2462588 2462587 2026-10-01T10:23:51Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . 2462588 wikitext text/x-wiki o3tuuyayil9d9iclc6abha3h35zah7l 2462589 2462588 2026-10-01T10:24:50Z Stashbot 7414 ammarpad@deploy1003: mwscript-k8s job started: extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=wikidatawiki --logwiki=metawiki Jakehekd Ivyeclipse # T439805 2462589 wikitext text/x-wiki rmht74yrycijl2krfw1rqyo3aolyz3w 2462590 2462589 2026-10-01T10:30:28Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply 2462590 wikitext text/x-wiki iq6bvf5hhegm0os1lei01zy04ycecan 2462591 2462590 2026-10-01T10:31:00Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply 2462591 wikitext text/x-wiki mnhyplwzxnxj3sy430r16q39v02r6cy 2462593 2462591 2026-10-01T10:40:02Z Stashbot 7414 javiermonton@deploy1003: Started scap sync-world: Backport for [[gerrit:1346516|stream: mediawiki.job.pageviewTrendingRelativeUpdate (T437875)]] 2462593 wikitext text/x-wiki es1tp8tqvqaqgnj7nnyo6jd4rjn5jlu 2462594 2462593 2026-10-01T10:44:12Z Stashbot 7414 javiermonton@deploy1003: javiermonton: Backport for [[gerrit:1346516|stream: mediawiki.job.pageviewTrendingRelativeUpdate (T437875)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462594 wikitext text/x-wiki bh0dqkk278sv7suardisiv0dt5oexrd 2462595 2462594 2026-10-01T10:45:34Z Stashbot 7414 javiermonton@deploy1003: javiermonton: Continuing with deployment 2462595 wikitext text/x-wiki fxf21atgpr8ffmfvb2igglxmxz8rtan 2462596 2462595 2026-10-01T10:50:53Z Stashbot 7414 javiermonton@deploy1003: Finished scap sync-world: Backport for [[gerrit:1346516|stream: mediawiki.job.pageviewTrendingRelativeUpdate (T437875)]] (duration: 10m 51s) 2462596 wikitext text/x-wiki n8eff1v5gkcipkpuk8blcsah0ia9353 2462597 2462596 2026-10-01T10:57:43Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling restart_daemons on A:swift-fe-codfw 2462597 wikitext text/x-wiki 376tgli0m56k4fv8y7gphk5j5i5tvg6 2462598 2462597 2026-10-01T11:06:15Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling restart_daemons on A:swift-fe-codfw 2462598 wikitext text/x-wiki hb2m3v8vel66qcwzepmanslr2fmxoq8 2462599 2462598 2026-10-01T11:28:05Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'sync'. 2462599 wikitext text/x-wiki jfww2l81je88amure0sedxcv5ftk029 2462600 2462599 2026-10-01T11:29:18Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'sync'. 2462600 wikitext text/x-wiki 15ja6fq1aqkbl7w7okhdrlbuahl3sfu 2462601 2462600 2026-10-01T11:30:22Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . 2462601 wikitext text/x-wiki 44kbyoexej6qaiuxiojs3y6r4tzx47j 2462602 2462601 2026-10-01T11:30:30Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/liftwing-studio: sync 2462602 wikitext text/x-wiki 7pftc5oe8dbzk3h8kzusdl73u2yzo0m 2462603 2462602 2026-10-01T11:31:47Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/liftwing-studio: sync 2462603 wikitext text/x-wiki mqt67k9z8dn9bbrqp0nn5oupi14phog 2462604 2462603 2026-10-01T11:32:22Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.dns.netbox 2462604 wikitext text/x-wiki mifak1wusyap87aak7q1pqw5so7scpo 2462605 2462604 2026-10-01T11:36:43Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add VIPs for ldap-ro-test in eqiad/codfw - jmm@cumin2003" 2462605 wikitext text/x-wiki iqqjn5xlxcxi58psqk9t8ap3clo1ovc 2462606 2462605 2026-10-01T11:43:30Z Stashbot 7414 jforrester@deploy1003: Started scap sync-world: Backport for [[gerrit:1347530|RepoHooks: Fix fresh-result rate limit blocking autoconfirmed users (T437533)]] 2462606 wikitext text/x-wiki tg5nb4lm338xcqaz2p6yp920bdp85if 2462607 2462606 2026-10-01T11:47:40Z Stashbot 7414 jforrester@deploy1003: jforrester: Backport for [[gerrit:1347530|RepoHooks: Fix fresh-result rate limit blocking autoconfirmed users (T437533)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462607 wikitext text/x-wiki 1rs4rsd7o4l4etuizspj86fncu8apqg 2462608 2462607 2026-10-01T11:50:13Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling restart_daemons on A:swift-fe-eqiad 2462608 wikitext text/x-wiki heaylocbv7qtyjj6uqnn4mz5gnbeewr 2462609 2462608 2026-10-01T11:50:28Z Stashbot 7414 jforrester@deploy1003: jforrester: Continuing with deployment 2462609 wikitext text/x-wiki fvac2mym9js6plydjzyywrqxickt28u 2462610 2462609 2026-10-01T11:55:57Z Stashbot 7414 jforrester@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347530|RepoHooks: Fix fresh-result rate limit blocking autoconfirmed users (T437533)]] (duration: 12m 27s) 2462610 wikitext text/x-wiki oti5tl2r0f9pyt3wzsroiowqwqv3p4t 2462611 2462610 2026-10-01T11:58:26Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling restart_daemons on A:swift-fe-eqiad 2462611 wikitext text/x-wiki g1a0g7a30sqeen9m45ewgactnc6kmdv 0 17290 2462411 2461948 2026-09-30T16:27:56Z Stashbot 7414 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1346671 2462411 wikitext text/x-wiki sppwnd9k55plub5znyfsssvv2ttwo22 2462416 2462411 2026-09-30T17:03:01Z Stashbot 7414 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1346701 2462416 wikitext text/x-wiki 223fgfr568rs8yp6zvb2b5wxgcwh6gf 2462434 2462416 2026-09-30T17:46:36Z Stashbot 7414 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1346728 2462434 wikitext text/x-wiki cdwra5k8tt8l4px8e8wlcyj4z6u7oub 2462444 2462434 2026-09-30T18:16:12Z Stashbot 7414 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1346733 2462444 wikitext text/x-wiki fkezm08x265dxy7batz22vy05o8wq91 2462459 2462444 2026-09-30T18:49:16Z Stashbot 7414 dancy: Rebuilding gitlab-cloud-runners (staging) (T425441) 2462459 wikitext text/x-wiki pp6ltblfx5g7brg0t76numpgqklx42a 2462484 2462459 2026-09-30T20:44:14Z Stashbot 7414 dancy: Done rebuilding gitlab-cloud-runners (staging) (T425441) 2462484 wikitext text/x-wiki 7vmao3dj7xmpczpw46krrdbrvyxv6ng 12 18681 2462279 2461754 2026-09-30T14:19:15Z Ahecht 5894 /* Latest supported pre-built images */ Mono pre-built images no longer appear to be valid options 2462279 wikitext text/x-wiki dy4ctiwujh8ykzbzjonlc43smyxp2ky 498 30942 2462129 2462073 2026-09-30T12:31:27Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.bootstrap_and_add 2462129 wikitext text/x-wiki 44y20crhq64ts6pz07x0n033coipe6g 2462130 2462129 2026-09-30T12:31:30Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=0) 2462130 wikitext text/x-wiki egr04jsump38utmr83xlns1ghvv9rh8 2462144 2462130 2026-09-30T12:43:02Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462144 wikitext text/x-wiki aj9jsdn5iuak8ckogejuzezbcx49psr 2462145 2462144 2026-09-30T12:43:06Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2462145 wikitext text/x-wiki qvv8ooxv1q1rqlnpk9hxgq8qaxts2qq 2462150 2462145 2026-09-30T12:45:33Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462150 wikitext text/x-wiki pjicwrixt5dycs729xl69agl7lsbk3b 2462151 2462150 2026-09-30T12:45:37Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2462151 wikitext text/x-wiki dqhaux9xxkbnvayurgfi4jr92ad5dbj 2462153 2462151 2026-09-30T12:46:17Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462153 wikitext text/x-wiki jhvrq419pknihcv5drcmtwybc0an4dx 2462154 2462153 2026-09-30T12:46:21Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2462154 wikitext text/x-wiki kemeggkb6glgrwbaab5qhdyovup1xyf 2462159 2462154 2026-09-30T12:48:25Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462159 wikitext text/x-wiki lbnbgovho5eaxf2xz7mx64h5dqobf2c 2462160 2462159 2026-09-30T12:48:30Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2462160 wikitext text/x-wiki 89vzlku9makdvnykoofxk78r5zmcc5y 2462438 2462160 2026-09-30T18:06:53Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462438 wikitext text/x-wiki c3a36ua7wt7h4tqetb8mwmx5y9em6p4 2462439 2462438 2026-09-30T18:06:58Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2462439 wikitext text/x-wiki ie7tq9bugm0h45oa5cui8c01bp8wxvr 2462440 2462439 2026-09-30T18:07:07Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462440 wikitext text/x-wiki 8jbrn6xw47ncjvqit64zyv7c7bzf04d 2462441 2462440 2026-09-30T18:07:13Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2462441 wikitext text/x-wiki fzyvlt99gm90bwe7mcsbbaojpc51zse 2462445 2462441 2026-09-30T18:21:17Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462445 wikitext text/x-wiki 9fjyopz3x3ub613bylw9jkf5ipn252m 2462446 2462445 2026-09-30T18:21:22Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2462446 wikitext text/x-wiki tw6y9z6630w86h5mp912ztetw9y3raa 2462447 2462446 2026-09-30T18:21:33Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462447 wikitext text/x-wiki m9im7g8c6hha0wsczgcjzbt9nu54g5n 2462448 2462447 2026-09-30T18:21:37Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2462448 wikitext text/x-wiki syxmccrcdhmg9swib82uj5ghd4tme49 2462449 2462448 2026-09-30T18:22:20Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.bootstrap_and_add 2462449 wikitext text/x-wiki kxf7nfrm5w5x1ziur9h8gk5lsbuy1xl 2462453 2462449 2026-09-30T18:30:37Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=0) 2462453 wikitext text/x-wiki n6bgyso1hh63zto8uqa0agb0gu0u7od 2462458 2462453 2026-09-30T18:47:05Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462458 wikitext text/x-wiki 8dzp1jndacj9yzmwvy1t2swse37d3k8 2462513 2462458 2026-10-01T04:25:15Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2462513 wikitext text/x-wiki 022ljx5fcz4st74rfqtak4pcrdzstex 2462520 2462513 2026-10-01T05:18:20Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.bootstrap_and_add 2462520 wikitext text/x-wiki cczbhla9uow61cdso06cd9krwyfh0tp 2462521 2462520 2026-10-01T05:25:10Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=99) 2462521 wikitext text/x-wiki g1qw0ranx43686p1azgriipdjyjope1 2462522 2462521 2026-10-01T05:25:31Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.bootstrap_and_add 2462522 wikitext text/x-wiki 5rh6nmvuonotlvugwe2tol7v54yhdgb 2462523 2462522 2026-10-01T05:27:56Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=0) 2462523 wikitext text/x-wiki sh2ufpjuvo25pgm1tkdvs5z2rtkp2fx 2462524 2462523 2026-10-01T05:31:03Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462524 wikitext text/x-wiki pnejruw852c75r2ywo7wox8mqh9jai2 0 91251 2462135 2461363 2026-09-30T12:35:14Z MMuhlenhoff (WMF) 5268 /* Adding/replacing URL downloader nodes */ Remove obsolete docs, as these are now behind LVS 2462135 wikitext text/x-wiki gpc5smvybbfknogixgvvc8aqaejabig 2462136 2462135 2026-09-30T12:37:31Z MMuhlenhoff (WMF) 5268 /* Troubleshooting */ Mention the new cookbook 2462136 wikitext text/x-wiki 5mthg58a3hvf87khjc7bjrn8zctyup0 12 442480 2462282 2159279 2026-09-30T14:21:23Z Ahecht 5894 /* Kubernetes pre-built image */ This image no longer exists 2462282 wikitext text/x-wiki 698mllw7wog1ehsiea13d8d2s3bsmje 2462487 2462282 2026-09-30T20:45:45Z Taavi-WMF 41365 Undid revision [[Special:Diff/2462282|2462282]] by [[Special:Contributions/Ahecht|Ahecht]] ([[User talk:Ahecht|talk]]) image is there, ref [[User_talk:Taavi#Toolforge_Image_Bot]] 2462487 wikitext text/x-wiki ifc9jb04wenyuho2orfsh3moy7w1i1f 498 443949 2462197 2461417 2026-09-30T13:19:56Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 672e794 (Reword queries drill-down page for clarity and beginner-friendliness) 2462197 wikitext text/x-wiki 1hhni4l0yps7famr560gizf95v4txdl 2462198 2462197 2026-09-30T13:19:57Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 21519fc (Add toolcounter page-view badge to footer) 2462198 wikitext text/x-wiki 1pikjicf4bzfasni8i8p5oaqdddgq0o 3 444547 2462277 2449671 2026-09-30T14:18:22Z Ahecht 5894 /* Toolforge Image Bot */ new section 2462277 wikitext text/x-wiki 18oad6xwv83rs56ih0635er8iq8c3ok 2462486 2462277 2026-09-30T20:45:09Z Taavi-WMF 41365 /* Toolforge Image Bot */ Reply 2462486 wikitext text/x-wiki 0cmtsufeldokpghjrqx7oi3vkfowg00 2462501 2462486 2026-09-30T21:44:30Z Ahecht 5894 /* Toolforge Image Bot */ Reply 2462501 wikitext text/x-wiki p7jpe766cfhigt4q6givqokb3wm1ba3 2462585 2462501 2026-10-01T10:11:47Z Taavi-WMF 41365 /* Toolforge Image Bot */ Reply 2462585 wikitext text/x-wiki r0az9qxb3whzev448qgqc1meblich51 0 447310 2462454 2459111 2026-09-30T18:34:58Z KReid-WMF 49588 /* Guides */ Add link to growthbook access guide 2462454 wikitext text/x-wiki d586i8tfgiuezm72dmy3e77esz35mzo 2462457 2462454 2026-09-30T18:42:28Z KReid-WMF 49588 /* Support */ Add note about office hours 2462457 wikitext text/x-wiki og0br4k9slrtnsdkm4etp410r7uzn8g 0 447761 2462576 2460877 2026-10-01T09:35:13Z ABran-WMF 37196 update page to avoid breaking numbered list 2462576 wikitext text/x-wiki n81rostt18f3x2wu7vk1k1lpsgiloyl 0 449160 2462505 2461983 2026-10-01T00:01:28Z Dexbot 30554 Bot: Updating the report 2462505 wikitext text/x-wiki r3qpz0duonqb0ybikv3e059lmzxg3jk 12 451664 2462306 2458740 2026-09-30T14:25:47Z Ahecht 5894 /* Overview */ promoting the use of python 3.12 when the pre-built images support 3.13 doesn't make sense. 2462306 wikitext text/x-wiki i2lwp8tboc9grntel8k7nuwx642k3dj 3 453064 2462417 2091708 2026-09-30T17:06:36Z Ben Creasy 7887 /* contribgraph review */ Reply to contribgraph review 2462417 wikitext text/x-wiki b8mb6uep2kksqvfgnp7mbz7bq0jjov4 116 453426 2462579 2461209 2026-10-01T09:36:38Z Phabbanbot 37210 privateRepo was disabled by Peachey88 2462579 wikitext text/x-wiki 7j5ev91q5ukidwfvmtqn2bzqgyn2h5p 498 455898 2462428 2457413 2026-09-30T17:27:10Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed e8a9a28785 + 91022ba47e (Toolforge Components Service, push-to-deploy) 2462428 wikitext text/x-wiki 70twybwqbx7zx1qsxmx824twgw84gy2 0 459290 2462186 2461631 2026-09-30T13:07:28Z APizzata-WMF 47941 2462186 wikitext text/x-wiki 576kgyjfvtq1mz8p8u6jimsxz3qy5u3 2462211 2462186 2026-09-30T13:28:08Z SNwachukwu (WMF) 39521 2462211 wikitext text/x-wiki huxcj8x3p71yo80ck5xpyt5d4e9cp7y 2462559 2462211 2026-10-01T07:45:05Z JAllemandou (WMF) 37709 Add deploy item for next deploy 2462559 wikitext text/x-wiki hq7pgg0o49dat3aushtvkwb46vnt8ar 0 459699 2462405 2461992 2026-09-30T16:12:28Z Stashbot 7414 larssandergreen: civicrm upgraded from 7ff3e786 to da999791 2462405 wikitext text/x-wiki 9dia2pdm0j2yowql4o54qz81tqxkvx4 2462436 2462405 2026-09-30T17:59:41Z Stashbot 7414 larssandergreen: config revision changed from 04ee3ccd to cf53192f 2462436 wikitext text/x-wiki 03pkmgx17mg6vbswsmfzn6wohcu97f5 2462443 2462436 2026-09-30T18:15:23Z Stashbot 7414 larssandergreen: config revision changed from cf53192f to a79709ab 2462443 wikitext text/x-wiki jykzca6jx4undsrypoy60lq3k9q9q13 2462452 2462443 2026-09-30T18:25:32Z Stashbot 7414 larssandergreen: config revision changed from a79709ab to 7a5261b9 2462452 wikitext text/x-wiki aosh3j3t6u2xsricwhfpjs1e54igz8o 2462502 2462452 2026-09-30T21:59:26Z Stashbot 7414 larssandergreen: civicrm upgraded from da999791 to b03afa89 2462502 wikitext text/x-wiki 03hhrjsyvkeps0j7xiv0047u03ccbki 2462504 2462502 2026-09-30T22:52:15Z Stashbot 7414 eileen: civicrm upgraded from b03afa89 to 3aadfcea 2462504 wikitext text/x-wiki taych9mc7ndtziih1aebiqpt18cj6bj 2462506 2462504 2026-10-01T00:30:38Z Stashbot 7414 larssandergreen: civicrm upgraded from 3aadfcea to 87705543 2462506 wikitext text/x-wiki bheo3gd39bhwg9tyzdu3ikuv7naqsn0 2462514 2462506 2026-10-01T04:34:44Z Stashbot 7414 eileen: civicrm upgraded from 87705543 to a30dfd91 2462514 wikitext text/x-wiki gx2txdw1a9jkv9wehd1y6m3m4dph1b3 0 460319 2462580 2425350 2026-10-01T09:42:34Z GGoncalves-WMF 42848 2462580 wikitext text/x-wiki 3dg18r4tp7f8c054wupbe6fbqdhpz3t 2462581 2462580 2026-10-01T09:42:48Z GGoncalves-WMF 42848 2462581 wikitext text/x-wiki o5cgws7jckg246rn1n3yzifvh22iqid 2 460457 2462563 2438693 2026-10-01T08:30:27Z MABot 10804 Bot: Redirect to a deleted or non-existent page 2462563 wikitext text/x-wiki aa6ur6i4lqssxioambp1pnfb0t0qkk5 1 460533 2462437 2444965 2026-09-30T18:01:00Z Fuzheado 1608 ask 2462437 wikitext text/x-wiki qsslbn868qr0y7hat1g2mnqqsxzjv7p 0 460602 2462511 2459094 2026-10-01T01:28:31Z AWesterinen-WMF 54189 /* Overview */ 2462511 wikitext text/x-wiki 2grfj5wziwe31dptvdbw0vp3kufwkz7 0 460709 2462517 2459054 2026-10-01T04:47:00Z AWesterinen-WMF 54189 2462517 wikitext text/x-wiki ru9jgd01cbe32ejvwdq6labh2o6uwjb 2462518 2462517 2026-10-01T04:47:45Z AWesterinen-WMF 54189 2462518 wikitext text/x-wiki frcw6ijkomnagypxvl98avm03depmpp 2462519 2462518 2026-10-01T04:59:43Z AWesterinen-WMF 54189 /* wikibase:box */ 2462519 wikitext text/x-wiki 5oe9nz32nqluqs334lpzskhlfntx1xg 0 460710 2462507 2461997 2026-10-01T01:23:03Z AWesterinen-WMF 54189 2462507 wikitext text/x-wiki iheblu52rynz0znuy4k54zaxkx2pwbp 2462508 2462507 2026-10-01T01:24:47Z AWesterinen-WMF 54189 /* bd:slice service */ 2462508 wikitext text/x-wiki idp4a2fzsk0jyfa6v0926z6sz1r2nuv 2462509 2462508 2026-10-01T01:26:29Z AWesterinen-WMF 54189 /* bd:slice service */ 2462509 wikitext text/x-wiki lohlwvd5onzrqmdgko8bby3h7qs2gj2 2462510 2462509 2026-10-01T01:27:19Z AWesterinen-WMF 54189 /* Query hints */ 2462510 wikitext text/x-wiki 52nj3ntd8dhrfdj1mg8praxhjdbkg7g 2462512 2462510 2026-10-01T03:34:34Z AWesterinen-WMF 54189 /* bd:slice service */ 2462512 wikitext text/x-wiki sailjr95tq9utsy9jb88am72j2eo8x3 2462515 2462512 2026-10-01T04:38:34Z AWesterinen-WMF 54189 /* bd:sample service */ 2462515 wikitext text/x-wiki jeytcvqo66cnfk9ui9krfxmwpogpnil 2462516 2462515 2026-10-01T04:42:03Z AWesterinen-WMF 54189 /* bd:sample service */ 2462516 wikitext text/x-wiki q5z3dbcqd99l5yl0ce8517d14vrw5io 0 460719 2462564 2462031 2026-10-01T08:34:26Z 1Veertje 1673 say what it is 2462564 wikitext text/x-wiki g40bdu49us9a9f3f9tudhl4osww7f3f 2462565 2462564 2026-10-01T08:34:59Z 1Veertje 1673 outside readers might not recognize the acronym 2462565 wikitext text/x-wiki fh2np8e0a0pmy4a7wndel8ofz5nfj1q 2462567 2462565 2026-10-01T08:35:22Z 1Veertje 1673 2462567 wikitext text/x-wiki eyd7rwitkixdgtvhx2e2k10dydsf7m8 2462568 2462567 2026-10-01T08:35:58Z 1Veertje 1673 2462568 wikitext text/x-wiki swnmendp8vwekuadz0ht5try6eass4x 2462569 2462568 2026-10-01T08:36:09Z 1Veertje 1673 2462569 wikitext text/x-wiki f4na0ns2ilcqa255h42oarw0wd4gc80 2462570 2462569 2026-10-01T08:37:17Z 1Veertje 1673 2462570 wikitext text/x-wiki 34r5qm5te9nkkjzzd4v13ki7dj0wvh8 12 460726 2462321 2461592 2026-09-30T14:30:58Z SSapaty (WMF) 28955 /* Test Push-to-Deploy */ add Gitlab pipeline link for deployment configurations 2462321 wikitext text/x-wiki pwt0kggbf73l5un4og6jnxtutzdn1rs 1 460736 2462412 2461985 2026-09-30T16:36:52Z BCornwall-WMF 31521 /* Don't think this belongs here */ Reply 2462412 wikitext text/x-wiki i7too29qlb51ekzt7yea9toppviir5l 0 460746 2462347 2462078 2026-09-30T14:44:42Z HNowlan (WMF) 15760 severities -> expectations 2462347 wikitext text/x-wiki 3eyrtt86uewc7u14oybvfvz391waxm9 3 460747 2462293 2026-09-30T14:23:46Z StrikerBot 8475 Welcome to Toolforge! 2462293 wikitext text/x-wiki svpnn750uhbc7tpes0utcq792zx268o 0 460748 2462401 2026-09-30T16:04:14Z DPogorzelski-WMF 48508 Created page with "{{SCROLL/Tools/Banner|requested}} <templatestyles src="User:Effie_Mouzeli_(WMF)/SCROLL/styles.css" /> ''Service Checklist for Readiness, Operations, Launch and Lifecycle'' __TOC__ {| style="width: 100%; border: none; background: transparent;" | style="vertical-align: top; width: 38%; padding-right: 2%;" | {| class="wikitable" style="font-size: 90%; width: 100%; border: 1px solid #a2a9b1;" |- ! colspan="2" class="scroll-card-header" | 📜 Service/Component identity |-..." 2462401 wikitext text/x-wiki f6huzhg9boe7w3i8nqisctz9692xo18 0 460749 2462525 2026-10-01T05:48:12Z PHedenskog (WMF) 5986 First baby step 2462525 wikitext text/x-wiki 3sfrrxswie3fvu3uj7u4p3w00qvcyzu