Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.21 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 1380 2460865 2459746 2026-09-25T07:30:33Z Stashbot 7414 godog: move integration VMs onto newer cloudvirts - T437407 2460865 wikitext text/x-wiki i98aix9elgq4v55p6d8p0pidoe0ducr 498 3086 2460568 2460315 2026-09-24T16:54:55Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T438255) 2460568 wikitext text/x-wiki h18iwcvrk97q2a2ycms3j9gk6u8ertb 2460574 2460568 2026-09-24T17:03:02Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster 2460574 wikitext text/x-wiki 7mysmorp34c691ihrjte4qc9wxiggxx 2460579 2460574 2026-09-24T17:11:16Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T438255) 2460579 wikitext text/x-wiki ee9yd1625qaei3b8x2va2oru55erw3k 2460586 2460579 2026-09-24T17:22:07Z Stashbot 7414 andrew@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-84.tools.eqiad1.wikimedia.cloud to the cluster 2460586 wikitext text/x-wiki gqz2e986o5pi3ejzbs94guu3wiqojgl 2460587 2460586 2026-09-24T17:22:09Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster 2460587 wikitext text/x-wiki trd7o2ya75sxbqf0mrx3kltg224qq9m 2460589 2460587 2026-09-24T17:25:23Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T438255) 2460589 wikitext text/x-wiki lzuzzktzlrb4t22d9ommubqrm2n2u7n 2460598 2460589 2026-09-24T17:38:15Z Stashbot 7414 andrew@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-85.tools.eqiad1.wikimedia.cloud to the cluster 2460598 wikitext text/x-wiki tge1jtitrire9y1hpzsxtc19s4i4xmd 2460599 2460598 2026-09-24T17:38:16Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster 2460599 wikitext text/x-wiki ajxbzcvcmgynwrfyx9y7ienvhvlxf7w 2460608 2460599 2026-09-24T17:52:49Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase by 192 cores, 262144 ram (T439157) 2460608 wikitext text/x-wiki fhbr4zzqxozoa2e6irfvskdhpegfu9n 2460609 2460608 2026-09-24T17:52:55Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) by 192 cores, 262144 ram (T439157) 2460609 wikitext text/x-wiki nj527c4zp1qttchvo55nupwxdylp43s 2460610 2460609 2026-09-24T17:53:22Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T438255) 2460610 wikitext text/x-wiki h3dpzj4mlhj79ljy56m4z8a4tp00lf4 2460624 2460610 2026-09-24T18:02:58Z Stashbot 7414 andrew@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-86.tools.eqiad1.wikimedia.cloud to the cluster 2460624 wikitext text/x-wiki dhwfrsxo4ck54yd3ple075s09dwqcup 2460625 2460624 2026-09-24T18:02:59Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster 2460625 wikitext text/x-wiki 5zsu5e2qzghlo0ojbv9fq0s33y758pm 2460650 2460625 2026-09-24T18:28:18Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T438255) 2460650 wikitext text/x-wiki j37mbe5zim0d5d9h49f5imp5oaj7jc5 2460662 2460650 2026-09-24T18:38:16Z Stashbot 7414 andrew@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-87.tools.eqiad1.wikimedia.cloud to the cluster 2460662 wikitext text/x-wiki auh96bau6h00m2rriqs6ifz81j1md14 2460663 2460662 2026-09-24T18:38:18Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster 2460663 wikitext text/x-wiki rvgy37yftdr4xlq8c3pr92u4wgod8oc 2460693 2460663 2026-09-24T19:07:24Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T438255) 2460693 wikitext text/x-wiki hbi6hbe09qz6uwwq7hwjhrbnax5eura 2460706 2460693 2026-09-24T19:20:53Z Stashbot 7414 andrew@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-88.tools.eqiad1.wikimedia.cloud to the cluster 2460706 wikitext text/x-wiki s862dq2a99f8bj54r8u4dgd054t42jz 2460707 2460706 2026-09-24T19:20:55Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster 2460707 wikitext text/x-wiki qrc1heccklxhnkqexiffoip7fjvjtm7 2460727 2460707 2026-09-24T19:53:54Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T438255) 2460727 wikitext text/x-wiki 4u3iep9yygmx6o5o66hfolb9k92pyt1 2460732 2460727 2026-09-24T20:05:59Z Stashbot 7414 andrew@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-89.tools.eqiad1.wikimedia.cloud to the cluster 2460732 wikitext text/x-wiki jkdq2951mylqybdxt0swkelmg5sdvaf 2460733 2460732 2026-09-24T20:06:01Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster 2460733 wikitext text/x-wiki 8j6z3wuf51sm877x0y4ru4wpotonl4b 2460734 2460733 2026-09-24T20:07:38Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster (T438255) 2460734 wikitext text/x-wiki g2n7dyncpqz18qfwoe7z61bzlsbeoxq 2460742 2460734 2026-09-24T20:17:07Z Stashbot 7414 andrew@cloudcumin1001: Added a new k8s worker tools-k8s-worker-114.tools.eqiad1.wikimedia.cloud to the cluster 2460742 wikitext text/x-wiki m9j1fsc1r9yo76gwqvwlpf6z63parfy 2460743 2460742 2026-09-24T20:17:09Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster 2460743 wikitext text/x-wiki io7p6t4ywsmkmrjpoea74ksx8axeo6i 2460744 2460743 2026-09-24T20:17:34Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster (T438255) 2460744 wikitext text/x-wiki 77gy1na0qw6qfdl5w8gsa2kfrslwmzd 2460749 2460744 2026-09-24T20:26:11Z Stashbot 7414 andrew@cloudcumin1001: Added a new k8s worker tools-k8s-worker-115.tools.eqiad1.wikimedia.cloud to the cluster 2460749 wikitext text/x-wiki bx8s7u5xhgdu0013dksc5qte31jt27x 2460750 2460749 2026-09-24T20:26:12Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster 2460750 wikitext text/x-wiki 2wqqm007f8kyczdwmrdw5kfo34ls974 2460851 2460750 2026-09-25T01:32:46Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster (T438255) 2460851 wikitext text/x-wiki sqn8yo0jbb090w6v8bsl49lwgjzx0d6 2460855 2460851 2026-09-25T01:43:43Z Stashbot 7414 andrew@cloudcumin1001: Added a new k8s worker tools-k8s-worker-115.tools.eqiad1.wikimedia.cloud to the cluster 2460855 wikitext text/x-wiki s02dz5u8rqve4dcooha98tt79b6cjv3 2460856 2460855 2026-09-25T01:43:44Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster 2460856 wikitext text/x-wiki 2647d6we14bd9tt0e0n7xow372vpkys 2460857 2460856 2026-09-25T01:45:27Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T438255) 2460857 wikitext text/x-wiki ggot9y8zvriwtddwjnblwg2pvi1g6v8 2460858 2460857 2026-09-25T01:57:16Z Stashbot 7414 andrew@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-89.tools.eqiad1.wikimedia.cloud to the cluster 2460858 wikitext text/x-wiki arn1p57t64amoimhkz8s25igmi0aa7r 2460859 2460858 2026-09-25T01:57:17Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster 2460859 wikitext text/x-wiki of7pyp0famtrtoa8qe9ue9jrwqiqx87 0 4108 2460374 2460369 2026-09-24T12:41:10Z ScheduleDeploymentBot 37566 Add [[gerrit:1344246]] to Thursday, September 24 UTC afternoon backport window 2460374 wikitext text/x-wiki cgig3md0to2z7bbi2uijmshshwensf2 2460398 2460374 2026-09-24T13:16:51Z MGrosse-WMF 13323 added my change to the current window 2460398 wikitext text/x-wiki 9leakifnec3rqdqos06x0h04aw28g4m 2460528 2460398 2026-09-24T15:46:07Z ScheduleDeploymentBot 37566 Add [[gerrit:1344714]] to Thursday, September 24 UTC late backport window 2460528 wikitext text/x-wiki 5b7ee8mm0aiayk5pi1x2g6qlif62w5t 2460553 2460528 2026-09-24T16:21:01Z ScheduleDeploymentBot 37566 Add [[gerrit:1344716]] to Thursday, September 24 UTC late backport window 2460553 wikitext text/x-wiki 14ezbib2nyjt2wq3p9mn7o459v3ib5h 2460556 2460553 2026-09-24T16:30:10Z ScheduleDeploymentBot 37566 Add [[gerrit:1344406]] to Thursday, September 24 UTC late backport window 2460556 wikitext text/x-wiki 8rsu1ksgim1cxdnsgb5kc67y35dp5n3 2460571 2460556 2026-09-24T16:59:41Z ScheduleDeploymentBot 37566 Add [[gerrit:1344388]] to Thursday, September 24 UTC late backport window 2460571 wikitext text/x-wiki 6rskzsc3lr2ivb388vo3y75q9nbvrb8 2460572 2460571 2026-09-24T16:59:50Z ScheduleDeploymentBot 37566 Add [[gerrit:1344389]] to Thursday, September 24 UTC late backport window 2460572 wikitext text/x-wiki t8lqrpelm4hm1wjjg2rysw9h20soety 2460590 2460572 2026-09-24T17:26:55Z ScheduleDeploymentBot 37566 Add [[gerrit:1344329]] to Thursday, September 24 UTC late backport window 2460590 wikitext text/x-wiki d130wrrwkacvztpd13ddjgs4p5qjkzz 2460592 2460590 2026-09-24T17:30:41Z ScheduleDeploymentBot 37566 Add [[gerrit:1344740]] to Thursday, September 24 UTC late backport window 2460592 wikitext text/x-wiki dfqh8i8irgbekmcduen0sdo8h5rssh0 2460653 2460592 2026-09-24T18:31:05Z ScheduleDeploymentBot 37566 Add [[gerrit:1344750]] to Thursday, September 24 UTC late backport window 2460653 wikitext text/x-wiki 4qflu9vbbstr7bvtlz92w6xaccjsx95 2460677 2460653 2026-09-24T18:43:12Z ScheduleDeploymentBot 37566 Add [[gerrit:1344752]] to Thursday, September 24 UTC late backport window 2460677 wikitext text/x-wiki da4ezxur88iu22k5u2npqiqsojqdx78 2460755 2460677 2026-09-24T20:42:38Z ScheduleDeploymentBot 37566 Add [[gerrit:1344774]] to Tuesday, September 29 UTC late backport window 2460755 wikitext text/x-wiki ic2xecz1sbt5jvghw8zlb87jkddzxjd 2460898 2460755 2026-09-25T10:56:45Z ScheduleDeploymentBot 37566 Add [[gerrit:1345070]] to Monday, September 28 UTC morning backport window 2460898 wikitext text/x-wiki tsfvptsrhvbsb8qorh23reooxz1ciiv 12 5649 2460411 2428742 2026-09-24T13:28:50Z DCaro (WMF) 18271 /* Main features of Toolforge */ updated the jobs link 2460411 wikitext text/x-wiki k78enw8i0ethl6rspl5ha2rv1rw3v4a 0 7919 2460370 2460368 2026-09-24T12:03:57Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2460370 wikitext text/x-wiki t0tftpz19dwffr8hld8crbhg5no12t4 2460371 2460370 2026-09-24T12:04:32Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2460371 wikitext text/x-wiki em15gtqnl7gb95l6kscbu4pr6qpbx0u 2460372 2460371 2026-09-24T12:34:12Z Stashbot 7414 urbanecm@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply 2460372 wikitext text/x-wiki mq04uq863suvr879k42y9wwfh8lh6jq 2460373 2460372 2026-09-24T12:34:47Z Stashbot 7414 urbanecm@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply 2460373 wikitext text/x-wiki ho6eiyk5ax7ro9mzpdvch2m8nix6rrm 2460375 2460373 2026-09-24T12:43:28Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460375 wikitext text/x-wiki nnf71n0zul2jrof31l5zs6huvhh6jl9 2460376 2460375 2026-09-24T12:43:31Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460376 wikitext text/x-wiki h1tkqdzpf1whoc2fqb12oc25uqk3shg 2460379 2460376 2026-09-24T12:52:41Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460379 wikitext text/x-wiki doo74veevsciv5exs3tcmbce5kzpmat 2460380 2460379 2026-09-24T12:52:43Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460380 wikitext text/x-wiki 8mdji7oyh42ybykywv9kary2qmvu8wi 2460381 2460380 2026-09-24T12:58:02Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460381 wikitext text/x-wiki 169lfj6gfi88tfrg4f68siqfyo2nhu1 2460382 2460381 2026-09-24T12:58:08Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460382 wikitext text/x-wiki 8y0weg26mellmiz6zfczzlglrv8zkzw 2460383 2460382 2026-09-24T12:59:23Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460383 wikitext text/x-wiki r6o3i4b4ac4zhfvc3q3wrfwy1uab3ua 2460384 2460383 2026-09-24T12:59:27Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460384 wikitext text/x-wiki 09l9p39fngpawi7e11b5z10mzufv1al 2460385 2460384 2026-09-24T13:01:59Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=yes; selector: name=cp2059.* 2460385 wikitext text/x-wiki 0h9ibqfglvhv31umg9p0tmysm751wsu 2460386 2460385 2026-09-24T13:02:15Z Stashbot 7414 vgutierrez@cumin1004: START - Cookbook sre.cdn.roll-upgrade-haproxy rolling upgrade of HAProxy on A:cp-text_eqsin and A:cp - 3.2.23 upgrade (T438828) 2460386 wikitext text/x-wiki q8iiebqrfmq5z4i7t5v7r8dac5rx88v 2460387 2460386 2026-09-24T13:02:51Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460387 wikitext text/x-wiki o25eb9z86b603wdck9ycsbm70h0360m 2460388 2460387 2026-09-24T13:02:53Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460388 wikitext text/x-wiki ly8qj6i8ms0o4m7yuqmrve1to41ji3e 2460389 2460388 2026-09-24T13:06:03Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460389 wikitext text/x-wiki 90lq3rs8atx8hdp717puwq6k54aaqg7 2460390 2460389 2026-09-24T13:06:07Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460390 wikitext text/x-wiki 3n9mfway2pw3lotwyicynagg2vr9r44 2460391 2460390 2026-09-24T13:06:27Z Stashbot 7414 mlitn@deploy1003: Started scap sync-world: Backport for [[gerrit:1344617|Instrument five-arm image carousel retest (T431362)]], [[gerrit:1344619|Wire image carousel retest instrumentation (T431362)]], [[gerrit:1344627|ThumbExtractor: trim nbsp and dangling colons from caption text (T435672)]], [[gerrit:1344630|ThumbExtractor: exclude lead infobox images from the carousel (T438907)]] 2460391 wikitext text/x-wiki 6f744luhypooyoz55dt10vec5x7pv22 2460392 2460391 2026-09-24T13:07:46Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460392 wikitext text/x-wiki 7xdqrtj6vi1btb6wx5u9zewosgt6gdj 2460393 2460392 2026-09-24T13:07:50Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2460393 wikitext text/x-wiki m6ieaw0oxdr07ai9j3mzqw8vy14pous 2460394 2460393 2026-09-24T13:08:23Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.reimage for host ncredir4004.ulsfo.wmnet with OS trixie 2460394 wikitext text/x-wiki kcm1hfxunwnh7i6hoez6kr26wk9pljy 2460396 2460394 2026-09-24T13:10:37Z Stashbot 7414 mlitn@deploy1003: mfossati, mlitn: Backport for [[gerrit:1344617|Instrument five-arm image carousel retest (T431362)]], [[gerrit:1344619|Wire image carousel retest instrumentation (T431362)]], [[gerrit:1344627|ThumbExtractor: trim nbsp and dangling colons from caption text (T435672)]], [[gerrit:1344630|ThumbExtractor: exclude lead infobox images from the carousel (T438907)]] synced to the testservers (see https://wiki 2460396 wikitext text/x-wiki 0rp1a68ta8fgll23hj9bs0vmlxc1vmz 2460397 2460396 2026-09-24T13:13:26Z Stashbot 7414 mlitn@deploy1003: mfossati, mlitn: Continuing with deployment 2460397 wikitext text/x-wiki a2ivl2utgce4s567obs1dpj4exb94lo 2460399 2460397 2026-09-24T13:18:52Z Stashbot 7414 mlitn@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344617|Instrument five-arm image carousel retest (T431362)]], [[gerrit:1344619|Wire image carousel retest instrumentation (T431362)]], [[gerrit:1344627|ThumbExtractor: trim nbsp and dangling colons from caption text (T435672)]], [[gerrit:1344630|ThumbExtractor: exclude lead infobox images from the carousel (T438907)]] (duration: 12m 25s) 2460399 wikitext text/x-wiki 7p5vh095xqn6a7j9xbzibiuvzttzd8a 2460403 2460399 2026-09-24T13:24:16Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/growthbook-next: apply 2460403 wikitext text/x-wiki ggb2b2x4ca9d93vtqes40zd8kvco4zl 2460404 2460403 2026-09-24T13:25:38Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/growthboo-next: apply 2460404 wikitext text/x-wiki 0dspds5373lv0nomhalmlv07d6lvfzi 2460405 2460404 2026-09-24T13:25:44Z Stashbot 7414 elukey@deploy1003: helmfile [staging-codfw] START helmfile.d/admin 'sync'. 2460405 wikitext text/x-wiki nk68xbz3fdw46fflw1h809ntb2iadvf 2460406 2460405 2026-09-24T13:25:50Z Stashbot 7414 elukey@deploy1003: helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. 2460406 wikitext text/x-wiki 2ovj32u65wycy9qtvx3aumoo7h1bq5a 2460407 2460406 2026-09-24T13:26:18Z Stashbot 7414 elukey@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'sync'. 2460407 wikitext text/x-wiki maq3c1hk5syyh9sxgj0una5ds11ubzj 2460408 2460407 2026-09-24T13:26:21Z Stashbot 7414 elukey@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'sync'. 2460408 wikitext text/x-wiki onei0eq2i1s8iyskp2dx20gajdh5wj0 2460409 2460408 2026-09-24T13:26:30Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/growthbook-next: apply 2460409 wikitext text/x-wiki 5y25sgrie276xpzcbc8f0xyv7avu7v1 2460410 2460409 2026-09-24T13:26:55Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/growthboo-next: apply 2460410 wikitext text/x-wiki sicvedmvvlpxtuvfg00obqpoxxg9lqb 2460412 2460410 2026-09-24T13:29:54Z Stashbot 7414 awight@deploy1003: Started scap sync-world: Backport for [[gerrit:1344246|Fixes failing edge when page is missing and entity usage remain. Updating ReallyDoQuery to function like an inner join. (T437687)]] 2460412 wikitext text/x-wiki 36w0htmld65t3uy1noasadzsh6dw970 2460413 2460412 2026-09-24T13:33:53Z Stashbot 7414 awight@deploy1003: awight: Backport for [[gerrit:1344246|Fixes failing edge when page is missing and entity usage remain. Updating ReallyDoQuery to function like an inner join. (T437687)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2460413 wikitext text/x-wiki jxqoojzhvo2dudtyd7qs1h5bji6e475 2460414 2460413 2026-09-24T13:34:24Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ncredir4004.ulsfo.wmnet with reason: host reimage 2460414 wikitext text/x-wiki 6so1u7bcibnkw8tj2rchzqg7i77x81w 2460415 2460414 2026-09-24T13:35:12Z Stashbot 7414 awight@deploy1003: awight: Continuing with deployment 2460415 wikitext text/x-wiki n67c4kb9sfdeqrjnyeb68cwhsj2oc09 2460416 2460415 2026-09-24T13:35:28Z Stashbot 7414 moritzm: installing nghttp2 security updates 2460416 wikitext text/x-wiki jlwbnwg8fwyuvgmgxltdd8wip6823l9 2460417 2460416 2026-09-24T13:39:26Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ncredir4004.ulsfo.wmnet with reason: host reimage 2460417 wikitext text/x-wiki 4wk6ryzoecte5z7d90pgjifta0t9qoh 2460418 2460417 2026-09-24T13:39:46Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2460418 wikitext text/x-wiki hdbe4yv6tabwlfbj55i8r9rcovoa7yg 2460419 2460418 2026-09-24T13:40:32Z Stashbot 7414 awight@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344246|Fixes failing edge when page is missing and entity usage remain. Updating ReallyDoQuery to function like an inner join. (T437687)]] (duration: 10m 38s) 2460419 wikitext text/x-wiki 3ihctawa4k8qo2pdfeb1op1or7sqk5z 2460420 2460419 2026-09-24T13:54:44Z Stashbot 7414 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1344662|fix(AccountSetup): ensure TestKitchen knows about new user in CentralAuth redirect (T436872)]] 2460420 wikitext text/x-wiki 9dp55hd7u7uq7uizbgsudmqpupaas6u 2460421 2460420 2026-09-24T13:55:09Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.reboot-nodes rolling reboot on P{dse-k8s-wdqs100[1-3].eqiad.wmnet} and (A:dse-k8s-master-eqiad or A:dse-k8s-worker-eqiad) 2460421 wikitext text/x-wiki ntpfg6fuucnddyo0o4nezwxt2r34kh5 2460422 2460421 2026-09-24T13:55:10Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs-next: apply 2460422 wikitext text/x-wiki 3jnpe046l78ou1hwfmugi0hllz1ljj7 2460423 2460422 2026-09-24T13:55:12Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-wdqs1001.eqiad.wmnet 2460423 wikitext text/x-wiki 7h4qyc04duinqwjaeollqv8gfwh0xjk 2460424 2460423 2026-09-24T13:55:20Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2460424 wikitext text/x-wiki 2xmw9qecgb9eb93nn2r111lep9n7kj3 2460425 2460424 2026-09-24T13:55:47Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-wdqs1001.eqiad.wmnet 2460425 wikitext text/x-wiki ccdh0ewmhxb4eizu1xt0cnh4exluzze 2460426 2460425 2026-09-24T13:58:51Z Stashbot 7414 urbanecm@deploy1003: migr, urbanecm: Backport for [[gerrit:1344662|fix(AccountSetup): ensure TestKitchen knows about new user in CentralAuth redirect (T436872)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2460426 wikitext text/x-wiki 2e16pwp52ltukt8ljk9ab00dtkg6jwu 2460427 2460426 2026-09-24T14:00:20Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ncredir4004.ulsfo.wmnet with OS trixie 2460427 wikitext text/x-wiki eu5iozvejci688kbdwlzcscsi9zqbjh 2460428 2460427 2026-09-24T14:01:04Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-wdqs1001.eqiad.wmnet 2460428 wikitext text/x-wiki pvku25a3mc5rzsrljanwywkubp59cn4 2460429 2460428 2026-09-24T14:01:06Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-wdqs1001.eqiad.wmnet 2460429 wikitext text/x-wiki r7riu9164hgxlce79atk6x5dhu7t8uq 2460430 2460429 2026-09-24T14:01:11Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-wdqs1002.eqiad.wmnet 2460430 wikitext text/x-wiki fsa37l7t745gcnh39iw1i72rca3l2gd 2460431 2460430 2026-09-24T14:01:40Z Stashbot 7414 urbanecm@deploy1003: migr, urbanecm: Continuing with deployment 2460431 wikitext text/x-wiki rmud1uvw1pnypz0ffdg775txrr6m866 2460432 2460431 2026-09-24T14:01:46Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-wdqs1002.eqiad.wmnet 2460432 wikitext text/x-wiki 31l9cxdfhyvfypo4zzkdheqca99y21d 2460434 2460432 2026-09-24T14:03:30Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs-next: apply 2460434 wikitext text/x-wiki d4xp6gortvhk358y5y506iyxqguh9zr 2460436 2460434 2026-09-24T14:05:12Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2460436 wikitext text/x-wiki 9a6pfr87qjzf0jmbammw9x46ywqwbik 2460437 2460436 2026-09-24T14:05:59Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs-next: apply 2460437 wikitext text/x-wiki hrtqiqlb59jwq14pg6pa0a24rl6t49a 2460438 2460437 2026-09-24T14:07:21Z Stashbot 7414 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344662|fix(AccountSetup): ensure TestKitchen knows about new user in CentralAuth redirect (T436872)]] (duration: 12m 27s) 2460438 wikitext text/x-wiki qnw9r0hasr63c4o8h4uwqooptclrwef 2460439 2460438 2026-09-24T14:07:22Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-wdqs1002.eqiad.wmnet 2460439 wikitext text/x-wiki 27ei83wdrxdea7ks0ocr09ptjrwk8jm 2460440 2460439 2026-09-24T14:07:23Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-wdqs1002.eqiad.wmnet 2460440 wikitext text/x-wiki k0cxl3j7tw7fy3vo4wuyzw9v3bpxip0 2460441 2460440 2026-09-24T14:07:27Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-wdqs1003.eqiad.wmnet 2460441 wikitext text/x-wiki 6dou0n59u0u9frhggzgsy6rrch4gmzi 2460442 2460441 2026-09-24T14:07:48Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=yes; selector: name=ncredir4004.* 2460442 wikitext text/x-wiki ko1euslezrznk41e1jsr109mxy2ycfe 2460443 2460442 2026-09-24T14:07:59Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-wdqs1003.eqiad.wmnet 2460443 wikitext text/x-wiki 4idsms7ss1p2c5xkiap6acizmevn4iw 2460444 2460443 2026-09-24T14:08:13Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2460444 wikitext text/x-wiki nmqjoewm8vmsni9v745ov0f34qr35kl 2460445 2460444 2026-09-24T14:09:02Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.reimage for host ncredir3005.esams.wmnet with OS trixie 2460445 wikitext text/x-wiki tbz7gf72wkdgdz97xf9fcz1zea2bgbe 2460446 2460445 2026-09-24T14:09:05Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs-next: apply 2460446 wikitext text/x-wiki i3799c510nfvftlpzvlg3gq3ohdublm 2460447 2460446 2026-09-24T14:11:16Z Stashbot 7414 vgutierrez@cumin1004: END (FAIL) - Cookbook sre.cdn.roll-upgrade-haproxy (exit_code=1) rolling upgrade of HAProxy on A:cp-text_eqsin and A:cp - 3.2.23 upgrade (T438828) 2460447 wikitext text/x-wiki 2gleatux7wyzvhgboa886q13u36akw8 2460448 2460447 2026-09-24T14:13:17Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-wdqs1003.eqiad.wmnet 2460448 wikitext text/x-wiki 4inb9vkx0hnwkwdrfedbrmb9hj0cnqk 2460449 2460448 2026-09-24T14:13:18Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-wdqs1003.eqiad.wmnet 2460449 wikitext text/x-wiki 1drw3x5q7x5dr57jjk0i69dhliotq2b 2460450 2460449 2026-09-24T14:13:19Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on P{dse-k8s-wdqs100[1-3].eqiad.wmnet} and (A:dse-k8s-master-eqiad or A:dse-k8s-worker-eqiad) 2460450 wikitext text/x-wiki pyimk3ion9wxnaf8yv0506oe8vzoon9 2460451 2460450 2026-09-24T14:17:22Z Stashbot 7414 moritzm: installing Bird security updates 2460451 wikitext text/x-wiki os3flfm4w7x4q7sx2zutf4r6cmmnxoa 2460452 2460451 2026-09-24T14:19:46Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.reboot-nodes rolling reboot on P{dse-k8s-wdqs-test1001.eqiad.wmnet} and (A:dse-k8s-master-eqiad or A:dse-k8s-worker-eqiad) 2460452 wikitext text/x-wiki 9jyvq3cfn89hswuklh80l4lz9kjcsv2 2460453 2460452 2026-09-24T14:19:49Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-wdqs-test1001.eqiad.wmnet 2460453 wikitext text/x-wiki h11etxbvhmai3mt1nw80gtxm21h63bv 2460454 2460453 2026-09-24T14:20:24Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-wdqs-test1001.eqiad.wmnet 2460454 wikitext text/x-wiki oou5op31foj0k8827uqd3b3flry5jud 2460455 2460454 2026-09-24T14:21:03Z Stashbot 7414 vgutierrez@cumin1004: START - Cookbook sre.cdn.roll-upgrade-haproxy rolling upgrade of HAProxy on P{cp[5025,5026].*} and A:cp - 3.2.23 upgrade (T438828) 2460455 wikitext text/x-wiki plw58o0brdt9jnrl4qmq5p8i9pvsx7s 2460456 2460455 2026-09-24T14:21:39Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2460456 wikitext text/x-wiki jy8mb4xuy9nm4wexidrfgc4oo97kz4p 2460457 2460456 2026-09-24T14:22:31Z Stashbot 7414 elukey: elukey@rdb2013:/srv/redis/appendonlydir$ sudo -u redis redis-check-aof --fix rdb2013-6380.aof.22039.incr.aof 2460457 wikitext text/x-wiki rr1xsvm2uia8od0q4cfcfddzjmpjc0v 2460458 2460457 2026-09-24T14:22:35Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2460458 wikitext text/x-wiki 37xyefkdhlzx3zka3azggw6nye1n6ki 2460459 2460458 2026-09-24T14:26:25Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-wdqs-test1001.eqiad.wmnet 2460459 wikitext text/x-wiki h7p3wxirei5n06q4utwi5s8gp07lzrb 2460460 2460459 2026-09-24T14:26:26Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-wdqs-test1001.eqiad.wmnet 2460460 wikitext text/x-wiki f2h0imk0kl2yglwy3q3e8oq6mgdi61m 2460461 2460460 2026-09-24T14:26:28Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on P{dse-k8s-wdqs-test1001.eqiad.wmnet} and (A:dse-k8s-master-eqiad or A:dse-k8s-worker-eqiad) 2460461 wikitext text/x-wiki 6qdfxcy3pm09szwbtiyme053oicn011 2460462 2460461 2026-09-24T14:27:05Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1344684|AbuseReview: Add local CheckUsers to vandalism alpha test (T438467)]], [[gerrit:1344677|AbuseReview: Inidicate if the queue hides recent edits (T438235)]] 2460462 wikitext text/x-wiki 3g9ustm2xlwcsaxn3sfkimpbxjpamn1 2460463 2460462 2026-09-24T14:30:44Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.reboot-nodes rolling reboot on P{dse-k8s-worker1*.eqiad.wmnet} and (A:dse-k8s-master-eqiad or A:dse-k8s-worker-eqiad) 2460463 wikitext text/x-wiki hrkwljbkulcu6fsauwswinhjz80h2gy 2460464 2460463 2026-09-24T14:30:47Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1001.eqiad.wmnet 2460464 wikitext text/x-wiki 6e458qkskdak7de10013wjlyt15h9lp 2460465 2460464 2026-09-24T14:31:19Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1001.eqiad.wmnet 2460465 wikitext text/x-wiki qk41r9hbjvw8yh7valszobvszweiwgw 2460466 2460465 2026-09-24T14:32:43Z Stashbot 7414 dkertesz: depooling cp7001|7011 to apply https://gerrit.wikimedia.org/r/c/operations/puppet/+/1344222 (context: https://phabricator.wikimedia.org/T343000) 2460466 wikitext text/x-wiki 2q6d12iptecpkrbtj5rfpgvm6fb6p55 2460467 2460466 2026-09-24T14:34:01Z Stashbot 7414 dkertesz@cumin1004: conftool action : set/pooled=no; selector: name=cp7001.* 2460467 wikitext text/x-wiki 7tinxhxgv2gied6j4stq87i9jo5rbn6 2460468 2460467 2026-09-24T14:34:53Z Stashbot 7414 dkertesz@cumin1004: conftool action : set/pooled=no; selector: name=cp7011.* 2460468 wikitext text/x-wiki mefuoufj1n9qqxi3tza80suz13fqkuk 2460469 2460468 2026-09-24T14:37:50Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1001.eqiad.wmnet 2460469 wikitext text/x-wiki 1tfq2z9f215pm1g5yu5o3lfosi0d5d5 2460470 2460469 2026-09-24T14:37:51Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1001.eqiad.wmnet 2460470 wikitext text/x-wiki qfwzr2vh9mj0nn6ypveh123ceulrd4p 2460471 2460470 2026-09-24T14:37:59Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1002.eqiad.wmnet 2460471 wikitext text/x-wiki hq1mlf9q8yitd42v21rn3c5mbbr2ntp 2460472 2460471 2026-09-24T14:39:30Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ncredir3005.esams.wmnet with reason: host reimage 2460472 wikitext text/x-wiki 9gkrgatnkaetjaubo24o2rlou8j3p2n 2460473 2460472 2026-09-24T14:39:44Z Stashbot 7414 vgutierrez@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-haproxy (exit_code=0) rolling upgrade of HAProxy on P{cp[5025,5026].*} and A:cp - 3.2.23 upgrade (T438828) 2460473 wikitext text/x-wiki mlunxnjix7ifzgisf994wim2w3569nr 2460474 2460473 2026-09-24T14:39:51Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/weight=10; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-worker1015.eqiad.wmnet 2460474 wikitext text/x-wiki r5ykike1mofipoobhjiypk7bbdqpk17 2460475 2460474 2026-09-24T14:39:55Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/weight=10; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-worker1016.eqiad.wmnet 2460475 wikitext text/x-wiki sh0q6h6fvltfxabpv3p3ry8ozheokmo 2460476 2460475 2026-09-24T14:40:03Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/pooled=yes; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-worker1015.eqiad.wmnet 2460476 wikitext text/x-wiki srdvzg13p5gq3dguem08fs1jntv7aba 2460477 2460476 2026-09-24T14:40:04Z Stashbot 7414 btullis@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host dse-k8s-worker1002.eqiad.wmnet 2460477 wikitext text/x-wiki 3t2na7onuedux4h469g882elmkd0pxa 2460478 2460477 2026-09-24T14:40:08Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/pooled=yes; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-worker1016.eqiad.wmnet 2460478 wikitext text/x-wiki bc0px9rh0vwkbfrhwvln1pqeygewjiw 2460479 2460478 2026-09-24T14:40:36Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/weight=10; selector: service=kubesvc,cluster=dse-k8s,dc=eqiad,name=dse-k8s-worker1015.eqiad.wmnet 2460479 wikitext text/x-wiki 6gwhf539334ci76ab4mg3zan3byprjd 2460480 2460479 2026-09-24T14:40:43Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/weight=10; selector: service=kubesvc,cluster=dse-k8s,dc=eqiad,name=dse-k8s-worker1016.eqiad.wmnet 2460480 wikitext text/x-wiki 1evd893l6qrr63wxnun16th2rtihxtb 2460481 2460480 2026-09-24T14:40:51Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/pooled=yes; selector: service=kubesvc,cluster=dse-k8s,dc=eqiad,name=dse-k8s-worker1015.eqiad.wmnet 2460481 wikitext text/x-wiki eyqd1698bu1eti09jakg9s0zsudyvf0 2460482 2460481 2026-09-24T14:40:56Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/pooled=yes; selector: service=kubesvc,cluster=dse-k8s,dc=eqiad,name=dse-k8s-worker1016.eqiad.wmnet 2460482 wikitext text/x-wiki d5f1zji904cxm3zd50ishk4ledinmlv 2460483 2460482 2026-09-24T14:43:05Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ncredir3005.esams.wmnet with reason: host reimage 2460483 wikitext text/x-wiki aplkqcbc5e6nvmpvteqg894maqbmhjc 2460484 2460483 2026-09-24T14:46:26Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1344684|AbuseReview: Add local CheckUsers to vandalism alpha test (T438467)]], [[gerrit:1344677|AbuseReview: Inidicate if the queue hides recent edits (T438235)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2460484 wikitext text/x-wiki 232ohqjzkad81pu6l2j3u3xi3lrhwai 2460485 2460484 2026-09-24T14:47:27Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2460485 wikitext text/x-wiki jm8srpeas4gsc4lds8gnasgvcjrwees 2460486 2460485 2026-09-24T14:49:49Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1002.eqiad.wmnet 2460486 wikitext text/x-wiki lqtwy3znkhumslqs3jq3e8jpjix7kni 2460487 2460486 2026-09-24T14:49:50Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1002.eqiad.wmnet 2460487 wikitext text/x-wiki ewspo4g44wzznf0gx9kfwaws4ssxhpr 2460488 2460487 2026-09-24T14:49:56Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1003.eqiad.wmnet 2460488 wikitext text/x-wiki 4s2es7d9q4qx9cw8a1g04rx34lwlip1 2460489 2460488 2026-09-24T14:50:36Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2460489 wikitext text/x-wiki 0740rbubvt1yaluo73cwxlyftpz0ice 2460490 2460489 2026-09-24T14:51:25Z Stashbot 7414 dkertesz: repooling cp7001|7011 after successful testing (T343000) 2460490 wikitext text/x-wiki 15s59ll307h9v9u8prnaa170oi8yfkm 2460491 2460490 2026-09-24T14:51:26Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs-next: apply 2460491 wikitext text/x-wiki codzkp4kuzhbibn8lpiptfjt1p6m7am 2460492 2460491 2026-09-24T14:53:07Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs: apply 2460492 wikitext text/x-wiki sce2e37cm92wadshz58hjpu5i58iq8c 2460493 2460492 2026-09-24T14:53:34Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs: apply 2460493 wikitext text/x-wiki 9frkktcwdunjvc18vain4f661iqnje0 2460494 2460493 2026-09-24T14:53:54Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs: apply 2460494 wikitext text/x-wiki seirj07i7cjoiupytvesq6xgnt6d8k6 2460495 2460494 2026-09-24T14:54:19Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs: apply 2460495 wikitext text/x-wiki pets25aq5i44pvpilak616twyi2er10 2460496 2460495 2026-09-24T14:54:33Z Stashbot 7414 dkertesz@cumin1004: conftool action : set/pooled=yes; selector: name=cp7001.* 2460496 wikitext text/x-wiki swhgmhuzikkl1u6xf2tdh60g8wm5szq 2460497 2460496 2026-09-24T14:54:39Z Stashbot 7414 dkertesz@cumin1004: conftool action : set/pooled=yes; selector: name=cp7011.* 2460497 wikitext text/x-wiki cattjwlbr0gjutfu10w2has69slkuor 2460498 2460497 2026-09-24T14:57:03Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2460498 wikitext text/x-wiki a5s0hvub8fu9sn3d1lnt310luphoqqt 2460499 2460498 2026-09-24T14:59:25Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344684|AbuseReview: Add local CheckUsers to vandalism alpha test (T438467)]], [[gerrit:1344677|AbuseReview: Inidicate if the queue hides recent edits (T438235)]] (duration: 32m 20s) 2460499 wikitext text/x-wiki tj4qgig5z31866wdu321m11lw10i6cw 2460500 2460499 2026-09-24T15:03:39Z Stashbot 7414 vgutierrez@puppetserver1001: conftool action : set/pooled=yes; selector: name=dns2.*,service=authdns-update 2460500 wikitext text/x-wiki aa42j0avxv54322sa63q1z2kl9z6pv1 2460501 2460500 2026-09-24T15:04:40Z Stashbot 7414 vgutierrez@dns1004: START - running authdns-update 2460501 wikitext text/x-wiki j139o2lm9gnpctp27qozpu41zpqhn39 2460502 2460501 2026-09-24T15:07:00Z Stashbot 7414 vgutierrez@dns1004: END - running authdns-update 2460502 wikitext text/x-wiki pb3570i1gmbafpkiq5e41208r6vhwk2 2460503 2460502 2026-09-24T15:12:14Z Stashbot 7414 vgutierrez@puppetserver1001: conftool action : set/pooled=yes; selector: dc=codfw,cluster=dnsbox 2460503 wikitext text/x-wiki c1pwrbefn6xmrkp3sr34h6qidz2zipj 2460504 2460503 2026-09-24T15:18:02Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ncredir3005.esams.wmnet with OS trixie 2460504 wikitext text/x-wiki jssl925n80en5101z2jpvxkv8brimd5 2460505 2460504 2026-09-24T15:20:12Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1344694|EventMapper::fetchByPage: Allow filtering by type (T438031)]] 2460505 wikitext text/x-wiki dcmq3byslmg0ot0mbb37qo3ira0sx3n 2460506 2460505 2026-09-24T15:20:14Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1003.eqiad.wmnet 2460506 wikitext text/x-wiki 9lrfdzigmtxm09hevnfil5chsy9uaim 2460509 2460506 2026-09-24T15:24:34Z Stashbot 7414 dreamyjazz@deploy1003: kharlan, dreamyjazz: Backport for [[gerrit:1344694|EventMapper::fetchByPage: Allow filtering by type (T438031)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2460509 wikitext text/x-wiki 4uclldxdys8ritrb8rxxdbd3ttlgeem 2460510 2460509 2026-09-24T15:25:12Z Stashbot 7414 dreamyjazz@deploy1003: kharlan, dreamyjazz: Continuing with deployment 2460510 wikitext text/x-wiki jncgrdl5s0inxwbiylqrx56vde7p7g0 2460511 2460510 2026-09-24T15:26:36Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1003.eqiad.wmnet 2460511 wikitext text/x-wiki g6d4rca68rf1fwm89k62q98cmko20hi 2460512 2460511 2026-09-24T15:26:37Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1003.eqiad.wmnet 2460512 wikitext text/x-wiki rjsqzjoyat64ow150fxac33ll78pyc2 2460513 2460512 2026-09-24T15:26:43Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1004.eqiad.wmnet 2460513 wikitext text/x-wiki ep6eo34kadhsx3zo6qk4s7qhc0aqlcd 2460514 2460513 2026-09-24T15:27:14Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply 2460514 wikitext text/x-wiki tsxb2nryecp05d7dxui50bqh4xrjgqa 2460515 2460514 2026-09-24T15:27:24Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2460515 wikitext text/x-wiki fs8wbco9g38vykicdayp6zmqqq7ii8n 2460516 2460515 2026-09-24T15:29:10Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply 2460516 wikitext text/x-wiki syghzloakesu6glt8emgf3srqxxon29 2460517 2460516 2026-09-24T15:30:30Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460517 wikitext text/x-wiki 8ynrd89klhemkd50eoy4ms4z9nq6d8g 2460518 2460517 2026-09-24T15:30:33Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460518 wikitext text/x-wiki lx981ohi7697082l0q3lb5si9dlffn3 2460520 2460518 2026-09-24T15:32:45Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344694|EventMapper::fetchByPage: Allow filtering by type (T438031)]] (duration: 12m 33s) 2460520 wikitext text/x-wiki d9qrez5lahznohts0s04ydxkzw8u2j2 2460521 2460520 2026-09-24T15:33:31Z Stashbot 7414 vgutierrez@dns1004: START - running authdns-update 2460521 wikitext text/x-wiki su5y6nw46030xro5kwe1jp1zg43ipig 2460522 2460521 2026-09-24T15:36:00Z Stashbot 7414 vgutierrez@dns1004: END - running authdns-update 2460522 wikitext text/x-wiki pw6goca87yj7xkkfpbc2r48uaukg7w2 2460524 2460522 2026-09-24T15:40:41Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1344711|AbuseReview: Enable on enwiki (T439149)]], [[gerrit:1344693|Sync wmf/1.47.0-wmf.20 with wmf/1.47.0-wmf.21 for vandalism alpha (T438467)]] 2460524 wikitext text/x-wiki 2nbh6jf3sog54iuqbqs5aadb8oj87ng 2460530 2460524 2026-09-24T15:48:00Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reboot-single for host dse-k8s-worker1017.eqiad.wmnet 2460530 wikitext text/x-wiki 0rl2kapxlmbfo201v94ckjvmc7u1zg4 2460531 2460530 2026-09-24T15:51:26Z Stashbot 7414 swfrench-wmf: begin rolling restarts of confds in eqsin, codfw, ulsfo to reflect etcd SRV record changes 2460531 wikitext text/x-wiki 97rhxepzeotttnuz3oopmh6k805of3q 2460532 2460531 2026-09-24T15:51:52Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=yes; selector: name=ncredir3005.* 2460532 wikitext text/x-wiki qjay36ecfvwed8arcmj6ccv86v5mg2z 2460533 2460532 2026-09-24T15:52:46Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.reimage for host ncredir1001.eqiad.wmnet with OS trixie 2460533 wikitext text/x-wiki owpe7ps7cjgw5lih76j3slomzn6cpbj 2460534 2460533 2026-09-24T15:53:55Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-worker1017.eqiad.wmnet 2460534 wikitext text/x-wiki t023s4vm2p89ccxscfxz5on3u87xd93 2460535 2460534 2026-09-24T15:56:56Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1004.eqiad.wmnet 2460535 wikitext text/x-wiki hqfdyy631otyii5kx6w58yp8cpe6smt 2460536 2460535 2026-09-24T15:57:29Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460536 wikitext text/x-wiki s3owf5e3xttjm24efd3ibacn4s5lko2 2460537 2460536 2026-09-24T15:57:32Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460537 wikitext text/x-wiki 8cloduyrp246pjtrycdp4w2545n8x0h 2460538 2460537 2026-09-24T16:00:01Z Stashbot 7414 dreamyjazz@deploy1003: kharlan, dreamyjazz: Backport for [[gerrit:1344711|AbuseReview: Enable on enwiki (T439149)]], [[gerrit:1344693|Sync wmf/1.47.0-wmf.20 with wmf/1.47.0-wmf.21 for vandalism alpha (T438467)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2460538 wikitext text/x-wiki qhag4ki5ivr5dov813th94mdvzgcygi 2460539 2460538 2026-09-24T16:01:27Z Stashbot 7414 dreamyjazz@deploy1003: kharlan, dreamyjazz: Continuing with deployment 2460539 wikitext text/x-wiki 7mddt00i5fpmjktngj5e7kjtplb40k8 2460540 2460539 2026-09-24T16:01:32Z Stashbot 7414 swfrench@cumin1004: START - Cookbook sre.loadbalancer.upgrade restart A:liberica-ulsfo 2460540 wikitext text/x-wiki bpt6nr9xy868jazdrd7xd9hhldolsb5 2460542 2460540 2026-09-24T16:01:52Z Stashbot 7414 hnowlan@cumin1004: START - Cookbook sre.switchdc.mediawiki.00-optional-warmup-caches for datacenter switchover from eqiad to codfw 2460542 wikitext text/x-wiki ix6esvkl8wieaopdrpp5rkj4yk10qfk 2460543 2460542 2026-09-24T16:03:15Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1004.eqiad.wmnet 2460543 wikitext text/x-wiki a8cfdatcpoweo2v8hi6o6mxyzbha81k 2460544 2460543 2026-09-24T16:03:16Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1004.eqiad.wmnet 2460544 wikitext text/x-wiki 6mg0d7wco1kadlv431qq6n8rqgvbe0p 2460545 2460544 2026-09-24T16:03:19Z Stashbot 7414 swfrench@cumin1004: END (PASS) - Cookbook sre.loadbalancer.upgrade (exit_code=0) restart A:liberica-ulsfo 2460545 wikitext text/x-wiki ewphbwb7b1e5bpg86bjcph586vcwqhk 2460546 2460545 2026-09-24T16:03:22Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1005.eqiad.wmnet 2460546 wikitext text/x-wiki 3ud414lp0he56mxtrr2bbe1o1qsodod 2460547 2460546 2026-09-24T16:08:07Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ncredir1001.eqiad.wmnet with reason: host reimage 2460547 wikitext text/x-wiki qvj3ie885dd6lvipy4b9pz7d2khupo2 2460548 2460547 2026-09-24T16:14:33Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344711|AbuseReview: Enable on enwiki (T439149)]], [[gerrit:1344693|Sync wmf/1.47.0-wmf.20 with wmf/1.47.0-wmf.21 for vandalism alpha (T438467)]] (duration: 33m 52s) 2460548 wikitext text/x-wiki 92v3ji5ijem9xl4ck3x1yc1yq8bg22b 2460549 2460548 2026-09-24T16:14:42Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ncredir1001.eqiad.wmnet with reason: host reimage 2460549 wikitext text/x-wiki fglpfp3sc8cz2b527uifdrbgiwdd0f8 2460550 2460549 2026-09-24T16:15:31Z Stashbot 7414 taavi@deploy1003: Locking from deployment [ALL REPOSITORIES]: locked for re-pooling codfw for read traffic, contact SRE for equestions 2460550 wikitext text/x-wiki bpk1ee4fr0pzfnddxwmbi10pnqedang 2460551 2460550 2026-09-24T16:19:59Z Stashbot 7414 hnowlan@cumin1004: END (FAIL) - Cookbook sre.switchdc.mediawiki.00-optional-warmup-caches (exit_code=99) for datacenter switchover from eqiad to codfw 2460551 wikitext text/x-wiki o62r9006flxw2fv7fx3bn1xnl9l259r 2460552 2460551 2026-09-24T16:20:38Z Stashbot 7414 hnowlan@cumin1004: START - Cookbook sre.switchdc.mediawiki.00-optional-warmup-caches for datacenter switchover from eqiad to codfw 2460552 wikitext text/x-wiki fznlz46zzt09tptblw67mk8u3t0tvn1 2460554 2460552 2026-09-24T16:23:20Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply 2460554 wikitext text/x-wiki 0zd534ximosled6vgl9vg0fz7v4zdiu 2460555 2460554 2026-09-24T16:23:49Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply 2460555 wikitext text/x-wiki buly207yvgc5qmiizvnbd1jgxswmzyu 2460557 2460555 2026-09-24T16:33:08Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ncredir1001.eqiad.wmnet with OS trixie 2460557 wikitext text/x-wiki c4cgnrvdgsrihyyzhkcilziyy36b84h 2460558 2460557 2026-09-24T16:33:21Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=mw-api-int-ro 2460558 wikitext text/x-wiki 55j4mwwsaz1vqw736ef1zeab8u6n7a2 2460559 2460558 2026-09-24T16:33:23Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1005.eqiad.wmnet 2460559 wikitext text/x-wiki 4ei3f8fuqhob6gypn4510dxzzbgx8im 2460560 2460559 2026-09-24T16:37:37Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=mw-web-ro 2460560 wikitext text/x-wiki 8lo35r12s1xhikyutapc5oyib4vd7sk 2460561 2460560 2026-09-24T16:40:08Z Stashbot 7414 dzahn@cumin2003: START - Cookbook sre.hosts.reimage for host zuul1005.eqiad.wmnet with OS trixie 2460561 wikitext text/x-wiki lyn3ec76qxyxax6k2gwaxeffa9ut8da 2460562 2460561 2026-09-24T16:40:15Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1005.eqiad.wmnet 2460562 wikitext text/x-wiki fi3ojh84bz0yof6hru92vjvohstxcpy 2460563 2460562 2026-09-24T16:40:16Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1005.eqiad.wmnet 2460563 wikitext text/x-wiki rsi5wes1tyh9dfjtoeeoumq4tzgji3t 2460564 2460563 2026-09-24T16:40:21Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1006.eqiad.wmnet 2460564 wikitext text/x-wiki ed73hooht0ok42flkib80hm9y5ienta 2460565 2460564 2026-09-24T16:49:38Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=mw-web-next-ro 2460565 wikitext text/x-wiki ahugoqjt3ha169al9yutt2czbuzvafz 2460566 2460565 2026-09-24T16:54:48Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460566 wikitext text/x-wiki 9n5e95d5zo9mjzw0h5gjz7x0wkzqk8x 2460567 2460566 2026-09-24T16:54:49Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460567 wikitext text/x-wiki pc2cqi0yn2e0ygoagako0wgwhd4o0ua 2460569 2460567 2026-09-24T16:55:43Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460569 wikitext text/x-wiki qujomlgoa3vbsidtme4wy5x0xyot53x 2460570 2460569 2026-09-24T16:55:48Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460570 wikitext text/x-wiki sxn9ffngsc70t4ypn2dbajr82nm0whb 2460575 2460570 2026-09-24T17:05:46Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=yes; selector: name=ncredir1001.* 2460575 wikitext text/x-wiki gdf4af246lhzqu9merhnfrlojwmknax 2460578 2460575 2026-09-24T17:10:23Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1006.eqiad.wmnet 2460578 wikitext text/x-wiki 202iar382lnb2leezaluylvmxtst5h0 2460580 2460578 2026-09-24T17:17:02Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1006.eqiad.wmnet 2460580 wikitext text/x-wiki bolu2lfp9hxw4cmx218gvl96ip9rqe1 2460581 2460580 2026-09-24T17:17:03Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1006.eqiad.wmnet 2460581 wikitext text/x-wiki fv08i9671md8y7x40xb7hiysr010apz 2460582 2460581 2026-09-24T17:17:09Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1007.eqiad.wmnet 2460582 wikitext text/x-wiki hg2r27m5t0xe0q14smdkn05kaewbxze 2460583 2460582 2026-09-24T17:17:24Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=urldownloader 2460583 wikitext text/x-wiki 3ggjg6d71vumca2tolssp5y90y2d635 2460584 2460583 2026-09-24T17:18:32Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=shellbox.* 2460584 wikitext text/x-wiki nzzpj2iafturyui498vzbw0uy6i3bg5 2460585 2460584 2026-09-24T17:19:19Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=thumbor 2460585 wikitext text/x-wiki sf92vditxqmlginhjk0mer72bf76r4j 2460588 2460585 2026-09-24T17:22:23Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.reimage for host ncredir5004.eqsin.wmnet with OS trixie 2460588 wikitext text/x-wiki 50lfa8ff1b0k79d8qi0zf2b844ogn1o 2460591 2460588 2026-09-24T17:30:24Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=apertium|schema|termbox|proton|cxserver 2460591 wikitext text/x-wiki c3p74jg4d3mbuumcu5iub4hef2m224x 2460593 2460591 2026-09-24T17:32:22Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=citoid|zotero 2460593 wikitext text/x-wiki l71b9sbz4n5yfnl69w3ir5jegshoen1 2460594 2460593 2026-09-24T17:34:02Z Stashbot 7414 dzahn@cumin2003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host zuul1005.eqiad.wmnet with OS trixie 2460594 wikitext text/x-wiki ed38rjyydvvn772kwdq3yztgamsp24m 2460595 2460594 2026-09-24T17:37:28Z Stashbot 7414 brett@cumin1004: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on A:cp-text_magru and not P{cp7001.magru.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2460595 wikitext text/x-wiki 8732ww5sjog0r1rc4a6wk3km3ju8d3a 2460596 2460595 2026-09-24T17:37:50Z Stashbot 7414 brett@cumin1004: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on A:cp-upload_magru and not P{cp7011.magru.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2460596 wikitext text/x-wiki t8kjsq1nwt4gzpv014z9z711vmcyt3w 2460597 2460596 2026-09-24T17:38:05Z Stashbot 7414 dzahn@cumin2003: START - Cookbook sre.hosts.reimage for host zuul1005.eqiad.wmnet with OS trixie 2460597 wikitext text/x-wiki 112susww5x6c0d29r7wx9ee6rmkr18n 2460600 2460597 2026-09-24T17:42:02Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=config-master|device-analytics|echostore|helm-charts|k8s-ingress-wikikube-ro|linkrecommendation|mathoid|restbase|restbase-async|rest-gateway-ro|mobileapps|mwdebug.*|push-notifications|recommendation-api|releases|wikifeeds 2460600 wikitext text/x-wiki h3fdayzwy5lemf7h4eonk8jb8iu9xvv 2460601 2460600 2026-09-24T17:44:52Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=swift 2460601 wikitext text/x-wiki 0nz7fjbzli0ld0b4ow85y646f5que97 2460602 2460601 2026-09-24T17:47:11Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1007.eqiad.wmnet 2460602 wikitext text/x-wiki nn8xb23n8pt0xj9i9b2zolp4y3rys7a 2460603 2460602 2026-09-24T17:47:17Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=mwdebug.* 2460603 wikitext text/x-wiki 9arw62i5jlo3m0rx271lu0y2y232zsy 2460604 2460603 2026-09-24T17:47:53Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=apus 2460604 wikitext text/x-wiki b8n015qdv36fl0uf9k7nb5jqk8566f7 2460605 2460604 2026-09-24T17:49:22Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=mw-.*-ro 2460605 wikitext text/x-wiki h8y7j5az7iht4cn7cdt6exwur5wr6ie 2460606 2460605 2026-09-24T17:52:07Z Stashbot 7414 swfrench@cumin1004: conftool action : set/pooled=false; selector: dnsdisc=mwdebug.*,name=codfw 2460606 wikitext text/x-wiki 4cuo93vx5uxpqax7ijl6waxdrjr6np6 2460607 2460606 2026-09-24T17:52:15Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=false; selector: name=codfw,dnsdisc=mwdebug.* 2460607 wikitext text/x-wiki en3v52bh5ksiszxncox9lidhww35fty 2460611 2460607 2026-09-24T17:54:05Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1007.eqiad.wmnet 2460611 wikitext text/x-wiki 0pxf9h3j5vlt2gw8qmrkrqenzt7dl01 2460612 2460611 2026-09-24T17:54:06Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1007.eqiad.wmnet 2460612 wikitext text/x-wiki n40wkgs5wan2550to5bsguw5l0bua59 2460613 2460612 2026-09-24T17:54:12Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1008.eqiad.wmnet 2460613 wikitext text/x-wiki maishhp0inhbd8i8qses9c68yfzxbyq 2460614 2460613 2026-09-24T17:54:44Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1008.eqiad.wmnet 2460614 wikitext text/x-wiki p9bp2hfax9n1egpdsbyzpcn3eynyvb3 2460615 2460614 2026-09-24T17:58:59Z Stashbot 7414 hnowlan@cumin1004: END (FAIL) - Cookbook sre.switchdc.mediawiki.00-optional-warmup-caches (exit_code=99) for datacenter switchover from eqiad to codfw 2460615 wikitext text/x-wiki 6s9rjcasbbb8sf9glhi3grlspalusdu 2460616 2460615 2026-09-24T17:59:40Z Stashbot 7414 cdanis@cumin1004: START - Cookbook sre.dns.admin DNS admin: pool codfw [reason: no reason specified, no task ID specified] 2460616 wikitext text/x-wiki p6uxenwairnj2x5m680sxqy4iughrkr 2460617 2460616 2026-09-24T17:59:48Z Stashbot 7414 cdanis@cumin1004: END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: pool codfw [reason: no reason specified, no task ID specified] 2460617 wikitext text/x-wiki a6ax0r9m1e8dz561t650m6zjr4ui401 2460618 2460617 2026-09-24T18:01:22Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1008.eqiad.wmnet 2460618 wikitext text/x-wiki e9i3f0phrd3a3nyrwizwbfk6o9spzsh 2460619 2460618 2026-09-24T18:01:23Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1008.eqiad.wmnet 2460619 wikitext text/x-wiki 7k7wt0ycm9aso5isls7ei7x8bhp77rw 2460620 2460619 2026-09-24T18:01:29Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1009.eqiad.wmnet 2460620 wikitext text/x-wiki tfuktei37tfcm99pnpn2fvi06eudr1e 2460621 2460620 2026-09-24T18:02:00Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460621 wikitext text/x-wiki mzqn77nxte2p86r1fbpqq57j47bqypq 2460622 2460621 2026-09-24T18:02:02Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460622 wikitext text/x-wiki bhe33ca06i9e1eo6nde0jzzu1w9m4zd 2460626 2460622 2026-09-24T18:04:42Z Stashbot 7414 cdobbins@cumin1004: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ncredir5004.eqsin.wmnet with OS trixie 2460626 wikitext text/x-wiki gx8y7v5o8d8ihkh4np0hg6jjt1m2uw3 2460627 2460626 2026-09-24T18:04:55Z Stashbot 7414 taavi@deploy1003: Unlocked for deployment [ALL REPOSITORIES]: locked for re-pooling codfw for read traffic, contact SRE for equestions (duration: 109m 23s) 2460627 wikitext text/x-wiki rii0h7syituw8sxlu9bqy169qwo0cto 2460628 2460627 2026-09-24T18:06:27Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2460628 wikitext text/x-wiki sgqb0udgw6suj7v7y6ocog1dm0iv1cw 2460629 2460628 2026-09-24T18:06:48Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply 2460629 wikitext text/x-wiki 76omlj8zl5p3clyglhddczphevi1ysk 2460630 2460629 2026-09-24T18:08:16Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2460630 wikitext text/x-wiki 8v48joyamsdxymdgjgc86my1gc61po2 2460631 2460630 2026-09-24T18:09:27Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs-next: apply 2460631 wikitext text/x-wiki p8yujuepga5j4wqjkl57ljrk9ui754j 2460632 2460631 2026-09-24T18:10:09Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2460632 wikitext text/x-wiki k2w2j10q1s1ip4y0qr0zzi8bf3becln 2460633 2460632 2026-09-24T18:11:06Z Stashbot 7414 brett@cumin1004: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on A:cp-text_ulsfo - 7.1.1-2~bpo13+wmf3 () 2460633 wikitext text/x-wiki 0s0qgbn3qet48jaafny0mt9rqtvg87l 2460634 2460633 2026-09-24T18:11:17Z Stashbot 7414 brett@cumin1004: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on A:cp-upload_ulsfo - 7.1.1-2~bpo13+wmf3 () 2460634 wikitext text/x-wiki cp0qjedbuliud97ogbifpfj4b2ha51t 2460635 2460634 2026-09-24T18:11:29Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs-next: apply 2460635 wikitext text/x-wiki 9g9kk5zfsklnxpeyheksgad76aekf1z 2460636 2460635 2026-09-24T18:12:44Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2460636 wikitext text/x-wiki ky09rjfzkd12dtjfimueh14awfs5qvo 2460637 2460636 2026-09-24T18:13:09Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2460637 wikitext text/x-wiki qrsxg2s00ez9ue90pugntr9bikwt4ut 2460638 2460637 2026-09-24T18:15:49Z Stashbot 7414 ryankemper: [WDQS] Preparing to repool codfw WDQS shortly; it's been operating single DC so this second DC should restore proper service availability 2460638 wikitext text/x-wiki f00icje3lq0gsjmrij3adt2eu4ua0qx 2460639 2460638 2026-09-24T18:16:50Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2460639 wikitext text/x-wiki 16rcf1t7pya3o18gatdzqlohx7lsmp3 2460640 2460639 2026-09-24T18:17:17Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2460640 wikitext text/x-wiki 6seis0dodtj7qhrh2y1m1osjwdgyml7 2460641 2460640 2026-09-24T18:18:22Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs: apply 2460641 wikitext text/x-wiki k5de4suvhp3i8ic3looyw03qk2sl4pc 2460642 2460641 2026-09-24T18:18:53Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs: apply 2460642 wikitext text/x-wiki mkn2uyivlsbk71jxlzd45fj47tbby8o 2460643 2460642 2026-09-24T18:19:13Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs: apply 2460643 wikitext text/x-wiki eh3zr884eg89bgjz0ooq8ohdet2crny 2460644 2460643 2026-09-24T18:19:22Z Stashbot 7414 jhuneidi@deploy1003: rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.21 refs T438217 2460644 wikitext text/x-wiki 8e9sm2weca3sjphxlb0ztc1enzingkw 2460645 2460644 2026-09-24T18:19:35Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs: apply 2460645 wikitext text/x-wiki 44bw23e2gbvdgfhcjidza70kb5109sb 2460646 2460645 2026-09-24T18:20:41Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.discovery.service-route pool wdqs-main in codfw: maintenance 2460646 wikitext text/x-wiki 7fqwdukv4s4kbe4l5ltrf44yuip1o9d 2460647 2460646 2026-09-24T18:25:33Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.reimage for host ncredir5004.eqsin.wmnet with OS trixie 2460647 wikitext text/x-wiki atn0jcc3rgv7swzng0gl7iacdah1sna 2460648 2460647 2026-09-24T18:25:46Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) pool wdqs-main in codfw: maintenance 2460648 wikitext text/x-wiki mdy5w8q0c7999tmajo4mt8u2tp8d8xp 2460649 2460648 2026-09-24T18:28:02Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.discovery.service-route pool wdqs-scholarly in codfw: maintenance 2460649 wikitext text/x-wiki 3e7fd57ekt8waed4993guncoqd5x8c0 2460651 2460649 2026-09-24T18:29:59Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2460651 wikitext text/x-wiki e1vqj6a8uf3zcnb4gyzhnn3cjkhcg4k 2460652 2460651 2026-09-24T18:30:16Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply 2460652 wikitext text/x-wiki mv2bfp85q8r8rv451x4kcirhzfmpftr 2460654 2460652 2026-09-24T18:31:36Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1009.eqiad.wmnet 2460654 wikitext text/x-wiki 34mebn4wdt6ya0nols1s3ieqr7b5med 2460655 2460654 2026-09-24T18:33:07Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) pool wdqs-scholarly in codfw: maintenance 2460655 wikitext text/x-wiki mndwxpbaeit4zn1blg6r4krmq3uc68c 2460656 2460655 2026-09-24T18:33:22Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2460656 wikitext text/x-wiki c02tevgwob9ajatbl1hq8tuutd82jku 2460657 2460656 2026-09-24T18:33:35Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.discovery.service-route pool wcqs in codfw: maintenance 2460657 wikitext text/x-wiki d78f65r7r830p974lnsqz0ncv8znq2x 2460658 2460657 2026-09-24T18:34:32Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs-next: apply 2460658 wikitext text/x-wiki ns0refudba64ahu1em4c7y91k50r19r 2460659 2460658 2026-09-24T18:35:01Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2460659 wikitext text/x-wiki slk8d9vsl49w1fdq9ucc9s3efvzehg5 2460660 2460659 2026-09-24T18:36:27Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs-next: apply 2460660 wikitext text/x-wiki snvu3b04x3zh5fbx8v7f33kpebqj0t0 2460661 2460660 2026-09-24T18:37:17Z Stashbot 7414 herron@puppetserver1001: conftool action : set/pooled=true; selector: dnsdisc=thanos-web.*,name=codfw 2460661 wikitext text/x-wiki 4qsml75ufycp8mu2ypi0tjharqtave2 2460664 2460661 2026-09-24T18:38:41Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) pool wcqs in codfw: maintenance 2460664 wikitext text/x-wiki ttmbivaid381gmgdhg41v896y8c9mxq 2460665 2460664 2026-09-24T18:39:05Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.discovery.service-route pool wdqs-internal-main in codfw: maintenance 2460665 wikitext text/x-wiki 7glwvn75dpl9w4jsd5tc1ylzduljvgw 2460667 2460665 2026-09-24T18:39:17Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2460667 wikitext text/x-wiki qdc2oy8lrkshsz201msb7f2cts6v55a 2460668 2460667 2026-09-24T18:39:33Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1009.eqiad.wmnet 2460668 wikitext text/x-wiki 0xr4ik5h8mf3n49iwx1c6aw40tegdxf 2460669 2460668 2026-09-24T18:39:34Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1009.eqiad.wmnet 2460669 wikitext text/x-wiki 62ajvvyf6a1e2i8egb7qohgl3drxzx6 2460670 2460669 2026-09-24T18:39:40Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1010.eqiad.wmnet 2460670 wikitext text/x-wiki b3cyeg66ikyq9j6usj9zn9xby1k2fh1 2460671 2460670 2026-09-24T18:39:47Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2460671 wikitext text/x-wiki f145uxoasbzzs969maadbej17vt07vb 2460672 2460671 2026-09-24T18:39:59Z Stashbot 7414 herron@puppetserver1001: conftool action : set/pooled=true; selector: dnsdisc=thanos-query,name=codfw 2460672 wikitext text/x-wiki 1adjkloudpkkxg9n3c6k3p9owjtkiy1 2460673 2460672 2026-09-24T18:40:16Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1010.eqiad.wmnet 2460673 wikitext text/x-wiki bufkrxsu2myq628zlcuxxdxup5lh073 2460674 2460673 2026-09-24T18:42:19Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2460674 wikitext text/x-wiki s04hvo6xrvar5fwq13sngyxicuvdpa6 2460675 2460674 2026-09-24T18:42:43Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2460675 wikitext text/x-wiki 9oi387k0xc49mk21o89zrwew8vwou65 2460676 2460675 2026-09-24T18:42:52Z Stashbot 7414 herron@puppetserver1001: conftool action : set/pooled=true; selector: dnsdisc=thanos-swift,name=codfw 2460676 wikitext text/x-wiki 4tmf3owdf72220dd895bphrvpfkdi6a 2460678 2460676 2026-09-24T18:44:11Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) pool wdqs-internal-main in codfw: maintenance 2460678 wikitext text/x-wiki jsz0nv7a85u9u1icwbpvgddehjunh7r 2460679 2460678 2026-09-24T18:44:22Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.discovery.service-route pool wdqs-internal-scholarly in codfw: maintenance 2460679 wikitext text/x-wiki q8cuuvqp1ithla3cchrph0wyg1dsv86 2460681 2460679 2026-09-24T18:48:39Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1010.eqiad.wmnet 2460681 wikitext text/x-wiki dft5ynl5mjxedx0it1tqek7xr9nm5tq 2460682 2460681 2026-09-24T18:48:40Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1010.eqiad.wmnet 2460682 wikitext text/x-wiki k5j40uspvh09bt4i2548qfdt9kr2yn4 2460683 2460682 2026-09-24T18:48:46Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1011.eqiad.wmnet 2460683 wikitext text/x-wiki 7et82pm307sd2rvmvdtpxrddc26h2h2 2460684 2460683 2026-09-24T18:49:19Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1011.eqiad.wmnet 2460684 wikitext text/x-wiki o6fhzjnm763xonjjsjsczmkf9g2vkil 2460685 2460684 2026-09-24T18:49:27Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) pool wdqs-internal-scholarly in codfw: maintenance 2460685 wikitext text/x-wiki p1n9yn1ehllrtaf8i9fr37jji7age0j 2460686 2460685 2026-09-24T18:52:30Z Stashbot 7414 jhuneidi@deploy1003: Started scap sync-world: Backport for [[gerrit:1344753|REST: restore PageContentHelper::checkAccess (fix live breakage)]] 2460686 wikitext text/x-wiki l6iuhoruh0an26equ9cxi3tul0wdgqv 2460687 2460686 2026-09-24T18:55:17Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1011.eqiad.wmnet 2460687 wikitext text/x-wiki dzau10ksmjtaeadotjtybdk9hdkuw18 2460688 2460687 2026-09-24T18:55:18Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1011.eqiad.wmnet 2460688 wikitext text/x-wiki hkpxygfp5r09pfar98qv95f3v1i6r6l 2460689 2460688 2026-09-24T18:55:24Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1012.eqiad.wmnet 2460689 wikitext text/x-wiki 8c01lkqvw5tqavdph8oowq47xnq0gz8 2460690 2460689 2026-09-24T18:56:51Z Stashbot 7414 jhuneidi@deploy1003: daniel, jhuneidi: Backport for [[gerrit:1344753|REST: restore PageContentHelper::checkAccess (fix live breakage)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2460690 wikitext text/x-wiki f10behwg6tdyv47uycbzdlrsh1hheik 2460691 2460690 2026-09-24T18:57:29Z Stashbot 7414 jhuneidi@deploy1003: daniel, jhuneidi: Continuing with deployment 2460691 wikitext text/x-wiki l3xthswr9czt6h7hailnk9p9bbx4o0p 2460692 2460691 2026-09-24T19:02:45Z Stashbot 7414 jhuneidi@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344753|REST: restore PageContentHelper::checkAccess (fix live breakage)]] (duration: 10m 15s) 2460692 wikitext text/x-wiki lh8l4j5evw3d341a63t3o03xleh4kuo 2460694 2460692 2026-09-24T19:07:54Z Stashbot 7414 cdobbins@cumin1004: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ncredir5004.eqsin.wmnet with OS trixie 2460694 wikitext text/x-wiki l3dijryo78lv7bl8ipvj6bchl9vy0l9 2460695 2460694 2026-09-24T19:11:50Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=(kartotherian|tegola-vector-tiles) 2460695 wikitext text/x-wiki r1422i72ux9p4sscaiuvtkf60tvjsn1 2460696 2460695 2026-09-24T19:11:52Z Stashbot 7414 ryankemper: [Cirrus] Repooling codfw, chi first followed by the small clusters 2460696 wikitext text/x-wiki jtvhbjyifhxlfjp05wosq06dbxt636b 2460697 2460696 2026-09-24T19:12:10Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.discovery.service-route pool search in codfw: maintenance 2460697 wikitext text/x-wiki sr9nmqxqdh2lgnuk86mq2jepva58rix 2460698 2460697 2026-09-24T19:13:44Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.reimage for host ncredir5004.eqsin.wmnet with OS trixie 2460698 wikitext text/x-wiki rer48465y3hn68grf2tcx1onipu5bt2 2460699 2460698 2026-09-24T19:17:10Z Stashbot 7414 cdanis@cumin1004: conftool action : set/ttl=300; selector: name=codfw 2460699 wikitext text/x-wiki nj9jgbytatpfynxahokjl0ifrdxe0lk 2460700 2460699 2026-09-24T19:17:16Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) pool search in codfw: maintenance 2460700 wikitext text/x-wiki 1crul0clu7hi6f9nm8bd6wbvdmwo7d0 2460701 2460700 2026-09-24T19:17:50Z Stashbot 7414 ryankemper: [Cirrus] codfw chi (big cluster) repooled; metrics are already improving, I see poolcounter rejections dropping significantly 2460701 wikitext text/x-wiki 1ey0l84aeip8vy8fwgel94kc56nz5dt 2460702 2460701 2026-09-24T19:18:12Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.discovery.service-route pool search-psi in codfw: maintenance 2460702 wikitext text/x-wiki du0640sligbl4ckot1n96aa4w2sdmw4 2460703 2460702 2026-09-24T19:18:47Z Stashbot 7414 dzahn@cumin2003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host zuul1005.eqiad.wmnet with OS trixie 2460703 wikitext text/x-wiki 4azpp0gnim3ztbvjogssbea7beoocul 2460704 2460703 2026-09-24T19:20:19Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460704 wikitext text/x-wiki 208tmc12wizou21o22dfxmxy0z9endw 2460705 2460704 2026-09-24T19:20:22Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460705 wikitext text/x-wiki 5jbb0eitig5hnxrchrnepkkgeb1m5tk 2460708 2460705 2026-09-24T19:23:14Z Stashbot 7414 ryankemper@cumin2003: END (FAIL) - Cookbook sre.discovery.service-route (exit_code=99) pool search-psi in codfw: maintenance 2460708 wikitext text/x-wiki k9jzd8iji5240zmgmm76ugs0kgiuzrh 2460709 2460708 2026-09-24T19:24:16Z Stashbot 7414 ryankemper: [Cirrus] `dns.resolver.NoAnswer: The DNS response does not contain an answer to the question: search-psi.svc.eqiad.wmnet` checking briefly if this is real failure or just some TTL wonkiness 2460709 wikitext text/x-wiki tt0rcvoxc333td0jj5rmwnahn7lq5yu 2460710 2460709 2026-09-24T19:25:28Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1012.eqiad.wmnet 2460710 wikitext text/x-wiki g4fl99sz1lba2ftem87hhf16cndttne 2460711 2460710 2026-09-24T19:26:41Z Stashbot 7414 ryankemper: [Cirrus] nevermind, that's just the cookbook assuming the DNS record should exist, which it doesn't because chi/psi/omega all share `search.svc.$DC.wmnet` 2460711 wikitext text/x-wiki khcggkteglqvbbpudgfts24njrgpzkj 2460712 2460711 2026-09-24T19:27:52Z Stashbot 7414 cdanis@cumin1004: conftool action : set/pooled=true; selector: name=codfw,dnsdisc=k8s-ingress-aux-ro 2460712 wikitext text/x-wiki kpcd8yl3ret3rgg0qf8fx70vpzr1160 2460713 2460712 2026-09-24T19:28:27Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.discovery.service-route pool search-omega in codfw: maintenance 2460713 wikitext text/x-wiki 24apv43aapxg6p9o9862pxzhbyclmm7 2460714 2460713 2026-09-24T19:29:24Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460714 wikitext text/x-wiki 7k15tiqtrp131uc465g5omc8zfighoz 2460715 2460714 2026-09-24T19:29:26Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460715 wikitext text/x-wiki bqenomkq7fnhkc3d8aitd6sgd3nakox 2460716 2460715 2026-09-24T19:31:25Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1012.eqiad.wmnet 2460716 wikitext text/x-wiki 8kpm5a9i9qrk5xu6rr2aa6q0zu59p46 2460717 2460716 2026-09-24T19:31:26Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1012.eqiad.wmnet 2460717 wikitext text/x-wiki ixv5dlt7e34cgutju20nnmmdnigmk9v 2460718 2460717 2026-09-24T19:31:32Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1013.eqiad.wmnet 2460718 wikitext text/x-wiki beo1o9944ao38b6ps3ssi5jkgi4u07v 2460719 2460718 2026-09-24T19:33:30Z Stashbot 7414 ryankemper@cumin2003: END (FAIL) - Cookbook sre.discovery.service-route (exit_code=99) pool search-omega in codfw: maintenance 2460719 wikitext text/x-wiki k8xt0wikg4qfa15lvf80oe1fyls9em2 2460720 2460719 2026-09-24T19:34:12Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460720 wikitext text/x-wiki lfodroorqemlrbuclabqvibqxzxmb1k 2460721 2460720 2026-09-24T19:34:16Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460721 wikitext text/x-wiki gft585xw3kapxz7ytxv8qippsez8dzs 2460723 2460721 2026-09-24T19:36:12Z Stashbot 7414 ryankemper: [Cirrus] All cirrus pools are serving again. Actively monitoring while the system returns to equilibrium, but all initial indications are that things are as they should be 2460723 wikitext text/x-wiki 4modv0yh1a1ji5nzqvlbmh9ezdvtfkq 2460725 2460723 2026-09-24T19:46:15Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.hosts.reimage for host zuul1005.eqiad.wmnet with OS trixie 2460725 wikitext text/x-wiki 6zh5uocwd41pqprrmvzkmv82fh68fmi 2460726 2460725 2026-09-24T19:50:28Z Stashbot 7414 brett@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on A:cp-upload_magru and not P{cp7011.magru.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2460726 wikitext text/x-wiki 09a5d7074z4hgfcydgut0zgfxynsutu 2460728 2460726 2026-09-24T19:56:06Z Stashbot 7414 cdobbins@cumin1004: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ncredir5004.eqsin.wmnet with OS trixie 2460728 wikitext text/x-wiki hxw4gydvh1u4ldh67avy1cy4g4ahohm 2460729 2460728 2026-09-24T19:56:43Z Stashbot 7414 cjming@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen: apply 2460729 wikitext text/x-wiki 854cshgdwtigl8xllzd9bjy9rtm1tbf 2460730 2460729 2026-09-24T19:57:22Z Stashbot 7414 cjming@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen: apply 2460730 wikitext text/x-wiki gkz0knz6apeyki4r5nqxbq287t5vrz5 2460731 2460730 2026-09-24T20:01:35Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1013.eqiad.wmnet 2460731 wikitext text/x-wiki gipaq8zkw1248aavrwkpypu7irdz9c8 2460735 2460731 2026-09-24T20:08:20Z Stashbot 7414 kemayo@deploy1003: Started scap sync-world: Backport for [[gerrit:1344714|EditCheck: add some statsv tracking of check/suggestion actions (T438916)]] 2460735 wikitext text/x-wiki h7vlz8t5omgb0mjs7dt5u8xdl8t8gy7 2460736 2460735 2026-09-24T20:09:36Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1013.eqiad.wmnet 2460736 wikitext text/x-wiki k6jrcbbqb7r152ndflo5a1ukjlk7o1d 2460737 2460736 2026-09-24T20:09:38Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1013.eqiad.wmnet 2460737 wikitext text/x-wiki nmhhfx8wm58jyqw8f3c5zgh8fd11nfy 2460738 2460737 2026-09-24T20:09:43Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1014.eqiad.wmnet 2460738 wikitext text/x-wiki 83fu8yvjn6qyhcgjgcf1th3x4pjmtxl 2460739 2460738 2026-09-24T20:12:34Z Stashbot 7414 kemayo@deploy1003: kemayo: Backport for [[gerrit:1344714|EditCheck: add some statsv tracking of check/suggestion actions (T438916)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2460739 wikitext text/x-wiki 5l6r8wyellnkbutl8cxvmq3x8qdrn3e 2460740 2460739 2026-09-24T20:14:26Z Stashbot 7414 kemayo@deploy1003: kemayo: Continuing with deployment 2460740 wikitext text/x-wiki rl8lfplry4h7nq9t5liulq95win3kmu 2460746 2460740 2026-09-24T20:19:44Z Stashbot 7414 kemayo@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344714|EditCheck: add some statsv tracking of check/suggestion actions (T438916)]] (duration: 11m 23s) 2460746 wikitext text/x-wiki h5w7hvub1ko4u3301lyjoufvuqzfzx7 2460747 2460746 2026-09-24T20:25:33Z Stashbot 7414 brett@cumin1004: END (FAIL) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=1) rolling upgrade of Varnish on A:cp-upload_ulsfo - 7.1.1-2~bpo13+wmf3 () 2460747 wikitext text/x-wiki 2aa318lx1jfxocjqgb4gdttqrfb8soo 2460748 2460747 2026-09-24T20:25:34Z Stashbot 7414 brett@cumin1004: END (FAIL) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=1) rolling upgrade of Varnish on A:cp-text_ulsfo - 7.1.1-2~bpo13+wmf3 () 2460748 wikitext text/x-wiki o6dcftbm7o57m1sb2kfekhhky4wrwur 2460751 2460748 2026-09-24T20:34:15Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460751 wikitext text/x-wiki pqf4mp8i44auk7xph9ginrwo7k77mxv 2460752 2460751 2026-09-24T20:34:19Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460752 wikitext text/x-wiki o9f2bymw1998k7nurlkcyzrj3uwl0jd 2460753 2460752 2026-09-24T20:39:11Z Stashbot 7414 catrope@deploy1003: Started scap sync-world: Backport for [[gerrit:1344388|HookHandler: Guard against recovery code expiry being null (T438593)]] 2460753 wikitext text/x-wiki bkhrnbf7ylb4ov2jca60gwfxa9b171v 2460754 2460753 2026-09-24T20:39:49Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1014.eqiad.wmnet 2460754 wikitext text/x-wiki ns1ikycyeiuud2crdk6qr0csxnprrrd 2460756 2460754 2026-09-24T20:43:23Z Stashbot 7414 catrope@deploy1003: catrope: Backport for [[gerrit:1344388|HookHandler: Guard against recovery code expiry being null (T438593)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2460756 wikitext text/x-wiki iud00hbcx6pbmw9l0lcvi910j7gb7ll 2460757 2460756 2026-09-24T20:43:38Z Stashbot 7414 brett@cumin1004: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on P{cp404[1-6].ulsfo.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2460757 wikitext text/x-wiki nudagn5sf5r10zrcz9yrhj0cvxork7l 2460758 2460757 2026-09-24T20:44:12Z Stashbot 7414 catrope@deploy1003: catrope: Continuing with deployment 2460758 wikitext text/x-wiki i53g1o5x1eqtxtxtj6dsoz3tbgbuufy 2460759 2460758 2026-09-24T20:44:32Z Stashbot 7414 brett@cumin1004: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on P{cp405[1-2].ulsfo.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2460759 wikitext text/x-wiki apl7s4dcxxtdrdr9hn68m35hd0wovy6 2460761 2460759 2026-09-24T20:47:55Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs: apply 2460761 wikitext text/x-wiki 7kvvlkz3pzmuilh12npdjn9cw2b0ers 2460762 2460761 2026-09-24T20:48:24Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs: apply 2460762 wikitext text/x-wiki ik4is5dg1o9bsztgh1wk17839byg43r 2460763 2460762 2026-09-24T20:48:44Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs: apply 2460763 wikitext text/x-wiki qq53pfl1ycuyhjnfuck1usgo5obaini 2460764 2460763 2026-09-24T20:49:07Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs: apply 2460764 wikitext text/x-wiki louhtfkb6006jhbky1j5a6m4a2wtxnf 2460765 2460764 2026-09-24T20:49:35Z Stashbot 7414 catrope@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344388|HookHandler: Guard against recovery code expiry being null (T438593)]] (duration: 10m 19s) 2460765 wikitext text/x-wiki redv5ozecz49j01y4777c54115v2638 2460766 2460765 2026-09-24T20:50:47Z Stashbot 7414 brett@cumin1004: END (FAIL) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=1) rolling upgrade of Varnish on P{cp405[1-2].ulsfo.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2460766 wikitext text/x-wiki dsalrl7qtufr81jg46tbnzcv96lmgw0 2460767 2460766 2026-09-24T20:53:17Z Stashbot 7414 catrope@deploy1003: Started scap sync-world: Backport for [[gerrit:1344750|Revert "cirrus: Send more_like traffic to eqiad"]], [[gerrit:1344329|prv: Enable parsoid rendering for 5 wikis (T438998)]] 2460767 wikitext text/x-wiki l4rjgn0zzz6uoe6q5qpb0tidtdb5aju 2460768 2460767 2026-09-24T20:55:07Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460768 wikitext text/x-wiki 10a6kuxvrsurnhf9wzbaxwloyrbdh20 2460769 2460768 2026-09-24T20:55:08Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2460769 wikitext text/x-wiki 3v80d2n4otqy31ayi289hqfylbqgr0g 2460770 2460769 2026-09-24T20:56:53Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1014.eqiad.wmnet 2460770 wikitext text/x-wiki 1shmi68gde8pm83vtsgorxypxzfwrhp 2460771 2460770 2026-09-24T20:56:54Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1014.eqiad.wmnet 2460771 wikitext text/x-wiki fs56venvlo2hwwj2n1trdwhhd3yp6jf 2460772 2460771 2026-09-24T20:56:56Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1015.eqiad.wmnet 2460772 wikitext text/x-wiki 6hlge6mwszweg4ppz9hx1wvt688tfwx 2460773 2460772 2026-09-24T20:57:14Z Stashbot 7414 brett@cumin1004: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on P{cp405[1-2].ulsfo.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2460773 wikitext text/x-wiki mut70j5ibgrkev1c7805ywvzsbkih5g 2460774 2460773 2026-09-24T20:57:26Z Stashbot 7414 catrope@deploy1003: catrope, ebernhardson, jgiannelos: Backport for [[gerrit:1344750|Revert "cirrus: Send more_like traffic to eqiad"]], [[gerrit:1344329|prv: Enable parsoid rendering for 5 wikis (T438998)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2460774 wikitext text/x-wiki cv03gkgqvgqxar8utzv69h0oqxafenl 2460775 2460774 2026-09-24T20:58:36Z Stashbot 7414 catrope@deploy1003: catrope, ebernhardson, jgiannelos: Continuing with deployment 2460775 wikitext text/x-wiki c74elatt7xdzu7bnrqq4kjltwdcc03b 2460776 2460775 2026-09-24T21:02:29Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp4041.* 2460776 wikitext text/x-wiki jshnxyxvr8i7y6et03yg9j2jfhcplj4 2460777 2460776 2026-09-24T21:03:56Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp4051.* 2460777 wikitext text/x-wiki qmxy4s874x67ixq45613dbjjaxgt1gl 2460778 2460777 2026-09-24T21:04:02Z Stashbot 7414 catrope@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344750|Revert "cirrus: Send more_like traffic to eqiad"]], [[gerrit:1344329|prv: Enable parsoid rendering for 5 wikis (T438998)]] (duration: 10m 45s) 2460778 wikitext text/x-wiki 6r1nd1f24u9lko8p0oqt1b3d1ccek6x 2460779 2460778 2026-09-24T21:08:38Z Stashbot 7414 catrope@deploy1003: Started scap sync-world: Backport for [[gerrit:1344406|ext.wikimediaEvents.testKitchen: Add withContext helper (T438898)]], [[gerrit:1344716|ReaderExperiments: add dewiki and svwiki (T438072)]], [[gerrit:1344740|Image Browsing carousel: taps outside the preview dialog should close it (T439006)]], [[gerrit:1344752|Cap the dialog viewport (T439007)]] 2460779 wikitext text/x-wiki 2jb9z0l32u5pw6fat9ya95br1q2iwry 2460780 2460779 2026-09-24T21:12:46Z Stashbot 7414 catrope@deploy1003: cjming, mfossati, catrope, mlitn: Backport for [[gerrit:1344406|ext.wikimediaEvents.testKitchen: Add withContext helper (T438898)]], [[gerrit:1344716|ReaderExperiments: add dewiki and svwiki (T438072)]], [[gerrit:1344740|Image Browsing carousel: taps outside the preview dialog should close it (T439006)]], [[gerrit:1344752|Cap the dialog viewport (T439007)]] synced to the testservers (see https://wi 2460780 wikitext text/x-wiki q50d3u6z486kidtajtt5ydxuao7ni11 2460781 2460780 2026-09-24T21:22:43Z Stashbot 7414 catrope@deploy1003: cjming, mfossati, catrope, mlitn: Continuing with deployment 2460781 wikitext text/x-wiki qr9fykmbvjp5gaxyqn2u6lyybgjde8d 2460782 2460781 2026-09-24T21:26:58Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1015.eqiad.wmnet 2460782 wikitext text/x-wiki li5qq6u11obwrqmsozfzvk4jmimh8rj 2460783 2460782 2026-09-24T21:28:04Z Stashbot 7414 catrope@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344406|ext.wikimediaEvents.testKitchen: Add withContext helper (T438898)]], [[gerrit:1344716|ReaderExperiments: add dewiki and svwiki (T438072)]], [[gerrit:1344740|Image Browsing carousel: taps outside the preview dialog should close it (T439006)]], [[gerrit:1344752|Cap the dialog viewport (T439007)]] (duration: 19m 27s) 2460783 wikitext text/x-wiki tl735xs7dymk1uhkdozxf9zdw2bbzaj 2460784 2460783 2026-09-24T21:28:54Z Stashbot 7414 catrope@deploy1003: Started scap sync-world: Backport for [[gerrit:1344795|Catch newline character in UserMailer to prevent it from allowing bad actors to create an additional header (T434545)]] 2460784 wikitext text/x-wiki nvas2sv5ptloeg2a1xk67wdg0hcvghf 2460785 2460784 2026-09-24T21:33:05Z Stashbot 7414 catrope@deploy1003: catrope: Backport for [[gerrit:1344795|Catch newline character in UserMailer to prevent it from allowing bad actors to create an additional header (T434545)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2460785 wikitext text/x-wiki 63az3vm3a76g4buzaqjd3ase8la35re 2460786 2460785 2026-09-24T21:34:04Z Stashbot 7414 brett@cumin1004: END (FAIL) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=1) rolling upgrade of Varnish on P{cp405[1-2].ulsfo.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2460786 wikitext text/x-wiki 37nzfjyqqe9m8aqgt2znbmc2lji1ofk 2460788 2460786 2026-09-24T21:34:57Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1015.eqiad.wmnet 2460788 wikitext text/x-wiki k62pk6co3pqdgc1bnoyvjgv3j4x28of 2460789 2460788 2026-09-24T21:34:59Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1015.eqiad.wmnet 2460789 wikitext text/x-wiki rxfegkqt0q0dlxrxx9m96cwf8hxybxq 2460790 2460789 2026-09-24T21:35:04Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1016.eqiad.wmnet 2460790 wikitext text/x-wiki kbe2g9rr6a0zvivyudap7fkhhoafed8 2460791 2460790 2026-09-24T21:35:35Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1016.eqiad.wmnet 2460791 wikitext text/x-wiki nfhe200bey7znw4nobjvle1c4ysmb14 2460792 2460791 2026-09-24T21:40:44Z Stashbot 7414 catrope@deploy1003: catrope: Continuing with deployment 2460792 wikitext text/x-wiki 4sf82wq4sd3k1wdsluw9j1cubda1ckn 2460793 2460792 2026-09-24T21:42:20Z Stashbot 7414 brett@cumin1004: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on P{cp4052.ulsfo.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2460793 wikitext text/x-wiki d34q98n0s1usm4cpk6ftql4ln7lxm3h 2460794 2460793 2026-09-24T21:45:59Z Stashbot 7414 catrope@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344795|Catch newline character in UserMailer to prevent it from allowing bad actors to create an additional header (T434545)]] (duration: 17m 05s) 2460794 wikitext text/x-wiki 4o4qiybbp0rd5q4r1brn8y7sf33wue6 2460795 2460794 2026-09-24T21:46:11Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1016.eqiad.wmnet 2460795 wikitext text/x-wiki 6oc9zhqubmr93dfr2n3035vc6xq0vys 2460796 2460795 2026-09-24T21:46:12Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1016.eqiad.wmnet 2460796 wikitext text/x-wiki argq2s02u9w42beqvpl446yr4mev0vq 2460797 2460796 2026-09-24T21:46:18Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1018.eqiad.wmnet 2460797 wikitext text/x-wiki fq9mh7nnd5iwjwl3plkqg084uf3gtr5 2460798 2460797 2026-09-24T21:46:56Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1018.eqiad.wmnet 2460798 wikitext text/x-wiki ein3ol2dmyfmuglme5zgpf6e9lwr9sj 2460799 2460798 2026-09-24T21:48:16Z Stashbot 7414 brett@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on P{cp4052.ulsfo.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2460799 wikitext text/x-wiki p72uh4502w0zn2wozv8r0a25bzwewnu 2460800 2460799 2026-09-24T21:53:06Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1018.eqiad.wmnet 2460800 wikitext text/x-wiki bdz81zphyg7vprofafvalcxndo1f80m 2460801 2460800 2026-09-24T21:53:07Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1018.eqiad.wmnet 2460801 wikitext text/x-wiki s8zpv3gs9xz1pt6euldktvlf912a9yk 2460802 2460801 2026-09-24T21:53:13Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1019.eqiad.wmnet 2460802 wikitext text/x-wiki biztsj2egktbuiwvcqp9jullca4ac7b 2460803 2460802 2026-09-24T21:53:45Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1019.eqiad.wmnet 2460803 wikitext text/x-wiki giszvtwkkqqtwfiy4zb0fmpmtcxre0s 2460804 2460803 2026-09-24T21:57:24Z Stashbot 7414 jhuneidi@deploy1003: rebuilt and synchronized wikiversions files: group2 to 1.47.0-wmf.21 refs T438217 2460804 wikitext text/x-wiki d0e1d6h17tst6mdivfhjias2aarb1ci 2460805 2460804 2026-09-24T21:58:25Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp4052.* 2460805 wikitext text/x-wiki q0u5gno565ziyt9htxmzjmiojbxzo20 2460806 2460805 2026-09-24T22:00:21Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1019.eqiad.wmnet 2460806 wikitext text/x-wiki rnj51bldkgzibbf180w018gqvz0nmb7 2460807 2460806 2026-09-24T22:00:23Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1019.eqiad.wmnet 2460807 wikitext text/x-wiki rigkybnnd2qqx2x4w23bt7fcehyoxnz 2460808 2460807 2026-09-24T22:00:28Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1020.eqiad.wmnet 2460808 wikitext text/x-wiki ogma0gwfa64g6fpilfylqxfl2w50ztd 2460809 2460808 2026-09-24T22:02:44Z Stashbot 7414 btullis@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host dse-k8s-worker1020.eqiad.wmnet 2460809 wikitext text/x-wiki 3lr2slp4v0djbxif85rt0l0w8qud9fo 2460810 2460809 2026-09-24T22:04:36Z Stashbot 7414 brett@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on A:cp-text_magru and not P{cp7001.magru.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2460810 wikitext text/x-wiki b7522dw3o5zj9451xlj0m2cnpuqq0ae 2460811 2460810 2026-09-24T22:10:08Z Stashbot 7414 egardner@deploy1003: Started scap sync-world: Backport for [[gerrit:1344049|ReaderExperiments: Set the preferred-sources debug flag on testwiki (T436692)]], [[gerrit:1344050|ReaderExperiments: Drop the stale ShareHighlight config var (T424764)]], [[gerrit:1344118|Enable ReadingList CTA on Minerva for our test wikis (inc beta cluster) (T438779)]], [[gerrit:1343560|Revert "Enable Reading Recommendations experiment on te 2460811 wikitext text/x-wiki lzu42g7v3csomlnylymczj7jt0x0x5c 2460812 2460811 2026-09-24T22:14:09Z Stashbot 7414 egardner@deploy1003: volker-e, egardner, jdlrobson: Backport for [[gerrit:1344049|ReaderExperiments: Set the preferred-sources debug flag on testwiki (T436692)]], [[gerrit:1344050|ReaderExperiments: Drop the stale ShareHighlight config var (T424764)]], [[gerrit:1344118|Enable ReadingList CTA on Minerva for our test wikis (inc beta cluster) (T438779)]], [[gerrit:1343560|Revert "Enable Reading Recommendations experiment 2460812 wikitext text/x-wiki dsvo1zwu58kn7n86uik9y30iv3y6sp2 2460813 2460812 2026-09-24T22:19:34Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1020.eqiad.wmnet 2460813 wikitext text/x-wiki 9l88i9h8c3328m7ho5mdwg6m2pxbaxe 2460814 2460813 2026-09-24T22:19:35Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1020.eqiad.wmnet 2460814 wikitext text/x-wiki oj5fj5wpmxhcc7r9eceg8myzutsw89k 2460815 2460814 2026-09-24T22:19:41Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1021.eqiad.wmnet 2460815 wikitext text/x-wiki 9b3uhl4dp3uyngcov0b97e32ukb5gf6 2460816 2460815 2026-09-24T22:21:45Z Stashbot 7414 btullis@cumin1004: END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host dse-k8s-worker1021.eqiad.wmnet 2460816 wikitext text/x-wiki lw8o0pzxfuzbsx3pmjvwxmwe60lzg2s 2460817 2460816 2026-09-24T22:22:17Z Stashbot 7414 egardner@deploy1003: volker-e, egardner, jdlrobson: Continuing with deployment 2460817 wikitext text/x-wiki n9kt730f0wsis0r90aw2at03977owhw 2460818 2460817 2026-09-24T22:27:37Z Stashbot 7414 egardner@deploy1003: Finished scap sync-world: Backport for [[gerrit:1344049|ReaderExperiments: Set the preferred-sources debug flag on testwiki (T436692)]], [[gerrit:1344050|ReaderExperiments: Drop the stale ShareHighlight config var (T424764)]], [[gerrit:1344118|Enable ReadingList CTA on Minerva for our test wikis (inc beta cluster) (T438779)]], [[gerrit:1343560|Revert "Enable Reading Recommendations experiment on t 2460818 wikitext text/x-wiki ahk2qbi6ccia67919vjlggq8gdz4kjy 2460820 2460818 2026-09-24T22:28:50Z Stashbot 7414 ryankemper: [WDQS] Expanding match in https://requestctl.wikimedia.org/pattern/ua/rocks to test a likely block candidate 2460820 wikitext text/x-wiki j7o9wiadj6ngpa3zbkwnw90jf0xmdil 2460821 2460820 2026-09-24T22:41:17Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1021.eqiad.wmnet 2460821 wikitext text/x-wiki 6fymgos9ppkfzch5vjx6ao72iczw4wz 2460822 2460821 2026-09-24T22:41:18Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1021.eqiad.wmnet 2460822 wikitext text/x-wiki juge4t4p7wqm7nuefi3wf5htlg8b32f 2460823 2460822 2026-09-24T22:41:24Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1022.eqiad.wmnet 2460823 wikitext text/x-wiki gr998qcekhci4r407zz1t0iy4vuymxn 2460824 2460823 2026-09-24T23:11:41Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1022.eqiad.wmnet 2460824 wikitext text/x-wiki g1e8ofyklj20mutnm65iy8adi7dtl90 2460825 2460824 2026-09-24T23:20:34Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1022.eqiad.wmnet 2460825 wikitext text/x-wiki d9exf6bljlw808s4vq0cvj3jgs6lnol 2460826 2460825 2026-09-24T23:20:35Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1022.eqiad.wmnet 2460826 wikitext text/x-wiki 9tfj4iiow5k1a3kdoysvaox3telgftz 2460827 2460826 2026-09-24T23:20:41Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1023.eqiad.wmnet 2460827 wikitext text/x-wiki 2hvz2rt1ejq3bb9571i6nm9er3s2isy 2460828 2460827 2026-09-24T23:32:17Z Stashbot 7414 brett@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on P{cp404[1-6].ulsfo.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2460828 wikitext text/x-wiki ltakjm2v4zxs5dwy9jdeuurh7ienosp 2460829 2460828 2026-09-24T23:50:45Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1023.eqiad.wmnet 2460829 wikitext text/x-wiki fvws604mlckv2v6be5mofrtln8dmwc6 2460830 2460829 2026-09-24T23:57:24Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1023.eqiad.wmnet 2460830 wikitext text/x-wiki qvnsc4vu3enqjmaabpzpam3ehzynau3 2460831 2460830 2026-09-24T23:57:25Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1023.eqiad.wmnet 2460831 wikitext text/x-wiki krtbpi97us12xw5md6ppz7v2xvu4y6b 2460832 2460831 2026-09-24T23:57:31Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1024.eqiad.wmnet 2460832 wikitext text/x-wiki od7wvmccg246z8t0frj3r4tiz8mkp3e 2460833 2460832 2026-09-24T23:58:06Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1024.eqiad.wmnet 2460833 wikitext text/x-wiki 9i9qy1jh6w4c26y92psktalk9wvnoi2 2460835 2460833 2026-09-25T00:06:58Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1024.eqiad.wmnet 2460835 wikitext text/x-wiki e4vvm4ugvlksk7n24rdt5vovp13uzy0 2460836 2460835 2026-09-25T00:06:59Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1024.eqiad.wmnet 2460836 wikitext text/x-wiki tjj103ezqxg24ivfvgzvh6rt0l9ky6k 2460837 2460836 2026-09-25T00:07:04Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1025.eqiad.wmnet 2460837 wikitext text/x-wiki ic8aauiooxmzsfi4yzgzo7mmu5u39yy 2460838 2460837 2026-09-25T00:07:40Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1025.eqiad.wmnet 2460838 wikitext text/x-wiki i3wig10so64p01djfwkq4cfwynjdykt 2460839 2460838 2026-09-25T00:14:21Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1025.eqiad.wmnet 2460839 wikitext text/x-wiki b5hoqn4y2h44i9un6xkaprvjqdg79rn 2460840 2460839 2026-09-25T00:14:22Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1025.eqiad.wmnet 2460840 wikitext text/x-wiki q3xvfc5w0gomq9hndg5uknyudqvpewc 2460841 2460840 2026-09-25T00:14:31Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1026.eqiad.wmnet 2460841 wikitext text/x-wiki 45kc2c56blgzb2o5k4air6m942iadpw 2460842 2460841 2026-09-25T00:44:34Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1026.eqiad.wmnet 2460842 wikitext text/x-wiki 6huncal92kxi2w6299em75v8wg3gnee 2460843 2460842 2026-09-25T00:53:15Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1026.eqiad.wmnet 2460843 wikitext text/x-wiki d6pbie6n6944n4p6ba6vhsvp7btdaqp 2460844 2460843 2026-09-25T00:53:17Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1026.eqiad.wmnet 2460844 wikitext text/x-wiki 2eqf9q7vw4i8rt3h0s9u1jqklhejpcm 2460845 2460844 2026-09-25T00:53:22Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1027.eqiad.wmnet 2460845 wikitext text/x-wiki 37qibvoxbk0jvhrsumwmwwn2ofme7ti 2460846 2460845 2026-09-25T00:53:57Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1027.eqiad.wmnet 2460846 wikitext text/x-wiki 79ts4z0f9s2ygsy0yshuajy5l5hot3y 2460847 2460846 2026-09-25T01:00:22Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1027.eqiad.wmnet 2460847 wikitext text/x-wiki sfbnie91z6ffltvo5g9b2yzq1pl69js 2460848 2460847 2026-09-25T01:00:23Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1027.eqiad.wmnet 2460848 wikitext text/x-wiki f2tiadphjqqv12e07p6sawe9bpt15wa 2460849 2460848 2026-09-25T01:00:28Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1028.eqiad.wmnet 2460849 wikitext text/x-wiki 71lfeyuqhbzwl4zljlj3r5uot2756vi 2460850 2460849 2026-09-25T01:30:30Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1028.eqiad.wmnet 2460850 wikitext text/x-wiki 67rg2uckz6fvkb5szpm60gozmtjtp4t 2460852 2460850 2026-09-25T01:41:42Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1028.eqiad.wmnet 2460852 wikitext text/x-wiki bs0kbbzs4f0qi3iqfcfnk6ahd3mqayq 2460853 2460852 2026-09-25T01:41:43Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1028.eqiad.wmnet 2460853 wikitext text/x-wiki 4c44mvi4zltxmsfb7iw84h1x0xtiaip 2460854 2460853 2026-09-25T01:41:43Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on P{dse-k8s-worker1*.eqiad.wmnet} and (A:dse-k8s-master-eqiad or A:dse-k8s-worker-eqiad) 2460854 wikitext text/x-wiki qq8foklxyxbjttjdl4j0qy7mlulapqi 2460860 2460854 2026-09-25T02:00:36Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2460860 wikitext text/x-wiki bwhyvcnac3isoodqse5wwmj2qc3p46a 2460861 2460860 2026-09-25T02:08:14Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 07m 38s) 2460861 wikitext text/x-wiki mdvmblr8zysorkojq5veyow8wiuw2c9 2460862 2460861 2026-09-25T05:21:41Z Stashbot 7414 ryankemper: [Cirrus] Stumble across orphaned index `sawikisource_content_1784136042`, deleted. The real index is `sawikisource_content_1784136826` which I've obviously left untouched 2460862 wikitext text/x-wiki ge0n2ht7bbs48mdjqk54md5ahl6c6cj 2460863 2460862 2026-09-25T05:58:05Z Stashbot 7414 dcausse@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-semantic-search-test: apply 2460863 wikitext text/x-wiki lq8xt5f29905s5qjo8g2hyps9r6hdyt 2460864 2460863 2026-09-25T05:58:14Z Stashbot 7414 dcausse@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-test: apply 2460864 wikitext text/x-wiki 7vs94a7j7xacagogs8md1myr59f9oew 2460867 2460864 2026-09-25T08:15:44Z Stashbot 7414 vgutierrez@puppetserver1001: conftool action : set/pooled=no; selector: dc=codfw,name=cp2059.* 2460867 wikitext text/x-wiki g4w0l13gm1x9zyyocfghmopf7q7uwfm 2460868 2460867 2026-09-25T08:20:13Z Stashbot 7414 vgutierrez@puppetserver1001: conftool action : set/weight=1; selector: dc=codfw,name=cp2059.* 2460868 wikitext text/x-wiki h3e1q4atpy3hjdf0u4kp2hpcy5n3dej 2460869 2460868 2026-09-25T08:22:57Z Stashbot 7414 elukey@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'sync'. 2460869 wikitext text/x-wiki c2jwhlhds1vke8v8ch212n04r4aqzsj 2460870 2460869 2026-09-25T08:23:18Z Stashbot 7414 elukey@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'sync'. 2460870 wikitext text/x-wiki fziksspr3m2jz0m5myiuxf7rpqobkdl 2460871 2460870 2026-09-25T08:23:53Z Stashbot 7414 elukey@deploy1003: helmfile [staging-codfw] START helmfile.d/admin 'sync'. 2460871 wikitext text/x-wiki 7klkpvij6p4ggxa2ft2jp0lrn3jv4tb 2460872 2460871 2026-09-25T08:24:35Z Stashbot 7414 elukey@deploy1003: helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. 2460872 wikitext text/x-wiki 8voa0eoq4acxlh3cyssz6ifjk6l89t3 2460874 2460872 2026-09-25T08:52:20Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.decommission for hosts build2001.codfw.wmnet 2460874 wikitext text/x-wiki g5j1qfdfb82zo3bjp07eyycr2ywyz9j 2460875 2460874 2026-09-25T08:57:21Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.dns.netbox 2460875 wikitext text/x-wiki 76wtxt4xu3se1p1l1evi0c908toykjn 2460876 2460875 2026-09-25T08:57:33Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reboot-single for host an-worker1207.eqiad.wmnet 2460876 wikitext text/x-wiki j5bm4excq2w0icu7ot48ngcaxyqpxki 2460878 2460876 2026-09-25T09:01:41Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: build2001.codfw.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin2003" 2460878 wikitext text/x-wiki 5nn2e6fwv8gnga394n00bdn4ry5tf75 2460879 2460878 2026-09-25T09:11:43Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host an-worker1207.eqiad.wmnet 2460879 wikitext text/x-wiki 4y3kmnplcvg1st5p7z62xlv56cco4uq 2460880 2460879 2026-09-25T09:28:41Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: build2001.codfw.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin2003" 2460880 wikitext text/x-wiki kea209z0610b2c7jzsozfu5tfi0u4ht 2460881 2460880 2026-09-25T09:28:42Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2460881 wikitext text/x-wiki hu0tnm90w6nt0ew5fz9u1rqrz36fa93 2460882 2460881 2026-09-25T09:28:43Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts build2001.codfw.wmnet 2460882 wikitext text/x-wiki 4uad5b1xicqubcxwkgkmsmbpevwwqsy 2460883 2460882 2026-09-25T09:35:03Z Stashbot 7414 cwilliams@cumin1004: START - Cookbook sre.switchdc.databases.prepare for the switch from eqiad to codfw for section test-s4 2460883 wikitext text/x-wiki mm2rd57yekfpfo2dwltzgye4tamqedq 2460884 2460883 2026-09-25T09:36:02Z Stashbot 7414 cwilliams@cumin1004: END (PASS) - Cookbook sre.switchdc.databases.prepare (exit_code=0) for the switch from eqiad to codfw for section test-s4 2460884 wikitext text/x-wiki 90rzkj9pfb58xgv52fw715ii4q5qz1i 2460885 2460884 2026-09-25T09:36:30Z Stashbot 7414 cwilliams@cumin1004: START - Cookbook sre.switchdc.databases.finalize for the switch from eqiad to codfw for section test-s4 2460885 wikitext text/x-wiki jntldktw0mdgww5re9uf7643a3tbcmc 2460886 2460885 2026-09-25T09:36:47Z Stashbot 7414 cwilliams@cumin1004: END (PASS) - Cookbook sre.switchdc.databases.finalize (exit_code=0) for the switch from eqiad to codfw for section test-s4 2460886 wikitext text/x-wiki tmpcs61ko8zb6kga537ckebqafkh76i 2460887 2460886 2026-09-25T09:43:37Z Stashbot 7414 tappof: reset modified_attributes for hosts and services that fully match the Puppet configuration in Icinga - T439105 2460887 wikitext text/x-wiki jaq41chyw91nkdp6l861t48tb2kcxyi 2460890 2460887 2026-09-25T09:48:06Z Stashbot 7414 cwilliams@cumin1004: START - Cookbook sre.switchdc.databases.prepare for the switch from codfw to eqiad for section test-s4 2460890 wikitext text/x-wiki 8qgfs4891ps6z18lt6bv5ssx5y422sh 2460891 2460890 2026-09-25T09:49:07Z Stashbot 7414 cwilliams@cumin1004: END (PASS) - Cookbook sre.switchdc.databases.prepare (exit_code=0) for the switch from codfw to eqiad for section test-s4 2460891 wikitext text/x-wiki 78y5pvrte54k223q8ub3mfl3o335hom 2460892 2460891 2026-09-25T09:49:28Z Stashbot 7414 cwilliams@cumin1004: START - Cookbook sre.switchdc.databases.finalize for the switch from codfw to eqiad for section test-s4 2460892 wikitext text/x-wiki haoq85snf9i5e2rf26cte70pwftr04t 2460893 2460892 2026-09-25T09:49:45Z Stashbot 7414 cwilliams@cumin1004: END (PASS) - Cookbook sre.switchdc.databases.finalize (exit_code=0) for the switch from codfw to eqiad for section test-s4 2460893 wikitext text/x-wiki 5e12brn5o83e3v0lcyrearbn5766kxe 2460894 2460893 2026-09-25T10:52:41Z Stashbot 7414 jelto@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on wikikube-worker1152.eqiad.wmnet with reason: hardware/networking issues 2460894 wikitext text/x-wiki a0pcgnauvvku8zil5ri29k2vbuq7rl6 2460895 2460894 2026-09-25T10:53:30Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reboot-single for host dse-k8s-ctrl1001.eqiad.wmnet 2460895 wikitext text/x-wiki 3zmdhmwndxfy0ix40kuo15hwuydh955 2460896 2460895 2026-09-25T10:54:44Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2460896 wikitext text/x-wiki 6zfx5o8wdlf96pm8sov8e6vegqdghwo 2460897 2460896 2026-09-25T10:54:54Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2460897 wikitext text/x-wiki da4632kdmzaph0oxiy9ldf54favhpwt 2460899 2460897 2026-09-25T10:58:30Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-ctrl1001.eqiad.wmnet 2460899 wikitext text/x-wiki ec0pbqs07gkgge7scevo10r7lklyjel 2460901 2460899 2026-09-25T11:17:33Z Stashbot 7414 kevinbazira@deploy1003: helmfile [ml-staging-codfw] 'sync' command on namespace 'tts-section-generator' for release 'main' . 2460901 wikitext text/x-wiki 1am590te84z8v65nsawn23xcbfekfkc 2460902 2460901 2026-09-25T11:19:15Z Stashbot 7414 kevinbazira@deploy1003: helmfile [ml-serve-eqiad] 'sync' command on namespace 'tts-section-generator' for release 'main' . 2460902 wikitext text/x-wiki 91jm1y3ffko87h7sqrqr6pqwkmops08 2460903 2460902 2026-09-25T11:20:52Z Stashbot 7414 kevinbazira@deploy1003: helmfile [ml-serve-codfw] 'sync' command on namespace 'tts-section-generator' for release 'main' . 2460903 wikitext text/x-wiki kqflodonpkmmmsijincxsx8mu49lwp6 2460907 2460903 2026-09-25T11:36:35Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reboot-single for host dse-k8s-ctrl1002.eqiad.wmnet 2460907 wikitext text/x-wiki 4c2mf6s1lfsc12evaymkld4xbv1urcc 2460908 2460907 2026-09-25T11:40:49Z Stashbot 7414 urbanecm@deploy1003: mwscript-k8s job started: foreachwikiindblist growthexperiments GrowthExperiments:revalidateLinkRecommendations.php --olderThan=1790175600 --verbose # T438366 2460908 wikitext text/x-wiki 9bhj4yd58germr0zjs521qb5pgmjxyw 2460909 2460908 2026-09-25T11:41:23Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-ctrl1002.eqiad.wmnet 2460909 wikitext text/x-wiki ghh5uu5cr39n04y8nvwyprbyannvvuw 0 17290 2460519 2460200 2026-09-24T15:30:48Z Stashbot 7414 dancy: Update gitlab-runner image to v19.2.6 (T439037) 2460519 wikitext text/x-wiki 0mh9jm8o4iwel2zhjvgljyk8bbfv2q6 2460523 2460519 2026-09-24T15:37:39Z Stashbot 7414 dancy: Upgrading gitlab-cloud-runners (staging) from 1.35.1-do.6 to 1.35.7-do.5 (T439035) 2460523 wikitext text/x-wiki ij2voev6l4to07ld5z7v8rmzhbfohbi 2460541 2460523 2026-09-24T16:01:48Z Stashbot 7414 dancy: Upgrading gitlab-cloud-runners (staging) from 1.35.7-do.5 to 1.36.3-do.5 (T439035) 2460541 wikitext text/x-wiki 0vxa16albmlvpx73ig0i7vp0tfmtytn 2460573 2460541 2026-09-24T17:02:47Z Stashbot 7414 dancy: Upgraded gitlab-cloud-runners (staging) from 1.35.7-do.5 to 1.36.3-do.5 (T439035) 2460573 wikitext text/x-wiki ljy3u4cy06pa4c3505jnmfqwee95ahd 498 30942 2460377 2460365 2026-09-24T12:45:14Z Stashbot 7414 filippo@cloudcumin1001: START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T435537) 2460377 wikitext text/x-wiki 1dvr001ntp9yo1r614veavn7vy5xrmc 2460378 2460377 2026-09-24T12:45:20Z Stashbot 7414 filippo@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) (T435537) 2460378 wikitext text/x-wiki guks3a44z4hhbaa3hpkjkeq00orc4gv 2460400 2460378 2026-09-24T13:19:57Z Stashbot 7414 filippo@cloudcumin1001: START - Cookbook wmcs.openstack.cloudvirt.set_maintenance (T437407) 2460400 wikitext text/x-wiki 70t6tdfbuyahotf8i89gk7mea1rajkv 2460401 2460400 2026-09-24T13:20:02Z Stashbot 7414 filippo@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.cloudvirt.set_maintenance (exit_code=0) (T437407) 2460401 wikitext text/x-wiki 36x8fzahv9x2lj1ouosimxz2nvlc3sk 2460507 2460401 2026-09-24T15:22:43Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch 2460507 wikitext text/x-wiki 9lv4cfdyvtvldgby1b4ic4ii8fi9f7i 2460508 2460507 2026-09-24T15:23:46Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch 2460508 wikitext text/x-wiki mnihgyb6b7am2nw9arjm8c5xa29p9gt 2460576 2460508 2026-09-24T17:08:21Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch 2460576 wikitext text/x-wiki 4dwx08vk3tfucwmql3vh8zskjgeh478 2460577 2460576 2026-09-24T17:09:27Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch 2460577 wikitext text/x-wiki qf62zqh9yvsdi1e87ctlxf2ruaxvlna 3 441296 2460623 2145009 2026-09-24T18:02:55Z Bumpf 48938 /* ZoomViewer 404 error! */ new section 2460623 wikitext text/x-wiki 7zehwn2pqr6xcqff4t20kr750csum2s 498 443946 2460741 2458559 2026-09-24T20:16:21Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed 473f09b916 (l10n updates: ne) [actually deployed 97394aca3a, temporary T431146 branch rebased on top of that] 2460741 wikitext text/x-wiki hchtuarweqd9n1f6yxb55uv89wdcpj4 498 443949 2460888 2458698 2026-09-25T09:47:09Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy c9189c5 (Upgrade Bootstrap 3 to 5 across templates) for T439032 2460888 wikitext text/x-wiki dsz8gtrxrh5x5aeondc7twq2gotirgf 2460889 2460888 2026-09-25T09:47:10Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 9ea4ac1 (Resolve grouping type before caching config) 2460889 wikitext text/x-wiki 1hlrkyhhyzakzroeqgsqiylh69w0cfy 2460904 2460889 2026-09-25T11:21:02Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 49b06bc (Add run-history read + summary methods to DashboardRegistry) for T438830 2460904 wikitext text/x-wiki 5k6tjowilvsl71eom47soh6xej4n6ry 2460905 2460904 2026-09-25T11:21:04Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 9e29a53 (Add per-dashboard run-history page) for T438830 2460905 wikitext text/x-wiki ed1j641s5ijq72qviqlo9idfkv18j5a 2460906 2460905 2026-09-25T11:21:07Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 62273b3 (Link dashboard rows to the run-history page) for T438830 2460906 wikitext text/x-wiki jz5g806gdckwkconih1ei94fs03rodv 0 446084 2460722 2428126 2026-09-24T19:34:44Z CDanis (WMF) 13326 Changed redirect target from [[mw:Wikimedia Production/Service Catalog/Ownership Roles and Responsibilities]] to [[mw:Wikimedia Production/Service Catalog]] 2460722 wikitext text/x-wiki m0yw39qj8yg09h9qylc0popm9gnv8f0 0 447761 2460877 2460310 2026-09-25T09:00:09Z ABran-WMF 37196 /* Failover (in case primary host is unavailable) */ 2460877 wikitext text/x-wiki 2g5ay3n5nlkzojjdem6uflxov0ln1ny 0 448297 2460900 1921742 2026-09-25T11:00:22Z MLechvien-WMF 49197 Updated to remove the Phabricator and Gerrit bits 2460900 wikitext text/x-wiki 0qc8st0rka0p12kjxs5i3ogangua5vb 0 449160 2460834 2460229 2026-09-25T00:02:47Z Dexbot 30554 Bot: Updating the report 2460834 wikitext text/x-wiki p67ltfd2tm96ey7qez3y0y1pygdndod 2 454501 2460525 2401639 2026-09-24T15:42:51Z BRouberol-WMF 39141 /* Kubernetes */ 2460525 wikitext text/x-wiki 57s087f44l92urtdywe94bf4gp4gppc 2460526 2460525 2026-09-24T15:43:11Z BRouberol-WMF 39141 /* Displaying the allocatable memory per Kubernetes worker node */ 2460526 wikitext text/x-wiki 6str9jbr0gtmea4du95ofgh9ma7bzn4 2460527 2460526 2026-09-24T15:45:34Z BRouberol-WMF 39141 /* Displaying the allocatable memory per Kubernetes worker node */ 2460527 wikitext text/x-wiki ezvkwkloit34q0cbvf8gz7sivdq3pd6 2460529 2460527 2026-09-24T15:47:36Z BRouberol-WMF 39141 /* Displaying the allocatable memory per Kubernetes worker node */ 2460529 wikitext text/x-wiki 9bi4kb90i5qsefyytdfppmvuqzyu7os 2460866 2460529 2026-09-25T07:56:57Z BRouberol-WMF 39141 /* Kubernetes */ 2460866 wikitext text/x-wiki jcxqqkgjsfc823ws4bkwwxmn6acev5a 498 454664 2460819 2193275 2026-09-24T22:28:09Z Stashbot 7414 wmbot~sportz@tools-bastion-15: gerrit-vue-watcher: Updating 6a90c89..02e4ca7 2460819 wikitext text/x-wiki 74fs0za4q6bfmva7z5ryao9fwlu4qr5 0 458648 2460395 2290227 2026-09-24T13:09:56Z BRouberol-WMF 39141 /* Performing a Point In Time Recovery (PITR) */ 2460395 wikitext text/x-wiki mt2jxj2vci4tkxqtzohg44ml956mx0a 0 459699 2460724 2460253 2026-09-24T19:39:38Z Stashbot 7414 ejegg: standalone (ipn listener) SmashPig upgraded from cbbb2d54 to 31c021ba 2460724 wikitext text/x-wiki 8pvmp941l1duw4ipq8ba8ny1p7rxde7 2460760 2460724 2026-09-24T20:46:18Z Stashbot 7414 ejegg: fundraising civicrm upgraded from 7545e2c4 to 51e3f920 2460760 wikitext text/x-wiki 793aqu3d7vrn7uu1ow0roijj9tdw80u 2460787 2460760 2026-09-24T21:34:08Z Stashbot 7414 ejegg: payments-wiki upgraded from 375adf37 to 49e656bc 2460787 wikitext text/x-wiki nnck919uf07a9gwm8sv79n72prl0fhm 0 460377 2460745 2459611 2026-09-24T20:19:20Z TBurmeister (WMF) 27417 Revise formatting and style 2460745 wikitext text/x-wiki 8qmp2dxiih8ixk6bhhgjh3ay76x8ciz 0 460585 2460433 2459107 2026-09-24T14:02:53Z TBurmeister (WMF) 27417 Remove column, update wording 2460433 wikitext text/x-wiki pv80mav7f03chvr3qlwrfextug8oc4l 2460435 2460433 2026-09-24T14:04:08Z TBurmeister (WMF) 27417 2460435 wikitext text/x-wiki rl9bd7oj2kw1icxbbptph7docr319xw 0 460667 2460666 2460249 2026-09-24T18:39:06Z TBurmeister (WMF) 27417 Remove template that is no longer active 2460666 wikitext text/x-wiki 9zuxgw4r6p5yeyus9pobl338j1htnpb 2460680 2460666 2026-09-24T18:45:28Z TBurmeister (WMF) 27417 Update intro sections 2460680 wikitext text/x-wiki 16ir5bsakmvsv6fo2dbis3q9tgm59mv 0 460718 2460402 2459849 2026-09-24T13:21:19Z GGoncalves-WMF 42848 /* User documentation */ Move Access up right at the start of User documentation. 2460402 wikitext text/x-wiki 7c8mo2jscbtx135he1jiznrb08lhxl2 2460873 2460402 2026-09-25T08:31:35Z JMonton-WMF 47887 remove code from Attribution API, it isn't ready yet. 2460873 wikitext text/x-wiki 6igh0pqyqnungnaetmw2vkiea0ttv9p