Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.7 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 2367 2429239 2211524 2026-06-22T19:05:19Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.nfs.migrate_service 2429239 wikitext text/x-wiki 69x1sp89h3ju7p4g4048aa36hx48jow 2429240 2429239 2026-06-22T19:05:27Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.nfs.migrate_service (exit_code=99) 2429240 wikitext text/x-wiki to2e4pwrl4fssfyxzt2ytnox8alstr6 2429241 2429240 2026-06-22T19:09:37Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.nfs.add_server 2429241 wikitext text/x-wiki ppu2stg6gsjjh6kvtda87ppoi7wss28 2429249 2429241 2026-06-22T19:23:40Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.nfs.add_server (exit_code=0) 2429249 wikitext text/x-wiki rfaft64zxpv0yzfe4eozcj6exvf7qur 2429263 2429249 2026-06-22T19:46:45Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.nfs.add_server 2429263 wikitext text/x-wiki przguvv74fvikpvi7zwex25dgkq1m6q 2429264 2429263 2026-06-22T19:47:24Z Stashbot 7414 andrew@cloudcumin1001: END (ERROR) - Cookbook wmcs.nfs.add_server (exit_code=97) 2429264 wikitext text/x-wiki stfxsnfhmcj279iidd1tewhmgcladjf 2429265 2429264 2026-06-22T19:47:28Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.nfs.add_server 2429265 wikitext text/x-wiki osnb6mj04sxwamqs78t6k1wx02r8so2 2429270 2429265 2026-06-22T19:59:25Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.nfs.add_server (exit_code=0) 2429270 wikitext text/x-wiki i7jejlxb9n0xg7dt4ymzouss1d2235a 498 3086 2429515 2429029 2026-06-23T08:27:13Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2429515 wikitext text/x-wiki 036yaaxo816ii36h2ujkevyglc22pki 2429530 2429515 2026-06-23T08:40:34Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2429530 wikitext text/x-wiki rw60gablzdrwis7i9pmotxqu8alwnz4 0 3387 2429277 2425840 2026-06-22T20:25:13Z WFan (WMF) 38087 /* Internal-facing */ 2429277 wikitext text/x-wiki 3a0b5gzvxcpx6mlwqiswxs8krglzj9y 2429281 2429277 2026-06-22T20:28:02Z WFan (WMF) 38087 /* Internal-facing */ 2429281 wikitext text/x-wiki 4kn8z99hcel8u1tvidpaov2bsqk419p 0 4108 2429065 2429043 2026-06-22T12:16:17Z Msz2001 16141 /* {{Deployment_day|date=2026-06-22}} */ 2429065 wikitext text/x-wiki ad9x8vg9odj43nfa6x9lwi8y2uie82e 2429076 2429065 2026-06-22T12:40:22Z ScheduleDeploymentBot 37566 Add [[gerrit:1304785]] to Tuesday, June 23 UTC morning backport window 2429076 wikitext text/x-wiki fj5xvnu7z134g8j1awjju0si68l4s25 2429077 2429076 2026-06-22T12:42:39Z ScheduleDeploymentBot 37566 Add [[gerrit:1300892]] to Monday, June 22 UTC afternoon backport window 2429077 wikitext text/x-wiki 7y54kjh79ioc6n67ffog2pm3t54zu7v 2429078 2429077 2026-06-22T12:51:26Z ScheduleDeploymentBot 37566 Add [[gerrit:1304222]] to Monday, June 22 UTC afternoon backport window 2429078 wikitext text/x-wiki lcz9rh5j0slu7qwtq788kl270nmzmpt 2429080 2429078 2026-06-22T13:00:58Z Msz2001 16141 /* {{Deployment_day|date=2026-06-22}} */ mine got deployed earlier 2429080 wikitext text/x-wiki acbpm2y94p38c3tuqy3lz36a2al22n7 2429233 2429080 2026-06-22T18:48:51Z ScheduleDeploymentBot 37566 Add [[gerrit:1304876]] to Tuesday, June 23 UTC late backport window 2429233 wikitext text/x-wiki n3vxw7vjptuhiilmprp9jgjg0jotl3j 2429266 2429233 2026-06-22T19:49:39Z ScheduleDeploymentBot 37566 Add [[gerrit:1304887]] to Monday, June 22 UTC late backport window 2429266 wikitext text/x-wiki 6he5iry51wizcnropoa1j0o10s15492 2429287 2429266 2026-06-22T20:43:26Z ScheduleDeploymentBot 37566 Add [[gerrit:1304896]] to Monday, June 22 UTC late backport window 2429287 wikitext text/x-wiki 80rga4649smi3bmdfjn7b7yx9jci62a 2429291 2429287 2026-06-22T21:04:56Z ScheduleDeploymentBot 37566 Add [[gerrit:1304904]] to Tuesday, June 23 UTC morning backport window 2429291 wikitext text/x-wiki 17ypqtb8uc4kb018tpe2fw13lk1jq6d 2429446 2429291 2026-06-23T05:27:12Z ScheduleDeploymentBot 37566 Add [[gerrit:1305015]] to Tuesday, June 23 UTC afternoon backport window 2429446 wikitext text/x-wiki 1g1ii6ct9dr93x9mc4ghs5jltt95dbk 2429448 2429446 2026-06-23T05:32:16Z ScheduleDeploymentBot 37566 Add [[gerrit:1305017]] to Tuesday, June 23 UTC afternoon backport window 2429448 wikitext text/x-wiki gnj2u6knewn9mxovxwlls05mxo79tib 2429449 2429448 2026-06-23T05:35:08Z ScheduleDeploymentBot 37566 Add [[gerrit:1305018]] to Tuesday, June 23 UTC afternoon backport window 2429449 wikitext text/x-wiki dr7u2c8857yzf3x2wmn70rbp061a1p5 2429451 2429449 2026-06-23T05:36:23Z ScheduleDeploymentBot 37566 Add [[gerrit:1305016]] to Tuesday, June 23 UTC afternoon backport window 2429451 wikitext text/x-wiki p9o44alhrl85o7n0nzircf6vdab6uvt 2429479 2429451 2026-06-23T06:35:08Z ScheduleDeploymentBot 37566 Add [[gerrit:1305025]] to Tuesday, June 23 UTC afternoon backport window 2429479 wikitext text/x-wiki g8dc43goz7qsqx6x670wi8v19wc16f0 2429480 2429479 2026-06-23T06:36:58Z ScheduleDeploymentBot 37566 Add [[gerrit:1305026]] to Tuesday, June 23 UTC afternoon backport window 2429480 wikitext text/x-wiki imhrtdivmm5ggosdcmk0tsyt1ndwja4 2429536 2429480 2026-06-23T09:05:21Z ScheduleDeploymentBot 37566 Add [[gerrit:1305049]] to Tuesday, June 23 UTC afternoon backport window 2429536 wikitext text/x-wiki 41f44jqx8phoi1pqriuzhv1ycgfd3oj 2429558 2429536 2026-06-23T09:15:57Z ScheduleDeploymentBot 37566 Add [[gerrit:1305051]] to Tuesday, June 23 UTC afternoon backport window 2429558 wikitext text/x-wiki pbemnfq1gm9btes1grgnqqzngrbacxv 2429568 2429558 2026-06-23T10:32:22Z ScheduleDeploymentBot 37566 Add [[gerrit:1305062]] to Tuesday, June 23 UTC afternoon backport window 2429568 wikitext text/x-wiki 7qklxaxz63iqhjuolm0zvivbjrarduz 498 6726 2429513 2429021 2026-06-23T08:14:11Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2429513 wikitext text/x-wiki ibm56pi75l04opryqifqj0uplwpq218 2429514 2429513 2026-06-23T08:26:14Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2429514 wikitext text/x-wiki 0mqrkbz6wm63ny6kgrdwdgvt2ny4u1k 0 7919 2429059 2429058 2026-06-22T12:06:14Z Stashbot 7414 kamila@deploy1003: Started scap sync-world: upgrade canaries to Debian Bookworm 2429059 wikitext text/x-wiki g404715cf58z6rupcmpmgvn93qr4vjw 2429060 2429059 2026-06-22T12:12:48Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/ratelimit: apply 2429060 wikitext text/x-wiki 136i1vhkw1387xh3ojdmr2l9lfnw3ge 2429061 2429060 2026-06-22T12:12:57Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/ratelimit: apply 2429061 wikitext text/x-wiki keharar5z4ehciel9y7iamwuw0bem8e 2429062 2429061 2026-06-22T12:13:04Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/ratelimit: apply 2429062 wikitext text/x-wiki 9ft54odarw5spx5wvdux04mk68s4bt9 2429063 2429062 2026-06-22T12:14:27Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/ratelimit: apply 2429063 wikitext text/x-wiki 5dh2hoafctfv7esc1jbpm7ys3reyg6n 2429064 2429063 2026-06-22T12:16:00Z Stashbot 7414 kamila@deploy1003: Finished scap sync-world: upgrade canaries to Debian Bookworm (duration: 10m 33s) 2429064 wikitext text/x-wiki m1ie8hlxk41ouzk84ivthtntvibbk2w 2429066 2429064 2026-06-22T12:18:43Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/ratelimit: apply 2429066 wikitext text/x-wiki q2iwdldo0b4lo2e7dh9yh4l1j53pr71 2429067 2429066 2026-06-22T12:19:05Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/ratelimit: apply 2429067 wikitext text/x-wiki opjto8107rc9he1290odb8xr8gki50f 2429068 2429067 2026-06-22T12:22:12Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1304761|hCaptcha: Skip blocked-IP risk-score collection for crawlers (T429755)]] 2429068 wikitext text/x-wiki enng5v88x87bsqsbko1903yh7lsli46 2429069 2429068 2026-06-22T12:28:06Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1304761|hCaptcha: Skip blocked-IP risk-score collection for crawlers (T429755)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429069 wikitext text/x-wiki me3b842nimnasetfqsjl1b4ngnhsyrb 2429070 2429069 2026-06-22T12:32:04Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2429070 wikitext text/x-wiki 8gp8wzepq7hox3i5bsy56rn9xvqoitl 2429071 2429070 2026-06-22T12:33:27Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist mwscript.dblist extensions/Translate/scripts/ttmserver-export.php --ttmserver eqiad-k8s # T425377 populating translation memory (dblist: https://phabricator.wikimedia.org/P94318) 2429071 wikitext text/x-wiki 8q04tqv0uwntk9d5lqezh7q272djuzp 2429072 2429071 2026-06-22T12:33:31Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist mwscript.dblist extensions/Translate/scripts/ttmserver-export.php --ttmserver codfw-k8s # T425377 populating translation memory (dblist: https://phabricator.wikimedia.org/P94319) 2429072 wikitext text/x-wiki 5avm3n2lfklf4smlqs6urwzf0twq7lf 2429073 2429072 2026-06-22T12:38:32Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304761|hCaptcha: Skip blocked-IP risk-score collection for crawlers (T429755)]] (duration: 16m 20s) 2429073 wikitext text/x-wiki mpx6essy7wesnt13gg9dn955w2ltacs 2429074 2429073 2026-06-22T12:38:56Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es1040: Migration of es1040.eqiad.wmnet completed 2429074 wikitext text/x-wiki krxjgzbrbsdnoe51etse0h0v5yegm49 2429075 2429074 2026-06-22T12:38:58Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2429075 wikitext text/x-wiki 40nc24or8ltyufof6szdqpxhb9eqvsd 2429079 2429075 2026-06-22T12:55:44Z Stashbot 7414 Msz2001: Deployed changes to private code (SuggestedInvestigations and PrivateSettings.php) 2429079 wikitext text/x-wiki 753lps6tzw0u4w9qah8ifsr75msrbqh 2429081 2429079 2026-06-22T13:02:39Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.remove-downtime for db2189.codfw.wmnet 2429081 wikitext text/x-wiki bqh02rbfzmno617tnga4ui4vcj7y100 2429082 2429081 2026-06-22T13:02:41Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for db2189.codfw.wmnet 2429082 wikitext text/x-wiki cfnh8caod91s77xex9hv1mdqhkspx0j 2429083 2429082 2026-06-22T13:03:07Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.remove-downtime for db2212.codfw.wmnet 2429083 wikitext text/x-wiki k9iw17b1bo5ixxvqy16r1y24aza8cuq 2429084 2429083 2026-06-22T13:03:09Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for db2212.codfw.wmnet 2429084 wikitext text/x-wiki c746xh8g3tg9e8u9f88pxwqdik63v0d 2429085 2429084 2026-06-22T13:03:46Z Stashbot 7414 oblivian@cumin2003: START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "Fix utf-8 name handling - oblivian@cumin2003" 2429085 wikitext text/x-wiki fce7rwvmawgg6rix2molguwo0qjt4pz 2429086 2429085 2026-06-22T13:03:49Z Stashbot 7414 oblivian@cumin2003: START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: Fix utf-8 name handling - oblivian@cumin2003 2429086 wikitext text/x-wiki 43pnctf54qmzdp7p38f9kljdnnnim97 2429087 2429086 2026-06-22T13:04:37Z Stashbot 7414 oblivian@cumin2003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: Fix utf-8 name handling - oblivian@cumin2003 2429087 wikitext text/x-wiki gusxnshuwktg7ykntpurha2wuy458ve 2429088 2429087 2026-06-22T13:04:39Z Stashbot 7414 oblivian@cumin2003: END (PASS) - Cookbook sre.deploy.hiddenparma (exit_code=0) Hiddenparma deployment to the alerting hosts with reason: "Fix utf-8 name handling - oblivian@cumin2003" 2429088 wikitext text/x-wiki rwkaj69gj9mc0u1t0x2fhm7vxi2qpbl 2429089 2429088 2026-06-22T13:11:51Z Stashbot 7414 tchin@deploy1003: Started scap sync-world: Backport for [[gerrit:1300892|[PageViewInfo] Add new config (T411771)]] 2429089 wikitext text/x-wiki 0a6918faepllgvhb3c1ra11vobgr2l8 2429090 2429089 2026-06-22T13:17:52Z Stashbot 7414 tchin@deploy1003: tchin: Backport for [[gerrit:1300892|[PageViewInfo] Add new config (T411771)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429090 wikitext text/x-wiki i7equli5mqid1k577oa9m015d17l8vm 2429091 2429090 2026-06-22T13:19:41Z Stashbot 7414 tchin@deploy1003: tchin: Continuing with deployment 2429091 wikitext text/x-wiki kzj37erx7wbfwpxgyc6uqbzqdcpgyzy 2429092 2429091 2026-06-22T13:24:52Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2429092 wikitext text/x-wiki 7i7gwl3bdmm16p68m4gg4f3e4b5jgi6 2429093 2429092 2026-06-22T13:24:57Z Stashbot 7414 marostegui@cumin1003: dbmaint on es7@eqiad T429463 2429093 wikitext text/x-wiki a5oxiks3re4yorot315hzwf53ifwayg 2429094 2429093 2026-06-22T13:25:02Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es1039: Upgrading es1039.eqiad.wmnet 2429094 wikitext text/x-wiki ivulgie94paz3i2rn242de85fl7ash1 2429095 2429094 2026-06-22T13:25:23Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool es1039: Upgrading es1039.eqiad.wmnet 2429095 wikitext text/x-wiki l8ytliyrvifh6v5za65mtxv9ux44ih7 2429096 2429095 2026-06-22T13:26:14Z Stashbot 7414 tchin@deploy1003: Finished scap sync-world: Backport for [[gerrit:1300892|[PageViewInfo] Add new config (T411771)]] (duration: 14m 24s) 2429096 wikitext text/x-wiki masuzespwdfo2esev1q4wx0oshyaoxn 2429097 2429096 2026-06-22T13:27:40Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es1039.eqiad.wmnet with OS trixie 2429097 wikitext text/x-wiki 0lp7jscx4owveiufhgtgoj5u1mgkk5h 2429098 2429097 2026-06-22T13:27:53Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2429098 wikitext text/x-wiki f1r2qtmdema5pxr612dh8a8l64t3r4k 2429099 2429098 2026-06-22T13:29:21Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2429099 wikitext text/x-wiki 4oilf6sclr7zi5bt1ciq0oj5hlpwrfn 2429100 2429099 2026-06-22T13:32:43Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2429100 wikitext text/x-wiki 6orgxfnjcp0wn8lff7szn0220f8gs9f 2429101 2429100 2026-06-22T13:33:13Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2429101 wikitext text/x-wiki ra3ignx82tghescmyeen471zth1jqcn 2429102 2429101 2026-06-22T13:33:31Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2429102 wikitext text/x-wiki sd3boa90tduib1v5z4mrr3oegu96mh6 2429103 2429102 2026-06-22T13:34:09Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2429103 wikitext text/x-wiki 4bz8mxikvg1d69zprkbrn3tus3v0hmx 2429104 2429103 2026-06-22T13:34:17Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2429104 wikitext text/x-wiki ritq1t8hdwplz60z9d08zk8avdssi0r 2429105 2429104 2026-06-22T13:34:36Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2429105 wikitext text/x-wiki foorlsvxtkclj2udqevdfczf5ivhucq 2429106 2429105 2026-06-22T13:35:00Z Stashbot 7414 tgr@deploy1003: Started scap sync-world: Backport for [[gerrit:1304125|Add email confirmation banner Test Kitchen instrumentation (long-term) (T428293)]] 2429106 wikitext text/x-wiki 3rh4x98dmmm2igczxihnf257vv6f8wl 2429108 2429106 2026-06-22T13:39:37Z Stashbot 7414 tgr@deploy1003: tgr, mmartorana: Backport for [[gerrit:1304125|Add email confirmation banner Test Kitchen instrumentation (long-term) (T428293)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429108 wikitext text/x-wiki ay94wg80zf5hfi5xa6e85qzvuc38qi6 2429109 2429108 2026-06-22T13:40:35Z Stashbot 7414 tgr@deploy1003: tgr, mmartorana: Continuing with deployment 2429109 wikitext text/x-wiki 2dvmtan4nq893sz2gxj0ezrgyksw70j 2429111 2429109 2026-06-22T13:43:42Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es1039.eqiad.wmnet with reason: host reimage 2429111 wikitext text/x-wiki 7ancelkkgjbhveblnu2q5c6lwph56i0 2429113 2429111 2026-06-22T13:47:00Z Stashbot 7414 tgr@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304125|Add email confirmation banner Test Kitchen instrumentation (long-term) (T428293)]] (duration: 11m 59s) 2429113 wikitext text/x-wiki icxej4n2yvpc8b0fxjwlvjp2kbe1h25 2429114 2429113 2026-06-22T13:49:39Z Stashbot 7414 tgr@deploy1003: Started scap sync-world: Backport for [[gerrit:1304122|config: Enable EmailConfirmationBanner on all wikis (T428292)]] 2429114 wikitext text/x-wiki py2stlez55i4h4tuezwbh3bgt4ruswv 2429115 2429114 2026-06-22T13:49:46Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es1039.eqiad.wmnet with reason: host reimage 2429115 wikitext text/x-wiki m1dg7knm62ht2jcyyn0v2rgumbvfa3t 2429116 2429115 2026-06-22T13:50:39Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . 2429116 wikitext text/x-wiki q3g7gzwr4oa36nnlimk0766depn4rar 2429117 2429116 2026-06-22T13:55:40Z Stashbot 7414 tgr@deploy1003: mmartorana, tgr: Backport for [[gerrit:1304122|config: Enable EmailConfirmationBanner on all wikis (T428292)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429117 wikitext text/x-wiki jahq8kwt32ra83w429ds2imbciy2bbz 2429118 2429117 2026-06-22T13:56:24Z Stashbot 7414 tgr@deploy1003: mmartorana, tgr: Continuing with deployment 2429118 wikitext text/x-wiki rb178uvix7pqz35a2mprwaunyv93mpm 2429119 2429118 2026-06-22T14:02:47Z Stashbot 7414 tgr@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304122|config: Enable EmailConfirmationBanner on all wikis (T428292)]] (duration: 13m 08s) 2429119 wikitext text/x-wiki 75qghwtlzdr59mv3zqlpskibqlfwhpu 2429120 2429119 2026-06-22T14:03:06Z Stashbot 7414 tgr_: UTC afternoon deploys done(ish; will do the remaining two patches in about an hour) 2429120 wikitext text/x-wiki cxyf4ufjmmcqua5ifasq3q9ycv5l604 2429121 2429120 2026-06-22T14:04:02Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-worker1375.eqiad.wmnet with OS trixie 2429121 wikitext text/x-wiki b28kqmyvj5wi4gz9sgu2e21915p9bqa 2429122 2429121 2026-06-22T14:06:40Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-worker1376.eqiad.wmnet with OS trixie 2429122 wikitext text/x-wiki qtylfpxf5cgxrnhptkgdpdo149gpe0s 2429123 2429122 2026-06-22T14:06:57Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-worker1377.eqiad.wmnet with OS trixie 2429123 wikitext text/x-wiki 3k1o9e3wns11hsd10qegc0nyxi0iwo2 2429124 2429123 2026-06-22T14:07:00Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es1039.eqiad.wmnet with OS trixie 2429124 wikitext text/x-wiki gdg3dx24scxhz8yroplpkfcbijwezgk 2429125 2429124 2026-06-22T14:07:11Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-worker1378.eqiad.wmnet with OS trixie 2429125 wikitext text/x-wiki ok5r58yljf1efaewmfousp2n97m5wh7 2429126 2429125 2026-06-22T14:07:18Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2429126 wikitext text/x-wiki h6abpx9ogjw7vqa6a2tyufjl5m3p8yo 2429127 2429126 2026-06-22T14:07:23Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-worker1379.eqiad.wmnet with OS trixie 2429127 wikitext text/x-wiki ohio9rv8te9tdznm629yef5zv9q22gc 2429128 2429127 2026-06-22T14:07:30Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2429128 wikitext text/x-wiki n5r64xe5as2d08lt7ar5bwjb6dt6gki 2429129 2429128 2026-06-22T14:07:38Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2429129 wikitext text/x-wiki n876xtosa24dqk23ty72rjyasr8gdec 2429130 2429129 2026-06-22T14:07:48Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2429130 wikitext text/x-wiki a7gfft33y8d9qdffpke2grz0jiq9k8u 2429131 2429130 2026-06-22T14:07:58Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2429131 wikitext text/x-wiki e5a11bhlxar1fqd07alddep0je8bdkv 2429132 2429131 2026-06-22T14:08:01Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2429132 wikitext text/x-wiki i5vq9ty97kh120xneukvwf65fef6618 2429133 2429132 2026-06-22T14:16:47Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1375.eqiad.wmnet with reason: host reimage 2429133 wikitext text/x-wiki hdh0xtkca4j2u5xup6ibfoqutw4bjls 2429134 2429133 2026-06-22T14:19:29Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es1039: Migration of es1039.eqiad.wmnet completed 2429134 wikitext text/x-wiki f1xwdzc7q4568aotjpaa7ydvg44mnfp 2429135 2429134 2026-06-22T14:19:34Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1376.eqiad.wmnet with reason: host reimage 2429135 wikitext text/x-wiki emhxwto6b9tg3ghfow9izfonopljf2e 2429137 2429135 2026-06-22T14:20:14Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1378.eqiad.wmnet with reason: host reimage 2429137 wikitext text/x-wiki ejg037zbjccrtd7rfagsolzimrdkvpa 2429139 2429137 2026-06-22T14:20:21Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1377.eqiad.wmnet with reason: host reimage 2429139 wikitext text/x-wiki t543kgttnjhptl7epyhfgayrouhoy2e 2429140 2429139 2026-06-22T14:20:24Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1379.eqiad.wmnet with reason: host reimage 2429140 wikitext text/x-wiki jnfbc0q8fl97qwp66urgctiow853i8k 2429141 2429140 2026-06-22T14:20:40Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1375.eqiad.wmnet with reason: host reimage 2429141 wikitext text/x-wiki 45myqbjcpjx8pge0vyrpxvv3kt7w2wy 2429143 2429141 2026-06-22T14:23:41Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1379.eqiad.wmnet with reason: host reimage 2429143 wikitext text/x-wiki esdiqlk3h3d4jwn3gin8obd6uycs2ti 2429144 2429143 2026-06-22T14:27:44Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1378.eqiad.wmnet with reason: host reimage 2429144 wikitext text/x-wiki gb4b3jwf2qfu85gyap6q94wwzfewj4q 2429145 2429144 2026-06-22T14:32:15Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1377.eqiad.wmnet with reason: host reimage 2429145 wikitext text/x-wiki b3xp9wodyutji5tt2kee11017ydwpbd 2429146 2429145 2026-06-22T14:36:21Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1376.eqiad.wmnet with reason: host reimage 2429146 wikitext text/x-wiki iwce62yvpvc65g1oc2hjee4fosx7xmg 2429147 2429146 2026-06-22T14:39:05Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1375.eqiad.wmnet with OS trixie 2429147 wikitext text/x-wiki eqq2ga2sbonbyd1bfudmibbyneevear 2429148 2429147 2026-06-22T14:41:07Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1379.eqiad.wmnet with OS trixie 2429148 wikitext text/x-wiki ok6fr0jqzznzhm60fw807d7om8w501c 2429149 2429148 2026-06-22T14:45:14Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1378.eqiad.wmnet with OS trixie 2429149 wikitext text/x-wiki prgc00rao5uic6uavcbyyi2wcxxcfzw 2429150 2429149 2026-06-22T14:49:36Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1377.eqiad.wmnet with OS trixie 2429150 wikitext text/x-wiki os2l50dr86jri0nolb2f75zbvlywy23 2429151 2429150 2026-06-22T14:52:26Z Stashbot 7414 elukey: deny ANONYMOUS write traffic on Kafka main for statsv (only varnishkafka will push events from now on) - T425528 2429151 wikitext text/x-wiki styn3s7v7yw9889bsdhfejwwxkhyfx0 2429154 2429151 2026-06-22T14:54:00Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1376.eqiad.wmnet with OS trixie 2429154 wikitext text/x-wiki fedu3j9486129bv9s2oqwouv1jeaju2 2429155 2429154 2026-06-22T14:55:26Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-worker1380.eqiad.wmnet with OS trixie 2429155 wikitext text/x-wiki gp8td68yocmdctyy8cejlixndxayk28 2429156 2429155 2026-06-22T14:55:33Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-worker1381.eqiad.wmnet with OS trixie 2429156 wikitext text/x-wiki qouw3y8yme14sltlpq0c7l6my8b7041 2429157 2429156 2026-06-22T14:55:42Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-worker1382.eqiad.wmnet with OS trixie 2429157 wikitext text/x-wiki rvbihooh1jjrzi1qesx56twr9nwk12y 2429158 2429157 2026-06-22T14:55:50Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-worker1383.eqiad.wmnet with OS trixie 2429158 wikitext text/x-wiki nqmd5fsi2lz2wclb01vk3wzyyod0kdx 2429159 2429158 2026-06-22T14:55:59Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-worker1384.eqiad.wmnet with OS trixie 2429159 wikitext text/x-wiki m1bkubttfoohki458hogwf89tydnf9s 2429162 2429159 2026-06-22T15:02:24Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2429162 wikitext text/x-wiki r52mohgng8edmbpbeyvijkq5pudkf0b 2429163 2429162 2026-06-22T15:02:30Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2429163 wikitext text/x-wiki tv8vpxslvb6agt5oeb946oe7uem4hcz 2429165 2429163 2026-06-22T15:04:59Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es1039: Migration of es1039.eqiad.wmnet completed 2429165 wikitext text/x-wiki 3gf4nqha9u3wrark3g35kt1223o09lr 2429166 2429165 2026-06-22T15:05:02Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2429166 wikitext text/x-wiki gb1djz03i8d2mrrvq3xvzamha6o5uq5 2429167 2429166 2026-06-22T15:08:25Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1382.eqiad.wmnet with reason: host reimage 2429167 wikitext text/x-wiki 7smfdd93twpvpuyiqr3aiifnzms6tmt 2429168 2429167 2026-06-22T15:08:48Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1381.eqiad.wmnet with reason: host reimage 2429168 wikitext text/x-wiki 4xaxyszuieiq8e381dvwfh542qa3wmf 2429169 2429168 2026-06-22T15:08:51Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1380.eqiad.wmnet with reason: host reimage 2429169 wikitext text/x-wiki 7fwnuqg84x45nda2zfuk0ipoquyiyjo 2429170 2429169 2026-06-22T15:09:01Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1384.eqiad.wmnet with reason: host reimage 2429170 wikitext text/x-wiki jlxnychf9571cc3h7k7eovxp4voclj8 2429171 2429170 2026-06-22T15:09:06Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1383.eqiad.wmnet with reason: host reimage 2429171 wikitext text/x-wiki jg4zudv65xtgt3dfrw2djpne0k7zxr6 2429172 2429171 2026-06-22T15:09:28Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2429172 wikitext text/x-wiki a7vkjrrm1bcmvy9ttvltpci4i10k0yk 2429173 2429172 2026-06-22T15:09:34Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2429173 wikitext text/x-wiki dkaw317yn78hy17iz8lvhf98rz3f24c 2429174 2429173 2026-06-22T15:11:13Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2429174 wikitext text/x-wiki ia7cc49myv4rdr7keas05xpidlslyz7 2429175 2429174 2026-06-22T15:12:13Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2429175 wikitext text/x-wiki lrsvcw8ardp2n0pgaojdekphvs71z4i 2429176 2429175 2026-06-22T15:13:29Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1382.eqiad.wmnet with reason: host reimage 2429176 wikitext text/x-wiki n378ow08kfinbxeb5teivta7lfgg8rp 2429177 2429176 2026-06-22T15:17:35Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1381.eqiad.wmnet with reason: host reimage 2429177 wikitext text/x-wiki s4nrq3kbjs9a2z2a3dlvt4hwf777le8 2429178 2429177 2026-06-22T15:22:27Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1380.eqiad.wmnet with reason: host reimage 2429178 wikitext text/x-wiki f5n2ch9wxv6pzt6bqy6blzk5wxvan32 2429179 2429178 2026-06-22T15:27:07Z Stashbot 7414 tgr@deploy1003: Started scap sync-world: Backport for [[gerrit:1304222|Fix displaying events with IP agents (T428198)]], [[gerrit:1304690|Preserve redoLocalAuthentication flag when returning from auth domain (T429495)]] 2429179 wikitext text/x-wiki 3bshx7e7cqno27pn9a4jephoh42ozh9 2429180 2429179 2026-06-22T15:27:19Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1384.eqiad.wmnet with reason: host reimage 2429180 wikitext text/x-wiki p4gwrnroj9o1fjrano67w7p37d3s5dn 2429181 2429180 2026-06-22T15:31:12Z Stashbot 7414 tgr@deploy1003: matmarex, tgr: Backport for [[gerrit:1304222|Fix displaying events with IP agents (T428198)]], [[gerrit:1304690|Preserve redoLocalAuthentication flag when returning from auth domain (T429495)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429181 wikitext text/x-wiki 9ycuha1qm9loj3637dwevz147ck3iro 2429182 2429181 2026-06-22T15:31:23Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1382.eqiad.wmnet with OS trixie 2429182 wikitext text/x-wiki d8w8uxjgk6v4dmj2m4bhkkhkapqnv3i 2429183 2429182 2026-06-22T15:31:54Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1383.eqiad.wmnet with reason: host reimage 2429183 wikitext text/x-wiki hergmchq6s098fsqeeq6y8w8e7cugua 2429184 2429183 2026-06-22T15:34:54Z Stashbot 7414 tgr@deploy1003: matmarex, tgr: Continuing with deployment 2429184 wikitext text/x-wiki 7w08b594uq7291za76elteoc77bn5di 2429185 2429184 2026-06-22T15:35:28Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1381.eqiad.wmnet with OS trixie 2429185 wikitext text/x-wiki etezok6t2l0ojpeg2382ci48p4mmnsl 2429186 2429185 2026-06-22T15:39:59Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1380.eqiad.wmnet with OS trixie 2429186 wikitext text/x-wiki o8fbbn9v1myp1sdba9ev3aflfbru17d 2429187 2429186 2026-06-22T15:41:20Z Stashbot 7414 tgr@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304222|Fix displaying events with IP agents (T428198)]], [[gerrit:1304690|Preserve redoLocalAuthentication flag when returning from auth domain (T429495)]] (duration: 14m 15s) 2429187 wikitext text/x-wiki k6tu2w9dvidbpoitgh9xsp7hs1kgivf 2429188 2429187 2026-06-22T15:41:43Z Stashbot 7414 tgr_: UTC afternoon deploys double done 2429188 wikitext text/x-wiki inm5vufdi98vufh2pyt16rzv2oz3612 2429192 2429188 2026-06-22T15:45:03Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1384.eqiad.wmnet with OS trixie 2429192 wikitext text/x-wiki n43jb9b2h3b0gbke0jqm22u5p56j3zi 2429193 2429192 2026-06-22T15:49:23Z Stashbot 7414 blake@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1383.eqiad.wmnet with OS trixie 2429193 wikitext text/x-wiki bd72zz9r8qxwv705ozy42tfsv6k1ziw 2429198 2429193 2026-06-22T16:23:57Z Stashbot 7414 arlolra@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2429198 wikitext text/x-wiki g0kbhjxc07oyxh0ii38uikontotq7uo 2429199 2429198 2026-06-22T16:30:40Z Stashbot 7414 arlolra@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2429199 wikitext text/x-wiki 8pdlysmm9r4ze9iwndz804ifombr92k 2429200 2429199 2026-06-22T16:30:42Z Stashbot 7414 arlolra@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2429200 wikitext text/x-wiki on0faevfj3ektef85lglx8v167neu1k 2429202 2429200 2026-06-22T16:32:00Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist mwscript.dblist extensions/Translate/scripts/ttmserver-export.php --ttmserver eqiad-k8s # T425377 populating translation memory (dblist: https://phabricator.wikimedia.org/P94329) 2429202 wikitext text/x-wiki r47znjbegqcvdg7y87uroy13qee13r5 2429205 2429202 2026-06-22T16:37:25Z Stashbot 7414 arlolra@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2429205 wikitext text/x-wiki tnm78gp6r0151h8qk9jxmpb1d90wqd4 2429206 2429205 2026-06-22T16:38:47Z Stashbot 7414 kamila@deploy1003: Started scap sync-world: upgrade all of prod to debian bookworm 2429206 wikitext text/x-wiki oojizk6fawqajsl9i7n70wr6mqef3rg 2429208 2429206 2026-06-22T16:43:18Z Stashbot 7414 arlolra@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2429208 wikitext text/x-wiki o1nhgx1au3alki0i8b13quapekidobh 2429209 2429208 2026-06-22T16:43:46Z Stashbot 7414 arlolra@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2429209 wikitext text/x-wiki pmylqel3r281qqy5fwtz9a81pubvr6c 2429210 2429209 2026-06-22T16:43:49Z Stashbot 7414 arlolra@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2429210 wikitext text/x-wiki e9ucb8s4p0t9erz1v9cayosbqhu00k6 2429211 2429210 2026-06-22T16:44:16Z Stashbot 7414 arlolra@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2429211 wikitext text/x-wiki 2el3djbabniai89l2i7lgxvys0105qc 2429213 2429211 2026-06-22T16:46:02Z Stashbot 7414 arlolra@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2429213 wikitext text/x-wiki twadoir995vse3a8lb2juim6gje5oef 2429214 2429213 2026-06-22T16:46:05Z Stashbot 7414 arlolra@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2429214 wikitext text/x-wiki 7gdvl06yrsfl7g0hhzxxv43e12gkh52 2429215 2429214 2026-06-22T16:46:07Z Stashbot 7414 arlolra@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2429215 wikitext text/x-wiki dxt58zp5shivwelzzz47pk8eevm5uhl 2429216 2429215 2026-06-22T16:46:09Z Stashbot 7414 arlolra@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2429216 wikitext text/x-wiki 5pfhow5809rtmjomzka51d4yz5zk1m6 2429217 2429216 2026-06-22T16:46:52Z Stashbot 7414 arlolra@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2429217 wikitext text/x-wiki kgl0wef1pubf4ucvb4j59ll78wv0ra7 2429218 2429217 2026-06-22T16:46:54Z Stashbot 7414 arlolra@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2429218 wikitext text/x-wiki 2i9h0q29wo6ey9xqzrd5n0j5hxcc634 2429219 2429218 2026-06-22T16:46:56Z Stashbot 7414 arlolra@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2429219 wikitext text/x-wiki 5osrukxhydjob3isieb0x8fxsfrq5hc 2429220 2429219 2026-06-22T16:46:59Z Stashbot 7414 arlolra@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2429220 wikitext text/x-wiki ogwiejpvlnkivk6ut58q2bfvwrc9onv 2429221 2429220 2026-06-22T16:47:10Z Stashbot 7414 kamila@deploy1003: Finished scap sync-world: upgrade all of prod to debian bookworm (duration: 09m 17s) 2429221 wikitext text/x-wiki pn54k68qje31kitb2voxydr9q5g1a6t 2429243 2429221 2026-06-22T19:12:07Z Stashbot 7414 reedy@deploy1003: Started scap sync-world: Backport for [[gerrit:1304867|Upgrading guzzlehttp/psr7 (2.11.0 => 2.11.1)]], [[gerrit:1304868|Upgrade guzzle/* (T429826)]], [[gerrit:1304879|Updated guzzlehttp/guzzle from 7.10.0 to 7.12.1 (T429826)]] 2429243 wikitext text/x-wiki na5mz358j613n8dk3dlzkfvmggk9ybz 2429244 2429243 2026-06-22T19:14:02Z Stashbot 7414 reedy@deploy1003: reedy: Backport for [[gerrit:1304867|Upgrading guzzlehttp/psr7 (2.11.0 => 2.11.1)]], [[gerrit:1304868|Upgrade guzzle/* (T429826)]], [[gerrit:1304879|Updated guzzlehttp/guzzle from 7.10.0 to 7.12.1 (T429826)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429244 wikitext text/x-wiki hkfyc2mo2kq08iex49fcgtckxdvsz8o 2429245 2429244 2026-06-22T19:14:26Z Stashbot 7414 reedy@deploy1003: reedy: Continuing with deployment 2429245 wikitext text/x-wiki dye6aqkh9ap33wv3pi4oabvyrnrq5xk 2429248 2429245 2026-06-22T19:21:13Z Stashbot 7414 reedy@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304867|Upgrading guzzlehttp/psr7 (2.11.0 => 2.11.1)]], [[gerrit:1304868|Upgrade guzzle/* (T429826)]], [[gerrit:1304879|Updated guzzlehttp/guzzle from 7.10.0 to 7.12.1 (T429826)]] (duration: 09m 06s) 2429248 wikitext text/x-wiki gmq8s8ffq0vb6l9hbulnir380okdqjk 2429251 2429248 2026-06-22T19:27:22Z Stashbot 7414 reedy@deploy1003: Started scap sync-world: Backport for [[gerrit:1304187|Add info-level logging to wmgMonologChannels for timeline (T429654)]] 2429251 wikitext text/x-wiki 0pejk53zy89o8ruknrn867lsgby2el6 2429253 2429251 2026-06-22T19:29:23Z Stashbot 7414 reedy@deploy1003: sbassett, reedy: Backport for [[gerrit:1304187|Add info-level logging to wmgMonologChannels for timeline (T429654)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429253 wikitext text/x-wiki 7p0sg0zz4yoq86mu15uyt9v669ehuly 2429254 2429253 2026-06-22T19:29:46Z Stashbot 7414 reedy@deploy1003: sbassett, reedy: Continuing with deployment 2429254 wikitext text/x-wiki 6pdrzwvxj51run5m31pjisn9i9m7boe 2429257 2429254 2026-06-22T19:34:03Z Stashbot 7414 reedy@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304187|Add info-level logging to wmgMonologChannels for timeline (T429654)]] (duration: 06m 41s) 2429257 wikitext text/x-wiki 7hop679a3zb6zo8i5zplewro933zucg 2429272 2429257 2026-06-22T20:09:44Z Stashbot 7414 catrope@deploy1003: Started scap sync-world: Backport for [[gerrit:1304156|Permissions: Create wmf-officeit group on collabwiki]] 2429272 wikitext text/x-wiki 7htoly4qeyttoa92pwbwaho2j1oq2of 2429274 2429272 2026-06-22T20:11:43Z Stashbot 7414 catrope@deploy1003: catrope: Backport for [[gerrit:1304156|Permissions: Create wmf-officeit group on collabwiki]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429274 wikitext text/x-wiki cnna4bg1czca8byf6vyx23v9vwqhmqg 2429275 2429274 2026-06-22T20:15:13Z Stashbot 7414 catrope@deploy1003: catrope: Continuing with deployment 2429275 wikitext text/x-wiki eackpxizf8s3o5tokcvq77jb6ntgi70 2429276 2429275 2026-06-22T20:19:29Z Stashbot 7414 catrope@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304156|Permissions: Create wmf-officeit group on collabwiki]] (duration: 09m 45s) 2429276 wikitext text/x-wiki 06o6vl8sdqqrorpxrk84wb7v9s14lf6 2429279 2429276 2026-06-22T20:27:25Z Stashbot 7414 bpirkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1304175|REST: adjust analytics and wikifunctions REST Sandbox visibility (T422770 T423058 T422771)]] 2429279 wikitext text/x-wiki praryk7q3p75orcdtpf2z625tq90i1h 2429282 2429279 2026-06-22T20:29:24Z Stashbot 7414 bpirkle@deploy1003: bpirkle: Backport for [[gerrit:1304175|REST: adjust analytics and wikifunctions REST Sandbox visibility (T422770 T423058 T422771)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429282 wikitext text/x-wiki pvqnz6xhyuugia4ymtrj28xsl0udy39 2429283 2429282 2026-06-22T20:30:35Z Stashbot 7414 bpirkle@deploy1003: bpirkle: Continuing with deployment 2429283 wikitext text/x-wiki fmf3dht4g7g20h9ynfmck6si03kchy1 2429286 2429283 2026-06-22T20:34:54Z Stashbot 7414 bpirkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304175|REST: adjust analytics and wikifunctions REST Sandbox visibility (T422770 T423058 T422771)]] (duration: 07m 30s) 2429286 wikitext text/x-wiki m99mnfc9vt2xeypadzb7jadfrm3ucxk 2429288 2429286 2026-06-22T20:47:01Z Stashbot 7414 arlolra@deploy1003: Started scap sync-world: Backport for [[gerrit:1304887|Add a hidden lint for pre ext tags expanding templates (T353697)]] 2429288 wikitext text/x-wiki e5ks7kbxdb2ax4cl5d9js95gu14guk6 2429292 2429288 2026-06-22T21:06:44Z Stashbot 7414 arlolra@deploy1003: arlolra: Backport for [[gerrit:1304887|Add a hidden lint for pre ext tags expanding templates (T353697)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429292 wikitext text/x-wiki je344876huyq1rqwbh90cygrm0hbo2s 2429294 2429292 2026-06-22T21:08:34Z Stashbot 7414 arlolra@deploy1003: arlolra: Continuing with deployment 2429294 wikitext text/x-wiki 1q5yu9578zw769vcfjfsc4jvjphxb2g 2429297 2429294 2026-06-22T21:18:45Z Stashbot 7414 cdobbins@dns1004: START - running authdns-update 2429297 wikitext text/x-wiki 8o0oak6znbg3g6gvauw6vz9nur8m2cq 2429298 2429297 2026-06-22T21:20:40Z Stashbot 7414 cdobbins@dns1004: END - running authdns-update 2429298 wikitext text/x-wiki 8g0h5b9fs7uoth4wy0zr34r4xnnmxjp 2429299 2429298 2026-06-22T21:20:46Z Stashbot 7414 arlolra@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304887|Add a hidden lint for pre ext tags expanding templates (T353697)]] (duration: 33m 42s) 2429299 wikitext text/x-wiki of8jpal7tbvpdt0u42lny3shdhk1xic 2429303 2429299 2026-06-22T21:32:07Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1304896|RiskScoreCollector: Make error_context a string map (T429594)]], [[gerrit:1304904|CaptchaPreAuthenticationProvider: Clear solved state on failure (T429705)]] 2429303 wikitext text/x-wiki dd59mxa153c6c7z83z8f4fc2zg5jlxc 2429304 2429303 2026-06-22T21:36:05Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1304896|RiskScoreCollector: Make error_context a string map (T429594)]], [[gerrit:1304904|CaptchaPreAuthenticationProvider: Clear solved state on failure (T429705)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429304 wikitext text/x-wiki dvq122uullf4v18uuay9qp6t30lxq1d 2429305 2429304 2026-06-22T21:36:52Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2429305 wikitext text/x-wiki olam3llvf3m98rpx7njru4a2kzwan25 2429306 2429305 2026-06-22T21:43:20Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304896|RiskScoreCollector: Make error_context a string map (T429594)]], [[gerrit:1304904|CaptchaPreAuthenticationProvider: Clear solved state on failure (T429705)]] (duration: 11m 12s) 2429306 wikitext text/x-wiki 249u0k8iyj1k72c0j4cbqq0sg91bckp 2429307 2429306 2026-06-22T21:53:00Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1304911|hCaptcha: Enable for badlogin on non-SUL wikis (T429843)]] 2429307 wikitext text/x-wiki j52a0ikf1u0jnaipdfa6c49l9xrrfbi 2429308 2429307 2026-06-22T21:55:03Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1304911|hCaptcha: Enable for badlogin on non-SUL wikis (T429843)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429308 wikitext text/x-wiki 53z6enazmu8dah9w8u09qnnwnhrq1aw 2429310 2429308 2026-06-22T22:04:03Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Rolling back deployment 2429310 wikitext text/x-wiki kfxz7cxmujglky8nvdp4v15twp676p8 2429311 2429310 2026-06-22T22:04:36Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304911|hCaptcha: Enable for badlogin on non-SUL wikis (T429843)]] (duration: 11m 37s) 2429311 wikitext text/x-wiki fj930f9lfl5gza0rvw0y02p7uz4d2w5 2429313 2429311 2026-06-22T22:10:36Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1304913|hCaptcha: Apply generic settings for bad login (T429843)]], [[gerrit:1304911|hCaptcha: Enable for badlogin on non-SUL wikis (T429843)]] 2429313 wikitext text/x-wiki ejrdieknxepkttazhi87xmvy4hmrb4z 2429314 2429313 2026-06-22T22:12:36Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1304913|hCaptcha: Apply generic settings for bad login (T429843)]], [[gerrit:1304911|hCaptcha: Enable for badlogin on non-SUL wikis (T429843)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429314 wikitext text/x-wiki imtnzdvvlpmn2i1z8oyptkoiin8wfrh 2429315 2429314 2026-06-22T22:13:30Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2429315 wikitext text/x-wiki qxmauqakl3x5ziomwv499uhk057wx54 2429317 2429315 2026-06-22T22:17:50Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304913|hCaptcha: Apply generic settings for bad login (T429843)]], [[gerrit:1304911|hCaptcha: Enable for badlogin on non-SUL wikis (T429843)]] (duration: 07m 14s) 2429317 wikitext text/x-wiki 6smw1z1jsqypd5l2dzn9cjgthdgamja 2429318 2429317 2026-06-22T22:29:48Z Stashbot 7414 cdobbins@dns1004: START - running authdns-update 2429318 wikitext text/x-wiki qtopf7lm86k34baknues4iqic2t7zhz 2429319 2429318 2026-06-22T22:31:39Z Stashbot 7414 cdobbins@dns1004: END - running authdns-update 2429319 wikitext text/x-wiki 9nlxuu78po5ao2ok0spz0itp5sbcaox 2429323 2429319 2026-06-22T22:35:42Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.dns.netbox 2429323 wikitext text/x-wiki py9az6w1ohjyhwqaky39pnjjgldiocp 2429324 2429323 2026-06-22T22:38:34Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1304920|hCaptcha: Enable for bad login on group1 (T429843)]] 2429324 wikitext text/x-wiki 7d83n426qi9d63ynhjd0asigyylbbxx 2429325 2429324 2026-06-22T22:39:29Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host clouddb1026.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429325 wikitext text/x-wiki spmxq7kbkswltgleilnummqeebt8xjl 2429326 2429325 2026-06-22T22:39:54Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding clouddb1031 to eqiad - jclark@cumin1003" 2429326 wikitext text/x-wiki kqytrtx6h6gi323hsp9hk1f1rysoq4u 2429327 2429326 2026-06-22T22:39:59Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding clouddb1031 to eqiad - jclark@cumin1003" 2429327 wikitext text/x-wiki jq16bqee477hm0fmjpentpvv6i97da3 2429328 2429327 2026-06-22T22:40:01Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2429328 wikitext text/x-wiki snjh0swufz3itvob0n8g30cjcfd72if 2429329 2429328 2026-06-22T22:40:29Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host clouddb1027.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429329 wikitext text/x-wiki qd7zep6wqv2koe0fg2wac3k3getbf1o 2429330 2429329 2026-06-22T22:40:35Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1304920|hCaptcha: Enable for bad login on group1 (T429843)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429330 wikitext text/x-wiki lbnrkxi3vpvpfe9hynz5l10fxlnnatt 2429331 2429330 2026-06-22T22:40:47Z Stashbot 7414 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host clouddb1027.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429331 wikitext text/x-wiki hl1y2kmcr6svwyvyvo6d6jto5pxpylw 2429332 2429331 2026-06-22T22:41:02Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2429332 wikitext text/x-wiki czpevo1a3ge8g1jryjluv8omv6r3pc9 2429333 2429332 2026-06-22T22:41:59Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host clouddb1027.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429333 wikitext text/x-wiki rfrd0y0cj5hxltg3ipqlbku6k9jondz 2429334 2429333 2026-06-22T22:43:12Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host clouddb1028.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429334 wikitext text/x-wiki inn0habg4ys5tp6klshkhy43uzlrx48 2429335 2429334 2026-06-22T22:44:21Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host clouddb1029.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429335 wikitext text/x-wiki rnfm1etfjyeneeww7v03zwx23kth7sq 2429336 2429335 2026-06-22T22:45:20Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304920|hCaptcha: Enable for bad login on group1 (T429843)]] (duration: 06m 45s) 2429336 wikitext text/x-wiki cyup8gr0p0p4sqjmgzowukox2fxx20l 2429337 2429336 2026-06-22T22:48:15Z Stashbot 7414 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host clouddb1027.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429337 wikitext text/x-wiki 69jre7anh0d9qzohnjlw8humm0b9eme 2429338 2429337 2026-06-22T22:49:42Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host clouddb1026.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429338 wikitext text/x-wiki 73ps4nk46jvln2ns3hspup44p68iamv 2429340 2429338 2026-06-22T22:53:30Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host clouddb1030.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429340 wikitext text/x-wiki ikj16fskw5eg4iqgze53ovra5j79eiu 2429341 2429340 2026-06-22T22:54:11Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host clouddb1029.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429341 wikitext text/x-wiki jlb5gqgn65sja6qnorzbr8cxkgt2aqz 2429342 2429341 2026-06-22T22:54:23Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host clouddb1031.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429342 wikitext text/x-wiki 997wl9v5lz6uxw3yvdn15ldug4stpdn 2429343 2429342 2026-06-22T22:54:26Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host clouddb1028.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429343 wikitext text/x-wiki 7sq2gifdrqr6wajnlsxowpzbh1x06z2 2429344 2429343 2026-06-22T22:55:01Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host clouddb1032.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429344 wikitext text/x-wiki 5aa95o29yuf8zevhncknhlqliwpyv0y 2429345 2429344 2026-06-22T22:55:42Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host clouddb1033.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429345 wikitext text/x-wiki s3rihs56ila6xsfqm64dpz9fh3um7jd 2429346 2429345 2026-06-22T23:03:03Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host clouddb1030.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429346 wikitext text/x-wiki s0v1gsb3764tzck25cy0n1j7d1118eq 2429347 2429346 2026-06-22T23:03:30Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host clouddb1031.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429347 wikitext text/x-wiki gs9nabmkflwq3ir3y4la1btutpvags5 2429348 2429347 2026-06-22T23:04:04Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host clouddb1032.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429348 wikitext text/x-wiki 11pi3dpxp7nx92hj4amkjq06m08pwka 2429349 2429348 2026-06-22T23:04:37Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host clouddb1033.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429349 wikitext text/x-wiki e3jqqu6ut7byzyu2jszlyg694ao60jn 2429350 2429349 2026-06-22T23:04:48Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.reimage for host clouddb1032.eqiad.wmnet with OS trixie 2429350 wikitext text/x-wiki dro36at7t1zf07lvfnili8okdl4zbtu 2429351 2429350 2026-06-22T23:05:14Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.reimage for host clouddb1033.eqiad.wmnet with OS trixie 2429351 wikitext text/x-wiki pt2cpf00swbljdg91g7wsiacny6tgym 2429352 2429351 2026-06-22T23:05:50Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.reimage for host clouddb1031.eqiad.wmnet with OS trixie 2429352 wikitext text/x-wiki 2uzzjwupsj8x0nrillzybxjvgw2o0ao 2429353 2429352 2026-06-22T23:06:06Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.reimage for host clouddb1030.eqiad.wmnet with OS trixie 2429353 wikitext text/x-wiki kasoqigkc4nn2apb7e2f5zb1i7s1cur 2429356 2429353 2026-06-22T23:21:14Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on clouddb1032.eqiad.wmnet with reason: host reimage 2429356 wikitext text/x-wiki te2p8evvafzmcmrmdoabw5zse6hrq67 2429357 2429356 2026-06-22T23:25:06Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddb1032.eqiad.wmnet with reason: host reimage 2429357 wikitext text/x-wiki es2l7q0vtuwmym7gyidjsli9o85b073 2429358 2429357 2026-06-22T23:25:51Z Stashbot 7414 jdlrobson@deploy1003: Started scap sync-world: Backport for [[gerrit:1304151|Ensure page tools icons are only shown on small viewports (T426131)]], [[gerrit:1304863|Fix duplicate print and other projects menu links in main menu (T429676)]] 2429358 wikitext text/x-wiki 95gpail3n9onixlzby8132b9js965mw 2429359 2429358 2026-06-22T23:26:05Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host clouddb1027.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429359 wikitext text/x-wiki a9ec5ghrbkuxelehe13ps750v4rquoe 2429360 2429359 2026-06-22T23:27:36Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on clouddb1030.eqiad.wmnet with reason: host reimage 2429360 wikitext text/x-wiki rvt2f9uipin5br72d9v3wfgrv3qhdhr 2429361 2429360 2026-06-22T23:27:39Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on clouddb1031.eqiad.wmnet with reason: host reimage 2429361 wikitext text/x-wiki ey5uhc2zbop5s6g99by2tvrezj9o4tt 2429362 2429361 2026-06-22T23:27:50Z Stashbot 7414 jdlrobson@deploy1003: jdlrobson: Backport for [[gerrit:1304151|Ensure page tools icons are only shown on small viewports (T426131)]], [[gerrit:1304863|Fix duplicate print and other projects menu links in main menu (T429676)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429362 wikitext text/x-wiki cffma4er4575kbbq3jij3bb976c13zt 2429363 2429362 2026-06-22T23:29:51Z Stashbot 7414 jdlrobson@deploy1003: jdlrobson: Continuing with deployment 2429363 wikitext text/x-wiki ciq9jfnvnt9cq0pqhdoesjybsy0hb4n 2429364 2429363 2026-06-22T23:31:55Z Stashbot 7414 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host clouddb1027.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429364 wikitext text/x-wiki hezyiu828k3kkhe5wk4u84bd96we692 2429365 2429364 2026-06-22T23:32:35Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddb1030.eqiad.wmnet with reason: host reimage 2429365 wikitext text/x-wiki gc576a0904q8lix8fmwvu0n84vi930a 2429366 2429365 2026-06-22T23:34:09Z Stashbot 7414 jdlrobson@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304151|Ensure page tools icons are only shown on small viewports (T426131)]], [[gerrit:1304863|Fix duplicate print and other projects menu links in main menu (T429676)]] (duration: 08m 18s) 2429366 wikitext text/x-wiki 0mjakfc1pdsuiwuv10aaolgtwz8xloy 2429367 2429366 2026-06-22T23:36:30Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddb1031.eqiad.wmnet with reason: host reimage 2429367 wikitext text/x-wiki crr2cg9hx7xcsdmb1i80vqxjxwsbx4t 2429368 2429367 2026-06-22T23:38:20Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host clouddb1027.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429368 wikitext text/x-wiki 01bcqhpjln493sk7j6l1n0wmbak4yz0 2429369 2429368 2026-06-22T23:39:55Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429369 wikitext text/x-wiki ca802j9au39ohhy4j5cht5jbgg24fbc 2429370 2429369 2026-06-22T23:40:18Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429370 wikitext text/x-wiki 5ufksrfepie3i08kwx7dmy6zv3kq00q 2429371 2429370 2026-06-22T23:40:20Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddb1032.eqiad.wmnet with OS trixie 2429371 wikitext text/x-wiki 49dqnj86onaonkgl8kly93dyj9g6lss 2429372 2429371 2026-06-22T23:40:50Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.reimage for host clouddb1029.eqiad.wmnet with OS trixie 2429372 wikitext text/x-wiki 4m34tirgv27tzobpixx57bq8kavvdzp 2429373 2429372 2026-06-22T23:47:05Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.reimage for host clouddb1028.eqiad.wmnet with OS trixie 2429373 wikitext text/x-wiki rymydn08d78vxsh5rc4gsmip35h9agh 2429374 2429373 2026-06-22T23:47:11Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429374 wikitext text/x-wiki lvwiekgx8xy94q17gvackuycel3v55y 2429375 2429374 2026-06-22T23:47:26Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429375 wikitext text/x-wiki t67ucg7onand0k6xr60uty0hhf22cv7 2429376 2429375 2026-06-22T23:47:28Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddb1030.eqiad.wmnet with OS trixie 2429376 wikitext text/x-wiki a4yurn9yfmx3109v9lsgctap9u3zlb7 2429377 2429376 2026-06-22T23:47:59Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.reimage for host clouddb1026.eqiad.wmnet with OS trixie 2429377 wikitext text/x-wiki 5umu69td0ypsl1f24du3xrcmvc9psn1 2429378 2429377 2026-06-22T23:50:01Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host clouddb1027.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2429378 wikitext text/x-wiki ir6lqtwlkwqo651hbcrudxiti6erwfv 2429379 2429378 2026-06-22T23:50:58Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429379 wikitext text/x-wiki 9aq0qogflr79el7c7s9ph3bid9zwzc0 2429380 2429379 2026-06-22T23:51:15Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on clouddb1029.eqiad.wmnet with reason: host reimage 2429380 wikitext text/x-wiki 8wvp7f6wcimcmx3u6l5owl8bsi5qkmt 2429381 2429380 2026-06-22T23:51:58Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429381 wikitext text/x-wiki blntgbzeag1hq8s5jp34gd4777yaaya 2429382 2429381 2026-06-22T23:52:00Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddb1031.eqiad.wmnet with OS trixie 2429382 wikitext text/x-wiki p5skqhtuydcizenuhpgv0axc4u4yti9 2429383 2429382 2026-06-22T23:52:02Z Stashbot 7414 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host clouddb1033.eqiad.wmnet with OS trixie 2429383 wikitext text/x-wiki phk8xy9zgpgwex7ae2kyfclkjxuuh6p 2429384 2429383 2026-06-22T23:53:01Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.reimage for host clouddb1033.eqiad.wmnet with OS trixie 2429384 wikitext text/x-wiki 6ko0kangeoov3s9powcaxkqyxmcjvxy 2429385 2429384 2026-06-22T23:55:40Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddb1029.eqiad.wmnet with reason: host reimage 2429385 wikitext text/x-wiki nxwt89cwbsqxah23mvfv0curtedab56 2429386 2429385 2026-06-22T23:57:24Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on clouddb1028.eqiad.wmnet with reason: host reimage 2429386 wikitext text/x-wiki 7tvz2941zvcrtedewyj1co6dkejinip 2429387 2429386 2026-06-22T23:57:31Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.reimage for host clouddb1027.eqiad.wmnet with OS trixie 2429387 wikitext text/x-wiki 3h2uyjzpps06em9hmm9wvy957aenj57 2429388 2429387 2026-06-22T23:57:58Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on clouddb1026.eqiad.wmnet with reason: host reimage 2429388 wikitext text/x-wiki a1033wffubbkrkk3pq2aq7of2fjgn99 2429390 2429388 2026-06-23T00:03:26Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on clouddb1033.eqiad.wmnet with reason: host reimage 2429390 wikitext text/x-wiki 10h9w48d4xvb8keqzf65h9rxbdxsbst 2429391 2429390 2026-06-23T00:03:59Z Stashbot 7414 egardner@deploy1003: Started scap sync-world: Backport for [[gerrit:1304914|Inject service RepoGroup into Hooks]], [[gerrit:1304909|MMV Beta Viewer: Improve loading/navigation UX (T429193)]], [[gerrit:1304910|Take the feature out of beta (T429509)]] 2429391 wikitext text/x-wiki gjujlsgjy3k3c56or4mumefycyzfi1i 2429392 2429391 2026-06-23T00:05:41Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddb1028.eqiad.wmnet with reason: host reimage 2429392 wikitext text/x-wiki t07p399itzokn1mwannqndbc8fvvjql 2429393 2429392 2026-06-23T00:07:46Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on clouddb1027.eqiad.wmnet with reason: host reimage 2429393 wikitext text/x-wiki o7gj88yye5w1musmui7evt58f8u9s81 2429394 2429393 2026-06-23T00:08:41Z Stashbot 7414 egardner@deploy1003: sync-world failed: <CalledProcessError> Command 'sudo -u mwbuilder /srv/mwbuilder/release/make-container-image/build-images.py --http-proxy http://webproxy:8080 --https-proxy http://webproxy:8080 /srv/mediawiki-staging/scap/image-build --staging-dir /srv/mediawiki-staging --mediawiki-versions 1.47.0-wmf.6,1.47.0-wmf.7,next --multiversion-image-basename docker-registry.discovery.wmnet/restricted/me 2429394 wikitext text/x-wiki l4grs4s74ag4fcjd0ujwd3nmjbelup9 2429395 2429394 2026-06-23T00:09:46Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddb1026.eqiad.wmnet with reason: host reimage 2429395 wikitext text/x-wiki 118deew95ez4zyg6sjh9ffhn9bgz2fw 2429396 2429395 2026-06-23T00:10:20Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429396 wikitext text/x-wiki 77ejrq9qchazj63trdoscte4spqpm91 2429397 2429396 2026-06-23T00:10:40Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429397 wikitext text/x-wiki o7q0sngpgmwfj4tkqbgg8njnnkagvvn 2429398 2429397 2026-06-23T00:10:42Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddb1029.eqiad.wmnet with OS trixie 2429398 wikitext text/x-wiki kojhxp1pyxp5szncxjpekisjzv9p7o4 2429399 2429398 2026-06-23T00:11:29Z Stashbot 7414 egardner@deploy1003: Started scap sync-world: Backport for [[gerrit:1304914|Inject service RepoGroup into Hooks]], [[gerrit:1304909|MMV Beta Viewer: Improve loading/navigation UX (T429193)]], [[gerrit:1304910|Take the feature out of beta (T429509)]] 2429399 wikitext text/x-wiki gp67zdtm1pek9177nh2d2jpiulz9eoi 2429400 2429399 2026-06-23T00:14:35Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddb1033.eqiad.wmnet with reason: host reimage 2429400 wikitext text/x-wiki 9j9drnmsybo3z1z468dcwbuxbjo570s 2429401 2429400 2026-06-23T00:15:15Z Stashbot 7414 egardner@deploy1003: sync-world failed: <CalledProcessError> Command 'sudo -u mwbuilder /srv/mwbuilder/release/make-container-image/build-images.py --http-proxy http://webproxy:8080 --https-proxy http://webproxy:8080 /srv/mediawiki-staging/scap/image-build --staging-dir /srv/mediawiki-staging --mediawiki-versions 1.47.0-wmf.6,1.47.0-wmf.7,next --multiversion-image-basename docker-registry.discovery.wmnet/restricted/me 2429401 wikitext text/x-wiki 6hpw9hrkp0razf0ufke8fdb6sx5ofg5 2429402 2429401 2026-06-23T00:17:57Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddb1027.eqiad.wmnet with reason: host reimage 2429402 wikitext text/x-wiki 6hqsl9dl7bgrnacntdjf0j7uhx35gk7 2429403 2429402 2026-06-23T00:20:31Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429403 wikitext text/x-wiki ljstb8rxdobt6guhpp6fsviysd5b74r 2429404 2429403 2026-06-23T00:20:52Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429404 wikitext text/x-wiki l0tzjgyjl51u30b32zvkw8vy0igxngo 2429405 2429404 2026-06-23T00:20:54Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddb1028.eqiad.wmnet with OS trixie 2429405 wikitext text/x-wiki koocwq6i2bzathdizs69vijs9u6vyyi 2429406 2429405 2026-06-23T00:24:11Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429406 wikitext text/x-wiki j2vtw1hetlsiuzjwqvz921f8999w9lr 2429407 2429406 2026-06-23T00:24:27Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429407 wikitext text/x-wiki p3venatvrk8vo4y3bayrtydyttynvhs 2429408 2429407 2026-06-23T00:24:31Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddb1026.eqiad.wmnet with OS trixie 2429408 wikitext text/x-wiki rkoiwrtz3q6afc0yg2vbtmp752x0c09 2429409 2429408 2026-06-23T00:29:54Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429409 wikitext text/x-wiki qf68rjn5eppbzr3dgdbzapn7nsv9bnj 2429410 2429409 2026-06-23T00:30:10Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429410 wikitext text/x-wiki bdii5966xs24s3kghv8dkz0kct7zaq1 2429411 2429410 2026-06-23T00:30:12Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddb1033.eqiad.wmnet with OS trixie 2429411 wikitext text/x-wiki 81phjyrxd4vkwog0njz4g2z497u1p3r 2429412 2429411 2026-06-23T00:33:08Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429412 wikitext text/x-wiki n3fgvvjcu6s27clcs3jlqwzjexfkojn 2429413 2429412 2026-06-23T00:33:17Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" 2429413 wikitext text/x-wiki lpviahaelfgcgbvrmn5pq7eh417aiad 2429414 2429413 2026-06-23T00:33:19Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddb1027.eqiad.wmnet with OS trixie 2429414 wikitext text/x-wiki 18d88c9rhzuuvs1pslq307in9hd4uza 2429415 2429414 2026-06-23T00:39:08Z Stashbot 7414 mutante: codesearch10: systemctl start codesearch-write-config; systemctl restart hound-operations (gerrit:1304848) (T429819) 2429415 wikitext text/x-wiki 84bpeqj70dvop9ja9elrmo4vh7sn3yz 2429416 2429415 2026-06-23T00:46:40Z Stashbot 7414 swfrench-wmf: manually started scap-clean-images.service on deply1003 to reclaim /srv space (93% -> 66% utilization) 2429416 wikitext text/x-wiki 5ezyjgom6g6l8a6lka6r0xkf1vcfr46 2429417 2429416 2026-06-23T00:51:24Z Stashbot 7414 egardner@deploy1003: Started scap sync-world: Backport for [[gerrit:1304914|Inject service RepoGroup into Hooks]], [[gerrit:1304909|MMV Beta Viewer: Improve loading/navigation UX (T429193)]], [[gerrit:1304910|Take the feature out of beta (T429509)]] 2429417 wikitext text/x-wiki 6jqwbjsjru0b6kdrpki9p8nmqf158f3 2429418 2429417 2026-06-23T00:56:15Z Stashbot 7414 brett@dns7002: START - running authdns-update 2429418 wikitext text/x-wiki qpl8uq6x5ovp2ro2w7pfhvag7bfrrim 2429419 2429418 2026-06-23T00:57:57Z Stashbot 7414 brett@dns7002: END - running authdns-update 2429419 wikitext text/x-wiki dt1ltuss9m1j2qae07fc6xa5em2fsua 2429420 2429419 2026-06-23T01:08:19Z Stashbot 7414 brett@dns7002: START - running authdns-update 2429420 wikitext text/x-wiki k0lfbqwd7iojr89r245qmg5otuda10l 2429421 2429420 2026-06-23T01:10:08Z Stashbot 7414 brett@dns7002: END - running authdns-update 2429421 wikitext text/x-wiki 00v2dr1fgj3hhxwi7c1dkehl5k4iwgw 2429422 2429421 2026-06-23T01:10:41Z Stashbot 7414 egardner@deploy1003: egardner: Backport for [[gerrit:1304914|Inject service RepoGroup into Hooks]], [[gerrit:1304909|MMV Beta Viewer: Improve loading/navigation UX (T429193)]], [[gerrit:1304910|Take the feature out of beta (T429509)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429422 wikitext text/x-wiki ra68d1lhwn89dwm694q61lhesmepex1 2429423 2429422 2026-06-23T01:12:41Z Stashbot 7414 egardner@deploy1003: egardner: Continuing with deployment 2429423 wikitext text/x-wiki 05ef9uc96v94s24nol3yn1led1klmx2 2429424 2429423 2026-06-23T01:24:47Z Stashbot 7414 egardner@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304914|Inject service RepoGroup into Hooks]], [[gerrit:1304909|MMV Beta Viewer: Improve loading/navigation UX (T429193)]], [[gerrit:1304910|Take the feature out of beta (T429509)]] (duration: 33m 22s) 2429424 wikitext text/x-wiki ma1yd0yy8o7liuzq5iapymrv1jlffh0 2429425 2429424 2026-06-23T01:30:09Z Stashbot 7414 egardner@deploy1003: Started scap sync-world: Backport for [[gerrit:1304859|Remove multimediaviewer-beta]] 2429425 wikitext text/x-wiki kmwqb1ku4spqnppi59b5vh6gl9psx7p 2429426 2429425 2026-06-23T01:34:14Z Stashbot 7414 egardner@deploy1003: egardner, ksarabia: Backport for [[gerrit:1304859|Remove multimediaviewer-beta]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429426 wikitext text/x-wiki hfzwxu4igdlyh9kim0vqpq1bmzqpux8 2429427 2429426 2026-06-23T01:34:45Z Stashbot 7414 egardner@deploy1003: egardner, ksarabia: Continuing with deployment 2429427 wikitext text/x-wiki 1v2rcslldvlg5cvqthtg0nqcgye67d0 2429428 2429427 2026-06-23T01:41:13Z Stashbot 7414 egardner@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304859|Remove multimediaviewer-beta]] (duration: 11m 14s) 2429428 wikitext text/x-wiki b03dfuqsvilznp4vijzx2i3ywewwgua 2429429 2429428 2026-06-23T02:00:42Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2429429 wikitext text/x-wiki 43bzlgdalhgcahh5ferubdhke4gnm33 2429430 2429429 2026-06-23T02:08:08Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 07m 25s) 2429430 wikitext text/x-wiki 2vapzy5sdelptbez6ffce0fbs7gtf3t 2429431 2429430 2026-06-23T03:08:38Z Stashbot 7414 mwpresync@deploy1003: Started scap sync-world: testwikis to 1.47.0-wmf.8 refs T423917 2429431 wikitext text/x-wiki 0coxfem7sscj81xissf4kplk9bgerj2 2429432 2429431 2026-06-23T03:44:19Z Stashbot 7414 mwpresync@deploy1003: Finished scap sync-world: testwikis to 1.47.0-wmf.8 refs T423917 (duration: 39m 23s) 2429432 wikitext text/x-wiki l1y7wq0kg4cmmlpjanlmqaukxucvmxz 2429433 2429432 2026-06-23T04:02:44Z Stashbot 7414 mwpresync@deploy1003: Pruned MediaWiki: 1.47.0-wmf.5 (duration: 02m 37s) 2429433 wikitext text/x-wiki mb8oxs5cr1sijyo5828zwliy687o7ov 2429434 2429433 2026-06-23T05:01:27Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 9 hosts with reason: Primary switchover es7 T429794 2429434 wikitext text/x-wiki gwlrfyftaq2qhx0v6janx76a2nx8ei0 2429435 2429434 2026-06-23T05:01:40Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Set es2039 with weight 0 T429794', diff saved to https://phabricator.wikimedia.org/P94330 and previous config saved to /var/cache/conftool/dbconfig/20260623-050137-marostegui.json 2429435 wikitext text/x-wiki ozhq8ggq0uqdfo0omsrsdhe038nkdul 2429437 2429435 2026-06-23T05:07:34Z Stashbot 7414 marostegui: Starting es7 codfw failover from es2038 to es2039 - T429794 2429437 wikitext text/x-wiki g7vhzeqv0qwdiseskku19a5newjxhex 2429438 2429437 2026-06-23T05:08:01Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Promote es2039 to es7 primary T429794', diff saved to https://phabricator.wikimedia.org/P94331 and previous config saved to /var/cache/conftool/dbconfig/20260623-050758-marostegui.json 2429438 wikitext text/x-wiki 2alqhweo3lydc797r528q4m2sqg41fx 2429439 2429438 2026-06-23T05:10:16Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool es2038 T429794', diff saved to https://phabricator.wikimedia.org/P94332 and previous config saved to /var/cache/conftool/dbconfig/20260623-051012-marostegui.json 2429439 wikitext text/x-wiki pkfzfus5zu18vt650riaj9r2wedwdrx 2429440 2429439 2026-06-23T05:12:17Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2429440 wikitext text/x-wiki 39r6l2jckp2wprhl1nmwf0upjex6q3x 2429441 2429440 2026-06-23T05:12:21Z Stashbot 7414 marostegui@cumin1003: dbmaint on es7@codfw T429463 2429441 wikitext text/x-wiki 1rmrpj28hpxgv2jnkbgoisicvd35i2s 2429442 2429441 2026-06-23T05:12:27Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es2038: Upgrading es2038.codfw.wmnet 2429442 wikitext text/x-wiki 2vil76o3mwv50h3niifbyrzt2tb4y7r 2429443 2429442 2026-06-23T05:12:37Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool es2038: Upgrading es2038.codfw.wmnet 2429443 wikitext text/x-wiki 63pdz0ymtna31t9od1sex4uqphqkwur 2429445 2429443 2026-06-23T05:13:40Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es2038.codfw.wmnet with OS trixie 2429445 wikitext text/x-wiki 4cyizwug2wjaif3li0om9vs8jb1zfgu 2429447 2429445 2026-06-23T05:31:07Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es2038.codfw.wmnet with reason: host reimage 2429447 wikitext text/x-wiki p1hn17msgzm6ph2gitc5vyyw6wlydfr 2429450 2429447 2026-06-23T05:35:42Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es2038.codfw.wmnet with reason: host reimage 2429450 wikitext text/x-wiki ivh86uqfetxi06phjl5ilv44hlskk45 2429452 2429450 2026-06-23T05:41:21Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on pc2018.codfw.wmnet with reason: Reimage to Trixie 2429452 wikitext text/x-wiki 16ey8efuxjyaum1qqdi3hv0zi2bo4lc 2429453 2429452 2026-06-23T05:41:24Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool pc2018: Reimage to Trixie 2429453 wikitext text/x-wiki a4eoy74fzleo1y27yfnrtyrj7mo2iwr 2429454 2429453 2026-06-23T05:41:26Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2429454 wikitext text/x-wiki 3a95spgz285zkwac2i4gvt9nptsbuhs 2429455 2429454 2026-06-23T05:41:32Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) 2429455 wikitext text/x-wiki 8apa5zutv2ix7sd0090x74awk4q5ply 2429456 2429455 2026-06-23T05:41:34Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool pc2018: Reimage to Trixie 2429456 wikitext text/x-wiki 5qgibk5pkz440a1jqoop2vuqosi21sn 2429457 2429456 2026-06-23T05:42:35Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host pc2018.codfw.wmnet with OS trixie 2429457 wikitext text/x-wiki b4pzly844u6fd6fko20ingedw0q4sfq 2429458 2429457 2026-06-23T05:53:24Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es2038.codfw.wmnet with OS trixie 2429458 wikitext text/x-wiki 9hse9ds6x7w52c38tdrnl783hsvnuv3 2429459 2429458 2026-06-23T05:59:45Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on pc2018.codfw.wmnet with reason: host reimage 2429459 wikitext text/x-wiki 2xpmftnb6ano7us4gng3fw9b9onjver 2429460 2429459 2026-06-23T06:00:43Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es2038: Migration of es2038.codfw.wmnet completed 2429460 wikitext text/x-wiki lam3wrsetu0qtc7kphpk2sn8l1vvgeu 2429461 2429460 2026-06-23T06:01:49Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 21 hosts with reason: Primary switchover s6 T429868 2429461 wikitext text/x-wiki oftm7avqdhom3pxuyzoilin3hoaqary 2429462 2429461 2026-06-23T06:02:00Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Set db1173 with weight 0 T429868', diff saved to https://phabricator.wikimedia.org/P94335 and previous config saved to /var/cache/conftool/dbconfig/20260623-060157-fceratto.json 2429462 wikitext text/x-wiki t70ctdrllszk3cz3dpk8kpwjuye0006 2429463 2429462 2026-06-23T06:04:30Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on pc2018.codfw.wmnet with reason: host reimage 2429463 wikitext text/x-wiki m33yf0qd95ljylqacj61jiowkq56p8s 2429465 2429463 2026-06-23T06:06:52Z Stashbot 7414 federico3: Starting s6 eqiad failover from db1201 to db1173 - T429868 2429465 wikitext text/x-wiki 253ze4prozb5hnd1942s8qsv8zfvc77 2429466 2429465 2026-06-23T06:07:11Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Set s6 eqiad as read-only for maintenance - T429868', diff saved to https://phabricator.wikimedia.org/P94336 and previous config saved to /var/cache/conftool/dbconfig/20260623-060708-fceratto.json 2429466 wikitext text/x-wiki 0e2jaa68nk8zrzuqpzd1anp314jaed9 2429467 2429466 2026-06-23T06:11:30Z Stashbot 7414 fceratto@dns1004: START - running authdns-update 2429467 wikitext text/x-wiki oteigbcs3azcfaz2f0xlumw6aaiad3s 2429468 2429467 2026-06-23T06:13:21Z Stashbot 7414 fceratto@dns1004: END - running authdns-update 2429468 wikitext text/x-wiki 9uho64g30f49va9fe9pfnrctm30hqwm 2429469 2429468 2026-06-23T06:14:19Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depool db1201 T429868', diff saved to https://phabricator.wikimedia.org/P94338 and previous config saved to /var/cache/conftool/dbconfig/20260623-061416-fceratto.json 2429469 wikitext text/x-wiki b4d4dd748xt7fun88ba1vpjui3tseag 2429470 2429469 2026-06-23T06:14:54Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.pool pool db1201: Repooling after switchover 2429470 wikitext text/x-wiki nz4vsfyzxgkuytp4vibnipocghnedx4 2429471 2429470 2026-06-23T06:27:30Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host pc2018.codfw.wmnet with OS trixie 2429471 wikitext text/x-wiki b8uywnfmc336roftd3ruodq0gp3ocju 2429472 2429471 2026-06-23T06:28:23Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool pc2018: after reimage to trixie 2429472 wikitext text/x-wiki sdchabus7m1uqw3mudp1a0l11ahouxl 2429473 2429472 2026-06-23T06:28:26Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.pool (exit_code=99) pool pc2018: after reimage to trixie 2429473 wikitext text/x-wiki 6jtjrcojll3s10pkm76bja6aspof6t8 2429474 2429473 2026-06-23T06:32:04Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on pc1018.eqiad.wmnet with reason: Reimage to Trixie 2429474 wikitext text/x-wiki op4yke1eqzgyxvzqpvw2cszmizql65w 2429475 2429474 2026-06-23T06:32:07Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool pc1018: Reimage to Trixie 2429475 wikitext text/x-wiki qjcfep2eo7ktyy24d2cjt3ienv3skln 2429476 2429475 2026-06-23T06:32:09Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2429476 wikitext text/x-wiki 219531qvbl77absxbekge2o7ujlw1oq 2429477 2429476 2026-06-23T06:32:13Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) 2429477 wikitext text/x-wiki 8qn7wnvobtx3e7ttmspx91ejvxux4ei 2429478 2429477 2026-06-23T06:32:15Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool pc1018: Reimage to Trixie 2429478 wikitext text/x-wiki 26m32gkt0mkvzwnfrh4l3o3m6h759s5 2429481 2429478 2026-06-23T06:37:28Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host pc1018.eqiad.wmnet with OS trixie 2429481 wikitext text/x-wiki ck68t2aatl6nkx4dujuka0lozfs0oed 2429482 2429481 2026-06-23T06:46:14Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es2038: Migration of es2038.codfw.wmnet completed 2429482 wikitext text/x-wiki ppcsat9mtxqv1yrfmgug4cphxziuz05 2429483 2429482 2026-06-23T06:46:17Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2429483 wikitext text/x-wiki bwrtnm5afdgz86j3bo36v80tk36flsa 2429484 2429483 2026-06-23T06:53:58Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on pc1018.eqiad.wmnet with reason: host reimage 2429484 wikitext text/x-wiki ls2rbkx3ikb53qh5goagoxdqpbmxhtm 2429485 2429484 2026-06-23T07:00:12Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on pc1018.eqiad.wmnet with reason: host reimage 2429485 wikitext text/x-wiki crb8b24rgzrmy43nl08khcx89buqg0c 2429486 2429485 2026-06-23T07:01:55Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1201: Repooling after switchover 2429486 wikitext text/x-wiki 8ih42c8ofhp0aq6se4ktpysen9p1nb9 2429487 2429486 2026-06-23T07:02:42Z Stashbot 7414 wmde-fisch@deploy1003: Started scap sync-world: Backport for [[gerrit:1304785|Global rollout - Sub-ref deployments to group 2 wikis (batch 1) (T428902)]] 2429487 wikitext text/x-wiki 5auu3nlsuqg0wdt3xvz30qp6iob3c5v 2429488 2429487 2026-06-23T07:07:32Z Stashbot 7414 wmde-fisch@deploy1003: lilients, wmde-fisch: Backport for [[gerrit:1304785|Global rollout - Sub-ref deployments to group 2 wikis (batch 1) (T428902)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429488 wikitext text/x-wiki 6fxxe56nn7422tht2jx7rgp4gl89a2f 2429489 2429488 2026-06-23T07:08:40Z Stashbot 7414 wmde-fisch@deploy1003: lilients, wmde-fisch: Continuing with deployment 2429489 wikitext text/x-wiki ppkfj6inu2va7p1a408vjao8tyae1w2 2429490 2429489 2026-06-23T07:15:20Z Stashbot 7414 wmde-fisch@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304785|Global rollout - Sub-ref deployments to group 2 wikis (batch 1) (T428902)]] (duration: 12m 37s) 2429490 wikitext text/x-wiki kq4nwehtmevfgomyboeoovrddlzmq4m 2429491 2429490 2026-06-23T07:22:42Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host pc1018.eqiad.wmnet with OS trixie 2429491 wikitext text/x-wiki e4f007lf88j6m9f0uzcoq83pcerzrsp 2429492 2429491 2026-06-23T07:23:35Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool pc1018: after reimage to trixie 2429492 wikitext text/x-wiki buwqdaelr9c6078nx1b4n6bx0pqd0p3 2429493 2429492 2026-06-23T07:23:37Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.pool (exit_code=99) pool pc1018: after reimage to trixie 2429493 wikitext text/x-wiki 165z406dii26rn1xty7948a9i6jni5f 2429494 2429493 2026-06-23T07:30:16Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool pc1018: repool after recloning another host 2429494 wikitext text/x-wiki ddicpa7m231dht75hj2xlycdrus71ju 2429495 2429494 2026-06-23T07:30:18Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.pool (exit_code=99) pool pc1018: repool after recloning another host 2429495 wikitext text/x-wiki brsy2e2q01ndtwg4r9wl7mqotoonlth 2429496 2429495 2026-06-23T07:30:41Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool pc2018: repool after recloning another host 2429496 wikitext text/x-wiki kephwo39ahb4vcdttans0c7wgiqrzam 2429497 2429496 2026-06-23T07:30:42Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.pool (exit_code=99) pool pc2018: repool after recloning another host 2429497 wikitext text/x-wiki aw4l60u0ijjdyisyziirchdd5sotiyf 2429498 2429497 2026-06-23T07:31:23Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool pc1018: repool after recloning another host 2429498 wikitext text/x-wiki sab263b3dd3ahtirkb5bw65yw7rc2vd 2429499 2429498 2026-06-23T07:31:25Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2429499 wikitext text/x-wiki mx5893q8fn9f0tcas01s3rbldlujas5 2429500 2429499 2026-06-23T07:31:37Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) 2429500 wikitext text/x-wiki 7uj54woi88jue6zxbjahdzmqb8ht0ih 2429501 2429500 2026-06-23T07:31:39Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool pc1018: repool after recloning another host 2429501 wikitext text/x-wiki cq9r14cj4komqmg5nvpkloqcd188avw 2429502 2429501 2026-06-23T07:34:21Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool pc1018: Maintenance on pc2 2429502 wikitext text/x-wiki pyxw050s7e3xo3i5arspg1o1fiilzgc 2429503 2429502 2026-06-23T07:34:23Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2429503 wikitext text/x-wiki 5vd39nlecymoe5gnj1tokgumm0cg7ev 2429504 2429503 2026-06-23T07:34:29Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) 2429504 wikitext text/x-wiki 45ujqvux2molecopejsbujcmp186hi4 2429505 2429504 2026-06-23T07:34:32Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool pc1018: Maintenance on pc2 2429505 wikitext text/x-wiki 5rkr0h4suu35doqm5daarhgc49gezuw 2429506 2429505 2026-06-23T07:34:39Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool pc1018: repool after recloning another host 2429506 wikitext text/x-wiki j65lkovrw7jltpz4hyekg5jr23rlhlf 2429507 2429506 2026-06-23T07:34:40Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.pool (exit_code=99) pool pc1018: repool after recloning another host 2429507 wikitext text/x-wiki b872qge1w3g7b12h3g9fs9tcvhrnkwr 2429508 2429507 2026-06-23T07:34:48Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool pc1018: repool after recloning another host 2429508 wikitext text/x-wiki t9o64seejpq15eptyjsueb90sdedgtx 2429509 2429508 2026-06-23T07:34:50Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2429509 wikitext text/x-wiki j1alfw48yr9asqf6n0lr45vp6ch9v8f 2429510 2429509 2026-06-23T07:35:01Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) 2429510 wikitext text/x-wiki 26ozlranfb5msib8hlh4o3a9fnu88b9 2429511 2429510 2026-06-23T07:35:03Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool pc1018: repool after recloning another host 2429511 wikitext text/x-wiki smm50omkldv8ikf2aalwa6y3cm6xajw 2429512 2429511 2026-06-23T07:46:19Z Stashbot 7414 Msz2001: Deployed update to SI private code 2429512 wikitext text/x-wiki 3okfudm066kmzf6i8y7ei0nmqtqn3xu 2429516 2429512 2026-06-23T08:30:40Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1201.eqiad.wmnet with reason: Maintenance 2429516 wikitext text/x-wiki b5c5eqjj4p7luvtrm30083utmqdelej 2429517 2429516 2026-06-23T08:30:48Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1201 (T426633)', diff saved to https://phabricator.wikimedia.org/P94351 and previous config saved to /var/cache/conftool/dbconfig/20260623-083046-fceratto.json 2429517 wikitext text/x-wiki nbuusye3sx7fz5hfqcz9m9p98iswqc2 2429519 2429517 2026-06-23T08:33:03Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool pc1018: Maintenance on pc8 2429519 wikitext text/x-wiki srw5f6t9zi6n6zsvvb12ly2ht9cwvrq 2429520 2429519 2026-06-23T08:33:06Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2429520 wikitext text/x-wiki n8layawz4gqvc1j5w2h9w6bdctbs7n6 2429521 2429520 2026-06-23T08:33:11Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) 2429521 wikitext text/x-wiki 4zytgqpuf9y1ksfkr8p7qcaav3v5dz3 2429522 2429521 2026-06-23T08:33:14Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool pc1018: Maintenance on pc8 2429522 wikitext text/x-wiki 8dlhid89o7yec59sow5016ptmhq0n52 2429523 2429522 2026-06-23T08:33:38Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool pc1018: test 2429523 wikitext text/x-wiki lly2gow73t8fmj73vxofj34977y6dvj 2429524 2429523 2026-06-23T08:33:40Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.pool (exit_code=99) pool pc1018: test 2429524 wikitext text/x-wiki ckgcvc34o9oukllh6fvzqupy80b23pt 2429525 2429524 2026-06-23T08:33:57Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool pc1018: test 2429525 wikitext text/x-wiki 5c3gup49bdp28tld8i6evb8d5lyliyp 2429526 2429525 2026-06-23T08:33:59Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2429526 wikitext text/x-wiki p1sy28sxzakgg28vscqjkb4t32ns6ld 2429527 2429526 2026-06-23T08:34:10Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) 2429527 wikitext text/x-wiki jtdjd9gmhs5vp93dovqrrgd71xqoesv 2429528 2429527 2026-06-23T08:34:12Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool pc1018: test 2429528 wikitext text/x-wiki b7j2gq0k3t1y9jeciwiik9q7ow7fixw 2429529 2429528 2026-06-23T08:37:31Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1201 (T426633)', diff saved to https://phabricator.wikimedia.org/P94354 and previous config saved to /var/cache/conftool/dbconfig/20260623-083729-fceratto.json 2429529 wikitext text/x-wiki 4adapm6g1wsxkjfc110wggfbncpi2vz 2429531 2429529 2026-06-23T08:47:39Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1201', diff saved to https://phabricator.wikimedia.org/P94355 and previous config saved to /var/cache/conftool/dbconfig/20260623-084737-fceratto.json 2429531 wikitext text/x-wiki rjlfaf3mhhgi13ee3rmnjsf5q6ew1hz 2429532 2429531 2026-06-23T08:48:17Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1262: After HW issues T428832 2429532 wikitext text/x-wiki orzrauo6x38u1qd83vwhjzbra4s28zh 2429533 2429532 2026-06-23T08:48:33Z Stashbot 7414 marostegui@cumin1003: END (ERROR) - Cookbook sre.mysql.pool (exit_code=97) pool db1262: After HW issues T428832 2429533 wikitext text/x-wiki ffa5l4hcvxhyz1qbnzwkvgwdxxc549v 2429534 2429533 2026-06-23T08:48:42Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1262: After HW issues T428832 2429534 wikitext text/x-wiki itf2cukxi8msdgrq1nlx3mgzqb0hzm6 2429535 2429534 2026-06-23T08:57:46Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1201', diff saved to https://phabricator.wikimedia.org/P94357 and previous config saved to /var/cache/conftool/dbconfig/20260623-085744-fceratto.json 2429535 wikitext text/x-wiki ah4ked9iiowflyl8q2uk32wm0usd3ys 2429537 2429535 2026-06-23T09:07:36Z Stashbot 7414 blake@cumin1003: conftool action : set/weight=10; selector: name=wikikube-worker1375.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429537 wikitext text/x-wiki qdhsp186rhjsqck74xvu4g5y6suidki 2429538 2429537 2026-06-23T09:07:55Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1201 (T426633)', diff saved to https://phabricator.wikimedia.org/P94359 and previous config saved to /var/cache/conftool/dbconfig/20260623-090752-fceratto.json 2429538 wikitext text/x-wiki afluaxgl9rs6zylbbkj6p9epnl2sfb4 2429539 2429538 2026-06-23T09:08:03Z Stashbot 7414 blake@cumin1003: conftool action : set/weight=10; selector: name=wikikube-worker1376.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429539 wikitext text/x-wiki gzlj1er23yyq473u25r3gg6yg0fvhfx 2429540 2429539 2026-06-23T09:08:10Z Stashbot 7414 blake@cumin1003: conftool action : set/weight=10; selector: name=wikikube-worker1377.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429540 wikitext text/x-wiki 9u3wdzaxmo0a9cwhhkft0m5r486git5 2429541 2429540 2026-06-23T09:08:15Z Stashbot 7414 blake@cumin1003: conftool action : set/weight=10; selector: name=wikikube-worker1378.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429541 wikitext text/x-wiki 5yf7x6wrsrg9ttl5r6z8t8m4dqbqinn 2429542 2429541 2026-06-23T09:08:21Z Stashbot 7414 blake@cumin1003: conftool action : set/weight=10; selector: name=wikikube-worker1379.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429542 wikitext text/x-wiki cq3w0wq48ssci5xr103jjprrk6sx36v 2429543 2429542 2026-06-23T09:08:27Z Stashbot 7414 blake@cumin1003: conftool action : set/weight=10; selector: name=wikikube-worker1380.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429543 wikitext text/x-wiki jodblhq4qqq10hhs9xmua4o8xjwl96v 2429544 2429543 2026-06-23T09:08:31Z Stashbot 7414 blake@cumin1003: conftool action : set/weight=10; selector: name=wikikube-worker1381.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429544 wikitext text/x-wiki ibxgz3w119efn7t55xj9ca9a9bw0xzc 2429545 2429544 2026-06-23T09:08:36Z Stashbot 7414 blake@cumin1003: conftool action : set/weight=10; selector: name=wikikube-worker1382.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429545 wikitext text/x-wiki h2vipk76uvl0xy77medefrihrx2vc3y 2429546 2429545 2026-06-23T09:08:40Z Stashbot 7414 blake@cumin1003: conftool action : set/weight=10; selector: name=wikikube-worker1383.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429546 wikitext text/x-wiki ha0ack1mnayzr7tlwcjd2dy308nbk24 2429547 2429546 2026-06-23T09:08:45Z Stashbot 7414 blake@cumin1003: conftool action : set/weight=10; selector: name=wikikube-worker1384.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429547 wikitext text/x-wiki 53ouf4svmeq7mb2ma6bgm0pojfxg6db 2429548 2429547 2026-06-23T09:13:06Z Stashbot 7414 blake@cumin1003: conftool action : set/pooled=yes; selector: name=wikikube-worker1375.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429548 wikitext text/x-wiki g7jggtj15ipfy9ql7bb7sesci6cn1la 2429549 2429548 2026-06-23T09:13:11Z Stashbot 7414 blake@cumin1003: conftool action : set/pooled=yes; selector: name=wikikube-worker1376.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429549 wikitext text/x-wiki 18sfl08b6fluf7p4xyb5im7m20mbl9h 2429550 2429549 2026-06-23T09:13:16Z Stashbot 7414 blake@cumin1003: conftool action : set/pooled=yes; selector: name=wikikube-worker1377.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429550 wikitext text/x-wiki g62p6zspyj2rntumz0vyzpub82qa1u7 2429551 2429550 2026-06-23T09:13:21Z Stashbot 7414 blake@cumin1003: conftool action : set/pooled=yes; selector: name=wikikube-worker1378.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429551 wikitext text/x-wiki eigqih1lqwv812e2rnbdsc9ww2ok1b0 2429552 2429551 2026-06-23T09:13:25Z Stashbot 7414 blake@cumin1003: conftool action : set/pooled=yes; selector: name=wikikube-worker1379.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429552 wikitext text/x-wiki 6x5iygtho2eo8is205fyet17ipqzckt 2429553 2429552 2026-06-23T09:13:29Z Stashbot 7414 blake@cumin1003: conftool action : set/pooled=yes; selector: name=wikikube-worker1380.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429553 wikitext text/x-wiki p909clx32m3ulskqg8svdns1cuei46o 2429554 2429553 2026-06-23T09:13:34Z Stashbot 7414 blake@cumin1003: conftool action : set/pooled=yes; selector: name=wikikube-worker1381.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429554 wikitext text/x-wiki am50pqd93mxrjajpi0norup0xphgar2 2429555 2429554 2026-06-23T09:13:39Z Stashbot 7414 blake@cumin1003: conftool action : set/pooled=yes; selector: name=wikikube-worker1382.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429555 wikitext text/x-wiki eyo09j5ae7miqtzx2vrudczkxb3ekr1 2429556 2429555 2026-06-23T09:13:43Z Stashbot 7414 blake@cumin1003: conftool action : set/pooled=yes; selector: name=wikikube-worker1383.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429556 wikitext text/x-wiki 0qa5cepvxvpqp0gukftocaer6agh4y7 2429557 2429556 2026-06-23T09:13:48Z Stashbot 7414 blake@cumin1003: conftool action : set/pooled=yes; selector: name=wikikube-worker1384.eqiad.wmnet,cluster=kubernetes,service=kubesvc 2429557 wikitext text/x-wiki o4ltmjlr62loa6583omj7fcgrlgfmk6 2429559 2429557 2026-06-23T09:21:31Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . 2429559 wikitext text/x-wiki d2n9y2hn3odybxe0r2k98mkq8wxus79 2429561 2429559 2026-06-23T09:34:08Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1262: After HW issues T428832 2429561 wikitext text/x-wiki 88uq5t9n0bvx0jnki74aqiozr11geqq 2429562 2429561 2026-06-23T09:41:04Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2429562 wikitext text/x-wiki chuuaaj6b1vo0qrlklymqe5wz73k25o 2429563 2429562 2026-06-23T10:14:58Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . 2429563 wikitext text/x-wiki osu4upybjr8ka3xj0ooqpwpxgpji9qu 2429564 2429563 2026-06-23T10:22:01Z Stashbot 7414 ozge@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . 2429564 wikitext text/x-wiki nmxnuvivjo971dw49mvuyrsztfgm82i 2429565 2429564 2026-06-23T10:24:03Z Stashbot 7414 ozge@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . 2429565 wikitext text/x-wiki g8osdwhicqvsw8zad6p5g2t304ut22d 2429566 2429565 2026-06-23T10:27:55Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-product: apply 2429566 wikitext text/x-wiki 9rwhpwl8gyhvb5bxwlgmb7y6amcb979 2429567 2429566 2026-06-23T10:28:34Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-product: apply 2429567 wikitext text/x-wiki 1cll5r1p2mbzw50sd4l3nzyq4ksspxl 2429569 2429567 2026-06-23T10:38:45Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1217.eqiad.wmnet with reason: Cloning db1290 2429569 wikitext text/x-wiki rnofqyta7epv7ij1elgb6q4t1fzqzr0 2429571 2429569 2026-06-23T11:15:51Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1305089|Tally: truncate BLT names by character so the result stays encodable (T427104)]], [[gerrit:1305088|Tally: truncate BLT names by character so the result stays encodable (T427104)]] 2429571 wikitext text/x-wiki ozynveh90lpfa739i817fkjj49s8ogr 2429572 2429571 2026-06-23T11:18:00Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1305089|Tally: truncate BLT names by character so the result stays encodable (T427104)]], [[gerrit:1305088|Tally: truncate BLT names by character so the result stays encodable (T427104)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429572 wikitext text/x-wiki g0sgrocwsh3jq4a2yyap4hjjx4gs2qi 2429573 2429572 2026-06-23T11:18:18Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2429573 wikitext text/x-wiki h6h914f81hpjnxsrcokr13we8o8jfzg 2429574 2429573 2026-06-23T11:22:39Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1305089|Tally: truncate BLT names by character so the result stays encodable (T427104)]], [[gerrit:1305088|Tally: truncate BLT names by character so the result stays encodable (T427104)]] (duration: 06m 48s) 2429574 wikitext text/x-wiki 65yju9n9tlax8q1l8iun17ifwzepcdj 2429575 2429574 2026-06-23T11:29:45Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1305092|hCaptcha: Log sitekeys on a sitekey-mismatch error (T429891)]] 2429575 wikitext text/x-wiki ai74t1yiizm51p5wx3loa1xf1mphr5z 2429576 2429575 2026-06-23T11:31:49Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1305092|hCaptcha: Log sitekeys on a sitekey-mismatch error (T429891)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429576 wikitext text/x-wiki qiwlydvt4apwk8r1brn46gdaj1cdmwy 2429577 2429576 2026-06-23T11:32:08Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2429577 wikitext text/x-wiki 86d23jdtb0hck06f7tt72y9asu4t96t 2429578 2429577 2026-06-23T11:36:22Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1305092|hCaptcha: Log sitekeys on a sitekey-mismatch error (T429891)]] (duration: 06m 37s) 2429578 wikitext text/x-wiki 6ltl8mg3pcxx0vybmroevmi9d64vpoh 2429579 2429578 2026-06-23T11:45:11Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1305096|hCaptcha: Log sitekeys on a sitekey-mismatch error (T429891)]] 2429579 wikitext text/x-wiki bdyd54p3fqbkywpr9dvq3fd6a3bhvka 2429580 2429579 2026-06-23T11:47:15Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1305096|hCaptcha: Log sitekeys on a sitekey-mismatch error (T429891)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2429580 wikitext text/x-wiki 42focfr69slcjrbavfog3qv6u9ah61p 2429581 2429580 2026-06-23T11:47:43Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2429581 wikitext text/x-wiki tlfl5n1muvqp428q0bu500c8hc5i737 2429582 2429581 2026-06-23T11:52:02Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1305096|hCaptcha: Log sitekeys on a sitekey-mismatch error (T429891)]] (duration: 06m 51s) 2429582 wikitext text/x-wiki nef24blbyco9yuf7z0qddjjcwxoq49g 0 17290 2429256 2428845 2026-06-22T19:30:20Z Stashbot 7414 thcipriani: thcipriani@integration-castor06:~$ sudo -u jenkins-deploy rm -rf /srv/castor/castor-mw-ext-and-skins/master/quibble-with-gated-extensions-vendor-mysql-php83 #T429824 2429256 wikitext text/x-wiki jiwc8t7io3kgt72xul33jeldqsaa9uv 2429322 2429256 2026-06-22T22:35:38Z Stashbot 7414 thcipriani: thcipriani@integration-castor06:~$ sudo -u jenkins-deploy rm -rf /srv/castor/castor-mw-ext-and-skins/master/mediawiki-node24/ #T429824 2429322 wikitext text/x-wiki nvyzqhtouwv3554ivdkglgb4gqjc6ck 2429354 2429322 2026-06-22T23:07:06Z Stashbot 7414 thcipriani: thcipriani@integration-castor06:~$ sudo -u jenkins-deploy rm -rf /srv/castor/castor-mw-ext-and-skins/master/mediawiki-node24/ #T429824 (again) 2429354 wikitext text/x-wiki rlgvx1t6wyp9xfimjrffs88gdbqo4c1 2429355 2429354 2026-06-22T23:19:42Z Stashbot 7414 thcipriani: thcipriani@integration-castor06:~$ sudo -u jenkins-deploy rm -rf /srv/castor/mediawiki-core/master/mediawiki-node24/ #T429824 2429355 wikitext text/x-wiki j1x7l9wsh0u1sczhpe0mpbbim9ocjvj 498 30942 2429236 2428174 2026-06-22T18:57:14Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.nfs.migrate_service 2429236 wikitext text/x-wiki 533hmturur90j1ts7fxm5puoudmcgcx 2429237 2429236 2026-06-22T18:57:18Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.nfs.migrate_service (exit_code=99) 2429237 wikitext text/x-wiki 8x9c2achdm7rpef35st8p7zxs2jed8k 498 55577 2429191 2421009 2026-06-22T15:43:13Z Stashbot 7414 andrewbogott: rm -rf /srv/paws/files-to-remove/* for T429578 2429191 wikitext text/x-wiki 7u8iwkvb8by1k2vy50lm0tifbrfog3c 0 440900 2429110 2420622 2026-06-22T13:42:38Z AndrewBogott (WMF) 51047 /* cloudvirtXXXX */ 2429110 wikitext text/x-wiki r4s8hxbhq5u475ybgw4o5ej2p6t9vmg 2429112 2429110 2026-06-22T13:44:16Z AndrewBogott (WMF) 51047 /* cloudvirtXXXX */ 2429112 wikitext text/x-wiki g1tgsqby6cca3snl01ji4o50y4m7efo 498 441911 2429234 2424696 2026-06-22T18:52:40Z Stashbot 7414 mutante: shutting down instance sourcebot1 - T429496 2429234 wikitext text/x-wiki cyu99vrqnd8unua76tryyfs1ura5yfy 2429235 2429234 2026-06-22T18:53:22Z Stashbot 7414 mutante: deleting proxy codesearch-sourcebot.wmcloud.org - T429496 2429235 wikitext text/x-wiki k0prchoiurb24sbot44j66hw5uuonnj 2429255 2429235 2026-06-22T19:29:57Z Stashbot 7414 mutante: deleted instance codesearch9 by accident :( - creating instance codesearch10 on trixie - T429828 T429496 2429255 wikitext text/x-wiki ist5lwdqkyt3qgfhfvwbod8vjw4hxz7 2429268 2429255 2026-06-22T19:56:49Z Stashbot 7414 mutante: creating instance codesearch11 on bookworm - hound services failed to start on trixie - verify if it's distro related - T429828 T429496 2429268 wikitext text/x-wiki 3gkqqi65g1imh5ae36hfijnq452c354 2429290 2429268 2026-06-22T20:58:02Z Stashbot 7414 mutante: - re-created on trixie! oof:)) should be working - https://codesearch.wmcloud.org/search/ T429828 2429290 wikitext text/x-wiki 61jbih3yngr3fwvh48vzk0ukfo6trl5 2429302 2429290 2026-06-22T21:30:22Z Stashbot 7414 mutante: - upgraded to Debian trixie 2429302 wikitext text/x-wiki qzoenio7ccv1jtjtoz2hhx4isb53y20 0 442343 2429222 2345061 2026-06-22T16:50:10Z Nintendofan885 17301 combine comments for Cloud Services domains 2429222 wikitext text/x-wiki jvyzfv5t86mvue9tpc31sa8o2uz9h56 0 444148 2429570 2422310 2026-06-23T11:12:18Z ISarantopoulos-WMF 34784 Remove duplicate row about ml-serve1014.eqiad.wmnet 2429570 wikitext text/x-wiki g8848gm89c0drki8bbgo1xhdy1621zp 0 445676 2429107 2428466 2026-06-22T13:35:34Z SLyngshede-WMF 31501 Opt-in MFA 2429107 wikitext text/x-wiki j7tqshnws6v9x1xckomizyg6p5kr45v 498 447034 2429227 2428695 2026-06-22T18:19:13Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed 934c610ff6 (l10n updates: ps) 2429227 wikitext text/x-wiki odmic5aaqjddh0ogwn43dlffaqexbg5 0 448996 2429267 2428641 2026-06-22T19:54:47Z LSandergreen-WMF 46345 /* Accounts & Privileges to get */ 2429267 wikitext text/x-wiki e6t91mrbrafyuzusqw0d9lni0cmi0tk 0 449027 2429258 2368096 2026-06-22T19:34:56Z WFan (WMF) 38087 2429258 wikitext text/x-wiki lc5cqpnpvnkd08mu9k9ee45bklbm1md 2429261 2429258 2026-06-22T19:39:27Z WFan (WMF) 38087 2429261 wikitext text/x-wiki qvba3i3zjr7898cqjmiovptn38qt1ia 2429262 2429261 2026-06-22T19:43:43Z WFan (WMF) 38087 /* URL Structure */ 2429262 wikitext text/x-wiki 7251sqv7mf2qfjttlgb66o95pgsmehy 2429271 2429262 2026-06-22T20:06:49Z WFan (WMF) 38087 2429271 wikitext text/x-wiki 0ls8ottz6wz750hzbprboocehbnnn0g 2429273 2429271 2026-06-22T20:10:20Z WFan (WMF) 38087 /* Email Update Flows before and after */ 2429273 wikitext text/x-wiki gfwgfkf9xcgvb1ztkrdkbfv255amiw9 2429285 2429273 2026-06-22T20:33:04Z WFan (WMF) 38087 2429285 wikitext text/x-wiki tewt8lfymtlx5yv6rxn52peu7m4ykz5 0 449160 2429389 2428933 2026-06-23T00:00:44Z Dexbot 30554 Bot: Updating the report 2429389 wikitext text/x-wiki 43hej5x66kwx1u4avm33nhw47hfl3os 2429436 2429389 2026-06-23T05:04:28Z Dexbot 30554 Bot: Updating the report 2429436 wikitext text/x-wiki pk1nxicxoxg78qnbi6uqvt7uza1o0az 2429444 2429436 2026-06-23T05:13:30Z Dexbot 30554 Bot: Updating the report 2429444 wikitext text/x-wiki bbxk5e5t1obq7q7s90f71j39laig7fi 2429464 2429444 2026-06-23T06:04:38Z Dexbot 30554 Bot: Updating the report 2429464 wikitext text/x-wiki lnn3as5d14i4gwlhxglw15ih2gsorve 2429518 2429464 2026-06-23T08:32:02Z Dexbot 30554 Bot: Updating the report 2429518 wikitext text/x-wiki 1bkckv6l1aji1c2bltrrqkb3ikrani6 0 450435 2429160 2428799 2026-06-22T14:57:59Z BJensen-WMF 48942 /* Prepare the clusters for the new service */ Caveat that PSS are only required in a namespace if it's hosting MediaWiki services. 2429160 wikitext text/x-wiki 3zvxwanj5o3duuqumoybz8usuwjpzig 2429161 2429160 2026-06-22T15:00:27Z BJensen-WMF 48942 /* Add a Kubernetes namespace */ reformat note 2429161 wikitext text/x-wiki 49blg1zhd6qzl9t91mk2tu8zvg3i2m9 2429164 2429161 2026-06-22T15:03:28Z BJensen-WMF 48942 /* Add a Kubernetes namespace */ formatting 2429164 wikitext text/x-wiki 512jgou6ogawdk0fgdeipeanb4jehag 2429190 2429164 2026-06-22T15:43:01Z BJensen-WMF 48942 /* Add a Kubernetes namespace */ formatting 2429190 wikitext text/x-wiki ge08ewf2dltidxuqz6sdl0kiam0gyjj 0 451748 2429301 2361420 2026-06-22T21:24:41Z RLazarus (WMF) 15215 /* Publication */ edit for clarity 2429301 wikitext text/x-wiki 1ltubmyiew6lauvxu7unqox4hhgdfw6 116 453906 2429212 2428907 2026-06-22T16:45:29Z Gitlabaccountapprovalbot 37332 @lucasbelo was approved. 2429212 wikitext text/x-wiki klox38p99s8gjcguihp1sas7kysqp4j 2429269 2429212 2026-06-22T19:57:23Z Gitlabaccountapprovalbot 37332 @gouvernathor was approved. 2429269 wikitext text/x-wiki hixt1mll6n0d1l4mpohd5gpch02pesr 0 455989 2429136 2427744 2026-06-22T14:19:36Z MPopov (WMF) 5891 2429136 wikitext text/x-wiki 8fcnbx7nlq0ayw3fpl4vvhfhh4dk447 0 459321 2429321 2384991 2026-06-22T22:34:44Z SFrench-WMF 37362 Update link to charlie (moved to gitlab) 2429321 wikitext text/x-wiki h60eqvi90wijlfo56hw1sjq7di9plg1 0 459699 2429204 2428401 2026-06-22T16:35:34Z Stashbot 7414 ejegg: payments-wiki upgraded from 873882a5 to d5f4b8aa 2429204 wikitext text/x-wiki s3ajg6vj9h5mmw6a9sm7xknwxtue7rz 2429280 2429204 2026-06-22T20:27:38Z Stashbot 7414 ejegg: donorwiki upgraded from 1f5f40f9 to 71cba440 2429280 wikitext text/x-wiki b3p0hzioeji22awx2fhb2fok01r1fm2 2429316 2429280 2026-06-22T22:15:30Z Stashbot 7414 eileen: civicrm upgraded from 602742db to 7130d7ff 2429316 wikitext text/x-wiki 7qr5bcl5v9n3hzqgzrfahrrfyzvr6zt 2429320 2429316 2026-06-22T22:32:04Z Stashbot 7414 eileen: config revision changed from c687d8f0 to 92ac127c 2429320 wikitext text/x-wiki do2e7fsylwf11gsmq3dhnk9oqygrq9k 2429339 2429320 2026-06-22T22:53:09Z Stashbot 7414 eileen: SmashPig upgraded from 9cd51fd1 to 38ffd696 2429339 wikitext text/x-wiki pywrx6riuysoxhhgpc7sa4ed3b5vr6x 2429560 2429339 2026-06-23T09:29:59Z Stashbot 7414 jgleeson: payments-wiki upgraded from 71cba440 to ab4fd16b 2429560 wikitext text/x-wiki o5yr7xpmjkwwwat9rwzo30u5z4oz2r3 0 460231 2429293 2428699 2026-06-22T21:08:15Z BKing (WMF) 29017 2429293 wikitext text/x-wiki a49ybgm666mvzj34fqq8fdnjbitxe5y 2429295 2429293 2026-06-22T21:12:00Z BKing (WMF) 29017 /* Cluster Quorum Issues */ 2429295 wikitext text/x-wiki jwv5j99vqe96r7s5ddb5xx1p6nns6mz 2429296 2429295 2026-06-22T21:15:46Z BKing (WMF) 29017 /* Extra Precautions for "perma-yellow" */ 2429296 wikitext text/x-wiki tal2ier6266doza7op14yb25el498gt 0 460232 2429138 2428685 2026-06-22T14:20:19Z MPopov (WMF) 5891 /* Tags */ 2429138 wikitext text/x-wiki 136k2q3709gs4149lhmzks9q4aicf04 2429152 2429138 2026-06-22T14:53:08Z MPopov (WMF) 5891 2429152 wikitext text/x-wiki r60ik19r20yx2oj9gavrj5ivjh7hwfo 0 460234 2429142 2428846 2026-06-22T14:20:48Z MPopov (WMF) 5891 /* Specify experiment details */ 2429142 wikitext text/x-wiki by23srhfjeg3hyofwadowg069nfaopd 2429153 2429142 2026-06-22T14:53:18Z MPopov (WMF) 5891 2429153 wikitext text/x-wiki am5tg2o3xxmevm8a4m43xhkw18vqz0l 2429242 2429153 2026-06-22T19:11:24Z KReid-WMF 49588 /* Specify experiment details */ Finish sentence on experiment keys and add note on essential-work tag 2429242 wikitext text/x-wiki pfek38ybaz96rwh3irzyoeq3y3rywhv 2429247 2429242 2026-06-22T19:19:48Z KReid-WMF 49588 /* Start the experiment */ Remove extra quote 2429247 wikitext text/x-wiki 9g2hxvrniiv1c7q02ghyncdp0lss14u 0 460366 2429189 2026-06-22T15:42:28Z AWesterinen-WMF 54189 Created initial page 2429189 wikitext text/x-wiki ng4xq0q23hvpw7uivhrphitd5xoide3 2429194 2429189 2026-06-22T15:54:35Z AWesterinen-WMF 54189 2429194 wikitext text/x-wiki 5wvu8uh1we8otfcbj1b5742jaulra8q 2429195 2429194 2026-06-22T15:55:38Z AWesterinen-WMF 54189 /* Reachability and Depth */ 2429195 wikitext text/x-wiki j5rqe5lbe13x9s9zl2az3soetopd7nd 2429196 2429195 2026-06-22T15:56:09Z AWesterinen-WMF 54189 /* Breadth First Search */ 2429196 wikitext text/x-wiki jlgw22vff0dclsd4hg7fkxl6dec5m1o 2429197 2429196 2026-06-22T16:15:57Z AWesterinen-WMF 54189 2429197 wikitext text/x-wiki kg4bclyaz99uh8235j39ku54aluvzj4 2429201 2429197 2026-06-22T16:31:25Z AWesterinen-WMF 54189 2429201 wikitext text/x-wiki qg46b41mtojcnfx9ands2nvq7tdgdl3 2429203 2429201 2026-06-22T16:33:04Z AWesterinen-WMF 54189 2429203 wikitext text/x-wiki n78wu8anegn1dqjjizh1lurc42wiw85 2429207 2429203 2026-06-22T16:39:07Z AWesterinen-WMF 54189 2429207 wikitext text/x-wiki i841win4n6k56r0oddet2620omspzme 2429223 2429207 2026-06-22T17:22:05Z AWesterinen-WMF 54189 2429223 wikitext text/x-wiki o94oelkqz8lrnzdbu1svhs29jrt6f5d 2429224 2429223 2026-06-22T17:57:07Z AWesterinen-WMF 54189 2429224 wikitext text/x-wiki o0wzel6x2duk3ghd1lg5s13ora7sgld 2429225 2429224 2026-06-22T18:09:15Z AWesterinen-WMF 54189 2429225 wikitext text/x-wiki 1oecm46zbymdkrktc1l5p50tuhrtmuh 2429226 2429225 2026-06-22T18:12:28Z AWesterinen-WMF 54189 2429226 wikitext text/x-wiki hszp5mw1s0cjtlrupjxile6zodns3ah 2429228 2429226 2026-06-22T18:19:28Z AWesterinen-WMF 54189 2429228 wikitext text/x-wiki prn3ufmfb2kvntc9qnd1i0xnahk65z3 2429229 2429228 2026-06-22T18:20:02Z AWesterinen-WMF 54189 2429229 wikitext text/x-wiki lap6wqz31zxijfhegbkwtlmjmfgwsuy 2429230 2429229 2026-06-22T18:28:05Z AWesterinen-WMF 54189 2429230 wikitext text/x-wiki nleglr8vuhoyl0tzoh50ifziejdabv0 2429231 2429230 2026-06-22T18:33:20Z AWesterinen-WMF 54189 2429231 wikitext text/x-wiki 7aejth0uaouhj4whaunn2n04sfm2bbq 2429232 2429231 2026-06-22T18:43:38Z AWesterinen-WMF 54189 2429232 wikitext text/x-wiki 1ww8ph86a55ioxcvrom27bwh51f3way 2429238 2429232 2026-06-22T19:04:26Z AWesterinen-WMF 54189 2429238 wikitext text/x-wiki k81pec2apsvjqe3vufib0q7c83jn8w4 2429246 2429238 2026-06-22T19:16:34Z AWesterinen-WMF 54189 2429246 wikitext text/x-wiki qmhb8r9voo1t13iosakw075kdjtqsp1 2429259 2429246 2026-06-22T19:36:57Z AWesterinen-WMF 54189 AWesterinen-WMF moved page [[Blazegraph Rewrite of GAS]] to [[Blazegraph Migration: Rewrite of GAS]]: Title unclear 2429246 wikitext text/x-wiki qmhb8r9voo1t13iosakw075kdjtqsp1 6 460367 2429250 2026-06-22T19:27:19Z AWesterinen-WMF 54189 {{File information |Description= |Source= |Date= |Author= |other_versions= }} 2429250 wikitext text/x-wiki s03jfytztn8v9r14jjn87mb6rqrtgs4 6 460368 2429252 2026-06-22T19:27:52Z AWesterinen-WMF 54189 {{File information |Description= |Source= |Date= |Author= |other_versions= }} 2429252 wikitext text/x-wiki s03jfytztn8v9r14jjn87mb6rqrtgs4 0 460369 2429260 2026-06-22T19:36:57Z AWesterinen-WMF 54189 AWesterinen-WMF moved page [[Blazegraph Rewrite of GAS]] to [[Blazegraph Migration: Rewrite of GAS]]: Title unclear 2429260 wikitext text/x-wiki mmefxyhqw075rx0ypsz7ui9oc9dgvtt 0 460370 2429278 2026-06-22T20:25:45Z WFan (WMF) 38087 Created page with "=== Overview === The donor portal is a standalone wiki that uses civiproxy to talk to civicrm where can manage donor's donations. We use vue.js for this extension. Note that the links below include checksums that will have expired by the time you read this page. To get an updated checksum for a contact, just browse to their contact summary page in your local Civi instance and look for the 'Donor Portal' link on the right side just under the '''Donor Links''' block. ==..." 2429278 wikitext text/x-wiki spoa8m5ltoykrj4rgcd39sfaiuj6iop 0 460371 2429284 2026-06-22T20:31:14Z WFan (WMF) 38087 Created page with "== Overview == The email preference center is a standalone wiki that uses civiproxy to talk to civicrm. Note that the links below include checksums that will have expired by the time you read this page. To get an updated checksum for a contact, just browse to their contact summary page in your local Civi instance and look for the 'email preference center' link on the right side just under the '''Donor Links''' block. File:Email_Prefs_CiviProxy_1.png|alt=quality art|f..." 2429284 wikitext text/x-wiki 8l5wk2dsfn7a88assukbyr8qin9whwm 0 460372 2429289 2026-06-22T20:47:20Z MPopov (WMF) 5891 Started drafting guide 2429289 wikitext text/x-wiki nfe46x8p4gvz86oyhb2jouckj2wa5o7 2429300 2429289 2026-06-22T21:22:52Z MPopov (WMF) 5891 2429300 wikitext text/x-wiki 6iyzvfs152mr4j17kh05dthcmwnpz0z 2429309 2429300 2026-06-22T21:55:27Z MPopov (WMF) 5891 2429309 wikitext text/x-wiki de0q4pwwq4cmkpm5pwj1plxfef3jo19 2429312 2429309 2026-06-22T22:07:52Z MPopov (WMF) 5891 /* How FY25/26 WE 1.8.3 overcame the challenges */ 2429312 wikitext text/x-wiki tqj0tucq2bsz9c4g5kiq976ogqoxc6k