Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.8 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 3086 2431399 2430798 2026-06-29T12:57:15Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2431399 wikitext text/x-wiki ivz2cov0sfz9d9pc2gwkstwbkhex6pt 2431408 2431399 2026-06-29T13:10:30Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2431408 wikitext text/x-wiki oeovi67cj30uxfhofsuiv1g6uux0hs8 2431421 2431408 2026-06-29T13:30:11Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2431421 wikitext text/x-wiki b9ut5orhb979hqmrflqp5304l8rlitf 2431426 2431421 2026-06-29T13:43:12Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2431426 wikitext text/x-wiki 93klk8p2mkqmhnwhu5afrmj4r6ado5p 2431445 2431426 2026-06-29T14:11:28Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api 2431445 wikitext text/x-wiki 7pxw6t9v3ohh3g01q4245s05x2o0cbb 2431454 2431445 2026-06-29T14:16:47Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api 2431454 wikitext text/x-wiki lld4lpl99q898uoh632ho8pa6c4dbuc 2431470 2431454 2026-06-29T15:17:18Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2431470 wikitext text/x-wiki 8ngl12lkykphzmpejlj8x1n8xgvtk5y 2431478 2431470 2026-06-29T15:31:18Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2431478 wikitext text/x-wiki pysvhgbl6wxsiwjsb9b2lchly5ltlxn 2431488 2431478 2026-06-29T15:44:02Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-cli 2431488 wikitext text/x-wiki mn25o27btqqrepo2ebqbhud5y2mjnjy 2431489 2431488 2026-06-29T15:46:14Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-cli 2431489 wikitext text/x-wiki kxri4e55fhq1584miuyvvkf01eo81dm 2431799 2431489 2026-06-30T09:20:55Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2431799 wikitext text/x-wiki n2eruqkjsbzghb947a7ix16huwud3id 2431804 2431799 2026-06-30T09:33:23Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2431804 wikitext text/x-wiki 78wawn5ow355w9kdbup2kwjbqtskjap 0 4108 2431452 2431320 2026-06-29T14:14:27Z ScheduleDeploymentBot 37566 Add [[gerrit:1306304]] to Wednesday, July 01 UTC morning backport window 2431452 wikitext text/x-wiki f9g3pwxs7j5bxblxc98cjz84u11frf0 2431455 2431452 2026-06-29T14:17:32Z ScheduleDeploymentBot 37566 Add [[gerrit:1306298]] to Monday, June 29 UTC late backport window 2431455 wikitext text/x-wiki i5xgfln46jkh8albp3tl4iygbaief1o 2431468 2431455 2026-06-29T15:15:35Z ScheduleDeploymentBot 37566 Add [[gerrit:1306341]] to Monday, June 29 UTC late backport window 2431468 wikitext text/x-wiki eb0ljdowgegqggvd5wahm8a2za7p75z 2431473 2431468 2026-06-29T15:19:55Z MGrosse-WMF 13323 moved my backport to tomorrow UTC afternoon 2431473 wikitext text/x-wiki bgk4e8storh1z6qcbw2ewe16gwa2w3c 2431474 2431473 2026-06-29T15:20:12Z ScheduleDeploymentBot 37566 Add [[gerrit:1306342]] to Tuesday, June 30 UTC afternoon backport window 2431474 wikitext text/x-wiki pcmbwmrsrj8c3p5hb9tvjn4kcf6adcm 2431503 2431474 2026-06-29T16:02:37Z ScheduleDeploymentBot 37566 Add [[gerrit:1305724]] to Monday, June 29 UTC late backport window 2431503 wikitext text/x-wiki exe07i3pgbjlhe40lqthmax581zibdc 2431514 2431503 2026-06-29T16:36:39Z ScheduleDeploymentBot 37566 Add [[gerrit:1306358]] to Monday, June 29 UTC late backport window 2431514 wikitext text/x-wiki 0xmnayg1optvh6xkobmon2wczi4dkos 2431515 2431514 2026-06-29T16:41:36Z ScheduleDeploymentBot 37566 Add [[gerrit:1306361]] to Monday, June 29 UTC late backport window 2431515 wikitext text/x-wiki a24k0y88y9tpqc67k3ni1bdikxqjevg 2431583 2431515 2026-06-29T19:40:38Z ScheduleDeploymentBot 37566 Add [[gerrit:1306401]] to Monday, June 29 UTC late backport window 2431583 wikitext text/x-wiki piy41zbu8r3l2bvjk3cpxpl0y513ai8 2431753 2431583 2026-06-30T07:41:32Z Msz2001 16141 move deployment 2431753 wikitext text/x-wiki ceca9l5wagavlqvq8zd2o15d9hp8l5d 2431766 2431753 2026-06-30T08:31:26Z ScheduleDeploymentBot 37566 Add [[gerrit:1306575]] to Tuesday, June 30 UTC afternoon backport window 2431766 wikitext text/x-wiki h1ag7e4rd3gf6py5fs6ot746la7ps4d 2431816 2431766 2026-06-30T09:57:58Z SFaci-WMF 35637 /* {{Deployment_day|date=2026-06-30}} */ 2431816 wikitext text/x-wiki 9v95m814b6fjhosj0gzo1f81ufd9u6c 498 6726 2431393 2430767 2026-06-29T12:44:11Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2431393 wikitext text/x-wiki gzt9iu8m5blq6s1pu0ro1mjdtq4617o 2431398 2431393 2026-06-29T12:56:15Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2431398 wikitext text/x-wiki a1l2iwp60urzthqyyzskwct30cvqtgi 2431415 2431398 2026-06-29T13:17:49Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2431415 wikitext text/x-wiki ofibq2mk8nmgok6gjkmjhyd1v15l3xn 2431420 2431415 2026-06-29T13:29:15Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2431420 wikitext text/x-wiki d0v5ywc34fhcba320hi1qusazzyk1s5 2431428 2431420 2026-06-29T13:49:23Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api 2431428 wikitext text/x-wiki 39jmw3vqtrx07p4dnlw9199pfn3g698 2431431 2431428 2026-06-29T13:52:42Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api 2431431 wikitext text/x-wiki nf1ykw5x0cmo0ikdrihh1nm45zkg250 2431436 2431431 2026-06-29T14:04:46Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api 2431436 wikitext text/x-wiki imsjgxdg7jgwgunuykhah9k9c7bj7o4 2431443 2431436 2026-06-29T14:08:52Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api 2431443 wikitext text/x-wiki 79nax3dwux29saeejl71lfowpxy20si 2431464 2431443 2026-06-29T15:03:58Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2431464 wikitext text/x-wiki 409l1frj4va1n8wjx8tqt2b6ikfnq6v 2431469 2431464 2026-06-29T15:16:14Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2431469 wikitext text/x-wiki 1izgituceupqf9xmfutre4dpr101fl0 2431485 2431469 2026-06-29T15:41:37Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-cli 2431485 wikitext text/x-wiki rwfzp2p72p5x400adwbcczgl6trf2k9 2431487 2431485 2026-06-29T15:43:26Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-cli 2431487 wikitext text/x-wiki qhf1ehjgml6d1ri4gwdh5rbg5znlm6b 2431775 2431487 2026-06-30T08:43:13Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2431775 wikitext text/x-wiki ghdo6bi0dh2l6kv26sce0lkvjw76cxm 2431779 2431775 2026-06-30T08:55:14Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2431779 wikitext text/x-wiki 4h30inl9ougd8z7rg4he22op11rix51 0 7919 2431360 2431358 2026-06-29T12:00:58Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-main: apply 2431360 wikitext text/x-wiki 9mg4inur7dwv3872jhbhpkh27jhwugi 2431361 2431360 2026-06-29T12:01:30Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-main: apply 2431361 wikitext text/x-wiki 8rbqug9dojo9cicj38c5br8sd2xpnbs 2431362 2431361 2026-06-29T12:04:48Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1252.eqiad.wmnet with reason: host reimage 2431362 wikitext text/x-wiki 43ioe1vuwf8kpocrnf9s6y3q0yn2n84 2431363 2431362 2026-06-29T12:07:40Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . 2431363 wikitext text/x-wiki feggx713bw52rywtayf1gjctfwt8z2l 2431364 2431363 2026-06-29T12:08:10Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-ml: apply 2431364 wikitext text/x-wiki odkrw44q2r0w97q12wht085g9shte84 2431365 2431364 2026-06-29T12:08:46Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-ml: apply 2431365 wikitext text/x-wiki qxqph8u44bzn7rmxyv4acjhyqwkuy8x 2431366 2431365 2026-06-29T12:09:59Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-platform-eng: apply 2431366 wikitext text/x-wiki 7ct1uy5hai1l0ha706z6ec9igpinlqc 2431367 2431366 2026-06-29T12:10:23Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-platform-eng: apply 2431367 wikitext text/x-wiki gtqz72a96urxtxz7fcka4z49v3nl0dr 2431368 2431367 2026-06-29T12:10:29Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-research: apply 2431368 wikitext text/x-wiki ps1n3uayy83epapgkbm34daxskvbavq 2431369 2431368 2026-06-29T12:11:01Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-research: apply 2431369 wikitext text/x-wiki jfcb3chhvoboovj1pe26ntyp38i5t4u 2431370 2431369 2026-06-29T12:11:41Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-search: apply 2431370 wikitext text/x-wiki cxmonxpw8y0qu77qjao6v2obeyb7zsf 2431371 2431370 2026-06-29T12:12:12Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-search: apply 2431371 wikitext text/x-wiki 0rc6malmbb938a4plp5tg9w40qa6bn9 2431372 2431371 2026-06-29T12:12:48Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-sre: apply 2431372 wikitext text/x-wiki acs9z890glnhz07mnmnwr8yt773uz5e 2431373 2431372 2026-06-29T12:13:29Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-sre: apply 2431373 wikitext text/x-wiki q06xeu8x4mpy8j2hiou471isms2kl0f 2431374 2431373 2026-06-29T12:16:23Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wikidata: apply 2431374 wikitext text/x-wiki kdwg2l8yn3mthvvd5g5ap1ebmcxbznh 2431375 2431374 2026-06-29T12:17:02Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wikidata: apply 2431375 wikitext text/x-wiki f2rycl75wdqu5yraacs5clficxcevbb 2431376 2431375 2026-06-29T12:18:34Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wmde: apply 2431376 wikitext text/x-wiki 3qw5lrn8e6rx652lup3s4s4gjy24ay5 2431377 2431376 2026-06-29T12:19:07Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wmde: apply 2431377 wikitext text/x-wiki sg2bp0lt1qb5mw3hlpm1o87xpvnnpkb 2431378 2431377 2026-06-29T12:21:50Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1252.eqiad.wmnet with OS trixie 2431378 wikitext text/x-wiki lg2la14gg7772nyhos5yu54o4c2dnse 2431379 2431378 2026-06-29T12:28:12Z Stashbot 7414 godog: add cloudvirt10[78-80] to nova -- with compute disabled - T429563 2431379 wikitext text/x-wiki 1tacii04kohuu7htapjikkndx3wnq5m 2431380 2431379 2026-06-29T12:29:26Z Stashbot 7414 dbrant@deploy1003: helmfile [staging] START helmfile.d/services/wikifeeds: apply 2431380 wikitext text/x-wiki qc49h9c52h3ecpxb4vxj423z2xoanq1 2431381 2431380 2026-06-29T12:29:49Z Stashbot 7414 dbrant@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifeeds: apply 2431381 wikitext text/x-wiki 0cksxbn1queotywtefznew1cnmvjg6q 2431382 2431381 2026-06-29T12:32:26Z Stashbot 7414 dbrant@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifeeds: apply 2431382 wikitext text/x-wiki gyuod6esdfhtb25c7azko2oskt5que6 2431383 2431382 2026-06-29T12:32:56Z Stashbot 7414 dbrant@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifeeds: apply 2431383 wikitext text/x-wiki gu3kfpvv7ebrfx6zpodlvs72ghloa6q 2431384 2431383 2026-06-29T12:33:05Z Stashbot 7414 dbrant@deploy1003: helmfile [codfw] START helmfile.d/services/wikifeeds: apply 2431384 wikitext text/x-wiki ke2re421nggepqlpwdy5a9ccq6pti92 2431385 2431384 2026-06-29T12:33:36Z Stashbot 7414 dbrant@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifeeds: apply 2431385 wikitext text/x-wiki oddla3l2kqgx471nruzl996wjuk1f3i 2431386 2431385 2026-06-29T12:34:12Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/datasets-config: apply 2431386 wikitext text/x-wiki 0zf2zta47eps5833ofhfk1ewhrt53ar 2431387 2431386 2026-06-29T12:34:25Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/datasets-config: apply 2431387 wikitext text/x-wiki 5imcvub7gpyqjffq46cw0bfmz0zh4qi 2431388 2431387 2026-06-29T12:34:39Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/spark-history: apply 2431388 wikitext text/x-wiki 44z8hyvwc7y5np6fha2px935tazsmii 2431389 2431388 2026-06-29T12:35:27Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/spark-history: apply 2431389 wikitext text/x-wiki jwgej78s125xqlgxgu0epjvbbwc0c45 2431390 2431389 2026-06-29T12:37:19Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1252: Migration of db1252.eqiad.wmnet completed 2431390 wikitext text/x-wiki 5nesnht2bepqu4zr0x59td673z2c1m7 2431391 2431390 2026-06-29T12:41:13Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2431391 wikitext text/x-wiki o2e681nmtvkqzmqr2hh2beetgfh46yw 2431392 2431391 2026-06-29T12:41:17Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2431392 wikitext text/x-wiki 66pdrdivyb320l3vf54sdvac9l7dvxw 2431394 2431392 2026-06-29T12:44:27Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook: apply 2431394 wikitext text/x-wiki 07cp13lqzx30q1qp655lm3fpcxf8s2i 2431395 2431394 2026-06-29T12:44:31Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook: apply 2431395 wikitext text/x-wiki gtzxipuksljb65kj3yftwnjhzfx62jd 2431396 2431395 2026-06-29T12:46:08Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2431396 wikitext text/x-wiki qp6gptfwmwbhh93n4ehz5q704mcq8e3 2431397 2431396 2026-06-29T12:46:12Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2431397 wikitext text/x-wiki pt9nfr1y7bvzd3tyajrpl54fxdhnt2o 2431401 2431397 2026-06-29T13:04:18Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: sync 2431401 wikitext text/x-wiki avkghvthwgsi1cxb2wwj1vloi7zvezg 2431402 2431401 2026-06-29T13:04:31Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: sync 2431402 wikitext text/x-wiki kog9idjzpszxvdf4p57vvdtwxplofwg 2431404 2431402 2026-06-29T13:05:12Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: sync 2431404 wikitext text/x-wiki r16sc0d0jbh0gr6gkihd4wp4461t6k3 2431406 2431404 2026-06-29T13:05:25Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: sync 2431406 wikitext text/x-wiki rql2h8cxbls03fl5yjni1hgf0mpco0l 2431409 2431406 2026-06-29T13:11:25Z Stashbot 7414 stran@deploy1003: Started scap sync-world: Backport for [[gerrit:1306056|User groups changes for English Wikiversity (T430416)]], [[gerrit:1305811|hrwiki: Add to wgCiteResponsiveReferences (T430182)]] 2431409 wikitext text/x-wiki l8tehgr7ifnmqc1t4u0zkg4royivc84 2431410 2431409 2026-06-29T13:13:19Z Stashbot 7414 stran@deploy1003: stran, vadymts1: Backport for [[gerrit:1306056|User groups changes for English Wikiversity (T430416)]], [[gerrit:1305811|hrwiki: Add to wgCiteResponsiveReferences (T430182)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431410 wikitext text/x-wiki jz67p8qq01aneltdvkuis2o54jaacpm 2431413 2431410 2026-06-29T13:15:56Z Stashbot 7414 stran@deploy1003: stran, vadymts1: Continuing with deployment 2431413 wikitext text/x-wiki 3ksg7fxgarx5k3u6j1e7irsun4g6e5v 2431416 2431413 2026-06-29T13:22:52Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1252: Migration of db1252.eqiad.wmnet completed 2431416 wikitext text/x-wiki 22roz0equl12cv29zih0hvpd8g9l5wt 2431417 2431416 2026-06-29T13:22:54Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2431417 wikitext text/x-wiki g8vqsiwu501wdww4recr7leoi8xey9z 2431419 2431417 2026-06-29T13:27:33Z Stashbot 7414 ozge@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . 2431419 wikitext text/x-wiki lvamzqb5ys2rt36noujac1jrpc3vvh2 2431422 2431419 2026-06-29T13:31:27Z Stashbot 7414 ozge@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . 2431422 wikitext text/x-wiki tgei97z657jewn4uf6mxxgovc6wb8oj 2431423 2431422 2026-06-29T13:31:54Z Stashbot 7414 stran@deploy1003: Scap cancelled without rolling back. 2431423 wikitext text/x-wiki j0kyo1coeg0mk0wavjbb3zk9cplkoux 2431424 2431423 2026-06-29T13:33:55Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'article-models' for release 'main' . 2431424 wikitext text/x-wiki 3jt8by8fmvb6hbd2xljitmnxo7yrga3 2431425 2431424 2026-06-29T13:39:39Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'article-models' for release 'main' . 2431425 wikitext text/x-wiki qnso457ur6qe50dbsncspcmj1j4jrpk 2431427 2431425 2026-06-29T13:49:18Z Stashbot 7414 Tran: Deployed patch for T427287 2431427 wikitext text/x-wiki myqbf5q6omh79svm7cwyewop8tiq4by 2431429 2431427 2026-06-29T13:49:51Z Stashbot 7414 ozge@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . 2431429 wikitext text/x-wiki 7pttz123q5t6lg8hfg9j8x3nxw5e77q 2431430 2431429 2026-06-29T13:51:18Z Stashbot 7414 ozge@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . 2431430 wikitext text/x-wiki mbo09fh4kptqz6j9jt6fkh5oobpiax9 2431432 2431430 2026-06-29T13:52:44Z Stashbot 7414 ozge@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . 2431432 wikitext text/x-wiki 8z93wc69zd46908hkoxyg9t1e1bnh49 2431433 2431432 2026-06-29T13:54:07Z Stashbot 7414 ozge@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . 2431433 wikitext text/x-wiki d07oqjmhlr2rc42wey227nt2tz7slwe 2431434 2431433 2026-06-29T13:55:03Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . 2431434 wikitext text/x-wiki 96htux7okavbtflb57nda6s21wm0n31 2431435 2431434 2026-06-29T13:57:09Z Stashbot 7414 ozge@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . 2431435 wikitext text/x-wiki bmyfczysvcjnlvk8dzt9dgs8hvel5a2 2431437 2431435 2026-06-29T14:06:01Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . 2431437 wikitext text/x-wiki hht6fjii48s2atd0rz11lq7wp51dkbz 2431439 2431437 2026-06-29T14:07:03Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2431439 wikitext text/x-wiki 2b31e7gqrgkp5dbjkoy4niw0eq1ujzp 2431440 2431439 2026-06-29T14:07:07Z Stashbot 7414 cwilliams@cumin1003: dbmaint on s4@eqiad T429893 2431440 wikitext text/x-wiki 25r66wch800ocsp697czdwr3uubwsqk 2431441 2431440 2026-06-29T14:07:23Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db1260: Upgrading db1260.eqiad.wmnet 2431441 wikitext text/x-wiki fr5nb2dz1dm4crz45qskdy4pqo3ba6l 2431442 2431441 2026-06-29T14:08:34Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1260: Upgrading db1260.eqiad.wmnet 2431442 wikitext text/x-wiki 79ows1r58udch061qvo0dyx6g98af5n 2431444 2431442 2026-06-29T14:10:59Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1306282|hCaptcha: Align the loginattempt CAPTCHA with badlogin (T428892)]] 2431444 wikitext text/x-wiki 8ewe09692ff4otr9ijge84uqr2e2knm 2431446 2431444 2026-06-29T14:11:44Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' . 2431446 wikitext text/x-wiki qvsdmp507ylvhdjbg8je4wg3l05lpgv 2431448 2431446 2026-06-29T14:12:53Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1306282|hCaptcha: Align the loginattempt CAPTCHA with badlogin (T428892)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431448 wikitext text/x-wiki i5oe7c4vdbd7i27e3psblbzlql24xn3 2431451 2431448 2026-06-29T14:14:05Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2431451 wikitext text/x-wiki 13vzenb4c2ban98xti1enwap0jslj9h 2431456 2431451 2026-06-29T14:18:24Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306282|hCaptcha: Align the loginattempt CAPTCHA with badlogin (T428892)]] (duration: 07m 25s) 2431456 wikitext text/x-wiki 6b7dgs7mg45s2bnar60yqdx3smwho60 2431457 2431456 2026-06-29T14:25:25Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' . 2431457 wikitext text/x-wiki j7wxovnaik71klawtfz0gz1aj2xlegv 2431459 2431457 2026-06-29T14:40:57Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1306298|[config] Fix code in core-Permissions.php]] 2431459 wikitext text/x-wiki 05d1cgu29xio600v7vjtvdqs6ra5qeg 2431460 2431459 2026-06-29T14:42:48Z Stashbot 7414 dreamyjazz@deploy1003: vadymts1, dreamyjazz: Backport for [[gerrit:1306298|[config] Fix code in core-Permissions.php]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431460 wikitext text/x-wiki am7t33edgqdkvvzv95j1gq2mz1hdo69 2431461 2431460 2026-06-29T14:43:43Z Stashbot 7414 dreamyjazz@deploy1003: vadymts1, dreamyjazz: Continuing with deployment 2431461 wikitext text/x-wiki 13j4miic18z5ov9c8rnj1d7f0l3fzjr 2431462 2431461 2026-06-29T14:48:01Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306298|[config] Fix code in core-Permissions.php]] (duration: 07m 03s) 2431462 wikitext text/x-wiki tfzpgcjlx5q8j6vp717dslj5n7d14it 2431465 2431462 2026-06-29T15:05:02Z Stashbot 7414 bking@cumin2003: conftool action : set/pooled=false; selector: dnsdisc=search,name=codfw 2431465 wikitext text/x-wiki dnvrdwr7rv2z9bfptlec3or0uvwinvb 2431466 2431465 2026-06-29T15:14:10Z Stashbot 7414 tchin@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-logging-external: apply 2431466 wikitext text/x-wiki lda641m306suwmgc1cn50u9t319c9rq 2431467 2431466 2026-06-29T15:14:25Z Stashbot 7414 tchin@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-logging-external: apply 2431467 wikitext text/x-wiki qfy451283iruobc0lcvatorb7qbc55k 2431471 2431467 2026-06-29T15:18:39Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2074.codfw.wmnet with OS trixie 2431471 wikitext text/x-wiki tsjlw936302lyjnwxsi16w9vvdikh3m 2431472 2431471 2026-06-29T15:19:30Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db1260.eqiad.wmnet with OS trixie 2431472 wikitext text/x-wiki quksqse4hu57zy7pg3c7cdp9n3b54e0 2431475 2431472 2026-06-29T15:24:50Z Stashbot 7414 tchin@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-logging-external: apply 2431475 wikitext text/x-wiki oqk1v23kgl1e9xq582ao3eig3y01ji6 2431476 2431475 2026-06-29T15:24:55Z Stashbot 7414 tchin@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-logging-external: apply 2431476 wikitext text/x-wiki j6gvhjkfwcou42g4o70njxptw8tlksk 2431477 2431476 2026-06-29T15:30:49Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2087.codfw.wmnet with OS trixie 2431477 wikitext text/x-wiki 6tt1gox57c0guwtg5a8mnlh4yjfiwvm 2431479 2431477 2026-06-29T15:32:45Z Stashbot 7414 moritzm: installing glib2.0 security updates 2431479 wikitext text/x-wiki n7c84v18h6ij2chiol92ikvl67lxy26 2431480 2431479 2026-06-29T15:34:15Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1260.eqiad.wmnet with reason: host reimage 2431480 wikitext text/x-wiki 6pa9bpvhiy0s4o3p4971yc8qq4ebhe9 2431481 2431480 2026-06-29T15:34:42Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2074.codfw.wmnet with reason: host reimage 2431481 wikitext text/x-wiki 7o1p7oir1ozeygp99qqxb8i5odz1awa 2431483 2431481 2026-06-29T15:38:54Z Stashbot 7414 moritzm: installing zsh updates from Trixie point release 2431483 wikitext text/x-wiki 4rc03fgw0hh4aurpfsw6huvhf09uoxi 2431484 2431483 2026-06-29T15:39:31Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1260.eqiad.wmnet with reason: host reimage 2431484 wikitext text/x-wiki lwor11muogs8jimbw485ojt8rvjntpn 2431486 2431484 2026-06-29T15:43:01Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2074.codfw.wmnet with reason: host reimage 2431486 wikitext text/x-wiki ds05xg3b54qftdhuww7r9mbow5oy86y 2431490 2431486 2026-06-29T15:47:14Z Stashbot 7414 bking@cumin2003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cirrussearch2074.codfw.wmnet with OS trixie 2431490 wikitext text/x-wiki 8vbe3pa84covovvs8fvox13z70ia2v4 2431491 2431490 2026-06-29T15:49:27Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2087.codfw.wmnet with reason: host reimage 2431491 wikitext text/x-wiki dbmpvjnnia5y67wqkyfh9wy9cfkme03 2431492 2431491 2026-06-29T15:50:06Z Stashbot 7414 moritzm: installing libconfig-inifiles-perl security updates 2431492 wikitext text/x-wiki 00d1v559dyyty81nvmiu6eqic0jx4hm 2431493 2431492 2026-06-29T15:50:39Z Stashbot 7414 cgoubert@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/admin 'apply'. 2431493 wikitext text/x-wiki r6w1aikr49tax39n6vft53310bjq5qv 2431494 2431493 2026-06-29T15:51:38Z Stashbot 7414 cgoubert@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2431494 wikitext text/x-wiki otl1imhwme5mty1scs4kvnns2czxebd 2431495 2431494 2026-06-29T15:54:05Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2087.codfw.wmnet with reason: host reimage 2431495 wikitext text/x-wiki kpg7bagemi8tfsib37a1tvdip8zwrzr 2431497 2431495 2026-06-29T15:56:37Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1260.eqiad.wmnet with OS trixie 2431497 wikitext text/x-wiki 6m36lnezijl1s6w3eb550375se4umi5 2431501 2431497 2026-06-29T15:57:53Z Stashbot 7414 jdrewniak@deploy1003: Started scap sync-world: Backport for [[gerrit:1306347|Assets build - 2026-06-29 15:36:06+00:00]] 2431501 wikitext text/x-wiki 2n6mqccey1cziaqr4n5w1faamey461i 2431502 2431501 2026-06-29T15:58:47Z Stashbot 7414 jdrewniak@deploy1003: portalsbuilder, jdrewniak: Backport for [[gerrit:1306347|Assets build - 2026-06-29 15:36:06+00:00]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431502 wikitext text/x-wiki 6l4zjm8rb9kxzgd9yfsn4436fp8odx5 2431505 2431502 2026-06-29T16:04:58Z Stashbot 7414 jdrewniak@deploy1003: portalsbuilder, jdrewniak: Continuing with deployment 2431505 wikitext text/x-wiki q0teivk2t0fssa0mj7hnnubxexv52ie 2431509 2431505 2026-06-29T16:06:31Z Stashbot 7414 jdrewniak@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306347|Assets build - 2026-06-29 15:36:06+00:00]] (duration: 08m 37s) 2431509 wikitext text/x-wiki 0pehie6ndatvgvzoy6qop3cl4fjb8i7 2431511 2431509 2026-06-29T16:24:40Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1260: Migration of db1260.eqiad.wmnet completed 2431511 wikitext text/x-wiki 8x872m4e9acuy4vnqyza6xikwcnors6 2431516 2431511 2026-06-29T17:03:12Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2088.codfw.wmnet with OS trixie 2431516 wikitext text/x-wiki 2rxl0xfrt5zin09ca1lbs2y60gf0sfd 2431517 2431516 2026-06-29T17:05:39Z Stashbot 7414 bking@cumin2003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cirrussearch2087.codfw.wmnet with OS trixie 2431517 wikitext text/x-wiki 06c0rlherojwr5jz8922slmjswtoadb 2431518 2431517 2026-06-29T17:10:11Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1260: Migration of db1260.eqiad.wmnet completed 2431518 wikitext text/x-wiki n9flumtkbcvm13vdbxehgcchcizb0ge 2431519 2431518 2026-06-29T17:10:14Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2431519 wikitext text/x-wiki 3z0jvco4fnqcel7eztlfljf8nqvu0na 2431520 2431519 2026-06-29T17:22:59Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2088.codfw.wmnet with reason: host reimage 2431520 wikitext text/x-wiki jkcirep41eju5zrq03azus5ytpmfist 2431521 2431520 2026-06-29T17:28:48Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2088.codfw.wmnet with reason: host reimage 2431521 wikitext text/x-wiki ly34hdweph4a2bnx1ljhv51e0s7xjet 2431522 2431521 2026-06-29T17:30:17Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2062.codfw.wmnet with OS trixie 2431522 wikitext text/x-wiki tj1ny5e40ho5rtmf4l66tohtjba9kqu 2431523 2431522 2026-06-29T17:48:25Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2062.codfw.wmnet with reason: host reimage 2431523 wikitext text/x-wiki rqxg127xko1128neg9t8m08xw3xdf6z 2431524 2431523 2026-06-29T17:52:00Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2088.codfw.wmnet with OS trixie 2431524 wikitext text/x-wiki nhj40w6d1u7t64wpnmwhyftxdmirhqk 2431525 2431524 2026-06-29T17:55:03Z Stashbot 7414 tchin@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-logging-external: apply 2431525 wikitext text/x-wiki hsrknintk4zoe8okcn3o744n2o63nyx 2431526 2431525 2026-06-29T17:55:17Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2062.codfw.wmnet with reason: host reimage 2431526 wikitext text/x-wiki pvll9ps924jay2156lz2t3xn3rxn7mg 2431527 2431526 2026-06-29T17:55:54Z Stashbot 7414 tchin@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-logging-external: apply 2431527 wikitext text/x-wiki 0ukuabb7djqzojc9kg0aay27syxubyy 2431528 2431527 2026-06-29T17:56:17Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2069.codfw.wmnet with OS trixie 2431528 wikitext text/x-wiki qbkezufc6fw5r3km8r76z4vgew4psm0 2431531 2431528 2026-06-29T18:02:53Z Stashbot 7414 tchin@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-logging-external: apply 2431531 wikitext text/x-wiki klk5nqitl4fiq9i9k6l0ka2hkwseclb 2431532 2431531 2026-06-29T18:03:37Z Stashbot 7414 tchin@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-logging-external: apply 2431532 wikitext text/x-wiki 96zft6cui5bdquwz5plc65e9w2dma1f 2431535 2431532 2026-06-29T18:15:04Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2069.codfw.wmnet with reason: host reimage 2431535 wikitext text/x-wiki 5re2pr1dprvexbczx9bmtjlyjequgf2 2431536 2431535 2026-06-29T18:17:30Z Stashbot 7414 mutante: contint1003 - maintenance reboot 2431536 wikitext text/x-wiki rgt9ikm7j57rjvepezo26b38t0o9qli 2431537 2431536 2026-06-29T18:17:38Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2062.codfw.wmnet with OS trixie 2431537 wikitext text/x-wiki sbegu8d9hd7b9wlhmypozf4lr8661ml 2431538 2431537 2026-06-29T18:20:44Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2069.codfw.wmnet with reason: host reimage 2431538 wikitext text/x-wiki eoqpfdcoouz7je7txxomcv82fewhxs1 2431541 2431538 2026-06-29T18:27:16Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 14 days, 0:00:00 on contint1003.wikimedia.org with reason: not active yet 2431541 wikitext text/x-wiki 00l9chxk3j2izwilhf6vkbj21juuwva 2431542 2431541 2026-06-29T18:27:36Z Stashbot 7414 mutante: contint2003 - maintenance reboot 2431542 wikitext text/x-wiki twbymysrfiuu80e5feiuuzgxj4l0exg 2431544 2431542 2026-06-29T18:33:59Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 14 days, 0:00:00 on contint2003.wikimedia.org with reason: not active yet 2431544 wikitext text/x-wiki ohd52cr2xqp3e4d6ig99dn2s61yo293 2431545 2431544 2026-06-29T18:39:04Z Stashbot 7414 tchin@deploy1003: helmfile [eqiad] START helmfile.d/services/eventgate-logging-external: apply 2431545 wikitext text/x-wiki 0deibgzae8ag4nrq8v65q967sy35fnl 2431546 2431545 2026-06-29T18:39:57Z Stashbot 7414 tchin@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventgate-logging-external: apply 2431546 wikitext text/x-wiki 9bf134js4jicnso1nzxfdcy66tfbklp 2431547 2431546 2026-06-29T18:42:38Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2069.codfw.wmnet with OS trixie 2431547 wikitext text/x-wiki tlbqtqixvukcz15nn2kfubogg0ke7wd 2431548 2431547 2026-06-29T18:43:49Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1305999|Remove remaining occurences of apiportalwiki (T418494)]] 2431548 wikitext text/x-wiki smqyhww0u6yxhpd4hn7ys53boix3kq6 2431549 2431548 2026-06-29T18:45:43Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1305999|Remove remaining occurences of apiportalwiki (T418494)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431549 wikitext text/x-wiki dkuvazyoz73d5kg2wmcqlplrw0y178t 2431551 2431549 2026-06-29T18:48:35Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2431551 wikitext text/x-wiki 01jmpp289yiukcf10pg7e3k2g1mmizd 2431552 2431551 2026-06-29T18:53:01Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1305999|Remove remaining occurences of apiportalwiki (T418494)]] (duration: 09m 11s) 2431552 wikitext text/x-wiki 9yki4508o1oiahyv7qcfngjmyb6i909 2431554 2431552 2026-06-29T18:55:59Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2075.codfw.wmnet with OS trixie 2431554 wikitext text/x-wiki tjynh14j6balp96a44zyyawtqb0t0yl 2431557 2431554 2026-06-29T19:11:38Z Stashbot 7414 tchin@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-analytics: apply 2431557 wikitext text/x-wiki chn3far1p0sjgm20d2cd2vqmi1co0d3 2431558 2431557 2026-06-29T19:12:06Z Stashbot 7414 tchin@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-analytics: apply 2431558 wikitext text/x-wiki bvt57epuhwojfxm6sd6evv8w99i6hue 2431559 2431558 2026-06-29T19:13:05Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2090.codfw.wmnet with OS trixie 2431559 wikitext text/x-wiki a0y6tp1r9og2sg2xosfr3zk5cckj5us 2431560 2431559 2026-06-29T19:13:32Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2075.codfw.wmnet with reason: host reimage 2431560 wikitext text/x-wiki dbxj3idjq3rggth1awa08oeyyu7llx7 2431561 2431560 2026-06-29T19:14:42Z Stashbot 7414 tchin@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-analytics: apply 2431561 wikitext text/x-wiki 41b7newzkluqqo0vkoj39fmv9hzmhup 2431562 2431561 2026-06-29T19:14:50Z Stashbot 7414 tchin@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-analytics: apply 2431562 wikitext text/x-wiki rcb1hejb87lxuq45kkh1a3x2c32x4ev 2431563 2431562 2026-06-29T19:15:04Z Stashbot 7414 tchin@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-analytics: apply 2431563 wikitext text/x-wiki filqtof8uz4517d6skj6txwvc4wqsxw 2431564 2431563 2026-06-29T19:15:54Z Stashbot 7414 tchin@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-analytics: apply 2431564 wikitext text/x-wiki 9gbqjhuq5olnhl734zlhersqpsbnl1j 2431565 2431564 2026-06-29T19:16:51Z Stashbot 7414 tchin@deploy1003: helmfile [eqiad] START helmfile.d/services/eventgate-analytics: apply 2431565 wikitext text/x-wiki e73a7byd7323k0wr6nylyv37w6wett5 2431566 2431565 2026-06-29T19:17:41Z Stashbot 7414 tchin@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventgate-analytics: apply 2431566 wikitext text/x-wiki bt410pheaw54cqra563bv4a1yi9silc 2431567 2431566 2026-06-29T19:19:08Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2075.codfw.wmnet with reason: host reimage 2431567 wikitext text/x-wiki tdsjqcwqxd5x29q8gt8wo6ye9hg8et2 2431568 2431567 2026-06-29T19:19:25Z Stashbot 7414 tchin@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-analytics-external: apply 2431568 wikitext text/x-wiki ceh9fh1e2ejdvsy734h6sszgdj116er 2431569 2431568 2026-06-29T19:19:36Z Stashbot 7414 tchin@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-analytics-external: apply 2431569 wikitext text/x-wiki lwpotkeqcre30tyj94co0135uz4mfpd 2431570 2431569 2026-06-29T19:21:18Z Stashbot 7414 tchin@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-analytics-external: apply 2431570 wikitext text/x-wiki 9mrdbsn5cmtj58svxajg7qudtykoi9u 2431571 2431570 2026-06-29T19:22:02Z Stashbot 7414 tchin@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-analytics-external: apply 2431571 wikitext text/x-wiki smbbegdtsby683rspdqwhi55uswrnw8 2431572 2431571 2026-06-29T19:27:19Z Stashbot 7414 tchin@deploy1003: helmfile [eqiad] START helmfile.d/services/eventgate-analytics-external: apply 2431572 wikitext text/x-wiki o6nzm2ibv3v7n6tt3wyjjhrjfqh5nap 2431573 2431572 2026-06-29T19:27:48Z Stashbot 7414 tchin@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventgate-analytics-external: apply 2431573 wikitext text/x-wiki 76rfvbt4jcsyxtljczjghg8dnkipbjb 2431574 2431573 2026-06-29T19:32:46Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2090.codfw.wmnet with reason: host reimage 2431574 wikitext text/x-wiki njm6xt7t87wef2qsqzhq2pdnk71gbmd 2431575 2431574 2026-06-29T19:33:43Z Stashbot 7414 tchin@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-main: apply 2431575 wikitext text/x-wiki hieckx1eb3m3cuq5v12vwgkgvr105um 2431576 2431575 2026-06-29T19:33:49Z Stashbot 7414 tchin@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-main: apply 2431576 wikitext text/x-wiki 1sigvgnxdym88o4l882k9m83mhcyizc 2431577 2431576 2026-06-29T19:35:58Z Stashbot 7414 tchin@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-main: apply 2431577 wikitext text/x-wiki r45ypl7lx66jra9iciau10bmcymkyb9 2431578 2431577 2026-06-29T19:36:15Z Stashbot 7414 tchin@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-main: apply 2431578 wikitext text/x-wiki rof33pyxu5otnekgurbxierjsa8i80k 2431579 2431578 2026-06-29T19:38:38Z Stashbot 7414 tchin@deploy1003: helmfile [eqiad] START helmfile.d/services/eventgate-main: apply 2431579 wikitext text/x-wiki h6fclreapmlmigxu7t54lcug5nx9l0t 2431580 2431579 2026-06-29T19:38:51Z Stashbot 7414 tchin@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventgate-main: apply 2431580 wikitext text/x-wiki 20sfi6kabsoicgwllwdbb84ca4unq3c 2431581 2431580 2026-06-29T19:39:04Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2075.codfw.wmnet with OS trixie 2431581 wikitext text/x-wiki lm4m88dfdm9bb7stcx6woss97m2ufo5 2431582 2431581 2026-06-29T19:39:11Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2090.codfw.wmnet with reason: host reimage 2431582 wikitext text/x-wiki 0zonjy4apwfs7bgo8tv93ranrx08ga1 2431584 2431582 2026-06-29T19:45:27Z Stashbot 7414 gmodena@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2431584 wikitext text/x-wiki qpyg6z4b4efjvewb06zmfn7zl5hduen 2431585 2431584 2026-06-29T19:49:36Z Stashbot 7414 gmodena@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2431585 wikitext text/x-wiki 56j0gf1eiyd97cpn74ub5n849b00fyc 2431586 2431585 2026-06-29T19:51:14Z Stashbot 7414 gmodena@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2431586 wikitext text/x-wiki iye0bkmdsbazkkwrv13cc6tfug1bwq5 2431587 2431586 2026-06-29T19:51:20Z Stashbot 7414 gmodena@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2431587 wikitext text/x-wiki cp0p0ae28zguemz5ornh2ezhy76qk6o 2431588 2431587 2026-06-29T19:54:08Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host restbase2039.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2431588 wikitext text/x-wiki ghmcmjh8sgugtcbwq4lhz496y9xaf1u 2431589 2431588 2026-06-29T19:54:22Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host restbase2039.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2431589 wikitext text/x-wiki jbly83yh3kpaaqk7ckdoy4q4qnpost5 2431590 2431589 2026-06-29T19:55:34Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host restbase2039.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2431590 wikitext text/x-wiki ab0x6c484t4934ze7m9naz0c3rprw64 2431591 2431590 2026-06-29T19:55:46Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host restbase2039.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2431591 wikitext text/x-wiki bnmbbsmq5omfekr5jcj3w4a2dxz6hpv 2431592 2431591 2026-06-29T19:58:39Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2090.codfw.wmnet with OS trixie 2431592 wikitext text/x-wiki b6izev62yigfb6j08bpkt0ba0tywfyg 2431593 2431592 2026-06-29T20:09:23Z Stashbot 7414 arlolra@deploy1003: Started scap sync-world: Backport for [[gerrit:1305724|Turn on Parsoid Read views for 5% of English Wikipedia desktop traffic (T430194)]], [[gerrit:1306401|Temporarily disable experimental ExtTagPFragment type (T430344 T429624)]] 2431593 wikitext text/x-wiki tgf69co0qnkfmsfyhsfbbukbpr5h6j0 2431594 2431593 2026-06-29T20:11:18Z Stashbot 7414 arlolra@deploy1003: arlolra, cscott: Backport for [[gerrit:1305724|Turn on Parsoid Read views for 5% of English Wikipedia desktop traffic (T430194)]], [[gerrit:1306401|Temporarily disable experimental ExtTagPFragment type (T430344 T429624)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431594 wikitext text/x-wiki 5hgjy55x19avlhb00nbtv2e6cw33jxd 2431595 2431594 2026-06-29T20:17:00Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host restbase2039.codfw.wmnet with OS bullseye 2431595 wikitext text/x-wiki pbqym8e53odfgaij37ujnkr83dqpiby 2431596 2431595 2026-06-29T20:18:18Z Stashbot 7414 arlolra@deploy1003: arlolra, cscott: Continuing with deployment 2431596 wikitext text/x-wiki mrl07ippj67ekk6rw8dpj5zeoss9hm2 2431597 2431596 2026-06-29T20:22:39Z Stashbot 7414 arlolra@deploy1003: Finished scap sync-world: Backport for [[gerrit:1305724|Turn on Parsoid Read views for 5% of English Wikipedia desktop traffic (T430194)]], [[gerrit:1306401|Temporarily disable experimental ExtTagPFragment type (T430344 T429624)]] (duration: 13m 16s) 2431597 wikitext text/x-wiki k3nfibn4j3a4w1evljaraetnk34e0fk 2431598 2431597 2026-06-29T20:31:44Z Stashbot 7414 kemayo@deploy1003: Started scap sync-world: Backport for [[gerrit:1306358|Add missing resolveUrlOrTitle helper function (T430450)]], [[gerrit:1306361|Add missing visualeditor-suggestion-link message to extension.json (T430450)]] 2431598 wikitext text/x-wiki 648g6h7cfr6sm2fzeylbkmn73yn7j5k 2431599 2431598 2026-06-29T20:33:34Z Stashbot 7414 kemayo@deploy1003: kemayo: Backport for [[gerrit:1306358|Add missing resolveUrlOrTitle helper function (T430450)]], [[gerrit:1306361|Add missing visualeditor-suggestion-link message to extension.json (T430450)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431599 wikitext text/x-wiki ddo2ql1dwy0bwx4sbzktq185iax480f 2431600 2431599 2026-06-29T20:35:01Z Stashbot 7414 kemayo@deploy1003: kemayo: Continuing with deployment 2431600 wikitext text/x-wiki s8gw52pwh136zwyk33fpld057vlnah4 2431601 2431600 2026-06-29T20:39:22Z Stashbot 7414 kemayo@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306358|Add missing resolveUrlOrTitle helper function (T430450)]], [[gerrit:1306361|Add missing visualeditor-suggestion-link message to extension.json (T430450)]] (duration: 07m 38s) 2431601 wikitext text/x-wiki g0g6vj9b34ih7l0h9y16o8q16alqreq 2431602 2431601 2026-06-29T20:49:24Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2091.codfw.wmnet with OS trixie 2431602 wikitext text/x-wiki pwnx0x76v8nugpsem5dg3wgidiqkpju 2431603 2431602 2026-06-29T20:53:50Z Stashbot 7414 kemayo@deploy1003: Started scap sync-world: Backport for [[gerrit:1306409|SuggestedLinkEditCheck: by default import the config of the growth task (T422730)]], [[gerrit:1306410|SuggestedLinkEditCheck: make non-experimental (T421968)]] 2431603 wikitext text/x-wiki 3pg2w640718cuttj6f9yfwci9oxfud9 2431604 2431603 2026-06-29T20:55:41Z Stashbot 7414 kemayo@deploy1003: kemayo: Backport for [[gerrit:1306409|SuggestedLinkEditCheck: by default import the config of the growth task (T422730)]], [[gerrit:1306410|SuggestedLinkEditCheck: make non-experimental (T421968)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431604 wikitext text/x-wiki enyoaaxk21rf05d7rauwetthtbgptnm 2431605 2431604 2026-06-29T20:55:44Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2073.codfw.wmnet with OS trixie 2431605 wikitext text/x-wiki jaut0sduaqjgrpz0z0238nwubvqbjfk 2431606 2431605 2026-06-29T20:59:48Z Stashbot 7414 kemayo@deploy1003: kemayo: Continuing with deployment 2431606 wikitext text/x-wiki msl9zz5e36asrn2b5uovwqqbufux6iz 2431607 2431606 2026-06-29T21:04:14Z Stashbot 7414 kemayo@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306409|SuggestedLinkEditCheck: by default import the config of the growth task (T422730)]], [[gerrit:1306410|SuggestedLinkEditCheck: make non-experimental (T421968)]] (duration: 10m 25s) 2431607 wikitext text/x-wiki 8i8bchwpcqiuljfn8ssd3531c32x4uo 2431609 2431607 2026-06-29T21:09:17Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2091.codfw.wmnet with reason: host reimage 2431609 wikitext text/x-wiki d54bz6ggd2hhbbr9lqvzzbs3qg7wb11 2431612 2431609 2026-06-29T21:12:45Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cirrussearch2089.codfw.wmnet'] 2431612 wikitext text/x-wiki 1iq5v4aah8y7cpd37ozrw4j3a97oxnd 2431613 2431612 2026-06-29T21:13:00Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2073.codfw.wmnet with reason: host reimage 2431613 wikitext text/x-wiki ia4fe7ojdrc4guign9qyw8qex86mir6 2431614 2431613 2026-06-29T21:13:51Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2091.codfw.wmnet with reason: host reimage 2431614 wikitext text/x-wiki l9fgxzsrmslmwm2j9zujifmj903z55s 2431615 2431614 2026-06-29T21:17:25Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2073.codfw.wmnet with reason: host reimage 2431615 wikitext text/x-wiki cl9elnshvm4fegfdq6tgzyhgv64b2ya 2431616 2431615 2026-06-29T21:21:26Z Stashbot 7414 maryum: Deployed security fix for T430548 2431616 wikitext text/x-wiki t4vme9xudyx39g03qfs5f1cx8hlp0k8 2431617 2431616 2026-06-29T21:23:18Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cirrussearch2089.codfw.wmnet'] 2431617 wikitext text/x-wiki 1aighl0im5emfv6tq8oivhokn016lwt 2431618 2431617 2026-06-29T21:30:01Z Stashbot 7414 bking@cumin2003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cirrussearch2091.codfw.wmnet with OS trixie 2431618 wikitext text/x-wiki oisakau6cau5pl9bmy0jwgq10xfiibt 2431621 2431618 2026-06-29T23:38:17Z Stashbot 7414 inflatador: bking@localhost raise the number of incoming shard recoveries from 4 to 7 on all search_codfw endpoints T429844 2431621 wikitext text/x-wiki cl52euzfcrauaisas7btfkd7fox8nqf 2431623 2431621 2026-06-30T00:08:38Z Stashbot 7414 ladsgroup@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2156.codfw.wmnet with reason: Maintenance 2431623 wikitext text/x-wiki 8tho2obg7x3diueceqx02h9coofxuzp 2431624 2431623 2026-06-30T00:08:47Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Depooling db2156 (T410589)', diff saved to https://phabricator.wikimedia.org/P94549 and previous config saved to /var/cache/conftool/dbconfig/20260630-000844-ladsgroup.json 2431624 wikitext text/x-wiki gdzlxah6sw4r2cewcpsjbg34jig5pd7 2431626 2431624 2026-06-30T00:16:38Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2156 (T410589)', diff saved to https://phabricator.wikimedia.org/P94550 and previous config saved to /var/cache/conftool/dbconfig/20260630-001634-ladsgroup.json 2431626 wikitext text/x-wiki 36rj9uk5io7f0eq15y538yk89s9xnjj 2431627 2431626 2026-06-30T00:26:44Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P94551 and previous config saved to /var/cache/conftool/dbconfig/20260630-002642-ladsgroup.json 2431627 wikitext text/x-wiki dvzsh5ei2t5uux67fz4s0872uj6be8u 2431628 2431627 2026-06-30T00:36:51Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P94552 and previous config saved to /var/cache/conftool/dbconfig/20260630-003650-ladsgroup.json 2431628 wikitext text/x-wiki szaahujl37o7mn3e7w1yonleq10sz1k 2431629 2431628 2026-06-30T00:47:01Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2156 (T410589)', diff saved to https://phabricator.wikimedia.org/P94553 and previous config saved to /var/cache/conftool/dbconfig/20260630-004657-ladsgroup.json 2431629 wikitext text/x-wiki j9foshppsunh2d24pbhv4wgc8b6xq76 2431630 2431629 2026-06-30T00:47:15Z Stashbot 7414 ladsgroup@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2177.codfw.wmnet with reason: Maintenance 2431630 wikitext text/x-wiki ot0ers09z3wa8j4cirdlxf4g9cfy4k5 2431631 2431630 2026-06-30T00:47:24Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Depooling db2177 (T410589)', diff saved to https://phabricator.wikimedia.org/P94554 and previous config saved to /var/cache/conftool/dbconfig/20260630-004721-ladsgroup.json 2431631 wikitext text/x-wiki l8bsryw88gy4j4q78yg9ayx5k7zh1yt 2431632 2431631 2026-06-30T00:53:26Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2063.codfw.wmnet with OS trixie 2431632 wikitext text/x-wiki nzeb50ypc1hqhcuvcxoilrfhzxyqo39 2431633 2431632 2026-06-30T00:55:21Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2177 (T410589)', diff saved to https://phabricator.wikimedia.org/P94555 and previous config saved to /var/cache/conftool/dbconfig/20260630-005517-ladsgroup.json 2431633 wikitext text/x-wiki rmwq2xbiklh1ujk85yf35zv2jgwfa5x 2431634 2431633 2026-06-30T01:05:26Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2177', diff saved to https://phabricator.wikimedia.org/P94556 and previous config saved to /var/cache/conftool/dbconfig/20260630-010525-ladsgroup.json 2431634 wikitext text/x-wiki 9wo7v42vamkfxxzigtdxsba6jeyyq38 2431635 2431634 2026-06-30T01:12:02Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2063.codfw.wmnet with reason: host reimage 2431635 wikitext text/x-wiki d12cyxcgrc8a4r4aqinpli939jb25gw 2431636 2431635 2026-06-30T01:15:34Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2177', diff saved to https://phabricator.wikimedia.org/P94557 and previous config saved to /var/cache/conftool/dbconfig/20260630-011533-ladsgroup.json 2431636 wikitext text/x-wiki 5el482j2ee1ujqjn5woqako8oql7702 2431637 2431636 2026-06-30T01:18:20Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2063.codfw.wmnet with reason: host reimage 2431637 wikitext text/x-wiki 1gypekoxlbyvuuh8dn9jjjaobrznfem 2431638 2431637 2026-06-30T01:25:44Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2177 (T410589)', diff saved to https://phabricator.wikimedia.org/P94558 and previous config saved to /var/cache/conftool/dbconfig/20260630-012540-ladsgroup.json 2431638 wikitext text/x-wiki 8fm60ivr7e77dztkcjhpp4tialfeapl 2431639 2431638 2026-06-30T01:25:59Z Stashbot 7414 ladsgroup@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2190.codfw.wmnet with reason: Maintenance 2431639 wikitext text/x-wiki 8i3ijj6r2pca781cxt5qkrxuxkqu8af 2431640 2431639 2026-06-30T01:26:09Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Depooling db2190 (T410589)', diff saved to https://phabricator.wikimedia.org/P94559 and previous config saved to /var/cache/conftool/dbconfig/20260630-012605-ladsgroup.json 2431640 wikitext text/x-wiki 99p4p09xttgt85caof5w06c71kec4kh 2431641 2431640 2026-06-30T01:33:33Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2190 (T410589)', diff saved to https://phabricator.wikimedia.org/P94560 and previous config saved to /var/cache/conftool/dbconfig/20260630-013329-ladsgroup.json 2431641 wikitext text/x-wiki ixilfa15yybg0pxz9984hgasm96ei11 2431642 2431641 2026-06-30T01:37:43Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2063.codfw.wmnet with OS trixie 2431642 wikitext text/x-wiki meokfd8b1s21g0bir3fqnu7podixr4a 2431643 2431642 2026-06-30T01:43:39Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2190', diff saved to https://phabricator.wikimedia.org/P94561 and previous config saved to /var/cache/conftool/dbconfig/20260630-014337-ladsgroup.json 2431643 wikitext text/x-wiki 5dzkegl8047f8ouwgsvx11vdq00myv2 2431644 2431643 2026-06-30T01:44:43Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.k8s.renumber-node Renumbering for host wikikube-worker1163.eqiad.wmnet 2431644 wikitext text/x-wiki kz1xssdl8y1egirjesjp0abfb4qzb7c 2431645 2431644 2026-06-30T01:44:48Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1163.eqiad.wmnet 2431645 wikitext text/x-wiki fged6omkflfbxk0i1p5reh5aa8zjgkk 2431646 2431645 2026-06-30T01:45:23Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1163.eqiad.wmnet 2431646 wikitext text/x-wiki 2tiszymoj89qglkc7c27jy5sj4bxxbs 2431647 2431646 2026-06-30T01:46:00Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker1163.eqiad.wmnet with OS trixie 2431647 wikitext text/x-wiki rxu3c83b8jvhm7ap5psoydyzggrqrwm 2431648 2431647 2026-06-30T01:46:32Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.hosts.move-vlan for host wikikube-worker1163 2431648 wikitext text/x-wiki 2gv929ytq919a5vv5yilu4bq87pmr6k 2431649 2431648 2026-06-30T01:47:16Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.dns.netbox 2431649 wikitext text/x-wiki oxdir3e0vlvboxzfe4oj4kc1cu010pa 2431650 2431649 2026-06-30T01:52:48Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1163 - jasmine@cumin2002" 2431650 wikitext text/x-wiki 2x1jsy5y1ioo0coku0abw6mqje2plp1 2431651 2431650 2026-06-30T01:52:54Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1163 - jasmine@cumin2002" 2431651 wikitext text/x-wiki bnqfx26n878ccdg08a9av8dxe9ky9sc 2431652 2431651 2026-06-30T01:52:56Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2431652 wikitext text/x-wiki 45s9skz3b47dwc007eodektkhlyfacc 2431653 2431652 2026-06-30T01:52:58Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.dns.wipe-cache wikikube-worker1163.eqiad.wmnet 55.48.64.10.in-addr.arpa 5.5.0.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2431653 wikitext text/x-wiki cj0ifmun0gsacc1p9e8e9mia1jawd7s 2431654 2431653 2026-06-30T01:53:01Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker1163.eqiad.wmnet 55.48.64.10.in-addr.arpa 5.5.0.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2431654 wikitext text/x-wiki kgqkt7n6p8dtzt8qnq8ueojhbbj6mar 2431655 2431654 2026-06-30T01:53:03Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker1163 2431655 wikitext text/x-wiki qjh8ocsfsvqofgikflsxjx9k7y7liao 2431656 2431655 2026-06-30T01:53:47Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2190', diff saved to https://phabricator.wikimedia.org/P94562 and previous config saved to /var/cache/conftool/dbconfig/20260630-015345-ladsgroup.json 2431656 wikitext text/x-wiki 1wdc6qsiojc8dr53mmpn7dvzl4uic4g 2431657 2431656 2026-06-30T01:54:56Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker1163 2431657 wikitext text/x-wiki cgutjtd36wg78zeijo90mju0chl6sl1 2431658 2431657 2026-06-30T01:54:58Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker1163 2431658 wikitext text/x-wiki t3ixgmtsuwq5lopvrdo2zlqayho76rs 2431659 2431658 2026-06-30T02:00:43Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2431659 wikitext text/x-wiki 282o553tx3y1xi1f0jjfll8hq0yw4op 2431660 2431659 2026-06-30T02:03:56Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2190 (T410589)', diff saved to https://phabricator.wikimedia.org/P94563 and previous config saved to /var/cache/conftool/dbconfig/20260630-020353-ladsgroup.json 2431660 wikitext text/x-wiki rzj4a86ieqntk11fi1ikl0uvfhoepbq 2431661 2431660 2026-06-30T02:04:10Z Stashbot 7414 ladsgroup@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2194.codfw.wmnet with reason: Maintenance 2431661 wikitext text/x-wiki dfk1bmfr3j8ydq1s9h2uq3hc4be68bg 2431662 2431661 2026-06-30T02:04:20Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Depooling db2194 (T410589)', diff saved to https://phabricator.wikimedia.org/P94564 and previous config saved to /var/cache/conftool/dbconfig/20260630-020416-ladsgroup.json 2431662 wikitext text/x-wiki ozyfytbs9wno88k3k3asy30qyw33i9j 2431663 2431662 2026-06-30T02:07:29Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 45s) 2431663 wikitext text/x-wiki jgseg4ej3yp06bbu30danw2xs1t9fq9 2431664 2431663 2026-06-30T02:09:34Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1163.eqiad.wmnet with reason: host reimage 2431664 wikitext text/x-wiki fxhvjnvwz53j0jpipc73ljp3xoed993 2431665 2431664 2026-06-30T02:11:52Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2194 (T410589)', diff saved to https://phabricator.wikimedia.org/P94565 and previous config saved to /var/cache/conftool/dbconfig/20260630-021149-ladsgroup.json 2431665 wikitext text/x-wiki b2kffbu2zwnqh7mpq6pmrhf73p34v97 2431666 2431665 2026-06-30T02:14:25Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1163.eqiad.wmnet with reason: host reimage 2431666 wikitext text/x-wiki 6mrmlq07o59cq2f8fcalj5jekkhugbr 2431667 2431666 2026-06-30T02:21:59Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2194', diff saved to https://phabricator.wikimedia.org/P94566 and previous config saved to /var/cache/conftool/dbconfig/20260630-022157-ladsgroup.json 2431667 wikitext text/x-wiki he7cmpct6nbt56rx9ljpww6jwob5jyx 2431668 2431667 2026-06-30T02:32:06Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2194', diff saved to https://phabricator.wikimedia.org/P94567 and previous config saved to /var/cache/conftool/dbconfig/20260630-023204-ladsgroup.json 2431668 wikitext text/x-wiki 71vwdmk3bif8bx0zvzyfpadt5l87d4p 2431669 2431668 2026-06-30T02:33:12Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1163.eqiad.wmnet with OS trixie 2431669 wikitext text/x-wiki 2wivprwvg295m70gvbb4wtwtjqsat42 2431670 2431669 2026-06-30T02:34:37Z Stashbot 7414 jasmine_: homer lsw1-d3-eqiad* commit 'T430226' 2431670 wikitext text/x-wiki rjfoey0mjhsztfg1kanj5xlj5rgwcfr 2431671 2431670 2026-06-30T02:42:15Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2194 (T410589)', diff saved to https://phabricator.wikimedia.org/P94568 and previous config saved to /var/cache/conftool/dbconfig/20260630-024212-ladsgroup.json 2431671 wikitext text/x-wiki s5dnc8qiujgxriqnkkp1u952tfra3nx 2431672 2431671 2026-06-30T02:42:19Z Stashbot 7414 ladsgroup@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2209.codfw.wmnet with reason: Maintenance 2431672 wikitext text/x-wiki 7xd19zuu6aslz3ptws4n5wqxo819nkb 2431673 2431672 2026-06-30T02:42:29Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Depooling db2209 (T410589)', diff saved to https://phabricator.wikimedia.org/P94569 and previous config saved to /var/cache/conftool/dbconfig/20260630-024225-ladsgroup.json 2431673 wikitext text/x-wiki 64hz1pruov8obg2xkdu0wa68kcsh253 2431674 2431673 2026-06-30T02:49:45Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2209 (T410589)', diff saved to https://phabricator.wikimedia.org/P94570 and previous config saved to /var/cache/conftool/dbconfig/20260630-024941-ladsgroup.json 2431674 wikitext text/x-wiki ptge2a6yp5xftonv1jr8qv7kh4pwsfi 2431675 2431674 2026-06-30T02:50:01Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2112.codfw.wmnet with OS trixie 2431675 wikitext text/x-wiki guh07usxc1nwddggpaeqqqc52i6a7jr 2431676 2431675 2026-06-30T02:59:50Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2209', diff saved to https://phabricator.wikimedia.org/P94571 and previous config saved to /var/cache/conftool/dbconfig/20260630-025948-ladsgroup.json 2431676 wikitext text/x-wiki dv7eoceg6wnn9llzo2rjpy285wbmg4n 2431677 2431676 2026-06-30T03:03:21Z Stashbot 7414 mwpresync@deploy1003: Started scap sync-world: testwikis to 1.47.0-wmf.9 refs T423918 2431677 wikitext text/x-wiki l6pvossdqycx5bzyr6919m43xlp7nqy 2431678 2431677 2026-06-30T03:06:35Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1163.eqiad.wmnet 2431678 wikitext text/x-wiki go1pshlccwa8nl3zk9r0ue04sbfgxif 2431679 2431678 2026-06-30T03:06:37Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1163.eqiad.wmnet 2431679 wikitext text/x-wiki j8mlbsx30940nimygffybu446k30uqz 2431680 2431679 2026-06-30T03:06:40Z Stashbot 7414 jasmine@cumin2002: END (FAIL) - Cookbook sre.k8s.renumber-node (exit_code=1) Renumbering for host wikikube-worker1163.eqiad.wmnet 2431680 wikitext text/x-wiki 0weezwmm4tuz1711wj2m1bhzpbh3q0m 2431681 2431680 2026-06-30T03:08:46Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2112.codfw.wmnet with reason: host reimage 2431681 wikitext text/x-wiki f9ohyetvikfa0k8klp8rvgqd1av5sie 2431682 2431681 2026-06-30T03:09:58Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2209', diff saved to https://phabricator.wikimedia.org/P94572 and previous config saved to /var/cache/conftool/dbconfig/20260630-030956-ladsgroup.json 2431682 wikitext text/x-wiki orsb5ex1q1iwlmwgzimgbnp50ix7jkg 2431683 2431682 2026-06-30T03:13:57Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2112.codfw.wmnet with reason: host reimage 2431683 wikitext text/x-wiki 49u0hm0kxajmko6b7s50r2u8f7aq25a 2431684 2431683 2026-06-30T03:20:07Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2209 (T410589)', diff saved to https://phabricator.wikimedia.org/P94573 and previous config saved to /var/cache/conftool/dbconfig/20260630-032004-ladsgroup.json 2431684 wikitext text/x-wiki fz1lwsvxlvgerzccjihae65zpshhkzd 2431685 2431684 2026-06-30T03:20:22Z Stashbot 7414 ladsgroup@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2227.codfw.wmnet with reason: Maintenance 2431685 wikitext text/x-wiki 8crzse34lx2azuydob6uinfrqzcp57x 2431686 2431685 2026-06-30T03:20:31Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Depooling db2227 (T410589)', diff saved to https://phabricator.wikimedia.org/P94574 and previous config saved to /var/cache/conftool/dbconfig/20260630-032028-ladsgroup.json 2431686 wikitext text/x-wiki couwf51j5fmdyb2hkke2k3tjzx88l1s 2431687 2431686 2026-06-30T03:28:05Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2227 (T410589)', diff saved to https://phabricator.wikimedia.org/P94575 and previous config saved to /var/cache/conftool/dbconfig/20260630-032802-ladsgroup.json 2431687 wikitext text/x-wiki ptcq0dh8ji6mhyxtd12sc2wt0t9pl0w 2431688 2431687 2026-06-30T03:34:34Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2112.codfw.wmnet with OS trixie 2431688 wikitext text/x-wiki jhpzbwrv7y0nrtu0quzzl6r4y6hxkcm 2431689 2431688 2026-06-30T03:38:11Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2227', diff saved to https://phabricator.wikimedia.org/P94576 and previous config saved to /var/cache/conftool/dbconfig/20260630-033809-ladsgroup.json 2431689 wikitext text/x-wiki l8d2d3rx8g7c4jmby8x36uy9upn8osb 2431690 2431689 2026-06-30T03:41:59Z Stashbot 7414 mwpresync@deploy1003: Finished scap sync-world: testwikis to 1.47.0-wmf.9 refs T423918 (duration: 38m 39s) 2431690 wikitext text/x-wiki 2a2ytaql9lakaextv70hp54p3tmustt 2431691 2431690 2026-06-30T03:48:19Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2227', diff saved to https://phabricator.wikimedia.org/P94577 and previous config saved to /var/cache/conftool/dbconfig/20260630-034818-ladsgroup.json 2431691 wikitext text/x-wiki 824o59azmku5x1iezbihkfwb6y5h3tm 2431692 2431691 2026-06-30T03:58:29Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2227 (T410589)', diff saved to https://phabricator.wikimedia.org/P94578 and previous config saved to /var/cache/conftool/dbconfig/20260630-035825-ladsgroup.json 2431692 wikitext text/x-wiki kmia7p5n5avltz8iaayqkm8gwhd6ifd 2431693 2431692 2026-06-30T03:58:42Z Stashbot 7414 ladsgroup@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2239.codfw.wmnet with reason: Maintenance 2431693 wikitext text/x-wiki 838e43zde2ymqeh3d8sa70twa8rul82 2431694 2431693 2026-06-30T04:02:45Z Stashbot 7414 mwpresync@deploy1003: Pruned MediaWiki: 1.47.0-wmf.6 (duration: 02m 36s) 2431694 wikitext text/x-wiki pkcf8y1gqc8lyzghaznjtnzetliq1sn 2431695 2431694 2026-06-30T04:48:51Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2114.codfw.wmnet with OS trixie 2431695 wikitext text/x-wiki 47xp5s9q8hrhvl4ye6hh5l86zmn2qqr 2431696 2431695 2026-06-30T05:07:18Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2114.codfw.wmnet with reason: host reimage 2431696 wikitext text/x-wiki gziyhd9mmqd0cje0nhh1iwu1ywmxjrn 2431697 2431696 2026-06-30T05:14:47Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2114.codfw.wmnet with reason: host reimage 2431697 wikitext text/x-wiki g6i6r8bd5o4fho29os01m1qlxcfdtku 2431698 2431697 2026-06-30T05:19:24Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Set db1210 with weight 0 T430540', diff saved to https://phabricator.wikimedia.org/P94579 and previous config saved to /var/cache/conftool/dbconfig/20260630-051920-marostegui.json 2431698 wikitext text/x-wiki 5ydj0vg0ez4w77sd2odxyi2xwrvblwr 2431699 2431698 2026-06-30T05:19:34Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 23 hosts with reason: Primary switchover s5 T430540 2431699 wikitext text/x-wiki j8aeckxkfhyad2zq19mx0zvyiijd4wz 2431701 2431699 2026-06-30T05:24:26Z Stashbot 7414 marostegui: Starting s5 eqiad failover from db1230 to db1210 - T430540 2431701 wikitext text/x-wiki f5q36niyz77r37sefkq60unhq083pzl 2431702 2431701 2026-06-30T05:25:26Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Set s5 eqiad as read-only for maintenance - T430540', diff saved to https://phabricator.wikimedia.org/P94580 and previous config saved to /var/cache/conftool/dbconfig/20260630-052523-marostegui.json 2431702 wikitext text/x-wiki 7cmlhuu4lojl983r745qib6shats029 2431703 2431702 2026-06-30T05:25:50Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Promote db1210 to s5 primary and set section read-write T430540', diff saved to https://phabricator.wikimedia.org/P94581 and previous config saved to /var/cache/conftool/dbconfig/20260630-052547-marostegui.json 2431703 wikitext text/x-wiki g8riy13q35cugul4627lt0etrbwz2fk 2431704 2431703 2026-06-30T05:26:27Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool db1230 T430540', diff saved to https://phabricator.wikimedia.org/P94582 and previous config saved to /var/cache/conftool/dbconfig/20260630-052624-marostegui.json 2431704 wikitext text/x-wiki 11urb6nxtywxoqpa856ra7h8b7i89eq 2431705 2431704 2026-06-30T05:26:45Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1230: Repooling after switchover 2431705 wikitext text/x-wiki 2p3ohj4nxi58begw0sbqkwb4hnb7cac 2431706 2431705 2026-06-30T05:28:11Z Stashbot 7414 marostegui@dns1004: START - running authdns-update 2431706 wikitext text/x-wiki hksrfdswyvly7k43jq2h9uvxlizy77n 2431707 2431706 2026-06-30T05:28:46Z Stashbot 7414 marostegui@cumin1003: END (ERROR) - Cookbook sre.mysql.pool (exit_code=97) pool db1230: Repooling after switchover 2431707 wikitext text/x-wiki o69rj4xe6g6g1q61fm1bhl9j9abbx5k 2431708 2431707 2026-06-30T05:28:58Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1230: Repooling after switchover 2431708 wikitext text/x-wiki a7kwbs4e92fgrl72i9mu8j9f66quki6 2431709 2431708 2026-06-30T05:30:11Z Stashbot 7414 marostegui@dns1004: END - running authdns-update 2431709 wikitext text/x-wiki 8rkxb98rqp36x6lemednqasbajpzjjw 2431710 2431709 2026-06-30T05:35:23Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2114.codfw.wmnet with OS trixie 2431710 wikitext text/x-wiki etjh6d256p9m5rt34aled4c01cuawnv 2431711 2431710 2026-06-30T05:54:23Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 24 hosts with reason: Primary switchover s3 T430610 2431711 wikitext text/x-wiki ejg2gog7hdon06xamgpwx8weg8ekdq6 2431712 2431711 2026-06-30T05:54:27Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Set db1189 with weight 0 T430610', diff saved to https://phabricator.wikimedia.org/P94585 and previous config saved to /var/cache/conftool/dbconfig/20260630-055423-marostegui.json 2431712 wikitext text/x-wiki teyylfz2bcibtuikvj3odh0knlngtcn 2431713 2431712 2026-06-30T05:55:34Z Stashbot 7414 marostegui: Starting s3 eqiad failover from db1223 to db1189 - T430610 2431713 wikitext text/x-wiki dave1jn6vjph9kmaw2qjbyqib1y48l3 2431715 2431713 2026-06-30T05:57:55Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Set s3 eqiad as read-only for maintenance - T430610', diff saved to https://phabricator.wikimedia.org/P94586 and previous config saved to /var/cache/conftool/dbconfig/20260630-055752-marostegui.json 2431715 wikitext text/x-wiki kteqs4se98vcbr6f8g1s53d9s75sdg8 2431716 2431715 2026-06-30T05:58:16Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Promote db1189 to s3 primary and set section read-write T430610', diff saved to https://phabricator.wikimedia.org/P94587 and previous config saved to /var/cache/conftool/dbconfig/20260630-055812-marostegui.json 2431716 wikitext text/x-wiki nazz2upgvvj5aese612xgx0fgtx44f5 2431717 2431716 2026-06-30T05:58:27Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2064.codfw.wmnet with OS trixie 2431717 wikitext text/x-wiki 25dbugzzrfven4g24yy8mfckw0h835s 2431718 2431717 2026-06-30T05:58:35Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2077.codfw.wmnet with OS trixie 2431718 wikitext text/x-wiki er1xe3lac8veyfspuvn5dt65hybmf6t 2431719 2431718 2026-06-30T05:58:39Z Stashbot 7414 marostegui@dns1004: START - running authdns-update 2431719 wikitext text/x-wiki 2fg5944siockbx1auvwn1xv26bqnr48 2431720 2431719 2026-06-30T05:59:10Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool db1223 T430610', diff saved to https://phabricator.wikimedia.org/P94588 and previous config saved to /var/cache/conftool/dbconfig/20260630-055906-marostegui.json 2431720 wikitext text/x-wiki jkpmelishe2led2rpke8911zkp6rw23 2431721 2431720 2026-06-30T05:59:27Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1223: Repooling after switchover 2431721 wikitext text/x-wiki 6yv6rgs7u737zk35v09k698k5hefjr9 2431722 2431721 2026-06-30T06:00:10Z Stashbot 7414 marostegui@cumin1003: END (ERROR) - Cookbook sre.mysql.pool (exit_code=97) pool db1223: Repooling after switchover 2431722 wikitext text/x-wiki 3qjfiiqrx580twugxn4diww666fydcz 2431723 2431722 2026-06-30T06:00:20Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1223: Repooling after switchover 2431723 wikitext text/x-wiki bk9ksgy57r20eiesrp5cms93b9x5wu2 2431724 2431723 2026-06-30T06:00:37Z Stashbot 7414 marostegui@dns1004: END - running authdns-update 2431724 wikitext text/x-wiki 0ppwbhpi62nkd6qi34b6dvmsbxpde04 2431725 2431724 2026-06-30T06:14:26Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1230: Repooling after switchover 2431725 wikitext text/x-wiki pc8d32k0jjp3u4cnv3huodhmemoh5bq 2431726 2431725 2026-06-30T06:15:53Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2077.codfw.wmnet with reason: host reimage 2431726 wikitext text/x-wiki c6e1vwbc04o8j156fr6xxbee6vc76vx 2431727 2431726 2026-06-30T06:16:09Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2064.codfw.wmnet with reason: host reimage 2431727 wikitext text/x-wiki qwy858xl7xmbe47s5dhnjsw79pfeg5f 2431728 2431727 2026-06-30T06:22:50Z Stashbot 7414 marostegui@dns1004: START - running authdns-update 2431728 wikitext text/x-wiki 8joak4drz13gi7fo52yw3g6qombmpty 2431729 2431728 2026-06-30T06:23:00Z Stashbot 7414 marostegui@dns1004: START - running authdns-update 2431729 wikitext text/x-wiki i9r2dv41z49whageb3ibennmi3jm94a 2431730 2431729 2026-06-30T06:25:53Z Stashbot 7414 ryankemper@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on cirrussearch2077.codfw.wmnet with reason: host reimage 2431730 wikitext text/x-wiki pby09vezun6wu0xo9d3clyyjzzix910 2431731 2431730 2026-06-30T06:26:44Z Stashbot 7414 marostegui@dns1004: END - running authdns-update 2431731 wikitext text/x-wiki hp6l3y9yy88kf14p3v2gg3a2hj07icu 2431732 2431731 2026-06-30T06:35:25Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2064.codfw.wmnet with reason: host reimage 2431732 wikitext text/x-wiki khwwjg2p72qs1jbplna4sfiwmi6lphf 2431733 2431732 2026-06-30T06:40:57Z Stashbot 7414 ryankemper@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cirrussearch2077.codfw.wmnet with OS trixie 2431733 wikitext text/x-wiki 7js1j4qqnvbb7vk10xoq1bg27um7utz 2431735 2431733 2026-06-30T06:45:48Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1223: Repooling after switchover 2431735 wikitext text/x-wiki l4zr9j42ijuen6wmxswcn2lin783nin 2431736 2431735 2026-06-30T06:56:42Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2064.codfw.wmnet with OS trixie 2431736 wikitext text/x-wiki nbotm9lvpl3l61g8ashzieo61dtjo06 2431738 2431736 2026-06-30T07:05:05Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1230.eqiad.wmnet with reason: Maintenance 2431738 wikitext text/x-wiki bqbah8fa0po4wpvpr3a01ysaubm55jt 2431739 2431738 2026-06-30T07:05:14Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1230 (T426633)', diff saved to https://phabricator.wikimedia.org/P94596 and previous config saved to /var/cache/conftool/dbconfig/20260630-070512-fceratto.json 2431739 wikitext text/x-wiki s1dkjo4j1roejsede41am3dcmjl46a7 2431741 2431739 2026-06-30T07:07:29Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2098.codfw.wmnet with OS trixie 2431741 wikitext text/x-wiki gk42gznjz2n5fk2weajapm5t2zrteay 2431742 2431741 2026-06-30T07:07:41Z Stashbot 7414 moritzm: installing libconfig-inifiles-perl security updates 2431742 wikitext text/x-wiki h3ge0k54tp4xnwligz11jlrv3pvjnz5 2431743 2431742 2026-06-30T07:12:08Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1230 (T426633)', diff saved to https://phabricator.wikimedia.org/P94597 and previous config saved to /var/cache/conftool/dbconfig/20260630-071206-fceratto.json 2431743 wikitext text/x-wiki qrx6xqvp5v4tse6xr0e9ll8vro0s1mc 2431744 2431743 2026-06-30T07:21:42Z Stashbot 7414 elukey: upgrade all bullseye hosts to pywmflib 3.0 - T430552 2431744 wikitext text/x-wiki 0zav1r26lumg8mupu4jpyrfntp4vjxr 2431745 2431744 2026-06-30T07:22:16Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1230', diff saved to https://phabricator.wikimedia.org/P94598 and previous config saved to /var/cache/conftool/dbconfig/20260630-072214-fceratto.json 2431745 wikitext text/x-wiki n5cxxg8yyvob28u3e1trexxjhejek1r 2431746 2431745 2026-06-30T07:23:24Z Stashbot 7414 moritzm: installing libgd-perl security updates 2431746 wikitext text/x-wiki ohneh6bssz1kg3fi8vfk80kaky8oqtm 2431747 2431746 2026-06-30T07:26:55Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2098.codfw.wmnet with reason: host reimage 2431747 wikitext text/x-wiki mtjw1fm9r2ftyb7aiio5exzto0hv7h9 2431748 2431747 2026-06-30T07:32:23Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1230', diff saved to https://phabricator.wikimedia.org/P94599 and previous config saved to /var/cache/conftool/dbconfig/20260630-073221-fceratto.json 2431748 wikitext text/x-wiki f0t28i46p0n89wy33sr6b3v6nve5w0d 2431749 2431748 2026-06-30T07:34:39Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2098.codfw.wmnet with reason: host reimage 2431749 wikitext text/x-wiki tcke4jmhya7g8mdxlun3h0izkj9w4fy 2431750 2431749 2026-06-30T07:37:13Z Stashbot 7414 moritzm: installing libhttp-daemon-perl security updates 2431750 wikitext text/x-wiki jf1ins6xkio61s81q9eq7rekbcute1g 2431752 2431750 2026-06-30T07:40:24Z Stashbot 7414 moritzm: installing libtext-csv-xs-perl security updates 2431752 wikitext text/x-wiki 5lc6mccxzh8y1d7d160ep0489vcm934 2431754 2431752 2026-06-30T07:42:31Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1230 (T426633)', diff saved to https://phabricator.wikimedia.org/P94600 and previous config saved to /var/cache/conftool/dbconfig/20260630-074229-fceratto.json 2431754 wikitext text/x-wiki 30xxn051ypw9byp1arf6j84gj9c555c 2431755 2431754 2026-06-30T07:45:46Z Stashbot 7414 moritzm: installing nodejs security updates 2431755 wikitext text/x-wiki 7ydotjp384mkma97nxglrqk7r9zywht 2431756 2431755 2026-06-30T07:53:08Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2099.codfw.wmnet with OS trixie 2431756 wikitext text/x-wiki 06lmc8szabnswfck4xso80248uesq8o 2431757 2431756 2026-06-30T07:57:10Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2098.codfw.wmnet with OS trixie 2431757 wikitext text/x-wiki szpa42gk9iceidh6xhulaky50jl37ay 2431758 2431757 2026-06-30T08:07:51Z Stashbot 7414 elukey: uploaded python3-wmflib_3.1.0 to apt.wikimedia.org bullseye-wikimedia,bookworm-wikimedia,trixie-wikimedia 2431758 wikitext text/x-wiki r9f0zyl7q21qlnvvknwfwv7k2cl5cjg 2431759 2431758 2026-06-30T08:08:52Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1223.eqiad.wmnet with reason: Maintenance 2431759 wikitext text/x-wiki df4lniluaxj23wd7sd3lad4m0w8n98q 2431760 2431759 2026-06-30T08:09:00Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1223 (T426633)', diff saved to https://phabricator.wikimedia.org/P94601 and previous config saved to /var/cache/conftool/dbconfig/20260630-080858-fceratto.json 2431760 wikitext text/x-wiki t6yg213l8d906aqzp2w093cz78aqg2f 2431762 2431760 2026-06-30T08:12:50Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2099.codfw.wmnet with reason: host reimage 2431762 wikitext text/x-wiki ho70vjdbjm1vyjsw6dccya1if36iqs5 2431763 2431762 2026-06-30T08:16:11Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1223 (T426633)', diff saved to https://phabricator.wikimedia.org/P94602 and previous config saved to /var/cache/conftool/dbconfig/20260630-081609-fceratto.json 2431763 wikitext text/x-wiki tb151a55izn66xyetobo2cixpjha6q9 2431764 2431763 2026-06-30T08:19:25Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2099.codfw.wmnet with reason: host reimage 2431764 wikitext text/x-wiki nn8rvmh8pw3tj4ksvxnxpg49a801t5l 2431765 2431764 2026-06-30T08:26:18Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1223', diff saved to https://phabricator.wikimedia.org/P94603 and previous config saved to /var/cache/conftool/dbconfig/20260630-082616-fceratto.json 2431765 wikitext text/x-wiki hfixpe5l4cjyvzjt9t6mxvio5ql3baq 2431767 2431765 2026-06-30T08:32:12Z Stashbot 7414 filippo@dns1004: START - running authdns-update 2431767 wikitext text/x-wiki 78yrp3cgmf3u6f3lmlzd3x0nonw2acd 2431768 2431767 2026-06-30T08:34:11Z Stashbot 7414 filippo@dns1004: END - running authdns-update 2431768 wikitext text/x-wiki 7nt5n42kxpe2e2f4xka1op71wzaabsb 2431769 2431768 2026-06-30T08:35:33Z Stashbot 7414 jmm@dns1004: START - running authdns-update 2431769 wikitext text/x-wiki rqz2frbjpcotofrwqu46511jio3a7t8 2431770 2431769 2026-06-30T08:36:26Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1223', diff saved to https://phabricator.wikimedia.org/P94604 and previous config saved to /var/cache/conftool/dbconfig/20260630-083624-fceratto.json 2431770 wikitext text/x-wiki 3lop8w70ganeebqwd6pxkzttl2o33kj 2431771 2431770 2026-06-30T08:37:34Z Stashbot 7414 jmm@dns1004: END - running authdns-update 2431771 wikitext text/x-wiki rhyphcv6g3xdm2wkjdqvf79scryz92e 2431773 2431771 2026-06-30T08:39:36Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2099.codfw.wmnet with OS trixie 2431773 wikitext text/x-wiki p5q497w955f080kp9a9mak603ml35ac 2431774 2431773 2026-06-30T08:40:11Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 25 hosts with reason: Primary switchover s2 T430624 2431774 wikitext text/x-wiki p85p5qi6qx9h83lmipbrnw8kac0xwnv 2431776 2431774 2026-06-30T08:44:39Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Set db2207 with weight 0 T430624', diff saved to https://phabricator.wikimedia.org/P94605 and previous config saved to /var/cache/conftool/dbconfig/20260630-084436-fceratto.json 2431776 wikitext text/x-wiki gccxc2572jzlrol73nypv91zqrz1s0z 2431777 2431776 2026-06-30T08:46:34Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1223 (T426633)', diff saved to https://phabricator.wikimedia.org/P94606 and previous config saved to /var/cache/conftool/dbconfig/20260630-084632-fceratto.json 2431777 wikitext text/x-wiki gftjlpb335hd9kmbfchqetwov7gemei 2431780 2431777 2026-06-30T08:55:16Z Stashbot 7414 jmm@dns1004: START - running authdns-update 2431780 wikitext text/x-wiki nw67xo0nrt26ku2ap8xcuq248ioru3o 2431781 2431780 2026-06-30T08:57:14Z Stashbot 7414 jmm@dns1004: END - running authdns-update 2431781 wikitext text/x-wiki dufop9uf9tihl1f3hdi3m8koipfmtzz 2431783 2431781 2026-06-30T09:01:13Z Stashbot 7414 marostegui@cumin1003: conftool action : set/pooled=no; selector: name=clouddb1027.eqiad.wmnet,service=s2 2431783 wikitext text/x-wiki cmg9hccsqn05tbol53uib6ssbecxybz 2431784 2431783 2026-06-30T09:01:18Z Stashbot 7414 marostegui@cumin1003: conftool action : set/pooled=no; selector: name=clouddb1027.eqiad.wmnet,service=s7 2431784 wikitext text/x-wiki 6krq1uf3cdalgq00pgh3x5egevwo6vh 2431785 2431784 2026-06-30T09:02:32Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on clouddb[1014,1027].eqiad.wmnet with reason: cloning 2431785 wikitext text/x-wiki rq6m46mkxf6scf2lvxfq0co3ncmtj4d 2431786 2431785 2026-06-30T09:02:40Z Stashbot 7414 marostegui@cumin1003: conftool action : set/pooled=no; selector: name=clouddb1014.eqiad.wmnet,service=s7 2431786 wikitext text/x-wiki 41mve87sdoce5dfl88bja4nz2ok8zwl 2431787 2431786 2026-06-30T09:02:44Z Stashbot 7414 marostegui@cumin1003: conftool action : set/pooled=no; selector: name=clouddb1014.eqiad.wmnet,service=s2 2431787 wikitext text/x-wiki 08nz503vog1um6acv8j1jx0q1hes6z5 2431788 2431787 2026-06-30T09:04:40Z Stashbot 7414 federico3: Starting s2 codfw failover from db2204 to db2207 - T430624 2431788 wikitext text/x-wiki 84m4j7h38s2ufdsjxaqmalv2jjdqsak 2431789 2431788 2026-06-30T09:05:33Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Promote db2207 to s2 primary T430624', diff saved to https://phabricator.wikimedia.org/P94609 and previous config saved to /var/cache/conftool/dbconfig/20260630-090530-fceratto.json 2431789 wikitext text/x-wiki h5obg47lhsf0giho0jd6clyojno0g0c 2431790 2431789 2026-06-30T09:08:44Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Set weight db2204 T430624', diff saved to https://phabricator.wikimedia.org/P94610 and previous config saved to /var/cache/conftool/dbconfig/20260630-090841-fceratto.json 2431790 wikitext text/x-wiki qofw8apkrvws3gp74uvhgb4wrk0jsap 2431791 2431790 2026-06-30T09:12:22Z Stashbot 7414 aklapper@deploy1003: Started scap sync-world: Backport for [[gerrit:1306629|Fix overflow menu for non-advanced users (T428220)]] 2431791 wikitext text/x-wiki 6189236kgb48z6nwgfm3vloiq8b9woh 2431792 2431791 2026-06-30T09:13:01Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2204.codfw.wmnet with reason: Maintenance 2431792 wikitext text/x-wiki a255z2zt7x62yi2vsrnamo2u70g98qv 2431793 2431792 2026-06-30T09:13:09Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db2204 (T426633)', diff saved to https://phabricator.wikimedia.org/P94611 and previous config saved to /var/cache/conftool/dbconfig/20260630-091307-fceratto.json 2431793 wikitext text/x-wiki m2mfrqmzefhcrngmrrvctrj1wavra61 2431795 2431793 2026-06-30T09:16:30Z Stashbot 7414 aklapper@deploy1003: aklapper: Backport for [[gerrit:1306629|Fix overflow menu for non-advanced users (T428220)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431795 wikitext text/x-wiki 81psya2uz361nhoil127towmptrc1qb 2431796 2431795 2026-06-30T09:17:40Z Stashbot 7414 aklapper@deploy1003: aklapper: Continuing with deployment 2431796 wikitext text/x-wiki rldtl0jutyk51wts76wc9nalo1a5nly 2431797 2431796 2026-06-30T09:19:17Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2204 (T426633)', diff saved to https://phabricator.wikimedia.org/P94612 and previous config saved to /var/cache/conftool/dbconfig/20260630-091915-fceratto.json 2431797 wikitext text/x-wiki rirrbd4g10zu2alatva6uq2v15w89xc 2431798 2431797 2026-06-30T09:19:21Z Stashbot 7414 filippo@puppetserver1001: conftool action : set/pooled=yes:weight=100; selector: service=dumps-nfs 2431798 wikitext text/x-wiki rr5vxbxbionxeqg35g043krng0eprtf 2431800 2431798 2026-06-30T09:24:02Z Stashbot 7414 aklapper@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306629|Fix overflow menu for non-advanced users (T428220)]] (duration: 11m 40s) 2431800 wikitext text/x-wiki kyiqych5rg1muqqnctv2ritai630mtl 2431803 2431800 2026-06-30T09:29:25Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2204', diff saved to https://phabricator.wikimedia.org/P94613 and previous config saved to /var/cache/conftool/dbconfig/20260630-092923-fceratto.json 2431803 wikitext text/x-wiki rx73krl8zph0uyq0q7qss6k86cvibip 2431805 2431803 2026-06-30T09:34:24Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2431805 wikitext text/x-wiki 6yem93tlo1jtehgxf27ig1d9ur66jnw 2431806 2431805 2026-06-30T09:34:28Z Stashbot 7414 cwilliams@cumin1003: dbmaint on s4@eqiad T429893 2431806 wikitext text/x-wiki 5l70fb8kapp5nfehd1p7o3pkq5q4dfa 2431807 2431806 2026-06-30T09:34:44Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db1261: Upgrading db1261.eqiad.wmnet 2431807 wikitext text/x-wiki btz3ju287ijp6ub7v4anzvs56woq79m 2431808 2431807 2026-06-30T09:35:15Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1261: Upgrading db1261.eqiad.wmnet 2431808 wikitext text/x-wiki 3i3mc2okix28wkbxm8ielzx1wyjnyg0 2431809 2431808 2026-06-30T09:38:07Z Stashbot 7414 aklapper@deploy1003: rebuilt and synchronized wikiversions files: group0 to 1.47.0-wmf.9 refs T423918 2431809 wikitext text/x-wiki 33xwh7ekk1ck48i7xpdeovrk99crk4y 2431810 2431809 2026-06-30T09:39:32Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2204', diff saved to https://phabricator.wikimedia.org/P94615 and previous config saved to /var/cache/conftool/dbconfig/20260630-093931-fceratto.json 2431810 wikitext text/x-wiki 7hq65kzo1fek4ax2o19xoenbhy107xq 2431811 2431810 2026-06-30T09:49:40Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2204 (T426633)', diff saved to https://phabricator.wikimedia.org/P94616 and previous config saved to /var/cache/conftool/dbconfig/20260630-094938-fceratto.json 2431811 wikitext text/x-wiki 1ejup48xgtf9y9h9yhftcmc5trdb1bv 2431812 2431811 2026-06-30T09:51:13Z Stashbot 7414 aikochou@deploy1003: helmfile [staging] START helmfile.d/services/changeprop: sync 2431812 wikitext text/x-wiki 1r8gjxn1l1xkocioprjryrbt6z89o0n 2431813 2431812 2026-06-30T09:51:21Z Stashbot 7414 aikochou@deploy1003: helmfile [staging] DONE helmfile.d/services/changeprop: sync 2431813 wikitext text/x-wiki e3b2jaatoqxtb0h7q874i573qnh9gke 2431814 2431813 2026-06-30T09:53:31Z Stashbot 7414 godog: restart pybal on A:lvs-secondary-eqiad 2431814 wikitext text/x-wiki pjlk2tl2o8fzhbtupcscrjtnt6tihij 2431815 2431814 2026-06-30T09:56:12Z Stashbot 7414 godog: restart pybal on A:lvs-high-traffic2-eqiad 2431815 wikitext text/x-wiki 3ykwcrilbflb96zywdk03s8lw0ha2aa 2431817 2431815 2026-06-30T10:00:15Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db1261.eqiad.wmnet with OS trixie 2431817 wikitext text/x-wiki 6vyx1g6ddxokp7vsrz5bejwmq4hwtmu 2431818 2431817 2026-06-30T10:07:36Z Stashbot 7414 daniel@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2431818 wikitext text/x-wiki 8ze6w3g4ktzmtcdh2ig6ssprtw50jvg 2431819 2431818 2026-06-30T10:12:39Z Stashbot 7414 daniel@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2431819 wikitext text/x-wiki juhuicmb0y3gsoixfjmiwojro5rgysm 2431820 2431819 2026-06-30T10:15:29Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1261.eqiad.wmnet with reason: host reimage 2431820 wikitext text/x-wiki gvup2h8cih5bqn02r270v08culpwphu 2431821 2431820 2026-06-30T10:18:23Z Stashbot 7414 daniel@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply 2431821 wikitext text/x-wiki 5vcmxsl23ts3yjnk9kvhpq9sd2cmajt 2431822 2431821 2026-06-30T10:19:47Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1261.eqiad.wmnet with reason: host reimage 2431822 wikitext text/x-wiki iexand8lh1pocavafnid9kdp8uyxrnm 2431823 2431822 2026-06-30T10:20:07Z Stashbot 7414 daniel@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply 2431823 wikitext text/x-wiki c5rxi2g2gp00br05y8xncwekfo2gvwf 2431824 2431823 2026-06-30T10:25:27Z Stashbot 7414 javiermonton@deploy1003: Started scap sync-world: Backport for [[gerrit:1306647|stream: webrequest.page_view.dev0 (T426091)]] 2431824 wikitext text/x-wiki sslzg3k51en4h370rhqiv0ce8ocegrc 2431825 2431824 2026-06-30T10:25:37Z Stashbot 7414 daniel@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply 2431825 wikitext text/x-wiki bw2d0n0ds12s2buqewgktyz5gjrmqg8 2431826 2431825 2026-06-30T10:25:58Z Stashbot 7414 daniel@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply 2431826 wikitext text/x-wiki 03d1csctwmfh6yrgfa6f53wluc9c1xv 2431827 2431826 2026-06-30T10:27:32Z Stashbot 7414 javiermonton@deploy1003: javiermonton: Backport for [[gerrit:1306647|stream: webrequest.page_view.dev0 (T426091)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431827 wikitext text/x-wiki hjh36zmd3xvgckmn79hxw6tml4gz850 2431828 2431827 2026-06-30T10:29:00Z Stashbot 7414 javiermonton@deploy1003: javiermonton: Continuing with deployment 2431828 wikitext text/x-wiki o4ox6rh0kndjcyhqflabnepyr9fvf9q 2431829 2431828 2026-06-30T10:30:22Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s_services/services/datahub: apply 2431829 wikitext text/x-wiki kh2cvm2ehivt5qcg64s5dafzhie2tn5 2431830 2431829 2026-06-30T10:32:52Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s_services/services/datahub: apply} 2431830 wikitext text/x-wiki tpt18rxerelttnsgixhl01ds9snwmn5 2431831 2431830 2026-06-30T10:33:24Z Stashbot 7414 javiermonton@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306647|stream: webrequest.page_view.dev0 (T426091)]] (duration: 07m 56s) 2431831 wikitext text/x-wiki 8jiujymcpovjwlwszgfkf5jsv8ruyb5 2431832 2431831 2026-06-30T10:35:50Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1261.eqiad.wmnet with OS trixie 2431832 wikitext text/x-wiki 95xfwwzqt6rlln9oq5rbdh5ur3twjgo 2431833 2431832 2026-06-30T10:38:31Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1217.eqiad.wmnet with reason: cloning 2431833 wikitext text/x-wiki ddf4rxgzspoyls7gorn6bb8cgbdi9ey 2431834 2431833 2026-06-30T10:40:04Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s_services/services/datahub-next: apply 2431834 wikitext text/x-wiki nzy273u6eulysbjwfvzglqae98dgif0 2431835 2431834 2026-06-30T10:40:19Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1228.eqiad.wmnet with reason: cloning 2431835 wikitext text/x-wiki 2newtuahpts71j6g8rwphar0a3b1llr 2431836 2431835 2026-06-30T10:43:39Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on dbproxy[1026,1028].eqiad.wmnet with reason: cloning 2431836 wikitext text/x-wiki siohd9fcntnvok35srqoevult639mbl 2431838 2431836 2026-06-30T10:50:14Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s_services/services/datahub-next: apply 2431838 wikitext text/x-wiki 2xl0pa714kmfwqfz0k7e72y9whhswr3 2431840 2431838 2026-06-30T10:55:55Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2431840 wikitext text/x-wiki qr8fhaqm1mz7g7kybqm0axpnxv1tw39 2431841 2431840 2026-06-30T10:56:05Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/webrequest-page-view-next: apply 2431841 wikitext text/x-wiki d9yrqkbyvosa7666zayss4oa2faz54q 2431843 2431841 2026-06-30T11:03:30Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/pageview-trending-relative-next: apply 2431843 wikitext text/x-wiki d9pmzivkjxsnlq443hkfj5ij9plttze 2431844 2431843 2026-06-30T11:03:40Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/pageview-trending-relative-next: apply 2431844 wikitext text/x-wiki mcl3wmhmdrvp29j8k54oht2twdsyhlt 2431845 2431844 2026-06-30T11:13:40Z Stashbot 7414 moritzm: installing Linux 6.12.94 on Trixie hosts 2431845 wikitext text/x-wiki 8ow08e67juzrvq0xfzocn31tn2lsp2y 2431846 2431845 2026-06-30T11:21:51Z Stashbot 7414 Tran: Deployed patch for T427287 2431846 wikitext text/x-wiki 3b48brq45tbxm41vtrcye8zswrmjjla 2431847 2431846 2026-06-30T11:23:17Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1261: Migration of db1261.eqiad.wmnet completed 2431847 wikitext text/x-wiki n0v81kaclm3q75facw4vpqxp63p9d0k 2431848 2431847 2026-06-30T11:26:34Z Stashbot 7414 moritzm: installing libpng1.6 security updates 2431848 wikitext text/x-wiki i605avl117fdo6hqdkqqnvfosko4u0y 2431849 2431848 2026-06-30T11:28:24Z Stashbot 7414 mszwarc@deploy1003: Started scap sync-world: Backport for [[gerrit:1306650|SuggestedInvestigations: Defer signal matching until transaction commits (T430617)]], [[gerrit:1306651|SuggestedInvestigations: Defer signal matching until transaction commits (T430617)]], [[gerrit:1306658|SuggestedInvestigations: Link to Interaction Timeline for shared pages (T429785)]] 2431849 wikitext text/x-wiki fljfmsnfogfs8wdb8jwdmk5do2w175n 2431850 2431849 2026-06-30T11:30:27Z Stashbot 7414 mszwarc@deploy1003: mszwarc: Backport for [[gerrit:1306650|SuggestedInvestigations: Defer signal matching until transaction commits (T430617)]], [[gerrit:1306651|SuggestedInvestigations: Defer signal matching until transaction commits (T430617)]], [[gerrit:1306658|SuggestedInvestigations: Link to Interaction Timeline for shared pages (T429785)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebu 2431850 wikitext text/x-wiki jgqn2aerzt6709p910gzfob79jn7agp 2431851 2431850 2026-06-30T11:31:54Z Stashbot 7414 mszwarc@deploy1003: mszwarc: Continuing with deployment 2431851 wikitext text/x-wiki bedqxicsco4efy6i5glujwxnt8pbgva 2431852 2431851 2026-06-30T11:36:11Z Stashbot 7414 mszwarc@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306650|SuggestedInvestigations: Defer signal matching until transaction commits (T430617)]], [[gerrit:1306651|SuggestedInvestigations: Defer signal matching until transaction commits (T430617)]], [[gerrit:1306658|SuggestedInvestigations: Link to Interaction Timeline for shared pages (T429785)]] (duration: 07m 48s) 2431852 wikitext text/x-wiki 26782yqjpf2nuugwlh27gprarmhw4vd 2431853 2431852 2026-06-30T11:38:01Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/growthbook-next: apply 2431853 wikitext text/x-wiki 7p8ham17012iu4eui9jusclz4f2zlzd 2431854 2431853 2026-06-30T11:39:05Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/growthboo-next: apply 2431854 wikitext text/x-wiki p3eq4xe63n1m6c0xl07vbdm7b8jvd78 2431855 2431854 2026-06-30T11:40:52Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1306667|SimpleCaptcha: Log skipcaptcha right in force-show trigger (T402595)]] 2431855 wikitext text/x-wiki 3stx6bs53mwl90llvu3ido299o9m12o 2431856 2431855 2026-06-30T11:42:50Z Stashbot 7414 marostegui@cumin1003: conftool action : set/pooled=yes; selector: name=clouddb1014.eqiad.wmnet,service=s2 2431856 wikitext text/x-wiki 46lews62p50mhiglwpm1dsikh69vjlu 2431857 2431856 2026-06-30T11:42:54Z Stashbot 7414 marostegui@cumin1003: conftool action : set/pooled=yes; selector: name=clouddb1014.eqiad.wmnet,service=s7 2431857 wikitext text/x-wiki r01abrx27ebwyz6lgueqc715imklxcx 2431858 2431857 2026-06-30T11:42:58Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1306667|SimpleCaptcha: Log skipcaptcha right in force-show trigger (T402595)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431858 wikitext text/x-wiki 8nqdo8t5xhmbbqks3aeb6psxt5ojbui 2431859 2431858 2026-06-30T11:43:04Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2104.codfw.wmnet with OS trixie 2431859 wikitext text/x-wiki 3hl92z7boei4a2i9c3fuhqmxwa984ms 2431860 2431859 2026-06-30T11:44:08Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2431860 wikitext text/x-wiki 4svazqjeh7rb7vveqgz044wbbg50vrc 2431861 2431860 2026-06-30T11:44:34Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2105.codfw.wmnet with OS trixie 2431861 wikitext text/x-wiki 3bnyn2qfume6x7im01m6bikwh4g9log 2431862 2431861 2026-06-30T11:48:27Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306667|SimpleCaptcha: Log skipcaptcha right in force-show trigger (T402595)]] (duration: 07m 39s) 2431862 wikitext text/x-wiki h5aapj9j0ifbbjxdw9v6hp1cidyaba3 498 30942 2431411 2430512 2026-06-29T13:14:28Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services 2431411 wikitext text/x-wiki b2iewzbzxlct3l1xq0gt4r9wkqv3r3y 2431414 2431411 2026-06-29T13:17:35Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment codfw1dev for all services 2431414 wikitext text/x-wiki reyroqlmqnlikq9xg6qm6y4589bl9qq 498 55577 2431412 2429890 2026-06-29T13:15:53Z Stashbot 7414 andrewbogott: rm -rf /srv/paws/files-to-remove/* on paws-nfs-2 2431412 wikitext text/x-wiki 8a9rt0dd5ixl326h78agqlkbdfomnl2 498 443628 2431619 2431188 2026-06-29T21:52:00Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/28404836827 (https://github.com/cluebotng/component-configs/commits/637e933186e3a7fedd0e7328d4c4e6035d12ec82) 2431619 wikitext text/x-wiki 4h0k0zbl9eoaupnnqntp9hfw00y77vc 0 444275 2431540 2396102 2026-06-29T18:25:49Z RLazarus (WMF) 15215 /* Roll out the new version on bare-metal hosts */ 2431540 wikitext text/x-wiki razy0pfreb44xq2qe0e490b7msszy80 0 446572 2431510 2401886 2026-06-29T16:12:35Z APaskulin (WMF) 14880 Mark as historical 2431510 wikitext text/x-wiki 494u4ktextcy8uwxdjmaxo966ucbsko 0 448033 2431482 2426562 2026-06-29T15:36:40Z Dreamyshade 53333 Link to ORES docs for context about migrated models 2431482 wikitext text/x-wiki cs6xaws1omhx89vjt4rj0taao80fcug 2431734 2431482 2026-06-30T06:44:35Z Dreamyshade 53333 Undid revision [[Special:Diff/2431482|2431482]] by [[Special:Contributions/Dreamyshade|Dreamyshade]] ([[User talk:Dreamyshade|talk]]) I see now that the "model card" column is supposed to give this context 2431734 wikitext text/x-wiki 5k605yzmg8ggywl2jc96eyadtunc1vf 0 448234 2431837 2196532 2026-06-30T10:47:59Z MLechvien-WMF 49197 Fix broken link 2431837 wikitext text/x-wiki smn009a3hafw8tdjjrwqwf4ta2ov1vd 0 448299 2431842 2229274 2026-06-30T10:56:19Z MLechvien-WMF 49197 2431842 wikitext text/x-wiki l1pgdk8lj28k84lw8hizrp11ewfjkoo 0 449160 2431622 2431232 2026-06-30T00:01:06Z Dexbot 30554 Bot: Updating the report 2431622 wikitext text/x-wiki gbgg5zd6m978mo5blxoylk4hunfp60q 2431625 2431622 2026-06-30T00:10:10Z Dexbot 30554 Bot: Updating the report 2431625 wikitext text/x-wiki rwb0m74rvkuc7nwkqonlona3zn63ok1 2431700 2431625 2026-06-30T05:20:00Z Dexbot 30554 Bot: Updating the report 2431700 wikitext text/x-wiki o9wtg2szw4hbx0at6lu7yajmnn9f1hh 2431714 2431700 2026-06-30T05:56:08Z Dexbot 30554 Bot: Updating the report 2431714 wikitext text/x-wiki ii6t17dxsu3qkwm2zl1s7imknz36td6 2431740 2431714 2026-06-30T07:05:19Z Dexbot 30554 Bot: Updating the report 2431740 wikitext text/x-wiki 62p0sg079ozn8rdwshvnt5xkecr09nb 2431761 2431740 2026-06-30T08:11:30Z Dexbot 30554 Bot: Updating the report 2431761 wikitext text/x-wiki 4go53sp5i5b7p22yatdnh2e4rflpj0k 2431778 2431761 2026-06-30T08:47:37Z Dexbot 30554 Bot: Updating the report 2431778 wikitext text/x-wiki mk25jczjvwfzm7oy7kgrdghaqa45tez 2431794 2431778 2026-06-30T09:14:44Z Dexbot 30554 Bot: Updating the report 2431794 wikitext text/x-wiki gx0m3y398psyp3xp83pwm4texbvp0cx 0 451075 2431608 2427704 2026-06-29T21:07:37Z WFan (WMF) 38087 2431608 wikitext text/x-wiki jprotcuy0m97ld99wsr4o7hnu3sdm12 0 452860 2431447 2395025 2026-06-29T14:12:52Z Krinkle 44 /* Activity feed */ 2431447 wikitext text/x-wiki 4kfubvbk4rvdgbmryrt469gf2157m26 2431449 2431447 2026-06-29T14:13:12Z Krinkle 44 /* Activity feed */ 2431449 wikitext text/x-wiki h7oubrjs1bvnkuhlbx01cv3tp17dng8 2431458 2431449 2026-06-29T14:40:43Z Krinkle 44 2431458 wikitext text/x-wiki 5c3uy86xthlkslxrer42r8r2lq95bxl 116 453906 2431556 2431204 2026-06-29T19:03:28Z Gitlabaccountapprovalbot 37332 @thisismattmiller was approved. 2431556 wikitext text/x-wiki p6e4tq8cmuiyvham3i779j5gd0fvazr 12 454293 2431550 2380038 2026-06-29T18:48:24Z ADancy (WMF) 17694 /* Step 1: Setup a puppetserver */ Add step for setting up profile::puppet::agent::puppetserver_ca_cert in project puppet hiera config 2431550 wikitext text/x-wiki n8sb642zcf2u881lici4bh4hzza95vg 2431555 2431550 2026-06-29T18:56:45Z ADancy (WMF) 17694 /* Step 2: Setup a puppet client */ Simplify client instructions (T429413) 2431555 wikitext text/x-wiki 8rympw9a7pf63l2rkggb1vugl7y9pos 0 455989 2431463 2430802 2026-06-29T14:58:04Z MPopov (WMF) 5891 /* Launch */ Add instructions for monitoring event production 2431463 wikitext text/x-wiki 28wyqln8y602ai29ojylizcxz287cx3 0 459290 2431418 2431214 2026-06-29T13:23:06Z JAllemandou (WMF) 37709 /* YYYY-MM-DD NEXT TUESDAY TRAIN (REPLACE THIS AFTER DEPLOY) */ Add new refinery change 2431418 wikitext text/x-wiki a4gm5vihl0pr8rxrzkz017a0pmpq4gl 2431512 2431418 2026-06-29T16:27:01Z AQuhen (WMF) 29318 /* Tuesday, June 30, 2026 TODO */ 2431512 wikitext text/x-wiki 8itr8m58tliw0sdw9g48rf8s32j829g 2431839 2431512 2026-06-30T10:50:15Z APizzata-WMF 47941 https://gerrit.wikimedia.org/r/c/analytics/refinery/source/+/1305674 2431839 wikitext text/x-wiki 3zs9zw7hi1c4vt9onspzziplza5ckku 0 459394 2431539 2360950 2026-06-29T18:23:31Z Reedy 9 fix number 2431539 wikitext text/x-wiki ph9dbw56829o5pe1ubi1uy5x9kav4fx 0 459699 2431533 2430966 2026-06-29T18:08:02Z Stashbot 7414 ejegg: payments-wiki upgraded from c2cdbf24 to 39df7f6b 2431533 wikitext text/x-wiki 5zl3hy1nwhazae4di1f3jf20h87l7ku 2431543 2431533 2026-06-29T18:32:21Z Stashbot 7414 wfan: payments-wiki upgraded from 39df7f6b to 32059801 2431543 wikitext text/x-wiki tqqv606f5m94yfq26lze6bpsh0osgxq 2431553 2431543 2026-06-29T18:54:31Z Stashbot 7414 ejegg: payments-wiki upgraded from 32059801 to 7e1a422e 2431553 wikitext text/x-wiki 07jea762d383jb24xrp3e4euyjuqzvx 2431620 2431553 2026-06-29T23:16:37Z Stashbot 7414 danielfm: payments-wiki upgraded from 7e1a422e to 28983fa4 2431620 wikitext text/x-wiki ipj0ecnq4d4y3zzorqxdst4vpzru1sb 2431737 2431620 2026-06-30T07:01:10Z Stashbot 7414 eileen: civicrm upgraded from 63606ee8 to c2fa32a6 2431737 wikitext text/x-wiki gf5dygpkvs9gm1qmfr7p2yi3cro5p3j 0 459903 2431772 2419439 2026-06-30T08:37:38Z BJensen-WMF 48942 Add instructions for building a new version. 2431772 wikitext text/x-wiki r6ldwvr4v628t9cei56ro6pmck0npxh 2431782 2431772 2026-06-30T08:57:41Z BJensen-WMF 48942 /* Building a new version */ Add a pointer to the upstream helm chart. 2431782 wikitext text/x-wiki 52zbrlige8chbeu3uau8ev21vrouj6e 498 460126 2431504 2419480 2026-06-29T16:04:39Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: [DEPLOY] Starting deployment | ref=v1.5.6 2431504 wikitext text/x-wiki dwfz8opz8nu8ujw9dvlkkhiezzns8mb 2431506 2431504 2026-06-29T16:06:11Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: [DEPLOY] Build triggered successfully | ref=v1.5.6 2431506 wikitext text/x-wiki djs6wwmfeo1qplzampzkxafc3vnr4u8 2431507 2431506 2026-06-29T16:06:12Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: [DEPLOY] Restarting buildservice | ref=v1.5.6 2431507 wikitext text/x-wiki t9tidqz9tm4dw6spzy5qbajzinpfg2k 2431508 2431507 2026-06-29T16:06:23Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: [DEPLOY] Deployment completed successfully | ref=v1.5.6 2431508 wikitext text/x-wiki gvvwnb9yfht17s0nywk6pz20mvsy5tg 498 460131 2431496 2408774 2026-06-29T15:56:06Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: [DEPLOY] Starting deployment | ref=refactor/migrate-to-mediawiki4j 2431496 wikitext text/x-wiki 5oj54amnjwy5tb83czahndohaz7p2r2 2431498 2431496 2026-06-29T15:57:34Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: [DEPLOY] Build triggered successfully | ref=refactor/migrate-to-mediawiki4j 2431498 wikitext text/x-wiki janmfxtm3rnl03zmx5fbb0mg1zdjsf9 2431499 2431498 2026-06-29T15:57:35Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: [DEPLOY] Restarting buildservice | ref=refactor/migrate-to-mediawiki4j 2431499 wikitext text/x-wiki 0ihkou7qdyp9fctuisz4ax6grmvv8b7 2431500 2431499 2026-06-29T15:57:45Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: [DEPLOY] Deployment completed successfully | ref=refactor/migrate-to-mediawiki4j 2431500 wikitext text/x-wiki 461faaiytfueee0vrz6vrmnif5jmib8 0 460178 2431802 2428816 2026-06-30T09:26:35Z MLechvien-WMF 49197 Adding DC Switchover question, to be consistent with template 2431802 wikitext text/x-wiki soezhsk0abfsw9r34rqs2i6ze0ypgx7 0 460192 2431801 2428847 2026-06-30T09:25:31Z MLechvien-WMF 49197 Adding question regarding DC Switchover 2431801 wikitext text/x-wiki qliybvt2ildrb604lrl6qittaz2hrqh 0 460231 2431534 2429296 2026-06-29T18:08:43Z BKing (WMF) 29017 /* Steps */ 2431534 wikitext text/x-wiki oyknshs3ldqd2q0ivm4n5pu9zgezw5n 0 460234 2431610 2431101 2026-06-29T21:12:19Z MPopov (WMF) 5891 /* Specify experiment details */ 2431610 wikitext text/x-wiki nwieb3al0gwwp1fk4wafcpcug0grd64 0 460268 2431359 2430351 2026-06-29T12:00:35Z Volker E. (WMF) 41575 /* General system expectations */ Fix sentence semantics 2431359 wikitext text/x-wiki tro6ec47iquhc6ku5dfwaa9l3o7ek7f 0 460388 2431529 2431291 2026-06-29T18:00:20Z AKhatun (WMF) 50461 Update access and usage sections 2431529 wikitext text/x-wiki m2clsgbfi4sdzaxusnclqqyjohjxa68 2431530 2431529 2026-06-29T18:01:33Z AKhatun (WMF) 50461 2431530 wikitext text/x-wiki ixhs15763uzbat5l4cv8yd2dmmf3nv0 2431751 2431530 2026-06-30T07:39:26Z GGoncalves-WMF 42848 /* Technical documentation */ Link to the original design. 2431751 wikitext text/x-wiki azz7rs4qa51k2105m653gxurc2cuyf5 0 460389 2431400 2431305 2026-06-29T13:03:17Z FNegri-WMF 32595 fix formatting 2431400 wikitext text/x-wiki hpmd821l5zb22phzxjiucr9jhf9g0hd 2431403 2431400 2026-06-29T13:05:10Z FNegri-WMF 32595 /* Tools using autocommit=False */ add "how to fix it" 2431403 wikitext text/x-wiki 37hotvxj96e6fx7vwkp9sjcnjdi8671 2431405 2431403 2026-06-29T13:05:24Z FNegri-WMF 32595 /* Tools using autocommit=False */ remove empty line 2431405 wikitext text/x-wiki 1asrgx8swidzec1tp5uv3eyg5h914ms 2431407 2431405 2026-06-29T13:06:22Z FNegri-WMF 32595 /* Tools making long queries */ add "how to fix it" 2431407 wikitext text/x-wiki qv1j2kp08j48xmy72jf1jrlmfpztq4l 3 460390 2431438 2026-06-29T14:06:43Z StrikerBot 8475 Welcome to Toolforge! 2431438 wikitext text/x-wiki s4osx9795el3i4izz111cou21plzxj3 2431513 2431438 2026-06-29T16:28:54Z Henryk Siuda 42006 /* Welcome to Toolforge! */ Reply 2431513 wikitext text/x-wiki c88n54ilq2ydy2z9ca5zpbev1iy9ciq 3 460391 2431450 2026-06-29T14:13:28Z StrikerBot 8475 Welcome to Toolforge! 2431450 wikitext text/x-wiki taweeai2qqyvkrokak0iozja92br1c1 3 460392 2431453 2026-06-29T14:16:04Z StrikerBot 8475 Welcome to Toolforge! 2431453 wikitext text/x-wiki 6c7u98aheqlzkijrb8zr9xzhwsz9gy3 0 460393 2431611 2026-06-29T21:12:45Z RLazarus (WMF) 15215 publish from WMF google doc 2431611 wikitext text/x-wiki n4t2sqb8950n6kidy94sn1rm2agq424