Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.22 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 3086 2462733 2462276 2026-10-01T14:28:15Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2462733 wikitext text/x-wiki jqhepxfkouh6kes74kbioubplgvuj0m 2462741 2462733 2026-10-01T14:43:13Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2462741 wikitext text/x-wiki 69n6v257liea2orwotbqhp3h1dtw9bx 2462771 2462741 2026-10-01T15:52:23Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-cli 2462771 wikitext text/x-wiki 60w901leczxi7iuqdmgua9ulorsetbv 2462773 2462771 2026-10-01T15:52:51Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-cli 2462773 wikitext text/x-wiki 3qaoxkrgy8wl6zl4qnsoimb7480jzz2 0 4108 2462684 2462592 2026-10-01T13:30:38Z ScheduleDeploymentBot 37566 Add [[gerrit:1347606]] to Thursday, October 01 UTC late backport window 2462684 wikitext text/x-wiki cyrp7tbp7ajebt4i9xywiran4pba2yt 2462785 2462684 2026-10-01T16:27:54Z ScheduleDeploymentBot 37566 Add [[gerrit:1347686]] to Thursday, October 01 UTC late backport window 2462785 wikitext text/x-wiki ijyjlz5dwoptow3ub2d85q5n2auno4r 2462786 2462785 2026-10-01T16:28:03Z ScheduleDeploymentBot 37566 Add [[gerrit:1347687]] to Thursday, October 01 UTC late backport window 2462786 wikitext text/x-wiki sxr9fz9nhifao1vn9jlcp0jocwudtpo 2462790 2462786 2026-10-01T16:38:13Z ScheduleDeploymentBot 37566 Add [[gerrit:1347692]] to Thursday, October 01 UTC late backport window 2462790 wikitext text/x-wiki 41ysuzcp2e49i1k34qlbdj8oh6kzuq5 2462791 2462790 2026-10-01T16:39:32Z Srapoj 44884 /* Thursday, October 1 */ 2462791 wikitext text/x-wiki 135eb1cbc7v3l98fh57fam0hppbvfjp 2462808 2462791 2026-10-01T17:45:28Z ScheduleDeploymentBot 37566 Add [[gerrit:1347712]] to Thursday, October 01 UTC late backport window 2462808 wikitext text/x-wiki ajqabh2opg3ukq6e6g7m2pwbgi5txi8 2462869 2462808 2026-10-01T18:55:36Z ScheduleDeploymentBot 37566 Add [[gerrit:1347729]] to Thursday, October 01 UTC late backport window 2462869 wikitext text/x-wiki h8sltwb0l4pk0wnnqx2a1uymr97p2sv 2462870 2462869 2026-10-01T18:59:11Z ScheduleDeploymentBot 37566 Add [[gerrit:1347735]] to Thursday, October 01 UTC late backport window 2462870 wikitext text/x-wiki pcitwfzk5oxqkbm2q7i28e28mrmilla 2462875 2462870 2026-10-01T19:17:12Z ScheduleDeploymentBot 37566 Add [[gerrit:1347743]] to Thursday, October 01 UTC late backport window 2462875 wikitext text/x-wiki 2b4srowao6hp53ny3v6it6wrkpkr0mm 2462879 2462875 2026-10-01T19:30:04Z Mmullie (WMF) 291 /* {{Deployment_day|date=2026-10-01}} */ 2462879 wikitext text/x-wiki mi5ghvgonyq44l4kmcx0bkl12xmc2eb 498 6726 2462721 2462245 2026-10-01T14:06:19Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2462721 wikitext text/x-wiki h5vpgd64u9afos634vlm8yr6yowkozk 2462722 2462721 2026-10-01T14:06:22Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api 2462722 wikitext text/x-wiki spk1mcikey9tkov8f1pfnzww4qjrx8f 2462727 2462722 2026-10-01T14:12:42Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2462727 wikitext text/x-wiki g7u23j44wdnxb59m0lu2nam0vg8om6j 2462732 2462727 2026-10-01T14:27:18Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2462732 wikitext text/x-wiki 1djxi13iqdimtthcpsjqq7gqvoje91b 2462768 2462732 2026-10-01T15:51:25Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-cli 2462768 wikitext text/x-wiki 8kt2uwg1dygsc9oh706m3tixdjrv4ll 2462770 2462768 2026-10-01T15:51:55Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-cli 2462770 wikitext text/x-wiki 0hszwlwa6x8wx4zihzwltypcp4b4ewe 2462999 2462770 2026-10-02T11:06:03Z Stashbot 7414 fnegri@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component alerts-api (T432865) 2462999 wikitext text/x-wiki 9vr4h2urc823ncungsc2gb5sdnvtsje 2463000 2462999 2026-10-02T11:07:08Z Stashbot 7414 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component alerts-api (T432865) 2463000 wikitext text/x-wiki 23fabcxwgdskbvmtetvlwrf7w4gw697 0 7919 2462612 2462611 2026-10-01T12:05:46Z Stashbot 7414 dkertesz@cumin1004: START - Cookbook sre.cdn.roll-upgrade-haproxy rolling upgrade of HAProxy on A:cp-eqsin and A:cp - upgrade haproxy to 3.2.25-1 (T439667) 2462612 wikitext text/x-wiki 890tqkxhh8m9z6i6is7kpfzeietof5h 2462613 2462612 2026-10-01T12:05:58Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2462613 wikitext text/x-wiki rp9s349g713ay7c79k6zlo4n5ttvnvf 2462614 2462613 2026-10-01T12:06:41Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2462614 wikitext text/x-wiki mqktjbl25okicmwf8fsovqr4458mfm5 2462615 2462614 2026-10-01T12:14:50Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-pretrain: apply 2462615 wikitext text/x-wiki 3clgagbfcoxlek1tjcfggxmqybp1alf 2462616 2462615 2026-10-01T12:15:06Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-pretrain: apply 2462616 wikitext text/x-wiki 4scsnv79jvh91x4cjexy0e68ec8opkz 2462617 2462616 2026-10-01T12:19:43Z Stashbot 7414 a-pizzata@deploy1003: Started deploy [analytics/refinery@1337993] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@13379939] 2462617 wikitext text/x-wiki s91a3jdf1i9f5rtxhzbcv6barjm1rmi 2462618 2462617 2026-10-01T12:20:29Z Stashbot 7414 a-pizzata@deploy1003: Finished deploy [analytics/refinery@1337993] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@13379939] (duration: 00m 46s) 2462618 wikitext text/x-wiki kemsm9b4zgar8at0z58fwbotpec3xou 2462619 2462618 2026-10-01T12:21:16Z Stashbot 7414 a-pizzata@deploy1003: Started deploy [analytics/refinery@1337993]: Regular analytics weekly train [analytics/refinery@13379939] 2462619 wikitext text/x-wiki l19e5tmguxfk874o34u4ix0eionpt4q 2462620 2462619 2026-10-01T12:22:34Z Stashbot 7414 moritzm: installing glibc security updates 2462620 wikitext text/x-wiki 1k9qu24jpxk8snirtw632e0eelwg8oh 2462621 2462620 2026-10-01T12:23:37Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2462621 wikitext text/x-wiki d6yp528e0pognx5sq34slxcurqirfj2 2462622 2462621 2026-10-01T12:24:17Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2462622 wikitext text/x-wiki begf5aiab2cqt9pe4aj9t2ts537o71l 2462623 2462622 2026-10-01T12:25:43Z Stashbot 7414 a-pizzata@deploy1003: Finished deploy [analytics/refinery@1337993]: Regular analytics weekly train [analytics/refinery@13379939] (duration: 04m 27s) 2462623 wikitext text/x-wiki jldzc8xg0jbvdhmogyc76mm4uocpwiy 2462624 2462623 2026-10-01T12:27:29Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-pretrain: apply 2462624 wikitext text/x-wiki kqyjrd163osei3ph5nqatwjhq2uoufk 2462625 2462624 2026-10-01T12:27:55Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-pretrain: apply 2462625 wikitext text/x-wiki ih05bu67sgweavynt4zkdj44c61pi0o 2462626 2462625 2026-10-01T12:32:13Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2462626 wikitext text/x-wiki 7o7rhguc5dojcqmdp9nhj1mfoqtpsrc 2462627 2462626 2026-10-01T12:33:29Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2462627 wikitext text/x-wiki liihrhnkzewk6fe1nth9i5yogbpn1pi 2462628 2462627 2026-10-01T12:34:20Z Stashbot 7414 moritzm: installing libtasn1-6 security updates 2462628 wikitext text/x-wiki pui8n9mpc6rfdpj86qaxju0torblgc5 2462629 2462628 2026-10-01T12:34:28Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . 2462629 wikitext text/x-wiki 8vo0jpb8bveu9vh9e5c23ntcyogl06h 2462630 2462629 2026-10-01T12:40:23Z Stashbot 7414 ihurbain@deploy1003: helmfile [eqiad] START helmfile.d/services/mobileapps: apply 2462630 wikitext text/x-wiki 84cbs0dm375nl5jo44zvksktzidblux 2462631 2462630 2026-10-01T12:44:09Z Stashbot 7414 ihurbain@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mobileapps: apply 2462631 wikitext text/x-wiki lyjg4smbb2m3z62cwpwayngbu42ntb6 2462632 2462631 2026-10-01T12:44:26Z Stashbot 7414 ihurbain@deploy1003: helmfile [codfw] START helmfile.d/services/mobileapps: apply 2462632 wikitext text/x-wiki bu61pjaznguodfq64a9a9k8oryba9wc 2462633 2462632 2026-10-01T12:45:27Z Stashbot 7414 ihurbain@deploy1003: helmfile [codfw] DONE helmfile.d/services/mobileapps: apply 2462633 wikitext text/x-wiki cjypd15vwy12jicntwuvmeouky0kokn 2462634 2462633 2026-10-01T12:53:41Z Stashbot 7414 mszwarc@deploy1003: mszwarc: Sync PrivateSettings.php synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462634 wikitext text/x-wiki 83hjwyl1s0uyi3zhko6t52nfjvgyoao 2462635 2462634 2026-10-01T12:54:25Z Stashbot 7414 mszwarc@deploy1003: mszwarc: Continuing with deployment 2462635 wikitext text/x-wiki gs3vv21ymm0ofaqossam8rh51gasxl8 2462636 2462635 2026-10-01T12:54:37Z Stashbot 7414 filippo@cumin1004: START - Cookbook sre.hosts.reimage for host cloudcephosd1055.eqiad.wmnet with OS bookworm 2462636 wikitext text/x-wiki g2a4rrglehp105oq8h3yrppjhfq3by6 2462637 2462636 2026-10-01T12:59:32Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/pageview-trending-relative-next: apply 2462637 wikitext text/x-wiki 5x77jgrx8zlhc4comn2sh58go23w376 2462638 2462637 2026-10-01T12:59:44Z Stashbot 7414 mszwarc@deploy1003: Synchronized private/PrivateSettings.php: Sync PrivateSettings.php (duration: 10m 02s) 2462638 wikitext text/x-wiki nhsk683d8qto4d369ya5a273nh87xlz 2462639 2462638 2026-10-01T12:59:58Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/pageview-trending-relative-next: apply 2462639 wikitext text/x-wiki hmvn5w9upgeauq6r1hv609y23cnx5p3 2462642 2462639 2026-10-01T13:05:46Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add VIPs for ldap-ro-test in eqiad/codfw - jmm@cumin2003" 2462642 wikitext text/x-wiki lrrj64usnl5t4h87u6l5yol1cr2kqn5 2462643 2462642 2026-10-01T13:05:47Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2462643 wikitext text/x-wiki cmmv364u7jdza6vtz2bo955tawh57vs 2462644 2462643 2026-10-01T13:06:43Z Stashbot 7414 oblivian@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-timeline: apply 2462644 wikitext text/x-wiki hidgqpk75yrp58znrrdg5grdf3fi22k 2462645 2462644 2026-10-01T13:07:03Z Stashbot 7414 oblivian@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-timeline: apply 2462645 wikitext text/x-wiki 6d42rq31xus85vlejhmt7h75zbiz8dz 2462646 2462645 2026-10-01T13:07:31Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply 2462646 wikitext text/x-wiki 32gmshb7gu4yg9vuzn5yiscwzsxekm3 2462647 2462646 2026-10-01T13:08:05Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply 2462647 wikitext text/x-wiki pmsrlnlaoc0d20ix3rjwgu6z7xiur5d 2462648 2462647 2026-10-01T13:08:06Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.swift.roll-restart-reboot-swift-thanos-proxies rolling restart_daemons on A:thanos-fe 2462648 wikitext text/x-wiki 7ncr1nxfc56bbmvsvun8ec5sxpaavtk 2462649 2462648 2026-10-01T13:12:18Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-thanos-proxies (exit_code=0) rolling restart_daemons on A:thanos-fe 2462649 wikitext text/x-wiki dpm3pmfhpd36uhhn7k85iiykfj6hc0m 2462650 2462649 2026-10-01T13:17:55Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2462650 wikitext text/x-wiki of5zdnrb8r6hvldabxp50zpjuilgh3q 2462651 2462650 2026-10-01T13:18:04Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2462651 wikitext text/x-wiki 8cdg0eoywqul78voxz962idvg3rw2el 2462652 2462651 2026-10-01T13:18:59Z Stashbot 7414 awight@deploy1003: Started scap sync-world: Backport for [[gerrit:1347422|instrument(AccountSetup): don't double count finish/skip clicks]], [[gerrit:1347423|stats(AccountSetup): Record onboarding ready time in Prometheus (T432810)]], [[gerrit:1347424|instrument(AccountSetup): Send exposure from server, send ready event (T432164)]] 2462652 wikitext text/x-wiki rrwm65ittgjmc2dhjrwivtnmxjeyo3m 2462653 2462652 2026-10-01T13:19:01Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-product: apply 2462653 wikitext text/x-wiki 9roly188a0lpjizl9ai90npagkqcxqu 2462654 2462653 2026-10-01T13:19:42Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-product: apply 2462654 wikitext text/x-wiki 1bbcvfnudc70xih06imo7tjmtpa5ni1 2462655 2462654 2026-10-01T13:20:57Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply 2462655 wikitext text/x-wiki tv3u5tpbmh3ycsmi8ursxypc4zepex4 2462656 2462655 2026-10-01T13:21:30Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply 2462656 wikitext text/x-wiki hf5s5b2ip44fx71agt51cjdyivz68gr 2462657 2462656 2026-10-01T13:21:41Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-experiment-platform: apply 2462657 wikitext text/x-wiki qa2i028m32rwcqg8uzkduz29wtqz7d3 2462658 2462657 2026-10-01T13:22:19Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-experiment-platform: apply 2462658 wikitext text/x-wiki sdj24yx9vnara8zew4nesof2alyi2do 2462659 2462658 2026-10-01T13:22:35Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-fr-tech: apply 2462659 wikitext text/x-wiki pd4zm4ugomhiz9vozl12f91qn3s91s2 2462660 2462659 2026-10-01T13:23:08Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-fr-tech: apply 2462660 wikitext text/x-wiki ou5lnxbouk2yhcx6cscx3z1zpkb9hzr 2462661 2462660 2026-10-01T13:23:15Z Stashbot 7414 awight@deploy1003: awight, migr: Backport for [[gerrit:1347422|instrument(AccountSetup): don't double count finish/skip clicks]], [[gerrit:1347423|stats(AccountSetup): Record onboarding ready time in Prometheus (T432810)]], [[gerrit:1347424|instrument(AccountSetup): Send exposure from server, send ready event (T432164)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be ve 2462661 wikitext text/x-wiki eo8x72mk8pde33jz9on03r951uwfb34 2462662 2462661 2026-10-01T13:23:35Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2462662 wikitext text/x-wiki dv4q84s1vgj96fanxzv7mbb8x70t0dn 2462663 2462662 2026-10-01T13:23:40Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2462663 wikitext text/x-wiki r16wxe7eqv6lfzoi5wapw0inxangbcx 2462664 2462663 2026-10-01T13:24:16Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-ml: apply 2462664 wikitext text/x-wiki px2zkn56vtia2h9t0rxevqn88w8z8a2 2462665 2462664 2026-10-01T13:24:43Z Stashbot 7414 oblivian@deploy1003: helmfile [staging] START helmfile.d/services/shellbox: sync 2462665 wikitext text/x-wiki lxdq03spkao4m8ggd1opb7952is5fmc 2462666 2462665 2026-10-01T13:24:46Z Stashbot 7414 oblivian@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox: sync 2462666 wikitext text/x-wiki pj6grfpzc7lmrzg10fx9552y3oty0hv 2462667 2462666 2026-10-01T13:24:48Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-ml: apply 2462667 wikitext text/x-wiki dj10653dqk1hpt157db7hjcqweq3vqw 2462668 2462667 2026-10-01T13:24:49Z Stashbot 7414 oblivian@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-timeline: sync 2462668 wikitext text/x-wiki e9xnt03sbl5ojpc8a0v31lsqz2q1el5 2462669 2462668 2026-10-01T13:24:54Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-platform-eng: apply 2462669 wikitext text/x-wiki ttb10qfout0gk4zrw1bh7vj6wxrj5md 2462670 2462669 2026-10-01T13:25:00Z Stashbot 7414 oblivian@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-timeline: sync 2462670 wikitext text/x-wiki c6q6vc782scnethgjn9sj7npbvj3zg2 2462671 2462670 2026-10-01T13:25:32Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-platform-eng: apply 2462671 wikitext text/x-wiki lewja4uk28j8qkjnu9mz14a4rjscsca 2462672 2462671 2026-10-01T13:25:42Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2462672 wikitext text/x-wiki eo94sq4a9rqn1511am9y6kb1csvhufj 2462673 2462672 2026-10-01T13:25:47Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2462673 wikitext text/x-wiki mjilobwkmn6s18aatqw5pgos8tbdode 2462674 2462673 2026-10-01T13:25:54Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-research: apply 2462674 wikitext text/x-wiki hmj5ab339niz4j4vbgui2ks2pb8bdda 2462675 2462674 2026-10-01T13:26:03Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2462675 wikitext text/x-wiki rszsfqw7u4ccewa0llwnoicazlmvmxg 2462676 2462675 2026-10-01T13:26:08Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2462676 wikitext text/x-wiki 4nz9b69m8tvscvk69osko470punp4g6 2462677 2462676 2026-10-01T13:26:28Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-research: apply 2462677 wikitext text/x-wiki ecr7x3sz2mw0ibpwcukeqv2i342ybpo 2462678 2462677 2026-10-01T13:29:22Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2462678 wikitext text/x-wiki 34nnazrmh7ek6zlqp64blcvrsac6uil 2462679 2462678 2026-10-01T13:29:39Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply 2462679 wikitext text/x-wiki o125ljvux13q4l5to0s7oz5uhnr9yj4 2462680 2462679 2026-10-01T13:30:08Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-search: apply 2462680 wikitext text/x-wiki omu7vmdvtb4t663xmdhewnp6zf3suqh 2462681 2462680 2026-10-01T13:30:11Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2462681 wikitext text/x-wiki lryzzhpurm3p8ytcj1mdcw3ktwn1i4h 2462682 2462681 2026-10-01T13:30:15Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply 2462682 wikitext text/x-wiki nhk6ojsfj232qu3kdpvava0hx6al8q9 2462683 2462682 2026-10-01T13:30:26Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2462683 wikitext text/x-wiki b893m8nkfgdholjknxxapq2l25t31w8 2462685 2462683 2026-10-01T13:30:42Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-search: apply 2462685 wikitext text/x-wiki kqz66ew6rsvgrp35q61ihxnbuft5ajt 2462687 2462685 2026-10-01T13:30:50Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-sre: apply 2462687 wikitext text/x-wiki kyck86p5ooowy5kcaub25x3w9wbxjfi 2462688 2462687 2026-10-01T13:31:19Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-sre: apply 2462688 wikitext text/x-wiki 0o2kja1wwkgo0rn1y5ss1oe5nghssh8 2462689 2462688 2026-10-01T13:31:30Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-main: apply 2462689 wikitext text/x-wiki c55a7bjt4kgy8qq7s56uob2b2inqgb4 2462690 2462689 2026-10-01T13:31:44Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wikidata: apply 2462690 wikitext text/x-wiki 8zd5m8idmzksebz2j0690sh08vnbwpj 2462691 2462690 2026-10-01T13:32:00Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-main: apply 2462691 wikitext text/x-wiki 3ircgmh5e4lq1oa2gnyxjl8gnqwnyd8 2462692 2462691 2026-10-01T13:32:18Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wikidata: apply 2462692 wikitext text/x-wiki ffw3rcbzqvqegx6g8i1zo11371nv7ya 2462693 2462692 2026-10-01T13:32:23Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wmde: apply 2462693 wikitext text/x-wiki q7gdszwqha9foza7zqbv3s3cf6lgnl2 2462694 2462693 2026-10-01T13:32:53Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs-next: apply 2462694 wikitext text/x-wiki rbm3kmrgmssioi0396gfmnp58y0pxov 2462695 2462694 2026-10-01T13:33:01Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wmde: apply 2462695 wikitext text/x-wiki ai880zqny4xzoxx9c6gkgm3i209l2q3 2462696 2462695 2026-10-01T13:33:26Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wmde: apply 2462696 wikitext text/x-wiki 1d4ow8zvz1p5pj3d0dnwqzwvv50w6up 2462697 2462696 2026-10-01T13:33:30Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wmde: apply 2462697 wikitext text/x-wiki 294si1cd57zzoe307k6m74xurvqkjnd 2462698 2462697 2026-10-01T13:34:17Z Stashbot 7414 filippo@cumin1004: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudcephosd1055.eqiad.wmnet with OS bookworm 2462698 wikitext text/x-wiki pqh1tlcq0y72u26lcgrtv6alnryvip7 2462699 2462698 2026-10-01T13:34:36Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2462699 wikitext text/x-wiki mvkmy5xlt347s3rq5i7dz2t1lhcj04v 2462700 2462699 2026-10-01T13:36:29Z Stashbot 7414 awight@deploy1003: awight, migr: Continuing with deployment 2462700 wikitext text/x-wiki cqg5xu0vzsm43hoeutqgipoxxjgysfa 2462702 2462700 2026-10-01T13:40:32Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2462702 wikitext text/x-wiki mgvp4af1f9bdgy3x0v8w2j6nb1wgrtv 2462703 2462702 2026-10-01T13:41:25Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs-next: apply 2462703 wikitext text/x-wiki nljv7pjqp6hdka64op7p5l5ize321xl 2462704 2462703 2026-10-01T13:41:51Z Stashbot 7414 awight@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347422|instrument(AccountSetup): don't double count finish/skip clicks]], [[gerrit:1347423|stats(AccountSetup): Record onboarding ready time in Prometheus (T432810)]], [[gerrit:1347424|instrument(AccountSetup): Send exposure from server, send ready event (T432164)]] (duration: 22m 53s) 2462704 wikitext text/x-wiki fdl9ij7g83u4m1s2201z6wjhw3u5l92 2462705 2462704 2026-10-01T13:42:59Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2462705 wikitext text/x-wiki nvodbn0wuco0jeagrtu801klc1uqecc 2462706 2462705 2026-10-01T13:43:22Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2462706 wikitext text/x-wiki bnvteub0mhs692trgjzviem0wtvfhb8 2462707 2462706 2026-10-01T13:45:37Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2462707 wikitext text/x-wiki 680rybmnfw93tqo7ghcg8ctxh7xomfs 2462708 2462707 2026-10-01T13:46:00Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2462708 wikitext text/x-wiki 94ia88plwvyvcilqrtqrxd2ggs993b6 2462709 2462708 2026-10-01T13:47:08Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs: apply 2462709 wikitext text/x-wiki fd7lhgzr07qt85yhjyfr77qflokn73p 2462710 2462709 2026-10-01T13:47:41Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs: apply 2462710 wikitext text/x-wiki j7duwa9hgk6ctxend6t3d24hj6cldmt 2462711 2462710 2026-10-01T13:48:11Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs: apply 2462711 wikitext text/x-wiki t1kncps8lch94ahesf0etpnyaowjc01 2462712 2462711 2026-10-01T13:48:36Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs: apply 2462712 wikitext text/x-wiki 64n81egc9hbrsnkg5o72y7p5p2aiuyv 2462715 2462712 2026-10-01T13:53:21Z Stashbot 7414 awight@deploy1003: Started scap sync-world: Backport for [[gerrit:1347483|Revert "Use explicit numbers in reference lists and Parsoid rendering" (T439790)]] 2462715 wikitext text/x-wiki ak4avsjk3yllb5y36mk5930vy9gigi3 2462716 2462715 2026-10-01T13:57:40Z Stashbot 7414 awight@deploy1003: lilients, awight: Backport for [[gerrit:1347483|Revert "Use explicit numbers in reference lists and Parsoid rendering" (T439790)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462716 wikitext text/x-wiki dad0u7yaudl52b3tjwiu78kitvy9uie 2462717 2462716 2026-10-01T13:58:31Z Stashbot 7414 awight@deploy1003: lilients, awight: Continuing with deployment 2462717 wikitext text/x-wiki f44r8nivx519ez2ci3n7e1xe1s2nz7m 2462718 2462717 2026-10-01T14:01:49Z Stashbot 7414 dkertesz@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-haproxy (exit_code=0) rolling upgrade of HAProxy on A:cp-eqsin and A:cp - upgrade haproxy to 3.2.25-1 (T439667) 2462718 wikitext text/x-wiki ri5h0x5w5mqraaz5766mh8pdgwjm52o 2462719 2462718 2026-10-01T14:03:47Z Stashbot 7414 awight@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347483|Revert "Use explicit numbers in reference lists and Parsoid rendering" (T439790)]] (duration: 10m 26s) 2462719 wikitext text/x-wiki nltdn1c33ze7pcsxxa1velibjs57cl3 2462720 2462719 2026-10-01T14:04:27Z Stashbot 7414 awight: EU afternoon backport complete 2462720 wikitext text/x-wiki lm7kp609p7nusjenf9fatqyb43160pl 2462723 2462720 2026-10-01T14:06:26Z Stashbot 7414 jelto@cumin1004: START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: wikikube-worker-codfw@codfw 2462723 wikitext text/x-wiki 5cv6y5iuk04ex0xu2boo49gxekwi4u4 2462724 2462723 2026-10-01T14:09:41Z Stashbot 7414 jelto@cumin1004: START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs 2462724 wikitext text/x-wiki jfm2tgxn6xxhuvyh5sus21358dntuw4 2462725 2462724 2026-10-01T14:10:47Z Stashbot 7414 jelto@cumin1004: END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs 2462725 wikitext text/x-wiki o1syzazgm9m8pqn0o5j8ddpd6k8ncla 2462726 2462725 2026-10-01T14:10:48Z Stashbot 7414 jelto@cumin1004: END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: wikikube-worker-codfw@codfw 2462726 wikitext text/x-wiki pe6uk85knevz6qbxx1tb2eagevilu93 2462728 2462726 2026-10-01T14:14:40Z Stashbot 7414 jelto@cumin1004: START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: wikikube-worker-eqiad@eqiad 2462728 wikitext text/x-wiki bqq4tuvyen0jlcq4o8mozu987pg02ml 2462729 2462728 2026-10-01T14:18:23Z Stashbot 7414 jelto@cumin1004: START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-eqiad or A:lvs-secondary-eqiad) and A:bullseye and A:lvs 2462729 wikitext text/x-wiki njqn2skc12b3o7y071ni25fpq6aedz3 2462730 2462729 2026-10-01T14:19:34Z Stashbot 7414 jelto@cumin1004: END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-eqiad or A:lvs-secondary-eqiad) and A:bullseye and A:lvs 2462730 wikitext text/x-wiki l6ncjn0flk14urwcm88oim81racx2ie 2462731 2462730 2026-10-01T14:19:35Z Stashbot 7414 jelto@cumin1004: END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: wikikube-worker-eqiad@eqiad 2462731 wikitext text/x-wiki rmw9ytvoqmijh32ttpbkz1n4ct9c20n 2462734 2462731 2026-10-01T14:29:13Z Stashbot 7414 dpogorzelski@dns1004: START - running authdns-update 2462734 wikitext text/x-wiki ke0m2264u5gqt9mygxax6papsvtxzib 2462735 2462734 2026-10-01T14:31:36Z Stashbot 7414 dpogorzelski@dns1004: END - running authdns-update 2462735 wikitext text/x-wiki jp6y2az5oxio2avhp7xjsg3st1fcsso 2462736 2462735 2026-10-01T14:32:47Z Stashbot 7414 moritzm: installing e2fsprogs bugfix update from trixie point release 2462736 wikitext text/x-wiki 0ewg4gjxm8lnwoditv6jkypq8kdt1kk 2462737 2462736 2026-10-01T14:37:31Z Stashbot 7414 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1347638|NotificationController: Use lazyPush() for the delete job (T439626)]], [[gerrit:1347639|NotificationController: Use lazyPush() for the delete job (T439626)]] 2462737 wikitext text/x-wiki eta368asarg9cnny2byfm2yxjym9loe 2462738 2462737 2026-10-01T14:41:47Z Stashbot 7414 urbanecm@deploy1003: urbanecm: Backport for [[gerrit:1347638|NotificationController: Use lazyPush() for the delete job (T439626)]], [[gerrit:1347639|NotificationController: Use lazyPush() for the delete job (T439626)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462738 wikitext text/x-wiki 52rqb30ertlt16y60xb8c0pa1nmx6wy 2462739 2462738 2026-10-01T14:42:17Z Stashbot 7414 moritzm: installing zsh bugfix update from trixie point release 2462739 wikitext text/x-wiki 268h9kzu6rc3tmuibex1qpg051zz65i 2462740 2462739 2026-10-01T14:42:20Z Stashbot 7414 urbanecm@deploy1003: urbanecm: Continuing with deployment 2462740 wikitext text/x-wiki 6c7a82aqgkdn63wqkk6jid31rfofrbi 2462742 2462740 2026-10-01T14:47:36Z Stashbot 7414 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347638|NotificationController: Use lazyPush() for the delete job (T439626)]], [[gerrit:1347639|NotificationController: Use lazyPush() for the delete job (T439626)]] (duration: 10m 05s) 2462742 wikitext text/x-wiki 3xrkmujy2v7xf54pi4icbz8vv5589yz 2462743 2462742 2026-10-01T14:48:28Z Stashbot 7414 moritzm: installing busybox bugfix updates from trixie point release 2462743 wikitext text/x-wiki a6ahhx47jts1typc2jjyc5uxs4takm2 2462744 2462743 2026-10-01T14:56:28Z Stashbot 7414 moritzm: installing initramfs-tools bugfix updates from trixie point release 2462744 wikitext text/x-wiki 57z7zc3ugwnoxfs6q1bqbfhxtxmtpqw 2462746 2462744 2026-10-01T15:08:09Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=no; selector: name=tcp-proxy.* 2462746 wikitext text/x-wiki 7veabyq4rm91ne271xqoyt0jsk1ut9s 2462747 2462746 2026-10-01T15:08:57Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=yes; selector: name=tcp-proxy.* 2462747 wikitext text/x-wiki i2hz8wvhq49rrri2pkmkd9sfakyvs1e 2462749 2462747 2026-10-01T15:09:52Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1347652|Add missing alias for filerevision]], [[gerrit:1347650|Add missing alias for filerevision]] 2462749 wikitext text/x-wiki iy5hkz8kjfastz0nwy7ddabfh3jp685 2462750 2462749 2026-10-01T15:11:24Z Stashbot 7414 moritzm: installing postgresql security updates 2462750 wikitext text/x-wiki tbriulbcg4yktjff845fnulcy2u1bhp 2462751 2462750 2026-10-01T15:14:02Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1347652|Add missing alias for filerevision]], [[gerrit:1347650|Add missing alias for filerevision]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462751 wikitext text/x-wiki 52h4pvg1e335d7pji20y37va970c35w 2462752 2462751 2026-10-01T15:14:47Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2462752 wikitext text/x-wiki dq2tv62gc01k8l2xf4s47pyfv4qe7ii 2462753 2462752 2026-10-01T15:20:05Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347652|Add missing alias for filerevision]], [[gerrit:1347650|Add missing alias for filerevision]] (duration: 10m 12s) 2462753 wikitext text/x-wiki qppejtblpi2yy2m2wq65cwq5lqdwf6j 2462754 2462753 2026-10-01T15:25:10Z Stashbot 7414 dpogorzelski@dns1004: START - running authdns-update 2462754 wikitext text/x-wiki heq228i9x3s94qkbvlvimlg202b93yd 2462755 2462754 2026-10-01T15:26:20Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.cdn.roll-restart-reboot-tcp-proxy rolling restart_daemons on A:tcpproxy and A:tcpproxy 2462755 wikitext text/x-wiki b195yncl6ttxh2us8i53lzm9zkd0nef 2462756 2462755 2026-10-01T15:29:23Z Stashbot 7414 dpogorzelski@dns1004: END - running authdns-update 2462756 wikitext text/x-wiki ikseemrv2lukar8fe14t9gyebxdebk4 2462760 2462756 2026-10-01T15:32:11Z Stashbot 7414 moritzm: installing socat security updates 2462760 wikitext text/x-wiki 4l1vpuoqvfyvmhxa2dat0c45zw1sea9 2462761 2462760 2026-10-01T15:39:13Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.cdn.roll-restart-reboot-tcp-proxy (exit_code=0) rolling restart_daemons on A:tcpproxy and A:tcpproxy 2462761 wikitext text/x-wiki drhbnoh51l1baaf3yeom1tg3npqd7q1 2462763 2462761 2026-10-01T15:39:59Z Stashbot 7414 brett@cumin1004: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on A:cp-text_eqiad - 7.1.1-2~bpo13+wmf3 () 2462763 wikitext text/x-wiki 4wx1gd3l82pk8huzd04z1ddommxwj2i 2462764 2462763 2026-10-01T15:40:00Z Stashbot 7414 brett@cumin1004: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on A:cp-upload_eqiad - 7.1.1-2~bpo13+wmf3 () 2462764 wikitext text/x-wiki 9gopxtji33qz65ko7o9hycdnwg805uu 2462779 2462764 2026-10-01T16:12:10Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.dns.roll-restart-reboot-durum rolling restart_daemons on A:durum and A:durum 2462779 wikitext text/x-wiki efcx5yx258j1s67fyygl84irmag0h07 2462781 2462779 2026-10-01T16:25:09Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.dns.roll-restart-reboot-durum (exit_code=0) rolling restart_daemons on A:durum and A:durum 2462781 wikitext text/x-wiki hs4r0ofv7zzl3vhz1to27v6mp4xkktl 2462796 2462781 2026-10-01T17:01:15Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462796 wikitext text/x-wiki 3z1rlvws2p82ge8efcred2qglz666yz 2462797 2462796 2026-10-01T17:01:19Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462797 wikitext text/x-wiki 314dmk3q4cvv1zba429bpdgixbf8b0u 2462800 2462797 2026-10-01T17:12:00Z Stashbot 7414 tappof@deploy1003: helmfile [eqiad] START helmfile.d/admin 'sync'. 2462800 wikitext text/x-wiki 2qoajfa9yufcflg93fq2jxhjcdtp85l 2462801 2462800 2026-10-01T17:12:16Z Stashbot 7414 tappof@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'sync'. 2462801 wikitext text/x-wiki ea3fahy30hzdsldbieoyvfljd3ha0gg 2462802 2462801 2026-10-01T17:12:59Z Stashbot 7414 tappof@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'. 2462802 wikitext text/x-wiki ffh8e34ih2mjrcuq5xy0ftmy6do20m6 2462803 2462802 2026-10-01T17:13:35Z Stashbot 7414 tappof@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'. 2462803 wikitext text/x-wiki 1prye904o8q46p51o7btxvbj1hkc7hl 2462804 2462803 2026-10-01T17:16:17Z Stashbot 7414 kamila@deploy1003: Started scap sync-world: rebuild for T439100 2462804 wikitext text/x-wiki l85m3jvs2uqmwdee8pokfxj45t853oc 2462809 2462804 2026-10-01T17:49:06Z Stashbot 7414 kamila@deploy1003: Finished scap sync-world: rebuild for T439100 (duration: 33m 25s) 2462809 wikitext text/x-wiki e2l41z2tveb6e34ilyuprd3xlpe0844 2462810 2462809 2026-10-01T17:52:04Z Stashbot 7414 swfrench@deploy1003: Started scap deploy-service shellbox: Test deploy-service configuration for Shellbox 2462810 wikitext text/x-wiki 5r8t43p4ambfe05gjcme686rjrankmd 2462811 2462810 2026-10-01T17:52:11Z Stashbot 7414 spiderpig@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-syntaxhighlight: apply 2462811 wikitext text/x-wiki rjb7aweo3xgvzzot04ic1wi8mqx7tgz 2462812 2462811 2026-10-01T17:52:12Z Stashbot 7414 spiderpig@deploy1003: helmfile [staging] START helmfile.d/services/shellbox: apply 2462812 wikitext text/x-wiki hsew4rj4mlufn594vd9qfv27ixwubtc 2462813 2462812 2026-10-01T17:52:13Z Stashbot 7414 spiderpig@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-timeline: apply 2462813 wikitext text/x-wiki 8xv72tnw7d3vbt7w1lcija7flx70ugm 2462814 2462813 2026-10-01T17:52:14Z Stashbot 7414 spiderpig@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply 2462814 wikitext text/x-wiki h6j93h1vvfqvoiyixtiegxscfgmxbca 2462815 2462814 2026-10-01T17:52:15Z Stashbot 7414 spiderpig@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-constraints: apply 2462815 wikitext text/x-wiki jj53japhi1ziv1bmfsxd98v2ofqi56l 2462816 2462815 2026-10-01T17:52:16Z Stashbot 7414 spiderpig@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-video: apply 2462816 wikitext text/x-wiki pl3n0bow6v5wme6bionx0ol0f105kci 2462817 2462816 2026-10-01T17:52:17Z Stashbot 7414 spiderpig@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox: apply 2462817 wikitext text/x-wiki ponv1upibg1md3tbj3vpg2a0wtykzk9 2462818 2462817 2026-10-01T17:52:18Z Stashbot 7414 spiderpig@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply 2462818 wikitext text/x-wiki i5gtofxo9k5kvk6v1k4l2c2o46vfp6n 2462819 2462818 2026-10-01T17:52:19Z Stashbot 7414 spiderpig@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-syntaxhighlight: apply 2462819 wikitext text/x-wiki 4ka16uh1ja0ys6qq9abs45g445agggi 2462820 2462819 2026-10-01T17:52:20Z Stashbot 7414 spiderpig@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-constraints: apply 2462820 wikitext text/x-wiki mhjzl3l0wlutpvg5qj68ws25dn3xn4b 2462821 2462820 2026-10-01T17:52:21Z Stashbot 7414 spiderpig@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-video: apply 2462821 wikitext text/x-wiki 3rjzzlvl8iza1t6w87f2oqxnvaam1e7 2462822 2462821 2026-10-01T17:52:22Z Stashbot 7414 spiderpig@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-timeline: apply 2462822 wikitext text/x-wiki shnpnt20jedjejthwal9hyojn89a46u 2462823 2462822 2026-10-01T17:53:14Z Stashbot 7414 spiderpig@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-timeline: apply 2462823 wikitext text/x-wiki erh4cityunazj1mgw7fpku08scyqna6 2462824 2462823 2026-10-01T17:53:16Z Stashbot 7414 spiderpig@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply 2462824 wikitext text/x-wiki fe9uzet3yjrj6pykk21lzxu79yv773j 2462825 2462824 2026-10-01T17:53:16Z Stashbot 7414 spiderpig@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-constraints: apply 2462825 wikitext text/x-wiki rv41zxxskambhfvb7c0r1ptl5zhzixx 2462826 2462825 2026-10-01T17:53:17Z Stashbot 7414 spiderpig@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-video: apply 2462826 wikitext text/x-wiki mhtnazoq38b1ay4wg35i9gsfqt71aps 2462827 2462826 2026-10-01T17:53:18Z Stashbot 7414 spiderpig@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-syntaxhighlight: apply 2462827 wikitext text/x-wiki 07a1jumff0up2mx53dkny6z99iuckzd 2462828 2462827 2026-10-01T17:53:20Z Stashbot 7414 spiderpig@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox: apply 2462828 wikitext text/x-wiki t5swew3rm3sgrbhwsezfw7qrvcuz8pb 2462829 2462828 2026-10-01T17:53:21Z Stashbot 7414 spiderpig@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox: apply 2462829 wikitext text/x-wiki l83jhc81wv0lqzmkseu2x3lczy4rpzj 2462830 2462829 2026-10-01T17:53:22Z Stashbot 7414 spiderpig@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-video: apply 2462830 wikitext text/x-wiki bp80yh5ypb8puwh90ynb9afo71mi8ia 2462831 2462830 2026-10-01T17:53:23Z Stashbot 7414 spiderpig@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-syntaxhighlight: apply 2462831 wikitext text/x-wiki j199w43i0giwd839i7thg12qipkva48 2462832 2462831 2026-10-01T17:53:24Z Stashbot 7414 spiderpig@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-constraints: apply 2462832 wikitext text/x-wiki 2z19hr2fj3n1hafpzs35o7hjt60ws1p 2462833 2462832 2026-10-01T17:53:25Z Stashbot 7414 spiderpig@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply 2462833 wikitext text/x-wiki s2agm5dbwxllqhhcqcpul426jlnh3y7 2462834 2462833 2026-10-01T17:53:26Z Stashbot 7414 spiderpig@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-timeline: apply 2462834 wikitext text/x-wiki hh97d3xgboeb9ld2byie3p7epipibfd 2462835 2462834 2026-10-01T17:54:48Z Stashbot 7414 spiderpig@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-constraints: apply 2462835 wikitext text/x-wiki qqs799yjpvj23qznxmixdl76fs1rqzy 2462836 2462835 2026-10-01T17:54:50Z Stashbot 7414 spiderpig@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-timeline: apply 2462836 wikitext text/x-wiki jsqdcr8tncxz3rk10hol7505rhg679v 2462837 2462836 2026-10-01T17:54:51Z Stashbot 7414 spiderpig@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply 2462837 wikitext text/x-wiki 0rv5wovczv2t3fwzdldzb5kw3yx8e4a 2462838 2462837 2026-10-01T17:54:52Z Stashbot 7414 spiderpig@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox: apply 2462838 wikitext text/x-wiki cft8qy207b7t7ns4vnp086xwbn8heq6 2462839 2462838 2026-10-01T17:54:53Z Stashbot 7414 spiderpig@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-syntaxhighlight: apply 2462839 wikitext text/x-wiki bne8t0n1q5b5lfn4t4770a5xwpzicff 2462840 2462839 2026-10-01T17:54:54Z Stashbot 7414 spiderpig@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-video: apply 2462840 wikitext text/x-wiki 78651gyk00tt1nzvraov9k3nx9ldll5 2462841 2462840 2026-10-01T17:54:55Z Stashbot 7414 spiderpig@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox: apply 2462841 wikitext text/x-wiki 6ox0vbsgh0fvvudqq0w7q630gz30tt8 2462842 2462841 2026-10-01T17:54:56Z Stashbot 7414 spiderpig@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-constraints: apply 2462842 wikitext text/x-wiki 3jyj1nxt8zl8gy1ujpsmtf0jta7emgf 2462843 2462842 2026-10-01T17:54:59Z Stashbot 7414 spiderpig@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-timeline: apply 2462843 wikitext text/x-wiki a73a8hsu9un78ricb54s3ndilrbek6k 2462844 2462843 2026-10-01T17:55:01Z Stashbot 7414 spiderpig@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply 2462844 wikitext text/x-wiki 0734rfe6rzsv0ylbpnsbci6p8wcns8g 2462845 2462844 2026-10-01T17:55:02Z Stashbot 7414 spiderpig@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-syntaxhighlight: apply 2462845 wikitext text/x-wiki ei5p85al3wiptb4r2g01rke5uv21tl6 2462846 2462845 2026-10-01T17:55:04Z Stashbot 7414 spiderpig@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-video: apply 2462846 wikitext text/x-wiki rcqdmebhwgbdoj4drb9wx4z1v7adilc 2462847 2462846 2026-10-01T17:55:32Z Stashbot 7414 swfrench@deploy1003: Finished scap deploy-service shellbox: Test deploy-service configuration for Shellbox (duration: 03m 28s) 2462847 wikitext text/x-wiki 5apw7ta5oli7j43du8z3wfmdded38ca 2462850 2462847 2026-10-01T18:12:19Z Stashbot 7414 brett@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on A:cp-upload_eqiad - 7.1.1-2~bpo13+wmf3 () 2462850 wikitext text/x-wiki pcvc74xidwt1p25javw6utjm3vxe6vd 2462851 2462850 2026-10-01T18:16:25Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462851 wikitext text/x-wiki lm3twbp0ajq0hqsd35gcoajvpdhfv7c 2462852 2462851 2026-10-01T18:16:29Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462852 wikitext text/x-wiki s25ketdqyuyv41cpxfl770ut25l0fig 2462857 2462852 2026-10-01T18:25:21Z Stashbot 7414 brennen: 1.47.0-wmf.22 train status (T438218): logs kind of a mess, but no current blockers, rolling to all wikis 2462857 wikitext text/x-wiki sfpni3byetzxx9czn86amohhzipfays 2462861 2462857 2026-10-01T18:36:23Z Stashbot 7414 jhathaway: disabling puppet fleet wide to test https://gerrit.wikimedia.org/r/c/operations/puppet/+/1347689 2462861 wikitext text/x-wiki nvnvgp1h3erxjfc75tbozb5tn6lltg9 2462862 2462861 2026-10-01T18:40:53Z Stashbot 7414 brennen@deploy1003: rebuilt and synchronized wikiversions files: group2 to 1.47.0-wmf.22 refs T438218 2462862 wikitext text/x-wiki 9j0yqnpy3ze8jnml8885797q1ru29uw 2462863 2462862 2026-10-01T18:42:00Z Stashbot 7414 brett@cumin1004: END (FAIL) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=1) rolling upgrade of Varnish on A:cp-text_eqiad - 7.1.1-2~bpo13+wmf3 () 2462863 wikitext text/x-wiki fnfqqr3arkci43b2dfxjzrmhlhn2505 2462871 2462863 2026-10-01T19:02:53Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462871 wikitext text/x-wiki o2oos1t6m8u0xee8dpju642yee4gjw4 2462872 2462871 2026-10-01T19:02:58Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462872 wikitext text/x-wiki pd43sxdueyydfvsfq21w6xxf4iqzi0x 2462876 2462872 2026-10-01T19:19:38Z Stashbot 7414 jhathaway: puppet re-enabled 2462876 wikitext text/x-wiki otvpty7w18rgeg00v16yk3gpex147z5 2462880 2462876 2026-10-01T19:32:10Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1347672|Move reads/writes of wbc_entity_usage to x1 for s5 wikis (T439753)]] 2462880 wikitext text/x-wiki 7m2x3uda5zaddz72h8532x7yu3m6bjz 2462881 2462880 2026-10-01T19:36:12Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1347672|Move reads/writes of wbc_entity_usage to x1 for s5 wikis (T439753)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462881 wikitext text/x-wiki 7d6hi0eyltuzye1wz4j8ewswkngw2cq 2462882 2462881 2026-10-01T19:36:50Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2462882 wikitext text/x-wiki t56pj0t58qc33gzxdudv8namqw4vx1m 2462883 2462882 2026-10-01T19:42:11Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347672|Move reads/writes of wbc_entity_usage to x1 for s5 wikis (T439753)]] (duration: 10m 01s) 2462883 wikitext text/x-wiki f53mzdmzbyl5atlfjxmc62nc19vjcl9 2462884 2462883 2026-10-01T19:42:31Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1347711|Add missing select of fr_timestamp]] 2462884 wikitext text/x-wiki smffr7vlzgrvtz1t4ws9znzou8n8mdm 2462885 2462884 2026-10-01T19:46:31Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1347711|Add missing select of fr_timestamp]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462885 wikitext text/x-wiki i45hnfvc7mj1048g1r7fwizaoouf177 2462886 2462885 2026-10-01T19:46:59Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2462886 wikitext text/x-wiki noashn6pc3dtrcafnbupnrimjm7o7i8 2462889 2462886 2026-10-01T19:51:38Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462889 wikitext text/x-wiki iby75zj7u3h1pwzdw8wqwzad8r6qmbf 2462890 2462889 2026-10-01T19:51:42Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462890 wikitext text/x-wiki 7p9y9udkcqktdhmer1wadw28s6ayaz1 2462891 2462890 2026-10-01T19:52:12Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347711|Add missing select of fr_timestamp]] (duration: 09m 40s) 2462891 wikitext text/x-wiki sx2sd1lnb4yphmkm05zpo0mklx8ui72 2462893 2462891 2026-10-01T20:02:55Z Stashbot 7414 jhathaway: flipping flag to disable legacy facts in puppet prod, attempt two 2462893 wikitext text/x-wiki o15jo0jpjcfexmptuzimd3ptd0iydrx 2462894 2462893 2026-10-01T20:14:34Z Stashbot 7414 cjming@deploy1003: Started scap sync-world: Backport for [[gerrit:1347606|fix(ReadingRecommendations): Update button style to primary (T436332)]], [[gerrit:1347712|instrument(WelcomeSurvey): align motivation event with AccountSetup (T439887)]], [[gerrit:1347743|instrument(AccountSetup): Record setup motivation on `page_visit` (T439217)]] 2462894 wikitext text/x-wiki djg5qgge6wcuhzot1wmfnlahpigu5s4 2462895 2462894 2026-10-01T20:18:27Z Stashbot 7414 cjming@deploy1003: cjming, volker-e: Backport for [[gerrit:1347606|fix(ReadingRecommendations): Update button style to primary (T436332)]], [[gerrit:1347712|instrument(WelcomeSurvey): align motivation event with AccountSetup (T439887)]], [[gerrit:1347743|instrument(AccountSetup): Record setup motivation on `page_visit` (T439217)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can 2462895 wikitext text/x-wiki p6pf1cugyh0f2i88epddbl7ye8zgg5q 2462896 2462895 2026-10-01T20:21:13Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hadoop.roll-restart-workers restart workers for Hadoop test cluster: Roll restart of jvm daemons for openjdk upgrade. 2462896 wikitext text/x-wiki jo2yo3b83uozrc4pgo3i1s895b8q6xc 2462897 2462896 2026-10-01T20:26:52Z Stashbot 7414 bking@cumin2003: END (ERROR) - Cookbook sre.hadoop.roll-restart-workers (exit_code=97) restart workers for Hadoop test cluster: Roll restart of jvm daemons for openjdk upgrade. 2462897 wikitext text/x-wiki 7r70vyf6n0cbdz1937huhye2ym76090 2462898 2462897 2026-10-01T20:27:10Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hadoop.roll-restart-workers restart workers for Hadoop test cluster: Roll restart of jvm daemons for openjdk upgrade. 2462898 wikitext text/x-wiki i773kdg5mygkgee9zyacrwuo9yycjke 2462899 2462898 2026-10-01T20:27:29Z Stashbot 7414 cjming@deploy1003: cjming, volker-e: Continuing with deployment 2462899 wikitext text/x-wiki 85eo5au1thu08h4l48abysuayhz5j7k 2462900 2462899 2026-10-01T20:31:54Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hadoop.roll-restart-workers (exit_code=0) restart workers for Hadoop test cluster: Roll restart of jvm daemons for openjdk upgrade. 2462900 wikitext text/x-wiki c7e9j1c59msvvnqd8o1v0r6h90tnpjf 2462901 2462900 2026-10-01T20:33:02Z Stashbot 7414 cjming@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347606|fix(ReadingRecommendations): Update button style to primary (T436332)]], [[gerrit:1347712|instrument(WelcomeSurvey): align motivation event with AccountSetup (T439887)]], [[gerrit:1347743|instrument(AccountSetup): Record setup motivation on `page_visit` (T439217)]] (duration: 18m 28s) 2462901 wikitext text/x-wiki suosfixra9q5omujicksh2x1sfp0tal 2462902 2462901 2026-10-01T20:34:42Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hadoop.roll-restart-workers restart workers for Hadoop test cluster: Roll restart of jvm daemons for openjdk upgrade. 2462902 wikitext text/x-wiki om13vaktmxrbgxxnx5pjekffxv8p9n8 2462903 2462902 2026-10-01T20:35:07Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hadoop.roll-restart-workers (exit_code=0) restart workers for Hadoop test cluster: Roll restart of jvm daemons for openjdk upgrade. 2462903 wikitext text/x-wiki 1futmitjx6p8t68peujczw04ei881uo 2462904 2462903 2026-10-01T20:37:54Z Stashbot 7414 mlitn@deploy1003: Started scap sync-world: Backport for [[gerrit:1347686|Lose DB call for page prop (T439182)]], [[gerrit:1347687|Extract images in parser post-processing (T439182)]], [[gerrit:1347729|Make carousel open MMV directly (T439611)]], [[gerrit:1347735|Removes image preview from Treatment 1-2 (T439611)]] 2462904 wikitext text/x-wiki j226mk7pmp617nvqa6wpvef2k2v6g75 2462905 2462904 2026-10-01T20:41:55Z Stashbot 7414 mlitn@deploy1003: mlitn, ksarabia: Backport for [[gerrit:1347686|Lose DB call for page prop (T439182)]], [[gerrit:1347687|Extract images in parser post-processing (T439182)]], [[gerrit:1347729|Make carousel open MMV directly (T439611)]], [[gerrit:1347735|Removes image preview from Treatment 1-2 (T439611)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462905 wikitext text/x-wiki fpw4t500o05oah2l476symp1mw0hbnp 2462906 2462905 2026-10-01T20:42:43Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462906 wikitext text/x-wiki ljgqcupq5vc54eh25efihpuj8goc2d6 2462907 2462906 2026-10-01T20:42:47Z Stashbot 7414 ebernhardson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply 2462907 wikitext text/x-wiki loxlnwftc05h7xh8gr8dngdkfbylqw5 2462908 2462907 2026-10-01T20:43:33Z Stashbot 7414 mlitn@deploy1003: mlitn, ksarabia: Continuing with deployment 2462908 wikitext text/x-wiki 1czwnx7eqfs8eh49ornu60l2xsszf4a 2462909 2462908 2026-10-01T20:44:36Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hadoop.roll-restart-workers restart workers for Hadoop test cluster: Roll restart of jvm daemons for openjdk upgrade. 2462909 wikitext text/x-wiki e8k4lyjw4i8gmphl15ql2aizgbqt9h6 2462910 2462909 2026-10-01T20:45:46Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hadoop.roll-restart-workers (exit_code=0) restart workers for Hadoop test cluster: Roll restart of jvm daemons for openjdk upgrade. 2462910 wikitext text/x-wiki i2qsuiofg54a72xftki9idw6z28l9or 2462911 2462910 2026-10-01T20:46:30Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hadoop.roll-restart-workers restart workers for Hadoop test cluster: Roll restart of jvm daemons for openjdk upgrade. 2462911 wikitext text/x-wiki 8x84yud0p3eqxmb6r6wazezr7m56gtb 2462912 2462911 2026-10-01T20:48:50Z Stashbot 7414 mlitn@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347686|Lose DB call for page prop (T439182)]], [[gerrit:1347687|Extract images in parser post-processing (T439182)]], [[gerrit:1347729|Make carousel open MMV directly (T439611)]], [[gerrit:1347735|Removes image preview from Treatment 1-2 (T439611)]] (duration: 10m 56s) 2462912 wikitext text/x-wiki nn09d4rr5kua68fqkxjyaf7tjoxe0aj 2462913 2462912 2026-10-01T20:53:41Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hadoop.roll-restart-workers (exit_code=0) restart workers for Hadoop test cluster: Roll restart of jvm daemons for openjdk upgrade. 2462913 wikitext text/x-wiki ivhrrx6c5th5qi1g5e0289m899u2pjr 2462914 2462913 2026-10-01T20:55:06Z Stashbot 7414 tappof: Rebalancing is in progress on kafka-logging in codfw to evacuate kafka-logging2001, kafka-logging2002 and kafka-logging2003 - T432444 2462914 wikitext text/x-wiki ptp8vaa2kl6tzxl4xjl98qh9wgxkiot 2462915 2462914 2026-10-01T20:55:43Z Stashbot 7414 brennen@deploy1003: Started scap sync-world: Backport for [[gerrit:1347692|Fix UTF-8 corruption in newline sanitization (T431960 T426631)]] 2462915 wikitext text/x-wiki 1oq0mft04fepwuc716g2o5c9y4q3srg 2462916 2462915 2026-10-01T20:59:38Z Stashbot 7414 brennen@deploy1003: brennen, srapoj: Backport for [[gerrit:1347692|Fix UTF-8 corruption in newline sanitization (T431960 T426631)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462916 wikitext text/x-wiki 5puqomi6625hszzl5od1qmc1gm8ep79 2462917 2462916 2026-10-01T21:01:00Z Stashbot 7414 brennen@deploy1003: brennen, srapoj: Continuing with deployment 2462917 wikitext text/x-wiki rxyb3kyeyikn1de3ustii0us1zmv13k 2462918 2462917 2026-10-01T21:06:25Z Stashbot 7414 brennen@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347692|Fix UTF-8 corruption in newline sanitization (T431960 T426631)]] (duration: 10m 43s) 2462918 wikitext text/x-wiki jbpclvpry3vq9p16fsbvcapci38qzdk 2462919 2462918 2026-10-01T21:13:49Z Stashbot 7414 egardner@deploy1003: Started scap sync-world: Backport for [[gerrit:1347718|Preferred Sources: Move all eligibility checks to the client (T436692)]] 2462919 wikitext text/x-wiki ka0vgw86qdvle1ney4owshwpivvidew 2462920 2462919 2026-10-01T21:17:47Z Stashbot 7414 egardner@deploy1003: egardner: Backport for [[gerrit:1347718|Preferred Sources: Move all eligibility checks to the client (T436692)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462920 wikitext text/x-wiki d0pxzkzgjtgyazqfn8dho2r3cmmq1kc 2462921 2462920 2026-10-01T21:19:33Z Stashbot 7414 egardner@deploy1003: egardner: Continuing with deployment 2462921 wikitext text/x-wiki 7d8oxitbso23ho2vkcd13khhi5cmuw5 2462922 2462921 2026-10-01T21:24:51Z Stashbot 7414 egardner@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347718|Preferred Sources: Move all eligibility checks to the client (T436692)]] (duration: 11m 02s) 2462922 wikitext text/x-wiki q4fc972rtahqli4pmnw4n00gmu9ed3q 2462923 2462922 2026-10-01T21:33:28Z Stashbot 7414 tappof: Rebalancing has been completed on kafka-logging in codfw: kafka-logging200[1-3] are no longer assigned any partitions — T432444 2462923 wikitext text/x-wiki bktkgr46w8ufhatvd8y7v0q28hum25e 2462924 2462923 2026-10-01T21:46:03Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1347806|WikimediaAntiAbuse: Enable personal info content policy on dewiki (T439928)]] 2462924 wikitext text/x-wiki dvyyhv9re23l6zw3g7jn61k0gtb8qbe 2462925 2462924 2026-10-01T21:50:03Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1347806|WikimediaAntiAbuse: Enable personal info content policy on dewiki (T439928)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2462925 wikitext text/x-wiki qw95712ycrlfvyu85c9q8fds7f1mtws 2462926 2462925 2026-10-01T21:51:06Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2462926 wikitext text/x-wiki hwqhe47qpvyidqln6698ccqsjlew8rs 2462927 2462926 2026-10-01T21:56:23Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1347806|WikimediaAntiAbuse: Enable personal info content policy on dewiki (T439928)]] (duration: 10m 19s) 2462927 wikitext text/x-wiki 7qafia17m4yn52lze7bsaiyk8fymxgr 2462933 2462927 2026-10-01T23:04:02Z Stashbot 7414 brett@cumin1004: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on P{cp11[08,10,12,14].eqiad.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2462933 wikitext text/x-wiki hi4kefahi2u7aa9xyyxvn7z1yo4t60q 2462934 2462933 2026-10-01T23:38:16Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp1108.* 2462934 wikitext text/x-wiki 5qngcekxe3dia6o1tfi05izvhlsbm32 2462939 2462934 2026-10-02T00:39:01Z Stashbot 7414 brett@cumin1004: END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on P{cp11[08,10,12,14].eqiad.wmnet} and A:cp - 7.1.1-2~bpo13+wmf3 () 2462939 wikitext text/x-wiki inuxwg90x1yfdzap5mzyn3zkce8js5p 2462940 2462939 2026-10-02T00:52:32Z Stashbot 7414 dzahn@deploy1003: helmfile [staging] START helmfile.d/services/miscweb: apply 2462940 wikitext text/x-wiki rz678u9bz523pzi5crl6xcpib0qkhva 2462941 2462940 2026-10-02T00:53:13Z Stashbot 7414 dzahn@deploy1003: helmfile [staging] DONE helmfile.d/services/miscweb: apply 2462941 wikitext text/x-wiki k87qec9uohequvlc43ao2puid3di5bs 2462942 2462941 2026-10-02T00:53:30Z Stashbot 7414 dzahn@deploy1003: helmfile [codfw] START helmfile.d/services/miscweb: apply 2462942 wikitext text/x-wiki 8jhse66mo9dob0g29ihdjsgcdqttu7k 2462943 2462942 2026-10-02T00:54:07Z Stashbot 7414 dzahn@deploy1003: helmfile [codfw] DONE helmfile.d/services/miscweb: apply 2462943 wikitext text/x-wiki ts0fa2qmltdv3m82udvjf1eepm9xyaa 2462944 2462943 2026-10-02T00:54:15Z Stashbot 7414 dzahn@deploy1003: helmfile [eqiad] START helmfile.d/services/miscweb: apply 2462944 wikitext text/x-wiki 9jtuq9qesn270y1h01d0auua5w1ndy7 2462945 2462944 2026-10-02T00:54:39Z Stashbot 7414 dzahn@deploy1003: helmfile [eqiad] DONE helmfile.d/services/miscweb: apply 2462945 wikitext text/x-wiki 1cqhelu7fczbi3rn3fuet0ngvakk60o 2462947 2462945 2026-10-02T02:00:50Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2462947 wikitext text/x-wiki efnyruswkwlffnxjio9zr0nyxorqr0x 2462948 2462947 2026-10-02T02:08:27Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 07m 36s) 2462948 wikitext text/x-wiki 7cfkdtguzrv548um0ctpl2b2w0qq9k5 2462949 2462948 2026-10-02T02:39:27Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1238.eqiad.wmnet with reason: maint 2462949 wikitext text/x-wiki d6pffas6yxh2h055dhqsie64jqeoqrp 2462964 2462949 2026-10-02T06:55:45Z Stashbot 7414 jmm@cumin2003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Dima koushha out of all services on: 2448 hosts 2462964 wikitext text/x-wiki rsg9fvgy1dywpudfybao8avpr32zxut 2462965 2462964 2026-10-02T07:47:55Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1006.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART (NO reboot will be performed automatically) 2462965 wikitext text/x-wiki hmmj4cmmcc4kfvwsoqh8vnjpg6imgn8 2462966 2462965 2026-10-02T07:52:02Z Stashbot 7414 oblivian@cumin1004: END (ERROR) - Cookbook sre.hosts.provision (exit_code=97) for host wikikube-worker1006.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART (NO reboot will be performed automatically) 2462966 wikitext text/x-wiki 0m49amkz9v62lsv9j8f81w0yu77iep1 2462967 2462966 2026-10-02T07:52:52Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1006.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART (NO reboot will be performed automatically) 2462967 wikitext text/x-wiki h66pxg562xfz98qx2lyt788nkfkkela 2462968 2462967 2026-10-02T07:53:18Z Stashbot 7414 oblivian@cumin1004: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker1006.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART (NO reboot will be performed automatically) 2462968 wikitext text/x-wiki ihrmy1t0ygcw03z219xwt9c3x4p7dx0 2462969 2462968 2026-10-02T07:58:37Z Stashbot 7414 oblivian@cumin1004: START - Cookbook sre.hosts.provision for host wikikube-worker1006.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART (NO reboot will be performed automatically) 2462969 wikitext text/x-wiki cfz7ascvtigwgn0tlrn9dewp4k2a3gf 2462972 2462969 2026-10-02T08:47:08Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2462972 wikitext text/x-wiki ocmez7oygx99tlsou8uopfmnmjbt5rn 2462973 2462972 2026-10-02T08:48:16Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2462973 wikitext text/x-wiki j5ybsrimyd4bycab1webegrau1tcdhq 2462974 2462973 2026-10-02T08:48:51Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2462974 wikitext text/x-wiki 6imxvpl6hcdbuzn30p1teb44yp406m5 2462975 2462974 2026-10-02T08:49:49Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2462975 wikitext text/x-wiki 0h3g1ofpr9hdp46dkw60p7w290e13de 2462976 2462975 2026-10-02T08:56:10Z Stashbot 7414 elukey@cumin1004: START - Cookbook sre.hosts.provision for host sretest2001.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2462976 wikitext text/x-wiki 4cjy4timnuj7cou1hdyjdeaod84t65f 2462977 2462976 2026-10-02T09:06:26Z Stashbot 7414 elukey@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host sretest2001.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2462977 wikitext text/x-wiki 76h7sfyhqto7x2teeisbooqo990jnn5 2462978 2462977 2026-10-02T09:30:57Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.dns.netbox 2462978 wikitext text/x-wiki jwp0gg1cbqxkyar07s2da86v7x12uvt 2462979 2462978 2026-10-02T09:41:57Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Re-commissioning ml-cache2001 - btullis@cumin1004" 2462979 wikitext text/x-wiki izey0pij0t5kr6mh5vy1b71g1p9ayut 2462980 2462979 2026-10-02T09:45:45Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Re-commissioning ml-cache2001 - btullis@cumin1004" 2462980 wikitext text/x-wiki 7cl74h996xyl1qo2v8y1nfsjks0oi9e 2462981 2462980 2026-10-02T09:45:47Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2462981 wikitext text/x-wiki eoinqzhzlp36ewe1y5vnild4myw4ufp 2462982 2462981 2026-10-02T09:53:35Z Stashbot 7414 oblivian@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker1006.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART (NO reboot will be performed automatically) 2462982 wikitext text/x-wiki 26zfsfyi9mvtt9ry4dvs2teo1xnmzwc 2462983 2462982 2026-10-02T10:11:13Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'llm' . 2462983 wikitext text/x-wiki m65kd6f7helld5nfnl284bcnk5pe2ib 2462984 2462983 2026-10-02T10:11:30Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'llm' . 2462984 wikitext text/x-wiki j45v75f5pcyjro1k9g91i30e8guos4z 2462985 2462984 2026-10-02T10:12:13Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'llm' . 2462985 wikitext text/x-wiki tq8xuyut7zyyenenbgzqhtnw26sfyrq 2462986 2462985 2026-10-02T10:12:59Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'llm' . 2462986 wikitext text/x-wiki 8fzk4r1olp37hpqljh4f7io5wfwhmc8 2462987 2462986 2026-10-02T10:14:26Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.provision for host ml-cache2001.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2462987 wikitext text/x-wiki a2b8vtm44a0b8xsoj1npuzlml83ss3s 2462988 2462987 2026-10-02T10:22:22Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ml-cache2001.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2462988 wikitext text/x-wiki p0xcuw98d41tjno7osugpizu2bbrpso 2462989 2462988 2026-10-02T10:23:04Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reimage for host ml-cache2001.codfw.wmnet with OS bookworm 2462989 wikitext text/x-wiki adanviksuoz4jsz4oao4h585iovr3ss 2462990 2462989 2026-10-02T10:29:00Z Stashbot 7414 moritzm: installing Linux 6.12.111 on Trixie hosts 2462990 wikitext text/x-wiki sue092acwct0rxssskt98h924555iz7 2462991 2462990 2026-10-02T10:39:30Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ml-cache2001.codfw.wmnet with reason: host reimage 2462991 wikitext text/x-wiki 7ch6upt3263v62ngsb67z1b2j4iyv49 2462992 2462991 2026-10-02T10:42:23Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ml-cache2001.codfw.wmnet with reason: host reimage 2462992 wikitext text/x-wiki 5a4pty4d7hq66kcz4lcm4wlzof9omzd 2462993 2462992 2026-10-02T10:47:09Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.dns.netbox 2462993 wikitext text/x-wiki 9lfvxzsua27z10e3zlxg1536z8s8llk 2462994 2462993 2026-10-02T10:54:56Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Re-adding ml-cache2002 - btullis@cumin1004" 2462994 wikitext text/x-wiki jx9znq9s4rk9vlou634uwndythiihl8 2462995 2462994 2026-10-02T10:55:00Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Re-adding ml-cache2002 - btullis@cumin1004" 2462995 wikitext text/x-wiki ox3btwwxcerjwokm58rbnowa1jz60yr 2462996 2462995 2026-10-02T10:55:01Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2462996 wikitext text/x-wiki 3e4ceam1tpqbbforczl4u19gjbdrjrt 2462997 2462996 2026-10-02T10:58:30Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - btullis@cumin1004" 2462997 wikitext text/x-wiki 67wwdtdz90ybhoqbxnpfpvzka1toao9 2463001 2462997 2026-10-02T11:08:09Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - btullis@cumin1004" 2463001 wikitext text/x-wiki hgkqg2h38kz2ov6udq98zqpzdg32112 2463002 2463001 2026-10-02T11:08:11Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ml-cache2001.codfw.wmnet with OS bookworm 2463002 wikitext text/x-wiki i2qla7zs7fbjgpuu5xp2gfs75el09xm 2463003 2463002 2026-10-02T11:37:33Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.dns.netbox 2463003 wikitext text/x-wiki 3dlgv8qd349vz36338j26spnvu0qmkc 2463004 2463003 2026-10-02T11:40:08Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2463004 wikitext text/x-wiki 87yguajjbzkomj78h0ekq481naqpnd3 2463005 2463004 2026-10-02T11:40:23Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.provision for host ml-cache2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2463005 wikitext text/x-wiki 4kaddu57gffgejdgqu3dxf2uutyjz84 2463006 2463005 2026-10-02T11:56:49Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ml-cache2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2463006 wikitext text/x-wiki b220ewt46q0viv13upnxyahzjzz8817 2463007 2463006 2026-10-02T11:58:44Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.provision for host ml-cache2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2463007 wikitext text/x-wiki gjov36z4b0qbx28z2vzhi6m831gzz2e 0 17290 2462713 2462484 2026-10-01T13:49:37Z Stashbot 7414 jakob_WMDE: ssh integration-castor06.integration.eqiad1.wikimedia.cloud sudo -u jenkins-deploy rm -rf /srv/castor/mediawiki-core/master/quibble-with-Wikibase-extensions-browser-tests-only-vendor-php83 2462713 wikitext text/x-wiki ojqpbrsz3rznt7fduvtciezxr9x3rpx 2462714 2462713 2026-10-01T13:50:34Z Stashbot 7414 hashar: gerrit: deleting orphan comment not showing up on the change UI. git -C /srv/gerrit/git/All-Users.git update-ref -d refs/draft-comments/39/1341139/24 2462714 wikitext text/x-wiki 658tk42b0h7lkhkov3kmlnnpguypmn9 0 20225 2462758 2460040 2026-10-01T15:31:25Z APizzata-WMF 47941 2462758 wikitext text/x-wiki 26qx3eri2i3t39zngjauz5wyf0xae8x 0 20832 2462787 2451786 2026-10-01T16:35:14Z Ameisenigel 16760 /* Creation requests */ fix 2462787 wikitext text/x-wiki nzhmt8njnukuvha11lblusezvdwnvly 0 22139 2462928 2459069 2026-10-01T22:01:09Z Neil Shah-Quinn (WMF) 36601 Copyedit info about 3.1 vs. 3.5. Move information on custom shuffler versions from intro to new section. 2462928 wikitext text/x-wiki mzvcok33k2rnrgncuuafp1bk9e8w7bi 2462932 2462928 2026-10-01T23:03:38Z Neil Shah-Quinn (WMF) 36601 Completely reorganize around the main sections "interfaces", "guides", "basic configuration", and "advanced configuration". Copyedit. Remove outdated information. 2462932 wikitext text/x-wiki t796yfhme4c993ont5izgyxcy0p4gkn 2462935 2462932 2026-10-01T23:40:48Z Neil Shah-Quinn (WMF) 36601 Document Zstandard errors and how use to Conda-Analytics-Next to work around them. Document Spark 3.5 CLIs. 2462935 wikitext text/x-wiki honvumibtlpze8r8s9taujb9obetf8b 2462936 2462935 2026-10-01T23:43:30Z Neil Shah-Quinn (WMF) 36601 Remove blank lines 2462936 wikitext text/x-wiki q41bukvzjjbbgtde60yntdk2z4zacc0 2462955 2462936 2026-10-02T03:51:27Z Neil Shah-Quinn (WMF) 36601 Copyedit 2462955 wikitext text/x-wiki 5x07ly1x18szlsz7saoes1i5dg43pe2 498 30942 2462701 2462524 2026-10-01T13:38:52Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2462701 wikitext text/x-wiki 5ohf6dkf3mkfltmw57z58zhn64bjn9k 2462780 2462701 2026-10-01T16:15:29Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462780 wikitext text/x-wiki 53r6x9ktw1sgpi88pqrtzdd41u98gco 2462853 2462780 2026-10-01T18:17:12Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2462853 wikitext text/x-wiki fvnr7xnnbniqg9hcahrioktu6o4y707 2462856 2462853 2026-10-01T18:22:44Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462856 wikitext text/x-wiki tkiw5k0m8dolq02ji2hoqu37153gano 2462873 2462856 2026-10-01T19:08:35Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.bootstrap_and_add 2462873 wikitext text/x-wiki 4alug3xxo4qr48wxi38ulrqhs1zaoa0 2462874 2462873 2026-10-01T19:08:39Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=99) 2462874 wikitext text/x-wiki pm56t6ef6ozd3r56c5y8b5s5360znf2 2462877 2462874 2026-10-01T19:19:48Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.bootstrap_and_add 2462877 wikitext text/x-wiki ef10501eg8m2s2au9mtcz1spma9t4qa 2462878 2462877 2026-10-01T19:27:45Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=0) 2462878 wikitext text/x-wiki 5x2jg53f030im4q5h2lq3qikoqf7r73 2462887 2462878 2026-10-01T19:48:40Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.bootstrap_and_add 2462887 wikitext text/x-wiki 2miy91u8436s3oeewhpwj1kxzicpvx3 2462892 2462887 2026-10-01T19:56:05Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=0) 2462892 wikitext text/x-wiki h5rb6h3awqa5ell6fwbr5653seceymj 2462952 2462892 2026-10-02T03:38:43Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462952 wikitext text/x-wiki 63mgi2vfh65wqohq1qlpf526wkyeake 2462954 2462952 2026-10-02T03:40:01Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2462954 wikitext text/x-wiki 4yf9zfqzd3ufhsvurjd677seis9qucj 2462956 2462954 2026-10-02T04:00:13Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.bootstrap_and_add 2462956 wikitext text/x-wiki 9h8ao6atbq8nji86upka2po19ymooo0 2462957 2462956 2026-10-02T04:07:59Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=0) 2462957 wikitext text/x-wiki hwpk0hr1ousek2bfo5a8nbitdttz62z 2462958 2462957 2026-10-02T04:10:09Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) 2462958 wikitext text/x-wiki 3jdnki4qfvzaqwyf2l6ntkbjoypdyvt 2462970 2462958 2026-10-02T08:13:47Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) 2462970 wikitext text/x-wiki 0brnscfyrg1xycornm434ryz20vrw5y 0 197480 2462888 2395469 2026-10-01T19:50:11Z TCipriani (WMF) 4895 Add link to using SpiderPig 2462888 wikitext text/x-wiki rkquox3drs1imlj5z0dw06gcco8h1qz 0 442288 2462778 2453543 2026-10-01T16:04:51Z TBurmeister (WMF) 27417 Add portal nav template 2462778 wikitext text/x-wiki iuj68lu7ygc5ipckg06bihs35pmlmbj 498 443949 2462640 2462198 2026-10-01T13:04:22Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy b707aca (Rename properties → columns template parameter) 2462640 wikitext text/x-wiki 8avk3aubca69msc0ijpvf9p12sha46v 2462641 2462640 2026-10-01T13:04:22Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 11f8bfa (Guard migrate_template_params against duplicate params) 2462641 wikitext text/x-wiki d1yigthtc0kafvwgjrhct1gr83qtpzw 498 446211 2462745 2461663 2026-10-01T15:07:02Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/36881408868 (https://github.com/cluebotng/component-configs/commits/2eac12b733c0a202e1363dde61dd15496bc3d385) 2462745 wikitext text/x-wiki 3m3bzunu1vyryp8gdmvh66eo54lpjxa 2462748 2462745 2026-10-01T15:09:27Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/36881820713 (https://github.com/cluebotng/component-configs/commits/860f13c0de005840e3e832d5dc6d17d9a4863409) 2462748 wikitext text/x-wiki ggqbn5s7dw9eotwctjwgtmg0rjqo18f 0 446433 2462931 2293638 2026-10-01T22:17:47Z Neil Shah-Quinn (WMF) 36601 Add info on Conda-Analytics-Next 2462931 wikitext text/x-wiki 2byzyhl286g06livl1ihmt0rfzddudi 2462937 2462931 2026-10-01T23:45:09Z Neil Shah-Quinn (WMF) 36601 /* Conda-Analytics-Next */ Copyedit 2462937 wikitext text/x-wiki h6ht98qmh15a8hy89ixj4exldz5fg3i 2462953 2462937 2026-10-02T03:38:50Z Neil Shah-Quinn (WMF) 36601 /* Conda-Analytics-Next */ Link to [[Data Platforms/Systems/Spark]] 2462953 wikitext text/x-wiki c2r6fs1l6f339g5qx7qlt90clf6k00c 0 447004 2462946 2387257 2026-10-02T01:36:51Z Neil Shah-Quinn (WMF) 36601 /* Managing your server */ Document how to start a server with a Conda-Analytics-Next environment 2462946 wikitext text/x-wiki i0yayi6pa30w4cw4rpr96o0ew8pcy9d 2462951 2462946 2026-10-02T03:33:49Z Neil Shah-Quinn (WMF) 36601 /* Sharing Notebooks */ Simply data publication guidelines warning 2462951 wikitext text/x-wiki 10yol0424mrhj4ex2olgo0h6xowr09v 0 449160 2462938 2462505 2026-10-02T00:01:23Z Dexbot 30554 Bot: Updating the report 2462938 wikitext text/x-wiki 5xpgvea8i31hzmc2yzambid040i8dqn 0 458585 2462998 2459819 2026-10-02T11:00:11Z TKlausmann (WMF) 18780 /* Replacing Failed Disks */ Add a section on how the cookbook works 2462998 wikitext text/x-wiki 9n0aogys9odq8isj0udc4g7uyvmmb5s 0 459290 2462757 2462559 2026-10-01T15:30:09Z APizzata-WMF 47941 2462757 wikitext text/x-wiki 8dhhkdu22t2f35ygu4s1wtf5cz1q7db 0 459699 2462783 2462514 2026-10-01T16:26:28Z Stashbot 7414 larssandergreen: civicrm upgraded from a30dfd91 to cb812976 2462783 wikitext text/x-wiki 18hgtedghlabyt5ckkw29wllobqnack 2462866 2462783 2026-10-01T18:51:57Z Stashbot 7414 damilare: civicrm upgraded from cb812976 to 40a4588a 2462866 wikitext text/x-wiki juximitkusq84myw2imhr8vhuugv1rt 2462950 2462866 2026-10-02T02:45:23Z Stashbot 7414 larssandergreen: civicrm upgraded from 40a4588a to 46fe8864 2462950 wikitext text/x-wiki 2i688uzkwvcaqyybzf8jdzkvd1vaixl 10 460153 2462777 2412293 2026-10-01T15:57:12Z TBurmeister (WMF) 27417 Update template to use portal nav and mirror wikidata content 2462777 wikitext text/x-wiki 9uefapsqtq6ux4zmgawbvqd0odmqb6g 2462788 2462777 2026-10-01T16:35:53Z TBurmeister (WMF) 27417 Modify header styling 2462788 wikitext text/x-wiki hxoxhbnmazlax8c7ovq4hdey317j5bz 2462789 2462788 2026-10-01T16:37:04Z TBurmeister (WMF) 27417 Adjust alignment and spacing 2462789 wikitext text/x-wiki 28vd3g8o9jjlzq9f7ujus704p28mxgw 0 460194 2462959 2455949 2026-10-02T04:59:09Z RLazarus (WMF) 15215 /* The checklist columns */ skip Phab task in trivial cases 2462959 wikitext text/x-wiki finixiobun4wba8a0bnek72ai1i4jir 2462963 2462959 2026-10-02T05:24:14Z RLazarus (WMF) 15215 /* The checklist columns */ YYYY-MM-DD 2462963 wikitext text/x-wiki 20wg6bzeadpq5mmpfohwxfgge3zoo1y 0 460344 2462960 2429974 2026-10-02T05:05:54Z RLazarus (WMF) 15215 What does "Has the design been reviewed by SRE?" require? If the component is simple, or just reorganizes existing code, should this be N/A? 2462960 wikitext text/x-wiki ox96lc3us717naf20vh2kss7gyxtuag 2462961 2462960 2026-10-02T05:11:03Z RLazarus (WMF) 15215 /* Process and scope */ Do MediaWiki extensions need an SLO (items 4.3 and 4.4)? 2462961 wikitext text/x-wiki 5utpfseu7pkwstljvojl5c7p6lqz3m7 2462962 2462961 2026-10-02T05:16:34Z RLazarus (WMF) 15215 /* Roles and sign-off */ If the developer team marks an item N/A, does it still need a sign-off? 2462962 wikitext text/x-wiki rjc3ra3ycuc26kz5795439fvpdxscep 2462971 2462962 2026-10-02T08:34:36Z MLechvien-WMF 49197 Remove Draft banner and update the Slack channel link 2462971 wikitext text/x-wiki b6p7j7i217eekknx5wn08v28w71mb95 0 460375 2462864 2459418 2026-10-01T18:42:20Z AWesterinen-WMF 54189 /* Example 2 */ 2462864 wikitext text/x-wiki klmipdei3kmwcknsti1hxzh9d09r9ic 2 460427 2462805 2435873 2026-10-01T17:23:14Z BPirkle (WMF) 12686 Trimmed some logstash links that have not proven useful, and added notes about possible improvements 2462805 wikitext text/x-wiki 2lhke40ocvcwghwc0nfm5p59w90vyfz 0 460598 2462930 2454883 2026-10-01T22:15:43Z AMastilovic-WMF 40067 Add the start_date section 2462930 wikitext text/x-wiki 8o27mca2o9p1eqknw9ob29sa5kbbvb3 0 460682 2462782 2460244 2026-10-01T16:26:01Z AndrewBogott (WMF) 51047 2462782 wikitext text/x-wiki a9azrm0a75154tvkr3f2uyyozkl0ewz 2462784 2462782 2026-10-01T16:27:12Z AndrewBogott (WMF) 51047 /* Fixing */ 2462784 wikitext text/x-wiki kgv695r30q9ysoxhzlsmdk1jdmdfktz 0 460709 2462759 2462519 2026-10-01T15:31:58Z AWesterinen-WMF 54189 /* geof:globe, geof:latitude and geof:longitude functions */ 2462759 wikitext text/x-wiki 75miaedgvd7oc9tb2z3jyykxrew53yr 2462762 2462759 2026-10-01T15:39:17Z AWesterinen-WMF 54189 /* geof:globe, geof:latitude and geof:longitude functions */ 2462762 wikitext text/x-wiki ee37fewxgqcbsy6uk92pgl1chddyfz4 2462765 2462762 2026-10-01T15:41:27Z AWesterinen-WMF 54189 /* geof:distance() */ 2462765 wikitext text/x-wiki 6p1ikjbu5e6i27ld7b52zpciykoc6b9 2462766 2462765 2026-10-01T15:43:00Z AWesterinen-WMF 54189 /* wikibase:around */ 2462766 wikitext text/x-wiki mqpmgdksc6vhouowvyunvzyp3zsqap6 2462767 2462766 2026-10-01T15:46:13Z AWesterinen-WMF 54189 /* wikibase:box */ 2462767 wikitext text/x-wiki ahr8ycc44g6mmycy2fggqsfjx2vbxfy 2462798 2462767 2026-10-01T17:09:55Z AWesterinen-WMF 54189 /* geof:globe, geof:latitude and geof:longitude functions */ 2462798 wikitext text/x-wiki rokvg6mak6erdiqatye0ix5zuns3gaa 2462799 2462798 2026-10-01T17:11:09Z AWesterinen-WMF 54189 /* geof:globe, geof:latitude and geof:longitude functions */ 2462799 wikitext text/x-wiki ocb0k05sfaal6na82dgo205w5kmd0uf 2462806 2462799 2026-10-01T17:23:18Z AWesterinen-WMF 54189 /* geof:distance() */ 2462806 wikitext text/x-wiki hbg3fnqlqjrwd2w0zlb05arft75x5u6 2462807 2462806 2026-10-01T17:41:36Z AWesterinen-WMF 54189 /* wikibase:around */ 2462807 wikitext text/x-wiki 7msks210w9rmgxag6yoqckq3616anra 2462848 2462807 2026-10-01T18:09:24Z AWesterinen-WMF 54189 /* Box with parameters, cornerSouthWest and cornerNorthEast */ 2462848 wikitext text/x-wiki lyymznfl4sxqg23tbtzx29p01ersg13 2462849 2462848 2026-10-01T18:10:13Z AWesterinen-WMF 54189 /* Box with parameters, cornerSouthWest and cornerNorthEast */ 2462849 wikitext text/x-wiki 867ccsftsbz41on40cxx8w0poew3k3y 2462854 2462849 2026-10-01T18:20:00Z AWesterinen-WMF 54189 /* Box with parameters, cornerWest and cornerEast */ 2462854 wikitext text/x-wiki nvcotf6a6sv7cyjfdypb100u5kn8sdl 2462855 2462854 2026-10-01T18:20:21Z AWesterinen-WMF 54189 /* Why are there north/south calculations and different results? */ 2462855 wikitext text/x-wiki dzly2aqnvl8cs5d8brgurftnbwgib15 2462860 2462855 2026-10-01T18:34:52Z AWesterinen-WMF 54189 /* Box with parameters, cornerSouthWest and cornerNorthEast */ 2462860 wikitext text/x-wiki 0um4s6jcw52akdlifhooyzzoyvypige 2462865 2462860 2026-10-01T18:48:22Z AWesterinen-WMF 54189 /* wikibase:box */ 2462865 wikitext text/x-wiki ikcd3x5por0ijqw44imkcpopvlgqkkq 2462867 2462865 2026-10-01T18:52:22Z AWesterinen-WMF 54189 /* Box with parameters, cornerWest and cornerEast */ 2462867 wikitext text/x-wiki ggf6jgmjgkpzf433vhuhfa9lcdradj3 2462868 2462867 2026-10-01T18:52:55Z AWesterinen-WMF 54189 /* Box with parameters, cornerWest and cornerEast */ 2462868 wikitext text/x-wiki avk4llucp047y0d9s03gkyaxwfdmst2 12 460735 2462792 2461759 2026-10-01T16:40:59Z DCaro (WMF) 18271 added link to the opt-in, and rewording for a shorter word 2462792 wikitext text/x-wiki ai92po23u57tpg5bzcdwg1q73g44v48 2462793 2462792 2026-10-01T16:41:47Z DCaro (WMF) 18271 /* Enabling the email notifications */ added a ... to hint that the file needs more stuff 2462793 wikitext text/x-wiki dvqx7420xtnw7yvj6ksereuajhqwr61 2462794 2462793 2026-10-01T16:43:18Z DCaro (WMF) 18271 /* Disabling the email notifications */ Added a note about the default being disabled 2462794 wikitext text/x-wiki dfrvkp0z3asa9t9nooh3u5jtxuaz0ol 2462795 2462794 2026-10-01T16:44:33Z DCaro (WMF) 18271 /* Known limitations */ Added a note that they need to define the components in the config 2462795 wikitext text/x-wiki gdfr4q6nx889x69wjpbspxz6nbzt94r 828 460751 2462769 2026-10-01T15:51:30Z TBurmeister (WMF) 27417 Create module, copied directly from https://www.wikidata.org/wiki/Module:Portal_navigation 2462769 Scribunto text/plain g91vh3761wenithfbbasd6ky21nb50z 828 460752 2462772 2026-10-01T15:52:50Z TBurmeister (WMF) 27417 Create docs, copied from https://www.wikidata.org/wiki/Template:Portal_navigation/doc 2462772 wikitext text/x-wiki bi45ac56w4ix9p0qcfm0mvzuek8zp37 828 460753 2462774 2026-10-01T15:53:39Z TBurmeister (WMF) 27417 Create css, copied from https://www.wikidata.org/wiki/Module:Portal_navigation/styles.css 2462774 sanitized-css text/css b3shfb2x1t4epvy9zdxwgl5vmcizcok 10 460754 2462775 2026-10-01T15:54:38Z TBurmeister (WMF) 27417 Create template, copied from https://www.wikidata.org/wiki/Template:Portal_navigation 2462775 wikitext text/x-wiki migvhqm18sh7wqcmrdm4p62edciezd4 10 460755 2462776 2026-10-01T15:56:05Z TBurmeister (WMF) 27417 Create docs, copied from https://www.wikidata.org/wiki/Template:Portal_navigation/doc 2462776 wikitext text/x-wiki bi45ac56w4ix9p0qcfm0mvzuek8zp37 3 460758 2462929 2026-10-01T22:09:39Z StrikerBot 8475 Welcome to Toolforge! 2462929 wikitext text/x-wiki ajr56wamncbc68ku5hje40cwtkr18zd