Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.16 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 3086 2450763 2450219 2026-08-24T10:46:10Z Stashbot 7414 taavi: disable access to elastic cluster as announced on cloud-announce T401818 2450763 wikitext text/x-wiki qam60i3vu8hejomqqhpe594bj16c3ql 0 4039 2450686 2439745 2026-08-23T20:14:10Z Chlod 18186 /* Emergency runbook for production config changes */ fix typo 2450686 wikitext text/x-wiki p5j6oub84ufeqecv4tb420tl1d1g447 0 7919 2450684 2450659 2026-08-23T17:01:09Z Stashbot 7414 urbanecm@deploy1003: mwscript-k8s job started: GrowthExperiments:fixLinkRecommendationData.php --wiki=enwiki --db-table --search-index --force --verbose # T434319 2450684 wikitext text/x-wiki 34qiqoo8tizhsvw5j37zk08je5s5lvb 2450685 2450684 2026-08-23T17:26:58Z Stashbot 7414 urbanecm@deploy1003: mwscript-k8s job started: GrowthExperiments:fixLinkRecommendationData.php --wiki=enwiki --db-table --force --verbose --verbose # T434319 2450685 wikitext text/x-wiki hbxrjviw84ghbveltenq3ceyde95zx6 2450688 2450685 2026-08-24T05:20:23Z Stashbot 7414 marostegui: wikifunctionsclient_usage from s2 T434541 2450688 wikitext text/x-wiki pcfn8ejynl1dkmdokm6qqti7u5f4aad 2450689 2450688 2026-08-24T05:23:34Z Stashbot 7414 marostegui: wikifunctionsclient_usage from s3 T434541 2450689 wikitext text/x-wiki ilgpvuo2hwvv4j6fcteumqtkkd46ce3 2450690 2450689 2026-08-24T05:48:29Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1282 to dbctl T407942', diff saved to https://phabricator.wikimedia.org/P96225 and previous config saved to /var/cache/conftool/dbconfig/20260824-054826-marostegui.json 2450690 wikitext text/x-wiki ntmvrgq5mqmafbnd8d8f3nuxc7hakjc 2450691 2450690 2026-08-24T05:48:42Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1282: Pool back 2450691 wikitext text/x-wiki j0w14vu6bbsyeelt6325zxzeb9br8xe 2450692 2450691 2026-08-24T05:50:14Z Stashbot 7414 marostegui: Stop mariadb on sanitarium s1,s3,s8,s5,x3 there will be lag on wikireplicas for those sections T407942 2450692 wikitext text/x-wiki msxf0z9762ul0vd7vv72qucljd1cx68 2450693 2450692 2026-08-24T05:50:18Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on 21 hosts with reason: Cloning 2450693 wikitext text/x-wiki 5k2bg40qem5eqz3j0gkbe1pyk7gsu4q 2450694 2450693 2026-08-24T06:33:07Z Stashbot 7414 marostegui: wikifunctionsclient_usage from s5 T434541 2450694 wikitext text/x-wiki pjhizx32t59v2qvj26u8vc5osazpcp1 2450695 2450694 2026-08-24T06:33:46Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1282: Pool back 2450695 wikitext text/x-wiki bgj6oru4e0jwiq50kci73oiwqyk1m9j 2450696 2450695 2026-08-24T06:46:18Z Stashbot 7414 marostegui: wikifunctionsclient_usage from s7 T434541 2450696 wikitext text/x-wiki gzdesan2r1lkhhcpn4ccd6phuqhqlgp 2450697 2450696 2026-08-24T06:55:50Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Remove db1182 from dbctl T434869', diff saved to https://phabricator.wikimedia.org/P96236 and previous config saved to /var/cache/conftool/dbconfig/20260824-065547-marostegui.json 2450697 wikitext text/x-wiki ib0j3j1tblqdy3cmxantzg29d2lmwn2 2450698 2450697 2026-08-24T07:50:01Z Stashbot 7414 chlod@deploy1003: Started scap sync-world: Backport for [[gerrit:1307270|InitialiseSettings: change enwiki extendedconfirmed autopromote settings (T431060)]] 2450698 wikitext text/x-wiki 2o7oou86dlj9jinz1ddivwyw8ks8ox2 2450699 2450698 2026-08-24T07:51:17Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revise-tone-task-generator' for release 'main' . 2450699 wikitext text/x-wiki 4i4xpl6mhddwxwz313y0bsrpval8b8r 2450700 2450699 2026-08-24T07:51:23Z Stashbot 7414 kevinbazira@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . 2450700 wikitext text/x-wiki msw7wldij9os6tyoa6no78atak698lz 2450701 2450700 2026-08-24T07:52:39Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revise-tone-task-generator' for release 'main' . 2450701 wikitext text/x-wiki 1qxira7nvkcr75frxn8s13ee4uawiuw 2450702 2450701 2026-08-24T07:53:33Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revise-tone-task-generator' for release 'main' . 2450702 wikitext text/x-wiki qtycu9hjepg86t9c6d3eoarxg3eyrdo 2450703 2450702 2026-08-24T08:11:12Z Stashbot 7414 chlod@deploy1003: novemlinguae, chlod: Backport for [[gerrit:1307270|InitialiseSettings: change enwiki extendedconfirmed autopromote settings (T431060)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2450703 wikitext text/x-wiki bzqp13dy0rvqhzk3a28iakfm3yc3cpa 2450704 2450703 2026-08-24T08:14:10Z Stashbot 7414 chlod@deploy1003: novemlinguae, chlod: Continuing with deployment 2450704 wikitext text/x-wiki p9itk4m9cf0r4if4vhkuvhyx7s6xlny 2450705 2450704 2026-08-24T08:17:52Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [staging] START helmfile.d/services/changeprop: apply 2450705 wikitext text/x-wiki cy8wy8qc6tawpuj3i9ko9bsn17oyr5k 2450706 2450705 2026-08-24T08:18:02Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [staging] DONE helmfile.d/services/changeprop: apply 2450706 wikitext text/x-wiki csqy4483hw2oiyjv1i5wd5r3p0w4iuo 2450707 2450706 2026-08-24T08:19:03Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [codfw] START helmfile.d/services/changeprop: sync 2450707 wikitext text/x-wiki tnpvf9fu09wunyk0jj22bnhmim49mm1 2450708 2450707 2026-08-24T08:19:26Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [codfw] DONE helmfile.d/services/changeprop: sync 2450708 wikitext text/x-wiki 6wkxy48q0cf4zzae0q0dqbfod0brsz0 2450709 2450708 2026-08-24T08:21:12Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host irc2003.wikimedia.org 2450709 wikitext text/x-wiki eswybecjvwyikz2hgpaa1755pqb91oz 2450710 2450709 2026-08-24T08:22:03Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [eqiad] START helmfile.d/services/changeprop: sync 2450710 wikitext text/x-wiki 9pcouq5u3vgevdh2zrp56hhivk5hpzy 2450711 2450710 2026-08-24T08:22:28Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [eqiad] DONE helmfile.d/services/changeprop: sync 2450711 wikitext text/x-wiki 4mfh4fehw8zlo748xi335fwehyuxomu 2450712 2450711 2026-08-24T08:25:07Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host irc2003.wikimedia.org 2450712 wikitext text/x-wiki o1ban9oca04ha492lyf93k0xmgxahwp 2450714 2450712 2026-08-24T08:25:36Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2450714 wikitext text/x-wiki pkf55kqs99x6s5z9ta4b3oiyij43hck 2450716 2450714 2026-08-24T08:26:45Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2450716 wikitext text/x-wiki qqd5n5zbbra0klczeu0kqj76s3gcsvh 2450718 2450716 2026-08-24T08:29:01Z Stashbot 7414 chlod@deploy1003: Finished scap sync-world: Backport for [[gerrit:1307270|InitialiseSettings: change enwiki extendedconfirmed autopromote settings (T431060)]] (duration: 39m 00s) 2450718 wikitext text/x-wiki sj7yte0vxetwgu6v0wogdp2yeodwpwz 2450720 2450718 2026-08-24T08:42:20Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2450720 wikitext text/x-wiki eze8bcn5un3ot63usyrxlap35oj3so5 2450722 2450720 2026-08-24T08:43:23Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2450722 wikitext text/x-wiki 2k7yeb77xqb4brozb4flmpihhb7opaw 2450723 2450722 2026-08-24T08:44:11Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2450723 wikitext text/x-wiki gnh9qp5x13ilnkg0fv29s12myhg4bhv 2450724 2450723 2026-08-24T08:45:04Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2450724 wikitext text/x-wiki 001fnuvq85vljjxym2hjhrpkrckx7wg 2450726 2450724 2026-08-24T08:45:57Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2450726 wikitext text/x-wiki q8kkqw2ey69385ajw5z2j63uy7b40k8 2450727 2450726 2026-08-24T08:46:56Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2450727 wikitext text/x-wiki bo532jrlh2p11g46t7w0elrqd1r6m1e 2450728 2450727 2026-08-24T08:48:38Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2450728 wikitext text/x-wiki 89drmdu26j42civvblqyw5preguok8c 2450729 2450728 2026-08-24T08:49:31Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2450729 wikitext text/x-wiki tio2n2egxhr79it1dzcxdbc811r41pi 2450731 2450729 2026-08-24T09:18:38Z Stashbot 7414 kevinbazira@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . 2450731 wikitext text/x-wiki j5k7a7wpoecgxf03m0wjdacw408firc 2450733 2450731 2026-08-24T09:28:59Z Stashbot 7414 taavi: test 2450733 wikitext text/x-wiki 9bmw8jj5fdlzbz68yyka2pcrhvqsxzn 2450736 2450733 2026-08-24T09:30:00Z Stashbot 7414 taavi: [09:25] kevinbazira@deploy1003 helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . 2450736 wikitext text/x-wiki jyd84m7gbjgpo7vilof9q56lwyyic5g 2450737 2450736 2026-08-24T09:30:09Z Stashbot 7414 taavi: [09:26] kevinbazira@deploy1003 helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . 2450737 wikitext text/x-wiki puzl22p3rjmmbe6hlzf91f633pmtvpb 2450738 2450737 2026-08-24T09:46:06Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db2901.codfw.wmnet decommissioned, removing all IPs except the asset tag one - fceratto@cumin1003" 2450738 wikitext text/x-wiki 27tl3iclerzpeiqwsxvkhp1v2jatejy 2450739 2450738 2026-08-24T09:46:09Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) 2450739 wikitext text/x-wiki kxf26l8wbgnrvu0jum276s0u6l8f1k4 2450740 2450739 2026-08-24T09:46:11Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts db2901.codfw.wmnet 2450740 wikitext text/x-wiki nnysu125xoitixqx3h64gh2o8elmygv 2450741 2450740 2026-08-24T09:47:45Z Stashbot 7414 gmodena@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2450741 wikitext text/x-wiki dassc41n2kot87vht7iuqg9kqd77r00 2450743 2450741 2026-08-24T09:52:30Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2450743 wikitext text/x-wiki gvvzciyhlw8195uocfdl7lmtcgqyxuf 2450744 2450743 2026-08-24T10:04:24Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Delete db[12]90[1-3] ipv6 addr - fceratto@cumin1003" 2450744 wikitext text/x-wiki 5xu8agnvogtmol7r76ak5luonljym01 2450745 2450744 2026-08-24T10:04:28Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Delete db[12]90[1-3] ipv6 addr - fceratto@cumin1003" 2450745 wikitext text/x-wiki rf2zy3h01nkn7ukydnp1p8bc29ibyzu 2450746 2450745 2026-08-24T10:04:30Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2450746 wikitext text/x-wiki 3lp0v9ungdyugzw0uxb3g04tejzqlai 2450747 2450746 2026-08-24T10:05:58Z Stashbot 7414 blake@deploy1003: Started scap sync-world: upgrade envoy in debug and canary for T421418 2450747 wikitext text/x-wiki mn1tri7m909ojb8cjbiu5b8uofhs3cd 2450748 2450747 2026-08-24T10:11:19Z Stashbot 7414 blake@deploy1003: Finished scap sync-world: upgrade envoy in debug and canary for T421418 (duration: 06m 46s) 2450748 wikitext text/x-wiki 0jegueu3v2fzmvpvz5zua8bedc0bwj1 2450751 2450748 2026-08-24T10:19:53Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-product: apply 2450751 wikitext text/x-wiki qyd8q4zun8wztga0n28g5i94tpkerir 2450752 2450751 2026-08-24T10:20:13Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-product: apply 2450752 wikitext text/x-wiki blfz5d8ual2vjypjjl3ez6qy660sk60 2450753 2450752 2026-08-24T10:20:43Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply 2450753 wikitext text/x-wiki irxl74it0d4rhl2ekvcrt9d7vlb450i 2450754 2450753 2026-08-24T10:21:20Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply 2450754 wikitext text/x-wiki nx5uf8n6swrdgfsxq161k75varogh0b 2450755 2450754 2026-08-24T10:22:04Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply 2450755 wikitext text/x-wiki ffe0e95113dsiys21zp0da5afwadkp5 2450756 2450755 2026-08-24T10:22:38Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply 2450756 wikitext text/x-wiki 4e88x6o82h888e9heozdnpxh77vext9 2450757 2450756 2026-08-24T10:23:07Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-fr-tech: apply 2450757 wikitext text/x-wiki ku0pcanuaq6i2qcl6khw22khx16b10a 2450758 2450757 2026-08-24T10:23:45Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-fr-tech: apply 2450758 wikitext text/x-wiki 0tkxol0llq09de4okxre04yigoq7k70 2450759 2450758 2026-08-24T10:24:42Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-main: apply 2450759 wikitext text/x-wiki hi33whsqdmfc7ib87pvmd21pw6j457a 2450760 2450759 2026-08-24T10:25:04Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-main: apply 2450760 wikitext text/x-wiki qauip5dxhcopbbqw9kt30qxh4r1zhq5 2450761 2450760 2026-08-24T10:38:35Z Stashbot 7414 gmodena@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2450761 wikitext text/x-wiki 2hvz30lgvr4uizgwy6ub1scv8fkhcqs 2450764 2450761 2026-08-24T10:50:08Z Stashbot 7414 marostegui@dns1004: START - running authdns-update 2450764 wikitext text/x-wiki 4qsiqeiuxfrjkj3txkqk3uls74she61 2450765 2450764 2026-08-24T10:52:29Z Stashbot 7414 marostegui@dns1004: END - running authdns-update 2450765 wikitext text/x-wiki n7b91bhilkntjvg6gc78vs8wkjot5mn 2450766 2450765 2026-08-24T10:54:10Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2450766 wikitext text/x-wiki del4nkyvpeu12ltz5um4nxp13qqu3t3 2450767 2450766 2026-08-24T10:54:20Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2450767 wikitext text/x-wiki nfq6kpdqrs4ydc4zbxz9vo3w5glv18n 2450768 2450767 2026-08-24T10:58:43Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] START helmfile.d/services/thumbor: apply 2450768 wikitext text/x-wiki 3x3wym0imhn7832vbix2s2o9uth5bn8 2450769 2450768 2026-08-24T11:00:03Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] DONE helmfile.d/services/thumbor: apply 2450769 wikitext text/x-wiki a7gu7471c8c0kxjazjygxv0xa3gsq52 2450770 2450769 2026-08-24T11:04:50Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] START helmfile.d/services/thumbor: apply 2450770 wikitext text/x-wiki hl8t2q0t46d943g5jw3cw5v6jvrxcdo 2450771 2450770 2026-08-24T11:05:33Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] DONE helmfile.d/services/thumbor: apply 2450771 wikitext text/x-wiki jp3ad3g253o6sk4olgd88uf0bfnc2o8 2450772 2450771 2026-08-24T11:09:44Z Stashbot 7414 moritzm: installing Bind security updates (client-side libs/tools) 2450772 wikitext text/x-wiki nfu4kqbo5e22ya0liokop3aetu0zymg 2450773 2450772 2026-08-24T11:21:46Z Stashbot 7414 topranks: re-enable codfw-magru traffic path over HE T435543 2450773 wikitext text/x-wiki h1mq0sv50uuba72999lqnj7z60u9rot 2450774 2450773 2026-08-24T11:21:55Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-ml: apply 2450774 wikitext text/x-wiki 66b2exy087jyeuzzvs5po1tk7k7k80t 2450775 2450774 2026-08-24T11:22:26Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-ml: apply 2450775 wikitext text/x-wiki og4zvea6r2ubvrah0vjnch9sug2s6vr 2450776 2450775 2026-08-24T11:24:18Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-platform-eng: apply 2450776 wikitext text/x-wiki 09xts4knkm35xhupg8h4jnwca823vgp 2450777 2450776 2026-08-24T11:24:40Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-platform-eng: apply 2450777 wikitext text/x-wiki 3f14q0z7x69ezriv20nzqxldcvpz7f1 2450778 2450777 2026-08-24T11:25:22Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-research: apply 2450778 wikitext text/x-wiki iqoqmo7hwmva5xavlnmlvkgkr7e4rm2 2450779 2450778 2026-08-24T11:25:54Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-research: apply 2450779 wikitext text/x-wiki 45ae9fdz5xsr5w7lo0s3ausl3n5dtoi 2450780 2450779 2026-08-24T11:26:14Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-search: apply 2450780 wikitext text/x-wiki 0d6ej5sm3pf49ljiigr5wyoxnc49h9l 2450781 2450780 2026-08-24T11:26:50Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-search: apply 2450781 wikitext text/x-wiki opnmhas0zmrgrozhynibbcmtececuvu 2450782 2450781 2026-08-24T11:27:33Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-sre: apply 2450782 wikitext text/x-wiki iexnbbrz1rgq7db1qk9ko5xawjxbofu 2450783 2450782 2026-08-24T11:28:05Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-sre: apply 2450783 wikitext text/x-wiki 2g86b1v9qshus1xg7b1rvpuscbc82pz 2450784 2450783 2026-08-24T11:28:13Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2450784 wikitext text/x-wiki qga8gd7yj161ztlkwjs4gw0tnmd4dqa 2450785 2450784 2026-08-24T11:28:49Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2450785 wikitext text/x-wiki bl7wqwe7lcte2784txdwpa5yooo9sgv 2450786 2450785 2026-08-24T11:31:43Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wikidata: apply 2450786 wikitext text/x-wiki dkjquxgh0ljxvzf6gqyi1fqurpy973f 2450787 2450786 2026-08-24T11:32:18Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wikidata: apply 2450787 wikitext text/x-wiki d1eohi115b4319ex3y91lfza5lpkcco 2450789 2450787 2026-08-24T11:33:07Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wmde: apply 2450789 wikitext text/x-wiki ex2izz61tlw7q5o2h0on1jv1b1c5hnc 2450790 2450789 2026-08-24T11:33:38Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wmde: apply 2450790 wikitext text/x-wiki 99d43gx1rxvk6rcyehl6syglp6y3ckn 2450791 2450790 2026-08-24T11:38:39Z Stashbot 7414 kartik@deploy1003: helmfile [staging] START helmfile.d/services/cxserver: apply 2450791 wikitext text/x-wiki fi54c4n7gjp7fcq1x36hfjbk5odphwf 2450792 2450791 2026-08-24T11:39:01Z Stashbot 7414 kartik@deploy1003: helmfile [staging] DONE helmfile.d/services/cxserver: apply 2450792 wikitext text/x-wiki mac12s1ifs16nl6dmm93netivavczos 2450793 2450792 2026-08-24T11:42:19Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1328542|Expose the new thumbnail endpoint on dewiki (T427465)]] 2450793 wikitext text/x-wiki c2ll20wz4ebauwtlqijt43xjdy0g6ld 2450794 2450793 2026-08-24T11:45:11Z Stashbot 7414 kartik@deploy1003: helmfile [codfw] START helmfile.d/services/cxserver: apply 2450794 wikitext text/x-wiki np66s5f3xyzlr6sldno6vuwi5cnej06 2450795 2450794 2026-08-24T11:45:44Z Stashbot 7414 kartik@deploy1003: helmfile [codfw] DONE helmfile.d/services/cxserver: apply 2450795 wikitext text/x-wiki 3n8drd9as3fjl7zlrsz9ov1arvn2nto 2450796 2450795 2026-08-24T11:47:02Z Stashbot 7414 kartik@deploy1003: helmfile [eqiad] START helmfile.d/services/cxserver: apply 2450796 wikitext text/x-wiki f07pjsbl6jsvgjyv2si9tg3gykhaxsc 2450797 2450796 2026-08-24T11:47:31Z Stashbot 7414 kartik@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cxserver: apply 2450797 wikitext text/x-wiki bcv3l5lrtqvhee1ulf1iny72bz7poly 2450798 2450797 2026-08-24T11:48:21Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1328542|Expose the new thumbnail endpoint on dewiki (T427465)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2450798 wikitext text/x-wiki 5nbkicrvnujh4q6o6l0vz82droaqksh 2450799 2450798 2026-08-24T11:48:24Z Stashbot 7414 moritzm: installing php7.4 security updates 2450799 wikitext text/x-wiki lnqslxbzn08z71uuzb2jnz77mi3jzax 2450800 2450799 2026-08-24T11:48:29Z Stashbot 7414 jayme@deploy1003: helmfile [eqiad] START helmfile.d/admin 'apply'. 2450800 wikitext text/x-wiki kdnj40nala46i2jgq6ujnld3fk0io1q 2450801 2450800 2026-08-24T11:49:26Z Stashbot 7414 jayme@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'apply'. 2450801 wikitext text/x-wiki gaj79k1ymg9wrm9i6m3gkv0zw4p98yp 2450802 2450801 2026-08-24T11:49:35Z Stashbot 7414 jayme@deploy1003: helmfile [codfw] START helmfile.d/admin 'apply'. 2450802 wikitext text/x-wiki b5ys4m30rvzj4wgkkbjrxumdvv7rwqo 2450803 2450802 2026-08-24T11:50:17Z Stashbot 7414 kart_: Updated cxserver to 2026-08-24-032908-production (T213262, T213277, T253501, T428612) 2450803 wikitext text/x-wiki g6qdsfhc6qmjx2ge382akg63owo406p 2450804 2450803 2026-08-24T11:50:28Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2450804 wikitext text/x-wiki bc55yxl7lcyc6f489stmehplr6tzj6z 2450805 2450804 2026-08-24T11:50:32Z Stashbot 7414 jayme@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'apply'. 2450805 wikitext text/x-wiki jv9alguvcuugz4m1k4ic5tymrxgyr5f 2450806 2450805 2026-08-24T11:50:43Z Stashbot 7414 jayme@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'apply'. 2450806 wikitext text/x-wiki 0uw7fod0r17wh8o3j43szc65zl7r1j1 2450807 2450806 2026-08-24T11:51:21Z Stashbot 7414 jayme@deploy1003: helmfile [eqiad] START helmfile.d/admin 'apply'. 2450807 wikitext text/x-wiki clbe405el4h87im3shfntyp6xsvc7mx 2450808 2450807 2026-08-24T11:52:29Z Stashbot 7414 jayme@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. 2450808 wikitext text/x-wiki p4q8ev4ebdjwroz9zypfx40sl9oitno 2450809 2450808 2026-08-24T11:52:38Z Stashbot 7414 jayme@deploy1003: helmfile [staging-codfw] START helmfile.d/admin 'apply'. 2450809 wikitext text/x-wiki 5el9ec2awx7hyl7vo5w6vst6i8xio87 2450810 2450809 2026-08-24T11:53:45Z Stashbot 7414 jayme@deploy1003: helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. 2450810 wikitext text/x-wiki mx36xghucycqowqqn899lnqxcm3u83w 2450811 2450810 2026-08-24T11:53:57Z Stashbot 7414 jayme@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'apply'. 2450811 wikitext text/x-wiki sn31usrfk6lr5hxn3jtk8j5bjrlcbcx 2450812 2450811 2026-08-24T11:54:11Z Stashbot 7414 jayme@deploy1003: helmfile [codfw] START helmfile.d/admin 'apply'. 2450812 wikitext text/x-wiki kz09cv628ckyyu5sxl3bkz7mbdaszr2 2450814 2450812 2026-08-24T11:56:51Z Stashbot 7414 jayme@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'apply'. 2450814 wikitext text/x-wiki iwx2lm99m9kwugeysfje7qz7qvzhj64 2450815 2450814 2026-08-24T11:56:59Z Stashbot 7414 jayme@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'apply'. 2450815 wikitext text/x-wiki 6cqzka1vf1xt8cwjxkiixe7inugzy3m 2450816 2450815 2026-08-24T11:57:53Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1328542|Expose the new thumbnail endpoint on dewiki (T427465)]] (duration: 15m 34s) 2450816 wikitext text/x-wiki 8gocg9m7xkp5y1617zupad4upr8y5nu 2450817 2450816 2026-08-24T11:58:34Z Stashbot 7414 moritzm: installing ruby2.7 security updates 2450817 wikitext text/x-wiki 7igsuqunkkufgrre8213c64ojboy4jc 0 17290 2450717 2450498 2026-08-24T08:26:57Z Stashbot 7414 hashar: integration: removing firewall rule for ssh from contint1002/contint2002 IPv4 # T418521 2450717 wikitext text/x-wiki 2e6hqhjwpz65c9e051cb0071lr4kjny 2450719 2450717 2026-08-24T08:29:58Z Stashbot 7414 hashar: integration: add firewall rule to ssh from contint1003/contint2003 IPv6 (IPv4 was already allowed) # T435756 2450719 wikitext text/x-wiki 2vk9ko1pwy8buk6gbt2skg2xa659kkj 498 46253 2450721 2390562 2026-08-24T08:43:21Z Stashbot 7414 taavi: migrating historic irc-* indexes to the new opensearch cluster T435223 2450721 wikitext text/x-wiki 7sl220jmdx7c2f7ughytircxcg6wf8m 2450732 2450721 2026-08-24T09:27:58Z Stashbot 7414 taavi: migrate bot with current indexes to new opensearch cluster T435223 2450732 wikitext text/x-wiki is98ms2sa26n9pdynqyj43g2hg9tsr9 498 59869 2450734 2259508 2026-08-24T09:29:25Z Stashbot 7414 taavi: update to read from new opensearch cluster T435223 2450734 wikitext text/x-wiki 5k0dzozxu2rqt6pv72hgbbnm0uvuafy 498 59880 2450735 2060453 2026-08-24T09:29:28Z Stashbot 7414 taavi: update to read from new opensearch cluster T435223 2450735 wikitext text/x-wiki o4k181euem660wdsklt95monsyuzjup 498 443628 2450742 2442863 2026-08-24T09:52:01Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/32713575575 (https://github.com/cluebotng/component-configs/commits/15461b10d7f124d327ec6065e11b3fce0465cc68) 2450742 wikitext text/x-wiki dgqu4ubg5k50wsouax55m8vp9oz1e7l 498 443947 2450679 2439780 2026-08-23T12:10:14Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed 6b1c827261 (l10n updates: cs, ha, pl) 2450679 wikitext text/x-wiki pgb0kbz74qcgp13ukkxayyfwla75jmp 0 443950 2450713 2389672 2026-08-24T08:25:26Z Lucas Werkmeister (WMDE) 10037 update commands one might run on the deployment server 2450713 wikitext text/x-wiki a487pdmi3ux2tsow7u20nht35ucyfbo 2450715 2450713 2026-08-24T08:26:07Z Lucas Werkmeister (WMDE) 10037 wrap a command in <code> 2450715 wikitext text/x-wiki tcrfogib2gun7s7ykd4lf4s8ezb73rh 0 444275 2450749 2431540 2026-08-24T10:16:09Z BJensen-WMF 48942 /* Roll out the new version in Kubernetes */ add link to patch 2450749 wikitext text/x-wiki buoktx9qymddgjx75hqj7nugpin6s9n 2450750 2450749 2026-08-24T10:16:39Z BJensen-WMF 48942 /* Roll out the new version in Kubernetes */ add link to patch 2450750 wikitext text/x-wiki 2j5gqv1bykb0xwsw2kgs0ce80u2i85r 0 448298 2450725 2406855 2026-08-24T08:45:35Z MLechvien-WMF 49197 Adding Kubernetes OH to our team intake 2450725 wikitext text/x-wiki 4fjdjxwtwk4cvdue0bkwmo4z2l3juxo 0 449160 2450687 2450668 2026-08-24T00:02:22Z Dexbot 30554 Bot: Updating the report 2450687 wikitext text/x-wiki rjzwjscxzbs052my2ls1cumpywm14te 0 450294 2450788 2445128 2026-08-24T11:32:59Z Atsuko-WMF 54082 /* List of instances */ added experiment-platform 2450788 wikitext text/x-wiki b14j45qya4a10e4k0bmlk9rula1bhza 12 452086 2450730 2435555 2026-08-24T09:08:31Z AKlapper (WMF) 421 "within the next twelve months" does not exist on wikis 2450730 wikitext text/x-wiki f5lgvwxhk0454bcnk65ustot6ktpzyn 2 460039 2450813 2450409 2026-08-24T11:55:59Z Atsuko-WMF 54082 /* Provision Airflow */ 2450813 wikitext text/x-wiki m1aqormkyr0ra3pnsjfs5ar9rmss4g6 498 460131 2450680 2441066 2026-08-23T14:13:24Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: [DEPLOY] Starting deployment | ref=CodeSparks45-fix-sse-timeout (for test T435612) 2450680 wikitext text/x-wiki ftby2wwsukhes7igvevbalvs80ojv92 2450681 2450680 2026-08-23T14:14:34Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: [DEPLOY] Build triggered successfully | ref=CodeSparks45-fix-sse-timeout (for test T435612) 2450681 wikitext text/x-wiki 3w7s6g63ba3o5zltc9zdd14y7cd0iak 2450682 2450681 2026-08-23T14:14:37Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: [DEPLOY] Restarting buildservice | ref=CodeSparks45-fix-sse-timeout (for test T435612) 2450682 wikitext text/x-wiki bkc8qwd62jyifb6j95n0cdcv33jxbf9 2450683 2450682 2026-08-23T14:14:45Z Stashbot 7414 wmbot~gergesshamon@tools-bastion-15: [DEPLOY] Deployment completed successfully | ref=CodeSparks45-fix-sse-timeout (for test T435612) 2450683 wikitext text/x-wiki bhkgxrmjz5sfvy3xoiff7mmbawgewn2 0 460561 2450762 2450618 2026-08-24T10:42:33Z GGoncalves-WMF 42848 /* mediawiki_api_request */ typo 2450762 wikitext text/x-wiki 5dm3xxvlc7chh14mw41a1rqawtrbl4t