Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.9 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 3086 2431907 2431804 2026-06-30T13:38:16Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2431907 wikitext text/x-wiki hki2pcohn1s3jyu6od7nf17hd4iho5l 2431915 2431907 2026-06-30T13:51:17Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2431915 wikitext text/x-wiki ithf4y8ifnoxelxbew2gi1vgboll8k4 0 3205 2431881 2399835 2026-06-30T13:00:16Z CMooney (WMF) 24288 /* See which LVS balancer is active for a given service */ 2431881 wikitext text/x-wiki h7ybpboymw5e1vvsz71emo2vncsy352 2431988 2431881 2026-06-30T15:17:43Z BCornwall-WMF 31521 /* See which LVS balancer is active for a given service */ use outdated template 2431988 wikitext text/x-wiki ox7zzqydp04h7t5irxsq4rnw7n2p2l7 0 3818 2431872 2401732 2026-06-30T12:32:41Z MMuhlenhoff (WMF) 5268 /* Making changes */ Remove misleading/outdated example 2431872 wikitext text/x-wiki 5oyv0lytp4bn8gkbb6i8msuc7zbeume 2431887 2431872 2026-06-30T13:06:26Z MMuhlenhoff (WMF) 5268 /* Making changes */ Document sre.puppet.disable-merges 2431887 wikitext text/x-wiki hifv4wmdcvupgtini0w2yb411ctd51j 2431891 2431887 2026-06-30T13:08:12Z MMuhlenhoff (WMF) 5268 /* Temporarily disabling Puppet merges */ Fix link 2431891 wikitext text/x-wiki qa5pbitjh2ei45q9tf8h5pbvu6tra5a 0 4108 2431879 2431816 2026-06-30T12:57:05Z ScheduleDeploymentBot 37566 Add [[gerrit:1304173]] to Tuesday, June 30 UTC afternoon backport window 2431879 wikitext text/x-wiki 5luky5ou7pyfmbsdewiktpvpyxgdejq 2431882 2431879 2026-06-30T13:00:26Z WBrown (WMF) 37696 /* {{Deployment_day|date=2026-06-30}} */ + 2431882 wikitext text/x-wiki dqysmxy4b8xokr7los12byr3mo8hrx0 2431928 2431882 2026-06-30T14:20:50Z ScheduleDeploymentBot 37566 Add [[gerrit:1306649]] to Wednesday, July 01 UTC morning backport window 2431928 wikitext text/x-wiki mpk5mzw4z43xc7k9lzcivriv1zozqrh 2431939 2431928 2026-06-30T14:38:05Z ScheduleDeploymentBot 37566 Add [[gerrit:1306710]] to Wednesday, July 01 UTC morning backport window 2431939 wikitext text/x-wiki qnlsey56om9oclv9tqexfcb0hbhjy3l 2431940 2431939 2026-06-30T14:38:16Z ScheduleDeploymentBot 37566 Add [[gerrit:1306711]] to Wednesday, July 01 UTC morning backport window 2431940 wikitext text/x-wiki nhs5idxn8q0q8dnduk7bs0bude41ek5 2432026 2431940 2026-06-30T16:50:11Z ScheduleDeploymentBot 37566 Add [[gerrit:1306735]] to Tuesday, June 30 UTC late backport window 2432026 wikitext text/x-wiki 127urcq3yz41mqflixmpmtz64ak5cj4 2432027 2432026 2026-06-30T16:50:34Z ScheduleDeploymentBot 37566 Add [[gerrit:1306737]] to Tuesday, June 30 UTC late backport window 2432027 wikitext text/x-wiki 97ta85ox23xxaxxuslap18tcqd34irx 2432028 2432027 2026-06-30T16:50:42Z ScheduleDeploymentBot 37566 Add [[gerrit:1306141]] to Tuesday, June 30 UTC late backport window 2432028 wikitext text/x-wiki fdlm4g5mhzqd1a8p2zakkrke6as4kzr 2432044 2432028 2026-06-30T17:54:46Z ScheduleDeploymentBot 37566 Add [[gerrit:1306739]] to Tuesday, June 30 UTC late backport window 2432044 wikitext text/x-wiki rdtrkorl8yepl84itx8mmi06eq6rm7d 2432046 2432044 2026-06-30T18:03:31Z ScheduleDeploymentBot 37566 Add [[gerrit:1306746]] to Tuesday, June 30 UTC late backport window 2432046 wikitext text/x-wiki tnzr0uroarshosxx0sprgt6nl4bo7v2 2432068 2432046 2026-06-30T19:18:36Z ScheduleDeploymentBot 37566 Add [[gerrit:1306414]] to Tuesday, June 30 UTC late backport window 2432068 wikitext text/x-wiki 2qwar9sh8u81a7rjd8en0kc515v7pl8 2432136 2432068 2026-06-30T21:59:33Z WBrown (WMF) 37696 /* {{Deployment_day|date=2026-07-02}} */ + 2432136 wikitext text/x-wiki porlo3fsily8zi167xznb4aj293esgk 2432197 2432136 2026-07-01T05:35:32Z ScheduleDeploymentBot 37566 Add [[gerrit:1306842]] to Wednesday, July 01 UTC afternoon backport window 2432197 wikitext text/x-wiki bm7deltly8j81w5muuwr2y3lt7laj0q 2432211 2432197 2026-07-01T06:31:02Z ScheduleDeploymentBot 37566 Add [[gerrit:1306850]] to Wednesday, July 01 UTC morning backport window 2432211 wikitext text/x-wiki q7v6oxrexor1hcrw9y94rfz69gdot0s 2432256 2432211 2026-07-01T08:49:05Z ScheduleDeploymentBot 37566 Add [[gerrit:1306873]] to Wednesday, July 01 UTC afternoon backport window 2432256 wikitext text/x-wiki fb9063ui38kcq2bjsyxfvykl7hxz1dq 0 4120 2431921 2423928 2026-06-30T14:01:27Z RScout-WMF 56866 /* Other tips */ 2431921 wikitext text/x-wiki ox3ody50da2samjhvbpquwtc6wq03jv 2432049 2431921 2026-06-30T18:05:05Z RScout-WMF 56866 We're shifting Gitlab to gitlab-ssh.wikimedia.org, so this SSH config was trying to proxy the connection and would fail. This fixes it so you can continue to connect to Gitlab 2432049 wikitext text/x-wiki ckmukqjr6baa9js6dkwaupyvtpbs3f9 0 4673 2432162 2421563 2026-07-01T00:28:56Z JrandWP 37706 /* 2025-present */ archive 2432162 wikitext text/x-wiki gfnoprevec5islys6ik1e1irzdmxrym 498 6726 2431888 2431779 2026-06-30T13:06:40Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2431888 wikitext text/x-wiki kosby5auhca40lbwo4i1gjz88vj2t0w 2431893 2431888 2026-06-30T13:09:32Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api 2431893 wikitext text/x-wiki 6n28brxgu72zh109h05f3cmvgw12m84 2431898 2431893 2026-06-30T13:25:28Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2431898 wikitext text/x-wiki krzqtdzvp7wji3r1gysch0t57ftnjsm 2431906 2431898 2026-06-30T13:37:15Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2431906 wikitext text/x-wiki 07meond5jhubpq1r3xbpaxs0x9v9v20 0 7919 2431863 2431862 2026-06-30T12:02:51Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2104.codfw.wmnet with reason: host reimage 2431863 wikitext text/x-wiki i58679nl6gkz4p3pt9tkdzgf7zwave0 2431864 2431863 2026-06-30T12:04:05Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2105.codfw.wmnet with reason: host reimage 2431864 wikitext text/x-wiki 25v00qvdys0mtb9y4aded3z4p8lpfep 2431866 2431864 2026-06-30T12:07:21Z Stashbot 7414 elukey: upgrade all bookworm hosts to pywmflib 3.0 - T430552 2431866 wikitext text/x-wiki m25vorvyfhgsxenfgjgtetsicko6kt0 2431867 2431866 2026-06-30T12:08:47Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1261: Migration of db1261.eqiad.wmnet completed 2431867 wikitext text/x-wiki mie72i3unfq322vcd6kqvcjv1fesv0k 2431868 2431867 2026-06-30T12:08:50Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2431868 wikitext text/x-wiki 0iizv87cv2jc7avnwmntlk16b8hi80z 2431869 2431868 2026-06-30T12:10:25Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2104.codfw.wmnet with reason: host reimage 2431869 wikitext text/x-wiki 0f7yzfxmw6pd116csytg144cktx3k0u 2431870 2431869 2026-06-30T12:14:25Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2105.codfw.wmnet with reason: host reimage 2431870 wikitext text/x-wiki 7vq7wlebvqhn06zd6yotl44hyt5268e 2431873 2431870 2026-06-30T12:51:17Z Stashbot 7414 klausman@cumin2002: START - Cookbook sre.dns.netbox 2431873 wikitext text/x-wiki dby86ue92dwp1erlso5b26mqfe806c2 2431875 2431873 2026-06-30T12:56:21Z Stashbot 7414 klausman@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: ml-cache[2001-2003].codfw.wmnet,ml-cache[1001-1003].eqiad.wmnet decommissioned, removing all IPs except the asset tag one - klausman@cumin2002" 2431875 wikitext text/x-wiki 343yecmuy5rwq1z40ettqgaqze30x0y 2431876 2431875 2026-06-30T12:56:58Z Stashbot 7414 klausman@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: ml-cache[2001-2003].codfw.wmnet,ml-cache[1001-1003].eqiad.wmnet decommissioned, removing all IPs except the asset tag one - klausman@cumin2002" 2431876 wikitext text/x-wiki 67zb601ktv2z40ugdag8dra3h28028u 2431877 2431876 2026-06-30T12:57:00Z Stashbot 7414 klausman@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2431877 wikitext text/x-wiki 91czydw2m5xwoydmnd17oozrww75tg7 2431878 2431877 2026-06-30T12:57:03Z Stashbot 7414 klausman@cumin2002: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts ml-cache[2001-2003].codfw.wmnet,ml-cache[1001-1003].eqiad.wmnet 2431878 wikitext text/x-wiki 9etpp21zl8wx8dyy1ak1vw6926srm7b 2431880 2431878 2026-06-30T12:59:33Z Stashbot 7414 ozge@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . 2431880 wikitext text/x-wiki beklvc14zlqk9nayxxka0sc7r1rasrj 2431883 2431880 2026-06-30T13:00:33Z Stashbot 7414 ozge@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . 2431883 wikitext text/x-wiki jqoe3gpr4u4v8m84y8nw85qowevxnl5 2431884 2431883 2026-06-30T13:03:52Z Stashbot 7414 ihurbain@deploy1003: Started scap sync-world: Backport for [[gerrit:1306575|Turn on Parsoid Read views for 25% of English Wikipedia desktop traffic (T430194)]] 2431884 wikitext text/x-wiki rn34fzeednoawqhj95fmwmuk4rmzr1z 2431885 2431884 2026-06-30T13:04:47Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2078.codfw.wmnet with OS trixie 2431885 wikitext text/x-wiki o7uy3vsfqhz4l7zaj7f7hgisp74b2ta 2431886 2431885 2026-06-30T13:05:59Z Stashbot 7414 ihurbain@deploy1003: ihurbain: Backport for [[gerrit:1306575|Turn on Parsoid Read views for 25% of English Wikipedia desktop traffic (T430194)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431886 wikitext text/x-wiki 0wp24yxzeckjt38grcxif48ubyqdwew 2431889 2431886 2026-06-30T13:06:59Z Stashbot 7414 ihurbain@deploy1003: ihurbain: Continuing with deployment 2431889 wikitext text/x-wiki 2pld86i0wtt4luyjun2q1wzc7go399d 2431890 2431889 2026-06-30T13:07:27Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2094.codfw.wmnet with OS trixie 2431890 wikitext text/x-wiki ctmidshudtvv9rlccvcuojray97opuy 2431892 2431890 2026-06-30T13:09:03Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2070.codfw.wmnet with OS trixie 2431892 wikitext text/x-wiki dk3ps31fdphy3zzqk4yf4hwneem5b14 2431894 2431892 2026-06-30T13:11:21Z Stashbot 7414 ihurbain@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306575|Turn on Parsoid Read views for 25% of English Wikipedia desktop traffic (T430194)]] (duration: 07m 29s) 2431894 wikitext text/x-wiki jw77os7ssro49r0z7e5n8kbjo1b2057 2431895 2431894 2026-06-30T13:22:17Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2078.codfw.wmnet with reason: host reimage 2431895 wikitext text/x-wiki cd0nk711q8aaieotcpbo8030ozoo5hh 2431896 2431895 2026-06-30T13:23:18Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Started scap sync-world: Backport for [[gerrit:1306341|postEdit: temp account experiment instrumentation (T429110)]], [[gerrit:1306342|maybeSendThankYouEdit: avoid sending notification to temp users (T429110 T424205)]] 2431896 wikitext text/x-wiki t35ishocd240oejpq0q8b5kjyx4rdgc 2431897 2431896 2026-06-30T13:25:23Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde, migr: Backport for [[gerrit:1306341|postEdit: temp account experiment instrumentation (T429110)]], [[gerrit:1306342|maybeSendThankYouEdit: avoid sending notification to temp users (T429110 T424205)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431897 wikitext text/x-wiki sb176m89vgmuwej9nuy5psjo37sekoq 2431899 2431897 2026-06-30T13:26:01Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db1262.eqiad.wmnet with OS trixie 2431899 wikitext text/x-wiki b3i2a7qpn3iyf3wg1cgyvvj2riy0wbp 2431901 2431899 2026-06-30T13:27:05Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2094.codfw.wmnet with reason: host reimage 2431901 wikitext text/x-wiki 1xwh9rpsf2czdo0sv1t3qdza7eyk97m 2431902 2431901 2026-06-30T13:27:42Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2070.codfw.wmnet with reason: host reimage 2431902 wikitext text/x-wiki nbcfilo8dysy0634dxm3h9yxbtyv2ny 2431903 2431902 2026-06-30T13:30:32Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2078.codfw.wmnet with reason: host reimage 2431903 wikitext text/x-wiki qzl40vmvsjk4j06m9okg1959yxynvyp 2431904 2431903 2026-06-30T13:34:08Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde, migr: Continuing with deployment 2431904 wikitext text/x-wiki ddzech3r5j2iqolzjpjyb66rrb14sn5 2431905 2431904 2026-06-30T13:35:06Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2070.codfw.wmnet with reason: host reimage 2431905 wikitext text/x-wiki cstw7omn7n68ui1vs01k83rw6j802ty 2431908 2431905 2026-06-30T13:38:30Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306341|postEdit: temp account experiment instrumentation (T429110)]], [[gerrit:1306342|maybeSendThankYouEdit: avoid sending notification to temp users (T429110 T424205)]] (duration: 15m 12s) 2431908 wikitext text/x-wiki 4p97k7h4w98bg9cnb3oepacb5qtby9b 2431909 2431908 2026-06-30T13:39:16Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2094.codfw.wmnet with reason: host reimage 2431909 wikitext text/x-wiki lnskgi42rjq415a56uwhrt0uv5rzoxm 2431910 2431909 2026-06-30T13:42:48Z Stashbot 7414 bpirkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1304173|REST: remove obsolete and unnecessary config entries (T422770 T423058 T422771)]] 2431910 wikitext text/x-wiki bspvjen4m9bfoo41hl2pb4idzhz4lca 2431911 2431910 2026-06-30T13:42:58Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1262.eqiad.wmnet with reason: host reimage 2431911 wikitext text/x-wiki gozav4091igxwt74c3cqyn9qt3e0muk 2431912 2431911 2026-06-30T13:45:03Z Stashbot 7414 bpirkle@deploy1003: bpirkle: Backport for [[gerrit:1304173|REST: remove obsolete and unnecessary config entries (T422770 T423058 T422771)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431912 wikitext text/x-wiki 3irywjfjuvbgj0ybdpb2t34n38w0qxi 2431913 2431912 2026-06-30T13:48:46Z Stashbot 7414 bpirkle@deploy1003: bpirkle: Continuing with deployment 2431913 wikitext text/x-wiki sxu2fghskh8svx5im8dlo5igvzgkdy5 2431914 2431913 2026-06-30T13:50:04Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1262.eqiad.wmnet with reason: host reimage 2431914 wikitext text/x-wiki tbgm7gf9o2z5aavpzb8uroiupuhmrxh 2431916 2431914 2026-06-30T13:52:17Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2078.codfw.wmnet with OS trixie 2431916 wikitext text/x-wiki 1tho9l3wa08llwcektmr72qv671m59x 2431917 2431916 2026-06-30T13:53:10Z Stashbot 7414 bpirkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1304173|REST: remove obsolete and unnecessary config entries (T422770 T423058 T422771)]] (duration: 10m 21s) 2431917 wikitext text/x-wiki qv34mnip2b9pio797v05fqrexb0fzpk 2431918 2431917 2026-06-30T13:56:44Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2070.codfw.wmnet with OS trixie 2431918 wikitext text/x-wiki bm5r3uye98kbcbe818fejn8rns0azuf 2431919 2431918 2026-06-30T14:00:03Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' . 2431919 wikitext text/x-wiki 3995kiw06bymehup880o7wc1ucpsoui 2431920 2431919 2026-06-30T14:01:24Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2094.codfw.wmnet with OS trixie 2431920 wikitext text/x-wiki 48p9cxbmhr2g87tj2xvqf6zoq3wvh5r 2431922 2431920 2026-06-30T14:06:57Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1262.eqiad.wmnet with OS trixie 2431922 wikitext text/x-wiki t68wnym9lh679h76jpy03rcimpidga2 2431923 2431922 2026-06-30T14:08:21Z Stashbot 7414 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1305867|[Growth] frwiki: Deploy automated mentor list cleaner (T427386)]] 2431923 wikitext text/x-wiki kwrjk8smbu87r9s87k234zy84ipkqni 2431924 2431923 2026-06-30T14:10:27Z Stashbot 7414 urbanecm@deploy1003: urbanecm: Backport for [[gerrit:1305867|[Growth] frwiki: Deploy automated mentor list cleaner (T427386)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431924 wikitext text/x-wiki rkitmo6r5cge2lefkizoch2sdj9k8g6 2431925 2431924 2026-06-30T14:10:51Z Stashbot 7414 urbanecm@deploy1003: urbanecm: Continuing with deployment 2431925 wikitext text/x-wiki sr58rh7xzarsfl7j3dmd2azzt77433d 2431926 2431925 2026-06-30T14:15:11Z Stashbot 7414 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1305867|[Growth] frwiki: Deploy automated mentor list cleaner (T427386)]] (duration: 06m 50s) 2431926 wikitext text/x-wiki lwggzche8mvuwutu5d01q196t65rkhw 2431927 2431926 2026-06-30T14:18:18Z Stashbot 7414 Lucas_WMDE: UTC afternoon backport+config window done 2431927 wikitext text/x-wiki 5f5305i4q9dxl5n3baxfs4xbj1aofdd 2431930 2431927 2026-06-30T14:23:39Z Stashbot 7414 klausman@cumin2002: START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: ml-staging-worker@codfw 2431930 wikitext text/x-wiki 9bzza1zu6drqxus199zeyiuptylegq4 2431931 2431930 2026-06-30T14:26:16Z Stashbot 7414 klausman@cumin2002: START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs 2431931 wikitext text/x-wiki t9k24h3f9hnsguguewa871ezuldakyy 2431933 2431931 2026-06-30T14:27:24Z Stashbot 7414 klausman@cumin2002: END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs 2431933 wikitext text/x-wiki jlg1tbxga5sn74xe5eu5lmcvi0e1b0f 2431934 2431933 2026-06-30T14:27:27Z Stashbot 7414 klausman@cumin2002: END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: ml-staging-worker@codfw 2431934 wikitext text/x-wiki qim1rac2gibend86lsdsxlg1f8vapc4 2431937 2431934 2026-06-30T14:32:41Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1262: Migration of db1262.eqiad.wmnet completed 2431937 wikitext text/x-wiki tmtat2tour0vhkvs5hfjm6a5svlx0af 2431938 2431937 2026-06-30T14:36:55Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2095.codfw.wmnet with OS trixie 2431938 wikitext text/x-wiki 37bdnu9qo33kmvbz7r8fwj83t1ybxmr 2431941 2431938 2026-06-30T14:38:53Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2096.codfw.wmnet with OS trixie 2431941 wikitext text/x-wiki qhyqguc4b6xsc1oejnzzqeqoop73rq3 2431942 2431941 2026-06-30T14:40:44Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2097.codfw.wmnet with OS trixie 2431942 wikitext text/x-wiki hy9ru6nz27ta0hou6l8remimsbggnng 2431944 2431942 2026-06-30T14:42:13Z Stashbot 7414 jforrester@deploy1003: Started scap sync-world: Backport for [[gerrit:1306652|abstractwiki: Show Abstract provenance notice to all readers, not just sysops (T422710)]], [[gerrit:1306709|abstractwiki: Stop mis-setting the relevant title on Abstract surfaces (T422655)]], [[gerrit:1306703|SkinComponentFooter: Show copyright for known, not just existing, pages (T422655)]] 2431944 wikitext text/x-wiki nqjmfj4q3wvb2ufg0pbv3mjnibt66kl 2431945 2431944 2026-06-30T14:42:22Z Stashbot 7414 klausman@cumin2002: START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: ml-staging-worker@codfw 2431945 wikitext text/x-wiki exryr4y2tqooqvy01od3pjw8rp2h2b6 2431946 2431945 2026-06-30T14:43:32Z Stashbot 7414 jforrester@deploy1003: Started scap sync-world: Backport for [[gerrit:1306652|abstractwiki: Show Abstract provenance notice to all readers, not just sysops (T422710)]], [[gerrit:1306709|abstractwiki: Stop mis-setting the relevant title on Abstract surfaces (T422655)]], [[gerrit:1306703|SkinComponentFooter: Show copyright for known, not just existing, pages (T422655)]] 2431946 wikitext text/x-wiki 07fhchjxb7a1i0a8hi1e79p7esx7lig 2431947 2431946 2026-06-30T14:45:34Z Stashbot 7414 jforrester@deploy1003: jforrester: Backport for [[gerrit:1306652|abstractwiki: Show Abstract provenance notice to all readers, not just sysops (T422710)]], [[gerrit:1306709|abstractwiki: Stop mis-setting the relevant title on Abstract surfaces (T422655)]], [[gerrit:1306703|SkinComponentFooter: Show copyright for known, not just existing, pages (T422655)]] synced to the testservers (see https://wikitech.wikimedia.org/w 2431947 wikitext text/x-wiki k30qrthrdsu1ia0sr8qygaxx2mkjs49 2431948 2431947 2026-06-30T14:46:01Z Stashbot 7414 jforrester@deploy1003: jforrester: Continuing with deployment 2431948 wikitext text/x-wiki 8uk4djncoy0x0lox92op67uarx66epq 2431949 2431948 2026-06-30T14:46:07Z Stashbot 7414 klausman@cumin2002: START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs 2431949 wikitext text/x-wiki qcmcfm4cf6zzpg19tlbfwkynxm85w7a 2431951 2431949 2026-06-30T14:47:14Z Stashbot 7414 klausman@cumin2002: END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs 2431951 wikitext text/x-wiki hibclif7e8hfgeopkztf9gbmuf0f995 2431952 2431951 2026-06-30T14:47:17Z Stashbot 7414 klausman@cumin2002: END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: ml-staging-worker@codfw 2431952 wikitext text/x-wiki c748y1r3svby93hlmp9tocvhl4qffpv 2431953 2431952 2026-06-30T14:48:46Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/growthbook: apply 2431953 wikitext text/x-wiki 59szvplcej1ngnly7xbs9ftxfxh8y4l 2431954 2431953 2026-06-30T14:49:06Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/growthbook: apply 2431954 wikitext text/x-wiki 0a6ao7z7qwa8c5svdnhxxe0fd0x9mnf 2431955 2431954 2026-06-30T14:50:27Z Stashbot 7414 jforrester@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306652|abstractwiki: Show Abstract provenance notice to all readers, not just sysops (T422710)]], [[gerrit:1306709|abstractwiki: Stop mis-setting the relevant title on Abstract surfaces (T422655)]], [[gerrit:1306703|SkinComponentFooter: Show copyright for known, not just existing, pages (T422655)]] (duration: 06m 55s) 2431955 wikitext text/x-wiki lklq3el2co35uqq3o8pd65eexj7a44k 2431956 2431955 2026-06-30T14:54:27Z Stashbot 7414 ottomata: roll restart eventgates to re-cache page_change related schemas - T423583#12067516 2431956 wikitext text/x-wiki 3031jjzfllhzgrswze86itkqqqky9gm 2431957 2431956 2026-06-30T14:54:57Z Stashbot 7414 otto@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-analytics: sync 2431957 wikitext text/x-wiki ll5ontqieb4s4mzlwu20qc4ni88zhrn 2431958 2431957 2026-06-30T14:55:01Z Stashbot 7414 otto@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-analytics: sync 2431958 wikitext text/x-wiki 6hor1ncbefoi83ba43u8aq9b5wy87eq 2431959 2431958 2026-06-30T14:55:12Z Stashbot 7414 otto@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-analytics: sync 2431959 wikitext text/x-wiki bqxx3cckyhl7thzm90oq3st6lrwfewc 2431961 2431959 2026-06-30T14:55:52Z Stashbot 7414 otto@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-analytics: sync 2431961 wikitext text/x-wiki 9hu37k0m6j76cbkaatbrxqqkovnw13u 2431962 2431961 2026-06-30T14:56:26Z Stashbot 7414 otto@deploy1003: helmfile [eqiad] START helmfile.d/services/eventgate-analytics: sync 2431962 wikitext text/x-wiki fz8qbqonexu7860m5qpgqgm0m34bpc9 2431963 2431962 2026-06-30T14:57:01Z Stashbot 7414 otto@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventgate-analytics: sync 2431963 wikitext text/x-wiki ehxqof4v7bwwuxxfz258gkfmg5s20qo 2431964 2431963 2026-06-30T14:57:06Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2095.codfw.wmnet with reason: host reimage 2431964 wikitext text/x-wiki gshjf39kn5tqaj5ju2vfsupkaf5ozy9 2431965 2431964 2026-06-30T14:57:22Z Stashbot 7414 otto@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-analytics-external: sync 2431965 wikitext text/x-wiki et3cfhw57x4jbox88j3wf38tqpl6pn4 2431966 2431965 2026-06-30T14:57:30Z Stashbot 7414 otto@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-analytics-external: sync 2431966 wikitext text/x-wiki 35cnrtlzb5a8ofofn8fuq3ngh6xblie 2431967 2431966 2026-06-30T14:57:43Z Stashbot 7414 otto@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-analytics-external: sync 2431967 wikitext text/x-wiki 2z7fp680vvw6zpqrmkngcyd9ba7vjjy 2431968 2431967 2026-06-30T14:58:09Z Stashbot 7414 otto@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-analytics-external: sync 2431968 wikitext text/x-wiki 34enxybcc1dn8u8x5ryyrao7y9c09zs 2431969 2431968 2026-06-30T14:58:18Z Stashbot 7414 otto@deploy1003: helmfile [eqiad] START helmfile.d/services/eventgate-analytics-external: sync 2431969 wikitext text/x-wiki 1p76qezxuyj08yevvxbulr41ierrnac 2431970 2431969 2026-06-30T14:58:39Z Stashbot 7414 otto@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventgate-analytics-external: sync 2431970 wikitext text/x-wiki lju70ocz7zfkhzc9dl4j4f1bsq2ms67 2431971 2431970 2026-06-30T14:59:08Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2096.codfw.wmnet with reason: host reimage 2431971 wikitext text/x-wiki 1733xh8o55alx1heimyqfc50faul56z 2431972 2431971 2026-06-30T14:59:12Z Stashbot 7414 otto@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-main: sync 2431972 wikitext text/x-wiki 75ce47hbkjb2xk8jebqo96ch10jgo5z 2431973 2431972 2026-06-30T14:59:22Z Stashbot 7414 otto@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-main: sync 2431973 wikitext text/x-wiki e184o3qlhx8df94lewgp9hxj6lmyots 2431974 2431973 2026-06-30T14:59:43Z Stashbot 7414 otto@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-main: sync 2431974 wikitext text/x-wiki egc3ysypzuwreclmy42tb9wgp0854yj 2431975 2431974 2026-06-30T14:59:52Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.network.peering with action 'configure' for AS: 30818 2431975 wikitext text/x-wiki ma011tu11sizpn53l5kmlhcdcm4b7gt 2431976 2431975 2026-06-30T15:00:09Z Stashbot 7414 otto@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-main: sync 2431976 wikitext text/x-wiki 6tpuzxtqcwpeflsqoey7wqae1nc0w20 2431977 2431976 2026-06-30T15:00:14Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2097.codfw.wmnet with reason: host reimage 2431977 wikitext text/x-wiki 1tihe4hbx8kd9evxc39324nk7w3eb1f 2431978 2431977 2026-06-30T15:00:48Z Stashbot 7414 otto@deploy1003: helmfile [eqiad] START helmfile.d/services/eventgate-main: sync 2431978 wikitext text/x-wiki pejzo6n5jsic3lihrnstktngxgc0w4e 2431979 2431978 2026-06-30T15:00:50Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 30818 2431979 wikitext text/x-wiki 10q6qb3yskq6ddz5hb254okn822ket0 2431980 2431979 2026-06-30T15:00:55Z Stashbot 7414 otto@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventgate-main: sync 2431980 wikitext text/x-wiki 5ue46uzcij3j273cl4nqp225moq0hc0 2431981 2431980 2026-06-30T15:05:50Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2095.codfw.wmnet with reason: host reimage 2431981 wikitext text/x-wiki cqsw5ugdi4f8pgwt7v9wmcivgq7o9vt 2431984 2431981 2026-06-30T15:09:38Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2096.codfw.wmnet with reason: host reimage 2431984 wikitext text/x-wiki kkjguolol7pv9bsauvr2x2auztqnlpg 2431986 2431984 2026-06-30T15:14:14Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2097.codfw.wmnet with reason: host reimage 2431986 wikitext text/x-wiki pcue2zuai0idd1hc259y49d6qg19upi 2431987 2431986 2026-06-30T15:17:28Z Stashbot 7414 urbanecm@deploy1003: mwscript-k8s job started: GrowthExperiments:cleanMentorList.php --wiki=frwiki # T427386 2431987 wikitext text/x-wiki 6g4wnuwa6xpzkouj5dguqhkefrsx75i 2431989 2431987 2026-06-30T15:18:12Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1262: Migration of db1262.eqiad.wmnet completed 2431989 wikitext text/x-wiki ewqjtmpuub33ql05qebn2f23wqwlfae 2431990 2431989 2026-06-30T15:18:14Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2431990 wikitext text/x-wiki eyw6i9os6q6wkgk6bzsdvi1vada4v0b 2431992 2431990 2026-06-30T15:23:32Z Stashbot 7414 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1306718|Revert "[Growth] frwiki: Deploy automated mentor list cleaner" (T427386)]] 2431992 wikitext text/x-wiki 7hz3oj6lhkb87x0a7pkfwkllz29ub38 2431993 2431992 2026-06-30T15:25:43Z Stashbot 7414 urbanecm@deploy1003: urbanecm: Backport for [[gerrit:1306718|Revert "[Growth] frwiki: Deploy automated mentor list cleaner" (T427386)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2431993 wikitext text/x-wiki 1fdre7l8hipqm4ecscuz2n7jhpzpl4b 2431994 2431993 2026-06-30T15:27:10Z Stashbot 7414 urbanecm@deploy1003: urbanecm: Continuing with deployment 2431994 wikitext text/x-wiki bbiagdrj18eb79uew2o7fnx82qxkf15 2431995 2431994 2026-06-30T15:28:44Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2095.codfw.wmnet with OS trixie 2431995 wikitext text/x-wiki jna1buxwuyc7gwsg36ka83sh1oz0reg 2431996 2431995 2026-06-30T15:29:25Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2096.codfw.wmnet with OS trixie 2431996 wikitext text/x-wiki 96524yvw209u251hzv6on8ppjeyf33q 2431997 2431996 2026-06-30T15:31:33Z Stashbot 7414 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306718|Revert "[Growth] frwiki: Deploy automated mentor list cleaner" (T427386)]] (duration: 08m 01s) 2431997 wikitext text/x-wiki 12o9k3ek3ol30h9m3u8ksru3ym10pps 2431999 2431997 2026-06-30T15:36:27Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2097.codfw.wmnet with OS trixie 2431999 wikitext text/x-wiki jqtss942333qefy4slb8m87lvrdnmej 2432000 2431999 2026-06-30T15:37:42Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2432000 wikitext text/x-wiki 150rbdt3niyjnlcrx70wtl1dxh30isx 2432001 2432000 2026-06-30T15:37:50Z Stashbot 7414 cwilliams@cumin1003: dbmaint on s4@eqiad T429893 2432001 wikitext text/x-wiki qluxtsop04yimpu3ntqgv5h7qod8ovu 2432002 2432001 2026-06-30T15:38:03Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db1263: Upgrading db1263.eqiad.wmnet 2432002 wikitext text/x-wiki ia2jn82rhwnoh71rsmfthn94oqopqto 2432003 2432002 2026-06-30T15:38:33Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1263: Upgrading db1263.eqiad.wmnet 2432003 wikitext text/x-wiki 53rvqu3lg9qwgm709ewgr01wxu89daw 2432004 2432003 2026-06-30T15:42:46Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1306689|ConfirmEdit: Show a clear message when a login CAPTCHA is missing (T428892)]], [[gerrit:1306688|ConfirmEdit: Show a clear message when a login CAPTCHA is missing (T428892)]] 2432004 wikitext text/x-wiki gaalgomb11fime6t3k5ccssz9ntewmf 2432005 2432004 2026-06-30T16:02:10Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1306689|ConfirmEdit: Show a clear message when a login CAPTCHA is missing (T428892)]], [[gerrit:1306688|ConfirmEdit: Show a clear message when a login CAPTCHA is missing (T428892)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2432005 wikitext text/x-wiki l4fgwc76g0srp8lypkzvp9uy1qehy2p 2432006 2432005 2026-06-30T16:02:53Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2432006 wikitext text/x-wiki 7o140ykts2eiv62fuh9l67rstbyw6p3 2432007 2432006 2026-06-30T16:11:34Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db1263.eqiad.wmnet with OS trixie 2432007 wikitext text/x-wiki tvc0w7ejkwrx5c7e6cc25cmtmjz7j4m 2432008 2432007 2026-06-30T16:16:28Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306689|ConfirmEdit: Show a clear message when a login CAPTCHA is missing (T428892)]], [[gerrit:1306688|ConfirmEdit: Show a clear message when a login CAPTCHA is missing (T428892)]] (duration: 33m 43s) 2432008 wikitext text/x-wiki a5dpl6e5a2yqzvjviuu24pysbx7e2u7 2432013 2432008 2026-06-30T16:28:04Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1263.eqiad.wmnet with reason: host reimage 2432013 wikitext text/x-wiki qlfi7c2ebfz636oijwawgobktcg2omt 2432015 2432013 2026-06-30T16:32:35Z Stashbot 7414 bking@cumin2003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cirrussearch2073.codfw.wmnet with OS trixie 2432015 wikitext text/x-wiki d51qby6ahwfwvlqvqumw1t7ejm5wkzr 2432016 2432015 2026-06-30T16:34:48Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cirrussearch2089.codfw.wmnet'] 2432016 wikitext text/x-wiki ho7oxbsjfomxtao4f7lnni3ki3kx3ou 2432017 2432016 2026-06-30T16:35:07Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1263.eqiad.wmnet with reason: host reimage 2432017 wikitext text/x-wiki het7c268uxtldpqsibdmehrwh25nyjt 2432018 2432017 2026-06-30T16:38:28Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2432018 wikitext text/x-wiki lw2lze612qjf46m11mmob83uwmvflxk 2432019 2432018 2026-06-30T16:39:21Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2432019 wikitext text/x-wiki 2ca1tc63neehndncd4be5beijbs76om 2432020 2432019 2026-06-30T16:40:18Z Stashbot 7414 bking@cumin2003: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['cirrussearch2089.codfw.wmnet'] 2432020 wikitext text/x-wiki 2b7cvnuvbxckuaj2rj6r0a5dz2re3s6 2432021 2432020 2026-06-30T16:43:59Z Stashbot 7414 rzl@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-cron: apply 2432021 wikitext text/x-wiki 63towq6l0fnlr968q8odgjsvy0oxcwz 2432022 2432021 2026-06-30T16:44:09Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s_services/services/datahub-next: apply 2432022 wikitext text/x-wiki hoxlvbzdqjep0o8kmhny58310ueefp9 2432023 2432022 2026-06-30T16:44:12Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s_services/services/datahub-next: apply 2432023 wikitext text/x-wiki k04kgxds99c1d396dk3zm9w6js7qumq 2432024 2432023 2026-06-30T16:44:28Z Stashbot 7414 rzl@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-cron: apply 2432024 wikitext text/x-wiki b6tu3dm44nnf74xs265v3ka2rrdzvpc 2432029 2432024 2026-06-30T16:52:54Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1263.eqiad.wmnet with OS trixie 2432029 wikitext text/x-wiki pvoetpfzk3dmxgk5zw72wd5wm2bsa12 2432033 2432029 2026-06-30T17:18:24Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1263: Migration of db1263.eqiad.wmnet completed 2432033 wikitext text/x-wiki 8ysmxr8rg2w9ba3lrshm68whqobrf2j 2432036 2432033 2026-06-30T17:41:59Z Stashbot 7414 sukhe@dns1004: START - running authdns-update 2432036 wikitext text/x-wiki 8msj5ywiz0g9jijcev8yzd5zsc9a0h1 2432037 2432036 2026-06-30T17:43:59Z Stashbot 7414 sukhe@dns1004: END - running authdns-update 2432037 wikitext text/x-wiki m4vbpsxdl9diprnno69da9gj2omjs6t 2432038 2432037 2026-06-30T17:44:46Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.dns.wipe-cache url-downloader.eqiad.wikimedia.org on all recursors 2432038 wikitext text/x-wiki p1t41jeld0vaiq5o5b0z9x76jton0z4 2432039 2432038 2026-06-30T17:44:50Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) url-downloader.eqiad.wikimedia.org on all recursors 2432039 wikitext text/x-wiki rn2dgp9ddfne4w3kau3fxodyc1nche4 2432040 2432039 2026-06-30T17:47:49Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2110.codfw.wmnet with OS trixie 2432040 wikitext text/x-wiki t3xkcmw970ejd5jfd09nlnmsu1ljer1 2432041 2432040 2026-06-30T17:48:18Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=no; selector: name=cp3081.* 2432041 wikitext text/x-wiki d09b74x7ude9psw54psa030lxt9vwml 2432042 2432041 2026-06-30T17:49:01Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2079.codfw.wmnet with OS trixie 2432042 wikitext text/x-wiki 56kl0p1shbsu78k3spj7pn5jb7hxcpv 2432043 2432042 2026-06-30T17:52:18Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2065.codfw.wmnet with OS trixie 2432043 wikitext text/x-wiki 0dq32516fqhaefworh81005hqrv2o3x 2432045 2432043 2026-06-30T17:58:53Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.reimage for host cp3081.esams.wmnet with OS trixie 2432045 wikitext text/x-wiki 1djxol266jmlpfnak81n26ypk8ffpm3 2432047 2432045 2026-06-30T18:03:55Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1263: Migration of db1263.eqiad.wmnet completed 2432047 wikitext text/x-wiki j8nhx8jo2yx1wt02umh2tkipnvkicjq 2432048 2432047 2026-06-30T18:03:57Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2432048 wikitext text/x-wiki cn0ra5j68gxrk8fbbjoz1pn2f8hw61c 2432050 2432048 2026-06-30T18:06:14Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2079.codfw.wmnet with reason: host reimage 2432050 wikitext text/x-wiki r0qntnw28f5gm3ryhobo77cqb0utj2g 2432052 2432050 2026-06-30T18:07:05Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2110.codfw.wmnet with reason: host reimage 2432052 wikitext text/x-wiki oggexdx38wk37v4kbec9hklxdtz1x4a 2432053 2432052 2026-06-30T18:09:00Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2079.codfw.wmnet with reason: host reimage 2432053 wikitext text/x-wiki 9elk5wec9ym84rcshywqd0xbggnsrn0 2432054 2432053 2026-06-30T18:10:22Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2065.codfw.wmnet with reason: host reimage 2432054 wikitext text/x-wiki 0nm0w9s4zmdt8ezb8d71wl7t1lsuju0 2432055 2432054 2026-06-30T18:12:01Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2110.codfw.wmnet with reason: host reimage 2432055 wikitext text/x-wiki baqsl92j9m8y1r4kd6f32gr8nl3fcbx 2432056 2432055 2026-06-30T18:19:41Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2065.codfw.wmnet with reason: host reimage 2432056 wikitext text/x-wiki rgbyn92o4olbj3s9oasw49o4nl13u8y 2432057 2432056 2026-06-30T18:23:25Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cp3081.esams.wmnet with reason: host reimage 2432057 wikitext text/x-wiki mugntnxorv008ouc6obwkhqijv0c2zw 2432058 2432057 2026-06-30T18:29:10Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp3081.esams.wmnet with reason: host reimage 2432058 wikitext text/x-wiki 3pnrwq0b291n2baanng7fx7rk88dm3t 2432059 2432058 2026-06-30T18:29:46Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2079.codfw.wmnet with OS trixie 2432059 wikitext text/x-wiki 5p9agpeh081u12mi254d7bnlsi7j778 2432060 2432059 2026-06-30T18:35:04Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2110.codfw.wmnet with OS trixie 2432060 wikitext text/x-wiki 9u17i2jmggi732plpk26qsehvdurjro 2432061 2432060 2026-06-30T18:40:31Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2065.codfw.wmnet with OS trixie 2432061 wikitext text/x-wiki c2qx6bsaw5y4wuiti1174jj8nzt6q24 2432062 2432061 2026-06-30T18:54:31Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp3081.esams.wmnet with OS trixie 2432062 wikitext text/x-wiki cwle558z76t1nnswc1zagvpy0xvj67u 2432063 2432062 2026-06-30T18:55:31Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp3081.* 2432063 wikitext text/x-wiki h7prszfwmdlp4yo69lze59lbtwomaf0 2432065 2432063 2026-06-30T19:13:05Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host cirrussearch2089.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2432065 wikitext text/x-wiki ltptpkufbenguv0a3db0gl4vizipl86 2432066 2432065 2026-06-30T19:13:25Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2082.codfw.wmnet with OS trixie 2432066 wikitext text/x-wiki 5nq3oc2nxbm3gm7dbjuw57w40v8zu1m 2432067 2432066 2026-06-30T19:15:54Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2066.codfw.wmnet with OS trixie 2432067 wikitext text/x-wiki 9ka8kwukjdf722ia1zqevgoglrncwt8 2432069 2432067 2026-06-30T19:24:38Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cirrussearch2089.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2432069 wikitext text/x-wiki 5qqn8quxj3w64bao2k8aeazsjql0kvi 2432070 2432069 2026-06-30T19:30:48Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2082.codfw.wmnet with reason: host reimage 2432070 wikitext text/x-wiki jnanntfg7ln0eyg1b9q2fiap916nqat 2432071 2432070 2026-06-30T19:31:02Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2101.codfw.wmnet with OS trixie 2432071 wikitext text/x-wiki 02p80aky40v6se2wc0ryyksoqp4tjy9 2432072 2432071 2026-06-30T19:34:26Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2066.codfw.wmnet with reason: host reimage 2432072 wikitext text/x-wiki 69tk6vhk9rp1duixuyfcfe2rform8kx 2432073 2432072 2026-06-30T19:34:57Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2082.codfw.wmnet with reason: host reimage 2432073 wikitext text/x-wiki 2irsx7qzflcr2muczaopfq8zmxdr7ts 2432075 2432073 2026-06-30T19:39:09Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2066.codfw.wmnet with reason: host reimage 2432075 wikitext text/x-wiki as31s3mwvnz49jou3qmlttzdixdhh4p 2432078 2432075 2026-06-30T19:50:50Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2089.codfw.wmnet with OS trixie 2432078 wikitext text/x-wiki 00qhwoyzkjcoe4mtk8gsy4hyyjb6zo5 2432079 2432078 2026-06-30T19:51:17Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2101.codfw.wmnet with reason: host reimage 2432079 wikitext text/x-wiki 9dnifjgrc9f6yt3seg94jqcfpo44w11 2432080 2432079 2026-06-30T19:55:31Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2101.codfw.wmnet with reason: host reimage 2432080 wikitext text/x-wiki 3ow7f5lvb34pkd6db8orb9pbh4p7rtn 2432081 2432080 2026-06-30T19:56:49Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2082.codfw.wmnet with OS trixie 2432081 wikitext text/x-wiki bmvyktl6rociyxujwvcxybtmd9crywr 2432082 2432081 2026-06-30T20:00:49Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2066.codfw.wmnet with OS trixie 2432082 wikitext text/x-wiki 50gm58eqh52fvbzo13btixx7fzh8m5d 2432083 2432082 2026-06-30T20:08:45Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s_services/services/datahub-next: apply 2432083 wikitext text/x-wiki l8hbt4668pg8w7px5eh1k39ondfhq69 2432084 2432083 2026-06-30T20:08:50Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s_services/services/datahub-next: apply 2432084 wikitext text/x-wiki asusocmglw356l9wjx47ap9wcu198bz 2432085 2432084 2026-06-30T20:11:18Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2089.codfw.wmnet with reason: host reimage 2432085 wikitext text/x-wiki 4pspl5p9qmi7nobky2p2398ul3iuyy3 2432086 2432085 2026-06-30T20:11:32Z Stashbot 7414 ebernhardson@deploy1003: Started scap sync-world: Backport for [[gerrit:1306414|Revert^3 "cirrus: AB test query suggester variants" (T407432)]], [[gerrit:1306210|Revert "nlwiki: change to Wikipedia 25 logo" (T424519)]], [[gerrit:1306739|ExtensionDistributor: mark 1.46 as stable (T423272)]] 2432086 wikitext text/x-wiki fdz81ejnfzgrrl9z8lqi73ut85rpksd 2432087 2432086 2026-06-30T20:15:06Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2089.codfw.wmnet with reason: host reimage 2432087 wikitext text/x-wiki 2sy5x9ky4o9dyut64qcn59a454i66o1 2432088 2432087 2026-06-30T20:17:38Z Stashbot 7414 ebernhardson@deploy1003: chlod, ebernhardson, ariel: Backport for [[gerrit:1306414|Revert^3 "cirrus: AB test query suggester variants" (T407432)]], [[gerrit:1306210|Revert "nlwiki: change to Wikipedia 25 logo" (T424519)]], [[gerrit:1306739|ExtensionDistributor: mark 1.46 as stable (T423272)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2432088 wikitext text/x-wiki c2n022rtk5bg4ewaygy3ln59o68wxgq 2432089 2432088 2026-06-30T20:18:11Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2101.codfw.wmnet with OS trixie 2432089 wikitext text/x-wiki 4cqg1tq4jo2eux7hjkn2mqtulnnge34 2432090 2432089 2026-06-30T20:18:38Z Stashbot 7414 ebernhardson@deploy1003: chlod, ebernhardson, ariel: Continuing with deployment 2432090 wikitext text/x-wiki roul2qu4lpmd66z7zyk6l2aak8miz2p 2432091 2432090 2026-06-30T20:25:13Z Stashbot 7414 ebernhardson@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306414|Revert^3 "cirrus: AB test query suggester variants" (T407432)]], [[gerrit:1306210|Revert "nlwiki: change to Wikipedia 25 logo" (T424519)]], [[gerrit:1306739|ExtensionDistributor: mark 1.46 as stable (T423272)]] (duration: 13m 43s) 2432091 wikitext text/x-wiki iey2uaqvzeqwhhbi9nbcf0pyjay64gz 2432092 2432091 2026-06-30T20:27:17Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2101.codfw.wmnet with OS trixie 2432092 wikitext text/x-wiki pdj21wbguiaqnd45tqln7y3n7jlt7x3 2432093 2432092 2026-06-30T20:30:39Z Stashbot 7414 cscott@deploy1003: Started scap sync-world: Backport for [[gerrit:1306746|Turn on Parsoid Read views for 50% of English Wikipedia desktop traffic (T430194)]] 2432093 wikitext text/x-wiki 9cblitill86uh57yfflcms3ipzeszn7 2432094 2432093 2026-06-30T20:32:47Z Stashbot 7414 cscott@deploy1003: cscott: Backport for [[gerrit:1306746|Turn on Parsoid Read views for 50% of English Wikipedia desktop traffic (T430194)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2432094 wikitext text/x-wiki spzx4xcpealvf82oymbk7s1v1ahh22r 2432095 2432094 2026-06-30T20:33:47Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2113.codfw.wmnet with OS trixie 2432095 wikitext text/x-wiki fbmy1wec624qzpz53g4ytg6el84prst 2432096 2432095 2026-06-30T20:34:56Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply 2432096 wikitext text/x-wiki 9odfrxagkxfhstlqv00e44hugr62q9d 2432097 2432096 2026-06-30T20:35:18Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply 2432097 wikitext text/x-wiki aios7b1tpl6ohfd1qfojuyaete8ngvd 2432098 2432097 2026-06-30T20:36:37Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2089.codfw.wmnet with OS trixie 2432098 wikitext text/x-wiki 2utftjayii002nheoso9glg8nt45x9x 2432099 2432098 2026-06-30T20:36:42Z Stashbot 7414 cscott@deploy1003: cscott: Continuing with deployment 2432099 wikitext text/x-wiki 2iufbdydfvcdyecvxia4r0vuh1din3o 2432100 2432099 2026-06-30T20:38:59Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s_services/services/datahub-next: apply 2432100 wikitext text/x-wiki s56i43envndsiq75sgon0tiqtoymxkd 2432101 2432100 2026-06-30T20:39:03Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s_services/services/datahub-next: apply 2432101 wikitext text/x-wiki dfs47z1dvphl65d7oicvoutmyf87ngf 2432102 2432101 2026-06-30T20:41:02Z Stashbot 7414 cscott@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306746|Turn on Parsoid Read views for 50% of English Wikipedia desktop traffic (T430194)]] (duration: 10m 24s) 2432102 wikitext text/x-wiki sszzd0dt8r83uhlrh6sgmm7uwijr2rh 2432103 2432102 2026-06-30T20:41:06Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s_services/services/datahub-next: apply 2432103 wikitext text/x-wiki ajzheuefpg260v2s3h4jjmprseuqqjh 2432104 2432103 2026-06-30T20:41:09Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s_services/services/datahub-next: apply 2432104 wikitext text/x-wiki 48x28zkpydsc3lwljfutew24acrse2e 2432105 2432104 2026-06-30T20:46:51Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2101.codfw.wmnet with reason: host reimage 2432105 wikitext text/x-wiki g61xrurlq22bxmxgo2frr4vihqsw0yj 2432106 2432105 2026-06-30T20:52:04Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2113.codfw.wmnet with reason: host reimage 2432106 wikitext text/x-wiki bbkk1i24ngkl1irdn1j9soadl7q4wrl 2432107 2432106 2026-06-30T20:53:03Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s_services/services/datahub-next: apply 2432107 wikitext text/x-wiki sgpxak73ci9nqwb0pby4psy8wduqhbi 2432108 2432107 2026-06-30T20:53:07Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s_services/services/datahub-next: apply 2432108 wikitext text/x-wiki 96bme12fuacre0jirsqa8u00ertyje9 2432109 2432108 2026-06-30T20:53:56Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2101.codfw.wmnet with reason: host reimage 2432109 wikitext text/x-wiki rx8y44l63rvu85mr3y7510hm0osfu5u 2432110 2432109 2026-06-30T20:55:24Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s_services/services/datahub-next: apply 2432110 wikitext text/x-wiki 7f9vid1ce5zmxverk64lffsm2owwwqw 2432111 2432110 2026-06-30T20:55:27Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s_services/services/datahub-next: apply 2432111 wikitext text/x-wiki 9rad1iaj5feowdqakc3b3n84rz8osvj 2432112 2432111 2026-06-30T20:55:38Z Stashbot 7414 tgr@deploy1003: Started scap sync-world: Backport for [[gerrit:1306735|SecurityLogs: Create by moving code from mediawiki-config (T430564)]], [[gerrit:1306737|SecurityLogs: Add tests (T430564)]], [[gerrit:1306141|Remove security-related log hooks (T430564)]] 2432112 wikitext text/x-wiki 5i71imvwjkanidq0rng3kjad1ebwqkf 2432113 2432112 2026-06-30T20:57:42Z Stashbot 7414 tgr@deploy1003: tgr: Backport for [[gerrit:1306735|SecurityLogs: Create by moving code from mediawiki-config (T430564)]], [[gerrit:1306737|SecurityLogs: Add tests (T430564)]], [[gerrit:1306141|Remove security-related log hooks (T430564)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2432113 wikitext text/x-wiki jjhi927eudtsm20oc4sh22uhymp8p7r 2432114 2432113 2026-06-30T20:58:04Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2113.codfw.wmnet with reason: host reimage 2432114 wikitext text/x-wiki dcwor4gjhyhj0lh8rahkukpm465z7s8 2432115 2432114 2026-06-30T21:09:00Z Stashbot 7414 tgr@deploy1003: tgr: Continuing with deployment 2432115 wikitext text/x-wiki msge6esj61g68i995l53yzci4myywg2 2432116 2432115 2026-06-30T21:11:13Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.netbox 2432116 wikitext text/x-wiki qy9n3af4rmeap0loa6hl9d6pa23rt2k 2432117 2432116 2026-06-30T21:13:21Z Stashbot 7414 tgr@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306735|SecurityLogs: Create by moving code from mediawiki-config (T430564)]], [[gerrit:1306737|SecurityLogs: Add tests (T430564)]], [[gerrit:1306141|Remove security-related log hooks (T430564)]] (duration: 17m 45s) 2432117 wikitext text/x-wiki 15dexh5gtwjwwkwut8cfhpkosvi3k1p 2432118 2432117 2026-06-30T21:14:26Z Stashbot 7414 Dreamy_Jazz: Destroyed NodeJS iPoid service deployments for T416623 2432118 wikitext text/x-wiki lit1xifog7t18rv2z50kvshe8kzbb0z 2432119 2432118 2026-06-30T21:14:38Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2101.codfw.wmnet with OS trixie 2432119 wikitext text/x-wiki dv1ssys8xch4o5jlhgjo6qcx6rc1cmz 2432120 2432119 2026-06-30T21:15:49Z Stashbot 7414 tgr_: UTC late deploysdone 2432120 wikitext text/x-wiki psuzcjg2cvbzc3wcd1mxhml17f0zvuk 2432121 2432120 2026-06-30T21:17:42Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add new link IP dns for trasnport circuits to drmrs - cmooney@cumin1003" 2432121 wikitext text/x-wiki sx2mf1zixa1cseg1vf3nhiu2w4kwwm7 2432122 2432121 2026-06-30T21:19:20Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2113.codfw.wmnet with OS trixie 2432122 wikitext text/x-wiki 4i8znsy8ziepuhqbpvegr487kh256o9 2432123 2432122 2026-06-30T21:19:36Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add new link IP dns for trasnport circuits to drmrs - cmooney@cumin1003" 2432123 wikitext text/x-wiki empsbvvc3288uw8yxurjvhk0qr1mkqe 2432124 2432123 2026-06-30T21:19:39Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2432124 wikitext text/x-wiki 85wcr3y0h8jjlrmsbra8zy9bogz7uim 2432125 2432124 2026-06-30T21:20:23Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2071.codfw.wmnet with OS trixie 2432125 wikitext text/x-wiki oocmxqe6iqumhx7dzxf0kx2i0hej48n 2432127 2432125 2026-06-30T21:25:51Z Stashbot 7414 cmooney@dns3003: START - running authdns-update 2432127 wikitext text/x-wiki 5vhwbw09kr8id7mvzrbi9aw3exi33db 2432128 2432127 2026-06-30T21:27:58Z Stashbot 7414 cmooney@dns3003: END - running authdns-update 2432128 wikitext text/x-wiki n0k28okapahs8uhiwgb17imw1bqussk 2432129 2432128 2026-06-30T21:35:17Z Stashbot 7414 arlolra@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2432129 wikitext text/x-wiki 0td3wejr7wq38t4mkj2muck6lxvgkhz 2432130 2432129 2026-06-30T21:35:47Z Stashbot 7414 arlolra@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2432130 wikitext text/x-wiki 1mjnxsyqqx7vh7yuzybrh1jzja3qmju 2432131 2432130 2026-06-30T21:35:50Z Stashbot 7414 arlolra@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2432131 wikitext text/x-wiki 65vipj2ilek507wxtzc5beip27dyid3 2432132 2432131 2026-06-30T21:36:19Z Stashbot 7414 arlolra@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2432132 wikitext text/x-wiki 7joukanm44akhicctxrjncopsqummwd 2432133 2432132 2026-06-30T21:38:45Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2071.codfw.wmnet with reason: host reimage 2432133 wikitext text/x-wiki ma9y6wvso6dgx9sz3ig0vemixsntc6f 2432134 2432133 2026-06-30T21:44:46Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2071.codfw.wmnet with reason: host reimage 2432134 wikitext text/x-wiki hj6oe61jcanlnd0ipig0f3tdpc9xupj 2432137 2432134 2026-06-30T22:06:17Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2071.codfw.wmnet with OS trixie 2432137 wikitext text/x-wiki gfdaq5ixchle42h31qf1w83vh4v0re7 2432139 2432137 2026-06-30T22:53:55Z Stashbot 7414 dr0ptp4kt: Deploying Refinery at 4e7a2b32 for changes: pageview allowlist 1305158 (+min.wikiquote) 1305162 (+bol.wikipedia), 1305156 (+isv.wikipedia); 1305980 (pv allowlist -api.wikimedia, sqoop +isvwiki); sqoop 1295064 (+globalimagelinks) 1295069 (+filerevision) 2432139 wikitext text/x-wiki h92kp5va494ze67j2fdcych22ep9vmq 2432140 2432139 2026-06-30T22:54:31Z Stashbot 7414 dr0ptp4kt@deploy1003: Started deploy [analytics/refinery@4e7a2b3] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@4e7a2b32] 2432140 wikitext text/x-wiki lytitebcw68x1urcveydgc0d33edjqz 2432141 2432140 2026-06-30T22:56:30Z Stashbot 7414 dr0ptp4kt@deploy1003: Finished deploy [analytics/refinery@4e7a2b3] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@4e7a2b32] (duration: 01m 59s) 2432141 wikitext text/x-wiki ijwgjj9gdpf9l11e5zo7o250rewbhyy 2432142 2432141 2026-06-30T23:00:19Z Stashbot 7414 dr0ptp4kt@deploy1003: Started deploy [analytics/refinery@4e7a2b3]: Regular analytics weekly train [analytics/refinery@4e7a2b32] 2432142 wikitext text/x-wiki qudfq4urzza08sno3ekanripz9xvb0l 2432143 2432142 2026-06-30T23:04:29Z Stashbot 7414 dr0ptp4kt@deploy1003: Finished deploy [analytics/refinery@4e7a2b3]: Regular analytics weekly train [analytics/refinery@4e7a2b32] (duration: 04m 11s) 2432143 wikitext text/x-wiki 9qek0glytzfezqk4yvusc33vvq19hlu 2432144 2432143 2026-06-30T23:04:49Z Stashbot 7414 dr0ptp4kt@deploy1003: Started deploy [analytics/refinery@4e7a2b3] (thin): Regular analytics weekly train THIN [analytics/refinery@4e7a2b32] 2432144 wikitext text/x-wiki 0n7yqh2v3b83ab4knxdq2o5xdtef9wx 2432145 2432144 2026-06-30T23:06:50Z Stashbot 7414 dr0ptp4kt@deploy1003: Finished deploy [analytics/refinery@4e7a2b3] (thin): Regular analytics weekly train THIN [analytics/refinery@4e7a2b32] (duration: 02m 00s) 2432145 wikitext text/x-wiki dihg23s3qkkkt34mnm72tfa65g4dzsp 2432146 2432145 2026-06-30T23:12:44Z Stashbot 7414 ryankemper: T429844 [opensearch] cleared stale/redundant cluster.routing.allocation.* transient settings from all production search clusters; also cleared redundant action.auto_create_index and cluster.routing.use_adaptive_replica_selection transients from codfw chi/psi, and stale transient DEBUG logger overrides from codfw chi 2432146 wikitext text/x-wiki dw4p46yrdxiie5ndmuv6oq3qcv54ikd 2432147 2432146 2026-06-30T23:30:15Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2103.codfw.wmnet with OS trixie 2432147 wikitext text/x-wiki 5fdlvwb5833u43i53ynlacrn7iafmv7 2432148 2432147 2026-06-30T23:31:28Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2102.codfw.wmnet with OS trixie 2432148 wikitext text/x-wiki jqbrh4jlf5ogfjxl1nlgq6vlviuro3u 2432149 2432148 2026-06-30T23:50:09Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2103.codfw.wmnet with reason: host reimage 2432149 wikitext text/x-wiki b6ws4bidyybv4wyvu2hu0gfydba7ikb 2432150 2432149 2026-06-30T23:51:40Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2102.codfw.wmnet with reason: host reimage 2432150 wikitext text/x-wiki nj70fks2v6kmg6x507r6mgjqj6m2mw1 2432151 2432150 2026-06-30T23:54:07Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2103.codfw.wmnet with reason: host reimage 2432151 wikitext text/x-wiki 4vthh8gs09f4wewg7rlidiu5hy6ipew 2432152 2432151 2026-06-30T23:58:00Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2102.codfw.wmnet with reason: host reimage 2432152 wikitext text/x-wiki s6uhi9io5i0fwgwnvcvbudir6yu5oaa 2432154 2432152 2026-07-01T00:05:21Z Stashbot 7414 dr0ptp4kt: DEPLOYED Refinery at 4e7a2b32 for changes: pageview allowlist 1305158 (+min.wikiquote) 1305162 (+bol.wikipedia), 1305156 (+isv.wikipedia); 1305980 (pv allowlist -api.wikimedia, sqoop +isvwiki); sqoop 1295064 (+globalimagelinks) 1295069 (+filerevision) using scap, then deployed onto HDFS (manual copyToLocal required additionally) 2432154 wikitext text/x-wiki nub1ohxvf5b180w6kvlo8ixphgyi82n 2432155 2432154 2026-07-01T00:15:02Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2103.codfw.wmnet with OS trixie 2432155 wikitext text/x-wiki 64ktyyiu588qoa7jqfjarjjxql5mrvg 2432156 2432155 2026-07-01T00:20:12Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2102.codfw.wmnet with OS trixie 2432156 wikitext text/x-wiki 8faslq8dxoa38j7prf1nh91nasrd6x6 2432161 2432156 2026-07-01T00:28:23Z JrandWP 37706 archive 106-107 2432161 wikitext text/x-wiki orz2x6ndjm5el6peo8wamlprneznbtt 2432163 2432161 2026-07-01T00:42:25Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Set es7 eqiad as read-only for maintenance - T430765', diff saved to https://phabricator.wikimedia.org/P94645 and previous config saved to /var/cache/conftool/dbconfig/20260701-004221-ladsgroup.json 2432163 wikitext text/x-wiki 4x2vfc1pqevmifiitojo4c1xkagm14e 2432164 2432163 2026-07-01T00:53:18Z Stashbot 7414 ladsgroup@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 9 hosts with reason: Primary switchover es7 T430765 2432164 wikitext text/x-wiki 1nrlwwkztkihcmbkqc2humlu9n5voib 2432165 2432164 2026-07-01T00:53:31Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Set es1035 with weight 0 T430765', diff saved to https://phabricator.wikimedia.org/P94646 and previous config saved to /var/cache/conftool/dbconfig/20260701-005329-ladsgroup.json 2432165 wikitext text/x-wiki h5gjca7kf9k1w11u6nqv89a9oqwtkrx 2432167 2432165 2026-07-01T00:58:28Z Stashbot 7414 Amir1: Starting es7 eqiad failover from es1039 to es1035 - T430765 2432167 wikitext text/x-wiki b1c90l0hsjbl7ax4ozs45r24181mmp2 2432168 2432167 2026-07-01T01:00:05Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Promote es1035 to es7 primary T430765', diff saved to https://phabricator.wikimedia.org/P94647 and previous config saved to /var/cache/conftool/dbconfig/20260701-010002-ladsgroup.json 2432168 wikitext text/x-wiki p4upb0onrvsrnvknxio5p2en1ik1i4c 2432169 2432168 2026-07-01T01:03:55Z Stashbot 7414 ladsgroup@dns1004: START - running authdns-update 2432169 wikitext text/x-wiki 29wmsg8uxn4gqpxji2pluh8j5e76pmi 2432170 2432169 2026-07-01T01:05:54Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Depool es1039 T430765', diff saved to https://phabricator.wikimedia.org/P94648 and previous config saved to /var/cache/conftool/dbconfig/20260701-010551-ladsgroup.json 2432170 wikitext text/x-wiki py8xkrs0gf8z4kxc315aiog0ob6f4st 2432171 2432170 2026-07-01T01:05:59Z Stashbot 7414 ladsgroup@dns1004: END - running authdns-update 2432171 wikitext text/x-wiki 2tmc1dq6lhqiq6xyglm3oyak5yjfu5f 2432172 2432171 2026-07-01T01:07:19Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Set es7 eqiad back to read-write - T430765', diff saved to https://phabricator.wikimedia.org/P94649 and previous config saved to /var/cache/conftool/dbconfig/20260701-010716-ladsgroup.json 2432172 wikitext text/x-wiki pum612ia9nkvyhd05iqa7vlic1i7ysm 2432173 2432172 2026-07-01T02:00:40Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2432173 wikitext text/x-wiki 4pqwk89rfyr032ktz938ubuendemz8d 2432174 2432173 2026-07-01T02:03:19Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2072.codfw.wmnet with OS trixie 2432174 wikitext text/x-wiki 9vt4gmrsbi7da3cn2c4xh0jzvxkt2fp 2432175 2432174 2026-07-01T02:07:35Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 54s) 2432175 wikitext text/x-wiki 4npqagp0o8lzovxpe9pkajbfvm10ht6 2432176 2432175 2026-07-01T02:09:42Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2085.codfw.wmnet with OS trixie 2432176 wikitext text/x-wiki puvl8pnim371tun39od1q2qgk4r144z 2432177 2432176 2026-07-01T02:22:14Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2072.codfw.wmnet with reason: host reimage 2432177 wikitext text/x-wiki 6q0aktu6iadnem7shqiy8vfepjtqrux 2432178 2432177 2026-07-01T02:26:56Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2085.codfw.wmnet with reason: host reimage 2432178 wikitext text/x-wiki 9rd1dek24sfkfh0qw2gihi3kse0ugtp 2432179 2432178 2026-07-01T02:30:23Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2072.codfw.wmnet with reason: host reimage 2432179 wikitext text/x-wiki dmsm1rf063h9ya6h81d1vqpzg5gx153 2432180 2432179 2026-07-01T02:31:45Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2107.codfw.wmnet with OS trixie 2432180 wikitext text/x-wiki 6bwzln4avgfn95h9admv283u56fwd8p 2432181 2432180 2026-07-01T02:35:26Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2085.codfw.wmnet with reason: host reimage 2432181 wikitext text/x-wiki hczf1t16uclh4axitbat0t19jqhku0j 2432183 2432181 2026-07-01T02:51:28Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2107.codfw.wmnet with reason: host reimage 2432183 wikitext text/x-wiki ksibixmdfo8z5rs7jiag843iys9jbxv 2432184 2432183 2026-07-01T02:51:42Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2072.codfw.wmnet with OS trixie 2432184 wikitext text/x-wiki poctrd1oeae7jimmmo0xd3n9biska6i 2432185 2432184 2026-07-01T02:55:21Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2085.codfw.wmnet with OS trixie 2432185 wikitext text/x-wiki s11yly9y2mueflvm1hz3jjkcdbm7wue 2432186 2432185 2026-07-01T02:59:26Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2107.codfw.wmnet with reason: host reimage 2432186 wikitext text/x-wiki 4yyu5fgmiuqu4pz6wqmgkq6e4lfnu7f 2432187 2432186 2026-07-01T03:21:24Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2107.codfw.wmnet with OS trixie 2432187 wikitext text/x-wiki gb2qknhti2w7cjfc67hz04dsl2zxsea 2432188 2432187 2026-07-01T03:47:37Z Stashbot 7414 slyngshede@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on es1039.eqiad.wmnet with reason: Hardware crash 2432188 wikitext text/x-wiki 33pxdo4mcu00yw5a60fzkjejd0tx01s 2432189 2432188 2026-07-01T04:27:30Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2067.codfw.wmnet with OS trixie 2432189 wikitext text/x-wiki 2m2rihohzzueq52t75pb1wq3ns9ut9v 2432190 2432189 2026-07-01T04:45:40Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2067.codfw.wmnet with reason: host reimage 2432190 wikitext text/x-wiki 3lx5kwbc6eu4ih737clhklg2nqnww1t 2432191 2432190 2026-07-01T04:49:29Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2067.codfw.wmnet with reason: host reimage 2432191 wikitext text/x-wiki ggkahgbatw1xj1z7m6gjb37gguiyj70 2432192 2432191 2026-07-01T04:56:48Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2068.codfw.wmnet with OS trixie 2432192 wikitext text/x-wiki 6bq2w0fd69vts5whthhla7ttmfvyqmn 2432193 2432192 2026-07-01T05:09:24Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2067.codfw.wmnet with OS trixie 2432193 wikitext text/x-wiki 4j9ofd3zp2jla5qlf5kon41qiaxhaum 2432194 2432193 2026-07-01T05:15:00Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2068.codfw.wmnet with reason: host reimage 2432194 wikitext text/x-wiki eiv2l9yqrtsjvyulzltb8pm8m5b9l4w 2432195 2432194 2026-07-01T05:16:02Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.reimage for host cirrussearch2109.codfw.wmnet with OS trixie 2432195 wikitext text/x-wiki 1bc58ec94v8iwx9sk9setd480h3llji 2432196 2432195 2026-07-01T05:20:02Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2068.codfw.wmnet with reason: host reimage 2432196 wikitext text/x-wiki 3jme4xhszuspvqf1ztcdg51mrd068c8 2432198 2432196 2026-07-01T05:36:07Z Stashbot 7414 ryankemper@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2109.codfw.wmnet with reason: host reimage 2432198 wikitext text/x-wiki nmh2nwoe7uoeeodn35ib3aki4u10h5s 2432199 2432198 2026-07-01T05:40:17Z Stashbot 7414 marostegui@cumin1003: conftool action : set/pooled=yes; selector: name=clouddb1027.eqiad.wmnet,service=s7 2432199 wikitext text/x-wiki 47n6wl8a6uxgplvfqne0tcki0wfmb2i 2432200 2432199 2026-07-01T05:40:20Z Stashbot 7414 marostegui@cumin1003: conftool action : set/pooled=yes; selector: name=clouddb1027.eqiad.wmnet,service=s2 2432200 wikitext text/x-wiki driheehjzu9qoio1rny89uzva57d1j2 2432201 2432200 2026-07-01T05:40:47Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2109.codfw.wmnet with reason: host reimage 2432201 wikitext text/x-wiki ndui1nrk1gg6plvgvq1snzhok5l8xs7 2432202 2432201 2026-07-01T05:40:50Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2068.codfw.wmnet with OS trixie 2432202 wikitext text/x-wiki qkeyof0j0k3ved16dj7x6527gpqxqp5 2432203 2432202 2026-07-01T05:41:07Z Stashbot 7414 marostegui@cumin1003: conftool action : set/weight=100; selector: name=clouddb1027.eqiad.wmnet 2432203 wikitext text/x-wiki eux9lwynkk786n97trkqtb9b0va888h 2432204 2432203 2026-07-01T05:45:52Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on es1039.eqiad.wmnet with reason: issues 2432204 wikitext text/x-wiki cw4sr6ky9cwufhdt1ucid7zqy64c66r 2432205 2432204 2026-07-01T06:01:19Z Stashbot 7414 ryankemper@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2109.codfw.wmnet with OS trixie 2432205 wikitext text/x-wiki or8gsdeh7epakussp68ycrtjdqtlmz3 2432207 2432205 2026-07-01T06:30:31Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "Various improvements - oblivian@cumin1003" 2432207 wikitext text/x-wiki lnzw737bo70lihbidw3qyoh8l0edsit 2432208 2432207 2026-07-01T06:30:32Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: Various improvements - oblivian@cumin1003 2432208 wikitext text/x-wiki 9lamaqu67x3w6ocbqv173ytvx69n5ck 2432209 2432208 2026-07-01T06:30:53Z Stashbot 7414 oblivian@cumin1003: END (FAIL) - Cookbook sre.deploy.python-code (exit_code=99) hiddenparma to alert[1002,2002].wikimedia.org with reason: Various improvements - oblivian@cumin1003 2432209 wikitext text/x-wiki t6cz2soj1kle1ygpp5bm247u1p0h98n 2432210 2432209 2026-07-01T06:30:54Z Stashbot 7414 oblivian@cumin1003: END (FAIL) - Cookbook sre.deploy.hiddenparma (exit_code=99) Hiddenparma deployment to the alerting hosts with reason: "Various improvements - oblivian@cumin1003" 2432210 wikitext text/x-wiki go68ia2ys09c9ddnzlga3m3bpgqfca5 2432212 2432210 2026-07-01T06:31:51Z Stashbot 7414 jmm@cumin2003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Niharika29 out of all services on: 2453 hosts 2432212 wikitext text/x-wiki 5e28op1nez84vy39z7pqoiknyee51pd 2432213 2432212 2026-07-01T06:34:27Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "Various improvements - oblivian@cumin1003" 2432213 wikitext text/x-wiki qmlpliz6toayp3n3xg869xg06unakay 2432214 2432213 2026-07-01T06:34:29Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: Various improvements - oblivian@cumin1003 2432214 wikitext text/x-wiki 8088hhb6l69r7aluvwp2sih7em9jtas 2432215 2432214 2026-07-01T06:35:18Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: Various improvements - oblivian@cumin1003 2432215 wikitext text/x-wiki 8cqoitt5dqdh3wvvcj1da75frrqnfiz 2432216 2432215 2026-07-01T06:35:19Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.hiddenparma (exit_code=0) Hiddenparma deployment to the alerting hosts with reason: "Various improvements - oblivian@cumin1003" 2432216 wikitext text/x-wiki 4gaqug6tukizll6l7o70qm8pm7wdc5c 2432217 2432216 2026-07-01T06:41:55Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "Fix template - oblivian@cumin1003" 2432217 wikitext text/x-wiki 7cd7v10ue9vp7eac50cal8vvff5khw9 2432218 2432217 2026-07-01T06:41:56Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: Fix template - oblivian@cumin1003 2432218 wikitext text/x-wiki p0w692zwht4grmozd0dto7xi5asl6mj 2432219 2432218 2026-07-01T06:42:41Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: Fix template - oblivian@cumin1003 2432219 wikitext text/x-wiki nwxtynp695v6xi0f7k85gbqkw3baly2 2432220 2432219 2026-07-01T06:42:43Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.hiddenparma (exit_code=0) Hiddenparma deployment to the alerting hosts with reason: "Fix template - oblivian@cumin1003" 2432220 wikitext text/x-wiki prg65ho8ryyayweqjpdk17vvbfakdud 2432221 2432220 2026-07-01T06:43:02Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "Fix template - oblivian@cumin1003" 2432221 wikitext text/x-wiki 7fvi3rquzs78erac8hrt0pucerl45wx 2432222 2432221 2026-07-01T06:43:04Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: Fix template - oblivian@cumin1003 2432222 wikitext text/x-wiki hs9cajrietdgla81xum96kswpgzyiq7 2432223 2432222 2026-07-01T06:43:53Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: Fix template - oblivian@cumin1003 2432223 wikitext text/x-wiki 6mx0q4iml6yyp1ho5vz26cbnijcrekp 2432224 2432223 2026-07-01T06:43:54Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.hiddenparma (exit_code=0) Hiddenparma deployment to the alerting hosts with reason: "Fix template - oblivian@cumin1003" 2432224 wikitext text/x-wiki 09a4ofb82jab5z4f9z8n936evv2arw0 2432225 2432224 2026-07-01T06:55:47Z Stashbot 7414 elukey: upgrade all trixie hosts to pywmflib 3.0 - T430552 2432225 wikitext text/x-wiki 99nxmmf5j87ytfc3gph8lbv4k385pv9 2432226 2432225 2026-07-01T07:04:31Z Stashbot 7414 mszwarc@deploy1003: Started scap sync-world: Backport for [[gerrit:1306304|frwiki: change to Wikipedia 25 logo (T430409)]], [[gerrit:1306221|Temporarily change plwiki tagline for 1.7M articles (T430512)]], [[gerrit:1306649|CommonSettings: add Ombuds to wgWMCGlobalGroupToRateLimitClass (T430641)]] 2432226 wikitext text/x-wiki l11ynh9sg5m0ik2do1wgpli5z5znktq 2432227 2432226 2026-07-01T07:06:49Z Stashbot 7414 mszwarc@deploy1003: mszwarc, chlod, revi: Backport for [[gerrit:1306304|frwiki: change to Wikipedia 25 logo (T430409)]], [[gerrit:1306221|Temporarily change plwiki tagline for 1.7M articles (T430512)]], [[gerrit:1306649|CommonSettings: add Ombuds to wgWMCGlobalGroupToRateLimitClass (T430641)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2432227 wikitext text/x-wiki 6w96e4s33jthwinqhwd5i0y18we9wly 2432228 2432227 2026-07-01T07:09:18Z Stashbot 7414 mszwarc@deploy1003: mszwarc, chlod, revi: Continuing with deployment 2432228 wikitext text/x-wiki 7ii33j0u0pzexpxhfhoz7d560li35z7 2432229 2432228 2026-07-01T07:13:44Z Stashbot 7414 mszwarc@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306304|frwiki: change to Wikipedia 25 logo (T430409)]], [[gerrit:1306221|Temporarily change plwiki tagline for 1.7M articles (T430512)]], [[gerrit:1306649|CommonSettings: add Ombuds to wgWMCGlobalGroupToRateLimitClass (T430641)]] (duration: 09m 13s) 2432229 wikitext text/x-wiki f8ww1rb19pkgz72i0qrydrzi5ntaa3z 2432230 2432229 2026-07-01T07:19:05Z Stashbot 7414 aqu@deploy1003: Started deploy [analytics/refinery@410f205] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@410f2050] 2432230 wikitext text/x-wiki 72vx4im8qeg69xwf54wvd5jouugzcmh 2432231 2432230 2026-07-01T07:20:58Z Stashbot 7414 mszwarc@deploy1003: Started scap sync-world: Backport for [[gerrit:1306710|Fix async loading in footnote click interaction experiment (T415904)]], [[gerrit:1306711|Fix async loading in footnote click interaction experiment (T415904)]] 2432231 wikitext text/x-wiki a5tyivaf3zex627jogvvrda5myjbgh9 2432232 2432231 2026-07-01T07:21:07Z Stashbot 7414 aqu@deploy1003: Finished deploy [analytics/refinery@410f205] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@410f2050] (duration: 02m 01s) 2432232 wikitext text/x-wiki i4au3hbn8zicuxuhx3e847efajblyfs 2432233 2432232 2026-07-01T07:21:40Z Stashbot 7414 aqu@deploy1003: Started deploy [analytics/refinery@410f205]: Regular analytics weekly train [analytics/refinery@410f2050] 2432233 wikitext text/x-wiki a7215zchu3pzmm905g033t28rped7jo 2432234 2432233 2026-07-01T07:23:03Z Stashbot 7414 mszwarc@deploy1003: wmde-fisch, mszwarc: Backport for [[gerrit:1306710|Fix async loading in footnote click interaction experiment (T415904)]], [[gerrit:1306711|Fix async loading in footnote click interaction experiment (T415904)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2432234 wikitext text/x-wiki gs7ec556o28wmi65askt0wo8gyt7x5k 2432235 2432234 2026-07-01T07:24:03Z Stashbot 7414 mszwarc@deploy1003: wmde-fisch, mszwarc: Continuing with deployment 2432235 wikitext text/x-wiki lzhn9hb9n40uissvh2q67ykze5g5pm8 2432236 2432235 2026-07-01T07:26:13Z Stashbot 7414 aqu@deploy1003: Finished deploy [analytics/refinery@410f205]: Regular analytics weekly train [analytics/refinery@410f2050] (duration: 04m 32s) 2432236 wikitext text/x-wiki 81czgeh52reywii9ac05l4hyo916ytu 2432237 2432236 2026-07-01T07:26:55Z Stashbot 7414 aqu@deploy1003: Started deploy [analytics/refinery@410f205] (thin): Regular analytics weekly train THIN [analytics/refinery@410f2050] 2432237 wikitext text/x-wiki suekn2io6qlaj7d4m8zphz2llm60rew 2432238 2432237 2026-07-01T07:28:17Z Stashbot 7414 mszwarc@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306710|Fix async loading in footnote click interaction experiment (T415904)]], [[gerrit:1306711|Fix async loading in footnote click interaction experiment (T415904)]] (duration: 07m 19s) 2432238 wikitext text/x-wiki s7nnqqqi8onrz4vqptwfinn1mf6m847 2432239 2432238 2026-07-01T07:28:55Z Stashbot 7414 aqu@deploy1003: Finished deploy [analytics/refinery@410f205] (thin): Regular analytics weekly train THIN [analytics/refinery@410f2050] (duration: 01m 59s) 2432239 wikitext text/x-wiki ly8mb57tmylw27m74n7hgj6knop289i 2432240 2432239 2026-07-01T07:29:50Z Stashbot 7414 aqu@deploy1003: Started deploy [analytics/refinery@410f205]: Regular analytics weekly train 2nd try [analytics/refinery@410f2050] 2432240 wikitext text/x-wiki t4f7z0hh192rdu2n55gs2b83beazhz7 2432241 2432240 2026-07-01T07:30:13Z Stashbot 7414 aqu@deploy1003: Finished deploy [analytics/refinery@410f205]: Regular analytics weekly train 2nd try [analytics/refinery@410f2050] (duration: 00m 22s) 2432241 wikitext text/x-wiki hffxpe76zzzaa2pu2ay6s51fnmw0v3s 2432242 2432241 2026-07-01T07:45:36Z Stashbot 7414 mszwarc@deploy1003: Started scap sync-world: Backport for [[gerrit:1306850|ULS rewrite: change description key in EmptySearchEntrypoint (T429882)]] 2432242 wikitext text/x-wiki maptzxrx33pi1s7s4e57ybmsx6fv93q 2432243 2432242 2026-07-01T07:51:52Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . 2432243 wikitext text/x-wiki 6wrpxh41vrob46uyhotl0a5ro3zlg34 2432244 2432243 2026-07-01T07:55:24Z Stashbot 7414 filippo@cumin1003: conftool action : set/pooled=no; selector: service=dumps-nfs,name=clouddumps1001.wikimedia.org 2432244 wikitext text/x-wiki jm3l8sukzpmoevink0f9netrmmtbfu9 2432245 2432244 2026-07-01T08:03:38Z Stashbot 7414 mszwarc@deploy1003: mszwarc, abi: Backport for [[gerrit:1306850|ULS rewrite: change description key in EmptySearchEntrypoint (T429882)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2432245 wikitext text/x-wiki lgmjiuvraec1871btg06oxdqhgyehnc 2432246 2432245 2026-07-01T08:09:12Z Stashbot 7414 mszwarc@deploy1003: mszwarc, abi: Continuing with deployment 2432246 wikitext text/x-wiki gt4w95haetoux0hgyba4emm2ko6qtsq 2432247 2432246 2026-07-01T08:15:50Z Stashbot 7414 filippo@cumin1003: conftool action : set/pooled=yes; selector: service=dumps-nfs,name=clouddumps1001.wikimedia.org 2432247 wikitext text/x-wiki tducjvfcph05qyr9vxks3zfwqk5x1gf 2432248 2432247 2026-07-01T08:21:47Z Stashbot 7414 mszwarc@deploy1003: Finished scap sync-world: Backport for [[gerrit:1306850|ULS rewrite: change description key in EmptySearchEntrypoint (T429882)]] (duration: 36m 11s) 2432248 wikitext text/x-wiki aj17mbm7823kndv1shx1be0g6yoqwou 2432249 2432248 2026-07-01T08:21:50Z Stashbot 7414 filippo@cumin1003: conftool action : set/pooled=no; selector: service=dumps-nfs,name=clouddumps1002.wikimedia.org 2432249 wikitext text/x-wiki 499j7192gwa7j3yubjqp70rkraq0xyc 2432252 2432249 2026-07-01T08:36:34Z Stashbot 7414 aklapper@deploy1003: rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.9 refs T423918 2432252 wikitext text/x-wiki b7wsf78ogmp5ky4kki4pylertfke3g9 2432253 2432252 2026-07-01T08:38:52Z Stashbot 7414 filippo@cumin1003: conftool action : set/pooled=yes; selector: service=dumps-nfs,name=clouddumps1002.wikimedia.org 2432253 wikitext text/x-wiki qm96o36pywvwsk0ild70edt3q6ndnjq 2432254 2432253 2026-07-01T08:38:58Z Stashbot 7414 filippo@cumin1003: conftool action : set/pooled=no; selector: service=dumps-nfs,name=clouddumps1001.wikimedia.org 2432254 wikitext text/x-wiki rmxddk5xsrw81ad6x3kq1r5i4vxp5su 2432257 2432254 2026-07-01T08:54:09Z Stashbot 7414 aklapper@deploy1003: rebuilt and synchronized wikiversions files: group0 to 1.47.0-wmf.9 refs T423918 2432257 wikitext text/x-wiki l7u6u8z0vhef7p7eii6v36xqk5nt7ca 2432259 2432257 2026-07-01T09:02:04Z Stashbot 7414 filippo@cumin1003: conftool action : set/pooled=yes; selector: service=dumps-nfs,name=clouddumps1001.wikimedia.org 2432259 wikitext text/x-wiki b61if9rgv4ti5ft5jms3aqs4g7ooun9 2432260 2432259 2026-07-01T09:02:12Z Stashbot 7414 filippo@cumin1003: conftool action : set/pooled=no; selector: service=dumps-nfs,name=clouddumps1002.wikimedia.org 2432260 wikitext text/x-wiki e6v0bfzi7chg9bkixwss56yyj9t6tg8 2432262 2432260 2026-07-01T09:14:37Z Stashbot 7414 filippo@cumin1003: conftool action : set/pooled=yes; selector: service=dumps-nfs,name=clouddumps1002.wikimedia.org 2432262 wikitext text/x-wiki 3q7yxh3ct94x0hzp84utim66jphbun8 2432263 2432262 2026-07-01T09:14:46Z Stashbot 7414 filippo@cumin1003: conftool action : set/pooled=no; selector: service=dumps-nfs,name=clouddumps1001.wikimedia.org 2432263 wikitext text/x-wiki e1ju5iy621urdlr0sqy74fzbrvllp9r 2432264 2432263 2026-07-01T09:21:03Z Stashbot 7414 filippo@cumin1003: conftool action : set/pooled=yes; selector: service=dumps-nfs,name=clouddumps1001.wikimedia.org 2432264 wikitext text/x-wiki kryrgqz83mip8hrdn566b2wc8b79rfu 2432265 2432264 2026-07-01T09:21:10Z Stashbot 7414 filippo@cumin1003: conftool action : set/pooled=no; selector: service=dumps-nfs,name=clouddumps1002.wikimedia.org 2432265 wikitext text/x-wiki fu33hxmgvn43cykn3nsigoytgo4ckuc 2432272 2432265 2026-07-01T09:39:10Z Stashbot 7414 mszwarc@deploy1003: Synchronized private/SuggestedInvestigationsSignals/SuggestedInvestigationsSignal4n.php: Update SI signal 4n (duration: 06m 08s) 2432272 wikitext text/x-wiki 2uemcg8ept3jhrkohtp4gghzrdb7t59 2432273 2432272 2026-07-01T09:51:24Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . 2432273 wikitext text/x-wiki ni2kpfp98l7vexd5v4204kekb5ammhj 2432274 2432273 2026-07-01T09:55:50Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "Fix template (take 2) - oblivian@cumin1003" 2432274 wikitext text/x-wiki nn7opnfw0czklu6vngr0i143yiw0vro 2432275 2432274 2026-07-01T09:55:51Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: Fix template (take 2) - oblivian@cumin1003 2432275 wikitext text/x-wiki pamzccbse2xg8lvh5uygvoyfd45xt95 2432276 2432275 2026-07-01T09:56:41Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: Fix template (take 2) - oblivian@cumin1003 2432276 wikitext text/x-wiki ht6dzzyg5n292tus5wplmjz798glov8 2432277 2432276 2026-07-01T09:56:43Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.hiddenparma (exit_code=0) Hiddenparma deployment to the alerting hosts with reason: "Fix template (take 2) - oblivian@cumin1003" 2432277 wikitext text/x-wiki 376038hgfcyeozqq2snki4g9ndoniyo 2432281 2432277 2026-07-01T10:15:13Z Stashbot 7414 cwilliams@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 40 hosts with reason: Primary switchover s4 T430127 2432281 wikitext text/x-wiki nmw1gae739mgwonnalixxbhupv8svhp 2432282 2432281 2026-07-01T10:15:34Z Stashbot 7414 cwilliams@cumin1003: dbctl commit (dc=all): 'Set db2179 with weight 0 T430127', diff saved to https://phabricator.wikimedia.org/P94651 and previous config saved to /var/cache/conftool/dbconfig/20260701-101531-cwilliams.json 2432282 wikitext text/x-wiki 3tikdjdb1hug75ezby42ggvqhkvd8u1 2432285 2432282 2026-07-01T10:20:48Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2115.codfw.wmnet with OS trixie 2432285 wikitext text/x-wiki 7o8e8s7r78ozyfp3r5nvn9r6twbwqo7 2432286 2432285 2026-07-01T10:23:10Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2106.codfw.wmnet with OS trixie 2432286 wikitext text/x-wiki ez10r5okt01ytk5jtsevd2zs676z19x 2432287 2432286 2026-07-01T10:23:15Z Stashbot 7414 cezmunsta: Starting s4 codfw failover from db2240 to db2179 - T430127 2432287 wikitext text/x-wiki g8bamxdc0d7pk4f63ydqgs76r34j5yl 2432288 2432287 2026-07-01T10:23:59Z Stashbot 7414 cwilliams@cumin1003: dbctl commit (dc=all): 'Promote db2179 to s4 primary T430127', diff saved to https://phabricator.wikimedia.org/P94652 and previous config saved to /var/cache/conftool/dbconfig/20260701-102356-cwilliams.json 2432288 wikitext text/x-wiki 2qtt9olsp15y6q6m2f3p909tky5716a 2432289 2432288 2026-07-01T10:26:02Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2086.codfw.wmnet with OS trixie 2432289 wikitext text/x-wiki 5tm58xdagt6b9etzyz3he2cf3wcyg1m 2432290 2432289 2026-07-01T10:26:36Z Stashbot 7414 moritzm: installing nginx security updates 2432290 wikitext text/x-wiki b8mn729xop73t1lflm2v04o53nsntk2 2432291 2432290 2026-07-01T10:27:01Z Stashbot 7414 cwilliams@cumin1003: dbctl commit (dc=all): 'Depool db2240 T430127', diff saved to https://phabricator.wikimedia.org/P94653 and previous config saved to /var/cache/conftool/dbconfig/20260701-102658-cwilliams.json 2432291 wikitext text/x-wiki kt4x8ep1vzc53dc2sgzwd36owb1rwrm 2432292 2432291 2026-07-01T10:39:49Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2115.codfw.wmnet with reason: host reimage 2432292 wikitext text/x-wiki fr2vti7mnbdg75jstksh89n6ovm3fnf 2432293 2432292 2026-07-01T10:40:59Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2432293 wikitext text/x-wiki oir7h68rl09p31ngtx4aef5i8rt3nec 2432294 2432293 2026-07-01T10:41:01Z Stashbot 7414 cwilliams@cumin1003: dbmaint on s4@codfw T429893 2432294 wikitext text/x-wiki qkllun67ozipd8ssrscuw799e0sxlq8 2432295 2432294 2026-07-01T10:41:09Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db2240: Upgrading db2240.codfw.wmnet 2432295 wikitext text/x-wiki ordwojcao9jouxtsql7mxgee8us5mmw 2432296 2432295 2026-07-01T10:41:20Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2240: Upgrading db2240.codfw.wmnet 2432296 wikitext text/x-wiki dpwgm8ud1kv1zkjlivawz7prcxgew8j 2432297 2432296 2026-07-01T10:42:44Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2106.codfw.wmnet with reason: host reimage 2432297 wikitext text/x-wiki jxah4e0g946cw1exqa7z4r4uvasjesi 2432298 2432297 2026-07-01T10:44:13Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2086.codfw.wmnet with reason: host reimage 2432298 wikitext text/x-wiki a6os2r0zb8ejn7io12x90ers9atxrgt 2432299 2432298 2026-07-01T10:44:31Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db2240.codfw.wmnet with OS trixie 2432299 wikitext text/x-wiki 9dapul8bceytbkmdvep5lzsy1kt4r8q 2432300 2432299 2026-07-01T10:44:58Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2115.codfw.wmnet with reason: host reimage 2432300 wikitext text/x-wiki j1sggekpf2ohaeavlhbe2bpkcqq0d5v 2432301 2432300 2026-07-01T10:49:10Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2086.codfw.wmnet with reason: host reimage 2432301 wikitext text/x-wiki d00026udporzlezz3zff1a93oko7a8g 2432302 2432301 2026-07-01T10:53:02Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2106.codfw.wmnet with reason: host reimage 2432302 wikitext text/x-wiki ow5rcpytvfuveb1olqcqv0r4z6hs4xl 2432303 2432302 2026-07-01T11:00:24Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2240.codfw.wmnet with reason: host reimage 2432303 wikitext text/x-wiki kmnodbrn6zdq3sbjs3r2h2598vrr9ph 2432304 2432303 2026-07-01T11:04:31Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2240.codfw.wmnet with reason: host reimage 2432304 wikitext text/x-wiki lmxotjk01b0flxkpgez2vazv90qwhbw 2432306 2432304 2026-07-01T11:09:33Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2115.codfw.wmnet with OS trixie 2432306 wikitext text/x-wiki 8irwzi16puihidyjpzwyi7wjnio5ugv 2432307 2432306 2026-07-01T11:12:03Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.netbox 2432307 wikitext text/x-wiki q5ilnqyttx0ghi6pj5n0bwixu87sfa6 2432308 2432307 2026-07-01T11:13:57Z Stashbot 7414 mvolz@deploy1003: helmfile [staging] START helmfile.d/services/citoid: apply 2432308 wikitext text/x-wiki 6e8kfg7ij8ggogwx48gtxq509nv8m9t 2432309 2432308 2026-07-01T11:14:20Z Stashbot 7414 mvolz@deploy1003: helmfile [staging] DONE helmfile.d/services/citoid: apply 2432309 wikitext text/x-wiki 5xqn66u9j0zymfpw0w64uaygj80ez4t 2432310 2432309 2026-07-01T11:14:28Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2106.codfw.wmnet with OS trixie 2432310 wikitext text/x-wiki 45d3tepbrkqy6jyx1hr7ix0jf8jldm8 2432311 2432310 2026-07-01T11:15:31Z Stashbot 7414 atsuko@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2086.codfw.wmnet with OS trixie 2432311 wikitext text/x-wiki 8lr03mgym26nxhyjf89c5fbvdtisaav 2432312 2432311 2026-07-01T11:16:22Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] START helmfile.d/services/citoid: apply 2432312 wikitext text/x-wiki q33s72j3ux5b6iibjb9lpu39zdmx333 2432313 2432312 2026-07-01T11:16:54Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] DONE helmfile.d/services/citoid: apply 2432313 wikitext text/x-wiki odoeoqgn70ao7ibm1gg25lwwsjdkrqc 2432314 2432313 2026-07-01T11:17:33Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add new link IP dns for trasnport circuits to magru - cmooney@cumin1003" 2432314 wikitext text/x-wiki bd6ykvfu98wam4i3a151m8viyhoqhvm 2432315 2432314 2026-07-01T11:20:54Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add new link IP dns for trasnport circuits to magru - cmooney@cumin1003" 2432315 wikitext text/x-wiki bs8mshgugbogrqlua7ape6ym4vm4vsd 2432316 2432315 2026-07-01T11:20:55Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2432316 wikitext text/x-wiki sm8q2ms7eb9ottn1h5v64clrp9g0405 2432317 2432316 2026-07-01T11:21:34Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2240.codfw.wmnet with OS trixie 2432317 wikitext text/x-wiki brfxacjm17ee5i7jtzthrcqbi0s523f 2432318 2432317 2026-07-01T11:23:52Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.netbox 2432318 wikitext text/x-wiki b9zl3mvvm5ob44byphgbwchhoolvndn 2432319 2432318 2026-07-01T11:27:08Z Stashbot 7414 mvolz@deploy1003: helmfile [eqiad] START helmfile.d/services/citoid: apply 2432319 wikitext text/x-wiki bch6ap137uslz5xnskbjri4w8i2tyf9 2432320 2432319 2026-07-01T11:27:24Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add new link IP dns for trasnport circuits to magru - cmooney@cumin1003" 2432320 wikitext text/x-wiki djrrowih4rogu0nihzacbyf8amgorxj 2432321 2432320 2026-07-01T11:27:29Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add new link IP dns for trasnport circuits to magru - cmooney@cumin1003" 2432321 wikitext text/x-wiki jvuvsz4q0eu7yu5jok01egcl9vg39ic 2432322 2432321 2026-07-01T11:27:30Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2432322 wikitext text/x-wiki 6el36ryzitv0f748mhypmbgs1ryvt2f 2432323 2432322 2026-07-01T11:27:44Z Stashbot 7414 mvolz@deploy1003: helmfile [eqiad] DONE helmfile.d/services/citoid: apply 2432323 wikitext text/x-wiki i0vl5n3ywrk69813jvenq0ebihro0zt 2432324 2432323 2026-07-01T11:28:56Z Stashbot 7414 mvolz@deploy1003: helmfile [staging] START helmfile.d/services/zotero: apply 2432324 wikitext text/x-wiki 2vl84zj7rayatyz9w6mzqzcdgfl1mzy 2432325 2432324 2026-07-01T11:30:15Z Stashbot 7414 mvolz@deploy1003: helmfile [staging] DONE helmfile.d/services/zotero: apply 2432325 wikitext text/x-wiki 1sotbapsx7rm87wqn573m6rx3koqojc 2432326 2432325 2026-07-01T11:31:41Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db2240: Migration of db2240.codfw.wmnet completed 2432326 wikitext text/x-wiki rd5lpcw7b0vi977g15flamclg7y1o0l 2432327 2432326 2026-07-01T11:36:00Z Stashbot 7414 mvolz@deploy1003: helmfile [eqiad] START helmfile.d/services/zotero: apply 2432327 wikitext text/x-wiki efzroxdnbzu7v0raspmn1cy3wvg44er 2432328 2432327 2026-07-01T11:36:30Z Stashbot 7414 mvolz@deploy1003: helmfile [eqiad] DONE helmfile.d/services/zotero: apply 2432328 wikitext text/x-wiki 6ki1nosf9db1yfkzlnyluh1wbmzjgk1 2432329 2432328 2026-07-01T11:40:10Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] START helmfile.d/services/zotero: apply 2432329 wikitext text/x-wiki qywi9gzhyij8bk2hufaa2now68a09uo 2432330 2432329 2026-07-01T11:40:40Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] DONE helmfile.d/services/zotero: apply 2432330 wikitext text/x-wiki 9i1fdw93ozq07gesmhald6xwc0gifdl 2432331 2432330 2026-07-01T11:48:45Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2100.codfw.wmnet with OS trixie 2432331 wikitext text/x-wiki lgjvj1y02pt7xhgr8ee44k15cxgennr 2432332 2432331 2026-07-01T11:49:01Z Stashbot 7414 cmooney@dns2005: START - running authdns-update 2432332 wikitext text/x-wiki e8xhkwc6jhl9idccmwf0xoi0cyan66k 2432333 2432332 2026-07-01T11:50:44Z Stashbot 7414 cmooney@dns2005: END - running authdns-update 2432333 wikitext text/x-wiki oazj5vkuhflbwnn6tktaahmaa33r5i2 2432334 2432333 2026-07-01T11:52:16Z Stashbot 7414 atsuko@cumin2003: START - Cookbook sre.hosts.reimage for host cirrussearch2083.codfw.wmnet with OS trixie 2432334 wikitext text/x-wiki n2pyhzmm4wmroxd3qzv47enb3t2ufp0 0 17290 2431985 2431297 2026-06-30T15:13:30Z Stashbot 7414 dancy: Rebooting deployment-mwlog02.deployment-prep to clear stuck udp2log processes 2431985 wikitext text/x-wiki 2azto9h1iu3p7nzukuzzldhxd5js2nw 2432182 2431985 2026-07-01T02:45:37Z Stashbot 7414 hashar: gerrit: on gerrit2003 deleted /srv/gerrit/java_pid3520115.hprof (the JVM apparently died at some point, I assume due to heavy crawling) 2432182 wikitext text/x-wiki 1mbkd1xf0qbb6lr248x23mdxoo58l6w 3 17939 2432012 129350 2026-06-30T16:26:09Z StrikerBot 8475 Welcome to Toolforge! 2432012 wikitext text/x-wiki jfab00cmn7lk4ac754vmbxs58eddftp 0 43014 2431960 2417259 2026-06-30T14:55:23Z APizzata-WMF 47941 added warn section for point 3 2431960 wikitext text/x-wiki k3gd6i641movwvwdfnbgpuxyvkglhtp 0 440900 2431871 2429112 2026-06-30T12:27:09Z Filippo Giunchedi 2939 /* new cloudvirt install */ expand new cloudvirt instructions 2431871 wikitext text/x-wiki smdlviq614uhwsaiiaoeen3tvw4w390 498 443628 2432009 2431619 2026-06-30T16:20:47Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/28459029072 (https://github.com/cluebotng/component-configs/commits/428e86ea3f09468d24a0ba98e75225c1c1698e53) 2432009 wikitext text/x-wiki 1a1luop5rwr2yif99v6e1dqsu7yben2 498 443946 2432064 2428696 2026-06-30T19:01:36Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed bab5e77be1 (mwoauth2 0.1.0) 2432064 wikitext text/x-wiki lrryu74q35sqlnrmztva38y2jig8ilg 0 446120 2432076 2417339 2026-06-30T19:45:05Z CMooney (WMF) 24288 /* Upgrade */ 2432076 wikitext text/x-wiki 7ofclrldtujyhe8m8k6jtbyzlrftjrk 2432077 2432076 2026-06-30T19:45:17Z CMooney (WMF) 24288 /* Upgrade */ 2432077 wikitext text/x-wiki 5rapf8oz0tj23sjmtm813r5gvs7z6oj 0 447216 2432271 1923238 2026-07-01T09:34:51Z BJensen-WMF 48942 /* Hands-on: The Deployment Pipeline Tutorial - https://wikitech.wikimedia.org/wiki/Deployment_pipeline/Tutorial */ Link to script 2432271 wikitext text/x-wiki lnrqar53z4lmkynn0uezty45ypnxsuh 0 447225 2432270 1923259 2026-07-01T09:34:19Z BJensen-WMF 48942 /* Hands-on: Configuring helm for production releases */ Link to script 2432270 wikitext text/x-wiki anz46xlj9vrhrd9fug3wyp81zxa3j2z 0 449132 2432266 2301551 2026-07-01T09:31:56Z BJensen-WMF 48942 /* More complex setups */ Add link to script. 2432266 wikitext text/x-wiki 0il9i2k19b4gfqnwze3h3i1hb0qmves 0 449160 2432153 2431794 2026-07-01T00:01:56Z Dexbot 30554 Bot: Updating the report 2432153 wikitext text/x-wiki e1x1qh47l84dsmtnzqqwded1hspgegp 2432166 2432153 2026-07-01T00:56:04Z Dexbot 30554 Bot: Updating the report 2432166 wikitext text/x-wiki lrpyuvaw6qc1g56scv2etzhqkkkchbw 2432284 2432166 2026-07-01T10:18:04Z Dexbot 30554 Bot: Updating the report 2432284 wikitext text/x-wiki esv6m6gvtkqof0p7e6jyjbn0lsbxzl6 0 450435 2432258 2429190 2026-07-01T08:54:45Z BJensen-WMF 48942 /* Deploy the service */ Switch to link, as template doesn't seem to work correctly. 2432258 wikitext text/x-wiki 9vqge754oc3ngwvxtchsrtnlqdsf8v5 0 450867 2432267 1989811 2026-07-01T09:32:37Z BJensen-WMF 48942 /* Hands-on Demo: The Deployment Pipeline Tutorial */ Link to script 2432267 wikitext text/x-wiki 6ygxc1jjxo78hpdoywn4vkfm98tphqo 0 450868 2432269 2094105 2026-07-01T09:33:39Z BJensen-WMF 48942 /* Helm for Production Releases */ Link to script 2432269 wikitext text/x-wiki pxt12ey5wsjlsc63daqprswmyvx60ip 12 451343 2431983 2380041 2026-06-30T15:07:12Z Hashar 32 /* OpenStack Commandline Interfaces */ apt installation most certainly requires sudo. Replace "apt-get" by "apt get" 2431983 wikitext text/x-wiki lhhlw6bucbb3q70xfzlq1xomkcgrnc3 2431991 2431983 2026-06-30T15:21:23Z Hashar 32 /* Application Credentials */ link to https://horizon.wikimedia.org/identity/application_credentials/ 2431991 wikitext text/x-wiki 5d6cx57ug12cs17pjle6i9ef1ivovig 0 451481 2432305 2288561 2026-07-01T11:04:38Z CGoubert-WMF 32775 /* Production */ Link to LVS service removal doc 2432305 wikitext text/x-wiki mph17kaftr0o40adkypsect8hzmknes 0 451533 2432031 2419510 2026-06-30T17:03:59Z SSapaty (WMF) 28955 /* Next meeting agenda */ add meeting notes and agenda cleanup 2432031 wikitext text/x-wiki m83494jck4w57hy8547l4qziozonyny 0 451600 2432268 2424647 2026-07-01T09:33:05Z BJensen-WMF 48942 /* Steps */ Link to script 2432268 wikitext text/x-wiki jz2pmr7w97yhcd1zmzt8kb7vi43s4lc 12 452086 2431982 2426465 2026-06-30T15:06:28Z Hashar 32 link to openstack cli config and how to get 'openstack coe' enabled 2431982 wikitext text/x-wiki 133ia65a8srzk99af9w8oe3aqzd5iyi 0 452882 2432135 2085824 2026-06-30T21:56:37Z WBrown (WMF) 37696 NodeJS iPoid is being removed, redirect to the new and in use one 2432135 wikitext text/x-wiki 4x72swzcv1kmihy68vmqdysk18lncrg 116 453426 2432034 2431246 2026-06-30T17:21:07Z Phabbanbot 37210 Tomasz_Bladyniec was disabled by WMFOffice 2432034 wikitext text/x-wiki f9u8ffxpbwyy1sfbsa2dlgjs80pp62a 116 453906 2431932 2431556 2026-06-30T14:27:13Z Gitlabaccountapprovalbot 37332 shivangisharma was rejected. 2431932 wikitext text/x-wiki 6zb1ymii1yu0u5u2npbp7na4itmvh8l 2432278 2431932 2026-07-01T09:57:11Z Gitlabaccountapprovalbot 37332 mugurolevy was rejected. 2432278 wikitext text/x-wiki 7tftb2ossuqklix8kk3g3ve4yfundia 0 454628 2431998 2428532 2026-06-30T15:34:54Z DCaro (WMF) 18271 /* Upgrades Overview */ Did another round of assignments of upgrade among the workgroup members 2431998 wikitext text/x-wiki 1hhxy7d99zpky80gcv684sucxm3b7tq 0 459290 2431865 2431839 2026-06-30T12:04:06Z APizzata-WMF 47941 2431865 wikitext text/x-wiki n6aaoqyswslyvqnl25m7pzghl3gaeyy 2432138 2431865 2026-06-30T22:25:12Z ABaso (WMF) 1044 /* YYYY-MM-DD NEXT TUESDAY TRAIN (REPLACE THIS AFTER DEPLOY) */ 2432138 wikitext text/x-wiki 8km6m496vqw1e9slxked6b21o67lfhz 2432206 2432138 2026-07-01T06:21:51Z ABaso (WMF) 1044 /* Tuesday, June 30, 2026 TODO */ 2432206 wikitext text/x-wiki 0clzcygzktsjunye1nhatrwdgtaagg6 0 459560 2432051 2427556 2026-06-30T18:07:04Z DAdedoyin (WMF) 39150 Test 2432051 wikitext text/x-wiki 3147ig3c362ri5lednyy2cgat6mdjro 0 459699 2432126 2431737 2026-06-30T21:25:25Z Stashbot 7414 eileen: SmashPig upgraded from 38ffd696 to d8393666 2432126 wikitext text/x-wiki 63r987hdbs05pq8m0m1uva8ynx493dt 0 459777 2432283 2407640 2026-07-01T10:17:10Z GGoncalves-WMF 42848 Link to the category of project pages. 2432283 wikitext text/x-wiki 9mlvct5dyrf6ujcp8gfb1w0vpxz2xf7 6 459942 2432255 2391722 2026-07-01T08:48:19Z DCaro (WMF) 18271 DCaro (WMF) uploaded a new version of [[File:Toolforge system overview.png]] 2391722 wikitext text/x-wiki 6r1vnlncuec02syi3jvuj9898hxw7kp 2432261 2432255 2026-07-01T09:04:39Z DCaro (WMF) 18271 DCaro (WMF) uploaded a new version of [[File:Toolforge system overview.png]] 2391722 wikitext text/x-wiki 6r1vnlncuec02syi3jvuj9898hxw7kp 0 460052 2432010 2424454 2026-06-30T16:24:12Z MPopov (WMF) 5891 Re-organize page, document GB settings 2432010 wikitext text/x-wiki 3zs1tld19khioqpy9rf9rkflg9yssyw 0 460142 2432250 2411061 2026-07-01T08:23:57Z MLechvien-WMF 49197 2432250 wikitext text/x-wiki jm44i9wgtwey5fybru6t3b04qjf1i57 2432251 2432250 2026-07-01T08:24:23Z MLechvien-WMF 49197 Adding Phabricator tags 2432251 wikitext text/x-wiki c5iduy8l25583p2e1p6tbaee4vysew7 0 460194 2431935 2429975 2026-06-30T14:28:45Z MLechvien-WMF 49197 Linking the new Phabricator form 2431935 wikitext text/x-wiki saflxqa5sxglz855q46ezkw4y8da5h0 2431936 2431935 2026-06-30T14:30:21Z MLechvien-WMF 49197 Linking the new Phabricator form 2431936 wikitext text/x-wiki o1phmvhxz9jy7r2xkul0v8xxu5ff0ya 0 460209 2431900 2428814 2026-06-30T13:26:46Z MLechvien-WMF 49197 Linking the new Phabricator form 2431900 wikitext text/x-wiki aq9mupgiydtxwn2byd87i8hn8cg4qji 0 460232 2432025 2431147 2026-06-30T16:49:26Z MPopov (WMF) 5891 /* View and understand experiment results */ Add section: exclude bots from results 2432025 wikitext text/x-wiki aoqvy864notseocky85k5krpouhdsv7 0 460388 2432280 2431751 2026-07-01T10:13:12Z GGoncalves-WMF 42848 Tag with Category:Prep Pantry Projects. 2432280 wikitext text/x-wiki 6tqfss87eg07m0pbhnuot0slagtmm6p 0 460393 2432074 2431611 2026-06-30T19:36:44Z RLazarus (WMF) 15215 ref 2432074 wikitext text/x-wiki 5076a676br7disvjb4kjeqipuh4nt8n 2 460394 2431874 2026-06-30T12:54:04Z Milimetric 612 Created page with "Just an unorganized list for now * Deal with warnings like <code>26/06/30 12:40:50 WARN HiveExternalCatalog: The table schema given by Hive metastore(struct<_schema:string,app_install_id:string,app_session_id:string,...) is different from the schema when this table was created by Spark (...) We have to fall back to the table schema from Hive metastore which is not case preserving.</code> * migrate a few jobs from MW History to MW History incremental * update `hack_logge..." 2431874 wikitext text/x-wiki fom76gdizuuhvkens6b3h497v5zsyu3 2431943 2431874 2026-06-30T14:41:02Z Milimetric 612 2431943 wikitext text/x-wiki 5cq456v6lfelecyz7bqfv3hu9ceonec 2431950 2431943 2026-06-30T14:46:17Z Milimetric 612 2431950 wikitext text/x-wiki aix01bapvzqfjldfjc48xhi48y7baqm 2432035 2431950 2026-06-30T17:25:07Z Milimetric 612 2432035 wikitext text/x-wiki 72pne5xner2gv8i7ziqm1lhm4jdxu04 0 460395 2431929 2026-06-30T14:23:11Z Revi C. 1919 Redirected page to [[CDN/Headers#X-Trusted-Request]] 2431929 wikitext text/x-wiki 3puk8qifqtfkg2fed18p1ysxcuxat3u 3 460396 2432011 2026-06-30T16:24:51Z StrikerBot 8475 Welcome to Toolforge! 2432011 wikitext text/x-wiki tign18u450eiw9xfgz8ucjuwennpwaq 3 460397 2432014 2026-06-30T16:30:01Z StrikerBot 8475 Welcome to Toolforge! 2432014 wikitext text/x-wiki kmgvu2wnlyi6c5v5ldj1u6usu38cg63 0 460398 2432030 2026-06-30T17:02:49Z SSapaty (WMF) 28955 Archive meeting notes from Google Doc 2432030 wikitext text/x-wiki 4qwhhlnyi16jhu0in2u8wbt9o2d1m9h 2432032 2432030 2026-06-30T17:06:13Z SSapaty (WMF) 28955 Format phab links 2432032 wikitext text/x-wiki 1y12gmqxizlp6bb2ek8l89385kc1p9w 0 460399 2432157 2026-07-01T00:21:45Z JrandWP 37706 archive 106 2432157 wikitext text/x-wiki eg1pa5708axrd0hyd99k7ihwvignkrq 0 460400 2432158 2026-07-01T00:25:04Z JrandWP 37706 archive 107 2432158 wikitext text/x-wiki te7wlhgx1033ur6qk906bkqh6227984 14 460401 2432279 2026-07-01T10:10:49Z GGoncalves-WMF 42848 Create a category page for Prep Pantry projects. 2432279 wikitext text/x-wiki 03iveouwy66eozwm2dj6pjxwexwm8d9