Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.18 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 1516 2454110 2453742 2026-09-02T21:03:48Z Stashbot 7414 Krinkle: Release two unused floating IPs, ref T395274 2454110 wikitext text/x-wiki tjckafgsqyeew0ptc0q4mc2xe8asgwy 0 3149 2454299 2378828 2026-09-03T10:49:47Z BTullis (WMF) 25295 2454299 wikitext text/x-wiki 0miyh360xhfla3gnkeiskfmfcgw5qfi 0 4108 2453905 2453874 2026-09-02T12:36:21Z ScheduleDeploymentBot 37566 Add [[gerrit:1333816]] to Wednesday, September 02 UTC afternoon backport window 2453905 wikitext text/x-wiki 1rgpylk5g59mda8rgs52rksm6j4o4zv 2453964 2453905 2026-09-02T14:13:01Z ScheduleDeploymentBot 37566 Add [[gerrit:1333866]] to Thursday, September 03 UTC morning backport window 2453964 wikitext text/x-wiki 8g8f5bq4k2l3ny5xu286j46g9sxigge 2453968 2453964 2026-09-02T14:25:34Z ScheduleDeploymentBot 37566 Add [[gerrit:1333872]] to Thursday, September 03 UTC morning backport window 2453968 wikitext text/x-wiki cfkrmindic9qxnk6wdb6ilhez56zah4 2454169 2453968 2026-09-03T03:30:40Z ScheduleDeploymentBot 37566 Add [[gerrit:1334268]] to Thursday, September 03 UTC late backport window 2454169 wikitext text/x-wiki s0ifbs1z386o58eusie7w9i9hyjdbau 2454181 2454169 2026-09-03T06:38:49Z Thiemo Kreuz (WMDE) 2595 /* {{Deployment_day|date=2026-09-03}} */ -1 2454181 wikitext text/x-wiki s73covsjj59b12q71yjpnyl09t7svqm 2454188 2454181 2026-09-03T07:02:13Z ScheduleDeploymentBot 37566 Add [[gerrit:1333866]] to Thursday, September 03 UTC afternoon backport window 2454188 wikitext text/x-wiki moe0m0y1xi206c8i62y5dt4ih51he8j 2454304 2454188 2026-09-03T11:08:40Z ScheduleDeploymentBot 37566 Add [[gerrit:1329314]] to Monday, September 07 UTC morning backport window 2454304 wikitext text/x-wiki iy2g1j561b647ld4mn5zsyqfkplf0ox 0 6479 2454161 2450022 2026-09-03T00:03:26Z ASarabadani (WMF) 39030 /* Cluster redis_lock */ 2454161 wikitext text/x-wiki bpasm1cjgu8z8ukitqdjo4iutd3oqd9 0 7919 2453881 2453871 2026-09-02T12:17:46Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1328201|Declare the webrequest.dumps.v1 stream in EventStreamConfig (T425087 T291645)]], [[gerrit:1329360|EventStreamConfig: Register the abuse_review_interaction stream (T435517)]] 2453881 wikitext text/x-wiki 4thjsxacu5850563iookz6sqs5rkcci 2453882 2453881 2026-09-02T12:20:37Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: wikikube-worker-codfw@codfw 2453882 wikitext text/x-wiki 04s5wozy7u98imhvprql7e4e0g6lfw3 2453884 2453882 2026-09-02T12:22:18Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz, btullis: Backport for [[gerrit:1328201|Declare the webrequest.dumps.v1 stream in EventStreamConfig (T425087 T291645)]], [[gerrit:1329360|EventStreamConfig: Register the abuse_review_interaction stream (T435517)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2453884 wikitext text/x-wiki fb6zrppjuciuz8k6vmq1v6nnovnk3hb 2453886 2453884 2026-09-02T12:24:08Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs 2453886 wikitext text/x-wiki 5zah0myf733fzjwnv6fvokjeyvmtriq 2453887 2453886 2026-09-02T12:24:59Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz, btullis: Continuing with deployment 2453887 wikitext text/x-wiki eb17orodk1sieauqo8isrmhh74xkevf 2453888 2453887 2026-09-02T12:25:04Z Stashbot 7414 jelto@cumin1003: END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs 2453888 wikitext text/x-wiki twaznzh0q39xysnq7fmyh0s2uuqajpp 2453889 2453888 2026-09-02T12:25:05Z Stashbot 7414 jelto@cumin1003: END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: wikikube-worker-codfw@codfw 2453889 wikitext text/x-wiki 0506z00kp1mfr0r8sooa4r7jf1qiann 2453895 2453889 2026-09-02T12:30:35Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1328201|Declare the webrequest.dumps.v1 stream in EventStreamConfig (T425087 T291645)]], [[gerrit:1329360|EventStreamConfig: Register the abuse_review_interaction stream (T435517)]] (duration: 12m 50s) 2453895 wikitext text/x-wiki c7vd58wuq1gv7u1a5hfbq3sbcgpzpsr 2453896 2453895 2026-09-02T12:34:14Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/push-notifications: apply 2453896 wikitext text/x-wiki 2rjcfn0i503b5gyrppb5n1fjxtvcz9k 2453897 2453896 2026-09-02T12:34:20Z Stashbot 7414 jgiannelos@deploy1003: helmfile [staging] START helmfile.d/services/push-notifications: apply 2453897 wikitext text/x-wiki s9nppw3n0x6ii8gqt2x11fht2rf91ll 2453898 2453897 2026-09-02T12:34:24Z Stashbot 7414 jgiannelos@deploy1003: helmfile [staging] DONE helmfile.d/services/push-notifications: apply 2453898 wikitext text/x-wiki 2ncama1totzxqakajek3zkcaup80kwq 2453899 2453898 2026-09-02T12:34:29Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/push-notifications: apply 2453899 wikitext text/x-wiki swsxlh6oe3ae2ox3rj9bw1vr3eycff1 2453900 2453899 2026-09-02T12:34:32Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/push-notifications: apply 2453900 wikitext text/x-wiki g9czmjadrh7xuza7wnpx0d8s9pi4bqy 2453901 2453900 2026-09-02T12:34:37Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/push-notifications: apply 2453901 wikitext text/x-wiki qbaaa7qrdyxifb4ze4tr4uoxtpx8nt2 2453902 2453901 2026-09-02T12:34:41Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/push-notifications: apply 2453902 wikitext text/x-wiki kxj241kkcpxju1rzh8y2pimsc0a76oj 2453903 2453902 2026-09-02T12:35:15Z Stashbot 7414 jgiannelos@deploy1003: helmfile [staging] START helmfile.d/services/push-notifications: apply 2453903 wikitext text/x-wiki s2nly5rcjvd6rw0wgofrfcct2fficoq 2453904 2453903 2026-09-02T12:35:22Z Stashbot 7414 jgiannelos@deploy1003: helmfile [staging] DONE helmfile.d/services/push-notifications: apply 2453904 wikitext text/x-wiki 9dialivaux0mrymw219wbeyy1mo9uv2 2453906 2453904 2026-09-02T12:41:42Z Stashbot 7414 atsuko@dns1004: START - running authdns-update 2453906 wikitext text/x-wiki s5qqshpx7nlwigv0jri00hgv9foiwiw 2453908 2453906 2026-09-02T12:44:10Z Stashbot 7414 atsuko@dns1004: END - running authdns-update 2453908 wikitext text/x-wiki a1b6jdvtxnjldr587v1w08unf0k8pr1 2453914 2453908 2026-09-02T13:02:05Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-experiment-platform: apply 2453914 wikitext text/x-wiki rvog17zkffgfimpxbddd75vzxgcbo1b 2453915 2453914 2026-09-02T13:02:09Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-experiment-platform: apply 2453915 wikitext text/x-wiki 93r2a9oi1jfn0nbfx4fznen7s7d8qj3 2453916 2453915 2026-09-02T13:04:14Z Stashbot 7414 moritzm: import librsvg 2.60.0+dfsg-1+wmf13u1 to component/thumbor for trixie-wikimedia T436505 2453916 wikitext text/x-wiki lo2gideqzi4p0r4fbjymlogzc4afkji 2453917 2453916 2026-09-02T13:23:26Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-experiment-platform: apply 2453917 wikitext text/x-wiki tcd4ly143l74f62wpyh6uve230wmbr2 2453918 2453917 2026-09-02T13:23:44Z Stashbot 7414 jgiannelos@deploy1003: helmfile [staging] START helmfile.d/services/push-notifications: apply 2453918 wikitext text/x-wiki n1wsu6cxrho4qztjai5dgtvrsz5l82x 2453919 2453918 2026-09-02T13:23:58Z Stashbot 7414 jgiannelos@deploy1003: helmfile [staging] DONE helmfile.d/services/push-notifications: apply 2453919 wikitext text/x-wiki t7e2g65dbxmrqwkfy61u8bh03iql4j8 2453920 2453919 2026-09-02T13:24:03Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/push-notifications: apply 2453920 wikitext text/x-wiki 4n6c6ed5zn4astb4mezirowy8hqzb79 2453921 2453920 2026-09-02T13:24:22Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-experiment-platform: apply 2453921 wikitext text/x-wiki t081r64owagrkd00x7tmu2tzmdnms06 2453922 2453921 2026-09-02T13:24:38Z Stashbot 7414 moritzm: installing wireshark security updates 2453922 wikitext text/x-wiki 0r9m6wlm4ep10hz70z4fsna2tk81yzg 2453923 2453922 2026-09-02T13:24:48Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/push-notifications: apply 2453923 wikitext text/x-wiki 1kg4clf87jzfmg6ieq2ld5b410oprys 2453924 2453923 2026-09-02T13:24:48Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/push-notifications: apply 2453924 wikitext text/x-wiki bfss6ip590fbq667hs186nlbov7fscq 2453925 2453924 2026-09-02T13:25:20Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/push-notifications: apply 2453925 wikitext text/x-wiki lw0j7mi8mce7r9638tq07icyzioybg0 2453926 2453925 2026-09-02T13:26:58Z Stashbot 7414 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1333264|Add thumb.* to allowed hosts for commons images (T436579)]], [[gerrit:1333263|Add thumb.* to allowed hosts for commons images (T436579)]], [[gerrit:1333816|Enable mobile MMV on all wikis (T429970)]] 2453926 wikitext text/x-wiki 8y82lmcq3y5asqbzj6dnjjtoqwd2e50 2453927 2453926 2026-09-02T13:31:25Z Stashbot 7414 samtar@deploy1003: jforrester, samtar, mlitn: Backport for [[gerrit:1333264|Add thumb.* to allowed hosts for commons images (T436579)]], [[gerrit:1333263|Add thumb.* to allowed hosts for commons images (T436579)]], [[gerrit:1333816|Enable mobile MMV on all wikis (T429970)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2453927 wikitext text/x-wiki cpv3vacnefx2yqehkt717mcwz14o8wn 2453928 2453927 2026-09-02T13:31:52Z Stashbot 7414 samtar@deploy1003: jforrester, samtar, mlitn: Continuing with deployment 2453928 wikitext text/x-wiki 0wbzofux2qgxhrwpwq82imx2yywcr34 2453931 2453928 2026-09-02T13:36:52Z Stashbot 7414 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1333264|Add thumb.* to allowed hosts for commons images (T436579)]], [[gerrit:1333263|Add thumb.* to allowed hosts for commons images (T436579)]], [[gerrit:1333816|Enable mobile MMV on all wikis (T429970)]] (duration: 09m 52s) 2453931 wikitext text/x-wiki 3lgu1bzorcy7x3bgzdeiioxns0ft0et 2453935 2453931 2026-09-02T13:44:06Z Stashbot 7414 moritzm: bounce tcpircbot-logmsgbot/tcpircbot-logmsgbot_cloud on alert1002 to allow cumin1004 T427897 2453935 wikitext text/x-wiki lkp1tp6x8r8my6k5ktzj0i61wp3beg5 2453948 2453935 2026-09-02T14:04:23Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2453948 wikitext text/x-wiki mr4elrgho3qoanp3k9xvrwkl6cn5zwt 2453949 2453948 2026-09-02T14:04:34Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2453949 wikitext text/x-wiki 2dm5561aizjk0xljnbynns7nbii3ybx 2453950 2453949 2026-09-02T14:06:06Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] START helmfile.d/services/thumbor: apply 2453950 wikitext text/x-wiki 55dmeyr937xjrla51yl601ro9a1n332 2453951 2453950 2026-09-02T14:06:07Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/pageview-trending-relative-next: apply 2453951 wikitext text/x-wiki b11hygp6as9nax8s639z52ro4h17h3d 2453952 2453951 2026-09-02T14:06:20Z Stashbot 7414 apine@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2453952 wikitext text/x-wiki 86yy7ox6kqx6qv0w60honekj91wrmu6 2453953 2453952 2026-09-02T14:06:36Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/pageview-trending-relative-next: apply 2453953 wikitext text/x-wiki kyczot8ysfwfzciqzawhfhw8o5wl6yb 2453954 2453953 2026-09-02T14:06:41Z Stashbot 7414 apine@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2453954 wikitext text/x-wiki 9j45t0zo9vd6g4ibxskxdvuf6qvh67o 2453955 2453954 2026-09-02T14:07:36Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2453955 wikitext text/x-wiki 3rwcc13onptzw39k12myuyvpo1abxrv 2453956 2453955 2026-09-02T14:08:12Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2453956 wikitext text/x-wiki hctnjr1jsll7jlod24xf7g6rthnz4wy 2453957 2453956 2026-09-02T14:08:17Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] DONE helmfile.d/services/thumbor: apply 2453957 wikitext text/x-wiki d2ix0rx1ca4f3zqxl159q3zxxzs98bi 2453958 2453957 2026-09-02T14:08:43Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2453958 wikitext text/x-wiki 6c6yoz0evcvk7rhmhz9e6i1ux7qriu6 2453959 2453958 2026-09-02T14:08:44Z Stashbot 7414 apine@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2453959 wikitext text/x-wiki e7sly5xbm8mfz3kwvx0ybhwbrtfusat 2453960 2453959 2026-09-02T14:09:04Z Stashbot 7414 apine@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2453960 wikitext text/x-wiki k8gjbekjx1yszqi4fa44e111tfp3ipu 2453961 2453960 2026-09-02T14:09:38Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2453961 wikitext text/x-wiki jtzjhy2ndvrqqiewdclkyd5opkkg07e 2453962 2453961 2026-09-02T14:09:49Z Stashbot 7414 apine@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2453962 wikitext text/x-wiki 9ajl31gi4pykz3j4pyc73jp5s1l9doc 2453963 2453962 2026-09-02T14:12:38Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] START helmfile.d/services/thumbor: apply 2453963 wikitext text/x-wiki 5zxkhcdcbjs1fagkx2pslgo94wpsf01 2453965 2453963 2026-09-02T14:15:20Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] DONE helmfile.d/services/thumbor: apply 2453965 wikitext text/x-wiki 1crhzshv1nfgvxh2yayz9zj8yy0imae 2453966 2453965 2026-09-02T14:20:08Z Stashbot 7414 apine@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2453966 wikitext text/x-wiki tglahqzm7rqzgtfzs4oo6t1er759iuz 2453969 2453966 2026-09-02T14:26:07Z Stashbot 7414 apine@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2453969 wikitext text/x-wiki r6uckw2sn5d23pqd20y6j6kknslkykd 2453970 2453969 2026-09-02T14:26:20Z Stashbot 7414 apine@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2453970 wikitext text/x-wiki hyt3mr7363ynes4qthyvou3nggveq9y 2453972 2453970 2026-09-02T14:27:05Z Stashbot 7414 apine@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2453972 wikitext text/x-wiki kdbbc2j8yuyregdgczag1epewh2tnpk 2453973 2453972 2026-09-02T14:27:26Z Stashbot 7414 apine@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2453973 wikitext text/x-wiki 0n2uu0xekr8vsv0smvwmqwdxp6maf9k 2453975 2453973 2026-09-02T14:32:48Z Stashbot 7414 moritzm: installing pdns-recursor security updates 2453975 wikitext text/x-wiki go3kqnldw7ckilb784jbbt3aosofcr6 2453976 2453975 2026-09-02T14:44:46Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.cdn.roll-restart-ats rolling restart_daemons on A:cp-text_magru 2453976 wikitext text/x-wiki 6ochqie3c3bxfshxchaablwnhy1t0bx 2453978 2453976 2026-09-02T14:57:00Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-ats (exit_code=0) rolling restart_daemons on A:cp-text_magru 2453978 wikitext text/x-wiki 4nelj2px5lw1cy8l9q1oekgiklvgdzi 2453980 2453978 2026-09-02T15:11:01Z Stashbot 7414 moritzm: import jenkins 2.568.3 to thirdparty/jenkins for trixie-wikimedia 2453980 wikitext text/x-wiki qtgt7z0vamn2dh3vyfpr2n7fyv7yoxb 2453982 2453980 2026-09-02T15:15:46Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.cdn.roll-restart-ats rolling restart_daemons on A:cp-upload_magru 2453982 wikitext text/x-wiki 7ra02ten44ysq9yjrkj5y3x4fam9llq 2453987 2453982 2026-09-02T15:17:37Z Stashbot 7414 sukhe: sukhe@lvs1020:~$ sudo systemctl restart pybal.service 2453987 wikitext text/x-wiki asyjh43qklll97jxenf6tu6hnbeqk29 2453988 2453987 2026-09-02T15:21:09Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.00-downtime-db-readonly-checks for datacenter switchover from codfw to eqiad 2453988 wikitext text/x-wiki 7rdni8x1ktjzhsu08ywi52o21diyh4i 2453989 2453988 2026-09-02T15:21:25Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.00-downtime-db-readonly-checks (exit_code=0) for datacenter switchover from codfw to eqiad 2453989 wikitext text/x-wiki 7c95sozgcopo84tohuiqcqztnjwd3qp 2453990 2453989 2026-09-02T15:22:08Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.00-lock-scap for datacenter switchover from codfw to eqiad 2453990 wikitext text/x-wiki 8kgox0znpcd6uezbwhf20br0nfv28fc 2453991 2453990 2026-09-02T15:22:11Z Stashbot 7414 root@deploy1003: Locking from deployment [ALL REPOSITORIES]: Datacenter switchover from codfw to eqiad - T436781 2453991 wikitext text/x-wiki 6askikzdryjqlrzczarcmrm4x29qzc9 2453992 2453991 2026-09-02T15:22:12Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.00-lock-scap (exit_code=0) for datacenter switchover from codfw to eqiad 2453992 wikitext text/x-wiki 3awh6w7esm64m7yvk28a16cqiem8jhn 2453993 2453992 2026-09-02T15:22:46Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.00-optional-warmup-caches for datacenter switchover from codfw to eqiad 2453993 wikitext text/x-wiki t3m19549a4wautwx6eg8b7zcc42euhn 2453995 2453993 2026-09-02T15:27:14Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.deploy.python-code homer to cumin1004.eqiad.wmnet with reason: deploy homer to cumin1004 - cmooney@cumin1003 2453995 wikitext text/x-wiki o0ohi469z3l0btz3wcmojdf42nxv8ip 2453996 2453995 2026-09-02T15:27:50Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-ats (exit_code=0) rolling restart_daemons on A:cp-upload_magru 2453996 wikitext text/x-wiki 52g8unne2gi53v3b5ujnhhpjbkr932e 2453997 2453996 2026-09-02T15:28:14Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin1004.eqiad.wmnet with reason: deploy homer to cumin1004 - cmooney@cumin1003 2453997 wikitext text/x-wiki 71609ilihe3rh0bormi11qjgtesipxl 2453999 2453997 2026-09-02T15:31:46Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.cdn.roll-restart-ats rolling restart_daemons on A:cp-text_eqsin 2453999 wikitext text/x-wiki i9jc64w5cinvkvz48gsk17u8ckxv7yy 2454002 2453999 2026-09-02T15:38:25Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2454002 wikitext text/x-wiki 0emeeu85lozqlwj1c7zsxhi8llzwxgr 2454003 2454002 2026-09-02T15:39:12Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2454003 wikitext text/x-wiki e3z6a92hvtg8g8sazspt45ok0op9x1o 2454004 2454003 2026-09-02T15:42:22Z Stashbot 7414 sukhe: sukhe@lvs1019:~$ sudo systemctl restart pybal.service 2454004 wikitext text/x-wiki tibg3zk07vsd2o096ky79rswwwu36sd 2454005 2454004 2026-09-02T15:44:10Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-ats (exit_code=0) rolling restart_daemons on A:cp-text_eqsin 2454005 wikitext text/x-wiki qkj19g57w3h0ojz0khm1u8rdiqrsxbr 2454006 2454005 2026-09-02T15:45:58Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.cdn.roll-restart-ats rolling restart_daemons on A:cp-upload_eqsin 2454006 wikitext text/x-wiki py8vlwrjyorcn5fqvp9kroie585vyt6 2454007 2454006 2026-09-02T15:46:39Z Stashbot 7414 slyngshede@cumin1003: END (ERROR) - Cookbook sre.switchdc.mediawiki.00-optional-warmup-caches (exit_code=97) for datacenter switchover from codfw to eqiad 2454007 wikitext text/x-wiki 6n6py2rxf4qke530w4cjox1rdsef547 2454008 2454007 2026-09-02T15:47:37Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.00-reduce-ttl for datacenter switchover from codfw to eqiad 2454008 wikitext text/x-wiki eknelxcs2xh9pfdwnjnqnz2j9jzloqd 2454010 2454008 2026-09-02T15:53:21Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.00-reduce-ttl (exit_code=0) for datacenter switchover from codfw to eqiad 2454010 wikitext text/x-wiki jewnvcjkqzzlxz8oxs1ujenrtxruhil 2454011 2454010 2026-09-02T15:53:30Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.01-stop-maintenance for datacenter switchover from codfw to eqiad 2454011 wikitext text/x-wiki ea08fehcvhdyjgo4mrtl8tokigwuv9n 2454012 2454011 2026-09-02T15:53:31Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.01-stop-maintenance (exit_code=0) for datacenter switchover from codfw to eqiad 2454012 wikitext text/x-wiki eqjtfuze1abbg1ykq84p6shwlbbbl4r 2454013 2454012 2026-09-02T15:53:48Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.02-set-readonly for datacenter switchover from codfw to eqiad 2454013 wikitext text/x-wiki o1ndzixracg48ltztyinrhbtw7ipq5j 2454014 2454013 2026-09-02T15:53:49Z Stashbot 7414 slyngshede@cumin1003: [NON-PRIMARY-DC] MediaWiki read-only period starts at: 2026-09-02 15:53:47.690918 2454014 wikitext text/x-wiki jynpvwmuyi437l6p18176d90a1rjpu4 2454015 2454014 2026-09-02T15:54:04Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.02-set-readonly (exit_code=0) for datacenter switchover from codfw to eqiad 2454015 wikitext text/x-wiki bfas2bivtpc2p7lzhfjef9zg7no4ika 2454016 2454015 2026-09-02T15:54:17Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.03-set-db-readonly for datacenter switchover from codfw to eqiad 2454016 wikitext text/x-wiki 7hdwzld8qwbyxngveow71hl03sjdgdv 2454017 2454016 2026-09-02T15:55:06Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.03-set-db-readonly (exit_code=0) for datacenter switchover from codfw to eqiad 2454017 wikitext text/x-wiki pvlmvkurmsxr7tf590xa0cm4h6x40lx 2454018 2454017 2026-09-02T15:55:27Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki for datacenter switchover from codfw to eqiad 2454018 wikitext text/x-wiki nq66m0enl64dgrlvx6l5dwrpecg3hat 2454019 2454018 2026-09-02T15:55:40Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki (exit_code=0) for datacenter switchover from codfw to eqiad 2454019 wikitext text/x-wiki 9o3df6yuc314m4b4ryl13gpwbz1x0rt 2454020 2454019 2026-09-02T15:55:46Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite for datacenter switchover from codfw to eqiad 2454020 wikitext text/x-wiki p4pc4vmo2ynkkiyx2ld2cczlka5pp6k 2454021 2454020 2026-09-02T15:55:52Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (exit_code=0) for datacenter switchover from codfw to eqiad 2454021 wikitext text/x-wiki bt8xylu4wielfqbr565acrku9nfalci 2454022 2454021 2026-09-02T15:56:12Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.07-set-readwrite for datacenter switchover from codfw to eqiad 2454022 wikitext text/x-wiki eikars49307dkkmbxcbu4jcbz5yojp1 2454023 2454022 2026-09-02T15:56:13Z Stashbot 7414 slyngshede@cumin1003: [NON-PRIMARY-DC] MediaWiki read-only period ends at: 2026-09-02 15:56:13.434320 2454023 wikitext text/x-wiki cigxv6fhlnwo2tixw6p7gsa0jq33xsf 2454024 2454023 2026-09-02T15:56:16Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.07-set-readwrite (exit_code=0) for datacenter switchover from codfw to eqiad 2454024 wikitext text/x-wiki kq88h74x434tly0uiujcakont85ifuc 2454025 2454024 2026-09-02T15:56:26Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.08-restart-mw-jobrunner for datacenter switchover from codfw to eqiad 2454025 wikitext text/x-wiki pc8xtrbr6g8wz360bm8v4ck80rc4y7c 2454026 2454025 2026-09-02T15:56:27Z Stashbot 7414 root@deploy1003: helmfile [codfw] START helmfile.d/services/mw-jobrunner: sync 2454026 wikitext text/x-wiki 61gi990yr0smh4hnzdbueboepf2yy3v 2454027 2454026 2026-09-02T15:56:50Z Stashbot 7414 root@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-jobrunner: sync 2454027 wikitext text/x-wiki 25rlowkx7dimune37tjixwjzpwhhyuq 2454028 2454027 2026-09-02T15:56:52Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.08-restart-mw-jobrunner (exit_code=0) for datacenter switchover from codfw to eqiad 2454028 wikitext text/x-wiki l1iqtp4jbls62i0jr1swa6uekelwomo 2454029 2454028 2026-09-02T15:57:00Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.08-start-maintenance for datacenter switchover from codfw to eqiad 2454029 wikitext text/x-wiki bqygc6lg9pswn6t8oh6pfpiwz12e5yv 2454030 2454029 2026-09-02T15:57:01Z Stashbot 7414 root@deploy1003: helmfile [codfw] START helmfile.d/services/mw-cron: apply 2454030 wikitext text/x-wiki 9bzigg5s7hopxpjrplkdlh8dy9csff4 2454031 2454030 2026-09-02T15:57:04Z Stashbot 7414 root@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-cron: apply 2454031 wikitext text/x-wiki 1l2kf7fnsm9ae4odwxgtja2526mg79r 2454032 2454031 2026-09-02T15:57:06Z Stashbot 7414 root@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-cron: apply 2454032 wikitext text/x-wiki fv0ahon43hinyzotn1l8elugryga9x7 2454033 2454032 2026-09-02T15:57:12Z Stashbot 7414 root@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-cron: apply 2454033 wikitext text/x-wiki 1vj542ndjkj0663gz4o9ml15l7o7bac 2454034 2454033 2026-09-02T15:57:15Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.08-start-maintenance (exit_code=0) for datacenter switchover from codfw to eqiad 2454034 wikitext text/x-wiki 57fd209y7uv3ngeywts2sn3o0q31ca5 2454035 2454034 2026-09-02T15:57:30Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.09-restore-ttl for datacenter switchover from codfw to eqiad 2454035 wikitext text/x-wiki 8egpba5zrqcmpcl1yug9ei9c6dj6w2j 2454036 2454035 2026-09-02T15:58:10Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.09-restore-ttl (exit_code=0) for datacenter switchover from codfw to eqiad 2454036 wikitext text/x-wiki 7ezhfchqnz9odriskh1uic0ooxz2a4y 2454037 2454036 2026-09-02T15:58:21Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-ats (exit_code=0) rolling restart_daemons on A:cp-upload_eqsin 2454037 wikitext text/x-wiki tip2mhk3wcpznw5bs7t7x709hr95spr 2454038 2454037 2026-09-02T15:58:36Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.09-run-puppet-on-db-masters for datacenter switchover from codfw to eqiad 2454038 wikitext text/x-wiki 1ppd0ygsrswdcczfer9zr7tebdcxwxl 2454039 2454038 2026-09-02T15:59:21Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.cdn.roll-restart-ats rolling restart_daemons on A:cp-text_eqsin 2454039 wikitext text/x-wiki cednppgcjhg1ld8m67rr7arnqd08b0n 2454040 2454039 2026-09-02T16:10:07Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.09-run-puppet-on-db-masters (exit_code=0) for datacenter switchover from codfw to eqiad 2454040 wikitext text/x-wiki 1iahv28sxaqv14429znyhae29fphekg 2454041 2454040 2026-09-02T16:10:21Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.switchdc.mediawiki.09-unlock-scap for datacenter switchover from codfw to eqiad 2454041 wikitext text/x-wiki e9wg8wlu8x0gwlntkfggtw73b2jd70w 2454042 2454041 2026-09-02T16:10:24Z Stashbot 7414 root@deploy1003: Forcefully removing global lock: Datacenter switchover from codfw to eqiad - T436781 2454042 wikitext text/x-wiki jyzh8j258cspom7b9xy714gokw34alx 2454043 2454042 2026-09-02T16:10:26Z Stashbot 7414 root@deploy1003: Unlocked for deployment [ALL REPOSITORIES]: Datacenter switchover from codfw to eqiad - T436781 (duration: 48m 09s) 2454043 wikitext text/x-wiki 5gtm7bubdhsghox36aw75ehdahxxndq 2454044 2454043 2026-09-02T16:10:27Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.switchdc.mediawiki.09-unlock-scap (exit_code=0) for datacenter switchover from codfw to eqiad 2454044 wikitext text/x-wiki cqwljp5ir8jhuonyfliote84cysusyl 2454045 2454044 2026-09-02T16:11:35Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-ats (exit_code=0) rolling restart_daemons on A:cp-text_eqsin 2454045 wikitext text/x-wiki 6nj0t7xcwr1lomwnhzcnske7fn8yjk0 2454046 2454045 2026-09-02T16:14:14Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.cdn.roll-restart-ats rolling restart_daemons on A:cp-upload_drmrs 2454046 wikitext text/x-wiki lx4vsgeos0z1zczg5x6hkkxjsc10ir2 2454047 2454046 2026-09-02T16:24:46Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-ats (exit_code=0) rolling restart_daemons on A:cp-upload_drmrs 2454047 wikitext text/x-wiki 2q7cec5b2jxvu1fhiiibk3untsdz4sr 2454048 2454047 2026-09-02T16:29:25Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.cdn.roll-restart-ats rolling restart_daemons on A:cp-text_drmrs 2454048 wikitext text/x-wiki mtwqiz82m0ws2vefiurxgsygahoy5qt 2454049 2454048 2026-09-02T16:34:23Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.dns.netbox 2454049 wikitext text/x-wiki s1tv3m1s2yeidbxtyoneztet7dimsz9 2454050 2454049 2026-09-02T16:38:56Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1065] - vriley@cumin1003" 2454050 wikitext text/x-wiki bse4pkmuhjiy5jkxvv5xikp70aezwjx 2454051 2454050 2026-09-02T16:39:00Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1065] - vriley@cumin1003" 2454051 wikitext text/x-wiki s1eoe43nk4pf8xjpvl032av20gxbljh 2454052 2454051 2026-09-02T16:39:02Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2454052 wikitext text/x-wiki kvjzohm35jajow6lo4747xz9jfrru91 2454053 2454052 2026-09-02T16:41:29Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-ats (exit_code=0) rolling restart_daemons on A:cp-text_drmrs 2454053 wikitext text/x-wiki imypastlnqbec0ebx8junap6ovsarwl 2454054 2454053 2026-09-02T16:41:31Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1065 2454054 wikitext text/x-wiki 9vf7kgc5hfxchazdwpkqhooxr6fhfdj 2454055 2454054 2026-09-02T16:41:58Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1065 2454055 wikitext text/x-wiki 28t5v8217b5l66p78fwqkra83gvyetg 2454057 2454055 2026-09-02T16:46:35Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1065.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2454057 wikitext text/x-wiki flwqf29hbuuavzjd7rdqmhv6mn6f0f7 2454058 2454057 2026-09-02T16:48:31Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.cdn.roll-restart-ats rolling restart_daemons on A:cp-upload_eqsin 2454058 wikitext text/x-wiki 2mjmzzk25o3dnc520sxih7hsv74mgad 2454060 2454058 2026-09-02T16:59:34Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1065.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2454060 wikitext text/x-wiki 41orjhines4ux9pdb5yjhoswn6bl7vn 2454061 2454060 2026-09-02T17:00:39Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1065.eqiad.wmnet with OS trixie 2454061 wikitext text/x-wiki nuufoy1bh3i2ngojtrvkrn6g3peook4 2454062 2454061 2026-09-02T17:00:52Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-ats (exit_code=0) rolling restart_daemons on A:cp-upload_eqsin 2454062 wikitext text/x-wiki tdvbwgrbhcmach4m3ym2nfuwyxn4iej 2454065 2454062 2026-09-02T17:20:03Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.cdn.roll-restart-ats rolling restart_daemons on A:cp-eqsin 2454065 wikitext text/x-wiki r0ohfmnhmjulr26poj7f69frjog34uu 2454067 2454065 2026-09-02T17:35:30Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2454067 wikitext text/x-wiki jcoawpxs6u6mw6lmj1d6sss51qi8alt 2454068 2454067 2026-09-02T17:35:48Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2454068 wikitext text/x-wiki 22k9moi5xiuggryumu8teva4athz9q7 2454069 2454068 2026-09-02T17:38:04Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply 2454069 wikitext text/x-wiki pty3l0uoole4rygvu2eu97os6962a9k 2454070 2454069 2026-09-02T17:38:29Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply 2454070 wikitext text/x-wiki ju0n4vpj0pbq2ao3iq92u817ack8mok 2454071 2454070 2026-09-02T17:45:13Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-ats (exit_code=0) rolling restart_daemons on A:cp-eqsin 2454071 wikitext text/x-wiki 1fqjhwti45ea6m416gv42be3le3xum4 2454072 2454071 2026-09-02T17:47:58Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply 2454072 wikitext text/x-wiki rvdlkmmbd4r8xb63itupk1o5m2i1w6h 2454073 2454072 2026-09-02T17:48:27Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply 2454073 wikitext text/x-wiki ju70547xsku0wlac12y8w2z6cgizngj 2454074 2454073 2026-09-02T17:49:26Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.cdn.roll-restart-ats rolling restart_daemons on A:cp-eqiad 2454074 wikitext text/x-wiki mndazgtxlh4pq44o1ihiq5d57tdggqe 2454075 2454074 2026-09-02T17:58:19Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2454075 wikitext text/x-wiki 32uab9zd3wpqqyoj0o5ynsr223kt1xc 2454076 2454075 2026-09-02T17:58:44Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2454076 wikitext text/x-wiki hjzts8o3qdf9zoampje4uqajotqj65n 2454077 2454076 2026-09-02T18:02:46Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2454077 wikitext text/x-wiki a07o5q205tem2us89yg5mhvy8fq0a14 2454079 2454077 2026-09-02T18:02:58Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2454079 wikitext text/x-wiki pktqqou5h59dpuqk19nk2uxadlrqwwb 2454080 2454079 2026-09-02T18:14:01Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2454080 wikitext text/x-wiki mqfe5fb0wujtzm5238qs2dzf8ftsuku 2454081 2454080 2026-09-02T18:14:15Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-ats (exit_code=0) rolling restart_daemons on A:cp-eqiad 2454081 wikitext text/x-wiki 1subkemyck7rhgtvp7vff3pcr38cilx 2454082 2454081 2026-09-02T18:14:43Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2454082 wikitext text/x-wiki 7fjzbjnc5n0hnkvixbond587iq75abp 2454083 2454082 2026-09-02T18:16:28Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2454083 wikitext text/x-wiki tnu2npf70k7sxgr37xttfrcyxdlchcw 2454084 2454083 2026-09-02T18:16:30Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1065.eqiad.wmnet with OS trixie 2454084 wikitext text/x-wiki 3fxlcjntlf8a0naloy0r5pu7gcf3wkl 2454085 2454084 2026-09-02T18:16:40Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2454085 wikitext text/x-wiki sz8ykwhqpwib021qmopvwrxdoe470qv 2454086 2454085 2026-09-02T18:17:04Z Stashbot 7414 dancy@deploy1003: rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.18 refs T430837 2454086 wikitext text/x-wiki hg6zzc9s6dsse8p2da2lprwbd4yi94s 2454087 2454086 2026-09-02T18:26:32Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2454087 wikitext text/x-wiki ss4ido0ixkaudb3f0u9uoi1vpov928p 2454088 2454087 2026-09-02T18:26:43Z Stashbot 7414 swfrench@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2454088 wikitext text/x-wiki gezlnxow2ms6ewr07axi3ok0euxpzn2 2454089 2454088 2026-09-02T18:28:37Z Stashbot 7414 sukhe@dns1004: START - running authdns-update 2454089 wikitext text/x-wiki 7v7q3ahnwigqp11lmc2cftmp9iagu1h 2454090 2454089 2026-09-02T18:31:03Z Stashbot 7414 sukhe@dns1004: END - running authdns-update 2454090 wikitext text/x-wiki 3ywcw644mz4zpa9dzg8nnf0e55r7nrd 2454094 2454090 2026-09-02T18:53:03Z Stashbot 7414 jforrester@deploy1003: Started scap sync-world: Backport for [[gerrit:1334006|Revert "Remove fallback to Special:Upload and redirect user to alternative form" (T436851)]] 2454094 wikitext text/x-wiki is8gejzpfjikp10f3mpptfqg285t4tb 2454096 2454094 2026-09-02T18:55:34Z Stashbot 7414 swfrench-wmf: deleted pods coredns-85b4f68d95-pk5sn coredns-85b4f68d95-22ddb coredns-85b4f68d95-49k5p in eqiad due to intermittent upstream resolution health check failures correlated with high DNS resolution latency 2454096 wikitext text/x-wiki ayrwofgwzz69e1x8f4pzbzmgphwhdl6 2454097 2454096 2026-09-02T18:57:15Z Stashbot 7414 jforrester@deploy1003: jforrester: Backport for [[gerrit:1334006|Revert "Remove fallback to Special:Upload and redirect user to alternative form" (T436851)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2454097 wikitext text/x-wiki 6a5h9obxzpbu72vbigfuhikwlyjwljx 2454098 2454097 2026-09-02T18:57:59Z Stashbot 7414 jforrester@deploy1003: jforrester: Continuing with deployment 2454098 wikitext text/x-wiki sdhu4tgwhs333qlhdqupuvgpn9lninq 2454100 2454098 2026-09-02T19:55:36Z Stashbot 7414 bking@cumin2003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on matomo1003.eqiad.wmnet with reason: Matomo version upgrade T431608 2454100 wikitext text/x-wiki jhc00s1txe5lhjrzv8s5vuhnttqnu4x 2454101 2454100 2026-09-02T19:57:30Z Stashbot 7414 jforrester@deploy1003: Finished scap sync-world: Backport for [[gerrit:1334006|Revert "Remove fallback to Special:Upload and redirect user to alternative form" (T436851)]] (duration: 64m 27s) 2454101 wikitext text/x-wiki annpit3fabkrkl3l489z2legi62wa6c 2454102 2454101 2026-09-02T20:16:12Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.288.0" for 3 host(s) 2454102 wikitext text/x-wiki q1r7739gv0kj3c74q7syvkcy5ymasqe 2454103 2454102 2026-09-02T20:18:08Z Stashbot 7414 dancy@deploy1003: Installation of scap version "4.288.0" completed for 3 hosts 2454103 wikitext text/x-wiki d0mrky77gn7693a3jk99g9tu2womr3y 2454104 2454103 2026-09-02T20:18:37Z Stashbot 7414 dancy@deploy1003: Started scap sync-world: testing 2454104 wikitext text/x-wiki 4j9v5pgra9mzykshq6ygkn8l65n7sss 2454105 2454104 2026-09-02T20:27:59Z Stashbot 7414 dancy@deploy1003: Finished scap sync-world: testing (duration: 09m 21s) 2454105 wikitext text/x-wiki omaaqu50ng5xki6z3a8hvnw33y0ktyv 2454106 2454105 2026-09-02T20:50:55Z Stashbot 7414 rzl@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2454106 wikitext text/x-wiki izhw3t3zil49tihkziqtht1wwpwccgz 2454107 2454106 2026-09-02T21:01:07Z Stashbot 7414 rzl@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2454107 wikitext text/x-wiki j1zmiasxadgu0o8bk03m6gqthwi899x 2454108 2454107 2026-09-02T21:01:46Z Stashbot 7414 rzl@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2454108 wikitext text/x-wiki qzbqllwzfo0ves3gwuref7lgegbn36d 2454109 2454108 2026-09-02T21:03:29Z Stashbot 7414 rzl@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2454109 wikitext text/x-wiki 6i3607ay3m8lq0opw1a5ay6r7z4n7c4 2454111 2454109 2026-09-02T21:04:02Z Stashbot 7414 rzl@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2454111 wikitext text/x-wiki qqg4fam5eeka1kheyybvvy5dxre42lw 2454112 2454111 2026-09-02T21:04:50Z Stashbot 7414 rzl@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2454112 wikitext text/x-wiki 9rzwmc3ll953vhjrfn9v2hacpph1sfq 2454113 2454112 2026-09-02T21:11:11Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2454113 wikitext text/x-wiki mp4l8v7ntpyfz2gshqv7gg3th8tns1q 2454114 2454113 2026-09-02T21:11:21Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2454114 wikitext text/x-wiki 41fdbazyl0vame3ygtrsk0ob3c0br2e 2454115 2454114 2026-09-02T21:11:33Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2454115 wikitext text/x-wiki divpt7d63t02sqltqk4hbgxtxxxac2k 2454116 2454115 2026-09-02T21:11:42Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2454116 wikitext text/x-wiki 5wv162f4wg98e8ua64vodyhqwm5517l 2454117 2454116 2026-09-02T21:11:52Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2454117 wikitext text/x-wiki dp40bekt9i1xllzkclytuxwx15ctc46 2454118 2454117 2026-09-02T21:12:34Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2454118 wikitext text/x-wiki 588u8iidifq11nclyk5zye94u0vi98n 2454119 2454118 2026-09-02T21:19:27Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2454119 wikitext text/x-wiki 1zrv86mbbh41aj1jpc6faee9xo2tz4a 2454120 2454119 2026-09-02T21:19:36Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2454120 wikitext text/x-wiki 4sw8rxonrr10s6w3uhb7e8w6tir84e5 2454121 2454120 2026-09-02T21:19:46Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2454121 wikitext text/x-wiki hn5kuj4qyz4sowwdle9o956upzicdys 2454122 2454121 2026-09-02T21:21:30Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reimage for host aqs1017.eqiad.wmnet with OS bookworm 2454122 wikitext text/x-wiki 1sr8zg9m6xjmf00c2vy2ce5n3azlkc3 2454123 2454122 2026-09-02T21:24:09Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2454123 wikitext text/x-wiki lhh8hh3i9grcg96ekfdmjcqv22aucia 2454124 2454123 2026-09-02T21:24:21Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2454124 wikitext text/x-wiki 38o7wv9ykokririxnrz3su0cq2vllk2 2454125 2454124 2026-09-02T21:28:44Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2454125 wikitext text/x-wiki 3vgugm2sscugfv7kg22vh91iwzhmczt 2454126 2454125 2026-09-02T21:30:16Z Stashbot 7414 jforrester@deploy1003: Started scap sync-world: Backport for [[gerrit:1324368|[WikiLambda] Log the …Orchestrator and …AbstractClient channels too]], [[gerrit:1333732|wikifunctions: Set up the functionmaintainer right for the community (T435637)]] 2454126 wikitext text/x-wiki m108qm8ajz0lk6y8ck7gdl5fexkiq8g 2454127 2454126 2026-09-02T21:34:20Z Stashbot 7414 inflatador: bking@apt1002 sudo -E reprepro -C main include bookworm-wikimedia matomo-plugin-marketingcampaignsreporting_5.2.2-3_amd64.changes T431608 2454127 wikitext text/x-wiki 9gjafyf7vaohfhks2pbqawiv920pboz 2454128 2454127 2026-09-02T21:34:59Z Stashbot 7414 jforrester@deploy1003: jforrester: Backport for [[gerrit:1324368|[WikiLambda] Log the …Orchestrator and …AbstractClient channels too]], [[gerrit:1333732|wikifunctions: Set up the functionmaintainer right for the community (T435637)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2454128 wikitext text/x-wiki kdvfx6pzcvffudingythqfflwbe81gt 2454129 2454128 2026-09-02T21:35:28Z Stashbot 7414 jforrester@deploy1003: jforrester: Continuing with deployment 2454129 wikitext text/x-wiki g3eu4cvtu23bnvs44nqk7i67f146ard 2454130 2454129 2026-09-02T21:40:09Z Stashbot 7414 jforrester@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324368|[WikiLambda] Log the …Orchestrator and …AbstractClient channels too]], [[gerrit:1333732|wikifunctions: Set up the functionmaintainer right for the community (T435637)]] (duration: 09m 48s) 2454130 wikitext text/x-wiki 079feo7tnbu64dzrkmufx75q3d7v25l 2454131 2454130 2026-09-02T21:44:52Z Stashbot 7414 inflatador: bking@apt1002 sudo -E private_reprepro --ignore=wrongdistribution -C matomo_plugins include bookworm-wikimedia-private matomo-plugin-customreports_5.5.0-1_amd64.changes T431608 2454131 wikitext text/x-wiki 3ysz53qcmi3nkqqpe5rdz38491f1l7e 2454132 2454131 2026-09-02T21:50:49Z Stashbot 7414 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1334068|Extract MathJax DOM filter into a separate file (T435274)]], [[gerrit:1334070|Respect contextual binomial sizing (T434477 T418144 T401718)]], [[gerrit:1334032|Remove the last vestiges of $wgVirtualRestConfig (T436054)]] 2454132 wikitext text/x-wiki i1487s56x629vatxsr0pi0224xjjfys 2454133 2454132 2026-09-02T21:55:05Z Stashbot 7414 krinkle@deploy1003: krinkle: Backport for [[gerrit:1334068|Extract MathJax DOM filter into a separate file (T435274)]], [[gerrit:1334070|Respect contextual binomial sizing (T434477 T418144 T401718)]], [[gerrit:1334032|Remove the last vestiges of $wgVirtualRestConfig (T436054)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2454133 wikitext text/x-wiki le6to7c2di7bp47ete35v6e93ql04q2 2454134 2454133 2026-09-02T21:59:54Z Stashbot 7414 krinkle@deploy1003: krinkle: Continuing with deployment 2454134 wikitext text/x-wiki fzb4vjy4nsf9vjpicuba5ofhn42h1y2 2454135 2454134 2026-09-02T22:05:01Z Stashbot 7414 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1334068|Extract MathJax DOM filter into a separate file (T435274)]], [[gerrit:1334070|Respect contextual binomial sizing (T434477 T418144 T401718)]], [[gerrit:1334032|Remove the last vestiges of $wgVirtualRestConfig (T436054)]] (duration: 14m 14s) 2454135 wikitext text/x-wiki 1yxcerc8i04kj3u0nmzcsbjskyp9ahy 2454138 2454135 2026-09-02T22:26:27Z Stashbot 7414 jdlrobson@deploy1003: Started scap sync-world: Backport for [[gerrit:1333307|Campaigns should not override existing campaign query strings (T436681)]] 2454138 wikitext text/x-wiki me269layy12q3vfiy5684iongto8bdj 2454139 2454138 2026-09-02T22:30:44Z Stashbot 7414 jdlrobson@deploy1003: jdlrobson: Backport for [[gerrit:1333307|Campaigns should not override existing campaign query strings (T436681)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2454139 wikitext text/x-wiki 8ec71yj4qeitcynsdfc682mh42vpazf 2454140 2454139 2026-09-02T22:32:54Z Stashbot 7414 jdlrobson@deploy1003: jdlrobson: Continuing with deployment 2454140 wikitext text/x-wiki d8tq659w25lb0wr6hb0gowqeyrxaxkd 2454141 2454140 2026-09-02T22:37:30Z Stashbot 7414 jdlrobson@deploy1003: Finished scap sync-world: Backport for [[gerrit:1333307|Campaigns should not override existing campaign query strings (T436681)]] (duration: 11m 03s) 2454141 wikitext text/x-wiki g8f4kgkku5aun0p936hgpm5y9uijee8 2454146 2454141 2026-09-02T23:14:40Z Stashbot 7414 eevans@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host aqs1017.eqiad.wmnet with OS bookworm 2454146 wikitext text/x-wiki ks4i4mvamku09jn3nm71b2utdwns2qf 2454147 2454146 2026-09-02T23:14:55Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reimage for host aqs1017.eqiad.wmnet with OS bookworm 2454147 wikitext text/x-wiki 8qyjwbfo2tqna4dzy0wl3x93z3e05gh 2454148 2454147 2026-09-02T23:27:26Z Stashbot 7414 eevans@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host aqs1017.eqiad.wmnet with OS bookworm 2454148 wikitext text/x-wiki 4bwk7asmsmc9h397eo4dguy2tw4raa6 2454149 2454148 2026-09-02T23:27:39Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reimage for host aqs1017.eqiad.wmnet with OS bookworm 2454149 wikitext text/x-wiki 901q91rbfvl8qjmdai5q5nbuc0r502c 2454151 2454149 2026-09-02T23:38:45Z Stashbot 7414 eevans@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host aqs1017.eqiad.wmnet with OS bookworm 2454151 wikitext text/x-wiki jcdt3zla5fpgulqhrwsuxbsum6wyv3n 2454152 2454151 2026-09-02T23:38:58Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reimage for host aqs1017.eqiad.wmnet with OS bookworm 2454152 wikitext text/x-wiki gok3ee07blld7skewjw512ede2alnxh 2454153 2454152 2026-09-02T23:45:13Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1334141|private/readme.php: Remove now removed secrets (T436880)]] 2454153 wikitext text/x-wiki 78rs9q5v93hqgsdbphflg6czf3dnsxi 2454154 2454153 2026-09-02T23:49:35Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1334141|private/readme.php: Remove now removed secrets (T436880)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2454154 wikitext text/x-wiki bb8livr2se5vm33cyc527fbsv4ybrpt 2454155 2454154 2026-09-02T23:50:58Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2454155 wikitext text/x-wiki la929wzotp58nn4uev55xzwmg6d30hq 2454156 2454155 2026-09-02T23:51:40Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs1017.eqiad.wmnet with reason: host reimage 2454156 wikitext text/x-wiki tlp0kyq3llmj9ktcxf1w4664e07wfse 2454157 2454156 2026-09-02T23:55:34Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1334141|private/readme.php: Remove now removed secrets (T436880)]] (duration: 10m 21s) 2454157 wikitext text/x-wiki 6h9l68vy2n2sxwo8s0jxcujeesohvy8 2454158 2454157 2026-09-02T23:57:36Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1017.eqiad.wmnet with reason: host reimage 2454158 wikitext text/x-wiki dzb08bfj72jixjnpyl83xco93dia9kf 2454162 2454158 2026-09-03T00:16:16Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs1017.eqiad.wmnet with OS bookworm 2454162 wikitext text/x-wiki egedt2ceaczgp7emc5w4pz25s2a2daa 2454165 2454162 2026-09-03T02:01:24Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2454165 wikitext text/x-wiki 54y5iiqfbd6bopgaj98fhffp4w882bb 2454166 2454165 2026-09-03T02:09:13Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 07m 48s) 2454166 wikitext text/x-wiki sc8xw7bwupdkkwzloa8u3ejxkee6ub3 2454171 2454166 2026-09-03T05:56:47Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host vrts2002.codfw.wmnet 2454171 wikitext text/x-wiki sqasj9m0e01jpqpt4ai8urs5d2gs44j 2454172 2454171 2026-09-03T06:00:31Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host vrts1004.eqiad.wmnet 2454172 wikitext text/x-wiki mm5d004mrq6ntqwbxn7afr4iv73vfg6 2454173 2454172 2026-09-03T06:03:18Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host vrts2002.codfw.wmnet 2454173 wikitext text/x-wiki hrn16ahpm35cevonkjfx5w49wjor2p5 2454174 2454173 2026-09-03T06:04:20Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host vrts1004.eqiad.wmnet 2454174 wikitext text/x-wiki d8ucm0at7v4vbcqoecgchusrajdsz03 2454175 2454174 2026-09-03T06:04:28Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host vrts1003.eqiad.wmnet 2454175 wikitext text/x-wiki 3bwkesaow36joq5xqas3sqkgoy1ygyh 2454176 2454175 2026-09-03T06:11:05Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host vrts1003.eqiad.wmnet 2454176 wikitext text/x-wiki ew5b2rxluykmn7ne2iyvy3dgfblx6t0 2454177 2454176 2026-09-03T06:27:04Z Stashbot 7414 hashar: Upgrading CI Jenkins on contint1003 # T436812 2454177 wikitext text/x-wiki dritp8jiuirbii8hdtwfqv2qfazgmco 2454183 2454177 2026-09-03T06:46:07Z Stashbot 7414 moritzm: installing libxml2 security updates 2454183 wikitext text/x-wiki 9azlg6v5cr02ul1oo1xct8090edvr2e 2454184 2454183 2026-09-03T06:52:17Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.wdqs.restart-nginx-envoy rolling restart_daemons on A:wcqs-public 2454184 wikitext text/x-wiki 4n0h7obymw4mzwi3l1ytxh92ihp4mxf 2454185 2454184 2026-09-03T06:54:05Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.wdqs.restart-nginx-envoy (exit_code=0) rolling restart_daemons on A:wcqs-public 2454185 wikitext text/x-wiki fj32g3m21xed3h0xjotqi9h10bl61mf 2454186 2454185 2026-09-03T06:58:40Z Stashbot 7414 arnaudb@dns1006: START - running authdns-update 2454186 wikitext text/x-wiki etffiot6iy5j5zul2xj933u2s1j1vl2 2454187 2454186 2026-09-03T07:01:17Z Stashbot 7414 arnaudb@dns1006: END - running authdns-update 2454187 wikitext text/x-wiki ofv7lye2o5i903sskvy6q5djqc0yhjy 2454189 2454187 2026-09-03T07:13:51Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1228.eqiad.wmnet with reason: Onsite maintenance 2454189 wikitext text/x-wiki e0ucrbpfm5f6ntyjdwz4t0vyp2namlr 2454190 2454189 2026-09-03T07:15:18Z Stashbot 7414 marostegui: Power off db1228 for maintenance 2454190 wikitext text/x-wiki 9ues2di3cvsceuj8pudy1bvh0bvob5z 2454191 2454190 2026-09-03T07:15:38Z Stashbot 7414 chlod@deploy1003: Started scap sync-world: Backport for [[gerrit:1333872|core-Permissions.php: Allow English Wikiquote administrators to grant and remove the confirmed user permission (T436651)]] 2454191 wikitext text/x-wiki ocub7xj5zuklt21p6kcros83z6lf10c 2454192 2454191 2026-09-03T07:20:13Z Stashbot 7414 chlod@deploy1003: chlod, tryvix1509: Backport for [[gerrit:1333872|core-Permissions.php: Allow English Wikiquote administrators to grant and remove the confirmed user permission (T436651)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2454192 wikitext text/x-wiki os0zvox29t9rbl6he4e09tqp9rzjvv4 2454193 2454192 2026-09-03T07:22:12Z Stashbot 7414 XioNoX: push pfw policies - T436729 2454193 wikitext text/x-wiki d9boz48hvpkqbsz48jgzt7avtya2rgr 2454194 2454193 2026-09-03T07:22:51Z Stashbot 7414 chlod@deploy1003: chlod, tryvix1509: Continuing with deployment 2454194 wikitext text/x-wiki msszxpjovig4xn3v7rljqs9g961s8kr 2454196 2454194 2026-09-03T07:27:32Z Stashbot 7414 chlod@deploy1003: Finished scap sync-world: Backport for [[gerrit:1333872|core-Permissions.php: Allow English Wikiquote administrators to grant and remove the confirmed user permission (T436651)]] (duration: 11m 54s) 2454196 wikitext text/x-wiki s65pzvmecuwosoukl26o7tlqo6z057v 2454197 2454196 2026-09-03T07:29:47Z Stashbot 7414 chlod: UTC morning backport window done 2454197 wikitext text/x-wiki pp9dhs1aouhxcln7rexb4m4ughx2axd 2454198 2454197 2026-09-03T07:39:34Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cassandra-dev2001.codfw.wmnet 2454198 wikitext text/x-wiki jbdqs35m6bf1q1mbykn23udetpjneln 2454199 2454198 2026-09-03T07:49:17Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.reboot-single for host cassandra-dev2001.codfw.wmnet 2454199 wikitext text/x-wiki q51ycfmkys74dfw0d4y99xup4hpp12z 2454200 2454199 2026-09-03T07:58:50Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on 23 hosts with reason: Changing sanitarium master in s2 2454200 wikitext text/x-wiki 9qcwqns3k91bknxrlq1uolrrz98znrf 2454201 2454200 2026-09-03T07:59:17Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cassandra-dev2001.codfw.wmnet 2454201 wikitext text/x-wiki 2o328m832vv8m3jbr0rbl25wgauqviq 2454202 2454201 2026-09-03T07:59:19Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts cassandra-dev2001.codfw.wmnet 2454202 wikitext text/x-wiki s9krf3d0jkwm2h1l9gt5w35ho4jxq8f 2454203 2454202 2026-09-03T07:59:26Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cassandra-dev2001.codfw.wmnet 2454203 wikitext text/x-wiki 2l6t1mr9w3w79f25niwofh74vgxydrz 2454205 2454203 2026-09-03T08:03:16Z Stashbot 7414 marostegui: Move s2 sanitarium from db1156 to db1271 T434287 2454205 wikitext text/x-wiki tl3xi2g5837ltll1vfnbtbd2sr1opgq 2454207 2454205 2026-09-03T08:11:07Z Stashbot 7414 moritzm: uploaded wmf-laptop 1.0.7 to apt.wikimedia.org 2454207 wikitext text/x-wiki rpnb03f4bvgg8foq59hbdian015ozc2 2454211 2454207 2026-09-03T08:26:26Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts cassandra-dev2001.codfw.wmnet 2454211 wikitext text/x-wiki ekm35v2by2x3182wzg6nt0ofv2d8grl 2454212 2454211 2026-09-03T08:27:49Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.reimage for host cassandra-dev2001.codfw.wmnet with OS bookworm 2454212 wikitext text/x-wiki s8jrw42lkahearo9uj5i92nnnftt21h 2454214 2454212 2026-09-03T08:32:31Z Stashbot 7414 jnuche@deploy1003: Started deploy [releng/jenkins-deploy@b146ae7] (releasing): T436812 to backup server 2454214 wikitext text/x-wiki c95du3kw4k5q9niuexidfegpgg0nw24 2454215 2454214 2026-09-03T08:33:06Z Stashbot 7414 jnuche@deploy1003: Finished deploy [releng/jenkins-deploy@b146ae7] (releasing): T436812 to backup server (duration: 01m 28s) 2454215 wikitext text/x-wiki r7fxgv796dn6jvvnulxk9m9qakjnkzd 2454216 2454215 2026-09-03T08:34:07Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host cloudcephosd1053.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2454216 wikitext text/x-wiki s2hndwumg9jozrb7m04zcdduej808uf 2454217 2454216 2026-09-03T08:36:41Z Stashbot 7414 jnuche@deploy1003: Started deploy [releng/jenkins-deploy@b146ae7] (releasing): T436812 to prod server 2454217 wikitext text/x-wiki r5pfw2l4xe0539qfnt90cvnu6zd9btx 2454218 2454217 2026-09-03T08:37:19Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudcephosd1053.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2454218 wikitext text/x-wiki f4jqolzh3r907lyrp8m5l22l9rbjf0e 2454219 2454218 2026-09-03T08:37:44Z Stashbot 7414 jnuche@deploy1003: Finished deploy [releng/jenkins-deploy@b146ae7] (releasing): T436812 to prod server (duration: 01m 21s) 2454219 wikitext text/x-wiki j4szo1149tm81jsy1jg94pj0j1w08cj 2454220 2454219 2026-09-03T08:37:51Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host cloudcephosd1056.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2454220 wikitext text/x-wiki i5nlgghlb04g3dpv3dk9ewoua2vhvc4 2454221 2454220 2026-09-03T08:38:01Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on A:cp-text_magru 2454221 wikitext text/x-wiki pq9ni8kb694kuzm3qltpoivh8vxit4u 2454222 2454221 2026-09-03T08:39:42Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on 24 hosts with reason: Changing sanitarium master in s5 2454222 wikitext text/x-wiki 31jv71m59mvk3saer3qhbtzzoac93bm 2454223 2454222 2026-09-03T08:42:06Z Stashbot 7414 marostegui: Move s5 sanitarium from db1161 to db1275 T434776 2454223 wikitext text/x-wiki 4b1zc51o46on7g5o11atrf5z3dw6f4v 2454224 2454223 2026-09-03T08:43:35Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudcephosd1056.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2454224 wikitext text/x-wiki ek7z6w3jvd1ortoo5qkfvg8y3vqkgl3 2454225 2454224 2026-09-03T08:45:46Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cassandra-dev2001.codfw.wmnet with reason: host reimage 2454225 wikitext text/x-wiki m1qbtan3tf317tc4dj2y6f3ph22y8im 2454226 2454225 2026-09-03T08:45:50Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-purged (exit_code=0) rolling restart_daemons on A:cp-text_magru 2454226 wikitext text/x-wiki k146qj1inj7eeineo7365yy5nqfs3wf 2454227 2454226 2026-09-03T08:49:02Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cassandra-dev2001.codfw.wmnet with reason: host reimage 2454227 wikitext text/x-wiki 9dzdrmpej76tna86uq90zvnb9yxws05 2454228 2454227 2026-09-03T08:49:19Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on A:cp-text_codfw 2454228 wikitext text/x-wiki lvzhobmiit5468kx5kkfpw550ezxkez 2454230 2454228 2026-09-03T08:55:42Z Stashbot 7414 blake@cumin1003: START - Cookbook sre.memcached.roll-reboot-restart rolling reboot on A:memcached-codfw 2454230 wikitext text/x-wiki 62avkb43c8f9k1kw7fpr9giqyhtu6d8 2454231 2454230 2026-09-03T08:57:05Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-purged (exit_code=0) rolling restart_daemons on A:cp-text_codfw 2454231 wikitext text/x-wiki bhtweziqn0e3a0rtucmwxrz5t8y10k4 2454232 2454231 2026-09-03T08:57:44Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on A:cp-text_drmrs 2454232 wikitext text/x-wiki jwyk2bb8g1561ze17zofdlymur21rz3 2454233 2454232 2026-09-03T08:58:16Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 24 hosts with reason: Changing sanitarium master in s6 2454233 wikitext text/x-wiki 2nrzrvzvun7urvk5829uwpbmfgd2bx7 2454234 2454233 2026-09-03T09:03:21Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host aqs1018.eqiad.wmnet with OS bookworm 2454234 wikitext text/x-wiki fy3kdwrgsbyyu60clmfplbyid2m449w 2454235 2454234 2026-09-03T09:03:28Z Stashbot 7414 marostegui: Move s6 sanitarium from db1165 to db1279 T434775 2454235 wikitext text/x-wiki qr2bnknac4u2igte7ic99j3pc0iuc17 2454236 2454235 2026-09-03T09:05:23Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-purged (exit_code=0) rolling restart_daemons on A:cp-text_drmrs 2454236 wikitext text/x-wiki pfvpn5zb79sbal8en6wzbc5ssh8z7vs 2454237 2454236 2026-09-03T09:06:54Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on A:cp-text_esams 2454237 wikitext text/x-wiki 57zf5a182rzhvyh7ibz036crrn3eosr 2454238 2454237 2026-09-03T09:08:57Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cassandra-dev2001.codfw.wmnet with OS bookworm 2454238 wikitext text/x-wiki s9ng8df6wix0b1catadicvzw6wn4rkw 2454239 2454238 2026-09-03T09:09:22Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.reimage for host cassandra-dev2001.codfw.wmnet with OS bookworm 2454239 wikitext text/x-wiki t1nnr0ic5wi2dazjlzvpu8iz2bm4pbh 2454240 2454239 2026-09-03T09:14:42Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-purged (exit_code=0) rolling restart_daemons on A:cp-text_esams 2454240 wikitext text/x-wiki rhj83etw25zb3k08sq9ot8iks5onwf9 2454241 2454240 2026-09-03T09:15:39Z Stashbot 7414 topranks: put traffic on Lumen codfw<->eqiad link as it is stable T435810 2454241 wikitext text/x-wiki nuw28k9dztbh40bqcfkx8641i7rduhj 2454242 2454241 2026-09-03T09:17:36Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs1018.eqiad.wmnet with reason: host reimage 2454242 wikitext text/x-wiki 6nkafar7k15exc01lt6id1dwg8kkebh 2454243 2454242 2026-09-03T09:18:00Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on A:cp-text_eqsin 2454243 wikitext text/x-wiki pbprn5229caxe5837lk6ogsi0b8oye5 2454244 2454243 2026-09-03T09:21:05Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1334743|HookHandler: Fix bail out condition in onLocalUserCreated subscriber (T436791)]] 2454244 wikitext text/x-wiki lbfiysxvjvrm2hew3t0hkjar2haozgx 2454245 2454244 2026-09-03T09:23:33Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1018.eqiad.wmnet with reason: host reimage 2454245 wikitext text/x-wiki obwwdrdekjfu75ivyvn7839kmq3k0a8 2454246 2454245 2026-09-03T09:24:50Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Remove db1174 from dbctl T436904', diff saved to https://phabricator.wikimedia.org/P96323 and previous config saved to /var/cache/conftool/dbconfig/20260903-092448-marostegui.json 2454246 wikitext text/x-wiki 0npy4lcmzmx8d6q0yluop8nvlaz3ts9 2454248 2454246 2026-09-03T09:25:25Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1334743|HookHandler: Fix bail out condition in onLocalUserCreated subscriber (T436791)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2454248 wikitext text/x-wiki lh1isd5fqveyy1121u0qbp7scrco0p5 2454249 2454248 2026-09-03T09:25:31Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-purged (exit_code=0) rolling restart_daemons on A:cp-text_eqsin 2454249 wikitext text/x-wiki dthgcqrljz1e5wxdqh8x24nl5zxlp5o 2454250 2454249 2026-09-03T09:25:55Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on A:cp-text_eqiad 2454250 wikitext text/x-wiki 4xeoprol91cd0lvhymz2wnfub8557uf 2454251 2454250 2026-09-03T09:25:58Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2454251 wikitext text/x-wiki mskh361b09zzu4tj14uftb225g72hwm 2454252 2454251 2026-09-03T09:27:34Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cassandra-dev2001.codfw.wmnet with reason: host reimage 2454252 wikitext text/x-wiki sfm4d0agoh51omqb1wst2jphcbcjjzu 2454253 2454252 2026-09-03T09:30:35Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1334743|HookHandler: Fix bail out condition in onLocalUserCreated subscriber (T436791)]] (duration: 09m 30s) 2454253 wikitext text/x-wiki e73vhjmtx98yxlzoy1ah6o8amwnjjjz 2454254 2454253 2026-09-03T09:33:28Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-purged (exit_code=0) rolling restart_daemons on A:cp-text_eqiad 2454254 wikitext text/x-wiki c4g8cgv7nizuz80iryiyaetqd5qynh8 2454255 2454254 2026-09-03T09:33:49Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on A:cp-text_ulsfo 2454255 wikitext text/x-wiki g3h822uj1fvg5cbssnd9id77ffquxt3 2454256 2454255 2026-09-03T09:34:53Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: wikikube-worker-eqiad@eqiad 2454256 wikitext text/x-wiki co1n8wqfcnrfa5o6miek1gkv6tw6hiv 2454257 2454256 2026-09-03T09:35:02Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 24 hosts with reason: Changing sanitarium master in s7 2454257 wikitext text/x-wiki 04rxdgndzfbt43ood7dw9crtzlo4ezf 2454258 2454257 2026-09-03T09:35:13Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cassandra-dev2001.codfw.wmnet with reason: host reimage 2454258 wikitext text/x-wiki hg3v6z6u6yu24qem30eph9lldhsfj6g 2454259 2454258 2026-09-03T09:35:28Z Stashbot 7414 marostegui: Move s7 sanitarium from db1158 to db1273 T434775 2454259 wikitext text/x-wiki kgphq1k0m5a8mi39k2r0j46i611a50t 2454260 2454259 2026-09-03T09:37:47Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.ganeti.makevm for new host netflow2005.codfw.wmnet 2454260 wikitext text/x-wiki ibq1geanso6tlnurr0e25hvba58ob43 2454261 2454260 2026-09-03T09:37:49Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.dns.netbox 2454261 wikitext text/x-wiki 8673r965u11om445ybvfwing8cz31rk 2454262 2454261 2026-09-03T09:38:05Z Stashbot 7414 marostegui: Move s7 sanitarium from db1158 to db1273 T434751 2454262 wikitext text/x-wiki k37uwlo6f9e1b8844wzy7z7fu9g7rv5 2454263 2454262 2026-09-03T09:38:35Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-eqiad or A:lvs-secondary-eqiad) and A:bullseye and A:lvs 2454263 wikitext text/x-wiki koph76z07qoq24bruaz5b2jxdoizdud 2454264 2454263 2026-09-03T09:39:12Z Stashbot 7414 hnowlan: fixed currently oncall pane in klaxon 2454264 wikitext text/x-wiki a7bfg8i2e78sovj2bcym5qf47zp0vyb 2454265 2454264 2026-09-03T09:39:16Z Stashbot 7414 ayounsi@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on netflow2003.codfw.wmnet with reason: host reimage 2454265 wikitext text/x-wiki egi9vrwftpya4dikqkew168d382qvzt 2454266 2454265 2026-09-03T09:39:33Z Stashbot 7414 jelto@cumin1003: END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-eqiad or A:lvs-secondary-eqiad) and A:bullseye and A:lvs 2454266 wikitext text/x-wiki 47lf64tx6zns3pjsne7qkzqmnvlf09e 2454267 2454266 2026-09-03T09:39:34Z Stashbot 7414 jelto@cumin1003: END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: wikikube-worker-eqiad@eqiad 2454267 wikitext text/x-wiki 38njpbif5d2h87yj1cewdvmc5w6fisp 2454268 2454267 2026-09-03T09:41:25Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-purged (exit_code=0) rolling restart_daemons on A:cp-text_ulsfo 2454268 wikitext text/x-wiki fxmkifs0ea766x64w6uoqdmpmc746et 2454269 2454268 2026-09-03T09:41:32Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs1018.eqiad.wmnet with OS bookworm 2454269 wikitext text/x-wiki j4hwyfaj6xserqlz7wxtm2hcw06h2cf 2454270 2454269 2026-09-03T09:41:50Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM netflow2005.codfw.wmnet - ayounsi@cumin1003" 2454270 wikitext text/x-wiki ilqbotl6oa2xar4n2jjwfq8i5pg1f4p 2454271 2454270 2026-09-03T09:41:54Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM netflow2005.codfw.wmnet - ayounsi@cumin1003" 2454271 wikitext text/x-wiki hraeh5aywqfxoif3odznyuzi7irwd6h 2454272 2454271 2026-09-03T09:41:55Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2454272 wikitext text/x-wiki 9l4w6covz3lisf3d2hwiy3lzlyal59h 2454273 2454272 2026-09-03T09:41:57Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.dns.wipe-cache netflow2005.codfw.wmnet on all recursors 2454273 wikitext text/x-wiki dgnph5758dvs1oos9gog0s91kt5703y 2454274 2454273 2026-09-03T09:41:58Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) netflow2005.codfw.wmnet on all recursors 2454274 wikitext text/x-wiki beq7kbf8q21u27x7u1yi6wb5p7qr1aq 2454275 2454274 2026-09-03T09:42:27Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM netflow2005.codfw.wmnet - ayounsi@cumin1003" 2454275 wikitext text/x-wiki blkwjfeyc4zwijfma9yactk84pp3fgm 2454276 2454275 2026-09-03T09:42:31Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM netflow2005.codfw.wmnet - ayounsi@cumin1003" 2454276 wikitext text/x-wiki 4cxypsp2wqtb7i0nf62zzp6cxz4m5o9 2454277 2454276 2026-09-03T09:43:26Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.hosts.reimage for host netflow2005.codfw.wmnet with OS trixie 2454277 wikitext text/x-wiki 35p3de7bg0bfqvyxjyfovogq0wjr9ti 2454278 2454277 2026-09-03T09:55:13Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cassandra-dev2001.codfw.wmnet with OS bookworm 2454278 wikitext text/x-wiki 8sb1w8xgixg09ccb3qwyltdb1eqb80j 2454279 2454278 2026-09-03T09:55:39Z Stashbot 7414 ayounsi@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host netflow2003.codfw.wmnet with OS trixie 2454279 wikitext text/x-wiki f66qe28mli7tta1wyzsla55iu91j0k4 2454280 2454279 2026-09-03T09:56:24Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.reimage for host cassandra-dev2001.codfw.wmnet with OS bookworm 2454280 wikitext text/x-wiki ak2qkzuq17b8p7c2crmv2uxd78islv4 2454281 2454280 2026-09-03T10:03:45Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on netflow2005.codfw.wmnet with reason: host reimage 2454281 wikitext text/x-wiki cbav2hgti2o8lxcglr4fn1qun2lae7y 2454283 2454281 2026-09-03T10:09:29Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on netflow2005.codfw.wmnet with reason: host reimage 2454283 wikitext text/x-wiki 533lvb9skb11kfe38hji840yfrvbpbe 2454284 2454283 2026-09-03T10:11:21Z Stashbot 7414 marostegui: Move s8 sanitarium from db1167 to db1281 T434778 2454284 wikitext text/x-wiki na5yxtp2pme99kxcz3r9jdsy869xkj6 2454285 2454284 2026-09-03T10:12:19Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 24 hosts with reason: Changing sanitarium master in s8 2454285 wikitext text/x-wiki 5q2p276vsed0ot4ot6dk78bpkblzri2 2454286 2454285 2026-09-03T10:15:49Z Stashbot 7414 btullis@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-analytics: sync 2454286 wikitext text/x-wiki 8bzsxyb2t1gh5qaf19zkuis4whqqqyw 2454287 2454286 2026-09-03T10:15:56Z Stashbot 7414 btullis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-analytics: sync 2454287 wikitext text/x-wiki 74idqkspgvbohh2fcdedpl85yinixpn 2454288 2454287 2026-09-03T10:16:02Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host aqs1019.eqiad.wmnet with OS bookworm 2454288 wikitext text/x-wiki dbnlu9ti8rmhcnhqi3s358fbrzreo6g 2454289 2454288 2026-09-03T10:16:30Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cassandra-dev2001.codfw.wmnet with reason: host reimage 2454289 wikitext text/x-wiki ia0u9p0t0kqt5oj3lxg5vr3mz5pp6xk 2454290 2454289 2026-09-03T10:17:17Z Stashbot 7414 btullis@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-analytics: sync 2454290 wikitext text/x-wiki szv4drvdpj46656ngohkwcmrp6c7xua 2454291 2454290 2026-09-03T10:17:42Z Stashbot 7414 btullis@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-analytics: sync 2454291 wikitext text/x-wiki m77b3wbjykitvpzq8yblyfmb2y86l67 2454292 2454291 2026-09-03T10:20:19Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cassandra-dev2001.codfw.wmnet with reason: host reimage 2454292 wikitext text/x-wiki 9y2z562wf4956ejmgcp4i885va328nd 2454293 2454292 2026-09-03T10:29:22Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host netflow2005.codfw.wmnet with OS trixie 2454293 wikitext text/x-wiki hmzgch24sx1serxd10icv139ordvab7 2454294 2454293 2026-09-03T10:29:23Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host netflow2005.codfw.wmnet 2454294 wikitext text/x-wiki mpfotv68sud68dcd08fcwkizcxpclz8 2454295 2454294 2026-09-03T10:39:29Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs1019.eqiad.wmnet with reason: host reimage 2454295 wikitext text/x-wiki jzjcj263zf83f820yybba2nai3etb93 2454296 2454295 2026-09-03T10:40:23Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cassandra-dev2001.codfw.wmnet with OS bookworm 2454296 wikitext text/x-wiki 6d9mjua8qvtjdrfrb00zigbfxst0f2t 2454297 2454296 2026-09-03T10:41:12Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . 2454297 wikitext text/x-wiki iba3j97yf14njdv73fukj3oedz84z3n 2454298 2454297 2026-09-03T10:43:22Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1019.eqiad.wmnet with reason: host reimage 2454298 wikitext text/x-wiki ms36pspc1wpscd9pafbtpc3o9dza4bj 2454300 2454298 2026-09-03T10:52:08Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on A:cp-upload_codfw 2454300 wikitext text/x-wiki e0lvsyqxik49htlast9ibhfpqestnzk 2454301 2454300 2026-09-03T10:59:31Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-purged (exit_code=0) rolling restart_daemons on A:cp-upload_codfw 2454301 wikitext text/x-wiki kx14lgivyt6o2rhy85uxculev7rkfgw 2454302 2454301 2026-09-03T11:01:26Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on A:cp-upload_drmrs 2454302 wikitext text/x-wiki abbtwuoxau0iv3diyocqov1zjm6pf2b 2454303 2454302 2026-09-03T11:02:00Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs1019.eqiad.wmnet with OS bookworm 2454303 wikitext text/x-wiki pp186kr7xhmnyb73h81kkwaaazgen21 2454305 2454303 2026-09-03T11:09:13Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-purged (exit_code=0) rolling restart_daemons on A:cp-upload_drmrs 2454305 wikitext text/x-wiki qqmkflbm4a869005rpj20dpzwlvvjep 2454306 2454305 2026-09-03T11:10:03Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on A:cp-upload_esams 2454306 wikitext text/x-wiki nejhw7te8x92opr1ltyb6f7dsl20zw6 2454307 2454306 2026-09-03T11:14:18Z Stashbot 7414 cgoubert@deploy1003: Started scap sync-world: mediawiki: enable forward of fatal metrics to statsd exporter 2454307 wikitext text/x-wiki sc9pvc47vrscj5x9tqu594e29dsgpvr 2454308 2454307 2026-09-03T11:15:26Z Stashbot 7414 cgoubert@deploy1003: cgoubert: mediawiki: enable forward of fatal metrics to statsd exporter synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2454308 wikitext text/x-wiki jo1x44abqm9dcnqqbj8nifgt7dh2c89 2454309 2454308 2026-09-03T11:17:40Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-purged (exit_code=0) rolling restart_daemons on A:cp-upload_esams 2454309 wikitext text/x-wiki 6imon8qubvb7hk6ny1bo6sajshunpc6 2454310 2454309 2026-09-03T11:18:46Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on A:cp-upload_eqsin 2454310 wikitext text/x-wiki ttvyo6aopaauj1mtw7f46xnbplwntde 2454311 2454310 2026-09-03T11:22:56Z Stashbot 7414 cgoubert@deploy1003: cgoubert: Continuing with deployment 2454311 wikitext text/x-wiki cscpn8hdvazssvvvy3as5ma8k9zeefp 2454312 2454311 2026-09-03T11:24:25Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2207: db2207 repool 2454312 wikitext text/x-wiki 77k89d64g2xrbqwjojeu5fja9ykza20 2454313 2454312 2026-09-03T11:24:59Z Stashbot 7414 cgoubert@deploy1003: Finished scap sync-world: mediawiki: enable forward of fatal metrics to statsd exporter (duration: 12m 01s) 2454313 wikitext text/x-wiki r11o9glz480xzftewllqhypcu44ni0g 2454315 2454313 2026-09-03T11:26:40Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-purged (exit_code=0) rolling restart_daemons on A:cp-upload_eqsin 2454315 wikitext text/x-wiki spkdjuvp06h6zx57coq4gi5s0qu601p 2454316 2454315 2026-09-03T11:32:46Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host aqs1020.eqiad.wmnet with OS bookworm 2454316 wikitext text/x-wiki qq7qasyyj5v1t2xd73gsn9n6g0otjpq 2454317 2454316 2026-09-03T11:35:43Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on A:cp-upload_eqiad 2454317 wikitext text/x-wiki 7jx171s4bpyhzxx7o1d352d40e14z5b 2454318 2454317 2026-09-03T11:43:27Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-purged (exit_code=0) rolling restart_daemons on A:cp-upload_eqiad 2454318 wikitext text/x-wiki 1akon4yv4195gpcmgt0d4s4r5tfad0r 2454319 2454318 2026-09-03T11:45:28Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1282 to dbctl T407942', diff saved to https://phabricator.wikimedia.org/P96328 and previous config saved to /var/cache/conftool/dbconfig/20260903-114526-marostegui.json 2454319 wikitext text/x-wiki r8yputopt5gxx3116k48u170hytajyp 2454320 2454319 2026-09-03T11:46:03Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1282: Pooling db1282 into s6 2454320 wikitext text/x-wiki mq2xzv3qm3pzyghfozk0whrxn9t4azj 2454321 2454320 2026-09-03T11:47:22Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs1020.eqiad.wmnet with reason: host reimage 2454321 wikitext text/x-wiki cvz6294p43q19pkx4bji5n9qdfmzq3a 2454322 2454321 2026-09-03T11:51:50Z Stashbot 7414 kartik@deploy1003: helmfile [staging] START helmfile.d/services/cxserver: apply 2454322 wikitext text/x-wiki gdk9v0is40zxrnzy9nnzhq0m159pplg 2454323 2454322 2026-09-03T11:52:02Z Stashbot 7414 kartik@deploy1003: helmfile [staging] DONE helmfile.d/services/cxserver: apply 2454323 wikitext text/x-wiki o8uev1bth707yotgwugiya18h3yobi3 2454324 2454323 2026-09-03T11:54:57Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1020.eqiad.wmnet with reason: host reimage 2454324 wikitext text/x-wiki rj8c929c7592yfriig1nl8hj3rfvbo6 2454325 2454324 2026-09-03T11:55:16Z Stashbot 7414 kartik@deploy1003: helmfile [codfw] START helmfile.d/services/cxserver: apply 2454325 wikitext text/x-wiki f58kdl5l2nhm76j99z3s0emezsrwaje 2454326 2454325 2026-09-03T11:55:43Z Stashbot 7414 moritzm: installing rsync security updates 2454326 wikitext text/x-wiki 1ym6ydv0neokv7t1npdk3tcb88u7h6x 2454327 2454326 2026-09-03T11:55:45Z Stashbot 7414 kartik@deploy1003: helmfile [codfw] DONE helmfile.d/services/cxserver: apply 2454327 wikitext text/x-wiki m96w1ya14y44lcnmtuap1aegrjms332 2454328 2454327 2026-09-03T11:56:46Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on A:cp-upload_magru 2454328 wikitext text/x-wiki k0y6lfomti7u1oybrg637kfeaxx54p5 2454329 2454328 2026-09-03T11:56:50Z Stashbot 7414 kartik@deploy1003: helmfile [eqiad] START helmfile.d/services/cxserver: apply 2454329 wikitext text/x-wiki hzyfbfokqm5bx7y0lfg8fkn1vtzjlh0 2454330 2454329 2026-09-03T11:57:17Z Stashbot 7414 kartik@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cxserver: apply 2454330 wikitext text/x-wiki fhyxuqrtgn0gpsul2csef919z8egemt 2454331 2454330 2026-09-03T11:58:42Z Stashbot 7414 kart_: cxserver: Use urldownloader LVS endpoint (T429175) 2454331 wikitext text/x-wiki abv93ec3a3xwl0j7kglz8h77casn8lu 498 9925 2454137 2453671 2026-09-02T22:21:27Z Stashbot 7414 Southparkfan: decommission deployment-webperf21 - T436464 2454137 wikitext text/x-wiki 3w4dpn3w4l7jscj03gryf8gyz4f489z 0 17290 2453977 2453670 2026-09-02T14:52:14Z Stashbot 7414 hashar: Tag Quibble 1.20.0 @ c6f13962c27685773fc1fb39d74189b202e0961d # T321617 T432966 T428642 T435975 2453977 wikitext text/x-wiki dhw1ba7rfrvrxqv5bur1t4ix8jg0pl7 2453981 2453977 2026-09-02T15:14:43Z Stashbot 7414 hashar: Building Quibble 1.20.0 images 2453981 wikitext text/x-wiki aulmhtfuuohnoubtrjz7z0x2eri0e7c 2454001 2453981 2026-09-02T15:37:59Z Stashbot 7414 hashar: jjb: update Quibble jobs to 1.20.0 # T321617 T432966 T428642 T435975 2454001 wikitext text/x-wiki lzov5qag4e4g3dfw6nxnedqq8mysi92 2454064 2454001 2026-09-02T17:09:14Z Stashbot 7414 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1333891 2454064 wikitext text/x-wiki 5r31yltvlgvurqu98zzkbt8en8m9ge3 2454136 2454064 2026-09-02T22:21:24Z Stashbot 7414 Southparkfan: decommission deployment-webperf21 - T436464 2454136 wikitext text/x-wiki g1yr1ahh6a94gem711bsm11j66a4sm6 2454170 2454136 2026-09-03T05:15:52Z Stashbot 7414 hashar: Updating tox jobs to change default python from 3.9 to 3.11 | https://gerrit.wikimedia.org/r/c/integration/config/+/1334023 | T436857 2454170 wikitext text/x-wiki 931wqvv0xqids7b1itom0pba7euwfld 2454178 2454170 2026-09-03T06:27:06Z Stashbot 7414 hashar: Upgrading CI Jenkins on contint1003 # T436812 2454178 wikitext text/x-wiki qw0mdxym2lropwtxl2p0fqnu6hx2m15 2454195 2454178 2026-09-03T07:25:53Z Stashbot 7414 hashar: integration: granted sudo access to Phedenskog 2454195 wikitext text/x-wiki q2ajwrx8szjhhexe8wxnca408y6suty 2454209 2454195 2026-09-03T08:13:45Z Stashbot 7414 James_F: Zuul: [mediawiki/services/similar-users] Archive service, for T368269 2454209 wikitext text/x-wiki 9cgca6vz2wf1nyymgn4m1hrpsb0cltu 2454314 2454209 2026-09-03T11:26:13Z Stashbot 7414 hashar: Deleted coverage report for SimilarEditors ( /srv/doc/cover-extensions/SimilarEditors ), extension is being archived # T436880 2454314 wikitext text/x-wiki d64tqw7wmc1zlgynaie6jqvy7rx64pg 498 22581 2454180 2395743 2026-09-03T06:38:32Z PHedenskog (WMF) 5986 Updated flavor 2454180 wikitext text/x-wiki ao7daiklfrij0n21ghlc0kh8yhyl98b 498 30942 2453967 2453865 2026-09-02T14:25:09Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch 2453967 wikitext text/x-wiki rb72dvx1vdm5e9p7l0wktq05coxu06a 2453971 2453967 2026-09-02T14:26:25Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch 2453971 wikitext text/x-wiki qg37ewpydte2b21494qcy9q9dfq1yt2 0 443299 2453875 2453867 2026-09-02T12:01:07Z Milimetric (WMF) 39785 2453875 wikitext text/x-wiki advlt72n5j58i3odg22e5fd9yhwrgrl 498 443628 2454282 2453643 2026-09-03T10:04:45Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/33742087870 (https://github.com/cluebotng/component-configs/commits/781284a9a6b62d000c7c570fb34d20bbe21933a0) 2454282 wikitext text/x-wiki t8phmno72fxgfvrqdhtz0gup85m77f8 0 445715 2453930 2085617 2026-09-02T13:35:03Z Nintendofan885 17301 historical (RIP tech blog 2.0) 2453930 wikitext text/x-wiki d7l81k2vyfftv7hl5wvvfag4euvil5c 2453933 2453930 2026-09-02T13:39:45Z Nintendofan885 17301 /* History */ merge 2453933 wikitext text/x-wiki tbyd8r8uwr4anoa9r6m8u0evvvv5nrd 2453934 2453933 2026-09-02T13:40:13Z Nintendofan885 17301 /* History */ link 2453934 wikitext text/x-wiki 42lyvmutly6kul9cg9rq4z7dxa3kh7q 498 446211 2454247 2453563 2026-09-03T09:25:05Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/33738502759 (https://github.com/cluebotng/component-configs/commits/a57be2aa8221abd40bf701a3d617fddcec5fd47b) 2454247 wikitext text/x-wiki oh5toepte2ynkf4s8qxlry1h1o2qu92 0 446902 2454143 2302512 2026-09-02T22:50:48Z WBrown (WMF) 37696 archive 2454143 wikitext text/x-wiki qrq4p7mobvih1er3f220o2o2dpgquh8 2454144 2454143 2026-09-02T22:51:20Z WBrown (WMF) 37696 archive 2454144 wikitext text/x-wiki 3jiq2ue7ejdr5uw4q6yp85rsalbax4f 2454145 2454144 2026-09-02T22:51:28Z WBrown (WMF) 37696 ce 2454145 wikitext text/x-wiki 8ik4c1jhwa4rb7v6b9r5jt13hr3peso 0 449053 2453983 2451873 2026-09-02T15:16:06Z TBurmeister (WMF) 27417 TBurmeister (WMF) moved page [[User:AWesterinen/Blazegraph Features and Capabilities]] to [[Wikidata Query Service/Migration/Blazegraph Features and Capabilities]]: Moving into mainspace to be collocated with other relevant content 2451873 wikitext text/x-wiki 8avpkt6u4igt422l2p6lz2c3ifrltmh 2454059 2453983 2026-09-02T16:59:00Z TBurmeister (WMF) 27417 Add historical template with note 2454059 wikitext text/x-wiki pwlfq431ncqz88tjg78snyrol9xn8t8 0 449160 2454159 2453682 2026-09-03T00:01:28Z Dexbot 30554 Bot: Updating the report 2454159 wikitext text/x-wiki 7hv3a0he9jrsoec6jzlhsa7dkn6g130 1 450752 2453985 2401162 2026-09-02T15:16:06Z TBurmeister (WMF) 27417 TBurmeister (WMF) moved page [[User talk:AWesterinen/Blazegraph Features and Capabilities]] to [[Talk:Wikidata Query Service/Migration/Blazegraph Features and Capabilities]]: Moving into mainspace to be collocated with other relevant content 2401162 wikitext text/x-wiki 0p0len05r2fexh606y2e2mjqx5c3lfq 0 453464 2454229 2368681 2026-09-03T08:52:06Z FNegri-WMF 32595 /* Procedures */ Add "Rotate passwords" section 2454229 wikitext text/x-wiki oosq4olzsexlbti015mnyzzjpelnv82 116 453906 2454056 2453654 2026-09-02T16:42:17Z Gitlabaccountapprovalbot 37332 maximocampo was rejected. 2454056 wikitext text/x-wiki r3764680rzdvzfdsap8h0oib7wwxv3p 2454150 2454056 2026-09-02T23:30:29Z Gitlabaccountapprovalbot 37332 @aplucas0703 was approved. 2454150 wikitext text/x-wiki cgygc5gv3jzgg75n9droh5b3iq57xb0 2454206 2454150 2026-09-03T08:09:28Z Gitlabaccountapprovalbot 37332 @real68er was approved. 2454206 wikitext text/x-wiki 7cnjv1zrkgynoae94zkmtrdwmslhgcg 12 454182 2453883 2153129 2026-09-02T12:20:48Z CGoubert-WMF 32775 Update stat1011 fingerprints 2453883 wikitext text/x-wiki kjd6eysiq2duqer87yvp9y65ki4njct 0 459087 2454095 2439654 2026-09-02T18:54:49Z GChoi-WMF 39250 /* Service Level Indicators (SLIs) */ 2454095 wikitext text/x-wiki 57qtj46lmiutn0iaj3p0dgkjw9ariht 0 459697 2454066 2451869 2026-09-02T17:28:08Z TBurmeister (WMF) 27417 Mark as archived 2454066 wikitext text/x-wiki bhc2fn3f3b2jywva0hosrs6dkqzhswo 0 460234 2454091 2453005 2026-09-02T18:38:25Z MPopov (WMF) 5891 Update for new experiment creation workflow in GB 5.0 (T435217) 2454091 wikitext text/x-wiki bzgszoouvwsmhv5nj294he9j4jctk04 2454099 2454091 2026-09-02T19:11:26Z MPopov (WMF) 5891 /* Create an experiment */ Link to pre-5.0 documentation 2454099 wikitext text/x-wiki lq822tedbw06x66isr4wtkn9v6g09q0 0 460388 2454142 2434626 2026-09-02T22:41:38Z AKhatun (WMF) 50461 Added TOC 2454142 wikitext text/x-wiki hs8pnfw7mz9c58yzfvmrahihtvswwkt 0 460417 2453876 2445801 2026-09-02T12:08:33Z GKyziridis-WMF 39160 Update the API reference Note adding the swagger LLM Endpoint in mediawiki 2453876 wikitext text/x-wiki 5eglqxm86dk7hr6chy6dd0nnfxp2feb 2453877 2453876 2026-09-02T12:09:19Z GKyziridis-WMF 39160 2453877 wikitext text/x-wiki cb0nsrqfhsrhuqk8bm2lx1w35gxntax 0 460515 2453929 2450253 2026-09-02T13:32:47Z Ottomata 88 /* Linked Artifacts Cache */ 2453929 wikitext text/x-wiki k3y0xm7bbmyfzxk3y4eagdwneupqrzj 0 460574 2453878 2452756 2026-09-02T12:11:09Z JGiannelos (WMF) 17083 2453878 wikitext text/x-wiki lix7wvkut5g36o2sjwds7y3zicg628q 2453890 2453878 2026-09-02T12:25:09Z JGiannelos (WMF) 17083 /* Troubleshooting */ 2453890 wikitext text/x-wiki qx5dil4cq8uan9y4lyimbwn5xiltbzm 2453891 2453890 2026-09-02T12:26:32Z JGiannelos (WMF) 17083 /* Realistic targets */ 2453891 wikitext text/x-wiki funlzvroxz1uski74udobi90wlritiq 2453892 2453891 2026-09-02T12:27:42Z JGiannelos (WMF) 17083 /* Service Level Objectives */ 2453892 wikitext text/x-wiki hmp4sy5md46x4gzvk9byg5jdfgwaahs 2453893 2453892 2026-09-02T12:28:37Z JGiannelos (WMF) 17083 /* Ideal targets */ 2453893 wikitext text/x-wiki f3qpgf16kf8wa2mz5ygyvp07pk5zkzj 2453894 2453893 2026-09-02T12:29:22Z JGiannelos (WMF) 17083 /* Service Level Objectives */ 2453894 wikitext text/x-wiki brjcq10vl2fqfyj06070l3aqdveh67c 2453907 2453894 2026-09-02T12:42:03Z JGiannelos (WMF) 17083 /* Ideal targets */ 2453907 wikitext text/x-wiki h9ta8krg4zna1daoh7x1lawfjhyee51 2453909 2453907 2026-09-02T12:45:56Z JGiannelos (WMF) 17083 /* Ideal targets */ 2453909 wikitext text/x-wiki seox5rt42dj6yw1umqj8waidadz9204 2453910 2453909 2026-09-02T12:48:22Z JGiannelos (WMF) 17083 /* Ideal targets */ 2453910 wikitext text/x-wiki 1o7gj8bna8nxtd26kkt3g3o500cd0ds 2453911 2453910 2026-09-02T12:48:44Z JGiannelos (WMF) 17083 /* Realistic targets (based on the initial baseline) */ 2453911 wikitext text/x-wiki 9rtjha91kx0nm0b0dsooq4wwyiu293p 2453912 2453911 2026-09-02T12:49:14Z JGiannelos (WMF) 17083 /* Realistic targets (based on the initial baseline - August 2026) */ 2453912 wikitext text/x-wiki 3rfdhfe6oewn4hb4gobqesersul0zqr 2453913 2453912 2026-09-02T12:55:19Z JGiannelos (WMF) 17083 /* Service Level Objectives */ 2453913 wikitext text/x-wiki cyfvbok63prsdk48dqto0jvcnlw0dya 0 460600 2453947 2453655 2026-09-02T14:00:41Z AKhatun (WMF) 50461 SRE meeting notes 2453947 wikitext text/x-wiki m0m7afcokkxt40lpvyi6lvak6s1ea0g 1 460617 2453942 2453763 2026-09-02T13:55:14Z GLavagetto (WMF) 2905 /* When do we need code review? */ Reply 2453942 wikitext text/x-wiki p1fsrs5ucrgut0qgr8ylbtchbwwin4x 2453945 2453942 2026-09-02T13:58:11Z GLavagetto (WMF) 2905 /* Pick one reviewer and ping them */ Reply 2453945 wikitext text/x-wiki 5r9zre0tqgen6gzjmg269ptn220g52t 2453946 2453945 2026-09-02T14:00:41Z GLavagetto (WMF) 2905 /* Do not comment on a patch without casting a vote */ Reply 2453946 wikitext text/x-wiki 7o5qd08kcqiamkqavtxloz7n5tu4ojr 2453974 2453946 2026-09-02T14:28:33Z CDanis (WMF) 13326 /* Another exception for pushing code without review */ Reply 2453974 wikitext text/x-wiki f8lm5n5eb8s5tpuun6nfjpruoidc7d2 2454213 2453974 2026-09-03T08:27:50Z AYounsi (WMF) 10040 /* Do not comment on a patch without casting a vote */ Reply 2454213 wikitext text/x-wiki l2h7crh6qhu808m4vd6b8z1hz8qung3 2 460620 2453879 2453872 2026-09-02T12:13:52Z STrug-WMF 51028 2453879 wikitext text/x-wiki aoqhszte5hzalieu615i3e53n7thejb 2453880 2453879 2026-09-02T12:14:56Z STrug-WMF 51028 2453880 wikitext text/x-wiki p9ouwjw6k8leuxppy2cll113lfir11s 2453885 2453880 2026-09-02T12:22:34Z STrug-WMF 51028 2453885 wikitext text/x-wiki k6p19dkdt0rj54r884txtongl6j9xk5 2453932 2453885 2026-09-02T13:39:05Z STrug-WMF 51028 2453932 wikitext text/x-wiki ldp1htsobkd3g6fwaq7ky951kbbm187 2454208 2453932 2026-09-03T08:13:16Z STrug-WMF 51028 2454208 wikitext text/x-wiki 6865oobydux844uleb32fyau3ixy1k2 2454210 2454208 2026-09-03T08:14:59Z STrug-WMF 51028 2454210 wikitext text/x-wiki 2owz82wj8y4nd1j3vffd1vqygqsxhbc 0 460621 2453936 2026-09-02T13:45:38Z JGiannelos (WMF) 17083 Created page with "Status: '''[[SLO/Runbook#Drafting|draft]]''' == Organizational == === Service === The service is MediaWiki core (with Parsoid as a parser backend) and more specifically this SLO focuses on the read view of an article. This covers only the backend part of the stack so only the read views that went through all the caching layers on edge. === Teams === * Core platform team ** Responsible for the performance of the MediaWiki core backend that is serving MediaWiki article..." 2453936 wikitext text/x-wiki cyfvbok63prsdk48dqto0jvcnlw0dya 2453937 2453936 2026-09-02T13:49:43Z JGiannelos (WMF) 17083 /* Ideal targets */ 2453937 wikitext text/x-wiki 6lyn1phb8mi663w63yxdfyi32hu9tjc 2453938 2453937 2026-09-02T13:50:52Z JGiannelos (WMF) 17083 /* Organizational */ 2453938 wikitext text/x-wiki 08yloybfdrkp210qjupf2y06qsztsdc 2453939 2453938 2026-09-02T13:51:35Z JGiannelos (WMF) 17083 /* Environmental dependencies */ 2453939 wikitext text/x-wiki nckb8057g5lq2m1w1mkij2inf28szyj 2453940 2453939 2026-09-02T13:54:22Z JGiannelos (WMF) 17083 /* Troubleshooting */ 2453940 wikitext text/x-wiki 6qeay5pnrpweo64ko3554gammwexev3 2453941 2453940 2026-09-02T13:54:49Z JGiannelos (WMF) 17083 /* Troubleshooting */ 2453941 wikitext text/x-wiki rllwv12rh79civptdrz0e0wq0moftde 2453943 2453941 2026-09-02T13:55:34Z JGiannelos (WMF) 17083 /* Troubleshooting */ 2453943 wikitext text/x-wiki 9rjvcigyot0t796oxh7v4ckv3mt61ss 2453944 2453943 2026-09-02T13:56:20Z JGiannelos (WMF) 17083 /* Troubleshooting */ 2453944 wikitext text/x-wiki my1pac6qnhg9npphcx6hy4yhlcl97vl 2453979 2453944 2026-09-02T15:09:10Z JGiannelos (WMF) 17083 /* Metric */ 2453979 wikitext text/x-wiki 0d8bqtnol68wkx6lpycbafxf5wwkm8p 2453994 2453979 2026-09-02T15:25:19Z JGiannelos (WMF) 17083 /* Service Level Indicators (SLIs) */ 2453994 wikitext text/x-wiki od9smlszv8fkbo92phmaulhli0zs9yl 2453998 2453994 2026-09-02T15:29:26Z JGiannelos (WMF) 17083 /* Service Level Indicators (SLIs) */ 2453998 wikitext text/x-wiki on3aa5egwdfaws2eoy41vjxpeh51p60 2454000 2453998 2026-09-02T15:34:42Z JGiannelos (WMF) 17083 /* Ideal targets */ 2454000 wikitext text/x-wiki 9os2rhas2q0p4p2pakg8orld0h2lmud 2454009 2454000 2026-09-02T15:50:48Z JGiannelos (WMF) 17083 /* Ideal targets */ 2454009 wikitext text/x-wiki 2efvgi0p06cewocguiiln1ph2q75fro 2454063 2454009 2026-09-02T17:07:04Z JGiannelos (WMF) 17083 2454063 wikitext text/x-wiki qhasz4f4sbfylwhqvqylvcgkcx9ckek 2 460622 2453984 2026-09-02T15:16:06Z TBurmeister (WMF) 27417 TBurmeister (WMF) moved page [[User:AWesterinen/Blazegraph Features and Capabilities]] to [[Wikidata Query Service/Migration/Blazegraph Features and Capabilities]]: Moving into mainspace to be collocated with other relevant content 2453984 wikitext text/x-wiki 4dphzf2m7ba8wzabmhqmlgnocnvs6nu 3 460623 2453986 2026-09-02T15:16:06Z TBurmeister (WMF) 27417 TBurmeister (WMF) moved page [[User talk:AWesterinen/Blazegraph Features and Capabilities]] to [[Talk:Wikidata Query Service/Migration/Blazegraph Features and Capabilities]]: Moving into mainspace to be collocated with other relevant content 2453986 wikitext text/x-wiki i3ldmnixmvx0j0hzpg9pletuptr8n7y 0 460624 2454078 2026-09-02T18:02:57Z SSastry (WMF) 344 Draft SCROLL page for Produnto -- needs a bit more to be filled out. Creating of a wikitech page for it might help fill out more of the required entries. 2454078 wikitext text/x-wiki t8wtt62i4nb7c096ufwm3e49fplsfnw 2454093 2454078 2026-09-02T18:51:38Z SSastry (WMF) 344 Fill out more info + remove phab template and use the direct phab:TXXX wikilink format. 2454093 wikitext text/x-wiki hr4surkzft7plampruxrbv3ndhsp51n 2454163 2454093 2026-09-03T00:38:09Z Tim Starling (WMF) 38382 remove placeholder tags; fill out quick references 2454163 wikitext text/x-wiki qgldvmjpt29w0dygpe3devkwdrtpksu 2454179 2454163 2026-09-03T06:27:53Z Tim Starling (WMF) 38382 filled out some boxes 2454179 wikitext text/x-wiki fj6v5jyweln5ouuzhst8ixrmud33j34 2454182 2454179 2026-09-03T06:39:57Z Tim Starling (WMF) 38382 filled in more boxes 2454182 wikitext text/x-wiki hh79zict152wjy9y58l3wibycv8w1bm 0 460625 2454092 2026-09-02T18:48:05Z SSastry (WMF) 344 A draft with some minimal information filled in. See https://wikitech.wikimedia.org/wiki/Duck for guidance for the sections and what might be useful here. 2454092 wikitext text/x-wiki 6endv7l6zumpzraby9p6g2sqd1i3otw 2454167 2454092 2026-09-03T02:45:18Z Tim Starling (WMF) 38382 full description of extension 2454167 wikitext text/x-wiki f4gfqq6iuaw9lvjvzdrb6p62x2s4prx 2454168 2454167 2026-09-03T02:47:55Z Tim Starling (WMF) 38382 remove placeholders 2454168 wikitext text/x-wiki cjokhctuobcyonco8bisi33uue96dfc 0 460626 2454160 2026-09-03T00:03:08Z ASarabadani (WMF) 39030 Created page with "This page documents how our [[w:Mutex|lock]] manager system works in our production. == MediaWiki == On MediaWiki side of things, there is a service called "LockManager" providing [https://gerrit.wikimedia.org/g/mediawiki/core/+/master/includes/libs/LockManager/ILockManager.php ILockManager] interface. And callers can lock/unlock a key using this service. The lock manager returned by this service can be set via $wgDefaultLockManager config variable. You'd need to intro..." 2454160 wikitext text/x-wiki bgzun5tgwukt8yux0fe9r856f2opzg4 0 460627 2454164 2026-09-03T01:37:44Z AKhatun (WMF) 50461 Stub created 2454164 wikitext text/x-wiki mnzkt5s9rxnp6yvofazw9b1jsrew54z 6 460628 2454204 2026-09-03T07:59:39Z APizzata-WMF 47941 {{File information |Description= Architecture drawing for the mediawiki.page_change.v1 stream, including hard and soft dependencies |Source= |Date= |Author= |other_versions= }} 2454204 wikitext text/x-wiki 2kvw6iuabruw5er743k53bte9pvol8c