Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.5 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 0 4108 2424028 2424017 2026-06-07T22:17:35Z TCipriani (WMF) 4895 /* Week of June 15 */ 2424028 wikitext text/x-wiki reyuarhj91bm8voe4nyhfmfd91lsh52 2424056 2424028 2026-06-08T06:45:27Z ScheduleDeploymentBot 37566 Add [[gerrit:1298418]] to Monday, June 08 UTC morning backport window 2424056 wikitext text/x-wiki sqsto57b98ytu1nnoy2p0e31krdjks9 2424072 2424056 2026-06-08T07:38:32Z MSzwarc-WMF 44780 /* {{Deployment_day|date=2026-06-09}} */ +1 2424072 wikitext text/x-wiki out0pua6tw0witcj15sh62ucgn0pe9i 2424073 2424072 2026-06-08T07:41:57Z ScheduleDeploymentBot 37566 Add [[gerrit:1298390]] to Monday, June 08 UTC late backport window 2424073 wikitext text/x-wiki t5h9ldx3pov2yc4uoo1d0flxhewj125 2424074 2424073 2026-06-08T07:42:16Z ScheduleDeploymentBot 37566 Add [[gerrit:1298328]] to Monday, June 08 UTC late backport window 2424074 wikitext text/x-wiki 1cgybryo8ewp8wph4z3eevutkt01jj2 2424083 2424074 2026-06-08T07:58:55Z ScheduleDeploymentBot 37566 Add [[gerrit:1298418]] to Monday, June 08 UTC afternoon backport window 2424083 wikitext text/x-wiki hxbxpq2y2kuigh32n34v0lo2no6hga6 2424090 2424083 2026-06-08T08:16:40Z ScheduleDeploymentBot 37566 Add [[gerrit:1298654]] to Monday, June 08 UTC afternoon backport window 2424090 wikitext text/x-wiki 125e54pvcqb2x87gxbbd2ahx2e6cemx 2424091 2424090 2026-06-08T08:18:28Z Neriah 36702 2424091 wikitext text/x-wiki ok6b5qyolwr2daq9xzaf34x5vfhzbqk 2424093 2424091 2026-06-08T08:25:51Z ScheduleDeploymentBot 37566 Add [[gerrit:1297644]] to Monday, June 08 UTC afternoon backport window 2424093 wikitext text/x-wiki jlcpogrxoek979hnb3kbr5gqswq7ssb 2424095 2424093 2026-06-08T08:49:04Z ScheduleDeploymentBot 37566 Add [[gerrit:1298709]] to Monday, June 08 UTC afternoon backport window 2424095 wikitext text/x-wiki 4i5si39bzm32aafa85tzm01khiwrwt6 2424096 2424095 2026-06-08T08:49:10Z ScheduleDeploymentBot 37566 Add [[gerrit:1298710]] to Monday, June 08 UTC afternoon backport window 2424096 wikitext text/x-wiki nr7jo0w76t6chuex8u07fgwjqpivvdz 2424097 2424096 2026-06-08T08:49:49Z Lucas Werkmeister (WMDE) 10037 /* {{Deployment_day|date=2026-06-08}} */ move my backports before Audrey’s config change which requires them 2424097 wikitext text/x-wiki eacltpxq95gfyaj8ykr3nyhdu1rv5kk 2424115 2424097 2026-06-08T09:35:05Z ScheduleDeploymentBot 37566 Add [[gerrit:1298717]] to Monday, June 08 UTC afternoon backport window 2424115 wikitext text/x-wiki 7f5jn77c6hvup06gn2ovbzkbaesouq3 2424177 2424115 2026-06-08T10:32:49Z ScheduleDeploymentBot 37566 Add [[gerrit:1298726]] to Monday, June 08 UTC afternoon backport window 2424177 wikitext text/x-wiki 3qq1jec7c8vuexcu8yucyxryv786sn2 2424205 2424177 2026-06-08T11:34:08Z ScheduleDeploymentBot 37566 Add [[gerrit:1298734]] to Monday, June 08 UTC afternoon backport window 2424205 wikitext text/x-wiki hfxxr4d2q0yqtdbrorhxx9pkxsxdk8p 2424206 2424205 2026-06-08T11:34:58Z Neriah 36702 no 2424206 wikitext text/x-wiki p60ju9mv60olykynqvc4p4yk2mziil0 2424207 2424206 2026-06-08T11:35:08Z MFossati (WMF) 29357 /* {{Deployment_day|date=2026-06-08}} */ remove patch: the window is too busy 2424207 wikitext text/x-wiki pc36ge0vpjcbre6t5ty4w00g8xhqrzu 0 4874 2424027 2388327 2026-06-07T18:53:25Z TTWIDEE 51245 /* Groups */ Updated info, and fixed punctuation and style 2424027 wikitext text/x-wiki h7ornluzqmyyb3ftgv6sd65q7cc6cc5 498 6726 2424102 2423728 2026-06-08T09:13:20Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster toolsbeta upgrade to 1.32.13 (T427918) 2424102 wikitext text/x-wiki rx9mesiicaynm0mktb5j7pzdjo3igf7 2424103 2424102 2026-06-08T09:13:28Z Stashbot 7414 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=99) for cluster toolsbeta upgrade to 1.32.13 (T427918) 2424103 wikitext text/x-wiki pakx3oao7bc0j0vstxhiy583a9ghzfc 2424104 2424103 2026-06-08T09:14:00Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.run_tests 2424104 wikitext text/x-wiki 4b58wnr8qbexcuwes23tizdolb63q3m 2424105 2424104 2026-06-08T09:14:05Z Stashbot 7414 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99) 2424105 wikitext text/x-wiki pk76e6u6r9k56up8uw4r4s1q2a3zayr 2424112 2424105 2026-06-08T09:25:00Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster toolsbeta upgrade to 1.32.13 (T427918) 2424112 wikitext text/x-wiki hmv8a0mmh599dz75y95k80nhtwk1568 2424116 2424112 2026-06-08T09:39:34Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=0) for cluster toolsbeta upgrade to 1.32.13 (T427918) 2424116 wikitext text/x-wiki fijgkeqzym1kt36lnif3ksi8kqdeg5z 2424117 2424116 2026-06-08T09:40:35Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-10 from 1.31.14 to 1.32.13 (T427918) 2424117 wikitext text/x-wiki bj6k0w93ok7oxvqpi2lz51br6bmdz0m 2424128 2424117 2026-06-08T09:49:42Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-10 from 1.31.14 to 1.32.13 (T427918) 2424128 wikitext text/x-wiki kcngnhrd3rzlvk0q5eqy86ueqs5f5uz 2424129 2424128 2026-06-08T09:49:49Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-11 from 1.31.14 to 1.32.13 (T427918) 2424129 wikitext text/x-wiki tp7gfgqlxi6t0hv5yh6yulu3i1o4pb0 2424135 2424129 2026-06-08T09:55:27Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-11 from 1.31.14 to 1.32.13 (T427918) 2424135 wikitext text/x-wiki nravfmm83o8fzmkqoz1kapzfqd5scy7 2424136 2424135 2026-06-08T09:55:35Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-12 from 1.31.14 to 1.32.13 (T427918) 2424136 wikitext text/x-wiki on0e26jj1zw945gt8ik9e3qtmsjekh4 2424137 2424136 2026-06-08T09:57:56Z Stashbot 7414 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-k8s-control-12 from 1.31.14 to 1.32.13 (T427918) 2424137 wikitext text/x-wiki nl2eif9o5rdb0wnp7n0ew782meu4v8w 2424138 2424137 2026-06-08T09:59:51Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers for toolsbeta-test-k8s-worker-nfs-10, toolsbeta-test-k8s-worker-nfs-11, toolsbeta-test-k8s-worker-nfs-7, toolsbeta-test-k8s-worker-nfs-8, toolsbeta-test-k8s-worker-nfs-9, toolsbeta-test-k8s-worker-12, toolsbeta-test-k8s-worker-13 (T427918) 2424138 wikitext text/x-wiki 01brx3mavp6woqs425fftt5in1s5s45 2424140 2424138 2026-06-08T10:03:15Z Stashbot 7414 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers (exit_code=99) for toolsbeta-test-k8s-worker-nfs-10, toolsbeta-test-k8s-worker-nfs-11, toolsbeta-test-k8s-worker-nfs-7, toolsbeta-test-k8s-worker-nfs-8, toolsbeta-test-k8s-worker-nfs-9, toolsbeta-test-k8s-worker-12, toolsbeta-test-k8s-worker-13 (T427918) 2424140 wikitext text/x-wiki g384bwlh7jxyx5th3ofpx0ps2rjrj1i 2424141 2424140 2026-06-08T10:04:18Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers for toolsbeta-test-k8s-worker-nfs-10, toolsbeta-test-k8s-worker-nfs-11, toolsbeta-test-k8s-worker-nfs-7, toolsbeta-test-k8s-worker-nfs-8, toolsbeta-test-k8s-worker-nfs-9 (T427918) 2424141 wikitext text/x-wiki gwlqlr3d8c4yt9k318k1f2ek0866n6z 2424142 2424141 2026-06-08T10:04:55Z Stashbot 7414 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers (exit_code=99) for toolsbeta-test-k8s-worker-nfs-10, toolsbeta-test-k8s-worker-nfs-11, toolsbeta-test-k8s-worker-nfs-7, toolsbeta-test-k8s-worker-nfs-8, toolsbeta-test-k8s-worker-nfs-9 (T427918) 2424142 wikitext text/x-wiki mwhmj5wlfvd4fr8p0t1jd5ud72bom2h 2424148 2424142 2026-06-08T10:09:48Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers for toolsbeta-test-k8s-worker-nfs-10, toolsbeta-test-k8s-worker-nfs-11, toolsbeta-test-k8s-worker-nfs-7, toolsbeta-test-k8s-worker-nfs-8, toolsbeta-test-k8s-worker-nfs-9 (T427918) 2424148 wikitext text/x-wiki bztzub9a5hmlwhamt8rykqm09vff89m 2424155 2424148 2026-06-08T10:14:29Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers (exit_code=0) for toolsbeta-test-k8s-worker-nfs-10, toolsbeta-test-k8s-worker-nfs-11, toolsbeta-test-k8s-worker-nfs-7, toolsbeta-test-k8s-worker-nfs-8, toolsbeta-test-k8s-worker-nfs-9 (T427918) 2424155 wikitext text/x-wiki qrsyg0acx6rvu62zmu44qp6igk56f7l 2424162 2424155 2026-06-08T10:17:49Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-gateway-1 from 1.31.14 to 1.32.13 (T427918) 2424162 wikitext text/x-wiki 81bu12l3h43vprhacmclbs5dtbepv9n 2424167 2424162 2026-06-08T10:18:44Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-gateway-1 from 1.31.14 to 1.32.13 (T427918) 2424167 wikitext text/x-wiki 3d5linjllgla315lid4iw22k149nynw 2424168 2424167 2026-06-08T10:18:47Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-gateway-2 from 1.31.14 to 1.32.13 (T427918) 2424168 wikitext text/x-wiki hryayggnrz78vsmcodr2t4szhxobvcv 2424170 2424168 2026-06-08T10:19:32Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-gateway-2 from 1.31.14 to 1.32.13 (T427918) 2424170 wikitext text/x-wiki qvj6a7o4pllkcyoiqjuuzhevq4rc2ha 2424171 2424170 2026-06-08T10:19:35Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-gateway-3 from 1.31.14 to 1.32.13 (T427918) 2424171 wikitext text/x-wiki morrroqlkp5d8p0ejuhz4klt2zjr55j 2424173 2424171 2026-06-08T10:20:30Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-gateway-3 from 1.31.14 to 1.32.13 (T427918) 2424173 wikitext text/x-wiki 4qz2zs9d7kdg891p82n6s9ktdu2u624 2424174 2424173 2026-06-08T10:20:48Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade_bastions for toolsbeta-bastion-7.toolsbeta.eqiad1.wikimedia.cloud, toolsbeta-bastion-6.toolsbeta.eqiad1.wikimedia.cloud (T427918) 2424174 wikitext text/x-wiki 6sv213t39cir8hknmwwmxfp9ybzw8ta 2424175 2424174 2026-06-08T10:21:03Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade_bastions (exit_code=0) for toolsbeta-bastion-7.toolsbeta.eqiad1.wikimedia.cloud, toolsbeta-bastion-6.toolsbeta.eqiad1.wikimedia.cloud (T427918) 2424175 wikitext text/x-wiki k0bkvvrzhlqvdtf6wg0igl2psxiydzs 0 7919 2424022 2424011 2026-06-07T16:29:03Z Stashbot 7414 elukey: restart varnish-frontend on cp6014 2424022 wikitext text/x-wiki ptcj8a6t80e0ip1mza9aqngpff1qpre 2424023 2424022 2026-06-07T16:32:32Z Stashbot 7414 elukey: `elukey@cumin1003:~$ sudo cumin 'cp6* and not cp6014* and not cp6010*' "varnish-frontend-restart" -b 1` 2424023 wikitext text/x-wiki h77omz2n2nuzu1bv7ec7tpg7s3za3p1 2424031 2424023 2026-06-08T05:18:02Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2424031 wikitext text/x-wiki 2e7p2uet7wt1wa8ojc41btqib1yg10e 2424032 2424031 2026-06-08T05:18:23Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es2052: Upgrading es2052.codfw.wmnet 2424032 wikitext text/x-wiki oflhkn7902ybmsuyxqplmlpjtlfdn5k 2424033 2424032 2026-06-08T05:18:44Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool es2052: Upgrading es2052.codfw.wmnet 2424033 wikitext text/x-wiki 5uzjy3q2kzk80agbd411i3gtvw15i9p 2424034 2424033 2026-06-08T05:19:57Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es2052.codfw.wmnet with OS trixie 2424034 wikitext text/x-wiki 7pg6o98zgxhbn1m22oifjtdl8gbkq0f 2424035 2424034 2026-06-08T05:31:59Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Promote es1054 to es3 eqiad primary T428050', diff saved to https://phabricator.wikimedia.org/P93895 and previous config saved to /var/cache/conftool/dbconfig/20260608-053156-marostegui.json 2424035 wikitext text/x-wiki rcwowyjnuuvtw4hfn6advmgr0zzcbq3 2424037 2424035 2026-06-08T05:33:25Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2424037 wikitext text/x-wiki i608487dlz1mhsty1f91qfhnntxuxmk 2424038 2424037 2026-06-08T05:33:45Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es1051: Upgrading es1051.eqiad.wmnet 2424038 wikitext text/x-wiki 3edgeg1jaqeo6e0ae8bc13r66lnwgoa 2424039 2424038 2026-06-08T05:34:02Z Stashbot 7414 marostegui@dns1004: START - running authdns-update 2424039 wikitext text/x-wiki kmr56jwk9f9d8jce5stjc258j8tobu2 2424040 2424039 2026-06-08T05:35:32Z Stashbot 7414 marostegui@dns1004: END - running authdns-update 2424040 wikitext text/x-wiki o0bkjv7ty3rmjft1owfeucrfgvg7l3c 2424041 2424040 2026-06-08T05:35:47Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es2052.codfw.wmnet with reason: host reimage 2424041 wikitext text/x-wiki aqi711taddq9fdf8kqc9093di4jkw65 2424042 2424041 2026-06-08T05:39:12Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es2052.codfw.wmnet with reason: host reimage 2424042 wikitext text/x-wiki 7rn1rw9xofsco5n5qj3yih6x0moimmq 2424043 2424042 2026-06-08T05:44:47Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.depool (exit_code=99) depool es1051: Upgrading es1051.eqiad.wmnet 2424043 wikitext text/x-wiki laqdqnak8pcgyfkv5t9klkzjrutp14e 2424044 2424043 2026-06-08T05:54:43Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es2052.codfw.wmnet with OS trixie 2424044 wikitext text/x-wiki pj349y7n7opppmwml7xenq8n6e168eh 2424045 2424044 2026-06-08T05:58:53Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es1051.eqiad.wmnet with OS trixie 2424045 wikitext text/x-wiki nzem3o6ozmwhiqxiwgsoca7ffxmhwlb 2424046 2424045 2026-06-08T06:15:19Z Stashbot 7414 taavi@cumin1003: START - Cookbook sre.wikireplicas.add-wiki for database urwikisource (T415977) 2424046 wikitext text/x-wiki 6qbnqpe5cwbrk5k16631a9j313eexl3 2424047 2424046 2026-06-08T06:15:56Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es1051.eqiad.wmnet with reason: host reimage 2424047 wikitext text/x-wiki rgiei6u92mvpjehcuo11419xtd7afg7 2424048 2424047 2026-06-08T06:20:24Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es1051.eqiad.wmnet with reason: host reimage 2424048 wikitext text/x-wiki imnehbn2vv3utby9nelwozhmekcw91t 2424049 2424048 2026-06-08T06:36:56Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es1051.eqiad.wmnet with OS trixie 2424049 wikitext text/x-wiki rnj2y632yiu9qbxayli8ri5nrh588ud 2424062 2424049 2026-06-08T07:11:59Z Stashbot 7414 wmde-fisch@deploy1003: Started scap sync-world: Backport for [[gerrit:1297681|Global rollout - Sub-ref deployments to Group 0, Group 1 and frwiki (T425662)]] 2424062 wikitext text/x-wiki i0f1j1om2z1ffk3eyo2sdt588uc7s41 2424063 2424062 2026-06-08T07:12:32Z Stashbot 7414 elukey: upgrade exim4 packages on seaborgium for security upgrades 2424063 wikitext text/x-wiki ktbkej6qtnkrv1ouy58lxgas0l8hc0x 2424064 2424063 2026-06-08T07:12:35Z Stashbot 7414 taavi@cumin1003: END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) for database urwikisource (T415977) 2424064 wikitext text/x-wiki l9ntb430l3lis312c4xz7i4gs5z6y9x 2424065 2424064 2026-06-08T07:17:13Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.major-upgrade (exit_code=99) 2424065 wikitext text/x-wiki hgj9iuzzxc4qmqzknf0pixh5etgewhf 2424066 2424065 2026-06-08T07:17:25Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.major-upgrade (exit_code=99) 2424066 wikitext text/x-wiki qyssewzletp71zkl2qadn6ygm4oaxsz 2424067 2424066 2026-06-08T07:18:21Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es1051: repool after maintenance 2424067 wikitext text/x-wiki dr9g0y99oblprgwyb2x89uf9j0g2ed5 2424068 2424067 2026-06-08T07:18:29Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es2052: repool after upgrade 2424068 wikitext text/x-wiki s4mcfv0342aze69ktn6ppj6twyk3d1h 2424069 2424068 2026-06-08T07:21:51Z Stashbot 7414 elukey: upgrade sudo package on an-* hosts for T428384 2424069 wikitext text/x-wiki jue03l4b7sxltlet6xqz3ywe2f3cug6 2424070 2424069 2026-06-08T07:29:39Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch, lilients: Backport for [[gerrit:1297681|Global rollout - Sub-ref deployments to Group 0, Group 1 and frwiki (T425662)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2424070 wikitext text/x-wiki 32qotb8yydk26zdqg1vqvv4dpzwuswn 2424071 2424070 2026-06-08T07:32:23Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch, lilients: Continuing with deployment 2424071 wikitext text/x-wiki 6mgaf61qtj4wzi45ptfs92d358adosu 2424075 2424071 2026-06-08T07:44:50Z Stashbot 7414 wmde-fisch@deploy1003: Finished scap sync-world: Backport for [[gerrit:1297681|Global rollout - Sub-ref deployments to Group 0, Group 1 and frwiki (T425662)]] (duration: 32m 51s) 2424075 wikitext text/x-wiki t1u35tb322r3z6v6gf4s324pxtg8v93 2424076 2424075 2026-06-08T07:50:03Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.sanitize-wiki Managing sanitization for wikis urwikisource in section s5 2424076 wikitext text/x-wiki iv5cln8bn5wm21qw0rnz6ep3f9012vd 2424077 2424076 2026-06-08T07:50:05Z Stashbot 7414 fceratto@cumin1003: END (ERROR) - Cookbook sre.mysql.sanitize-wiki (exit_code=97) Managing sanitization for wikis urwikisource in section s5 2424077 wikitext text/x-wiki 9ykbwjktd7umm06nv284xwvuxe7nhen 2424078 2424077 2026-06-08T07:50:13Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.sanitize-wiki Checking sanitization for wikis urwikisource in section s5 2424078 wikitext text/x-wiki qrizap7jy3yfcsrnz1keuz8uj6ybe4h 2424079 2424078 2026-06-08T07:52:38Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.sanitize-wiki (exit_code=0) Checking sanitization for wikis urwikisource in section s5 2424079 wikitext text/x-wiki 94dui9peiylzdacl8iglf9v9azg309b 2424080 2424079 2026-06-08T07:53:04Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.sanitize-wiki Managing sanitization for wikis urwikisource in section s5 2424080 wikitext text/x-wiki 7czshpqsd6i6b3ckwy43kbwc4bnehb9 2424081 2424080 2026-06-08T07:53:27Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1217.eqiad.wmnet with reason: reimage 2424081 wikitext text/x-wiki 9a7lsf5nrxzmxz880xdog4g3dy1gx89 2424082 2424081 2026-06-08T07:55:44Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1217.eqiad.wmnet with OS trixie 2424082 wikitext text/x-wiki k0j75lfu6vnji1yhr4silsomzguvlp2 2424084 2424082 2026-06-08T08:03:24Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.sanitize-wiki (exit_code=0) Managing sanitization for wikis urwikisource in section s5 2424084 wikitext text/x-wiki nfa0o71951rmw28osf87081msn1qzat 2424085 2424084 2026-06-08T08:03:46Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es1051: repool after maintenance 2424085 wikitext text/x-wiki hze7c1yypf5coonq6x4c3ifph0s3c98 2424086 2424085 2026-06-08T08:03:55Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es2052: repool after upgrade 2424086 wikitext text/x-wiki 84mg7mebl0yb1ugh71oaw5l9bh7ppdo 2424087 2424086 2026-06-08T08:11:49Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1217.eqiad.wmnet with reason: host reimage 2424087 wikitext text/x-wiki r3a8ba16oseckr41iihkd7bk2od0ekp 2424088 2424087 2026-06-08T08:14:57Z Stashbot 7414 taavi@cumin1003: START - Cookbook sre.wikireplicas.add-wiki for database urwikisource (T415977) 2424088 wikitext text/x-wiki qqct0086d8lxzutho1zy8mvtzjrlzsx 2424089 2424088 2026-06-08T08:15:11Z Stashbot 7414 taavi@cumin1003: END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) for database urwikisource (T415977) 2424089 wikitext text/x-wiki aakq3qelby2936eij1lo5xmoxq5qxdl 2424092 2424089 2026-06-08T08:19:05Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1217.eqiad.wmnet with reason: host reimage 2424092 wikitext text/x-wiki pujsqphca67ucyspg24f2swn50j03d0 2424094 2424092 2026-06-08T08:41:45Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1217.eqiad.wmnet with OS trixie 2424094 wikitext text/x-wiki io2ovw908hg2hdpd97br4fxipxtfmoi 2424098 2424094 2026-06-08T09:05:50Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2424098 wikitext text/x-wiki jakmnv9d99mgyg6prxhidadbxrfgem6 2424099 2424098 2026-06-08T09:06:12Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es2043: Upgrading es2043.codfw.wmnet 2424099 wikitext text/x-wiki 4xx892uq7vcqmq7xu5zpo4zhtvcvrxd 2424100 2424099 2026-06-08T09:06:33Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool es2043: Upgrading es2043.codfw.wmnet 2424100 wikitext text/x-wiki apradl43ervpgqcphlf0m17gaqm88uy 2424101 2424100 2026-06-08T09:07:48Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es2043.codfw.wmnet with OS trixie 2424101 wikitext text/x-wiki j63n36sd563a8t7gii0ul4rpssex00o 2424106 2424101 2026-06-08T09:15:47Z Stashbot 7414 ozge@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: sync 2424106 wikitext text/x-wiki phmenn8u809xzo428tqspc92988kw5c 2424107 2424106 2026-06-08T09:15:53Z Stashbot 7414 ozge@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: sync 2424107 wikitext text/x-wiki 2pqdfns44wpx20yiqpqss79gglut0s7 2424108 2424107 2026-06-08T09:17:39Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.hosts.reboot-single for host gitlab1004.wikimedia.org 2424108 wikitext text/x-wiki b8v2i7fx66eb33i6bavx4yz0kicvs6u 2424111 2424108 2026-06-08T09:23:58Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es2043.codfw.wmnet with reason: host reimage 2424111 wikitext text/x-wiki hrsyebgkq9ik1svfwb7a40tic6nhun9 2424113 2424111 2026-06-08T09:27:14Z Stashbot 7414 jelto@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host gitlab1004.wikimedia.org 2424113 wikitext text/x-wiki blzgq998gr2t2nghfgw351udzruy1fa 2424114 2424113 2026-06-08T09:29:28Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es2043.codfw.wmnet with reason: host reimage 2424114 wikitext text/x-wiki d2mh0z2eqouza7kl8dwemkmb4ssywu2 2424118 2424114 2026-06-08T09:41:12Z Stashbot 7414 ozge@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: sync 2424118 wikitext text/x-wiki n9iis4vud0dyrnp4kepwgjus2f9m3qw 2424119 2424118 2026-06-08T09:41:26Z Stashbot 7414 ozge@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: sync 2424119 wikitext text/x-wiki s6b4i0r15m2jl25kff6yjwc0b71jfis 2424120 2424119 2026-06-08T09:41:44Z Stashbot 7414 ozge@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: sync 2424120 wikitext text/x-wiki d8v7r8k144zbajormzr4ku2ds49qhdf 2424121 2424120 2026-06-08T09:41:58Z Stashbot 7414 ozge@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: sync 2424121 wikitext text/x-wiki 5unr7r59wz5xnt3a558u6ktthf460cd 2424122 2424121 2026-06-08T09:42:03Z Stashbot 7414 ozge@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: sync 2424122 wikitext text/x-wiki n3o4mn3dcew47sjbykwnb5fybqitw8j 2424123 2424122 2026-06-08T09:42:06Z Stashbot 7414 ozge@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: sync 2424123 wikitext text/x-wiki ayfhxog721w2gvc85sf4as1n1kuvqm3 2424124 2424123 2026-06-08T09:44:05Z Stashbot 7414 mvolz@deploy1003: helmfile [staging] START helmfile.d/services/citoid: apply 2424124 wikitext text/x-wiki 9h5rp8u1d5nagzehbe2t431ach5yeue 2424125 2424124 2026-06-08T09:44:23Z Stashbot 7414 mvolz@deploy1003: helmfile [staging] DONE helmfile.d/services/citoid: apply 2424125 wikitext text/x-wiki b73f4oovpxi84cxfx9hjosdopuiyqdz 2424126 2424125 2026-06-08T09:46:49Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es2043.codfw.wmnet with OS trixie 2424126 wikitext text/x-wiki en72jo3tn4mbq698s37g0ox5e1j6gly 2424127 2424126 2026-06-08T09:49:05Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.major-upgrade (exit_code=99) 2424127 wikitext text/x-wiki sud4yyylqggzqkzhw1knt1mhxail0oa 2424130 2424127 2026-06-08T09:49:51Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es2043: repool after upgrade 2424130 wikitext text/x-wiki n35mikc3c8559u42d2l2zaws7uzzbvd 2424131 2424130 2026-06-08T09:49:53Z Stashbot 7414 mvolz@deploy1003: helmfile [eqiad] START helmfile.d/services/citoid: apply 2424131 wikitext text/x-wiki lcx2dqjinpw2vwmw3snbte0goyevsev 2424132 2424131 2026-06-08T09:50:22Z Stashbot 7414 mvolz@deploy1003: helmfile [eqiad] DONE helmfile.d/services/citoid: apply 2424132 wikitext text/x-wiki 8weusiuq7wai57rzkdvweifvqm08oih 2424133 2424132 2026-06-08T09:52:09Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] START helmfile.d/services/citoid: apply 2424133 wikitext text/x-wiki sprx58shr3pj13pkha2h68aiql1jssa 2424134 2424133 2026-06-08T09:52:39Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] DONE helmfile.d/services/citoid: apply 2424134 wikitext text/x-wiki 8m9zkgvpdlp5htb6tp8azatzcy9saem 2424143 2424134 2026-06-08T10:06:36Z Stashbot 7414 ihurbain@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2424143 wikitext text/x-wiki hqdvsxykgoneux7vtt1qirzc5ll43sf 2424144 2424143 2026-06-08T10:07:18Z Stashbot 7414 ihurbain@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2424144 wikitext text/x-wiki 6i2nt5t8t6y94o9jvwc0qgf0lnwogn1 2424145 2424144 2026-06-08T10:07:20Z Stashbot 7414 ihurbain@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2424145 wikitext text/x-wiki bgdmasufocgco2yne0en5anpg8bt2va 2424146 2424145 2026-06-08T10:08:00Z Stashbot 7414 ihurbain@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2424146 wikitext text/x-wiki 730vdanwpl66510l4lt1f8l9qs2iqee 2424147 2424146 2026-06-08T10:09:32Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.swift.convert-disks for host ms-be2062 2424147 wikitext text/x-wiki ab7zfz8n7g80gyquxo0fn7ilfrd2npp 2424149 2424147 2026-06-08T10:12:41Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.swift.convert-disks for host ms-be2063 2424149 wikitext text/x-wiki jsrhcufb3671p4oxrhd13qz4ofrnhnb 2424150 2424149 2026-06-08T10:13:35Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2424150 wikitext text/x-wiki jijjanneqehcw7zjv33lnnzxtigpl03 2424151 2424150 2026-06-08T10:13:55Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es1042: Upgrading es1042.eqiad.wmnet 2424151 wikitext text/x-wiki ggrvnwia4vphw53lbxbsrn5paihzy1n 2424152 2424151 2026-06-08T10:14:22Z Stashbot 7414 ihurbain@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2424152 wikitext text/x-wiki q14tgrp8gdvftzrgprtwhxlbvh569fx 2424153 2424152 2026-06-08T10:14:24Z Stashbot 7414 ihurbain@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2424153 wikitext text/x-wiki qcievi97es8eofncrj48u3gk9b48e97 2424154 2424153 2026-06-08T10:14:26Z Stashbot 7414 ihurbain@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2424154 wikitext text/x-wiki 780rvkp98ngg922qnuyca0m7hhhap1x 2424156 2424154 2026-06-08T10:14:32Z Stashbot 7414 ihurbain@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2424156 wikitext text/x-wiki sq0vbp65wcv61ggdzklw0tagyx8kj40 2424157 2424156 2026-06-08T10:15:15Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool es1042: Upgrading es1042.eqiad.wmnet 2424157 wikitext text/x-wiki 617pq8qcrgaygcy28gnz1ydyie970uq 2424158 2424157 2026-06-08T10:16:44Z Stashbot 7414 ihurbain@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2424158 wikitext text/x-wiki 0dzyyw2rweuo4qehz4i44ghzczulxjx 2424159 2424158 2026-06-08T10:16:46Z Stashbot 7414 ihurbain@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2424159 wikitext text/x-wiki jfyvd9b5r0m4eox21d254c7mttx4no5 2424160 2424159 2026-06-08T10:16:48Z Stashbot 7414 ihurbain@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2424160 wikitext text/x-wiki gqa0ci4a2e53ijrwoa4yz23jhsvjft9 2424161 2424160 2026-06-08T10:16:51Z Stashbot 7414 ihurbain@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2424161 wikitext text/x-wiki k95kw1fcm2grh53b9u5ez48wjmgawbt 2424163 2424161 2026-06-08T10:18:03Z Stashbot 7414 ihurbain@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2424163 wikitext text/x-wiki oob6wwnxrnxuusrl7rkfm6czaejza4z 2424164 2424163 2026-06-08T10:18:06Z Stashbot 7414 ihurbain@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2424164 wikitext text/x-wiki n0b61lodoja7lx859x5t8nu6q191kk2 2424165 2424164 2026-06-08T10:18:08Z Stashbot 7414 ihurbain@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2424165 wikitext text/x-wiki 0i93qblil81ax6lb7yuvbsmz3v0nxe6 2424166 2424165 2026-06-08T10:18:10Z Stashbot 7414 ihurbain@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2424166 wikitext text/x-wiki mo75m1m4tef6hykyql7ughdl9e5xy14 2424169 2424166 2026-06-08T10:19:01Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es1042.eqiad.wmnet with OS trixie 2424169 wikitext text/x-wiki 9jr82zwg3z9ie0mqxp1i50as0ynnuvi 2424176 2424169 2026-06-08T10:30:53Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1298721|GuessedThumbnailInfo: Also allow showing webp originals (T428202)]] 2424176 wikitext text/x-wiki jqiz5437c9281x48fvwy2t2tdqvb1br 2424178 2424176 2026-06-08T10:34:19Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es1042.eqiad.wmnet with reason: host reimage 2424178 wikitext text/x-wiki ckj2sedvr0swy4nwy0k1dumx3l21qgu 2424179 2424178 2026-06-08T10:34:40Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1298721|GuessedThumbnailInfo: Also allow showing webp originals (T428202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2424179 wikitext text/x-wiki drikq1xex2mp74yw4d7efhrsm7lp2ma 2424180 2424179 2026-06-08T10:35:03Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2160.codfw.wmnet with reason: Reboot 2424180 wikitext text/x-wiki mfllojfv1ps6223nbq1051kgx6zf6vh 2424181 2424180 2026-06-08T10:35:15Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es2043: repool after upgrade 2424181 wikitext text/x-wiki tg137j60qqi69x5sbf16nvxvhdnl2do 2424183 2424181 2026-06-08T10:36:37Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.remove-downtime for db2160.codfw.wmnet 2424183 wikitext text/x-wiki 8qr251ph9mpodt8paap5w7n1or7t6sz 2424184 2424183 2026-06-08T10:36:39Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for db2160.codfw.wmnet 2424184 wikitext text/x-wiki lx3f0yjkp1g36x4wyluut0kexy5qhps 2424185 2424184 2026-06-08T10:38:56Z Stashbot 7414 kamila@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-constraints: apply 2424185 wikitext text/x-wiki 08e7fzxtywzy4o2m6d814hhrxslqe6t 2424186 2424185 2026-06-08T10:39:17Z Stashbot 7414 kamila@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-constraints: apply 2424186 wikitext text/x-wiki bjq3tjs2ia7ub8g26o566tm99hvl7n5 2424187 2424186 2026-06-08T10:39:37Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2424187 wikitext text/x-wiki 3vb60sohqrshkpr599o7742hq9du31m 2424188 2424187 2026-06-08T10:39:52Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es1042.eqiad.wmnet with reason: host reimage 2424188 wikitext text/x-wiki lg59eas8nz414t5owpdhifjcm67z4ks 2424189 2424188 2026-06-08T10:47:33Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1298721|GuessedThumbnailInfo: Also allow showing webp originals (T428202)]] (duration: 16m 41s) 2424189 wikitext text/x-wiki izho85zib9av7gwjwkfy3kia9yyk1bo 2424190 2424189 2026-06-08T10:56:50Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es1042.eqiad.wmnet with OS trixie 2424190 wikitext text/x-wiki anhuhdq5knzpcb189y4mez2v3blxgw0 2424191 2424190 2026-06-08T10:58:27Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.major-upgrade (exit_code=99) 2424191 wikitext text/x-wiki hjpv57c258x8yufrez4macgy3vi2ikw 2424192 2424191 2026-06-08T10:58:39Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es1042: repool after maintenance 2424192 wikitext text/x-wiki 6rfl3v2lfun4nytboe86yhq0xa1pv1e 2424193 2424192 2026-06-08T11:02:18Z Stashbot 7414 mvernon@cumin2002: END (FAIL) - Cookbook sre.swift.convert-disks (exit_code=99) for host ms-be2063 2424193 wikitext text/x-wiki 9emysjhw95541jztc2w9eljr9b1xd5p 2424194 2424193 2026-06-08T11:02:21Z Stashbot 7414 mvernon@cumin2002: END (FAIL) - Cookbook sre.swift.convert-disks (exit_code=99) for host ms-be2062 2424194 wikitext text/x-wiki mkk2nig07cikv6hlwls4e8lckgyw4qv 2424195 2424194 2026-06-08T11:12:24Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1298728|SpecialMediaSearch: Prefer thumb steps over thumb limits (T424032)]] 2424195 wikitext text/x-wiki akks8gelrvwno4rvx62j5z8cp7kv7hg 2424196 2424195 2026-06-08T11:12:36Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-constraints: apply 2424196 wikitext text/x-wiki lywm22plgaxngdcuq2lpur5cwddpr18 2424197 2424196 2026-06-08T11:13:51Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-constraints: apply 2424197 wikitext text/x-wiki 84xce2x83gltz8ij7gf35bd1jfgogq6 2424198 2424197 2026-06-08T11:14:09Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1298728|SpecialMediaSearch: Prefer thumb steps over thumb limits (T424032)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2424198 wikitext text/x-wiki 0hi5b4ask37f991aejiycez97puncv8 2424199 2424198 2026-06-08T11:14:26Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-syntaxhighlight: apply 2424199 wikitext text/x-wiki oetn56rdxm84wfxc82jikdts52mefp3 2424200 2424199 2026-06-08T11:15:16Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-syntaxhighlight: apply 2424200 wikitext text/x-wiki lf3nwvh7zubmlt621zg4iacpz00vt9w 2424201 2424200 2026-06-08T11:18:29Z Stashbot 7414 Raine: progressively switching shellbox to bookworm (start) 2424201 wikitext text/x-wiki hmao4hpj5re5mek1o9dx881ehzy9euu 2424202 2424201 2026-06-08T11:25:44Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2424202 wikitext text/x-wiki cze74dqg4kesdviqgnr2tt75ilzydb7 2424204 2424202 2026-06-08T11:30:03Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1298728|SpecialMediaSearch: Prefer thumb steps over thumb limits (T424032)]] (duration: 17m 39s) 2424204 wikitext text/x-wiki dbykc5nau55q43u4o49xcpfv0ciceq7 2424208 2424204 2026-06-08T11:42:51Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reimage for host ms-be2062.codfw.wmnet with OS bullseye 2424208 wikitext text/x-wiki qprqcgs0u2ha1s9rqhmsoxx4du5h4bm 2424209 2424208 2026-06-08T11:43:10Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.move-vlan for host ms-be2062 2424209 wikitext text/x-wiki rs7a26pnopxjuhc92so9c7d4k9de57z 2424210 2424209 2026-06-08T11:43:20Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.dns.netbox 2424210 wikitext text/x-wiki i6tob51h6y54a7685kxcngcnpzapkoh 2424211 2424210 2026-06-08T11:44:01Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es1042: repool after maintenance 2424211 wikitext text/x-wiki dldjatto9u8ysnl2gnnbbnd6813t01j 2424212 2424211 2026-06-08T11:44:31Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2424212 wikitext text/x-wiki hutbkle8wfyxs5k2p7xq3wbn5xy34fo 2424213 2424212 2026-06-08T11:44:50Z Stashbot 7414 marostegui@cumin1003: END (ERROR) - Cookbook sre.mysql.major-upgrade (exit_code=97) 2424213 wikitext text/x-wiki j2hxywd2q3d0np59spl33rj2n6aqkgw 2424214 2424213 2026-06-08T11:44:56Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2424214 wikitext text/x-wiki 4ooq95955ekzs4w8zcp0t5x9p4hy087 2424215 2424214 2026-06-08T11:45:16Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es2041: Upgrading es2041.codfw.wmnet 2424215 wikitext text/x-wiki q9kxi20p0imo2m3mnhgaa02c5qd8ybr 2424216 2424215 2026-06-08T11:45:38Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool es2041: Upgrading es2041.codfw.wmnet 2424216 wikitext text/x-wiki om0mxl4ivacn7l1ad3tsjk450i0py64 2424217 2424216 2026-06-08T11:47:22Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es2041.codfw.wmnet with OS trixie 2424217 wikitext text/x-wiki o9uwkzjevsitkoip8n6vnwywq9hroyg 2424218 2424217 2026-06-08T11:49:20Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host ms-be2062 - mvernon@cumin2002" 2424218 wikitext text/x-wiki frisgxlpucikrpyfmjyph5hqvjgjow6 2424219 2424218 2026-06-08T11:49:26Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host ms-be2062 - mvernon@cumin2002" 2424219 wikitext text/x-wiki h69pi3fr97jkvcgrwjrd42vs7gi10b3 2424220 2424219 2026-06-08T11:49:28Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2424220 wikitext text/x-wiki 15qrltpz3l4j3vw049yi1281kvfj84q 2424221 2424220 2026-06-08T11:49:31Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.dns.wipe-cache ms-be2062.codfw.wmnet 123.0.192.10.in-addr.arpa 3.2.1.0.0.0.0.0.2.9.1.0.0.1.0.0.1.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2424221 wikitext text/x-wiki bz5u2cvm5yritagma99g2342ap62wbu 2424222 2424221 2026-06-08T11:49:33Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) ms-be2062.codfw.wmnet 123.0.192.10.in-addr.arpa 3.2.1.0.0.0.0.0.2.9.1.0.0.1.0.0.1.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2424222 wikitext text/x-wiki t29nh51n51exxm09i70ohrhqy7pvf5a 2424223 2424222 2026-06-08T11:49:42Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host ms-be2062 2424223 wikitext text/x-wiki 1d60rlcclj9l7mbvktk1w7ro2u0xa5a 2424224 2424223 2026-06-08T11:50:21Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ms-be2062 2424224 wikitext text/x-wiki fmxy9329w11pdzm382nsfssf7dle6mr 2424225 2424224 2026-06-08T11:50:24Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host ms-be2062 2424225 wikitext text/x-wiki gar44pl6tqtk3oobc8tyk4vf0p1ww2o 2424226 2424225 2026-06-08T11:50:46Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reimage for host ms-be2063.codfw.wmnet with OS bullseye 2424226 wikitext text/x-wiki cs8w3giw98crbwymry0vlpc3qrv9o8p 2424227 2424226 2026-06-08T11:51:07Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.move-vlan for host ms-be2063 2424227 wikitext text/x-wiki 5buy2zh1ldnv3dtyrvihce99i4q8u1f 2424228 2424227 2026-06-08T11:51:17Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.dns.netbox 2424228 wikitext text/x-wiki rho9jv899zelgpgoeh07nc3zjjayqlv 2424229 2424228 2026-06-08T11:56:50Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host ms-be2063 - mvernon@cumin2002" 2424229 wikitext text/x-wiki dg9y00hf3p8b5cgjzpu5pie9i9blvwk 2424230 2424229 2026-06-08T11:56:55Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host ms-be2063 - mvernon@cumin2002" 2424230 wikitext text/x-wiki 9vmtc1c4b916z707sdjc0m98efv7mx5 2424231 2424230 2026-06-08T11:56:57Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2424231 wikitext text/x-wiki sb0hu217bnnhn6v15gdnewzpccfepsa 2424232 2424231 2026-06-08T11:56:59Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.dns.wipe-cache ms-be2063.codfw.wmnet 52.16.192.10.in-addr.arpa 2.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2424232 wikitext text/x-wiki ie24vn5eq8fxlajv1tznyopv4vm0h3h 2424233 2424232 2026-06-08T11:57:01Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) ms-be2063.codfw.wmnet 52.16.192.10.in-addr.arpa 2.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2424233 wikitext text/x-wiki e6o2b6uh15fhgmadybpc8pgzlkcvwd9 2424234 2424233 2026-06-08T11:57:03Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host ms-be2063 2424234 wikitext text/x-wiki 7xrc7qf12pc7075udyjgy0wwvjq6gh6 2424235 2424234 2026-06-08T11:57:15Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ms-be2063 2424235 wikitext text/x-wiki ndx2ke48idi5k83rlg4a5oroabjpvhe 2424236 2424235 2026-06-08T11:57:17Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host ms-be2063 2424236 wikitext text/x-wiki 693lhi8hf1959u8iedz6kwzvkjart75 498 30942 2424024 2422743 2026-06-07T17:40:02Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.restart_openstack on deployment eqiad1 for all services 2424024 wikitext text/x-wiki a0p504rsu9wn2oiz26k6qrb0yk9rva6 2424025 2424024 2026-06-07T17:40:25Z Stashbot 7414 andrewbogott: wmcs.openstack.restart_openstack --cluster-name eqiad1 --all as step one in troubleshooting T428312 2424025 wikitext text/x-wiki t8rtu6dqjwdaf86gyb4mq4pkrwzh9s0 2424026 2424025 2026-06-07T17:55:28Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment eqiad1 for all services 2424026 wikitext text/x-wiki b7jvwa11sny60foeanr9rlg2crqrnbw 0 447862 2424182 2418718 2026-06-08T10:35:37Z FCeratto-WMF 39104 Use && 2424182 wikitext text/x-wiki 6jpnsim0w3cs9lder39a84df6taapcf 2424203 2424182 2026-06-08T11:25:53Z FCeratto-WMF 39104 Add task updater 2424203 wikitext text/x-wiki pu2r0p14hgisjbgk7jwkqoub0hn9h1t 0 449160 2424030 2424019 2026-06-08T00:01:38Z Dexbot 30554 Bot: Updating the report 2424030 wikitext text/x-wiki ku0z90lx3r9lcjxek22rrgkdg7odzj2 2424036 2424030 2026-06-08T05:32:18Z Dexbot 30554 Bot: Updating the report 2424036 wikitext text/x-wiki 97n3t1116arbnpzd9ekfruo188mssq0 116 453906 2424029 2424021 2026-06-07T23:03:14Z Gitlabaccountapprovalbot 37332 shadowthewuff was rejected. 2424029 wikitext text/x-wiki hj62rhkc0cocudm4bkpda1u5vj95wmc 3 456712 2424057 2268858 2026-06-08T06:46:45Z StrikerBot 8475 Welcome to Toolforge! 2424057 wikitext text/x-wiki a0vn6yqc5v387cz2e0zw85z77yv4b60 0 459290 2424139 2423948 2026-06-08T10:02:49Z JAllemandou (WMF) 37709 New section for today's deployment 2424139 wikitext text/x-wiki jo1ym2s10u0o1k49cjd735zg9l0ihe3 2424172 2424139 2026-06-08T10:20:14Z JAllemandou (WMF) 37709 /* Monday, June 8th, 2026 */ Add a new rinery source line about fixing MWH-revert bug 2424172 wikitext text/x-wiki spfrqxe4lih76i3rmsr97apuahk0mzd 116 460070 2424109 2421197 2026-06-08T09:23:28Z EtherpadBackupBot 54504 umlsisXFA4ql4IF1gi2- archived by Aristorkle 2424109 wikitext text/x-wiki k82hgf7tq62tlhfhs6a01dngl2vl6vf 2424110 2424109 2026-06-08T09:23:30Z EtherpadBackupBot 54504 umlsisXFA4ql4IF1gi2- archived by Aristorkle 2424110 wikitext text/x-wiki rk594vfrtbxsufikpyn3oadxfd511n6 3 460295 2424050 2026-06-08T06:42:13Z StrikerBot 8475 Welcome to Toolforge! 2424050 wikitext text/x-wiki lv1n9b57z6ur8locmjinph7hk2z8t61 3 460296 2424051 2026-06-08T06:42:45Z StrikerBot 8475 Welcome to Toolforge! 2424051 wikitext text/x-wiki 725u8hdwv2y4jecqvct2f23r08kpze0 3 460297 2424052 2026-06-08T06:43:29Z StrikerBot 8475 Welcome to Toolforge! 2424052 wikitext text/x-wiki pqwgyg5shwv6prhww67fun0hxg1qnye 3 460298 2424053 2026-06-08T06:43:45Z StrikerBot 8475 Welcome to Toolforge! 2424053 wikitext text/x-wiki g853zwr0v1fnyjqq57vhuee78ms7sky 3 460299 2424054 2026-06-08T06:44:17Z StrikerBot 8475 Welcome to Toolforge! 2424054 wikitext text/x-wiki tk8y6nbyqqmbs4vpmh5utabpqt95muu 3 460300 2424055 2026-06-08T06:44:45Z StrikerBot 8475 Welcome to Toolforge! 2424055 wikitext text/x-wiki bd22m6oib2g0dys579kw85auma8rx7v 3 460301 2424058 2026-06-08T06:47:39Z StrikerBot 8475 Welcome to Toolforge! 2424058 wikitext text/x-wiki cdsqm5w8969x8z5r3tf7503ibwj8kf3 3 460302 2424059 2026-06-08T06:48:07Z StrikerBot 8475 Welcome to Toolforge! 2424059 wikitext text/x-wiki epg90rlsueyao8zers0cyhgrd0sarz6 3 460303 2424060 2026-06-08T06:50:49Z StrikerBot 8475 Welcome to Toolforge! 2424060 wikitext text/x-wiki avpy2bznagnrskkkx534gkuvrisofe7 3 460304 2424061 2026-06-08T06:54:02Z StrikerBot 8475 Welcome to Toolforge! 2424061 wikitext text/x-wiki 3v4mg9s94ieppd2hyfy63rdx8shzpwv