Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.6 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 0 3387 2425840 2417418 2026-06-10T19:28:58Z DAdedoyin (WMF) 39150 /* Service Providers */ 2425840 wikitext text/x-wiki 6d02cx93a51hrads08ghnl0hvtjru4p 0 4108 2425510 2425255 2026-06-10T12:30:01Z ScheduleDeploymentBot 37566 Add [[gerrit:1297237]] to Wednesday, June 10 UTC afternoon backport window 2425510 wikitext text/x-wiki kgzvkn8pm27rh8lgj1dh2ma41g6jh1u 2425519 2425510 2026-06-10T12:38:19Z ScheduleDeploymentBot 37566 Add [[gerrit:1299516]] to Wednesday, June 10 UTC afternoon backport window 2425519 wikitext text/x-wiki ivkzd7zpgq9yxqklwzkrbt2nxjmzzzt 2425731 2425519 2026-06-10T16:39:52Z APaskulin (WMF) 14880 Fix typo 2425731 wikitext text/x-wiki pwmbs44u5ccawtyjii3x7w1tvy9itm4 2425733 2425731 2026-06-10T16:40:44Z ScheduleDeploymentBot 37566 Add [[gerrit:1300073]] to Wednesday, June 10 UTC late backport window 2425733 wikitext text/x-wiki 976ocmlupc4g622ck0s0owuejsv3nq6 2425821 2425733 2026-06-10T18:31:53Z ScheduleDeploymentBot 37566 Add [[gerrit:1300226]] to Wednesday, June 10 UTC late backport window 2425821 wikitext text/x-wiki 3dl6ljfmbvpmrjyj0i3l8n9mj7j4yv8 2425965 2425821 2026-06-11T05:00:05Z ScheduleDeploymentBot 37566 Add [[gerrit:1300301]] to Monday, June 15 UTC morning backport window 2425965 wikitext text/x-wiki t88xi3ke29pdh0qjnqfdg07k9sr2tvp 2426097 2425965 2026-06-11T10:10:39Z ScheduleDeploymentBot 37566 Add [[gerrit:1300736]] to Thursday, June 11 UTC afternoon backport window 2426097 wikitext text/x-wiki hp0u30hodcjzm2eiwa1lqx16pdfw49p 2426145 2426097 2026-06-11T11:14:52Z ScheduleDeploymentBot 37566 Add [[gerrit:1300731]] to Thursday, June 11 UTC afternoon backport window 2426145 wikitext text/x-wiki 2q5xem0qgsjbk29xsdpke3i6rvkcin4 0 6479 2426107 2425007 2026-06-11T10:30:27Z Effie Mouzeli (WMF) 12880 scrollbot update 2426107 wikitext text/x-wiki 3wufrg7fryhb0a8ro5lru0wuleks470 2426109 2426107 2026-06-11T10:31:24Z Effie Mouzeli (WMF) 12880 scrollbot update 2426109 wikitext text/x-wiki huk4v86oepa0u55oupk0itbglukwrvl 0 7919 2425479 2425478 2026-06-10T11:59:12Z Stashbot 7414 reedy@deploy1003: reedy: Backport for [[gerrit:1300104|Mandatory2FAChecker: Allow getGroupsRequiring2FA() to work on implicit groups (T420792)]], [[gerrit:1300102|Mandatory2FAChecker: Allow getGroupsRequiring2FA() to work on implicit groups (T420792)]], [[gerrit:1299643|wmf-config: Add $wmgOATHAuthRequire2FAForAll config (T420792)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes c 2425479 wikitext text/x-wiki sxkuahj9rkq1gewzly3zluukn4zp86a 2425480 2425479 2026-06-10T12:02:12Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1186.eqiad.wmnet with reason: host reimage 2425480 wikitext text/x-wiki j8nd0ehcn76kkcpkob468jzm9cvkb5d 2425481 2425480 2026-06-10T12:03:51Z Stashbot 7414 reedy@deploy1003: reedy: Continuing with deployment 2425481 wikitext text/x-wiki hg1satoo40hfjigttcwi57nzsvyklb5 2425482 2425481 2026-06-10T12:06:36Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1186.eqiad.wmnet with reason: host reimage 2425482 wikitext text/x-wiki iy54kyycxlibb7je2f0fts2r84eic40 2425483 2425482 2026-06-10T12:08:11Z Stashbot 7414 reedy@deploy1003: Finished scap sync-world: Backport for [[gerrit:1300104|Mandatory2FAChecker: Allow getGroupsRequiring2FA() to work on implicit groups (T420792)]], [[gerrit:1300102|Mandatory2FAChecker: Allow getGroupsRequiring2FA() to work on implicit groups (T420792)]], [[gerrit:1299643|wmf-config: Add $wmgOATHAuthRequire2FAForAll config (T420792)]] (duration: 11m 06s) 2425483 wikitext text/x-wiki 5icnthy72jy7p7cti6l8s113xjfj6qi 2425484 2425483 2026-06-10T12:12:01Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2170.codfw.wmnet with reason: host reimage 2425484 wikitext text/x-wiki sebzuk6d12byncsa6to98i9wqcezhhx 2425485 2425484 2026-06-10T12:12:58Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host rdb1014.eqiad.wmnet with OS trixie 2425485 wikitext text/x-wiki orl860ts3npdses7z8k83ghlmrmj987 2425486 2425485 2026-06-10T12:13:25Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.move-vlan for host rdb1014 2425486 wikitext text/x-wiki 163xf2lucd2geycs4u07jvjo9vdex6b 2425492 2425486 2026-06-10T12:16:17Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.dns.netbox 2425492 wikitext text/x-wiki 3stq4kvnsh43rl0q4h3ecehv5iues47 2425497 2425492 2026-06-10T12:19:24Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2170.codfw.wmnet with reason: host reimage 2425497 wikitext text/x-wiki dxz8lw5kcpvfy5ndhl2u2lnhiac54d3 2425498 2425497 2026-06-10T12:20:24Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti5004.eqsin.wmnet with reason: host reimage 2425498 wikitext text/x-wiki 1mf8tq3o4t66z4h036x4mj50cpqf68m 2425500 2425498 2026-06-10T12:21:16Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host rdb1014 - jiji@cumin1003" 2425500 wikitext text/x-wiki nffcftg0c6oeimy6swwimkokmd0iqfk 2425501 2425500 2026-06-10T12:21:21Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host rdb1014 - jiji@cumin1003" 2425501 wikitext text/x-wiki gwrv6xg2isfry89n7kyshvqj58g9e9e 2425502 2425501 2026-06-10T12:21:24Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2425502 wikitext text/x-wiki ef01i5fg7e244dn59qucb4w4yg6d3ce 2425503 2425502 2026-06-10T12:21:26Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.dns.wipe-cache rdb1014.eqiad.wmnet 42.48.64.10.in-addr.arpa 2.4.0.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2425503 wikitext text/x-wiki 2mbq9m286t2rt4tnhsa8scapgfrf3tg 2425504 2425503 2026-06-10T12:21:28Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) rdb1014.eqiad.wmnet 42.48.64.10.in-addr.arpa 2.4.0.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2425504 wikitext text/x-wiki 7kwhcwl5vhja4v56tyl6m04yl29s6it 2425505 2425504 2026-06-10T12:21:30Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host rdb1014 2425505 wikitext text/x-wiki fx3cee7883ye61b5wx55nalo4strvo1 2425506 2425505 2026-06-10T12:23:32Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1186.eqiad.wmnet with OS trixie 2425506 wikitext text/x-wiki tmn7i0l3af0byb3pax3kqcegud4sw2m 2425507 2425506 2026-06-10T12:24:07Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host rdb1014 2425507 wikitext text/x-wiki 618ehs079hbmn126kyrsaaddwx022nu 2425508 2425507 2026-06-10T12:24:10Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host rdb1014 2425508 wikitext text/x-wiki 8kb559ewv5btabsxhuprhhuazj8aget 2425509 2425508 2026-06-10T12:24:25Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ganeti5004.eqsin.wmnet with reason: host reimage 2425509 wikitext text/x-wiki h3fz2777xr8q6xzf0ljdodgez6mxc9i 2425512 2425509 2026-06-10T12:33:20Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1186: Migration of db1186.eqiad.wmnet completed 2425512 wikitext text/x-wiki 0r2pzu6lygwpo4hculaf31eun3rxdio 2425514 2425512 2026-06-10T12:36:53Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2170.codfw.wmnet with OS trixie 2425514 wikitext text/x-wiki 0kh0ztpjq33g1aje0xlj09cocz42ur3 2425524 2425514 2026-06-10T12:41:45Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on rdb1014.eqiad.wmnet with reason: host reimage 2425524 wikitext text/x-wiki i0qxxth81blu9w1hl6hekevzzyolgwh 2425526 2425524 2026-06-10T12:42:36Z Stashbot 7414 topranks: re-map DSCP AF41 from 'low' to 'normal' priority qos class on network T424640 2425526 wikitext text/x-wiki hrnndanlg02obhq65wu2dl5kuc1jtsp 2425527 2425526 2026-06-10T12:45:02Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on rdb1014.eqiad.wmnet with reason: host reimage 2425527 wikitext text/x-wiki 91fbnsl6j1rf2drhtzhqhx0cx524jif 2425528 2425527 2026-06-10T12:46:28Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-ipoid: apply 2425528 wikitext text/x-wiki lz1ff479e1ltxl71v68ze9yktjoixek 2425529 2425528 2026-06-10T12:46:32Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-ipoid: apply 2425529 wikitext text/x-wiki cpkgdrl2o5ja81nd4nx1jp68jdjlqzg 2425530 2425529 2026-06-10T12:46:42Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-ipoid: apply 2425530 wikitext text/x-wiki cfyv8cqwm4iuw7r3cok74s3a4lmxrhw 2425531 2425530 2026-06-10T12:46:46Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti5004.eqsin.wmnet with OS bookworm 2425531 wikitext text/x-wiki az8869o8fjnx17sr4zcbgqctfotuy1w 2425532 2425531 2026-06-10T12:46:50Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-ipoid: apply 2425532 wikitext text/x-wiki dcqcbadvx4r8xz3ovtdbdhmmrcbtbfp 2425533 2425532 2026-06-10T12:47:45Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db2170: Migration of db2170.codfw.wmnet completed 2425533 wikitext text/x-wiki im2ppcav8ytc9mzr6k2m9w7p8bkc636 2425543 2425533 2026-06-10T13:02:57Z Stashbot 7414 sbisson@deploy1003: Started scap sync-world: Backport for [[gerrit:1299676|Enable ULS v2 on group0 wikis]] 2425543 wikitext text/x-wiki lhlv78ldd8rg1nxovtvudcwnsoouo2b 2425544 2425543 2026-06-10T13:03:02Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host rdb1014.eqiad.wmnet with OS trixie 2425544 wikitext text/x-wiki 3dt1anykcb1efeu9hayfwzpqy0l5qrw 2425545 2425544 2026-06-10T13:05:06Z Stashbot 7414 sbisson@deploy1003: sbisson, abi: Backport for [[gerrit:1299676|Enable ULS v2 on group0 wikis]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2425545 wikitext text/x-wiki hhmo5p221ebnnsqvg5lwlwvpnwpxqw8 2425546 2425545 2026-06-10T13:10:40Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.cdn.roll-restart-reboot-tcp-proxy rolling restart_daemons on A:tcpproxy and A:tcpproxy 2425546 wikitext text/x-wiki no9jqp3q9058vbliuzha84q5f20imk2 2425547 2425546 2026-06-10T13:15:40Z Stashbot 7414 sbisson@deploy1003: sbisson, abi: Continuing with deployment 2425547 wikitext text/x-wiki ko3jb48kptalyqotv4zrr8ya8vrjwgk 2425548 2425547 2026-06-10T13:18:08Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-test: apply 2425548 wikitext text/x-wiki bnm7d1ihvoo3lde6bc58g1p86y84kdv 2425549 2425548 2026-06-10T13:18:13Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-test: apply 2425549 wikitext text/x-wiki m1lp2mb25flv1n8qmycnr226sjo2x6l 2425550 2425549 2026-06-10T13:18:19Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-test: apply 2425550 wikitext text/x-wiki 4qcf3p470stgt7rpohv9fe7anrqi0cd 2425551 2425550 2026-06-10T13:18:26Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-test: apply 2425551 wikitext text/x-wiki 7c32wa93ogmaaf0vjirt2phtivr8uek 2425552 2425551 2026-06-10T13:18:50Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1186: Migration of db1186.eqiad.wmnet completed 2425552 wikitext text/x-wiki k1e7g6mmecmi1tf4qfbt3yyhbkrwd8b 2425553 2425552 2026-06-10T13:18:52Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2425553 wikitext text/x-wiki folj7jad763kmc7hsd424pvjmeuv8nn 2425554 2425553 2026-06-10T13:19:35Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.dns.roll-restart-reboot-wikimedia-dns rolling restart_daemons on A:wikidough 2425554 wikitext text/x-wiki tih699nsduyeydcf8vs6lipbkwxalfq 2425555 2425554 2026-06-10T13:19:58Z Stashbot 7414 sbisson@deploy1003: Finished scap sync-world: Backport for [[gerrit:1299676|Enable ULS v2 on group0 wikis]] (duration: 17m 00s) 2425555 wikitext text/x-wiki 4p1oqgrgd1uwz6mfal1i74xaa46p61w 2425556 2425555 2026-06-10T13:20:18Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.cdn.roll-restart-reboot-hcaptcha-proxy rolling restart_daemons on A:hcaptcha-proxy and A:hcaptcha-proxy 2425556 wikitext text/x-wiki iypnnxoqzqih8so8g3al9nk62h4oz55 2425557 2425556 2026-06-10T13:20:41Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.dns.roll-restart-reboot-durum rolling restart_daemons on A:durum and A:durum 2425557 wikitext text/x-wiki 6h0sezi5hs2vek5mqesynukqegncn36 2425558 2425557 2026-06-10T13:22:18Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.dns.roll-restart rolling restart_daemons on A:dnsbox and (A:dnsbox) 2425558 wikitext text/x-wiki 6uxxm8gnauzpnfudedkuvqjsx8wmint 2425559 2425558 2026-06-10T13:25:07Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=no; selector: name=cp5018.eqsin.wmnet,service=(cdn|ats-be) 2425559 wikitext text/x-wiki k9ggjye3t79bf8aw4kxhn0vzjof6nb7 2425561 2425559 2026-06-10T13:28:03Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-reboot-tcp-proxy (exit_code=0) rolling restart_daemons on A:tcpproxy and A:tcpproxy 2425561 wikitext text/x-wiki 1iw95lfzbrql225jhivw3fo4a56rdvv 2425562 2425561 2026-06-10T13:28:38Z Stashbot 7414 sukhe@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on cp5018.eqsin.wmnet with reason: host down 2425562 wikitext text/x-wiki fncygpzhdom88ub2kd4d0weuhy2jswj 2425563 2425562 2026-06-10T13:29:40Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Started scap sync-world: Backport for [[gerrit:1297237|wmf-config: Update private subnets to include additions (T427393)]] 2425563 wikitext text/x-wiki 5182ie8w8unvi1nflhs2ig8buh3hce5 2425564 2425563 2026-06-10T13:31:38Z Stashbot 7414 eevans@deploy1003: helmfile [staging] START helmfile.d/services/data-gateway: apply 2425564 wikitext text/x-wiki kxkico3u7xcth7mfiosv8ej5zds5ehx 2425565 2425564 2026-06-10T13:31:44Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde, brett: Backport for [[gerrit:1297237|wmf-config: Update private subnets to include additions (T427393)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2425565 wikitext text/x-wiki 7auzuijx07s721tod6nro3z9oul9zip 2425566 2425565 2026-06-10T13:31:52Z Stashbot 7414 eevans@deploy1003: helmfile [staging] DONE helmfile.d/services/data-gateway: apply 2425566 wikitext text/x-wiki b4puqb36nym4on483ha2l0umiriwdoy 2425567 2425566 2026-06-10T13:32:32Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.dns.roll-restart-reboot-wikimedia-dns (exit_code=0) rolling restart_daemons on A:wikidough 2425567 wikitext text/x-wiki lvgluw3y1ab1nprfoyl5d82fao1ou67 2425568 2425567 2026-06-10T13:32:39Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde, brett: Continuing with deployment 2425568 wikitext text/x-wiki jz402mtfupajl0b07lkg5d6z7ny375z 2425569 2425568 2026-06-10T13:32:49Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2425569 wikitext text/x-wiki dpofd8pr78ouosxkj1lrw2auqu6ei1g 2425570 2425569 2026-06-10T13:33:09Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db1195: Upgrading db1195.eqiad.wmnet 2425570 wikitext text/x-wiki 68n7env657djwx1ve4s1w3y58u89rtf 2425571 2425570 2026-06-10T13:33:14Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2170: Migration of db2170.codfw.wmnet completed 2425571 wikitext text/x-wiki ml8gp4tbreh2q2v8pqfmkq5lw7lclxv 2425572 2425571 2026-06-10T13:33:17Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2425572 wikitext text/x-wiki azjr6p1n274z4v82e81rnsbwhjl681j 2425573 2425572 2026-06-10T13:33:36Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.dns.roll-restart-reboot-durum (exit_code=0) rolling restart_daemons on A:durum and A:durum 2425573 wikitext text/x-wiki dxct42epysco25j4uv553huoqx37wgy 2425574 2425573 2026-06-10T13:33:39Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-reboot-hcaptcha-proxy (exit_code=0) rolling restart_daemons on A:hcaptcha-proxy and A:hcaptcha-proxy 2425574 wikitext text/x-wiki mvm4sb664u7jj91abjzr72q77oyb864 2425575 2425574 2026-06-10T13:33:41Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1195: Upgrading db1195.eqiad.wmnet 2425575 wikitext text/x-wiki i2ebex8mkfuddpx4sn9b0vl3azl1ex0 2425576 2425575 2026-06-10T13:37:01Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Finished scap sync-world: Backport for [[gerrit:1297237|wmf-config: Update private subnets to include additions (T427393)]] (duration: 07m 20s) 2425576 wikitext text/x-wiki f0z5su281k5zciy8gj03wo90ydq24bb 2425577 2425576 2026-06-10T13:42:16Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db1195.eqiad.wmnet with OS trixie 2425577 wikitext text/x-wiki 8gxvt2s209ke1y9hijb1lxwk8bcrhu4 2425578 2425577 2026-06-10T13:42:59Z Stashbot 7414 Lucas_WMDE: UTC afternoon backport+config window done 2425578 wikitext text/x-wiki m1q24ndj7bth57q2wee1tpotlgfhwcb 2425581 2425578 2026-06-10T13:54:49Z Stashbot 7414 root@cumin2002: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Atieno out of all services on: 2436 hosts 2425581 wikitext text/x-wiki bnfhmzcxrobzmx4h02szh3wsx97lvmc 2425582 2425581 2026-06-10T13:56:29Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1195.eqiad.wmnet with reason: host reimage 2425582 wikitext text/x-wiki 8p3jbs5kdx39o898zovjnbhevoewulq 2425583 2425582 2026-06-10T13:58:40Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: extensions/Translate/scripts/ttmserver-export.php --wiki=default --ttmserver eqiad-test # T425377 populating production index on test cluster to estimate time required for the release 2425583 wikitext text/x-wiki 86gzemrzq3fihjwihku0ypdtvgrl7pu 2425584 2425583 2026-06-10T13:58:46Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2425584 wikitext text/x-wiki mxa0bqgys314udps5k4dnp2lr5nvrl6 2425585 2425584 2026-06-10T13:59:08Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db2173: Upgrading db2173.codfw.wmnet 2425585 wikitext text/x-wiki 8xmu7tzpk298dxztwn2tk4wi0fnq3ba 2425586 2425585 2026-06-10T13:59:29Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2173: Upgrading db2173.codfw.wmnet 2425586 wikitext text/x-wiki c9xggdy7gk6bzeixmr7yvnely03rb84 2425587 2425586 2026-06-10T14:00:24Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2425587 wikitext text/x-wiki h8zmhskuswimat25oehd9kmte9l016j 2425588 2425587 2026-06-10T14:00:45Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1195.eqiad.wmnet with reason: host reimage 2425588 wikitext text/x-wiki o4yv1k0iwot4fmuggqvc4vrsm0pfjcn 2425589 2425588 2026-06-10T14:01:21Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2425589 wikitext text/x-wiki 1eha5zqo2x7qtelea2wzeuvtirk3u27 2425590 2425589 2026-06-10T14:02:27Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db2173.codfw.wmnet with OS trixie 2425590 wikitext text/x-wiki 4a1nzvp8b4w12041zl16z6yn2v94ro1 2425591 2425590 2026-06-10T14:05:41Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-product: apply 2425591 wikitext text/x-wiki r93vnrc6wbg6ohgfxgi6r7uv5rnpor3 2425592 2425591 2026-06-10T14:06:47Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-product: apply 2425592 wikitext text/x-wiki 7b1f676776ujw4e3idyphyvbhou56ay 2425593 2425592 2026-06-10T14:07:16Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply 2425593 wikitext text/x-wiki 1olc5aii00nhdkeahb05rz7fszjqzyc 2425594 2425593 2026-06-10T14:07:42Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply 2425594 wikitext text/x-wiki 9t5rfmg9tyekhckbe0qgbrkcb2e9f1a 2425595 2425594 2026-06-10T14:07:59Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2425595 wikitext text/x-wiki b18m1n9kcrxinjvxpkn5q1zwm7qxzc6 2425596 2425595 2026-06-10T14:08:03Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-fr-tech: apply 2425596 wikitext text/x-wiki dxcx1hrui14nk09vhmntql5voql4q2i 2425597 2425596 2026-06-10T14:08:51Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2425597 wikitext text/x-wiki titbfux0lffjvw426mtm9vcm28xn9fz 2425598 2425597 2026-06-10T14:09:04Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-fr-tech: apply 2425598 wikitext text/x-wiki mi6qs17l11n95lzp3j15ux74evmjcxi 2425599 2425598 2026-06-10T14:09:48Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2425599 wikitext text/x-wiki 4jjojkvtjjqf6408jrc71mx5g5e0bhr 2425600 2425599 2026-06-10T14:10:30Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-main: apply 2425600 wikitext text/x-wiki ntb02umobw5uuitcq8wpyqsvez3ifaq 2425601 2425600 2026-06-10T14:11:02Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-main: apply 2425601 wikitext text/x-wiki nrko1pai3qurhz0no21tzhwp76ae4ch 2425602 2425601 2026-06-10T14:11:23Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-ml: apply 2425602 wikitext text/x-wiki 2wh0xwbdaln6e0djooydxkqomq2vy22 2425603 2425602 2026-06-10T14:12:18Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-ml: apply 2425603 wikitext text/x-wiki a0rwi9hjkshmle2eiqzvctckrp9fdwv 2425604 2425603 2026-06-10T14:13:02Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-platform-eng: apply 2425604 wikitext text/x-wiki 2tcuzbg2laudka3ecxigrx4tvo0cy5y 2425605 2425604 2026-06-10T14:13:04Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2425605 wikitext text/x-wiki kxdt71mfa74eaqg8vpusau4py4lwhyu 2425606 2425605 2026-06-10T14:13:08Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2425606 wikitext text/x-wiki p9r06undmw6h2u2j0frkam8f0wcn13z 2425608 2425606 2026-06-10T14:13:36Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-platform-eng: apply 2425608 wikitext text/x-wiki a69bs6hohnf39dushrle6297vlvz9gu 2425610 2425608 2026-06-10T14:14:13Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-research: apply 2425610 wikitext text/x-wiki r4kggeacotheodaemrdchaosd1mkkxf 2425611 2425610 2026-06-10T14:14:35Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-research: apply 2425611 wikitext text/x-wiki be5ljv8jen5jhh5a7o3zs2uf84u0b7d 2425612 2425611 2026-06-10T14:15:01Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-search: apply 2425612 wikitext text/x-wiki a7fh8cq2xsb1fvv89gauw5pp8lgvcp2 2425613 2425612 2026-06-10T14:15:34Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-search: apply 2425613 wikitext text/x-wiki a8hxmtp6ypoqio217c8xbee7u1m5et2 2425614 2425613 2026-06-10T14:15:38Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2425614 wikitext text/x-wiki 41s5vfnsnhxe90u2r1ka46ufd4s8wz7 2425615 2425614 2026-06-10T14:16:23Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-sre: apply 2425615 wikitext text/x-wiki 3tggia0t9fljp90tpm6w47c1ngscuu6 2425616 2425615 2026-06-10T14:17:01Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-sre: apply 2425616 wikitext text/x-wiki lracbv8r7hlg1gzyrh4nt70cg81qf5y 2425617 2425616 2026-06-10T14:17:23Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wikidata: apply 2425617 wikitext text/x-wiki 7gg597oqf2w84sd9pz2mhp3prfn54p8 2425618 2425617 2026-06-10T14:17:48Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wikidata: apply 2425618 wikitext text/x-wiki dd4ktainvwe7e5ggtw2ii5wua9a6rxk 2425619 2425618 2026-06-10T14:17:55Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wmde: apply 2425619 wikitext text/x-wiki qcvr1fztcad4d0a3pwojejyqpmqs4i2 2425620 2425619 2026-06-10T14:18:05Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1195.eqiad.wmnet with OS trixie 2425620 wikitext text/x-wiki r5k69jxpve8j5ge7qidim2v86wbwtlu 2425621 2425620 2026-06-10T14:18:25Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2425621 wikitext text/x-wiki 5sephi06nd482d1ymhwzksqo0phmmv5 2425622 2425621 2026-06-10T14:18:34Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wmde: apply 2425622 wikitext text/x-wiki 7756wmn1ald1oppl17u4jd3mrnbagzo 2425623 2425622 2026-06-10T14:18:43Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2425623 wikitext text/x-wiki 8hzirwqf65awfhxauybrkeumrsg0nmr 2425624 2425623 2026-06-10T14:18:52Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2425624 wikitext text/x-wiki csdm6bppwgp19pbu34j47uy23r1f37k 2425625 2425624 2026-06-10T14:19:37Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2425625 wikitext text/x-wiki 8zdydslg34ewhctt0ppait2suo84vgs 2425626 2425625 2026-06-10T14:19:43Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2425626 wikitext text/x-wiki 3ktweixqwca6idvv8pvaq30ktf9wp33 2425627 2425626 2026-06-10T14:20:29Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2425627 wikitext text/x-wiki 6lw8medrcybknasse5d89bazsun1qrn 2425628 2425627 2026-06-10T14:20:34Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2173.codfw.wmnet with reason: host reimage 2425628 wikitext text/x-wiki 4ubhqfhastn1j3oily91kx4rbrg5ecv 2425629 2425628 2026-06-10T14:20:53Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.dns.roll-restart (exit_code=0) rolling restart_daemons on A:dnsbox and (A:dnsbox) 2425629 wikitext text/x-wiki hi81odlelpwyope57a9xkirdy9802f8 2425630 2425629 2026-06-10T14:21:02Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/ratelimit: apply 2425630 wikitext text/x-wiki 88as9rqwxaexrht1wdso76xuv83i86l 2425632 2425630 2026-06-10T14:22:47Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/ratelimit: apply 2425632 wikitext text/x-wiki 1vf20u62gdj2x4yeif229xqftndtuxs 2425633 2425632 2026-06-10T14:22:52Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/ratelimit: apply 2425633 wikitext text/x-wiki kww3073u8sdpsf9p5b251e4l2denz8e 2425634 2425633 2026-06-10T14:23:13Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/ratelimit: apply 2425634 wikitext text/x-wiki eg2s75osw9310qbwztfbjz46lj2x07f 2425635 2425634 2026-06-10T14:24:30Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2173.codfw.wmnet with reason: host reimage 2425635 wikitext text/x-wiki 5lsm15ex9gjyqk9x1qec0w371iyb8vx 2425636 2425635 2026-06-10T14:24:57Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist translate extensions/Translate/scripts/ttmserver-export.php --ttmserver eqiad-test # T425377 populating ttmserver index on test cluster to estimate time required for the release, now with dblist translate 2425636 wikitext text/x-wiki j797omo3134gpuyhnh2ktxxmwfgxk5s 2425637 2425636 2026-06-10T14:26:21Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/ratelimit: apply 2425637 wikitext text/x-wiki 6p2wrc81ozydo8cak16k33ojzvpawzs 2425638 2425637 2026-06-10T14:26:39Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/ratelimit: apply 2425638 wikitext text/x-wiki frgoxtrrgwazz4fltivf3oruonvgnoc 2425639 2425638 2026-06-10T14:27:10Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host dse-k8s-worker1009.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2425639 wikitext text/x-wiki 6dns506ymgbwghnvwd5owtib4p06emw 2425640 2425639 2026-06-10T14:28:17Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.deploy.python-code homer to cumin[2002-2003].codfw.wmnet,cumin1003.eqiad.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - cmooney@cumin1003 2425640 wikitext text/x-wiki 9qk2i8zadm84e7aq0unk60ztwzumeal 2425641 2425640 2026-06-10T14:29:37Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1195: Migration of db1195.eqiad.wmnet completed 2425641 wikitext text/x-wiki qusc624cd6p4klkpjlne9mjx3k9if8q 2425642 2425641 2026-06-10T14:33:11Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-worker1009.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2425642 wikitext text/x-wiki jrlxrp6w0izg8zvo3gfmy02it1tcbhu 2425643 2425642 2026-06-10T14:34:23Z Stashbot 7414 cmooney@cumin1003: END (FAIL) - Cookbook sre.deploy.python-code (exit_code=99) homer to cumin[2002-2003].codfw.wmnet,cumin1003.eqiad.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - cmooney@cumin1003 2425643 wikitext text/x-wiki ie7asd0fdss1wjpx1pcezotj2nz7800 2425644 2425643 2026-06-10T14:34:26Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.deploy.python-code homer to cumin[2002-2003].codfw.wmnet,cumin1003.eqiad.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - cmooney@cumin1003 2425644 wikitext text/x-wiki dc1xgnaeknkasyx43t6fmpqnyx9s9ic 2425645 2425644 2026-06-10T14:38:02Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.dns.roll-restart-ntp rolling restart_daemons on A:dnsbox and (A:dnsbox) 2425645 wikitext text/x-wiki iioq6caxo5lajff87i8lbodf1lkapyw 2425646 2425645 2026-06-10T14:38:33Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Started scap sync-world: Backport for [[gerrit:1299614|Add my public key to mediawiki.org/keys (T423267)]] 2425646 wikitext text/x-wiki lnanl5vsjwqu123ukrt25rp84gays4h 2425647 2425646 2026-06-10T14:40:15Z Stashbot 7414 cmooney@cumin1003: END (FAIL) - Cookbook sre.deploy.python-code (exit_code=99) homer to cumin[2002-2003].codfw.wmnet,cumin1003.eqiad.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - cmooney@cumin1003 2425647 wikitext text/x-wiki j4reveio4ecb89enkunjpfsu1ecy4ov 2425648 2425647 2026-06-10T14:40:33Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.deploy.python-code homer to cumin[2002-2003].codfw.wmnet,cumin1003.eqiad.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - cmooney@cumin1003 2425648 wikitext text/x-wiki g09wobv5rvw10iyrb0hm92ry7qdwitn 2425649 2425648 2026-06-10T14:40:39Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde, matmarex: Backport for [[gerrit:1299614|Add my public key to mediawiki.org/keys (T423267)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2425649 wikitext text/x-wiki rp1hnse47b5d18gyr80ebp5ezlos2jc 2425650 2425649 2026-06-10T14:41:34Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2173.codfw.wmnet with OS trixie 2425650 wikitext text/x-wiki n3gdpyvg0nw4lh7hdmmyd04bj95q479 2425651 2425650 2026-06-10T14:42:48Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde, matmarex: Continuing with deployment 2425651 wikitext text/x-wiki g62e8hglemso2opj76vh6sfeb7d8mw3 2425652 2425651 2026-06-10T14:46:15Z Stashbot 7414 cmooney@cumin1003: END (FAIL) - Cookbook sre.deploy.python-code (exit_code=99) homer to cumin[2002-2003].codfw.wmnet,cumin1003.eqiad.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - cmooney@cumin1003 2425652 wikitext text/x-wiki f5124hyt1xn6kg9kcgy9yl222d70tyk 2425653 2425652 2026-06-10T14:47:06Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Finished scap sync-world: Backport for [[gerrit:1299614|Add my public key to mediawiki.org/keys (T423267)]] (duration: 08m 33s) 2425653 wikitext text/x-wiki gifape8vwh4preo63xtmgkcugva9qmj 2425654 2425653 2026-06-10T14:48:35Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.deploy.python-code homer to cumin2002.codfw.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - ayounsi@cumin1003 2425654 wikitext text/x-wiki ix9mlr6yp3oq7umne185gfnc77mfxkf 2425655 2425654 2026-06-10T14:49:23Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2002.codfw.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - ayounsi@cumin1003 2425655 wikitext text/x-wiki cpncky19ra0jqf971om7hyaclr8q5an 2425656 2425655 2026-06-10T14:50:27Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.deploy.python-code homer to cumin2003.codfw.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - ayounsi@cumin1003 2425656 wikitext text/x-wiki t46usb2wkb3afa01feamc1vcgvd2s64 2425657 2425656 2026-06-10T14:50:40Z Stashbot 7414 ayounsi@cumin1003: END (FAIL) - Cookbook sre.deploy.python-code (exit_code=99) homer to cumin2003.codfw.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - ayounsi@cumin1003 2425657 wikitext text/x-wiki f0cz3kapq3gty3t75ucf4kihkvjtv58 2425658 2425657 2026-06-10T14:53:04Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db2173: Migration of db2173.codfw.wmnet completed 2425658 wikitext text/x-wiki q1twb6xiywvqsyzp1vd8wyqb3c48jvi 2425659 2425658 2026-06-10T14:54:17Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist translate extensions/Translate/scripts/ttmserver-export.php --ttmserver eqiad-test # T425377 populating ttmserver index on test cluster to estimate time required for the release, now with correct schema 2425659 wikitext text/x-wiki 08w479wuwn4czee7umrbx4wed3o5263 2425660 2425659 2026-06-10T14:55:26Z Stashbot 7414 Lucas_WMDE: lucaswerkmeister-wmde@deploy1003 $ printf 'https://www.mediawiki.org/keys/%s\n' '' 'keys.txt' 'keys.html' | mwscript-k8s --attach --comment=T423267 purgeList mediawikiwiki 2425660 wikitext text/x-wiki 8a2kl9us9o96gbcda1sihxgvjvso381 2425662 2425660 2026-06-10T14:58:36Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.deploy.python-code homer to cumin2002.codfw.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - cmooney@cumin1003 2425662 wikitext text/x-wiki 6zi7zzl2noywvv83jdk8lmpoy72l6av 2425664 2425662 2026-06-10T14:59:25Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Started scap sync-world: Backport for [[gerrit:1300169|Fix snak value display for rtl languages (T360854)]], [[gerrit:1300168|Fix snak value display for rtl languages (T360854)]] 2425664 wikitext text/x-wiki idlhdsxqpwye06ixbzrn3ldu8cq33ci 2425665 2425664 2026-06-10T14:59:28Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2002.codfw.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - cmooney@cumin1003 2425665 wikitext text/x-wiki j89i65f2336cpz504nj6fv4aod1sq0r 2425672 2425665 2026-06-10T15:01:42Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde: Backport for [[gerrit:1300169|Fix snak value display for rtl languages (T360854)]], [[gerrit:1300168|Fix snak value display for rtl languages (T360854)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2425672 wikitext text/x-wiki q95p2k48e3mss41djzrnqp6cvyn1fpl 2425673 2425672 2026-06-10T15:03:43Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: lucaswerkmeister-wmde: Continuing with deployment 2425673 wikitext text/x-wiki krur9m4hkrz2bnoclkvhhjho0mitj8s 2425675 2425673 2026-06-10T15:08:05Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Finished scap sync-world: Backport for [[gerrit:1300169|Fix snak value display for rtl languages (T360854)]], [[gerrit:1300168|Fix snak value display for rtl languages (T360854)]] (duration: 08m 39s) 2425675 wikitext text/x-wiki qk8intqfdtxivewfj4829nqt0zbfk99 2425676 2425675 2026-06-10T15:11:21Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.deploy.python-code homer to cumin1003.codfw.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - cmooney@cumin1003 2425676 wikitext text/x-wiki 8ve1me1o24p06hxhi2urt0kfyueqn8l 2425677 2425676 2026-06-10T15:11:38Z Stashbot 7414 cmooney@cumin1003: END (FAIL) - Cookbook sre.deploy.python-code (exit_code=99) homer to cumin1003.codfw.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - cmooney@cumin1003 2425677 wikitext text/x-wiki 0g9nrcf50jr63j4dddba1cl0hcvcfz7 2425678 2425677 2026-06-10T15:11:42Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.deploy.python-code homer to cumin1003.eqiad.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - cmooney@cumin1003 2425678 wikitext text/x-wiki 2a2yp4bc6emdew1897v23a5l4j8x6zt 2425679 2425678 2026-06-10T15:12:32Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin1003.eqiad.wmnet with reason: add new eqsin vlans as legacy temp workaround in wmf-plugin.py - cmooney@cumin1003 2425679 wikitext text/x-wiki 5kd0pui0aasirlnafecvuxqgrxtb3oa 2425680 2425679 2026-06-10T15:15:07Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1195: Migration of db1195.eqiad.wmnet completed 2425680 wikitext text/x-wiki lnrkk0719ykaknghyo0n7k471iqb6u7 2425681 2425680 2026-06-10T15:15:10Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2425681 wikitext text/x-wiki h4zieixvwkbm625xebpgm3n0b5kmh4i 2425682 2425681 2026-06-10T15:15:34Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.dns.netbox 2425682 wikitext text/x-wiki c3n55hrld35hvx92ccd7p65ojyzrnpm 2425683 2425682 2026-06-10T15:18:33Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2425683 wikitext text/x-wiki 3rqjbw99gkjywaxb4g1ssnnnc56h0n8 2425684 2425683 2026-06-10T15:18:36Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.dns.wipe-cache cp5018.eqsin.wmnet 18.0.132.10.in-addr.arpa 8.1.0.0.0.0.0.0.2.3.1.0.0.1.0.0.1.0.1.0.0.0.5.e.2.f.d.0.1.0.0.2.ip6.arpa on all recursors 2425684 wikitext text/x-wiki s6vxwiq2w8ji3ycnxe9x0uv1g5fp5ga 2425685 2425684 2026-06-10T15:18:38Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) cp5018.eqsin.wmnet 18.0.132.10.in-addr.arpa 8.1.0.0.0.0.0.0.2.3.1.0.0.1.0.0.1.0.1.0.0.0.5.e.2.f.d.0.1.0.0.2.ip6.arpa on all recursors 2425685 wikitext text/x-wiki mq24q59herdftxz3zzjnuc0mtbs0n10 2425686 2425685 2026-06-10T15:18:41Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host cp5018 2425686 wikitext text/x-wiki nofajk1vaffrc6shrre5lcnmol08eil 2425687 2425686 2026-06-10T15:18:55Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.dns.netbox 2425687 wikitext text/x-wiki 3nrgiktkbgqqpraxhm1ifbwa7lyfyky 2425688 2425687 2026-06-10T15:19:23Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cp5018 2425688 wikitext text/x-wiki 14agz9pg452hjh1qn0mff87rqauldaw 2425689 2425688 2026-06-10T15:19:26Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host cp5018 2425689 wikitext text/x-wiki kzmb8o3xo22fo14rd6z6scce0z887tg 2425690 2425689 2026-06-10T15:20:43Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist translate extensions/Translate/scripts/ttmserver-export.php --ttmserver eqiad-test # T425377 populating ttmserver index on test cluster to estimate time required for the release 2425690 wikitext text/x-wiki q79j9rkf219bexar4tddwmdmgvsvtwl 2425691 2425690 2026-06-10T15:21:56Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2425691 wikitext text/x-wiki dyax2vsgwuxxxn0vuiqh8ii1oh6arsk 2425692 2425691 2026-06-10T15:23:07Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host dse-k8s-worker1009 2425692 wikitext text/x-wiki jdlmgpi71qsc6rc1rvpc87n5w58yv2a 2425693 2425692 2026-06-10T15:24:06Z Stashbot 7414 root@cumin2002: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Harroyo-wmf out of all services on: 2436 hosts 2425693 wikitext text/x-wiki cd11c7qj6h87h17mz4zjvv0bdfzn4b1 2425694 2425693 2026-06-10T15:24:08Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host dse-k8s-worker1009 2425694 wikitext text/x-wiki f784wsn6vw137ymzjfvnampnosdaeoq 2425695 2425694 2026-06-10T15:24:34Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host dse-k8s-worker1009.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2425695 wikitext text/x-wiki 9c0fl5vs07w8211984sakdhp6nzsnd2 2425697 2425695 2026-06-10T15:25:34Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-worker1009.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2425697 wikitext text/x-wiki b80fcc93po0wpzdyz3xjy5yvl6hu8d5 2425698 2425697 2026-06-10T15:28:02Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2425698 wikitext text/x-wiki 6agqeyzexmj9nz7146owozdxl1i0q37 2425699 2425698 2026-06-10T15:28:19Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db1206: Upgrading db1206.eqiad.wmnet 2425699 wikitext text/x-wiki nnmwtk9bdso3tcbpp3sr30xiavushq3 2425700 2425699 2026-06-10T15:28:59Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1206: Upgrading db1206.eqiad.wmnet 2425700 wikitext text/x-wiki htqt48fuv6ms6y6vv6ez1db7toqmjiu 2425701 2425700 2026-06-10T15:30:34Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db1206.eqiad.wmnet with OS trixie 2425701 wikitext text/x-wiki rr0ery8br2vr99abymx3dn8y2nhij5y 2425702 2425701 2026-06-10T15:33:32Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist mwscript.dblist extensions/Translate/scripts/ttmserver-export.php --ttmserver eqiad-test # T425377 populating ttmserver index on test cluster to estimate time required for the release (dblist: https://phabricator.wikimedia.org/P94013) 2425702 wikitext text/x-wiki rgokx78tzg4f4h3cqme27fxok6cb6m9 2425703 2425702 2026-06-10T15:35:00Z Stashbot 7414 topranks: drain traffic through cr2-drmrs to reset pic0 2425703 wikitext text/x-wiki hhulkity7sxgabbq7jznn431tnkixfz 2425704 2425703 2026-06-10T15:38:34Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2173: Migration of db2173.codfw.wmnet completed 2425704 wikitext text/x-wiki 6gzhxloj3uszwu72zrqhovqh23ksple 2425705 2425704 2026-06-10T15:38:37Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2425705 wikitext text/x-wiki 32fwkba2no8zbrnlcsbkmku59olceyz 2425706 2425705 2026-06-10T15:42:30Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.dns.admin DNS admin: depool drmrs [reason: no reason specified, no task ID specified] 2425706 wikitext text/x-wiki 2rqanjaokqx0i4bno1utvwjphmh3qdh 2425707 2425706 2026-06-10T15:43:41Z Stashbot 7414 sukhe@cumin1003: END (FAIL) - Cookbook sre.dns.admin (exit_code=99) DNS admin: depool drmrs [reason: no reason specified, no task ID specified] 2425707 wikitext text/x-wiki 7z3npm1ew5hzagh752u8bavod0w2wfx 2425708 2425707 2026-06-10T15:45:42Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1206.eqiad.wmnet with reason: host reimage 2425708 wikitext text/x-wiki f1df0mbpew5ywloml8ihy2mtb85jvuw 2425710 2425708 2026-06-10T15:50:26Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1206.eqiad.wmnet with reason: host reimage 2425710 wikitext text/x-wiki gwvqtgt7qzx39k05mgv2x8qd9soul31 2425711 2425710 2026-06-10T15:51:46Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cp5018.eqsin.wmnet with reason: host reimage 2425711 wikitext text/x-wiki e4g5m1ynvs86o49065wlde24rthth8x 2425712 2425711 2026-06-10T15:52:40Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host dse-k8s-worker1009.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2425712 wikitext text/x-wiki 6853vy1us8gnjitz4zkvhecv7em95qi 2425713 2425712 2026-06-10T15:53:45Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-worker1009.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2425713 wikitext text/x-wiki dxo2opst1i8yzkn8dkc6kj6r7cpj19c 2425714 2425713 2026-06-10T15:59:20Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp5018.eqsin.wmnet with reason: host reimage 2425714 wikitext text/x-wiki 0yi3io6ktroa105cac3ah53740yiou9 2425715 2425714 2026-06-10T16:01:44Z Stashbot 7414 blblack: apt: uploaded libvmod-wmfuniq 0.3.0 for trixie 2425715 wikitext text/x-wiki 8gkavsymf0a9fbn936nv7rducrqqwep 2425718 2425715 2026-06-10T16:07:21Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1206.eqiad.wmnet with OS trixie 2425718 wikitext text/x-wiki mfepi821pzasxs65aoyt8zsrvlsfgbl 2425719 2425718 2026-06-10T16:11:35Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/admin 'apply'. 2425719 wikitext text/x-wiki t9cuitergc9zr6c946ot3dkiisrtrrq 2425720 2425719 2026-06-10T16:12:26Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'apply'. 2425720 wikitext text/x-wiki 25n864pj335rtois49vqgkfbzr0gwk4 2425721 2425720 2026-06-10T16:12:42Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/admin 'apply'. 2425721 wikitext text/x-wiki jqejx2mh7d9kz1akdfw5anbrqub6idp 2425722 2425721 2026-06-10T16:14:27Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'apply'. 2425722 wikitext text/x-wiki q9fulzgvjuk30mzkznq7co69xwaaoa1 2425723 2425722 2026-06-10T16:15:24Z Stashbot 7414 hnowlan@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2425723 wikitext text/x-wiki qwjnfsx0yhm8r9klgk14eydh142wfy7 2425724 2425723 2026-06-10T16:15:32Z Stashbot 7414 hnowlan@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2425724 wikitext text/x-wiki e6srlf0qxm4ffv65hd3npv2c6kdu5z9 2425725 2425724 2026-06-10T16:18:00Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1206: Migration of db1206.eqiad.wmnet completed 2425725 wikitext text/x-wiki qmofo0ydydffch09ugw2qw9xskpyzw1 2425726 2425725 2026-06-10T16:20:29Z Stashbot 7414 hnowlan@deploy1003: helmfile [codfw] START helmfile.d/services/thumbor: apply 2425726 wikitext text/x-wiki rx2p9xst7high1b6me28weh4q8x5gc1 2425727 2425726 2026-06-10T16:22:36Z Stashbot 7414 hnowlan@deploy1003: helmfile [codfw] DONE helmfile.d/services/thumbor: apply 2425727 wikitext text/x-wiki m1yuzw6a5x5wk9zz9anfbu9f3fjr1et 2425728 2425727 2026-06-10T16:28:26Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp5018.eqsin.wmnet with OS trixie 2425728 wikitext text/x-wiki 02w9nunzffkuh1ngueca3xadg3gs0u1 2425729 2425728 2026-06-10T16:34:02Z Stashbot 7414 bblack@cumin1003: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on P{cp7008.magru.wmnet} and A:cp - Upgrade wmfuniq to 0.3.0 () 2425729 wikitext text/x-wiki bay9bo7wdfo4t4a9s7du3ns3inoea09 2425730 2425729 2026-06-10T16:39:09Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] START helmfile.d/services/thumbor: apply 2425730 wikitext text/x-wiki cxc1cda1w88ge4ulbvb2wd49apkoi33 2425732 2425730 2026-06-10T16:39:57Z Stashbot 7414 bblack@cumin1003: END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on P{cp7008.magru.wmnet} and A:cp - Upgrade wmfuniq to 0.3.0 () 2425732 wikitext text/x-wiki ks8qga4198repp74k3vjo6ca4nh7ohv 2425734 2425732 2026-06-10T16:41:15Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] DONE helmfile.d/services/thumbor: apply 2425734 wikitext text/x-wiki 8glpw6pskb4pd53xfolv00kqokv50cc 2425735 2425734 2026-06-10T16:50:24Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.hosts.reimage for host kafka-main2010.codfw.wmnet with OS trixie 2425735 wikitext text/x-wiki 9gq0y8g52kqzfm5epsxec2y3hmi2vio 2425736 2425735 2026-06-10T16:50:45Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.hosts.move-vlan for host kafka-main2010 2425736 wikitext text/x-wiki amr7sz5gwtbbr7tbz184r3z5jl7lg61 2425737 2425736 2026-06-10T16:57:27Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.dns.netbox 2425737 wikitext text/x-wiki eiuwdfztxt2a6sf9ew3mkazbhonqdzj 2425738 2425737 2026-06-10T17:01:53Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-main2010 - jasmine@cumin2002" 2425738 wikitext text/x-wiki it0vtg0x46bac5abw3wgrz3dt2rl5yi 2425739 2425738 2026-06-10T17:02:15Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-main2010 - jasmine@cumin2002" 2425739 wikitext text/x-wiki g8hs1mslbvj3f2ht2b6ewb0xizrzw5w 2425740 2425739 2026-06-10T17:02:17Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2425740 wikitext text/x-wiki hy9c84pjir60y1hz85lia7akskwynln 2425741 2425740 2026-06-10T17:02:19Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.dns.wipe-cache kafka-main2010.codfw.wmnet 35.48.192.10.in-addr.arpa 5.3.0.0.8.4.0.0.2.9.1.0.0.1.0.0.4.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2425741 wikitext text/x-wiki khq3diuihc7i3goives64fzdx84mga2 2425742 2425741 2026-06-10T17:02:21Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-main2010.codfw.wmnet 35.48.192.10.in-addr.arpa 5.3.0.0.8.4.0.0.2.9.1.0.0.1.0.0.4.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2425742 wikitext text/x-wiki bkmdz1xhv4hcbl3ee20djcumyogwn61 2425743 2425742 2026-06-10T17:02:24Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host kafka-main2010 2425743 wikitext text/x-wiki jjg9yc7llq0tck5bbxrpc3zwviklnfa 2425744 2425743 2026-06-10T17:02:40Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-main2010 2425744 wikitext text/x-wiki 1sy8gt84axwcyrxur3xxabfgviiiuvo 2425745 2425744 2026-06-10T17:02:42Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-main2010 2425745 wikitext text/x-wiki 7lr6wpsqk9mgnidv5216mw899q5mlvg 2425746 2425745 2026-06-10T17:03:29Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1206: Migration of db1206.eqiad.wmnet completed 2425746 wikitext text/x-wiki 5sf8xbt8p8ex6g2s853z3pzm4dhscw8 2425747 2425746 2026-06-10T17:03:32Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2425747 wikitext text/x-wiki 9wty42436zsl8oiricanuqrpwi8arwr 2425748 2425747 2026-06-10T17:12:50Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.dns.roll-restart-ntp (exit_code=0) rolling restart_daemons on A:dnsbox and (A:dnsbox) 2425748 wikitext text/x-wiki kgsg5ijc943qyey26ceh4cy321ud46l 2425749 2425748 2026-06-10T17:13:59Z Stashbot 7414 jasmine@deploy1003: helmfile [eqiad] START helmfile.d/admin 'apply'. 2425749 wikitext text/x-wiki iqmx7mus1zm0zpup8w6nbfxh3cns2jl 2425750 2425749 2026-06-10T17:14:01Z Stashbot 7414 jasmine@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'apply'. 2425750 wikitext text/x-wiki enploo9np86klv2sloevr6qn8hgm6yp 2425751 2425750 2026-06-10T17:14:08Z Stashbot 7414 jasmine@deploy1003: helmfile [codfw] START helmfile.d/admin 'apply'. 2425751 wikitext text/x-wiki ssxr5wyhhawhffdyimfslqkeeq3uepa 2425752 2425751 2026-06-10T17:14:11Z Stashbot 7414 jasmine@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'apply'. 2425752 wikitext text/x-wiki na1070ud3w5vluq0wotyv2llyscrxj1 2425753 2425752 2026-06-10T17:14:18Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'apply'. 2425753 wikitext text/x-wiki mn9a9xxdidpfz39un2eiql0oft4hqcc 2425754 2425753 2026-06-10T17:14:21Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. 2425754 wikitext text/x-wiki 1xy8wcqbpfstezausijmjaldhlmd09z 2425755 2425754 2026-06-10T17:14:27Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-codfw] START helmfile.d/admin 'apply'. 2425755 wikitext text/x-wiki 6fuuh488ui3azhqsbfo54rnzxegpam7 2425756 2425755 2026-06-10T17:14:29Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. 2425756 wikitext text/x-wiki cxhq0cx837vnabnw18pr85rlffz6uhi 2425757 2425756 2026-06-10T17:14:36Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. 2425757 wikitext text/x-wiki h0bare2m9ai1ofxqunkn9z3w7cm01fo 2425758 2425757 2026-06-10T17:14:39Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. 2425758 wikitext text/x-wiki l3skog1d9hiu7r608ekkm0eh3hxnkqe 2425759 2425758 2026-06-10T17:14:45Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-codfw] START helmfile.d/admin 'apply'. 2425759 wikitext text/x-wiki 0earc33rm4ramgsqusjxvgb4jqk3aak 2425760 2425759 2026-06-10T17:14:47Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-codfw] DONE helmfile.d/admin 'apply'. 2425760 wikitext text/x-wiki ixiqhzl2kv66awliyt0s6gewtverxfu 2425761 2425760 2026-06-10T17:14:53Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'apply'. 2425761 wikitext text/x-wiki lji05di0uwxsh3mkp9hi4b114kh0utd 2425762 2425761 2026-06-10T17:14:56Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'apply'. 2425762 wikitext text/x-wiki 0dhh3f31wsm6rcxyd82ykz12rdc1xyh 2425763 2425762 2026-06-10T17:15:03Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2425763 wikitext text/x-wiki 9hgzy8o197mezh9363nlyyj64pmxpcn 2425764 2425763 2026-06-10T17:15:05Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2425764 wikitext text/x-wiki fqzlie8nvgc0plnpypti31f3kbt4pwo 2425765 2425764 2026-06-10T17:15:11Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2425765 wikitext text/x-wiki 4gccdesc319kh2vuonzlyx053rku7hv 2425766 2425765 2026-06-10T17:15:14Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2425766 wikitext text/x-wiki h1mx23i2ps1nswdl52trkiyk9u00a73 2425767 2425766 2026-06-10T17:15:19Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/admin 'apply'. 2425767 wikitext text/x-wiki fii22ovn7mk9kw5h70is2fiqbhcgalw 2425768 2425767 2026-06-10T17:15:21Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2425768 wikitext text/x-wiki pu1u1ha5utuhh9o9cp9ftgnppdcfgvp 2425769 2425768 2026-06-10T17:15:27Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/admin 'apply'. 2425769 wikitext text/x-wiki hid8e6xqc8xux6ukx9obl0sanq2wbzb 2425770 2425769 2026-06-10T17:15:29Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/admin 'apply'. 2425770 wikitext text/x-wiki fwgk8p1ufsm9oo6w7hni1zpxcprl8bo 2425771 2425770 2026-06-10T17:17:24Z Stashbot 7414 jasmine@deploy1003: helmfile [eqiad] START helmfile.d/admin 'apply'. 2425771 wikitext text/x-wiki dgfkwnbk3qu3c9b08ji4vbpce8wv8e4 2425772 2425771 2026-06-10T17:17:45Z Stashbot 7414 jasmine@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'apply'. 2425772 wikitext text/x-wiki mxx31tysf8cdsgi9uc3jwqehadszvl7 2425773 2425772 2026-06-10T17:17:53Z Stashbot 7414 jasmine@deploy1003: helmfile [codfw] START helmfile.d/admin 'apply'. 2425773 wikitext text/x-wiki f0usc6ej272n9996pyqk5abgbiqdxeb 2425774 2425773 2026-06-10T17:18:35Z Stashbot 7414 jasmine@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'apply'. 2425774 wikitext text/x-wiki 8oqjq5l1ywfy6kqj96ml4y2l0oxtgwo 2425775 2425774 2026-06-10T17:18:43Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'apply'. 2425775 wikitext text/x-wiki cxqwwbptmq6gogv41ypijoe9c0g8zts 2425776 2425775 2026-06-10T17:19:12Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. 2425776 wikitext text/x-wiki 6cn48uauhdlzmfqzv1d49691nf7e58b 2425777 2425776 2026-06-10T17:19:20Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-codfw] START helmfile.d/admin 'apply'. 2425777 wikitext text/x-wiki lsix77h0icrt2f0cd3l0v5jovva1cds 2425778 2425777 2026-06-10T17:20:09Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. 2425778 wikitext text/x-wiki eh52l00ln9wnsyiidn38xhc659bdlvd 2425779 2425778 2026-06-10T17:20:17Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. 2425779 wikitext text/x-wiki 1d8y7onsykf759k9cygferbp7eezb1h 2425780 2425779 2026-06-10T17:20:44Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. 2425780 wikitext text/x-wiki 3qn99pgsg4019rx0sj1jiljhk8p2vt7 2425781 2425780 2026-06-10T17:20:52Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-codfw] START helmfile.d/admin 'apply'. 2425781 wikitext text/x-wiki d0p28vka7iukv7yujpf2mirwkx4jkdn 2425782 2425781 2026-06-10T17:21:42Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-codfw] DONE helmfile.d/admin 'apply'. 2425782 wikitext text/x-wiki n7eur7w0ot47vq0c25pcvzdb0m5p56t 2425783 2425782 2026-06-10T17:21:50Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'apply'. 2425783 wikitext text/x-wiki i3on9q3jlumo9hgs0olgp705g02sb46 2425784 2425783 2026-06-10T17:22:09Z Stashbot 7414 bblack@cumin1003: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on A:cp-text and not P{cp7008*} and A:cp - Upgrade wmfuniq to 0.3.0 () 2425784 wikitext text/x-wiki i32zj3sqqypt6ousmxnntc389aoru0a 2425785 2425784 2026-06-10T17:22:34Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2425785 wikitext text/x-wiki brn56bvadijqqs5ym9vmoqni0a5a4d1 2425786 2425785 2026-06-10T17:22:40Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'apply'. 2425786 wikitext text/x-wiki 6in2i4j5dxh1lli4yavntv2r3o0hn8e 2425787 2425786 2026-06-10T17:22:43Z Stashbot 7414 bblack@cumin1003: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on A:cp-upload and not P{cp7008.magru.wmnet} and A:cp - Upgrade wmfuniq to 0.3.0 () 2425787 wikitext text/x-wiki c1vz39lxs7vf97k6dkigfjs7xqt0nup 2425788 2425787 2026-06-10T17:22:49Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2425788 wikitext text/x-wiki l6q2qx4lhlxoo101v398kcikwnqb8bm 2425789 2425788 2026-06-10T17:22:55Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db2174: Upgrading db2174.codfw.wmnet 2425789 wikitext text/x-wiki mianyqq0kumsq4pkz4f4qu57lz01mzd 2425790 2425789 2026-06-10T17:23:15Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2425790 wikitext text/x-wiki 70eo7a27xnqu6m0rlfuaemvwbcbf505 2425791 2425790 2026-06-10T17:23:20Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-main2010.codfw.wmnet with reason: host reimage 2425791 wikitext text/x-wiki 9i4iwrcan4umh9jda2zyzduatd4i01e 2425792 2425791 2026-06-10T17:23:22Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2425792 wikitext text/x-wiki j9oihbyr85dsvskav09ottjl0sqykfw 2425793 2425792 2026-06-10T17:23:28Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2174: Upgrading db2174.codfw.wmnet 2425793 wikitext text/x-wiki 0mzyinkwqigdwiybsudtdorvjnqtymt 2425794 2425793 2026-06-10T17:23:49Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2425794 wikitext text/x-wiki cfbv1xtg3vxk730jcryo2wu0g4h9bzg 2425795 2425794 2026-06-10T17:24:09Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db1218: Upgrading db1218.eqiad.wmnet 2425795 wikitext text/x-wiki fbhb2zn0wsyv51kxjtc0jmxhpifvssb 2425796 2425795 2026-06-10T17:24:12Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2425796 wikitext text/x-wiki ro9s283lhejfjq4cowdsf1fe39ih1vs 2425797 2425796 2026-06-10T17:24:18Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/admin 'apply'. 2425797 wikitext text/x-wiki s36pkai0mbffz66j058wtmo938p6inm 2425798 2425797 2026-06-10T17:24:40Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1218: Upgrading db1218.eqiad.wmnet 2425798 wikitext text/x-wiki j628zw0u2jtd8mqbd3px7pv1u2efrx3 2425799 2425798 2026-06-10T17:24:49Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2425799 wikitext text/x-wiki 80yqkg9jnrm37p7san3yfg1tj5u8xdl 2425800 2425799 2026-06-10T17:24:56Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/admin 'apply'. 2425800 wikitext text/x-wiki pecwcuvoz93kh3zr7vh0xe6tjx4bc2c 2425801 2425800 2026-06-10T17:25:46Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/admin 'apply'. 2425801 wikitext text/x-wiki adgva7a6xjlfn3r1f9zqolhx9rm2m83 2425802 2425801 2026-06-10T17:26:01Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db2174.codfw.wmnet with OS trixie 2425802 wikitext text/x-wiki lionnkqwwvslwvipf6g04cuuhbkv7p5 2425803 2425802 2026-06-10T17:26:21Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db1218.eqiad.wmnet with OS trixie 2425803 wikitext text/x-wiki a4p35yzhhkz3zlp82rsw7u6czjvnmt0 2425804 2425803 2026-06-10T17:29:30Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-main2010.codfw.wmnet with reason: host reimage 2425804 wikitext text/x-wiki 9c32u9xkyahkcvtqr2rh2m0arxnh1c3 2425805 2425804 2026-06-10T17:33:25Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist mwscript.dblist extensions/Translate/scripts/ttmserver-export.php --ttmserver eqiad-test # T425377 populating ttmserver index on test cluster to estimate time required for the release (dblist: https://phabricator.wikimedia.org/P94021) 2425805 wikitext text/x-wiki akkqkaetrw5kxmzx62zc4ylfejryfc2 2425806 2425805 2026-06-10T17:42:55Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1218.eqiad.wmnet with reason: host reimage 2425806 wikitext text/x-wiki gapm9g5qz29tm6h18vfv78jc207m4bs 2425807 2425806 2026-06-10T17:44:25Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2174.codfw.wmnet with reason: host reimage 2425807 wikitext text/x-wiki pkfvimzh241uwgecmkbuqk6v2wtfhiu 2425808 2425807 2026-06-10T17:45:02Z Stashbot 7414 jasmine@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-main: sync 2425808 wikitext text/x-wiki 0sk4yv6l3ko9ibylomgh551nmo3gdoe 2425809 2425808 2026-06-10T17:45:21Z Stashbot 7414 jasmine@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-main: sync 2425809 wikitext text/x-wiki amv2i5h71pq1rrvkeqqkiqhvmkw53ry 2425810 2425809 2026-06-10T17:46:44Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-main2010.codfw.wmnet with OS trixie 2425810 wikitext text/x-wiki 78kwaminevsp2i7hlkua22pi8vuwzgq 2425811 2425810 2026-06-10T17:49:12Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1218.eqiad.wmnet with reason: host reimage 2425811 wikitext text/x-wiki mshwxm5fy0g0s529dp7ludrkzx9vwa0 2425812 2425811 2026-06-10T17:52:45Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2174.codfw.wmnet with reason: host reimage 2425812 wikitext text/x-wiki is72h0igtjyl9wzqqecdug951bt4s3z 2425813 2425812 2026-06-10T18:06:25Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1218.eqiad.wmnet with OS trixie 2425813 wikitext text/x-wiki qhxv40mjp0phv6kxk9ymaaqqxiyq4yk 2425814 2425813 2026-06-10T18:10:27Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2174.codfw.wmnet with OS trixie 2425814 wikitext text/x-wiki 47wiocfy1jmlf92t79ngnyije5fz5m8 2425815 2425814 2026-06-10T18:16:53Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp5018.* 2425815 wikitext text/x-wiki 6bpxf09p42ihkasytghoi7uorv2clbs 2425816 2425815 2026-06-10T18:17:07Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1218: Migration of db1218.eqiad.wmnet completed 2425816 wikitext text/x-wiki 2gqew5o8xsqv5mkvr5ef5fbg2s9ooq1 2425817 2425816 2026-06-10T18:20:29Z Stashbot 7414 dduvall@deploy1003: rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.6 refs T423915 2425817 wikitext text/x-wiki 6iifxz3cog1kn5efao06oqx7xv7wpaf 2425818 2425817 2026-06-10T18:22:20Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db2174: Migration of db2174.codfw.wmnet completed 2425818 wikitext text/x-wiki kty8m0j0ea35pv2edvfbo8ny3ijze3z 2425819 2425818 2026-06-10T18:23:51Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.reimage for host cp5020.eqsin.wmnet with OS trixie 2425819 wikitext text/x-wiki cw878r9byccy968aty4v4u9e7eoovci 2425820 2425819 2026-06-10T18:24:22Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.move-vlan for host cp5020 2425820 wikitext text/x-wiki lc4ysno1x0jbl0l16hvnvk6ooe30npk 2425822 2425820 2026-06-10T19:02:37Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1218: Migration of db1218.eqiad.wmnet completed 2425822 wikitext text/x-wiki 4mqcpl8vc6uh5rpwb4d7zxqqazeyp43 2425823 2425822 2026-06-10T19:02:40Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2425823 wikitext text/x-wiki cyfc214d1rjb5rn4lf3v6ni3h8i6kds 2425824 2425823 2026-06-10T19:07:50Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2174: Migration of db2174.codfw.wmnet completed 2425824 wikitext text/x-wiki pw6tsjiuxxt6ybjzngqlggytd9ipfdk 2425825 2425824 2026-06-10T19:07:52Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2425825 wikitext text/x-wiki 3ivqe5okup8w32fkr0w6xw85vbesnam 2425826 2425825 2026-06-10T19:11:08Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.dns.netbox 2425826 wikitext text/x-wiki 5sclvz8wa0gsesuujvq4ih2zv0yyhq0 2425827 2425826 2026-06-10T19:14:34Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on P{cp2044.codfw.wmnet} and A:cp - testing 1300236 () 2425827 wikitext text/x-wiki 5jgxezcytxmszp04h8kjvf6dig0zs8r 2425828 2425827 2026-06-10T19:17:55Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host cp5020 - brett@cumin2002" 2425828 wikitext text/x-wiki prqf872br5jgmmktbhou8n71ubjtsip 2425829 2425828 2026-06-10T19:18:00Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host cp5020 - brett@cumin2002" 2425829 wikitext text/x-wiki n1b1evx2yr6rj8t13yabu8f4z78tyl3 2425830 2425829 2026-06-10T19:18:02Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2425830 wikitext text/x-wiki d67xmfdihpjij7j4ijjsz60gyvmxi3i 2425831 2425830 2026-06-10T19:18:05Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.dns.wipe-cache cp5020.eqsin.wmnet 24.0.132.10.in-addr.arpa 4.2.0.0.0.0.0.0.2.3.1.0.0.1.0.0.1.0.1.0.0.0.5.e.2.f.d.0.1.0.0.2.ip6.arpa on all recursors 2425831 wikitext text/x-wiki t847okz84qkalqaj6j08896s1cl1jdw 2425832 2425831 2026-06-10T19:18:08Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) cp5020.eqsin.wmnet 24.0.132.10.in-addr.arpa 4.2.0.0.0.0.0.0.2.3.1.0.0.1.0.0.1.0.1.0.0.0.5.e.2.f.d.0.1.0.0.2.ip6.arpa on all recursors 2425832 wikitext text/x-wiki ks7inbjqwifdnvhng2x2o4r9ioyrpf6 2425833 2425832 2026-06-10T19:18:10Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host cp5020 2425833 wikitext text/x-wiki ewmlqxo9ejty5g53qly5tqb8soooe3a 2425834 2425833 2026-06-10T19:18:55Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on P{cp2044.codfw.wmnet} and A:cp - testing 1300236 () 2425834 wikitext text/x-wiki 2ptsa9e030cnzxhkb59jpq6vkaqhhrz 2425835 2425834 2026-06-10T19:18:59Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cp5020 2425835 wikitext text/x-wiki 7v7cqztixy31g1xkv37tulusaimy0e3 2425836 2425835 2026-06-10T19:19:01Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host cp5020 2425836 wikitext text/x-wiki 2o988fpbrh25solzvdgg3hhoed4l8be 2425837 2425836 2026-06-10T19:19:34Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on P{cp2046.codfw.wmnet} and A:cp - testing 1300236 () 2425837 wikitext text/x-wiki js6w1p5m6ohn7ayvll69v7ckl94xeut 2425838 2425837 2026-06-10T19:23:26Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on P{cp2046.codfw.wmnet} and A:cp - testing 1300236 () 2425838 wikitext text/x-wiki ky5ijerod2x3y1ba52acgfivuzo0o2e 2425839 2425838 2026-06-10T19:27:44Z Stashbot 7414 bblack@cumin1003: END (FAIL) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=1) rolling upgrade of Varnish on A:cp-upload and not P{cp7008.magru.wmnet} and A:cp - Upgrade wmfuniq to 0.3.0 () 2425839 wikitext text/x-wiki hyixkyu20ad8l0fpdybmrtq6rn04vnz 2425841 2425839 2026-06-10T19:30:15Z Stashbot 7414 bblack@cumin1003: START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on A:cp-upload and not P{cp7008.magru.wmnet} and A:cp - Upgrade wmfuniq to 0.3.0 () 2425841 wikitext text/x-wiki ohiew8oqc0wiup3zgf8gliumeaqmomq 2425845 2425841 2026-06-10T19:53:17Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cp5020.eqsin.wmnet with reason: host reimage 2425845 wikitext text/x-wiki m3o0bytsbrj4jh1f31by150g1lwxxee 2425846 2425845 2026-06-10T19:59:31Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp5020.eqsin.wmnet with reason: host reimage 2425846 wikitext text/x-wiki fn3nzfcgs92z57f7idfmuw9phc3xy4n 2425847 2425846 2026-06-10T20:20:13Z Stashbot 7414 catrope@deploy1003: Started scap sync-world: Backport for [[gerrit:1300226|[arzwiki] Change the wordmark (T427720)]] 2425847 wikitext text/x-wiki i9afhdnomkpib6pxauyf7504xy1phi0 2425848 2425847 2026-06-10T20:22:32Z Stashbot 7414 catrope@deploy1003: gergesshamon, catrope: Backport for [[gerrit:1300226|[arzwiki] Change the wordmark (T427720)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2425848 wikitext text/x-wiki lqauha7cr5za8k3vnmkr6pa55ed34p3 2425849 2425848 2026-06-10T20:25:38Z Stashbot 7414 catrope@deploy1003: gergesshamon, catrope: Continuing with deployment 2425849 wikitext text/x-wiki 40vgnxydlx3towfkinzng5yq4a0qqvb 2425851 2425849 2026-06-10T20:30:02Z Stashbot 7414 catrope@deploy1003: Finished scap sync-world: Backport for [[gerrit:1300226|[arzwiki] Change the wordmark (T427720)]] (duration: 09m 49s) 2425851 wikitext text/x-wiki a0rodm0rj41y1mukwt3357uhy95w18r 2425852 2425851 2026-06-10T20:30:58Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp5020.eqsin.wmnet with OS trixie 2425852 wikitext text/x-wiki 1wz1nz0p6n1u3exyd4vlnek1x227t32 2425853 2425852 2026-06-10T20:32:31Z Stashbot 7414 catrope@deploy1003: Started scap sync-world: Backport for [[gerrit:1300073|wgRestSandboxSpecs: Add Lift Wing API to documentation wikis (T427902)]] 2425853 wikitext text/x-wiki m2vabfu9povuriw8q2grx37k3limdwj 2425854 2425853 2026-06-10T20:34:38Z Stashbot 7414 catrope@deploy1003: catrope, gkyziridis: Backport for [[gerrit:1300073|wgRestSandboxSpecs: Add Lift Wing API to documentation wikis (T427902)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2425854 wikitext text/x-wiki jp3femmbhxulth5dm8xd69pg44ni8qc 2425855 2425854 2026-06-10T20:40:05Z Stashbot 7414 catrope@deploy1003: catrope, gkyziridis: Continuing with deployment 2425855 wikitext text/x-wiki 9uvz5r0jvqalwcbjxlvacl80qq30za1 2425856 2425855 2026-06-10T20:44:26Z Stashbot 7414 catrope@deploy1003: Finished scap sync-world: Backport for [[gerrit:1300073|wgRestSandboxSpecs: Add Lift Wing API to documentation wikis (T427902)]] (duration: 11m 55s) 2425856 wikitext text/x-wiki pw8t669nyhomuif5kcrvjdmbm7gz748 2425857 2425856 2026-06-10T20:48:23Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp5020.* 2425857 wikitext text/x-wiki fz0tezp1osuc3sx3yx3xj4orxmvroym 2425858 2425857 2026-06-10T20:49:32Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.reimage for host cp5024.eqsin.wmnet with OS trixie 2425858 wikitext text/x-wiki lap1drmm2wa8uqk79b5sirzf4pvv089 2425859 2425858 2026-06-10T20:50:05Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.move-vlan for host cp5024 2425859 wikitext text/x-wiki 7vvlmiec2nnk1mr87nlnfnwn0unjcoo 2425860 2425859 2026-06-10T20:54:32Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.dns.netbox 2425860 wikitext text/x-wiki 53wkhvcyym75dxt21tznlpd7te0tpc8 2425861 2425860 2026-06-10T20:55:27Z Stashbot 7414 catrope@deploy1003: Started scap sync-world: Backport for [[gerrit:1300247|Revert "wgRestSandboxSpecs: Add Lift Wing API to documentation wikis" (T427902)]] 2425861 wikitext text/x-wiki i89424c0eo8nycs35oxhrkifa7xyyrm 2425863 2425861 2026-06-10T20:57:32Z Stashbot 7414 catrope@deploy1003: catrope: Backport for [[gerrit:1300247|Revert "wgRestSandboxSpecs: Add Lift Wing API to documentation wikis" (T427902)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2425863 wikitext text/x-wiki evvl95nqml0achf7kgcnhkkdpfhflkm 2425864 2425863 2026-06-10T20:58:00Z Stashbot 7414 catrope@deploy1003: catrope: Continuing with deployment 2425864 wikitext text/x-wiki gava70gzwk23xbjks7v8qo0pp5zcsi4 2425865 2425864 2026-06-10T20:59:57Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host cp5024 - brett@cumin2002" 2425865 wikitext text/x-wiki rkdvampqpeomyfx1vsl2kwsypohvsap 2425866 2425865 2026-06-10T21:00:02Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host cp5024 - brett@cumin2002" 2425866 wikitext text/x-wiki mmyr4f7q8yrufyxwll5gvzvhh1n4u7j 2425867 2425866 2026-06-10T21:00:04Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2425867 wikitext text/x-wiki qdd08man7yvpzjc8cvc590b30ty7lvv 2425868 2425867 2026-06-10T21:00:07Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.dns.wipe-cache cp5024.eqsin.wmnet 35.0.132.10.in-addr.arpa 5.3.0.0.0.0.0.0.2.3.1.0.0.1.0.0.1.0.1.0.0.0.5.e.2.f.d.0.1.0.0.2.ip6.arpa on all recursors 2425868 wikitext text/x-wiki ievynesa2m0kbbj2ne0gzqv4ol6souk 2425869 2425868 2026-06-10T21:00:10Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) cp5024.eqsin.wmnet 35.0.132.10.in-addr.arpa 5.3.0.0.0.0.0.0.2.3.1.0.0.1.0.0.1.0.1.0.0.0.5.e.2.f.d.0.1.0.0.2.ip6.arpa on all recursors 2425869 wikitext text/x-wiki bt416rxxol3mrfivp8te7d4yw73g85q 2425870 2425869 2026-06-10T21:00:12Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host cp5024 2425870 wikitext text/x-wiki d3wpbmvy0jklz5cn96bzl7fv26aioe2 2425871 2425870 2026-06-10T21:02:18Z Stashbot 7414 catrope@deploy1003: Finished scap sync-world: Backport for [[gerrit:1300247|Revert "wgRestSandboxSpecs: Add Lift Wing API to documentation wikis" (T427902)]] (duration: 06m 51s) 2425871 wikitext text/x-wiki n265i6xl8ph5usg55fq1p9gdxrh0jrk 2425872 2425871 2026-06-10T21:03:00Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cp5024 2425872 wikitext text/x-wiki 5pzh190ptml3gcl1fnxnrgymfgyfcw1 2425873 2425872 2026-06-10T21:03:03Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host cp5024 2425873 wikitext text/x-wiki re2dhkvwurza0ylyvf7mc11jps9vgue 2425874 2425873 2026-06-10T21:13:38Z Stashbot 7414 jforrester@deploy1003: Started scap sync-world: Backport for [[gerrit:1300250|ExecuteTestAndCacheJob: Fix stdClasses serialised wrongly by JobQueue (T428801)]], [[gerrit:1300248|tests: Fix StandaloneHooksTest ordering, now broken by DB upgrade]] 2425874 wikitext text/x-wiki 89z5qzghylxxumoeippicnsreuk6zx4 2425876 2425874 2026-06-10T21:15:44Z Stashbot 7414 jforrester@deploy1003: jforrester: Backport for [[gerrit:1300250|ExecuteTestAndCacheJob: Fix stdClasses serialised wrongly by JobQueue (T428801)]], [[gerrit:1300248|tests: Fix StandaloneHooksTest ordering, now broken by DB upgrade]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2425876 wikitext text/x-wiki a0w0099kvn3hfmvslk14pisvxv3tx3c 2425877 2425876 2026-06-10T21:17:42Z Stashbot 7414 jforrester@deploy1003: jforrester: Continuing with deployment 2425877 wikitext text/x-wiki 1m3kjl48ktud0n4vgda3x34twdclytw 2425878 2425877 2026-06-10T21:22:01Z Stashbot 7414 jforrester@deploy1003: Finished scap sync-world: Backport for [[gerrit:1300250|ExecuteTestAndCacheJob: Fix stdClasses serialised wrongly by JobQueue (T428801)]], [[gerrit:1300248|tests: Fix StandaloneHooksTest ordering, now broken by DB upgrade]] (duration: 08m 23s) 2425878 wikitext text/x-wiki edpnikxj4x19aq9iz0g9tdn25oprgnc 2425879 2425878 2026-06-10T21:37:18Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cp5024.eqsin.wmnet with reason: host reimage 2425879 wikitext text/x-wiki eh5cbjn7u7w5p8idlmgs6tsqwqd86ry 2425880 2425879 2026-06-10T21:44:46Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp5024.eqsin.wmnet with reason: host reimage 2425880 wikitext text/x-wiki b3xp85lqlqrk1q3pyz7bmr81o8yvoxm 2425881 2425880 2026-06-10T22:06:02Z Stashbot 7414 mutante: gerrit-replica: restarting gerrit 2425881 wikitext text/x-wiki ej1k6v3uednz12y8r000bv9korxcj1e 2425882 2425881 2026-06-10T22:06:59Z Stashbot 7414 mutante: gerrit-spare: restarting gerrit 2425882 wikitext text/x-wiki 1x6drh77uqg6o8pqi2hcjfwkb7oix04 2425883 2425882 2026-06-10T22:09:33Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:10:00 on gerrit2003.wikimedia.org with reason: service restart 2425883 wikitext text/x-wiki d1rwjym7euyxopvyrlre2v1bsctjlq7 2425884 2425883 2026-06-10T22:11:42Z Stashbot 7414 dzahn@cumin2002: DONE (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 0:10:00 on gerrit.wikimedia.org with reason: service restart 2425884 wikitext text/x-wiki 9e9r1vo88ipov64cd04wkmblktwm6cz 2425885 2425884 2026-06-10T22:13:36Z Stashbot 7414 mutante: gerrit - restarting service for logging change 2425885 wikitext text/x-wiki 2b5c9qscpd27p1x5cyqj7glha3i1jg2 2425886 2425885 2026-06-10T22:13:57Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp5024.eqsin.wmnet with OS trixie 2425886 wikitext text/x-wiki qms5frjpziynoedxpcd4kv4q3ocq1r5 2425887 2425886 2026-06-10T22:55:32Z Stashbot 7414 ladsgroup@dns1004: START - running authdns-update 2425887 wikitext text/x-wiki 16pfez253f9mhaae306n60lc8nn40yx 2425888 2425887 2026-06-10T22:57:09Z Stashbot 7414 ladsgroup@dns1004: END - running authdns-update 2425888 wikitext text/x-wiki hpaqrvfzxu26jwvy9yvriwpegy1a1i4 2425889 2425888 2026-06-10T23:04:05Z Stashbot 7414 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1300154|Disable ShortUrl on bdwikimedia, bhwiki, bnwiki, bnwikisource, eswikibooks, gomwiki (T107188)]] 2425889 wikitext text/x-wiki 904nl4ldx4crzyayxo27xsf2cqqxspw 2425890 2425889 2026-06-10T23:06:14Z Stashbot 7414 krinkle@deploy1003: krinkle: Backport for [[gerrit:1300154|Disable ShortUrl on bdwikimedia, bhwiki, bnwiki, bnwikisource, eswikibooks, gomwiki (T107188)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2425890 wikitext text/x-wiki 9lj90vah0x7czvunorurhitk5rvc8dw 2425891 2425890 2026-06-10T23:11:18Z Stashbot 7414 krinkle@deploy1003: krinkle: Continuing with deployment 2425891 wikitext text/x-wiki 2nszxftwmgltz2d59o82krxpl18gjyj 2425892 2425891 2026-06-10T23:15:43Z Stashbot 7414 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1300154|Disable ShortUrl on bdwikimedia, bhwiki, bnwiki, bnwikisource, eswikibooks, gomwiki (T107188)]] (duration: 11m 37s) 2425892 wikitext text/x-wiki 0ibb94ttzrjz1dqhy7k4g5w7rzigip9 2425893 2425892 2026-06-10T23:53:46Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp5024.* 2425893 wikitext text/x-wiki f6gl9bills7t44gnl7sce4zlfr6ix6p 2425895 2425893 2026-06-11T00:03:23Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=no; selector: name=cp5024.* 2425895 wikitext text/x-wiki ema3oqyabmd2iea7qnbb693lc8xmsoe 2425897 2425895 2026-06-11T00:30:29Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.hosts.reimage for host kafka-main1009.eqiad.wmnet with OS trixie 2425897 wikitext text/x-wiki hmv50frvkm6ye4c1yqmyfqdgm69xbrz 2425898 2425897 2026-06-11T00:31:00Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.hosts.move-vlan for host kafka-main1009 2425898 wikitext text/x-wiki 8gprdw3zaks2xhmr60alqv25b38og8c 2425899 2425898 2026-06-11T00:34:22Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=no; selector: name=cp5020.* 2425899 wikitext text/x-wiki fzf5wis28c1cfy0zmr1y22ekubjbkwx 2425900 2425899 2026-06-11T00:36:22Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.dns.netbox 2425900 wikitext text/x-wiki h1dvbolxliwu47si5whwaqgfctayjul 2425901 2425900 2026-06-11T00:39:52Z Stashbot 7414 cdanis@cumin1003: dbctl commit (dc=all): 'depool db1262', diff saved to https://phabricator.wikimedia.org/P94032 and previous config saved to /var/cache/conftool/dbconfig/20260611-003950-cdanis.json 2425901 wikitext text/x-wiki dgz4lttp4bgtvtc3ngsvsjts5sb9ac9 2425902 2425901 2026-06-11T00:40:56Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-main1009 - jasmine@cumin2002" 2425902 wikitext text/x-wiki 6z0sq3q8td6qq0ndd3hvr9pq2uw12gm 2425903 2425902 2026-06-11T00:41:01Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-main1009 - jasmine@cumin2002" 2425903 wikitext text/x-wiki tjjm0ej3loymksn4laaw6r31qhnk91v 2425904 2425903 2026-06-11T00:41:03Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2425904 wikitext text/x-wiki skh5us7e1tisngrxk6vvvyhc4wsvubt 2425905 2425904 2026-06-11T00:41:05Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.dns.wipe-cache kafka-main1009.eqiad.wmnet 37.48.64.10.in-addr.arpa 7.3.0.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2425905 wikitext text/x-wiki eiohb68sccngjnw5zptn47z10yv10be 2425906 2425905 2026-06-11T00:41:08Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-main1009.eqiad.wmnet 37.48.64.10.in-addr.arpa 7.3.0.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2425906 wikitext text/x-wiki 8aqm8pnktec4jctctrrsx7476rskakn 2425907 2425906 2026-06-11T00:41:10Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host kafka-main1009 2425907 wikitext text/x-wiki r72ucdgibg2rs8iwcxoq6gcpc8tczl3 2425908 2425907 2026-06-11T00:41:32Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-main1009 2425908 wikitext text/x-wiki mg05ea4b1zhin485kiodkpq7be7ridg 2425909 2425908 2026-06-11T00:41:47Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-main1009 2425909 wikitext text/x-wiki ox9eszge8r2m2ndhixfnthngx9pi32r 2425910 2425909 2026-06-11T00:51:02Z Stashbot 7414 jasmine@deploy1003: helmfile [eqiad] START helmfile.d/admin 'apply'. 2425910 wikitext text/x-wiki 1essnz6b750leupq23rcj7izjxk6t59 2425911 2425910 2026-06-11T00:51:09Z Stashbot 7414 jasmine@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'apply'. 2425911 wikitext text/x-wiki 82av5es2sc1cp8th83gegebd9ghul6o 2425912 2425911 2026-06-11T00:51:18Z Stashbot 7414 jasmine@deploy1003: helmfile [codfw] START helmfile.d/admin 'apply'. 2425912 wikitext text/x-wiki 66ndb6gdd3jfd16jmn0ahquv7fzqoc8 2425913 2425912 2026-06-11T00:51:25Z Stashbot 7414 jasmine@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'apply'. 2425913 wikitext text/x-wiki pdg0h6sar0a1fzwl5bg9528vt3mm9u3 2425914 2425913 2026-06-11T00:51:34Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'apply'. 2425914 wikitext text/x-wiki ftptdoowj91dfvrrxmbrrkn1ipta4pr 2425915 2425914 2026-06-11T00:51:41Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. 2425915 wikitext text/x-wiki el4bc7foyvx70r4kwf8e3hqbwukjsr4 2425916 2425915 2026-06-11T00:51:49Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-codfw] START helmfile.d/admin 'apply'. 2425916 wikitext text/x-wiki ldyerzwtz89muwwchx67vqtpgpxise2 2425917 2425916 2026-06-11T00:51:57Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. 2425917 wikitext text/x-wiki h2u4ibyw1o8azwnewu6wu9t0y3owdg8 2425918 2425917 2026-06-11T00:52:05Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. 2425918 wikitext text/x-wiki iym5zxwr8pgqm9p2brdmivdb44g7s2q 2425919 2425918 2026-06-11T00:52:12Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. 2425919 wikitext text/x-wiki 2tn8qsgm5ama3ut352jsjudse1p28tj 2425920 2425919 2026-06-11T00:52:20Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-codfw] START helmfile.d/admin 'apply'. 2425920 wikitext text/x-wiki s58j4cyxk226q1nwb5rt4faann3kdad 2425921 2425920 2026-06-11T00:52:28Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-codfw] DONE helmfile.d/admin 'apply'. 2425921 wikitext text/x-wiki eijq6j8s9nayh8f8zzmvzs9rlinjt61 2425922 2425921 2026-06-11T00:52:35Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'apply'. 2425922 wikitext text/x-wiki 6gdufotlf5xyqu8ay1opcb59cxgq2d1 2425923 2425922 2026-06-11T00:52:44Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'apply'. 2425923 wikitext text/x-wiki 9wgc7jffqqjgjnpf105knbm38uohb75 2425924 2425923 2026-06-11T00:52:52Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2425924 wikitext text/x-wiki 7b6uaoutijjakrqhodlhxxxhrz6dup7 2425925 2425924 2026-06-11T00:53:00Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2425925 wikitext text/x-wiki p48gld7m0h1gq4dyaillbzmd8dj5h8o 2425926 2425925 2026-06-11T00:53:07Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2425926 wikitext text/x-wiki qavazcvolvrzftxmy5h80n9g95v61d2 2425927 2425926 2026-06-11T00:53:16Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2425927 wikitext text/x-wiki httm0hz3xr6usbfrhfugqnjo05kmf6w 2425928 2425927 2026-06-11T00:53:22Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/admin 'apply'. 2425928 wikitext text/x-wiki 1qe9j0csqoic5tegwxykg9ia1er1wgr 2425929 2425928 2026-06-11T00:53:30Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2425929 wikitext text/x-wiki 5vvwvtp972gct7y2p5ekxbfhfl27nio 2425930 2425929 2026-06-11T00:53:37Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/admin 'apply'. 2425930 wikitext text/x-wiki 8dyr9vp0e7w3i8c65o9ihikbxujuw0w 2425931 2425930 2026-06-11T00:54:28Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/admin 'apply'. 2425931 wikitext text/x-wiki g3v8h9ziaan61dg26f0cz8niwm46qr2 2425932 2425931 2026-06-11T00:58:45Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-main1009.eqiad.wmnet with reason: host reimage 2425932 wikitext text/x-wiki cz4c3gf43hiue424jdea4vq2yi2frwp 2425933 2425932 2026-06-11T01:02:31Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-main1009.eqiad.wmnet with reason: host reimage 2425933 wikitext text/x-wiki 8vb4gixjnz8nds04r9g6q9ed3ss3706 2425934 2425933 2026-06-11T01:05:12Z Stashbot 7414 jasmine@deploy1003: helmfile [eqiad] START helmfile.d/admin 'apply'. 2425934 wikitext text/x-wiki batnqrnlcnyr6n834bwdjjtejl07knc 2425935 2425934 2026-06-11T01:05:34Z Stashbot 7414 jasmine@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'apply'. 2425935 wikitext text/x-wiki 1r4rxurhyq7w7i4l6k9kkgh6q5pekbb 2425936 2425935 2026-06-11T01:05:42Z Stashbot 7414 jasmine@deploy1003: helmfile [codfw] START helmfile.d/admin 'apply'. 2425936 wikitext text/x-wiki f25bmqd12xgbx6w09s68mgtafpq1hwl 2425937 2425936 2026-06-11T01:06:24Z Stashbot 7414 jasmine@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'apply'. 2425937 wikitext text/x-wiki k1j0gzglj1x1kdsxiuzqpgk9nz3oy41 2425938 2425937 2026-06-11T01:06:32Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'apply'. 2425938 wikitext text/x-wiki ao6989fuesfdpsymr91gipz667eeflx 2425939 2425938 2026-06-11T01:07:00Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. 2425939 wikitext text/x-wiki to2czb80luj1est7b4m9e2aj2qz9zt5 2425940 2425939 2026-06-11T01:07:08Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-codfw] START helmfile.d/admin 'apply'. 2425940 wikitext text/x-wiki dt1f8z570ot5yz3p5onqzqcioduhqlw 2425941 2425940 2026-06-11T01:07:57Z Stashbot 7414 jasmine@deploy1003: helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. 2425941 wikitext text/x-wiki gyqwn0c6wup9mtr0ozmyyqm933ibxad 2425942 2425941 2026-06-11T01:08:05Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. 2425942 wikitext text/x-wiki 8lerkjc6o0k6vjxpqynqo4h7uqzjuz7 2425943 2425942 2026-06-11T01:08:33Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. 2425943 wikitext text/x-wiki qv3k6ac9o9cob480w7yadffydfigkch 2425944 2425943 2026-06-11T01:08:40Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-codfw] START helmfile.d/admin 'apply'. 2425944 wikitext text/x-wiki mxbj2r9d6bj7gb7ne70pbuwduji42i6 2425945 2425944 2026-06-11T01:09:30Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-serve-codfw] DONE helmfile.d/admin 'apply'. 2425945 wikitext text/x-wiki n9td1sq0rt2jctp96htny32bd46sq1m 2425946 2425945 2026-06-11T01:09:37Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'apply'. 2425946 wikitext text/x-wiki kh1w6w93zdf34sbudbmq1mi2gwxwi67 2425947 2425946 2026-06-11T01:10:28Z Stashbot 7414 jasmine@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'apply'. 2425947 wikitext text/x-wiki 84xw3mgs57lltxdsy1hqk2gvvcrup5u 2425948 2425947 2026-06-11T01:10:36Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2425948 wikitext text/x-wiki sq0nuw4vfmyjvztcccilx6p7tlimvvo 2425949 2425948 2026-06-11T01:11:03Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2425949 wikitext text/x-wiki 5r9p1mzdp8pxf2eqc8rta0oyx6y1vgp 2425950 2425949 2026-06-11T01:11:10Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2425950 wikitext text/x-wiki qwfhqccefyrgqs1zuzxuezenb8znv7f 2425951 2425950 2026-06-11T01:11:59Z Stashbot 7414 jasmine@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2425951 wikitext text/x-wiki 670nodiixci544arqtnpsmpb9bl21ax 2425952 2425951 2026-06-11T01:12:09Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/admin 'apply'. 2425952 wikitext text/x-wiki 9iecjwmh5y3en1fy4wpouyknbz452xc 2425953 2425952 2026-06-11T01:12:36Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2425953 wikitext text/x-wiki mxla3v6jcpuimmvfw93f52buzy19qox 2425954 2425953 2026-06-11T01:12:43Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/admin 'apply'. 2425954 wikitext text/x-wiki mjq9qcp3ltwazx2at1oaxzmpvgq9gr0 2425955 2425954 2026-06-11T01:12:50Z Stashbot 7414 jasmine@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/admin 'apply'. 2425955 wikitext text/x-wiki nl6pp18swzkfca1tj8hoq1ag1uosgp2 2425956 2425955 2026-06-11T01:18:44Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-main1009.eqiad.wmnet with OS trixie 2425956 wikitext text/x-wiki nl6nm1te87vjmd6ce3ihfd99dtwo3pw 2425957 2425956 2026-06-11T01:18:56Z Stashbot 7414 jasmine@deploy1003: helmfile [eqiad] START helmfile.d/services/eventgate-main: sync 2425957 wikitext text/x-wiki 7yd4crgvrjkk1ovj1ngay1fn9eyqhxy 2425958 2425957 2026-06-11T01:19:17Z Stashbot 7414 jasmine@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventgate-main: sync 2425958 wikitext text/x-wiki 3h7qj92i19lbizn9o8j6r9zojdjx61b 2425959 2425958 2026-06-11T01:23:45Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp2046.* 2425959 wikitext text/x-wiki fxy576el0c1a0iafic30qxx55rry175 2425961 2425959 2026-06-11T02:00:52Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2425961 wikitext text/x-wiki etwkkxkohm2tzolqpwgn4gggx2jwomy 2425962 2425961 2026-06-11T02:07:37Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 44s) 2425962 wikitext text/x-wiki noqpm6bkpx80xb2z4sqm10sycygbpdr 2425967 2425962 2026-06-11T05:16:26Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2425967 wikitext text/x-wiki ec7fq4ew0vmt0fu5twetawbnp8b069k 2425968 2425967 2026-06-11T05:16:47Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es2045: Upgrading es2045.codfw.wmnet 2425968 wikitext text/x-wiki cznz5rfm107f1dn497f2iaj2lkt6hp6 2425969 2425968 2026-06-11T05:17:09Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool es2045: Upgrading es2045.codfw.wmnet 2425969 wikitext text/x-wiki k4rdokxsfrqbpvs1r3ksahd7p1kqvpc 2425970 2425969 2026-06-11T05:17:53Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es2045.codfw.wmnet with OS trixie 2425970 wikitext text/x-wiki 8dbhc8unzwkyp8nw9fspayllxjl9sqf 2425971 2425970 2026-06-11T05:18:10Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab2002.wikimedia.org with reason: Upgrade gitlab 2425971 wikitext text/x-wiki 6f4xyh3hyniev3q8n8svbg1w1hcwcrx 2425972 2425971 2026-06-11T05:27:52Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab2002.wikimedia.org with reason: Upgrade gitlab 2425972 wikitext text/x-wiki r4rk0z512ut7t32rzwxw2mrm4kcv6gv 2425973 2425972 2026-06-11T05:28:46Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1003.wikimedia.org with reason: Upgrade gitlab 2425973 wikitext text/x-wiki t7y4amm2l6ocs8s54zkxrhdt79ahxjq 2425974 2425973 2026-06-11T05:37:56Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1003.wikimedia.org with reason: Upgrade gitlab 2425974 wikitext text/x-wiki bcxill305p1kj4bo965kyziqphc3ar8 2425976 2425974 2026-06-11T06:22:16Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 30 hosts with reason: Primary switchover s1 T426083 2425976 wikitext text/x-wiki 791l0fpk0ocphigegx0cwkg2cafl5ja 2425977 2425976 2026-06-11T06:22:28Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Set db1184 with weight 0 T426083', diff saved to https://phabricator.wikimedia.org/P94035 and previous config saved to /var/cache/conftool/dbconfig/20260611-062224-fceratto.json 2425977 wikitext text/x-wiki 35zgc4w87elcnk9vahq8rp6gocj8e6r 2425979 2425977 2026-06-11T06:29:38Z Stashbot 7414 federico3: Starting s1 eqiad failover from db1163 to db1184 - T426083 2425979 wikitext text/x-wiki iy8wy6hs23ml7rb3ma6hueayvu15ata 2425980 2425979 2026-06-11T06:29:50Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.global-read-only 2425980 wikitext text/x-wiki 7vp0x2gjrv7j6npo0qjgtmpu0qmqlxd 2425981 2425980 2026-06-11T06:30:09Z Stashbot 7414 fceratto@cumin1003: Dbctl change: Setting sections s1 as read-only for T426083: 'Maintenance until 06:15 UTC' 2425981 wikitext text/x-wiki bisu66quxezrdvxr81wcy90zec3q7qt 2425982 2425981 2026-06-11T06:30:16Z Stashbot 7414 fceratto@cumin1003: MariaDB change: Setting sections s1 as read-only for T426083: 'Maintenance until 06:15 UTC' 2425982 wikitext text/x-wiki rsi3618o3cibbn2hlhwgfqxl8fnan4k 2425983 2425982 2026-06-11T06:30:19Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.global-read-only (exit_code=0) 2425983 wikitext text/x-wiki bh69bruf6i9f7qkq2wcvzfsdvmg5fb2 2425984 2425983 2026-06-11T06:31:04Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Set s1 eqiad as read-only for maintenance - T426083', diff saved to https://phabricator.wikimedia.org/P94037 and previous config saved to /var/cache/conftool/dbconfig/20260611-063100-fceratto.json 2425984 wikitext text/x-wiki mkwi04nwvicp1lq99ku5z2lo1i3jxhh 2425985 2425984 2026-06-11T06:32:00Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.global-read-only 2425985 wikitext text/x-wiki k8ydbctydm79sugq2kaxxin4lbx6hx0 2425986 2425985 2026-06-11T06:32:04Z Stashbot 7414 fceratto@cumin1003: MariaDB change: Setting sections s1 as read-write for T426083: 'Maintenance until 06:15 UTC' 2425986 wikitext text/x-wiki ae94igcfyqjri2141w855szf81w6nxd 2425987 2425986 2026-06-11T06:32:11Z Stashbot 7414 fceratto@cumin1003: Dbctl change: Setting sections s1 as read-write for T426083: 'Maintenance until 06:15 UTC' 2425987 wikitext text/x-wiki 8o5ld097k28ma8ms55kt37j16js029q 2425988 2425987 2026-06-11T06:32:14Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.global-read-only (exit_code=0) 2425988 wikitext text/x-wiki 0qiwlkhwd336rshb7cmv10dh76rtggo 2425989 2425988 2026-06-11T06:32:55Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Set s1 eqiad as read-only for maintenance - T426083', diff saved to https://phabricator.wikimedia.org/P94039 and previous config saved to /var/cache/conftool/dbconfig/20260611-063251-fceratto.json 2425989 wikitext text/x-wiki 3bctck586pkclggkhcr07wkeui0wtn3 2425990 2425989 2026-06-11T06:33:27Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Promote db1184 to s1 primary and set section read-write T426083', diff saved to https://phabricator.wikimedia.org/P94040 and previous config saved to /var/cache/conftool/dbconfig/20260611-063323-fceratto.json 2425990 wikitext text/x-wiki qezha2qko6ljvwt47hf1qblaappcppg 2425991 2425990 2026-06-11T06:33:35Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.global-read-only 2425991 wikitext text/x-wiki nwt4fum8p3qi050nr4ya2j051cbydtj 2425992 2425991 2026-06-11T06:33:43Z Stashbot 7414 fceratto@cumin1003: MariaDB change: Setting sections s1 as read-write for T426083: 'Maintenance until 06:15 UTC' 2425992 wikitext text/x-wiki 4mwt7jk69uhj41ntgkb002uqfwl6oqi 2425993 2425992 2026-06-11T06:33:50Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.global-read-only (exit_code=0) 2425993 wikitext text/x-wiki p1ukz9n5j4rgh8vcw8wduv34oft7fyy 2425995 2425993 2026-06-11T06:40:30Z Stashbot 7414 fceratto@dns1005: START - running authdns-update 2425995 wikitext text/x-wiki easx4ak463bkakipb3qogyobia8jex0 2425996 2425995 2026-06-11T06:42:07Z Stashbot 7414 fceratto@dns1005: END - running authdns-update 2425996 wikitext text/x-wiki jnnas4n0x1c2oj4i0o672vemzfeox91 2425997 2425996 2026-06-11T06:43:23Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depool db1163 T426083', diff saved to https://phabricator.wikimedia.org/P94041 and previous config saved to /var/cache/conftool/dbconfig/20260611-064319-fceratto.json 2425997 wikitext text/x-wiki afl68rhwafb2o0ckzru9g98c8esl420 2425998 2425997 2026-06-11T06:44:26Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.pool pool db1163: Repooling 2425998 wikitext text/x-wiki fvseskwo5nawzaid9mm04yagn0u62jw 2425999 2425998 2026-06-11T06:50:29Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool es2042', diff saved to https://phabricator.wikimedia.org/P94043 and previous config saved to /var/cache/conftool/dbconfig/20260611-065027-marostegui.json 2425999 wikitext text/x-wiki qjv51xb3sl14xj5bqpoxoj8ogdky30a 2426000 2425999 2026-06-11T06:50:51Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Repool es2042', diff saved to https://phabricator.wikimedia.org/P94044 and previous config saved to /var/cache/conftool/dbconfig/20260611-065049-marostegui.json 2426000 wikitext text/x-wiki heu18hwuy3672pxi3j9z7t0ojue7kc9 2426001 2426000 2026-06-11T06:51:28Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es2045.codfw.wmnet with OS trixie 2426001 wikitext text/x-wiki 83iqqd9w5fmd0ivs9vgu3rrwhsfvz4f 2426003 2426001 2026-06-11T07:19:12Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1003.wikimedia.org with reason: Upgrade gitlab 2426003 wikitext text/x-wiki dfr3ojj0jygrmkza5v1o2fcosygb560 2426004 2426003 2026-06-11T07:29:56Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1163: Repooling 2426004 wikitext text/x-wiki kvb69risbkaachnl46f7uvsz4zky72f 2426005 2426004 2026-06-11T07:30:19Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1003.wikimedia.org with reason: Upgrade gitlab 2426005 wikitext text/x-wiki 47lic5v5u03x6ff0r62cdt2dpjt8eec 2426006 2426005 2026-06-11T07:31:09Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2426006 wikitext text/x-wiki bht3h9r9ojbi7pq1vz39vxxb73w6pd4 2426007 2426006 2026-06-11T07:31:12Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab2002.wikimedia.org with reason: Upgrade gitlab 2426007 wikitext text/x-wiki 4jqg5rujy0m5mk4o7fkvfzba1qeklj3 2426008 2426007 2026-06-11T07:31:29Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db1219: Upgrading db1219.eqiad.wmnet 2426008 wikitext text/x-wiki 4k7r6gbi48grk16pvkfd99zbvg6eoag 2426009 2426008 2026-06-11T07:31:39Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2426009 wikitext text/x-wiki 11bw2dgx2d4zxo2j3j00vxweiod2zs7 2426010 2426009 2026-06-11T07:32:00Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db2176: Upgrading db2176.codfw.wmnet 2426010 wikitext text/x-wiki at4xjh3dgglkq24129xzjle27o6r1eu 2426011 2426010 2026-06-11T07:32:10Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1219: Upgrading db1219.eqiad.wmnet 2426011 wikitext text/x-wiki f4r2olmqr9re2v54ysyk82f78dk115r 2426012 2426011 2026-06-11T07:32:22Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2176: Upgrading db2176.codfw.wmnet 2426012 wikitext text/x-wiki cdm4yxeo6a0jj896drazb4dezat7eds 2426013 2426012 2026-06-11T07:35:04Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host rdb1013.eqiad.wmnet with OS trixie 2426013 wikitext text/x-wiki qnuzf7fbh9egr92vgkumnaoz7uuk59i 2426015 2426013 2026-06-11T07:42:28Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab2002.wikimedia.org with reason: Upgrade gitlab 2426015 wikitext text/x-wiki j7oraqu194bfnu9ln1u1ywmunqrjddr 2426016 2426015 2026-06-11T07:43:10Z Stashbot 7414 moritzm: imported Jenkins 2.541.3 for thirdparty/ci (Bullseye) and thirdparty/jenkins (Bookworm, Trixie) 2426016 wikitext text/x-wiki 5g4rkn7e5t6bjhok3kchmnogj8pagku 2426017 2426016 2026-06-11T07:43:41Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db1219.eqiad.wmnet with OS trixie 2426017 wikitext text/x-wiki gtvg5e49c46jria85l8o802lh6mv4pu 2426018 2426017 2026-06-11T07:46:05Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db2176.codfw.wmnet with OS trixie 2426018 wikitext text/x-wiki k4my3403cz41k4phcsqnon000jchwd8 2426019 2426018 2026-06-11T07:47:32Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply 2426019 wikitext text/x-wiki i4y91iwspmmedbraax2htxfijk41jhv 2426020 2426019 2026-06-11T07:47:39Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply 2426020 wikitext text/x-wiki m4qcqi55drwfc80fxfdmejd0d1w12aw 2426021 2426020 2026-06-11T07:49:35Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti5004.eqsin.wmnet 2426021 wikitext text/x-wiki 0v5jp3zqdnms0bxth6o3kg5zp3wo3ds 2426022 2426021 2026-06-11T07:49:50Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply 2426022 wikitext text/x-wiki 5eg1jyl4mkit6mgpufszbk8gqnesmhk 2426023 2426022 2026-06-11T07:49:54Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply 2426023 wikitext text/x-wiki 2ne69b4shs1ytc64ytn67p7llvnhnl7 2426024 2426023 2026-06-11T07:50:19Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on rdb1013.eqiad.wmnet with reason: host reimage 2426024 wikitext text/x-wiki 6mdutwf3d5jaj5hr7r47hc122dwr9vk 2426025 2426024 2026-06-11T07:54:34Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on rdb1013.eqiad.wmnet with reason: host reimage 2426025 wikitext text/x-wiki af7nbi2hu6t7p1lyepb4kayi3ft4j6l 2426026 2426025 2026-06-11T07:56:51Z Stashbot 7414 marostegui: install mariadb 10.11.17 on pc1 T427345 2426026 wikitext text/x-wiki dpqr6fwoefgics15t8lmpomwwwwuv79 2426027 2426026 2026-06-11T07:58:46Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1219.eqiad.wmnet with reason: host reimage 2426027 wikitext text/x-wiki ln46lcxhbt2775nyhjvxduofp3vinkm 2426028 2426027 2026-06-11T07:59:23Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti5004.eqsin.wmnet 2426028 wikitext text/x-wiki jw250zhsbcjj69svra778t2fntj2vn2 2426029 2426028 2026-06-11T08:03:54Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool pc1021: Migration to 10.11.17 T427345 2426029 wikitext text/x-wiki 8jhyd6l64n6kjk00xqytmnu6hgxvds4 2426030 2426029 2026-06-11T08:03:57Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2426030 wikitext text/x-wiki 8i4x2w0m1rsb8tm7felr8q45pqdvkvz 2426031 2426030 2026-06-11T08:03:59Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) 2426031 wikitext text/x-wiki f5i5gy52qs9szqir5u85ws137kpt33o 2426032 2426031 2026-06-11T08:04:03Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool pc1021: Migration to 10.11.17 T427345 2426032 wikitext text/x-wiki 2rtrvc5tv9zsn9xxa62pr69vjoyvp14 2426033 2426032 2026-06-11T08:04:31Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2176.codfw.wmnet with reason: host reimage 2426033 wikitext text/x-wiki i94nivrr0k2l6tpd22qbkgaubjhyhda 2426034 2426033 2026-06-11T08:05:03Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool pc1021: Migration to 10.11.17 T427345 2426034 wikitext text/x-wiki 2yqkgirlv8nsh7rmto95zuu3bxg3vx9 2426035 2426034 2026-06-11T08:05:07Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool pc1021: Migration to 10.11.17 T427345 2426035 wikitext text/x-wiki nf4kc80btanybpsmgd3jhvzrz0z5pea 2426036 2426035 2026-06-11T08:05:09Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.addnode for new host ganeti5004.eqsin.wmnet to cluster eqsin02 and group 01 2426036 wikitext text/x-wiki t7ye5ts60moyc76d6yace3mq7s2xxgs 2426037 2426036 2026-06-11T08:05:20Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1219.eqiad.wmnet with reason: host reimage 2426037 wikitext text/x-wiki jxwds92s5yr67haxw09rmatpinngmyl 2426038 2426037 2026-06-11T08:05:49Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on pc2021.codfw.wmnet,pc1021.eqiad.wmnet with reason: upgrade 2426038 wikitext text/x-wiki l6w6grmprzbyo28eidy81meoanjij61 2426039 2426038 2026-06-11T08:06:48Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply 2426039 wikitext text/x-wiki bs901x53u58tuyvy60ozz3rmmc4ei22 2426040 2426039 2026-06-11T08:06:51Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply 2426040 wikitext text/x-wiki ad94ngdnxqz9tihymtg0qlv3bljzgwb 2426041 2426040 2026-06-11T08:06:54Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.addnode (exit_code=0) for new host ganeti5004.eqsin.wmnet to cluster eqsin02 and group 01 2426041 wikitext text/x-wiki c4qqwdsj0tl7f8vcsjyhvcf6rs984tr 2426042 2426041 2026-06-11T08:08:56Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host rdb1013.eqiad.wmnet with OS trixie 2426042 wikitext text/x-wiki 0r85i3amzelh6hjwjioyvexntpw3z97 2426043 2426042 2026-06-11T08:09:35Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2176.codfw.wmnet with reason: host reimage 2426043 wikitext text/x-wiki tembhx1i0e78f0sggnr1vd0l8nbhomu 2426044 2426043 2026-06-11T08:11:23Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply 2426044 wikitext text/x-wiki 5uw0qbfx5ck2b9jlva39ydj738scnks 2426045 2426044 2026-06-11T08:11:25Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply 2426045 wikitext text/x-wiki 4hhwqg8ka4x3fq0lrygwnwh610fid7z 2426046 2426045 2026-06-11T08:14:58Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply 2426046 wikitext text/x-wiki 3gsw8no79kvryi177tpiuyivovup8mw 2426047 2426046 2026-06-11T08:15:01Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply 2426047 wikitext text/x-wiki 2als67rdpppiooi1zuaix5ff10glv2s 2426048 2426047 2026-06-11T08:17:03Z Stashbot 7414 moritzm: installing PHP 8.2 security updates 2426048 wikitext text/x-wiki rmku3yfinsuplss5l3tovayz73d94l2 2426049 2426048 2026-06-11T08:22:03Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1219.eqiad.wmnet with OS trixie 2426049 wikitext text/x-wiki bfs3no5bucqpk9tsh1zuw6bszwpxh41 2426050 2426049 2026-06-11T08:22:21Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist mwscript.dblist extensions/Translate/scripts/ttmserver-export.php --ttmserver eqiad-test # T425377 populating ttmserver index on test cluster to estimate time required for the release (dblist: https://phabricator.wikimedia.org/P94051) 2426050 wikitext text/x-wiki 1fq027qxq9jhw9wney7yjuh8xyiuaoi 2426051 2426050 2026-06-11T08:23:53Z Stashbot 7414 jnuche@deploy1003: Started deploy [releng/jenkins-deploy@6200ab1] (releasing): Testing upgrade for T428823 2426051 wikitext text/x-wiki etak401edpaqfeegztaz9uluzw1w144 2426052 2426051 2026-06-11T08:24:57Z Stashbot 7414 jnuche@deploy1003: Finished deploy [releng/jenkins-deploy@6200ab1] (releasing): Testing upgrade for T428823 (duration: 01m 17s) 2426052 wikitext text/x-wiki 0ojawy5eiy1void1v0yuqxt2k4h8boq 2426053 2426052 2026-06-11T08:25:04Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist mwscript.dblist extensions/Translate/scripts/ttmserver-export.php --ttmserver eqiad-test # T425377 populating ttmserver index on test cluster to estimate time required for the release (dblist: https://phabricator.wikimedia.org/P94052) 2426053 wikitext text/x-wiki 85yy6gmkufvrc356gm4xih2etevq6t9 2426054 2426053 2026-06-11T08:25:06Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool pc1021: Migration to 10.11.17 2426054 wikitext text/x-wiki puyq7zw7xd2o10s7pvkko6jcoawgyo0 2426055 2426054 2026-06-11T08:25:11Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2426055 wikitext text/x-wiki hwuawn6egtm5myuamxfw8y1i796im4u 2426056 2426055 2026-06-11T08:25:20Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) 2426056 wikitext text/x-wiki hze4kavhsuuorz16y0pxvfuk69byu2m 2426057 2426056 2026-06-11T08:25:28Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool pc1021: Migration to 10.11.17 2426057 wikitext text/x-wiki dlzlcjhhakkpn93vx69emnznebtkxez 2426058 2426057 2026-06-11T08:27:03Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2176.codfw.wmnet with OS trixie 2426058 wikitext text/x-wiki l728z66c6o6cz51a0zpnm82he2y8rky 2426059 2426058 2026-06-11T08:29:41Z Stashbot 7414 jnuche@deploy1003: Started deploy [releng/jenkins-deploy@6200ab1] (releasing): T428823 2426059 wikitext text/x-wiki 6g45p1ix1hg84cihh4ldkn8jb1zyhh5 2426060 2426059 2026-06-11T08:30:52Z Stashbot 7414 jnuche@deploy1003: Finished deploy [releng/jenkins-deploy@6200ab1] (releasing): T428823 (duration: 01m 18s) 2426060 wikitext text/x-wiki d0s0t4to21f1w9j727vjf8zgeglsftl 2426061 2426060 2026-06-11T08:34:00Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist mwscript.dblist extensions/Translate/scripts/ttmserver-export.php --ttmserver eqiad-test # T425377 populating ttmserver index on test cluster to estimate time required for the release (dblist: https://phabricator.wikimedia.org/P94053) 2426061 wikitext text/x-wiki f5wgho16wvk174uro7ovf7qm56kuvyu 2426062 2426061 2026-06-11T08:34:29Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1219: Migration of db1219.eqiad.wmnet completed 2426062 wikitext text/x-wiki 1269ef8nc3hyw1z79evt1b78vb572hw 2426063 2426062 2026-06-11T08:34:46Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist mwscript.dblist extensions/Translate/scripts/ttmserver-export.php --ttmserver eqiad-test # T425377 populating ttmserver index on test cluster to estimate time required for the release (dblist: https://phabricator.wikimedia.org/P94055) 2426063 wikitext text/x-wiki bnkgc8vji4fu7wpyd5ckagqbmap2wap 2426064 2426063 2026-06-11T08:39:55Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db2176: Migration of db2176.codfw.wmnet completed 2426064 wikitext text/x-wiki cviiwmcfwdov3211u0xxtwidxwf379e 2426067 2426064 2026-06-11T08:57:37Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es2045.codfw.wmnet with OS bookworm 2426067 wikitext text/x-wiki 70ow4tuu2toso0cnb9l3pyypq8wi31i 2426070 2426067 2026-06-11T09:11:27Z Stashbot 7414 claime: cumin -x 'A:swift-fe' "disable-puppet 'Disabling puppet for ratelimit deploy - cgoubert'" 2426070 wikitext text/x-wiki jutd79m3q2xd5o4sua444dqck6rwcwn 2426073 2426070 2026-06-11T09:19:58Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1219: Migration of db1219.eqiad.wmnet completed 2426073 wikitext text/x-wiki nri6pfe4ja6dwie768ne4459s5sfi2n 2426074 2426073 2026-06-11T09:20:00Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2426074 wikitext text/x-wiki 3e2sb5dpgiveb0ed0mudjt1n9ot9vyh 2426075 2426074 2026-06-11T09:25:25Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2176: Migration of db2176.codfw.wmnet completed 2426075 wikitext text/x-wiki edt8n7ikxa1z53e3shsc8z6nj7h8ld2 2426076 2426075 2026-06-11T09:25:28Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2426076 wikitext text/x-wiki rjc08v4ust7ni7vvrlsa723q0pl8gpt 2426077 2426076 2026-06-11T09:26:22Z Stashbot 7414 marostegui@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host es2045.codfw.wmnet with OS bookworm 2426077 wikitext text/x-wiki 9ny3z5lf009wh5oofk5gxo0oyvi9cy2 2426078 2426077 2026-06-11T09:26:43Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es2045.codfw.wmnet with OS trixie 2426078 wikitext text/x-wiki s42txts1gs7qygi4en384ooql9re4ou 2426080 2426078 2026-06-11T09:37:52Z Stashbot 7414 elukey: uploaded spicerack_12.8.0 to apt.wikimedia.org bookworm-wikimedia,trixie-wikimedia 2426080 wikitext text/x-wiki 6w6rngb1lex79ticdnsj1zx35cqnosg 2426081 2426080 2026-06-11T09:42:02Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es2045.codfw.wmnet with reason: host reimage 2426081 wikitext text/x-wiki hg6zp30ts9r94aa92gmda5p19cjpjlj 2426082 2426081 2026-06-11T09:43:22Z Stashbot 7414 jiji@deploy1003: Started scap sync-world: Backport for [[gerrit:1300580|ProductionServices.php: switch filebackend.php back to rdb1013 (T291916 T419976)]] 2426082 wikitext text/x-wiki c3gnnkvdl7o6do20kxsv0cnnxawk2g5 2426084 2426082 2026-06-11T09:45:42Z Stashbot 7414 jiji@deploy1003: jiji: Backport for [[gerrit:1300580|ProductionServices.php: switch filebackend.php back to rdb1013 (T291916 T419976)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2426084 wikitext text/x-wiki qrqae4kn1arx55bnhhgsmv0ylwwm5dd 2426085 2426084 2026-06-11T09:46:21Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es2045.codfw.wmnet with reason: host reimage 2426085 wikitext text/x-wiki qiiuuodt1goctqxgvzb0cosmribzt3a 2426086 2426085 2026-06-11T09:54:38Z Stashbot 7414 jiji@deploy1003: jiji: Continuing with deployment 2426086 wikitext text/x-wiki fr8lyk673xpt3uqagnc2ccb17o4onbe 2426087 2426086 2026-06-11T09:59:04Z Stashbot 7414 jiji@deploy1003: Finished scap sync-world: Backport for [[gerrit:1300580|ProductionServices.php: switch filebackend.php back to rdb1013 (T291916 T419976)]] (duration: 15m 41s) 2426087 wikitext text/x-wiki 7srcsc6i56i70jf0b2ta7e2kk8tsoig 2426088 2426087 2026-06-11T10:01:08Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2426088 wikitext text/x-wiki dbs8xtjnx1vexf7ui2hmxzb0110yywj 2426089 2426088 2026-06-11T10:01:47Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool es2046', diff saved to https://phabricator.wikimedia.org/P94068 and previous config saved to /var/cache/conftool/dbconfig/20260611-100145-marostegui.json 2426089 wikitext text/x-wiki mhuae9q8npyyvgd4z3zhop2lyi5u1dx 2426090 2426089 2026-06-11T10:02:23Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Repool es2046', diff saved to https://phabricator.wikimedia.org/P94069 and previous config saved to /var/cache/conftool/dbconfig/20260611-100221-marostegui.json 2426090 wikitext text/x-wiki 1vzxhas3jyes22ef7ls4md9ivk62e3t 2426091 2426090 2026-06-11T10:02:54Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2426091 wikitext text/x-wiki j3gdvtr4v7zchb6266uxr76wbiyngg0 2426092 2426091 2026-06-11T10:05:05Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2426092 wikitext text/x-wiki aln1ou8bd36j2nhodxrmqrwwrrqhpbi 2426093 2426092 2026-06-11T10:06:30Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2426093 wikitext text/x-wiki 2k9j92liaho2bs3g2tly3oj9nh30ap1 2426094 2426093 2026-06-11T10:08:28Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2426094 wikitext text/x-wiki 6gbjg5cb5yhs3sphu91aeyp9ot0whk9 2426095 2426094 2026-06-11T10:09:03Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2426095 wikitext text/x-wiki rdq3u2ss3sg1ibe6ex0je6g5uxto79p 2426096 2426095 2026-06-11T10:09:14Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es2045.codfw.wmnet with OS trixie 2426096 wikitext text/x-wiki 911kb7d44wvnt89gvvq9m0nve6nq0gw 2426098 2426096 2026-06-11T10:10:45Z Stashbot 7414 hnowlan@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2426098 wikitext text/x-wiki dnnstaaruxupsg18i4dwn59b854ysez 2426099 2426098 2026-06-11T10:10:52Z Stashbot 7414 hnowlan@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2426099 wikitext text/x-wiki 5306hd7yv1cw3zkjc5ndp0ea40jeuyo 2426100 2426099 2026-06-11T10:18:29Z Stashbot 7414 daniel@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2426100 wikitext text/x-wiki j5ofbb8pzownfmygyem5gq1hhr3wggs 2426101 2426100 2026-06-11T10:19:01Z Stashbot 7414 daniel@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2426101 wikitext text/x-wiki serrfcksrz5o6wr3szb1ts10vu0w4bz 2426102 2426101 2026-06-11T10:20:11Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1300734|HCaptcha: Return 'forceshowcaptcha' error when CAPTCHA forced (T426476)]], [[gerrit:1300727|hCaptcha: Enable for DiscussionTools on group 1 wikis (T426039)]] 2426102 wikitext text/x-wiki i573e03o1tye5cqif8iu4sb28oyysrd 2426103 2426102 2026-06-11T10:22:21Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1300734|HCaptcha: Return 'forceshowcaptcha' error when CAPTCHA forced (T426476)]], [[gerrit:1300727|hCaptcha: Enable for DiscussionTools on group 1 wikis (T426039)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2426103 wikitext text/x-wiki c6jqtatsjishbu0l1l7hggvufb3xfib 2426104 2426103 2026-06-11T10:23:05Z Stashbot 7414 daniel@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply 2426104 wikitext text/x-wiki jhfq6pquhub7cck4bz8rm5y7o3nfxui 2426105 2426104 2026-06-11T10:23:35Z Stashbot 7414 daniel@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply 2426105 wikitext text/x-wiki 8mpv5e6an513phr2hebbhvhx66a7qdj 2426106 2426105 2026-06-11T10:26:49Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2426106 wikitext text/x-wiki rqysz11u2puiqeombyu3skv75ua47qj 2426108 2426106 2026-06-11T10:31:13Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1300734|HCaptcha: Return 'forceshowcaptcha' error when CAPTCHA forced (T426476)]], [[gerrit:1300727|hCaptcha: Enable for DiscussionTools on group 1 wikis (T426039)]] (duration: 11m 01s) 2426108 wikitext text/x-wiki ngbzvyfgxj2pniqyrzp88cqfzt3ybwg 2426110 2426108 2026-06-11T10:32:28Z Stashbot 7414 daniel@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply 2426110 wikitext text/x-wiki ogy5lnfrc2oqsx6eaoi1vz5e91ktmft 2426111 2426110 2026-06-11T10:33:08Z Stashbot 7414 daniel@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply 2426111 wikitext text/x-wiki 7xseo7fz7jakf17xob4epz5dpct14px 2426113 2426111 2026-06-11T10:40:27Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2426113 wikitext text/x-wiki l85fs2rju7v9bgrojg1jb28k864leru 2426114 2426113 2026-06-11T10:40:30Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2426114 wikitext text/x-wiki 2hswbpjb5zmovph1z45sst7wfanute8 2426117 2426114 2026-06-11T10:48:17Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2426117 wikitext text/x-wiki jw6xkya3hjh08nrfjqr5u4tbgmuhiqx 2426118 2426117 2026-06-11T10:48:37Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db1232: Upgrading db1232.eqiad.wmnet 2426118 wikitext text/x-wiki jg42kc5ie2nfhzqi6p458unzp0h9i8c 2426119 2426118 2026-06-11T10:48:58Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1232: Upgrading db1232.eqiad.wmnet 2426119 wikitext text/x-wiki r6bg7i75l61hv4n85dwzouv5iizbq3y 2426120 2426119 2026-06-11T10:52:14Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db1232.eqiad.wmnet with OS trixie 2426120 wikitext text/x-wiki a2l0mm03z3huw78s4s1lcskro93y1w4 2426121 2426120 2026-06-11T10:52:17Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2426121 wikitext text/x-wiki b18zpg65sqmd8sv7ly2ev812b1wet43 2426122 2426121 2026-06-11T10:52:37Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db2188: Upgrading db2188.codfw.wmnet 2426122 wikitext text/x-wiki nfet6cf4fyaqwv16j58x9qg5riclatx 2426123 2426122 2026-06-11T10:52:59Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2188: Upgrading db2188.codfw.wmnet 2426123 wikitext text/x-wiki s09659063t74uhstg3eqv9gggz7t719 2426124 2426123 2026-06-11T10:55:37Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db2188.codfw.wmnet with OS trixie 2426124 wikitext text/x-wiki j1jxbfhntqfagsy8dcw81w5bthi6ev0 2426125 2426124 2026-06-11T10:58:41Z Stashbot 7414 hnowlan@deploy1003: helmfile [codfw] START helmfile.d/services/thumbor: apply 2426125 wikitext text/x-wiki naljcag9vgdri5pz99cekt74pbpadmo 2426126 2426125 2026-06-11T10:59:44Z Stashbot 7414 jmm@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2426126 wikitext text/x-wiki s2gedcc609dm9ge70ttvzlcew4jgyxy 2426127 2426126 2026-06-11T11:00:01Z Stashbot 7414 jmm@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2426127 wikitext text/x-wiki brpc4j3ihcahxkjlgbvu7o7xy8q7lyd 2426128 2426127 2026-06-11T11:00:32Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reboot-single for host testreduce1002.eqiad.wmnet 2426128 wikitext text/x-wiki 40tku4f86o08mp3zgvm305gyff8o6lg 2426129 2426128 2026-06-11T11:00:45Z Stashbot 7414 hnowlan@deploy1003: helmfile [codfw] DONE helmfile.d/services/thumbor: apply 2426129 wikitext text/x-wiki gi7eilsj5xyzhci26deru5aks0kc9nj 2426130 2426129 2026-06-11T11:03:00Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on db1262.eqiad.wmnet with reason: crash 2426130 wikitext text/x-wiki 2p45la57njb1tj8gvjt9hm687457g4h 2426131 2426130 2026-06-11T11:04:08Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2426131 wikitext text/x-wiki l9z3eze5kdri1rxmw41qbzq5jkli6nt 2426132 2426131 2026-06-11T11:04:29Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host testreduce1002.eqiad.wmnet 2426132 wikitext text/x-wiki pkxy01j5qd22h07eyd3n0ss6rv6ubc9 2426133 2426132 2026-06-11T11:04:33Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reboot-single for host mc-misc2001.codfw.wmnet 2426133 wikitext text/x-wiki 9jg96mab0sqfftrfmxqja87gismn8t5 2426134 2426133 2026-06-11T11:05:10Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2426134 wikitext text/x-wiki s2uqmays6gmrvxayeocpu92232c5mnh 2426135 2426134 2026-06-11T11:08:13Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5006.eqsin.wmnet 2426135 wikitext text/x-wiki 1myld2kq0js6d81adhfbj2klby2fnzv 2426136 2426135 2026-06-11T11:09:12Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1232.eqiad.wmnet with reason: host reimage 2426136 wikitext text/x-wiki 835mk7i4ju6p1a9y7fuf9b169ga75t8 2426137 2426136 2026-06-11T11:09:51Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc-misc2001.codfw.wmnet 2426137 wikitext text/x-wiki c65ob2hswkko6na8xvz77w3qlvl2dgv 2426138 2426137 2026-06-11T11:11:09Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] START helmfile.d/services/thumbor: apply 2426138 wikitext text/x-wiki 5ka6ocxj1w8bmychi0ohztz5u0krj9p 2426139 2426138 2026-06-11T11:12:27Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti5006.eqsin.wmnet 2426139 wikitext text/x-wiki b44fgg5rk9m8f44ipxm4cru9q721r5k 2426140 2426139 2026-06-11T11:12:47Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5006.eqsin.wmnet 2426140 wikitext text/x-wiki cunpypwjvly61m75tsslenyc90sipem 2426141 2426140 2026-06-11T11:13:15Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] DONE helmfile.d/services/thumbor: apply 2426141 wikitext text/x-wiki d68jvq55a6k94to5leumvi1h13rfvdo 2426142 2426141 2026-06-11T11:13:40Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reboot-single for host mc-misc2002.codfw.wmnet 2426142 wikitext text/x-wiki 9bs08xmp42unwzjbfntuifk56c9o2o5 2426143 2426142 2026-06-11T11:14:28Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1232.eqiad.wmnet with reason: host reimage 2426143 wikitext text/x-wiki lyg3ph8jyzz0gghu32yfouxwgkgwf5t 2426146 2426143 2026-06-11T11:15:09Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2188.codfw.wmnet with reason: host reimage 2426146 wikitext text/x-wiki aso6sq6elvrh20nfmd3eg8w3yp7m3of 2426147 2426146 2026-06-11T11:17:02Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1300749|HCaptcha: Return 'forceshowcaptcha' error when CAPTCHA forced (T426476)]], [[gerrit:1300751|hCaptcha: Enable for DiscussionTools on all wikis (T426039)]] 2426147 wikitext text/x-wiki nwqq823lz7bxo4y3rtqxlkdqpdsq5xz 2426149 2426147 2026-06-11T11:19:00Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2188.codfw.wmnet with reason: host reimage 2426149 wikitext text/x-wiki fsytn38vljiivk6vj0z1ya6zn4cc9bq 2426150 2426149 2026-06-11T11:19:13Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1300749|HCaptcha: Return 'forceshowcaptcha' error when CAPTCHA forced (T426476)]], [[gerrit:1300751|hCaptcha: Enable for DiscussionTools on all wikis (T426039)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2426150 wikitext text/x-wiki 10tilzlbua6u4r3ztqv2dh1wphzxt8i 2426151 2426150 2026-06-11T11:21:19Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2426151 wikitext text/x-wiki 1tmp780nbj0go6kby70ndb5yl98i81i 2426154 2426151 2026-06-11T11:25:41Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1300749|HCaptcha: Return 'forceshowcaptcha' error when CAPTCHA forced (T426476)]], [[gerrit:1300751|hCaptcha: Enable for DiscussionTools on all wikis (T426039)]] (duration: 08m 38s) 2426154 wikitext text/x-wiki l9co3t3tvloashcong10b3yglefel9t 2426155 2426154 2026-06-11T11:27:33Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc-misc2002.codfw.wmnet 2426155 wikitext text/x-wiki stb7v4ix5cxhl5bgr4e7sprwxm0lb0k 2426156 2426155 2026-06-11T11:32:22Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1232.eqiad.wmnet with OS trixie 2426156 wikitext text/x-wiki cvuts2hzflxievy85tpccys08utjolw 2426157 2426156 2026-06-11T11:32:43Z Stashbot 7414 jmm@deploy1003: helmfile [staging] START helmfile.d/services/proton: apply 2426157 wikitext text/x-wiki n91fi9jlliae1sa8mm0b25hhp0qkpml 2426158 2426157 2026-06-11T11:33:36Z Stashbot 7414 jmm@deploy1003: helmfile [staging] DONE helmfile.d/services/proton: apply 2426158 wikitext text/x-wiki mzqpj66t1j0bsxvn9v9ajo75a0hjf6z 2426159 2426158 2026-06-11T11:34:11Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.decommission for hosts rdb2010.codfw.wmnet 2426159 wikitext text/x-wiki mmwdloubbjb7f5hpdik71q1jbv2cwy5 2426160 2426159 2026-06-11T11:34:16Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.decommission for hosts rdb2008.codfw.wmnet 2426160 wikitext text/x-wiki kl9lhnsiybyjn1heex8vmttfrpo7l27 2426161 2426160 2026-06-11T11:35:02Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.decommission for hosts rdb1012.eqiad.wmnet 2426161 wikitext text/x-wiki 3iun1weueh5chzl4s56ldxsbdiih1or 2426162 2426161 2026-06-11T11:35:51Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2188.codfw.wmnet with OS trixie 2426162 wikitext text/x-wiki inosymyug1znknfjzvtbnvo6nz5scyj 2426163 2426162 2026-06-11T11:36:26Z Stashbot 7414 jmm@deploy1003: helmfile [codfw] START helmfile.d/services/proton: apply 2426163 wikitext text/x-wiki hah2rg9cut6x0471c4ey2npta38f5go 2426164 2426163 2026-06-11T11:37:45Z Stashbot 7414 jmm@deploy1003: helmfile [codfw] DONE helmfile.d/services/proton: apply 2426164 wikitext text/x-wiki ozmq3uwxo3nv7worvk5667c8i19abdn 2426165 2426164 2026-06-11T11:37:59Z Stashbot 7414 jmm@deploy1003: helmfile [eqiad] START helmfile.d/services/proton: apply 2426165 wikitext text/x-wiki i4yt9bsdyfaqibwi98jxgsh6vw804b4 2426166 2426165 2026-06-11T11:38:49Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.dns.netbox 2426166 wikitext text/x-wiki 9098y2jhwwbcjz34tw61fdf38yl8hbt 2426168 2426166 2026-06-11T11:43:05Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1232: Migration of db1232.eqiad.wmnet completed 2426168 wikitext text/x-wiki c5mbnx815unpho351b0d91av0tqsn1b 2426169 2426168 2026-06-11T11:43:11Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: rdb2010.codfw.wmnet decommissioned, removing all IPs except the asset tag one - jiji@cumin1003" 2426169 wikitext text/x-wiki knpfxuy6cw2wzs3gtm8j7v27sqwxidw 2426170 2426169 2026-06-11T11:43:53Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.dns.netbox 2426170 wikitext text/x-wiki 522xebuvg5zmz8ut9cla173myuodvo7 2426171 2426170 2026-06-11T11:44:31Z Stashbot 7414 jmm@deploy1003: helmfile [eqiad] DONE helmfile.d/services/proton: apply 2426171 wikitext text/x-wiki i4ur1bovyrx2i6vpxllooevi05na15p 2426172 2426171 2026-06-11T11:46:20Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db2188: Migration of db2188.codfw.wmnet completed 2426172 wikitext text/x-wiki e69ldd8n1jdrxepzadydirig814scez 2426173 2426172 2026-06-11T11:46:45Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2426173 wikitext text/x-wiki 210bdmfnko84cl3gynlr866w7rkw8uj 2426174 2426173 2026-06-11T11:46:47Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts rdb2008.codfw.wmnet 2426174 wikitext text/x-wiki ey02zuoefwqeeh30z5urs61nd71q6qc 2426175 2426174 2026-06-11T11:46:51Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.dns.netbox 2426175 wikitext text/x-wiki ns3wicksyqndmkl0epxkwbte2pa43y0 2426178 2426175 2026-06-11T11:49:10Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: rdb2010.codfw.wmnet decommissioned, removing all IPs except the asset tag one - jiji@cumin1003" 2426178 wikitext text/x-wiki ll4irnxho6w7rrcq52898dvia388bqi 2426179 2426178 2026-06-11T11:49:13Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2426179 wikitext text/x-wiki l318exv44amv827byid7gcdsku7nn6a 2426180 2426179 2026-06-11T11:49:15Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts rdb2010.codfw.wmnet 2426180 wikitext text/x-wiki jfapfabey1xa6cauim5hvzb4m9i4h3s 2426181 2426180 2026-06-11T11:49:53Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2426181 wikitext text/x-wiki f3zpfpas47u3b6kpa50hf6i5uxx1w5l 2426182 2426181 2026-06-11T11:49:55Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts rdb1012.eqiad.wmnet 2426182 wikitext text/x-wiki i0sp5z6ypr4kif1g8pm85mbpf0ytvga 2426183 2426182 2026-06-11T11:56:48Z Stashbot 7414 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1300764|Remove GrowthExperiments extension from closed wikis (T428884)]] 2426183 wikitext text/x-wiki m8gevh23zkpbhb3rgmqi2i9otwi9ckl 2426184 2426183 2026-06-11T11:58:55Z Stashbot 7414 urbanecm@deploy1003: urbanecm: Backport for [[gerrit:1300764|Remove GrowthExperiments extension from closed wikis (T428884)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2426184 wikitext text/x-wiki gszcn1xphss5wh9edhurtmbt78fc2e5 3 11976 2425862 1777160 2026-06-10T20:56:57Z BryanDavis 1604 /* Interested in adding your toolviews visualization to toolviews.toolforge.org? */ new section 2425862 wikitext text/x-wiki nxjyadhorco9jiwijbraxpj300khiix 0 17290 2425717 2423877 2026-06-10T16:06:50Z Stashbot 7414 James_F: Docker: Provide quibble-bookworm, for T362705 2425717 wikitext text/x-wiki 8b1serv3bgfebwwpk0znqoxbh5p5d9k 2426065 2425717 2026-06-11T08:40:19Z Stashbot 7414 hashar: gerrit: deleted mediawiki/core branch "development" that pointed to 7f622781cc31053b121f6f4ddbff506cba10d38e (which is contained by master). Had probably been created by a direct push. 2426065 wikitext text/x-wiki iv5ajmz05zfrj4p91s1dvey9smpdoir 2426066 2426065 2026-06-11T08:56:59Z Stashbot 7414 hashar: Converting REL1_39 branches to tags # T428864 2426066 wikitext text/x-wiki jvafkhwzf75ubnqunfpikoog635rzvb 2426069 2426066 2026-06-11T09:05:21Z Stashbot 7414 hashar: Converting REL1_40 branches to tags # T428864 2426069 wikitext text/x-wiki 8xrqnam357isscu13wp412x78xr69fc 2426071 2426069 2026-06-11T09:18:32Z Stashbot 7414 hashar: Converting REL1_41 branches to tags # T428864 2426071 wikitext text/x-wiki qltzrb61jmytvf18w5v8m8mgodgk5zs 2426072 2426071 2026-06-11T09:18:58Z Stashbot 7414 hashar: Converting REL1_42 branches to tags # T428864 2426072 wikitext text/x-wiki 5lpzsozqrpmkvc0g84ly2lt480471fa 2426083 2426072 2026-06-11T09:45:11Z Stashbot 7414 hashar: Converted mediawiki/core branches REL1_39, REL1_40, REL1_41, REL1_42 to tags # T428864 2426083 wikitext text/x-wiki fcqc167gtjjh8ruoky1ajjx9avtavin 2 441756 2425511 2412197 2026-06-10T12:32:03Z Tommy Kronkvist 11506 User statistics. 2425511 wikitext text/x-wiki 50ca1tkyifc7d96k06ty92lcrv5dmnp 498 443628 2425487 2425050 2026-06-10T12:13:30Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27275339407 (https://github.com/cluebotng/component-configs/commits/2d4571e6f74a6269bb7fbd7a03cc1cd1114f0a11) 2425487 wikitext text/x-wiki 7ljqk1k3p4z0ru4fgb88p887pd0zeao 2425494 2425487 2026-06-10T12:17:38Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27275397723 (https://github.com/cluebotng/component-configs/commits/33d203fb0e6b88ac6dc34e82ee630f7d4e6fdb56) 2425494 wikitext text/x-wiki qrioapoqjl6902mldq953lwchtxhvgl 2425499 2425494 2026-06-10T12:21:06Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27275411819 (https://github.com/cluebotng/component-configs/commits/869dfb8d1487914e36184a9d1c5aae1e26dbba01) 2425499 wikitext text/x-wiki 19u9o6kq0fecx1gorko8yvwybkmqod6 2425517 2425499 2026-06-10T12:37:32Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27276510139 (https://github.com/cluebotng/component-configs/commits/4442f7413e6335776bd1b8b0a660e20ae1256ae1) 2425517 wikitext text/x-wiki l5zws2h3i3czgajvoo0gv1mxh26o673 2425521 2425517 2026-06-10T12:40:24Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27276699371 (https://github.com/cluebotng/component-configs/commits/8be9293c4b541d74d39482efd21163eb36cda6bd) 2425521 wikitext text/x-wiki 6ghoi53s7r32psy97ccq0xq6b54g3dh 2425536 2425521 2026-06-10T12:56:59Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27277788838 (https://github.com/cluebotng/component-configs/commits/39ecf0765b86afbcbd1be02c9f9a5519245ab884) 2425536 wikitext text/x-wiki jsg2y14z8que39d73t40ckkzq3pcxzl 2425560 2425536 2026-06-10T13:25:50Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27279260980 (https://github.com/cluebotng/component-configs/commits/83e16c74dca286ed8f7104d49a271dee18c41854) 2425560 wikitext text/x-wiki l5diz5w5r37vd7m01tlnqh5z898nsqp 2425670 2425560 2026-06-10T15:01:18Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27285125849 (https://github.com/cluebotng/component-configs/commits/3a4f641c7199ec2c34cd294d0baf97b9be997e7b) 2425670 wikitext text/x-wiki hea8piwgm4fg5479lg5x9tjc1ar7qty 498 443949 2426115 2423849 2026-06-11T10:40:45Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 338027c (Rename ?prop to ?statement in PropertyColumn SPARQL) 2426115 wikitext text/x-wiki mgiwcp45azuezmgav39hagpv8iyajhw 2426116 2426115 2026-06-11T10:40:46Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 3b935df (Extract QualifierColumn from PropertyColumn) 2426116 wikitext text/x-wiki q4jyuyjjl6lvuj8tmj16vi3s01w2ef0 0 444008 2426167 2302625 2026-06-11T11:41:09Z BJensen-WMF 48942 /* Apache */ Remove 'testing config', which contained instructions pertinent to turned-down testing infrastructure. Rename 'deploying config' to 'Testing and deploying config', as the section also contains testing instructions. 2426167 wikitext text/x-wiki 9iaaegwp6lef1lb3gb4rsqok0goni1j 12 444256 2425709 2413594 2026-06-10T15:46:18Z Ankitwarora 56213 recommendations on key gen using PuttyGen 2425709 wikitext text/x-wiki mhnam9ktbbh0oxnsayz3bz0g5hsmh1m 0 444372 2425843 2336378 2026-06-10T19:35:47Z Ottomata 88 /* Timestamps: meta.dt and dt */ 2425843 wikitext text/x-wiki 1lsi749rx5oomzeubbf32qsv9ijlxx3 0 445686 2425844 2401048 2026-06-10T19:41:00Z RLazarus (WMF) 15215 Revert edits by Star walker 2425844 wikitext text/x-wiki nb216nukmf7k56fd67zs16bl9jn8y1k 498 446211 2425535 2425367 2026-06-10T12:56:59Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27277788675 (https://github.com/cluebotng/component-configs/commits/39ecf0765b86afbcbd1be02c9f9a5519245ab884) 2425535 wikitext text/x-wiki 1kesshv6xxd100ytrvggmr4wbk0767n 2425668 2425535 2026-06-10T15:00:27Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27285126281 (https://github.com/cluebotng/component-configs/commits/3a4f641c7199ec2c34cd294d0baf97b9be997e7b) 2425668 wikitext text/x-wiki 7liysb3uknabzkjktjn3xc2e66pd5c3 0 446705 2425525 2420189 2026-06-10T12:42:08Z JMeybohm (WMF) 16709 /* Publishing */ 2425525 wikitext text/x-wiki p357l2hfx8wthl73wuji1b0c9b9xt6r 2425534 2425525 2026-06-10T12:48:10Z JMeybohm (WMF) 16709 /* Packaging */ 2425534 wikitext text/x-wiki mcrri92tx41ihf2nzpukjxk2bp6c4cn 0 446741 2425580 2205306 2026-06-10T13:52:24Z Jynus 38174 +cat 2425580 wikitext text/x-wiki 96t413bbzha7f1p4poxvmyhhmmolnzs 0 447310 2425696 2423992 2026-06-10T15:24:51Z Phuedx (WMF) 47850 /* Test Kitchen maintainer documentation */ 2425696 wikitext text/x-wiki jcwwmw5g56x6j5psq404zd9xrx3krno 0 449160 2425894 2425169 2026-06-11T00:01:45Z Dexbot 30554 Bot: Updating the report 2425894 wikitext text/x-wiki mir9t9oxv8lvlhfhv3ke5d1nrf713jf 2425978 2425894 2026-06-11T06:23:33Z Dexbot 30554 Bot: Updating the report 2425978 wikitext text/x-wiki 6zkfseguw5zbi9v8z6ozvk982hpw06k 0 449344 2425609 2423989 2026-06-10T14:13:38Z CGoubert-WMF 32775 /* Look for existing request pattern */ Add WebUI search 2425609 wikitext text/x-wiki p7svdeue4kih6eneagnrd7cchh6849q 0 451449 2425842 2026743 2026-06-10T19:33:49Z CStone (WMF) 38439 2425842 wikitext text/x-wiki hn7ubyuo4i0htssm62p1r868zfcg6se 498 452890 2425491 2425051 2026-06-10T12:15:35Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27275339348 (https://github.com/cluebotng/component-configs/commits/2d4571e6f74a6269bb7fbd7a03cc1cd1114f0a11) 2425491 wikitext text/x-wiki 86nxlrcxqw3t5p0l6r7dymnc6vskmuw 2425493 2425491 2026-06-10T12:17:21Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27275411814 (https://github.com/cluebotng/component-configs/commits/869dfb8d1487914e36184a9d1c5aae1e26dbba01) 2425493 wikitext text/x-wiki ogrdmok0gx3d0c7jqoydllzmg4mu26e 2425518 2425493 2026-06-10T12:37:37Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27276511398 (https://github.com/cluebotng/component-configs/commits/4442f7413e6335776bd1b8b0a660e20ae1256ae1) 2425518 wikitext text/x-wiki joa6qjxi74ig8yxxvkx15cm2vjq0137 2425523 2425518 2026-06-10T12:40:59Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27276698071 (https://github.com/cluebotng/component-configs/commits/8be9293c4b541d74d39482efd21163eb36cda6bd) 2425523 wikitext text/x-wiki fg0xwovowgo5d485n81c2my4u6q86y5 2425537 2425523 2026-06-10T12:57:00Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27277789050 (https://github.com/cluebotng/component-configs/commits/39ecf0765b86afbcbd1be02c9f9a5519245ab884) 2425537 wikitext text/x-wiki p0dsfeuh2g3ht1zjo25nazswlyteru7 2425671 2425537 2026-06-10T15:01:19Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27285125877 (https://github.com/cluebotng/component-configs/commits/3a4f641c7199ec2c34cd294d0baf97b9be997e7b) 2425671 wikitext text/x-wiki r3jufje8g2pn9p90p4lmcaw85gj7r9l 116 453906 2426002 2425263 2026-06-11T07:15:28Z Gitlabaccountapprovalbot 37332 @vesihiisi was approved. 2426002 wikitext text/x-wiki czjq4flh4u7kbyhi2xmeznlu4y11poi 2426148 2426002 2026-06-11T11:18:14Z Gitlabaccountapprovalbot 37332 wikipiggy was rejected. 2426148 wikitext text/x-wiki 4jyqbvmo5f5miq4nye3itu7k25vhqmi 2426177 2426148 2026-06-11T11:48:26Z Gitlabaccountapprovalbot 37332 @cadddr was approved. 2426177 wikitext text/x-wiki kluno4pay7npr0p5lzluykptzmghohl 0 454026 2426079 2401090 2026-06-11T09:29:12Z Phuedx (WMF) 47850 Components -> Component Catalog. Re-order MediaWiki extensions based on which extensions are the most important/likely to be worked on 2426079 wikitext text/x-wiki ajijjkobx7lraqxc9ilo8ivi1996rmu 2426144 2426079 2026-06-11T11:14:30Z Phuedx (WMF) 47850 More work on the Component Catalog and the Dashboards sections 2426144 wikitext text/x-wiki jreytup4bef531nausl2wy9o4vuifcd 2426153 2426144 2026-06-11T11:24:26Z Phuedx (WMF) 47850 Remove per-language development instructions as we're focussing on on-wiki development 2426153 wikitext text/x-wiki 8k12igmliqfnfzao1ogh42dvhc75wj9 498 458787 2425489 2423954 2026-06-10T12:15:01Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27275339730 (https://github.com/cluebotng/component-configs/commits/2d4571e6f74a6269bb7fbd7a03cc1cd1114f0a11) 2425489 wikitext text/x-wiki tpnzhee3cskhljrptxehezeola55p8s 2425495 2425489 2026-06-10T12:18:19Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27275397658 (https://github.com/cluebotng/component-configs/commits/33d203fb0e6b88ac6dc34e82ee630f7d4e6fdb56) 2425495 wikitext text/x-wiki qzvgsigskizcs7tvh43xo03wf2zpasx 2425515 2425495 2026-06-10T12:37:04Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27276510216 (https://github.com/cluebotng/component-configs/commits/4442f7413e6335776bd1b8b0a660e20ae1256ae1) 2425515 wikitext text/x-wiki brfsy59h6evdj03xyz8ndzkj4yr5rh9 2425520 2425515 2026-06-10T12:39:56Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27276642792 (https://github.com/cluebotng/component-configs/commits/0fef439bee61144769bfe62dddc830113380624e) 2425520 wikitext text/x-wiki icujt4aw6vhgt2dtz5xebqpgyejajle 2425541 2425520 2026-06-10T12:59:59Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27277788856 (https://github.com/cluebotng/component-configs/commits/39ecf0765b86afbcbd1be02c9f9a5519245ab884) 2425541 wikitext text/x-wiki h4v9q0hy5xfgcjenpg8ufutkp3yoifc 2425579 2425541 2026-06-10T13:45:39Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27280628230 (https://github.com/cluebotng/component-configs/commits/9b8bd8bb539a243486f5bdbbee92001379161762) 2425579 wikitext text/x-wiki 43pt9g8m2u4pvw8gnq9zgxajmldkzrh 2425667 2425579 2026-06-10T15:00:23Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27285127629 (https://github.com/cluebotng/component-configs/commits/3a4f641c7199ec2c34cd294d0baf97b9be997e7b) 2425667 wikitext text/x-wiki n41e95dt8s0q0gyypq3bm7lcas36905 498 459243 2425538 2421544 2026-06-10T12:57:43Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27277788903 (https://github.com/cluebotng/component-configs/commits/39ecf0765b86afbcbd1be02c9f9a5519245ab884) 2425538 wikitext text/x-wiki 5xa656wero2fb1zrezkg89rwb43awz3 2425661 2425538 2026-06-10T14:56:40Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27285126180 (https://github.com/cluebotng/component-configs/commits/3a4f641c7199ec2c34cd294d0baf97b9be997e7b) 2425661 wikitext text/x-wiki faqhqf4sa3piyjcnyaxayxgg4ab29pi 498 459244 2425490 2425049 2026-06-10T12:15:15Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27275339656 (https://github.com/cluebotng/component-configs/commits/2d4571e6f74a6269bb7fbd7a03cc1cd1114f0a11) 2425490 wikitext text/x-wiki 1cbz6q6mew7krbccpez8ruzyfxn8e82 2425496 2425490 2026-06-10T12:18:56Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27275411775 (https://github.com/cluebotng/component-configs/commits/869dfb8d1487914e36184a9d1c5aae1e26dbba01) 2425496 wikitext text/x-wiki f3dst04zfnpysaijuesh4q9qwebff4u 2425516 2425496 2026-06-10T12:37:10Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27276510520 (https://github.com/cluebotng/component-configs/commits/4442f7413e6335776bd1b8b0a660e20ae1256ae1) 2425516 wikitext text/x-wiki mg0ie7a1xiditlyuvi9vsojr5u97v2x 2425522 2425516 2026-06-10T12:40:34Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27276698067 (https://github.com/cluebotng/component-configs/commits/8be9293c4b541d74d39482efd21163eb36cda6bd) 2425522 wikitext text/x-wiki d7sddnllrj0s2mhrtl5tvohnfqrff8u 2425542 2425522 2026-06-10T13:00:50Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27277788620 (https://github.com/cluebotng/component-configs/commits/39ecf0765b86afbcbd1be02c9f9a5519245ab884) 2425542 wikitext text/x-wiki fyybflnrl1nn2l8kstoucvp6gh93k7z 2425669 2425542 2026-06-10T15:00:50Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27285125894 (https://github.com/cluebotng/component-configs/commits/3a4f641c7199ec2c34cd294d0baf97b9be997e7b) 2425669 wikitext text/x-wiki l9kz3ufwrhi8p9n6lhwta4bakpeeouz 498 459255 2425539 2423125 2026-06-10T12:57:55Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27277788656 (https://github.com/cluebotng/component-configs/commits/39ecf0765b86afbcbd1be02c9f9a5519245ab884) 2425539 wikitext text/x-wiki dx4xngipbxwbzm3ak4aade5icroqjlb 2425666 2425539 2026-06-10T14:59:31Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27285126038 (https://github.com/cluebotng/component-configs/commits/3a4f641c7199ec2c34cd294d0baf97b9be997e7b) 2425666 wikitext text/x-wiki l1yzt39d2nxgp1v8m06bpndm92vv38z 498 459263 2425540 2421528 2026-06-10T12:59:50Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27277788890 (https://github.com/cluebotng/component-configs/commits/39ecf0765b86afbcbd1be02c9f9a5519245ab884) 2425540 wikitext text/x-wiki t5l4i4m4eqb9bpnsim77wgn5uz30foz 2425663 2425540 2026-06-10T14:59:03Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27285125950 (https://github.com/cluebotng/component-configs/commits/3a4f641c7199ec2c34cd294d0baf97b9be997e7b) 2425663 wikitext text/x-wiki recd3ehtf0o37cfuos5mbpitt9ve4zd 498 459362 2425488 2421540 2026-06-10T12:14:37Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27275339527 (https://github.com/cluebotng/component-configs/commits/2d4571e6f74a6269bb7fbd7a03cc1cd1114f0a11) 2425488 wikitext text/x-wiki nzgl1me0fa2por9knt8y2e69hwqv0n5 2425513 2425488 2026-06-10T12:36:38Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27276510887 (https://github.com/cluebotng/component-configs/commits/4442f7413e6335776bd1b8b0a660e20ae1256ae1) 2425513 wikitext text/x-wiki fllzwhf2dyzl8s2i7t4yt0h32vsvm5b 0 459699 2425716 2425460 2026-06-10T16:04:40Z Stashbot 7414 wfan: civicrm upgraded from d7113e04 to 819c4ede 2425716 wikitext text/x-wiki nfmlw689186uffbpdz3id7ww318gaye 2425896 2425716 2026-06-11T00:22:50Z Stashbot 7414 eileen: civicrm upgraded from 819c4ede to f46a6066 2425896 wikitext text/x-wiki o7s8j8px1un4dn3dg4ra5uai5lcgm68 2425960 2425896 2026-06-11T01:30:48Z Stashbot 7414 eileen: civicrm upgraded from f46a6066 to 2962b28d 2425960 wikitext text/x-wiki dytwwf3zketstm9nev1vuklxcivyji8 2425963 2425960 2026-06-11T02:42:37Z Stashbot 7414 eileen: civicrm upgraded from 2962b28d to 8f98770f 2425963 wikitext text/x-wiki o8v9j94ab07xt9vz8pbh1ebhtqhnsho 2425964 2425963 2026-06-11T02:59:32Z Stashbot 7414 larssandergreen: civicrm upgraded from 8f98770f to 77961b3e 2425964 wikitext text/x-wiki hajfba83qsh4kyzb868p0zleibm9v05 2425975 2425964 2026-06-11T06:05:37Z Stashbot 7414 eileen: civicrm upgraded from 77961b3e to 0b8db587 2425975 wikitext text/x-wiki 58yhwe117p6ok0lxjxp5i5ji4oahq2r 0 459895 2426112 2391908 2026-06-11T10:33:29Z KSoučková-WMF 35993 tiny tweaks; consider this SRE reviewed 2426112 wikitext text/x-wiki hkf4fbjlrmt5ity4kxmfuqqx0oqub27 0 459966 2425850 2419917 2026-06-10T20:27:23Z MPopov (WMF) 5891 2425850 wikitext text/x-wiki 0460zpt6qq02y2b66aq9ntl30j03rol 0 460289 2426014 2423915 2026-06-11T07:38:23Z AKlapper (WMF) 421 2426014 wikitext text/x-wiki 6dfz1pafrtkilxoo5e4e1bf5ifdz0ee 0 460318 2425607 2425385 2026-06-10T14:13:32Z Atsuko-WMF 54082 Maintenance 2425607 wikitext text/x-wiki 0uxxl7e6xgvdfuwjdb6ttmtkj7yd8f1 2425631 2425607 2026-06-10T14:22:02Z Atsuko-WMF 54082 /* Maintenance */ dblist 2425631 wikitext text/x-wiki oyg8zcdexjdibbbit5obr85zxbq5zk4 2426068 2425631 2026-06-11T09:01:02Z Atsuko-WMF 54082 /* Maintenance */ dblist exclusions 2426068 wikitext text/x-wiki o6dcutwncuorqck5g42peqgmjpmjqk4 3 460320 2425674 2026-06-10T15:06:12Z StrikerBot 8475 Welcome to Toolforge! 2425674 wikitext text/x-wiki 9l8xjkkzztxlkxuyol8jwown7e3u9wo 3 460321 2425875 2026-06-10T21:14:33Z StrikerBot 8475 Welcome to Toolforge! 2425875 wikitext text/x-wiki d3vlsybkexoo223ny9yimvdbxvl4svx 2 460322 2425966 2026-06-11T05:04:44Z Ordiopside 56415 Created page with "== About me == Hi, I'm Or. I recently started contributing to Wikipedia and got hooked fast. I work in software, and I want to build tools that help the community. == Project: AfC Draft Quality Dashboard == I'm building a dashboard that scores the quality of Articles for Creation (AfC) drafts. The goal is to help reviewers and new editors, not to replace human review. === What it does === * Reads public draft data through the MediaWiki API and the Wiki Replicas. * P..." 2425966 wikitext text/x-wiki mlqg41cu3u7oa9n8gv2jx40jesi267b 2425994 2425966 2026-06-11T06:37:45Z Ordiopside 56415 2425994 wikitext text/x-wiki 2dvbcfpx73fyc87yviwusoivvs6g8fa 0 460323 2426152 2026-06-11T11:24:09Z GKyziridis-WMF 39160 Add LiftWing-openapi-server documentation 2426152 wikitext text/x-wiki 6mu1l8kdkx1kla2nwf0ptjlsrk22o85 3 460324 2426176 2026-06-11T11:48:00Z StrikerBot 8475 Welcome to Toolforge! 2426176 wikitext text/x-wiki liqc5xntkl4wpflqbkref052lseoxpr