Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.18 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 3086 2455274 2454857 2026-09-07T14:19:32Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component misctools-cli 2455274 wikitext text/x-wiki fbu0mw6vu45l0lrwbhlwb29n4s0ryv8 2455275 2455274 2026-09-07T14:19:51Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component misctools-cli 2455275 wikitext text/x-wiki 7cguqk7nmcgbo4979skjx8cwito9mn3 2455334 2455275 2026-09-07T15:47:56Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2455334 wikitext text/x-wiki 7ug8qp009ekhd5w5n56rjbxjj7izhoa 2455338 2455334 2026-09-07T15:53:05Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api 2455338 wikitext text/x-wiki dfzqecr5ouvrzbvof62zx9mny3ujryq 0 4108 2455189 2454967 2026-09-07T13:09:21Z ScheduleDeploymentBot 37566 Add [[gerrit:1335752]] to Monday, September 07 UTC afternoon backport window 2455189 wikitext text/x-wiki a73m9fxzmw59gz0cb23u9l77w7f36ir 2455190 2455189 2026-09-07T13:09:37Z ScheduleDeploymentBot 37566 Add [[gerrit:1336611]] to Monday, September 07 UTC afternoon backport window 2455190 wikitext text/x-wiki hjvxnb6q1qyj4w48ky1x6ovf1zkg359 2455191 2455190 2026-09-07T13:09:51Z ScheduleDeploymentBot 37566 Add [[gerrit:1336609]] to Monday, September 07 UTC afternoon backport window 2455191 wikitext text/x-wiki r2nl7pekwfgt10q32l730bh7noler04 2455192 2455191 2026-09-07T13:10:05Z ScheduleDeploymentBot 37566 Add [[gerrit:1336613]] to Monday, September 07 UTC afternoon backport window 2455192 wikitext text/x-wiki pncodca36whoz5yrvpp26ttm6toy5l4 2455307 2455192 2026-09-07T15:03:55Z ScheduleDeploymentBot 37566 Add [[gerrit:1326400]] to Monday, September 07 UTC late backport window 2455307 wikitext text/x-wiki he79k4ecjgoitlnoi7yr4vq0a1ly6h1 2455412 2455307 2026-09-08T04:56:35Z ScheduleDeploymentBot 37566 Add [[gerrit:1335706]] to Tuesday, September 08 UTC morning backport window 2455412 wikitext text/x-wiki at83z5mdowwm6ffl4d5yn44ua9bop47 2455447 2455412 2026-09-08T08:02:59Z ScheduleDeploymentBot 37566 Add [[gerrit:1337610]] to Tuesday, September 08 UTC afternoon backport window 2455447 wikitext text/x-wiki ibtiw3ycw0qhjl0ym2orrssalbytqnl 498 6726 2455253 2454800 2026-09-07T14:02:41Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component misctools-cli 2455253 wikitext text/x-wiki j8nta7gkeea1xju0c62hvxwhqak5rn5 2455254 2455253 2026-09-07T14:02:43Z Stashbot 7414 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component misctools-cli 2455254 wikitext text/x-wiki qe9fneufbgf3snb7ovm6w3ausww4xfc 2455255 2455254 2026-09-07T14:03:21Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component misctools-cli 2455255 wikitext text/x-wiki oa97l499o8hgne0oqpykluoz75xjlrt 2455266 2455255 2026-09-07T14:16:54Z Stashbot 7414 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component misctools-cli 2455266 wikitext text/x-wiki pg7zz2685mdzhzovghmbyanoibebupx 2455322 2455266 2026-09-07T15:33:36Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2455322 wikitext text/x-wiki l9qlizin1eugv2wnbr7au2hxbtd48es 2455323 2455322 2026-09-07T15:35:30Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api 2455323 wikitext text/x-wiki m662h5at7fjcnm4gvu9iqhqln94m0wv 2455324 2455323 2026-09-07T15:36:55Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2455324 wikitext text/x-wiki l11xeg6dhmq6l16jwza8bz38k9fxhj7 2455325 2455324 2026-09-07T15:37:15Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api 2455325 wikitext text/x-wiki 25zaqlegrhz7flef1oogok2qp4isoy8 2455327 2455325 2026-09-07T15:44:30Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2455327 wikitext text/x-wiki 4y98u30egj3eqsa7ryqcf1l8drhzpuy 2455332 2455327 2026-09-07T15:47:24Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api 2455332 wikitext text/x-wiki 9icd4hwbzozziymknn2gw9ibsww0nvw 0 7919 2455155 2455154 2026-09-07T12:01:24Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.provision for host aqs2002.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455155 wikitext text/x-wiki a7twfxrlv88pi57r7ztkdd2ddl5izgq 2455156 2455155 2026-09-07T12:02:59Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ldap-replica2007.wikimedia.org with reason: host reimage 2455156 wikitext text/x-wiki 8c1324m4pqe0dtacrm060kc89n1gfti 2455157 2455156 2026-09-07T12:03:10Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs2002.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455157 wikitext text/x-wiki pq0fqgp5gv9amgtzuyhr70d5aesxw99 2455158 2455157 2026-09-07T12:04:21Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host aqs2002.codfw.wmnet with OS bookworm 2455158 wikitext text/x-wiki 5d0u00kx0nt8f5w851p4k3ikgtkzsny 2455160 2455158 2026-09-07T12:07:38Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ldap-replica2007.wikimedia.org with reason: host reimage 2455160 wikitext text/x-wiki 46m1qorb8sxvxjmjsq09287laeq30b4 2455161 2455160 2026-09-07T12:14:38Z Stashbot 7414 elukey: moved most of the Docker Registry's prefixes to a new internal S3 backend. For any docker pull failure that worked in the past, please ping me or drop a note in T435499 or contact the oncall SREs 2455161 wikitext text/x-wiki 3msmll0zovefg5r9snt9v0fvdaky0nq 2455162 2455161 2026-09-07T12:22:35Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ldap-replica2007.wikimedia.org with OS trixie 2455162 wikitext text/x-wiki ffrcusjyzhjsdatln39g2ypdsl6lafm 2455163 2455162 2026-09-07T12:22:37Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host ldap-replica2007.wikimedia.org 2455163 wikitext text/x-wiki marca8mcjn9jzw5y6luw5thovshixy1 2455164 2455163 2026-09-07T12:22:47Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs2002.codfw.wmnet with reason: host reimage 2455164 wikitext text/x-wiki gzhkck5lmi18r6911rmhtzex304r9jv 2455165 2455164 2026-09-07T12:28:18Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs2002.codfw.wmnet with reason: host reimage 2455165 wikitext text/x-wiki m00j29v377a6t7cunc61osla6cx1p6f 2455166 2455165 2026-09-07T12:29:55Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1181: Pool db1181.eqiad.wmnet in after cloning 2455166 wikitext text/x-wiki eetrlvcpmg101t2f204ncfgwxpuah3u 2455167 2455166 2026-09-07T12:46:14Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs2002.codfw.wmnet with OS bookworm 2455167 wikitext text/x-wiki s7s88aa5xc1hcgm6c9kkrvitarey943 2455168 2455167 2026-09-07T12:47:53Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2003.codfw.wmnet 2455168 wikitext text/x-wiki rzj6cfjayif4n5d872nkelvg0f516o6 2455173 2455168 2026-09-07T12:54:37Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.ganeti.makevm for new host ldap-replica2008.wikimedia.org 2455173 wikitext text/x-wiki 42v2pggivtn3zi3q1pwuaah1omteshf 2455174 2455173 2026-09-07T12:54:38Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.dns.netbox 2455174 wikitext text/x-wiki e7rf2vp7lyazinseg9f9cjutozhb7s6 2455176 2455174 2026-09-07T12:58:40Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: wikikube-worker-codfw@codfw 2455176 wikitext text/x-wiki rka2hrwkklj7z8sqqp0jzbj1a0igg5f 2455177 2455176 2026-09-07T13:00:18Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reboot-single for host aqs2003.codfw.wmnet 2455177 wikitext text/x-wiki 7a7xfu0t90cgvihefdnbkrt7a56o4q8 2455178 2455177 2026-09-07T13:01:36Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ldap-replica2008.wikimedia.org - jmm@cumin1004" 2455178 wikitext text/x-wiki 2k6pltwc9s7yxxaenc1q2y566xjntvl 2455179 2455178 2026-09-07T13:02:03Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs 2455179 wikitext text/x-wiki 17ngoo1fzjkckoogt0rtmb48xlocs2c 2455180 2455179 2026-09-07T13:02:48Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ldap-replica2008.wikimedia.org - jmm@cumin1004" 2455180 wikitext text/x-wiki d97wxcsd9dbjejapxi0l6xtb5p2droc 2455181 2455180 2026-09-07T13:02:49Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2455181 wikitext text/x-wiki j64t8n83lryx9xqzl8zs9rxraoizdo1 2455182 2455181 2026-09-07T13:02:50Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.dns.wipe-cache ldap-replica2008.wikimedia.org on all recursors 2455182 wikitext text/x-wiki 6wj39grq3jgptpafh7q8wj4q7qjh4gv 2455183 2455182 2026-09-07T13:02:52Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) ldap-replica2008.wikimedia.org on all recursors 2455183 wikitext text/x-wiki e1pbknx0zqmu309y0yseyziu6din7cp 2455184 2455183 2026-09-07T13:03:00Z Stashbot 7414 jelto@cumin1003: END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs 2455184 wikitext text/x-wiki 1qk84l6iwoagvfce7d7xrxx1gtd15m1 2455185 2455184 2026-09-07T13:03:01Z Stashbot 7414 jelto@cumin1003: END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: wikikube-worker-codfw@codfw 2455185 wikitext text/x-wiki a3p46tgueiewr7grzpr26nh0iufpcd0 2455186 2455185 2026-09-07T13:03:24Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ldap-replica2008.wikimedia.org - jmm@cumin1004" 2455186 wikitext text/x-wiki o6ax7yf0xuhudhg5hiihkekjop0v251 2455187 2455186 2026-09-07T13:03:27Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ldap-replica2008.wikimedia.org - jmm@cumin1004" 2455187 wikitext text/x-wiki au5qzf7ita1enq7kyrgn3tmdzlo2z1x 2455188 2455187 2026-09-07T13:03:32Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.hosts.reimage for host ldap-replica2008.wikimedia.org with OS trixie 2455188 wikitext text/x-wiki g1s7d8dmyt2dqzj1gz1ywoh4og7k83f 2455193 2455188 2026-09-07T13:10:59Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host aqs2003.codfw.wmnet 2455193 wikitext text/x-wiki 2zgw8095474a5w7bnavw5qz8zypfypw 2455194 2455193 2026-09-07T13:11:01Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts aqs2003.codfw.wmnet 2455194 wikitext text/x-wiki mo358tfoqk0gi5e01gbdtanzlz3b9kq 2455195 2455194 2026-09-07T13:12:22Z Stashbot 7414 stran@deploy1003: Started scap sync-world: Backport for [[gerrit:1337450|SI: Use new InfoChip text class in case status updater (T437020)]] 2455195 wikitext text/x-wiki glwq8t5psyxebmbo4cp2q7x15rtl8bf 2455196 2455195 2026-09-07T13:15:04Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1181: Pool db1181.eqiad.wmnet in after cloning 2455196 wikitext text/x-wiki 3txyqjc8ob5jonl7i787gyxhq1qpijf 2455197 2455196 2026-09-07T13:15:13Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1181.eqiad.wmnet onto db1190.eqiad.wmnet 2455197 wikitext text/x-wiki kafzq2rsoutj69hagrcfhl932qazymr 2455198 2455197 2026-09-07T13:15:29Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.provision for host aqs2003.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455198 wikitext text/x-wiki 1l21fuu534li6gxcxrppam7081epg43 2455199 2455198 2026-09-07T13:16:22Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: wikikube-worker-eqiad@eqiad 2455199 wikitext text/x-wiki fvm5grdmd8o2lspuasxn99lbgi14hmo 2455200 2455199 2026-09-07T13:16:29Z Stashbot 7414 stran@deploy1003: stran: Backport for [[gerrit:1337450|SI: Use new InfoChip text class in case status updater (T437020)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2455200 wikitext text/x-wiki 26s1fvrfflw4ej2jxd2lnyrjhuqbc23 2455201 2455200 2026-09-07T13:17:16Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs2003.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455201 wikitext text/x-wiki mhfidizolaud827qja05p6omx2pdso8 2455202 2455201 2026-09-07T13:17:47Z Stashbot 7414 stran@deploy1003: stran: Continuing with deployment 2455202 wikitext text/x-wiki 0e4kclp54t8uww7suusx2qba8ovdq0u 2455203 2455202 2026-09-07T13:17:59Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host aqs2003.codfw.wmnet with OS bookworm 2455203 wikitext text/x-wiki 9bugl0bsfwjxwgg7y6uj72akeakwtnl 2455205 2455203 2026-09-07T13:22:28Z Stashbot 7414 stran@deploy1003: Finished scap sync-world: Backport for [[gerrit:1337450|SI: Use new InfoChip text class in case status updater (T437020)]] (duration: 10m 06s) 2455205 wikitext text/x-wiki f369srp5o9exoi8pn88dixtagpj7v3m 2455206 2455205 2026-09-07T13:23:06Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ldap-replica2008.wikimedia.org with reason: host reimage 2455206 wikitext text/x-wiki bjkl2279zdhdnzr207926vp7xhbkave 2455207 2455206 2026-09-07T13:24:40Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1340.eqiad.wmnet 2455207 wikitext text/x-wiki thtq5s2oct3nl8hgarrdaw21uvjb80y 2455208 2455207 2026-09-07T13:24:43Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1340.eqiad.wmnet 2455208 wikitext text/x-wiki t4e8h0ajx7dlzzh0gg8356t29yf5gcp 2455209 2455208 2026-09-07T13:24:45Z Stashbot 7414 cgoubert@cumin2003: END (FAIL) - Cookbook sre.k8s.renumber-node (exit_code=1) Renumbering for host wikikube-worker1340.eqiad.wmnet 2455209 wikitext text/x-wiki ewg8u94fgpb7jedflsifn4a2calyos3 2455210 2455209 2026-09-07T13:24:53Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-eqiad or A:lvs-secondary-eqiad) and A:bullseye and A:lvs 2455210 wikitext text/x-wiki ldjitfmkjb158hiug9nabl417t7yqeb 2455211 2455210 2026-09-07T13:25:06Z Stashbot 7414 moritzm: installing openssh security updates 2455211 wikitext text/x-wiki 51e78a86bktcin8q880s0j3kuzg3o7q 2455212 2455211 2026-09-07T13:26:08Z Stashbot 7414 jelto@cumin1003: END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-eqiad or A:lvs-secondary-eqiad) and A:bullseye and A:lvs 2455212 wikitext text/x-wiki 3zqog0eouv6d7eqnyvjm7ssm29jxyna 2455213 2455212 2026-09-07T13:26:09Z Stashbot 7414 jelto@cumin1003: END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: wikikube-worker-eqiad@eqiad 2455213 wikitext text/x-wiki gmaegn7im27l3sythbp4edxh2a8dvdb 2455214 2455213 2026-09-07T13:27:01Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudvirt1065.eqiad.wmnet 2455214 wikitext text/x-wiki k41u5gt19tf7e2lqsir9d9hqdl33fym 2455215 2455214 2026-09-07T13:28:18Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ldap-replica2008.wikimedia.org with reason: host reimage 2455215 wikitext text/x-wiki 6qujxh9t9iqruiw6nvx2rf894iuf3jk 2455216 2455215 2026-09-07T13:33:07Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudvirt1065.eqiad.wmnet 2455216 wikitext text/x-wiki q24sxta5li5ey7h4a41eewjoig4unuu 2455217 2455216 2026-09-07T13:33:11Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudvirt1066.eqiad.wmnet 2455217 wikitext text/x-wiki cx2raezv9uv65j6uc7tffpf9w2wlxl0 2455218 2455217 2026-09-07T13:34:54Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs2003.codfw.wmnet with reason: host reimage 2455218 wikitext text/x-wiki ftusr5z3fyy7vq3tmgq158j14518w0h 2455219 2455218 2026-09-07T13:36:05Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1190: Repooling after cloning 2455219 wikitext text/x-wiki 2ql70ufi3ot70mqw59u5z3cll6dsnre 2455220 2455219 2026-09-07T13:37:03Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1228: Repooling db1228 into s4 2455220 wikitext text/x-wiki 6f9vwldq66tvawtrolrtb2uwpvbmfhn 2455221 2455220 2026-09-07T13:38:06Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs2003.codfw.wmnet with reason: host reimage 2455221 wikitext text/x-wiki 6s3vq0w7v3hviu3vxw7b8a0yql7hjkb 2455222 2455221 2026-09-07T13:39:23Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/pageview-trending-relative-next: apply 2455222 wikitext text/x-wiki jh4hqq01ftcwg5vqqgq39ndxstgiw86 2455223 2455222 2026-09-07T13:39:27Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/pageview-trending-relative-next: apply 2455223 wikitext text/x-wiki cs0ngozcid1svrb9ik8l3gms4nlqjxu 2455224 2455223 2026-09-07T13:39:40Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudvirt1066.eqiad.wmnet 2455224 wikitext text/x-wiki o2zr1jv3ii39rqetjaedkshm1sl1el4 2455225 2455224 2026-09-07T13:39:45Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudvirt1067.eqiad.wmnet 2455225 wikitext text/x-wiki lod0mx0p0hhaqxhb6pru34i9x2n7ltd 2455226 2455225 2026-09-07T13:44:20Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ldap-replica2008.wikimedia.org with OS trixie 2455226 wikitext text/x-wiki nh7you4o6git1vocdxxcqvsn6pdaz71 2455227 2455226 2026-09-07T13:44:21Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host ldap-replica2008.wikimedia.org 2455227 wikitext text/x-wiki fyj794cv2fqaaivnw8pu4153kwapln4 2455228 2455227 2026-09-07T13:50:03Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.renumber-node Renumbering for host wikikube-worker1341.eqiad.wmnet 2455228 wikitext text/x-wiki d46dbrazcan6wkkorgwz1ylotbk8xdn 2455229 2455228 2026-09-07T13:50:07Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1341.eqiad.wmnet 2455229 wikitext text/x-wiki bkldk1ih9acu2rw3g9krdv2s7a46gbd 2455230 2455229 2026-09-07T13:51:19Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1341.eqiad.wmnet 2455230 wikitext text/x-wiki f47krwz0jor8maqwhs3tixg1y6rdb0b 2455231 2455230 2026-09-07T13:52:30Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.hosts.reimage for host wikikube-worker1341.eqiad.wmnet with OS trixie 2455231 wikitext text/x-wiki 8fz07ioctcjoo32y3j7dz6ik4uafj3h 2455232 2455231 2026-09-07T13:52:45Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.hosts.move-vlan for host wikikube-worker1341 2455232 wikitext text/x-wiki kypttj3pwpf68ab6mnhc4mr4a7ku62p 2455233 2455232 2026-09-07T13:55:28Z Stashbot 7414 filippo@cumin1003: END (ERROR) - Cookbook sre.hosts.reboot-single (exit_code=97) for host cloudvirt1067.eqiad.wmnet 2455233 wikitext text/x-wiki exhagj93e0jbc18djeq86et0w5e33km 2455234 2455233 2026-09-07T13:55:32Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudvirt1072.eqiad.wmnet 2455234 wikitext text/x-wiki i8yuu7czeqm54k0cwgg680nz9jbfo5x 2455235 2455234 2026-09-07T13:55:41Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.dns.netbox 2455235 wikitext text/x-wiki q0zrx4hcvv54d0oz6ko9tjbctzqqnvq 2455236 2455235 2026-09-07T13:57:03Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs2003.codfw.wmnet with OS bookworm 2455236 wikitext text/x-wiki gh1vpzdn22gixfi6701oe573h1z7fp1 2455237 2455236 2026-09-07T13:59:12Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2004.codfw.wmnet 2455237 wikitext text/x-wiki 7wni6xiw6m8zz8640wgvodaejfvub2r 2455238 2455237 2026-09-07T13:59:55Z Stashbot 7414 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1335754|tests: Add user_id and actor in testLogDatabaseRowsForHiddenUser (T436971)]], [[gerrit:1336613|User: Use ActorStore on User::load (T428517)]], [[gerrit:1335752|Fix empty bases in m(under|over) (T436876)]], [[gerrit:1336611|Use Core-compatible output for cancellation (T379359)]], [[gerrit:1336609|Page: Fix absence caching when combined with mult 2455238 wikitext text/x-wiki dvqobujamz2am10qsy39i8m0ls96xdk 2455239 2455238 2026-09-07T13:59:58Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1341 - cgoubert@cumin2003" 2455239 wikitext text/x-wiki jdd8o88c7g23fac6erw3tna3ypxeqes 2455240 2455239 2026-09-07T14:00:23Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1341 - cgoubert@cumin2003" 2455240 wikitext text/x-wiki cdwpm2pa9kdxryikjy4p0t3d4iwq0rn 2455241 2455240 2026-09-07T14:00:24Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2455241 wikitext text/x-wiki kv78d04730qw0zi12la2irdbsxwbs44 2455242 2455241 2026-09-07T14:00:25Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.dns.wipe-cache wikikube-worker1341.eqiad.wmnet 159.32.64.10.in-addr.arpa 9.5.1.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2455242 wikitext text/x-wiki bi2dgyw6lwtg42zew4bt8ax7j5feb4q 2455243 2455242 2026-09-07T14:00:27Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker1341.eqiad.wmnet 159.32.64.10.in-addr.arpa 9.5.1.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2455243 wikitext text/x-wiki g37x3cnl12a53b2qcsva731oq7c55vu 2455244 2455243 2026-09-07T14:00:28Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker1341 2455244 wikitext text/x-wiki ky28n59c6hxdzl4keu3qwvoik5ywy0q 2455245 2455244 2026-09-07T14:01:12Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2455245 wikitext text/x-wiki e7prdt05nv4rqu2a3889rz9ygdbnc8d 2455246 2455245 2026-09-07T14:01:13Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker1341 2455246 wikitext text/x-wiki 2xgirael2th5bff4xv96o7ru5eow0rz 2455247 2455246 2026-09-07T14:01:14Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker1341 2455247 wikitext text/x-wiki 3on0plkeo4ykqvdur1mhujldmb5iqs8 2455248 2455247 2026-09-07T14:01:21Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2455248 wikitext text/x-wiki rvbehfh70fqyk7191tfp96u323w38l8 2455250 2455248 2026-09-07T14:02:06Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudvirt1072.eqiad.wmnet 2455250 wikitext text/x-wiki 4lddjg3uiifsgd2byc75rbrmtwpceeu 2455251 2455250 2026-09-07T14:02:07Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudvirt1073.eqiad.wmnet 2455251 wikitext text/x-wiki 7e0aapeqilu4thgvsd9tf98l9jawt85 2455256 2455251 2026-09-07T14:04:13Z Stashbot 7414 krinkle@deploy1003: krinkle: Backport for [[gerrit:1335754|tests: Add user_id and actor in testLogDatabaseRowsForHiddenUser (T436971)]], [[gerrit:1336613|User: Use ActorStore on User::load (T428517)]], [[gerrit:1335752|Fix empty bases in m(under|over) (T436876)]], [[gerrit:1336611|Use Core-compatible output for cancellation (T379359)]], [[gerrit:1336609|Page: Fix absence caching when combined with multiple properties 2455256 wikitext text/x-wiki 590r1yv6ukcnlhgdxc3exvaa4h0joc0 2455257 2455256 2026-09-07T14:07:09Z Stashbot 7414 krinkle@deploy1003: krinkle: Continuing with deployment 2455257 wikitext text/x-wiki ezj512tmavzf579k1rkdekxr3wnf2gk 2455258 2455257 2026-09-07T14:08:11Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudvirt1073.eqiad.wmnet 2455258 wikitext text/x-wiki g4hrjhin1a7bbzwks9wulchaf3oespc 2455259 2455258 2026-09-07T14:08:15Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudvirt1074.eqiad.wmnet 2455259 wikitext text/x-wiki 06vp783v7jsyatgpxv1er03snmqourg 2455260 2455259 2026-09-07T14:09:40Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reboot-single for host aqs2004.codfw.wmnet 2455260 wikitext text/x-wiki 5u97tr7gu7m60uxc60qv04q806xqnf3 2455261 2455260 2026-09-07T14:11:49Z Stashbot 7414 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1335754|tests: Add user_id and actor in testLogDatabaseRowsForHiddenUser (T436971)]], [[gerrit:1336613|User: Use ActorStore on User::load (T428517)]], [[gerrit:1335752|Fix empty bases in m(under|over) (T436876)]], [[gerrit:1336611|Use Core-compatible output for cancellation (T379359)]], [[gerrit:1336609|Page: Fix absence caching when combined with mul 2455261 wikitext text/x-wiki k8e5x4zcql8fklgv44xt2e613i459zd 2455262 2455261 2026-09-07T14:13:05Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'sync'. 2455262 wikitext text/x-wiki 69izysuko610w5kf2qjar9ljwdzf3fq 2455263 2455262 2026-09-07T14:13:39Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1341.eqiad.wmnet with reason: host reimage 2455263 wikitext text/x-wiki 4ijru6ifjlmq8mzl7xgykmsvwvbogqo 2455264 2455263 2026-09-07T14:14:20Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'sync'. 2455264 wikitext text/x-wiki ik1aktolon6zfrzjmvk8deaxklvjs2p 2455265 2455264 2026-09-07T14:14:47Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudvirt1074.eqiad.wmnet 2455265 wikitext text/x-wiki hfy5ejx4fdoa2i43vzbrryjx6xbv5eg 2455267 2455265 2026-09-07T14:18:37Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1341.eqiad.wmnet with reason: host reimage 2455267 wikitext text/x-wiki 7numy14zdpmtbijtbp3bbv8fvbqk1x8 2455268 2455267 2026-09-07T14:18:38Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-liftwing-studio: sync 2455268 wikitext text/x-wiki 72tnnyaz8n4nvu10qc5eybfcqmwixvj 2455269 2455268 2026-09-07T14:18:40Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-liftwing-studio: sync 2455269 wikitext text/x-wiki gpr1wxujss8fpllssn646oehu8zjrjt 2455270 2455269 2026-09-07T14:18:47Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2455270 wikitext text/x-wiki 3gud279jhfjruu6aoofy7n56p4znaii 2455271 2455270 2026-09-07T14:18:48Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host aqs2004.codfw.wmnet 2455271 wikitext text/x-wiki 7djudzvv9ld4luu3erw8q2rhp5pljna 2455272 2455271 2026-09-07T14:18:51Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts aqs2004.codfw.wmnet 2455272 wikitext text/x-wiki adnanob4u4tp964p81ahyzp3thqnvy2 2455273 2455272 2026-09-07T14:19:08Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.provision for host aqs2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455273 wikitext text/x-wiki ixewmvj6qp92kr2i2d46a4d52nmjml6 2455276 2455273 2026-09-07T14:20:54Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455276 wikitext text/x-wiki 1d1q2ehidkcg6g8tr7aoh6zkn4jhaa0 2455277 2455276 2026-09-07T14:21:11Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1190: Repooling after cloning 2455277 wikitext text/x-wiki 3jfp4wxbkqmntn8gmrewhhp7ehfzjdd 2455278 2455277 2026-09-07T14:21:38Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host aqs2004.codfw.wmnet with OS bookworm 2455278 wikitext text/x-wiki au07ms0w4n4udf5zqogw81oxxfocdoa 2455279 2455278 2026-09-07T14:22:10Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1228: Repooling db1228 into s4 2455279 wikitext text/x-wiki oxv24lmg0j3afjjk335yddq696b4ujv 2455282 2455279 2026-09-07T14:25:53Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: wikikube-worker-codfw@codfw 2455282 wikitext text/x-wiki t2uoww765u12o980bw9icy26bbm7viy 2455283 2455282 2026-09-07T14:29:04Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs 2455283 wikitext text/x-wiki 5n562i4a34m1igeiewxhgdsfqo0x959 2455284 2455283 2026-09-07T14:30:09Z Stashbot 7414 jelto@cumin1003: END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs 2455284 wikitext text/x-wiki 8fivm65w3htz7pyxtvw5e9htbp1rut4 2455285 2455284 2026-09-07T14:30:10Z Stashbot 7414 jelto@cumin1003: END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: wikikube-worker-codfw@codfw 2455285 wikitext text/x-wiki 8tf8dqy41gmig2vrtu0o1xr1lbddmnd 2455286 2455285 2026-09-07T14:32:38Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2455286 wikitext text/x-wiki swz2iun7re8empvmzbtonlo0vmxfdgt 2455287 2455286 2026-09-07T14:33:15Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2455287 wikitext text/x-wiki 3hhx67hufhzz9q5rs7pcc8hwa0uz0kf 2455288 2455287 2026-09-07T14:33:16Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2455288 wikitext text/x-wiki 5b6j9sida9sjejrckej82gn9dm81gnt 2455289 2455288 2026-09-07T14:33:56Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2455289 wikitext text/x-wiki 1oo4ga2w0a4o8iihcwlvbxbijti9pve 2455290 2455289 2026-09-07T14:35:05Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2455290 wikitext text/x-wiki i63u5kxevomr7lesz8t7hu4kxv3gtrg 2455291 2455290 2026-09-07T14:35:08Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2455291 wikitext text/x-wiki r2p1w30nr5vcfx6u50b7e6gscq9sduf 2455292 2455291 2026-09-07T14:35:09Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-liftwing-studio: apply 2455292 wikitext text/x-wiki ptcmcoepsyx058vr1na553au0mmd0w0 2455293 2455292 2026-09-07T14:35:10Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2455293 wikitext text/x-wiki kkojqknfxky88gn299fphqzjenp1jt2 2455294 2455293 2026-09-07T14:35:11Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-liftwing-studio: apply 2455294 wikitext text/x-wiki kwp5u52jmhg0x6q4e91w79qavdks9i7 2455295 2455294 2026-09-07T14:35:12Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2455295 wikitext text/x-wiki 1kq34p52f7okfrci4k0vpq9t1qu1szp 2455296 2455295 2026-09-07T14:37:15Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: wikikube-worker-eqiad@eqiad 2455296 wikitext text/x-wiki q1gs71iujn4repl2cprb4nhj8tw02dv 2455297 2455296 2026-09-07T14:37:37Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1341.eqiad.wmnet with OS trixie 2455297 wikitext text/x-wiki g3rdqafipojhedq7k8n9te3p3gxlql1 2455298 2455297 2026-09-07T14:40:23Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-eqiad or A:lvs-secondary-eqiad) and A:bullseye and A:lvs 2455298 wikitext text/x-wiki 40iez9qy9zop9ew1ds0epzomem95tnf 2455299 2455298 2026-09-07T14:40:30Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs2004.codfw.wmnet with reason: host reimage 2455299 wikitext text/x-wiki odzsvuzb1t2x86u2qtm5iydp5st0tcv 2455300 2455299 2026-09-07T14:41:32Z Stashbot 7414 jelto@cumin1003: END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-eqiad or A:lvs-secondary-eqiad) and A:bullseye and A:lvs 2455300 wikitext text/x-wiki oxgsbqea5xgp9dgw2bvmjzlr1e6lz3z 2455301 2455300 2026-09-07T14:41:33Z Stashbot 7414 jelto@cumin1003: END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: wikikube-worker-eqiad@eqiad 2455301 wikitext text/x-wiki 5kb046ku9y4rl12n1hluo57botxsdw5 2455302 2455301 2026-09-07T14:44:45Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs2004.codfw.wmnet with reason: host reimage 2455302 wikitext text/x-wiki o3vxmhok6piwn55uney5npodkrdjqbg 2455303 2455302 2026-09-07T14:58:44Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'sync'. 2455303 wikitext text/x-wiki 3r89u3blsia54mdseavpnyrmsewhtfi 2455304 2455303 2026-09-07T15:00:06Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'sync'. 2455304 wikitext text/x-wiki 5maq0kiv7igfsgy8jahd3iekbmqc3u1 2455305 2455304 2026-09-07T15:02:08Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs2004.codfw.wmnet with OS bookworm 2455305 wikitext text/x-wiki 4ebb85mc77yzkotdasgfe4s2yhnhcw5 2455306 2455305 2026-09-07T15:03:55Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2005.codfw.wmnet 2455306 wikitext text/x-wiki 2iqojpra59iiziqbdfq12xpsuipw9jc 2455308 2455306 2026-09-07T15:04:12Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1341.eqiad.wmnet 2455308 wikitext text/x-wiki c5r3w5f78dnhjepebpd44rx00d72zcw 2455309 2455308 2026-09-07T15:04:14Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1341.eqiad.wmnet 2455309 wikitext text/x-wiki 7z7snd4a82ec4aq6bxlu2u0lk7sxqy8 2455310 2455309 2026-09-07T15:04:17Z Stashbot 7414 cgoubert@cumin2003: END (FAIL) - Cookbook sre.k8s.renumber-node (exit_code=1) Renumbering for host wikikube-worker1341.eqiad.wmnet 2455310 wikitext text/x-wiki fu5v72v7f4zn5psbggfs17u7ffs7l2z 2455311 2455310 2026-09-07T15:11:34Z Stashbot 7414 moritzm: installing rsync security updates 2455311 wikitext text/x-wiki ciogpy2zrch3om7fuqtqakj8rhhbh3v 2455312 2455311 2026-09-07T15:14:06Z Stashbot 7414 mvernon@cumin2003: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts aqs2005.codfw.wmnet 2455312 wikitext text/x-wiki 99l281wna591nf92u4m1p9nf0k82vt5 2455313 2455312 2026-09-07T15:14:41Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2005.codfw.wmnet 2455313 wikitext text/x-wiki 2g0hrslyq2b7ef9aop15jnyj8bxmgkb 2455314 2455313 2026-09-07T15:15:07Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reboot-single for host aqs2005.codfw.wmnet 2455314 wikitext text/x-wiki row8o5rp9putc5fizj0bkqcg37ry0ep 2455315 2455314 2026-09-07T15:24:05Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host aqs2005.codfw.wmnet 2455315 wikitext text/x-wiki sl645www76e8mmsq6mlulx6sk44iwux 2455316 2455315 2026-09-07T15:24:07Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts aqs2005.codfw.wmnet 2455316 wikitext text/x-wiki g33w9x1brq4wm7fcrsi9c5lcqzzqhpe 2455317 2455316 2026-09-07T15:24:29Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.provision for host aqs2005.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455317 wikitext text/x-wiki cycmzm1m752bvqwoqabdn9dum27477t 2455318 2455317 2026-09-07T15:26:15Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs2005.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455318 wikitext text/x-wiki bm5e6ezycig6e5dm3p7tu2n1cujwcfd 2455319 2455318 2026-09-07T15:26:56Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host aqs2005.codfw.wmnet with OS bookworm 2455319 wikitext text/x-wiki g4khj2s9at9qu06xlytb3w88fs42p47 2455320 2455319 2026-09-07T15:27:12Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2455320 wikitext text/x-wiki 0uduxfoxmt6id0tvnucg9f361dw6j38 2455326 2455320 2026-09-07T15:44:20Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2455326 wikitext text/x-wiki 9h9ysbp4frgvgxsb10la6114glkkfax 2455328 2455326 2026-09-07T15:44:32Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2455328 wikitext text/x-wiki gb8shge3bo8aowewk1dcrr3bagez89w 2455330 2455328 2026-09-07T15:45:38Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs2005.codfw.wmnet with reason: host reimage 2455330 wikitext text/x-wiki tp929xdvyopi0r3te4d6v14k3d4b77i 2455331 2455330 2026-09-07T15:46:00Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] START helmfile.d/services/thumbor: apply 2455331 wikitext text/x-wiki 47xvwuh4y302nzsc43esy56j97tkoac 2455333 2455331 2026-09-07T15:47:25Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] DONE helmfile.d/services/thumbor: apply 2455333 wikitext text/x-wiki e1zzsec8gqdnownamlfqrz9mzjcimyw 2455335 2455333 2026-09-07T15:49:34Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs2005.codfw.wmnet with reason: host reimage 2455335 wikitext text/x-wiki 5t6ghfx61762z9auq3bkxib1m50vm19 2455337 2455335 2026-09-07T15:51:13Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1336829|Disable query pages not yet compatible with commons split (T437108)]] 2455337 wikitext text/x-wiki qlistxghfp0vwssy3an9paxpjyqpx8x 2455339 2455337 2026-09-07T15:55:15Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1336829|Disable query pages not yet compatible with commons split (T437108)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2455339 wikitext text/x-wiki hwxmev21qaev9ierbvzdq1qa8hdih1v 2455340 2455339 2026-09-07T15:56:44Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2455340 wikitext text/x-wiki 1s74vmx3blwn5k61wvdue53k77da17m 2455341 2455340 2026-09-07T15:57:30Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] START helmfile.d/services/thumbor: apply 2455341 wikitext text/x-wiki 8q5t8a0if4c51cm3e2ur9wgfvaerqxk 2455342 2455341 2026-09-07T15:59:32Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] DONE helmfile.d/services/thumbor: apply 2455342 wikitext text/x-wiki stpeg1jh7219ulotjbk2simd57utwgw 2455343 2455342 2026-09-07T16:01:35Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1336829|Disable query pages not yet compatible with commons split (T437108)]] (duration: 10m 22s) 2455343 wikitext text/x-wiki 0w2yoj544nmkclvea91okhshgi9zwul 2455344 2455343 2026-09-07T16:07:00Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs2005.codfw.wmnet with OS bookworm 2455344 wikitext text/x-wiki oynmba6cmx0l11s2ax6vw1a3ujh5mdl 2455345 2455344 2026-09-07T16:11:32Z Bellevarunchayap 59319 #REDIRECT[[]] · <code></code> · <syntaxhighlight lang="bash"></syntaxhighlight> · 2455345 wikitext text/x-wiki bgts49usvvrqhlg92dgct5e0f0f83sy 2455346 2455345 2026-09-07T16:12:05Z Syunsyunminmin 33710 Reverted edits by [[Special:Contribs/Bellevarunchayap|Bellevarunchayap]] ([[User talk:Bellevarunchayap|talk]]) to last version by Stashbot: test edits, please use the sandbox 2455344 wikitext text/x-wiki oynmba6cmx0l11s2ax6vw1a3ujh5mdl 2455353 2455346 2026-09-07T18:22:31Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1337646|SpecialMostGloballyLinkedFiles: Recache from the globalusage domain (T400824)]] 2455353 wikitext text/x-wiki ffi8qr0iqa0czrize29dlzg6mxkknuf 2455354 2455353 2026-09-07T18:26:39Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1337646|SpecialMostGloballyLinkedFiles: Recache from the globalusage domain (T400824)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2455354 wikitext text/x-wiki o18adqtnrin89h2922q6rpj9fifue09 2455355 2455354 2026-09-07T18:27:05Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2455355 wikitext text/x-wiki 2wsngza8hmwwgoe6olh73pwh0jprjs7 2455356 2455355 2026-09-07T18:31:43Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1337646|SpecialMostGloballyLinkedFiles: Recache from the globalusage domain (T400824)]] (duration: 09m 12s) 2455356 wikitext text/x-wiki cvx1wa87zre2pqp2nc5q1skywutcpkx 2455357 2455356 2026-09-07T18:32:41Z Stashbot 7414 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply 2455357 wikitext text/x-wiki ctyz9fsad6zsiwbu249pp1jd9a33fuc 2455358 2455357 2026-09-07T18:33:13Z Stashbot 7414 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply 2455358 wikitext text/x-wiki qercfvuu6fua33bcv4t5qzl49as2hw7 2455361 2455358 2026-09-07T18:53:11Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1328365|Set RemoteVirtualDomainsMapping for test-commons]] 2455361 wikitext text/x-wiki o7xddvzd9g2f9eotv7dx0sibfy7yt5p 2455362 2455361 2026-09-07T18:57:17Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1328365|Set RemoteVirtualDomainsMapping for test-commons]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2455362 wikitext text/x-wiki fmyfdbo1g6ba6cqm5hnmoudixa71ewz 2455363 2455362 2026-09-07T19:00:50Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2455363 wikitext text/x-wiki 45ocp8sqpo4e1flb3dun56i8svzengb 2455368 2455363 2026-09-07T19:07:28Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1328365|Set RemoteVirtualDomainsMapping for test-commons]] (duration: 14m 17s) 2455368 wikitext text/x-wiki pnytgznwwt9q0rgve2w78gde54jw15g 2455369 2455368 2026-09-07T19:19:39Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1336120|Move production reads for commons link tables to x4 for 1% of requests (T437108)]] 2455369 wikitext text/x-wiki rypvf9w373nssqcovnnlyzrzit4jeuh 2455370 2455369 2026-09-07T19:23:56Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1336120|Move production reads for commons link tables to x4 for 1% of requests (T437108)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2455370 wikitext text/x-wiki 19goxbprqvwu7wffb2fcmcgxm3oajy9 2455371 2455370 2026-09-07T19:26:36Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2455371 wikitext text/x-wiki 2p1cw08af6k6sempo0jp36pu7k2vxty 2455372 2455371 2026-09-07T19:31:19Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1336120|Move production reads for commons link tables to x4 for 1% of requests (T437108)]] (duration: 11m 40s) 2455372 wikitext text/x-wiki rrxdvgehbs37g30go9sopfjyin8jx9q 2455375 2455372 2026-09-07T19:48:18Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1337650|Move production reads for commons link tables to x4 for 10% of requests (T437108)]] 2455375 wikitext text/x-wiki 9cng11np1ayjcia7l1lojci7pdv4yno 2455377 2455375 2026-09-07T19:52:28Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1337650|Move production reads for commons link tables to x4 for 10% of requests (T437108)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2455377 wikitext text/x-wiki 9ebjj44c0btwm47zj2myaj3vss1svn6 2455378 2455377 2026-09-07T19:55:06Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2455378 wikitext text/x-wiki ppr456okh155z8tktnb0m7ctiyy7ns9 2455379 2455378 2026-09-07T19:59:46Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1337650|Move production reads for commons link tables to x4 for 10% of requests (T437108)]] (duration: 11m 27s) 2455379 wikitext text/x-wiki qh0ocgj7j18zsfj3wnvmrb8yur2t3to 2455380 2455379 2026-09-07T20:02:32Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1326400|Stop setting wgCampaignEventsEnableWorklists (T429510)]] 2455380 wikitext text/x-wiki mlv3alaza9cok0tvqz6wcjhb0u6arra 2455381 2455380 2026-09-07T20:06:40Z Stashbot 7414 zabe@deploy1003: zabe, daimona: Backport for [[gerrit:1326400|Stop setting wgCampaignEventsEnableWorklists (T429510)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2455381 wikitext text/x-wiki lrljmu9b15wbjyyj7gp99x8kwcy1win 2455382 2455381 2026-09-07T20:07:59Z Stashbot 7414 zabe@deploy1003: zabe, daimona: Continuing with deployment 2455382 wikitext text/x-wiki 7qw9ca6cd5vjgxqncrcxfjepu4pxtud 2455383 2455382 2026-09-07T20:12:38Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326400|Stop setting wgCampaignEventsEnableWorklists (T429510)]] (duration: 10m 06s) 2455383 wikitext text/x-wiki gfwnu2rhcq1co3gze8yf5ywcdeftqo2 2455384 2455383 2026-09-07T20:14:20Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1337655|Move production reads for commons link tables to x4 for 25% of requests (T437108)]] 2455384 wikitext text/x-wiki 7uigp8k2pveomupbuzthol0jc9iek8b 2455385 2455384 2026-09-07T20:18:38Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1337655|Move production reads for commons link tables to x4 for 25% of requests (T437108)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2455385 wikitext text/x-wiki dusp7e857hu1p8rth5meqlkawrk6f6d 2455386 2455385 2026-09-07T20:19:14Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2455386 wikitext text/x-wiki fu1ahtqmzxd44c4g6urpiy04bsh97fu 2455387 2455386 2026-09-07T20:23:48Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1337655|Move production reads for commons link tables to x4 for 25% of requests (T437108)]] (duration: 09m 28s) 2455387 wikitext text/x-wiki rupv7x3fexu1c9kzcqt9oooirkr36xr 2455388 2455387 2026-09-07T20:38:07Z Stashbot 7414 ladsgroup@cumin1003: dbctl commit (dc=all): 'Set weight of db1261 to zero in s4 (T437108)', diff saved to https://phabricator.wikimedia.org/P96385 and previous config saved to /var/cache/conftool/dbconfig/20260907-203804-ladsgroup.json 2455388 wikitext text/x-wiki g4vw9d6lmk279s84pzlxlalum0kt2w3 2455390 2455388 2026-09-07T20:52:38Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1337656|Move production reads for commons link tables to x4 for 50% of requests (T437108)]] 2455390 wikitext text/x-wiki jhx7jcf9u8bs3kefrtns1p18x0rgfq0 2455391 2455390 2026-09-07T20:57:07Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1337656|Move production reads for commons link tables to x4 for 50% of requests (T437108)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2455391 wikitext text/x-wiki o2tnicy8m21izv69rg2nzms3szv2qa9 2455392 2455391 2026-09-07T20:58:04Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2455392 wikitext text/x-wiki 4wfwnx94nvorg9vollj8gp6ys2kul1h 2455393 2455392 2026-09-07T21:03:05Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1337656|Move production reads for commons link tables to x4 for 50% of requests (T437108)]] (duration: 10m 27s) 2455393 wikitext text/x-wiki 81x9whw8oqxwbbyuqd3zxvbhwdptcah 2455394 2455393 2026-09-07T21:28:09Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1337319|Move production reads for commons link tables to x4 for all requests (T437108)]] 2455394 wikitext text/x-wiki pmqlmgt6m45py9xv83eyir05b1voqf9 2455395 2455394 2026-09-07T21:32:30Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1337319|Move production reads for commons link tables to x4 for all requests (T437108)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2455395 wikitext text/x-wiki lu7wu3h4v474japum9t7268nu37or4c 2455396 2455395 2026-09-07T21:33:03Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2455396 wikitext text/x-wiki 3cz4bjvt7tpao4oz14m53713k61mkc8 2455397 2455396 2026-09-07T21:37:43Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1337319|Move production reads for commons link tables to x4 for all requests (T437108)]] (duration: 09m 34s) 2455397 wikitext text/x-wiki fiy8le94s7gijv11ctpjd6o3e3sx6a2 2455398 2455397 2026-09-07T21:41:00Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1337635|Move globalimagelinks queries to x4 (T437108)]] 2455398 wikitext text/x-wiki h1r6cqibawzomz0v9fyypl8tmeqjlmq 2455399 2455398 2026-09-07T21:45:09Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1337635|Move globalimagelinks queries to x4 (T437108)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2455399 wikitext text/x-wiki eni5hhjyt4edr63qppeyqq22s5khs0e 2455400 2455399 2026-09-07T21:47:23Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2455400 wikitext text/x-wiki 6ajmmkzgpgku4xl15jjwrqiokp8tsij 2455401 2455400 2026-09-07T21:52:01Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1337635|Move globalimagelinks queries to x4 (T437108)]] (duration: 11m 00s) 2455401 wikitext text/x-wiki pnc63i1tmbtiovwz6ksjtyqaygtatm4 2455404 2455401 2026-09-08T02:01:47Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2455404 wikitext text/x-wiki t725wa84thlgiavf6ivqhvwd2i43vq7 2455405 2455404 2026-09-08T02:09:27Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 07m 41s) 2455405 wikitext text/x-wiki jpzco4onkiuo21a6092a9u5nxyxr72z 2455407 2455405 2026-09-08T03:03:23Z Stashbot 7414 mwpresync@deploy1003: Started scap sync-world: testwikis to 1.47.0-wmf.19 refs T430838 2455407 wikitext text/x-wiki gu19nw6y5myg2109k87su3s524xxavd 2455408 2455407 2026-09-08T03:39:53Z Stashbot 7414 mwpresync@deploy1003: Finished scap sync-world: testwikis to 1.47.0-wmf.19 refs T430838 (duration: 36m 30s) 2455408 wikitext text/x-wiki 1pkttgxgg1aubocz0t26ufser6aemcq 2455409 2455408 2026-09-08T04:02:30Z Stashbot 7414 mwpresync@deploy1003: Pruned MediaWiki: 1.47.0-wmf.16 (duration: 02m 27s) 2455409 wikitext text/x-wiki 0hennkhsbf2h4y9qco0jd740jdot6xk 2455413 2455409 2026-09-08T05:07:31Z Stashbot 7414 denisse: Add grafana-plugins 0.10 to bookworm-wikimedia - T436056 2455413 wikitext text/x-wiki 9l0iv6rik1on9nn2paqarkjd3ceixd0 2455414 2455413 2026-09-08T07:12:01Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Remove hosts from s4 as they should only be in x4 T404715', diff saved to https://phabricator.wikimedia.org/P96386 and previous config saved to /var/cache/conftool/dbconfig/20260908-071159-marostegui.json 2455414 wikitext text/x-wiki hc0ohdhu4x8x45jlswjt680f4vy6tf5 2455416 2455414 2026-09-08T07:12:19Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Remove hosts from s4 as they should only be in x4 T404715', diff saved to https://phabricator.wikimedia.org/P96387 and previous config saved to /var/cache/conftool/dbconfig/20260908-071216-marostegui.json 2455416 wikitext text/x-wiki titbd1eg6k53kro8rkvmebvx05nr5zw 2455417 2455416 2026-09-08T07:12:34Z Stashbot 7414 jmm@dns1004: START - running authdns-update 2455417 wikitext text/x-wiki 2bt3qqbrqwuesg6zbgebjzsjmw9frje 2455418 2455417 2026-09-08T07:13:11Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Remove hosts from s4 as they should only be in x4 T404715', diff saved to https://phabricator.wikimedia.org/P96388 and previous config saved to /var/cache/conftool/dbconfig/20260908-071308-marostegui.json 2455418 wikitext text/x-wiki h6bhgvim1su4lohv8srlso4xw6kmg8c 2455419 2455418 2026-09-08T07:14:44Z Stashbot 7414 jmm@dns1004: END - running authdns-update 2455419 wikitext text/x-wiki d83z5b23796c27hco1nevynwp54o4y2 2455421 2455419 2026-09-08T07:18:14Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2006.codfw.wmnet 2455421 wikitext text/x-wiki 7z8kv7bta5wuedtrikjx9btk296d8zv 2455422 2455421 2026-09-08T07:22:29Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.ganeti.makevm for new host ldap-replica1006.wikimedia.org 2455422 wikitext text/x-wiki cc8z5xv7lfbl28y6mtvgmjwlskzmsa1 2455423 2455422 2026-09-08T07:22:31Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.dns.netbox 2455423 wikitext text/x-wiki 2ycj8h74zqg1jf2n390ll4h2la3klhp 2455424 2455423 2026-09-08T07:26:19Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ldap-replica1006.wikimedia.org - jmm@cumin1004" 2455424 wikitext text/x-wiki tt5qliru2f8zivjd18y895vwl67h8z9 2455425 2455424 2026-09-08T07:26:21Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ldap-replica1006.wikimedia.org - jmm@cumin1004" 2455425 wikitext text/x-wiki rvg7axa1gq2y45vsvpkcrlmo3noqgiv 2455426 2455425 2026-09-08T07:26:22Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2455426 wikitext text/x-wiki qpia1epawfdindrwg8xt16bwnhfatw9 2455427 2455426 2026-09-08T07:26:23Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.dns.wipe-cache ldap-replica1006.wikimedia.org on all recursors 2455427 wikitext text/x-wiki 3bf1p4dpppe3etfmk6obb7hp9q4y6tj 2455428 2455427 2026-09-08T07:26:24Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) ldap-replica1006.wikimedia.org on all recursors 2455428 wikitext text/x-wiki nrp0p0dw0275cgfygyeu3z3sbb44uv3 2455429 2455428 2026-09-08T07:27:04Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ldap-replica1006.wikimedia.org - jmm@cumin1004" 2455429 wikitext text/x-wiki c812pqnjj1icscp5as7bgkrfwmvf2gj 2455430 2455429 2026-09-08T07:27:08Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ldap-replica1006.wikimedia.org - jmm@cumin1004" 2455430 wikitext text/x-wiki tolytzgpv1i48lldddyaqt5qapshkli 2455431 2455430 2026-09-08T07:27:12Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.hosts.reimage for host ldap-replica1006.wikimedia.org with OS trixie 2455431 wikitext text/x-wiki goqrz2dayf6ltdha2d2f9fx15sj4ka9 2455432 2455431 2026-09-08T07:28:40Z Stashbot 7414 mvernon@cumin2003: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts aqs2006.codfw.wmnet 2455432 wikitext text/x-wiki nfpldao84d205bjxn988al4bq831ywb 2455433 2455432 2026-09-08T07:28:53Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2006.codfw.wmnet 2455433 wikitext text/x-wiki fq1e0e8dgz7lzcpca2zbx55vrxaeyw9 2455434 2455433 2026-09-08T07:29:22Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reboot-single for host aqs2006.codfw.wmnet 2455434 wikitext text/x-wiki fg3gjozbw0qg1x1niax356cyr8dxrgw 2455435 2455434 2026-09-08T07:30:07Z Stashbot 7414 denisse: Add grafana-plugins 0.15 to bookworm-wikimedia - T436056 2455435 wikitext text/x-wiki gfvwsdnmu0e10s0jj61gaq5wcjgvj8h 2455436 2455435 2026-09-08T07:37:42Z Stashbot 7414 jmm@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ldap-replica1006.wikimedia.org with reason: host reimage 2455436 wikitext text/x-wiki 8ti5echu8ptbp5sgrrikrksjczoa9eu 2455437 2455436 2026-09-08T07:38:31Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host aqs2006.codfw.wmnet 2455437 wikitext text/x-wiki kmrtlmnp61t36gk7qf78jhcrlcmrzn9 2455438 2455437 2026-09-08T07:38:33Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts aqs2006.codfw.wmnet 2455438 wikitext text/x-wiki 03u3g1fm3ep16obvi6ldmstubaabnqn 2455439 2455438 2026-09-08T07:39:00Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.provision for host aqs2006.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455439 wikitext text/x-wiki bakeeqs7u05sb3kvipuntgqze0nbbwz 2455440 2455439 2026-09-08T07:40:45Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs2006.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455440 wikitext text/x-wiki d5nax86tabl7tz99nmlqklgv1o2ekm0 2455441 2455440 2026-09-08T07:41:17Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host aqs2006.codfw.wmnet with OS bookworm 2455441 wikitext text/x-wiki 00vni972hfl28hcs34w58ym4zrr3uqv 2455442 2455441 2026-09-08T07:43:16Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ldap-replica1006.wikimedia.org with reason: host reimage 2455442 wikitext text/x-wiki ilamltydcn7hdsvmwbkbqkmey6xha0u 2455443 2455442 2026-09-08T07:50:29Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2007.codfw.wmnet 2455443 wikitext text/x-wiki jgjjo7b078086fhom9sfeodcnwwmxey 2455444 2455443 2026-09-08T07:58:25Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs2006.codfw.wmnet with reason: host reimage 2455444 wikitext text/x-wiki oofb8kzzwty9imx6z8xiylabbx9jsz4 2455445 2455444 2026-09-08T07:58:54Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ldap-replica1006.wikimedia.org with OS trixie 2455445 wikitext text/x-wiki jbx77okkcb51up2wpwmla2sdtideg9s 2455446 2455445 2026-09-08T07:58:55Z Stashbot 7414 jmm@cumin1004: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host ldap-replica1006.wikimedia.org 2455446 wikitext text/x-wiki nyzr6y6dtv1h95k9uicniveugcv5562 2455448 2455446 2026-09-08T08:03:53Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs2006.codfw.wmnet with reason: host reimage 2455448 wikitext text/x-wiki 9bqp7ei76rjmcu76bws8h0u2srcg32o 2455449 2455448 2026-09-08T08:25:27Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs2006.codfw.wmnet with OS bookworm 2455449 wikitext text/x-wiki 4ntewkp9my83tlo747t1d9npieqlsm1 2455450 2455449 2026-09-08T08:26:12Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hosts.reboot-single for host aqs2007.codfw.wmnet 2455450 wikitext text/x-wiki 3ydxy61s4v9h9rjsfou7vnos588spiq 2455451 2455450 2026-09-08T08:29:13Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.deploy.python-code homer to cumin1004.eqiad.wmnet with reason: Release v0.11.3 - ayounsi@cumin1003 2455451 wikitext text/x-wiki 8me6rtx9lu71z21zw4yg40zmo90dfmq 2455452 2455451 2026-09-08T08:30:03Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin1004.eqiad.wmnet with reason: Release v0.11.3 - ayounsi@cumin1003 2455452 wikitext text/x-wiki 4zuwyzn9rg4h218evlg43j674tazlgm 2455453 2455452 2026-09-08T08:34:43Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.deploy.python-code homer to cumin1003.eqiad.wmnet with reason: Release v0.11.3 - ayounsi@cumin1003 2455453 wikitext text/x-wiki na3c2oq4i7xkujeggd5zhshllepjstc 2455454 2455453 2026-09-08T08:35:33Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin1003.eqiad.wmnet with reason: Release v0.11.3 - ayounsi@cumin1003 2455454 wikitext text/x-wiki 4afchgiayl99ynhu3khu0jcivi9odv2 2455455 2455454 2026-09-08T08:36:46Z Stashbot 7414 mvernon@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host aqs2007.codfw.wmnet 2455455 wikitext text/x-wiki 9an3vec42ijonsxvmc7mcqjwk66i6zp 2455456 2455455 2026-09-08T08:36:48Z Stashbot 7414 mvernon@cumin1003: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts aqs2007.codfw.wmnet 2455456 wikitext text/x-wiki 60gawbxzq8e78t08r8iya3hchtpgr63 2455457 2455456 2026-09-08T08:37:50Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.deploy.python-code homer to cumin2003.codfw.wmnet with reason: Release v0.11.3 - ayounsi@cumin1003 2455457 wikitext text/x-wiki 8ujwv44qyl90vmcot14aktagjdoc3xo 2455458 2455457 2026-09-08T08:37:54Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.provision for host aqs2007.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455458 wikitext text/x-wiki pl54glk3jr67o1t5gg2pn61szampk62 2455459 2455458 2026-09-08T08:38:38Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2003.codfw.wmnet with reason: Release v0.11.3 - ayounsi@cumin1003 2455459 wikitext text/x-wiki f85trstanxyr2mgocm9ab0b2yvvgbai 2455460 2455459 2026-09-08T08:39:39Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs2007.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455460 wikitext text/x-wiki ko22gqocbmuhs7jteu7qf32i00tqfp9 2455461 2455460 2026-09-08T08:41:28Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host aqs2007.codfw.wmnet with OS bookworm 2455461 wikitext text/x-wiki qujoqusq1nzzyuz0eghhqukxcf1w0sj 2455462 2455461 2026-09-08T08:43:57Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 32 hosts with reason: x4 split 2455462 wikitext text/x-wiki 0iabyp71wuoimwkd7xonfi628kjckqh 2455463 2455462 2026-09-08T08:55:10Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-liftwing-studio: sync 2455463 wikitext text/x-wiki mzqq049c5koumyx8mv4bewt4o2i3txp 2455464 2455463 2026-09-08T08:55:13Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-liftwing-studio: sync 2455464 wikitext text/x-wiki cmz3xkz4lwjzez6ha7w2k9n54decffc 2455465 2455464 2026-09-08T08:58:33Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2008.codfw.wmnet 2455465 wikitext text/x-wiki atka24h5s115hsg4pteinic6qke2tx5 2455467 2455465 2026-09-08T09:00:12Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs2007.codfw.wmnet with reason: host reimage 2455467 wikitext text/x-wiki ku6pjso0n9by4wp22futfv60paigm4j 2455469 2455467 2026-09-08T09:02:01Z Stashbot 7414 marostegui: Starting x4 split from s4, RO time on commons needed T404715 2455469 wikitext text/x-wiki rjw91pgbqbzzxy8n66dcdsi02gklkoq 2455470 2455469 2026-09-08T09:02:31Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Set s4 commons to read-only for maintenance T404715', diff saved to https://phabricator.wikimedia.org/P96389 and previous config saved to /var/cache/conftool/dbconfig/20260908-090228-marostegui.json 2455470 wikitext text/x-wiki eunez4qdeeiopzhzvaw45cfif30unrn 2455471 2455470 2026-09-08T09:03:19Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs2007.codfw.wmnet with reason: host reimage 2455471 wikitext text/x-wiki s3aujo7dowevnqjn2dxmrw4hy2gpwdi 2455472 2455471 2026-09-08T09:05:20Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Set x4 masters T404715', diff saved to https://phabricator.wikimedia.org/P96391 and previous config saved to /var/cache/conftool/dbconfig/20260908-090517-marostegui.json 2455472 wikitext text/x-wiki dqd5m9ney310zpq3vd8l0kvj9egwkav 2455475 2455472 2026-09-08T09:07:52Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Remove old s4 masters from x4 T404715', diff saved to https://phabricator.wikimedia.org/P96392 and previous config saved to /var/cache/conftool/dbconfig/20260908-090749-marostegui.json 2455475 wikitext text/x-wiki bpo1eq05vg2dnfdptu30ion4lfih483 2455476 2455475 2026-09-08T09:11:24Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Make x4 and s4 RW again T404715', diff saved to https://phabricator.wikimedia.org/P96393 and previous config saved to /var/cache/conftool/dbconfig/20260908-091121-marostegui.json 2455476 wikitext text/x-wiki t546vzn5ktfr2e6b85p4djkdbyo3wo4 2455479 2455476 2026-09-08T09:22:28Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hosts.reboot-single for host aqs2008.codfw.wmnet 2455479 wikitext text/x-wiki b36jervgjq2a0moq5y3a7d20pmnk27r 2455480 2455479 2026-09-08T09:22:29Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs2007.codfw.wmnet with OS bookworm 2455480 wikitext text/x-wiki r9kdf7uxl48c5o1iy9z2p2ctvhl5vu7 2455481 2455480 2026-09-08T09:23:34Z Stashbot 7414 moritzm: installing rsync security updates 2455481 wikitext text/x-wiki 2bcrpgwdw7au8m7c0s41tcrst676cbc 2455482 2455481 2026-09-08T09:29:48Z Stashbot 7414 XioNoX: remove GRE tunnels eqiad-drmrs eqdfw-ulsfo 2455482 wikitext text/x-wiki e8z6kxqt9ekc7zr7a0hfntqp672qvbs 2455483 2455482 2026-09-08T09:31:01Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.provision for host aqs2008.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455483 wikitext text/x-wiki ikh27earn6x4m5m1h3i7ew4woc759ad 2455484 2455483 2026-09-08T09:31:24Z Stashbot 7414 mvernon@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host aqs2008.codfw.wmnet 2455484 wikitext text/x-wiki 2371qzr00x0gvk2rdhne0yjnixvvvlt 2455485 2455484 2026-09-08T09:31:24Z Stashbot 7414 mvernon@cumin1003: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts aqs2008.codfw.wmnet 2455485 wikitext text/x-wiki nreg9r1ivi7di9vdfobn292lj9x4t19 2455486 2455485 2026-09-08T09:32:09Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.dns.netbox 2455486 wikitext text/x-wiki qb42gpdl2ksmbnjpl5t32yrwv1d0yem 2455487 2455486 2026-09-08T09:32:43Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1337856|Use virtual domain in NameTableStore for collation (T405812)]], [[gerrit:1337857|Use virtual domain in NameTableStore for collation (T405812)]] 2455487 wikitext text/x-wiki 44yf0rjfp1pfz5xgw1nlqkceenmni9y 2455488 2455487 2026-09-08T09:32:47Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs2008.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455488 wikitext text/x-wiki iawan29lbxhaebkk6h3ubdfqaucvxjk 2455489 2455488 2026-09-08T09:33:21Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host aqs2008.codfw.wmnet with OS bookworm 2455489 wikitext text/x-wiki tsg9h6uzca9wsegiowy8wtl59z8xk31 2455490 2455489 2026-09-08T09:34:59Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: remove include for former GRE tunnels v6 PTR - ayounsi@cumin1003" 2455490 wikitext text/x-wiki 3kzmg1wh263bsvg3xgj0fciy0tl8s49 2455491 2455490 2026-09-08T09:35:42Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: remove include for former GRE tunnels v6 PTR - ayounsi@cumin1003" 2455491 wikitext text/x-wiki 0pc4tkwi15ir769bsze2wxypzbkdexb 2455492 2455491 2026-09-08T09:35:43Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2455492 wikitext text/x-wiki evj7skf2m2xace213c97k4qkf5cal35 2455493 2455492 2026-09-08T09:37:10Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1337856|Use virtual domain in NameTableStore for collation (T405812)]], [[gerrit:1337857|Use virtual domain in NameTableStore for collation (T405812)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2455493 wikitext text/x-wiki rpjib2l9ondbngjaaabstsxmcfrhh8k 2455494 2455493 2026-09-08T09:39:14Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with deployment 2455494 wikitext text/x-wiki smw4szue8pslm6nna10eoidkv86kzqw 2455495 2455494 2026-09-08T09:43:10Z Stashbot 7414 ayounsi@dns1004: START - running authdns-update 2455495 wikitext text/x-wiki pwhr9k0qcvestdsqrd6oqc6qvzbkjps 2455496 2455495 2026-09-08T09:44:19Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2009.codfw.wmnet 2455496 wikitext text/x-wiki 7dh2yrrkb8p5jn1v76p68p32de1cmxn 2455497 2455496 2026-09-08T09:45:23Z Stashbot 7414 ayounsi@dns1004: END - running authdns-update 2455497 wikitext text/x-wiki k165osrpk6w4ty3fk9tr4apx295dg7w 2455498 2455497 2026-09-08T09:45:58Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1337856|Use virtual domain in NameTableStore for collation (T405812)]], [[gerrit:1337857|Use virtual domain in NameTableStore for collation (T405812)]] (duration: 13m 15s) 2455498 wikitext text/x-wiki tpdmt55n11xlsva2pi00i484qvqzz83 2455499 2455498 2026-09-08T09:51:51Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs2008.codfw.wmnet with reason: host reimage 2455499 wikitext text/x-wiki cy5gmaorny3hk3f0g16b7y9cgdryt76 2455500 2455499 2026-09-08T09:55:00Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs2008.codfw.wmnet with reason: host reimage 2455500 wikitext text/x-wiki 9o4lweequn4sb8njuku9cavly6wugz0 2455501 2455500 2026-09-08T09:59:04Z Stashbot 7414 mvernon@cumin1003: END (ERROR) - Cookbook sre.hardware.upgrade-firmware (exit_code=97) upgrade firmware for hosts aqs2009.codfw.wmnet 2455501 wikitext text/x-wiki dvzn4as8tot9hrmwnuegs4wms4b3qg4 2455502 2455501 2026-09-08T09:59:09Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2009.codfw.wmnet 2455502 wikitext text/x-wiki jtxgexys894b17gopfqqg6w476c7rlh 2455503 2455502 2026-09-08T10:01:38Z Stashbot 7414 mvernon@cumin1003: END (ERROR) - Cookbook sre.hardware.upgrade-firmware (exit_code=97) upgrade firmware for hosts aqs2009.codfw.wmnet 2455503 wikitext text/x-wiki rx0lorn91a7x1991doqmy4dybfb6ed5 2455504 2455503 2026-09-08T10:04:19Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2009.codfw.wmnet 2455504 wikitext text/x-wiki s1en73thnrksdbkqvxwk4cojuxr3z70 2455505 2455504 2026-09-08T10:04:52Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-liftwing-studio: sync 2455505 wikitext text/x-wiki j9mib3spxgfpch59ynpvytp7t143cmk 2455506 2455505 2026-09-08T10:04:54Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-liftwing-studio: sync 2455506 wikitext text/x-wiki 0wuf1bne8ug6s0zgvgw7wmmisl6qj5v 2455507 2455506 2026-09-08T10:05:07Z Stashbot 7414 mvernon@cumin1003: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts aqs2009.codfw.wmnet 2455507 wikitext text/x-wiki 8ly42qe3xheudbea908rzbnhx3wb4s9 2455508 2455507 2026-09-08T10:07:50Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2009.codfw.wmnet 2455508 wikitext text/x-wiki lmgfjrsqmtev8c7wqk1ui054ktkqwe5 2455509 2455508 2026-09-08T10:12:30Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs2008.codfw.wmnet with OS bookworm 2455509 wikitext text/x-wiki 3kom5hwyjb5nk2bmv2nr19qptlwqjf9 2455510 2455509 2026-09-08T10:18:42Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hosts.reboot-single for host aqs2009.codfw.wmnet 2455510 wikitext text/x-wiki lkpz8hwlia62y28nnbs5lhp5jhxnx43 2455511 2455510 2026-09-08T10:26:36Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.provision for host aqs2009.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455511 wikitext text/x-wiki q794q4uw1lgxtrllmo8lxcrep812asr 2455512 2455511 2026-09-08T10:27:50Z Stashbot 7414 mvernon@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host aqs2009.codfw.wmnet 2455512 wikitext text/x-wiki gxw9vxbclvqftc88v48ieoere6lsmu3 2455513 2455512 2026-09-08T10:27:52Z Stashbot 7414 mvernon@cumin1003: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts aqs2009.codfw.wmnet 2455513 wikitext text/x-wiki 16fjhfhhwkpwhf2n0ad17bl8pozw3mi 2455514 2455513 2026-09-08T10:28:22Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs2009.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455514 wikitext text/x-wiki 4e2y2u4zl7y9q66xielh4b537nzu9bl 2455515 2455514 2026-09-08T10:28:40Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host aqs2009.codfw.wmnet with OS bookworm 2455515 wikitext text/x-wiki hyzw1feto1nyo7axlx15sgx7zfd9tb8 2455516 2455515 2026-09-08T10:30:11Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2010.codfw.wmnet 2455516 wikitext text/x-wiki duhr8xg7r9l9an73t5kem6lcakfmgf8 2455517 2455516 2026-09-08T10:32:05Z Stashbot 7414 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1307862|IS/IS-labs: wgEnableWatchstarPopover default false, enable for enwiki beta (T431355)]] 2455517 wikitext text/x-wiki f0ckvv1tqk6xw1depz2y0joo03ee40u 2455518 2455517 2026-09-08T10:34:49Z Stashbot 7414 mvernon@cumin1003: END (ERROR) - Cookbook sre.hardware.upgrade-firmware (exit_code=97) upgrade firmware for hosts aqs2010.codfw.wmnet 2455518 wikitext text/x-wiki ji10tn3ccp1uyhrlpkq4dgfs2gh0nub 2455519 2455518 2026-09-08T10:36:50Z Stashbot 7414 samtar@deploy1003: samtar: Backport for [[gerrit:1307862|IS/IS-labs: wgEnableWatchstarPopover default false, enable for enwiki beta (T431355)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2455519 wikitext text/x-wiki omasll0qqqqgugn8ednft7jrslb7utj 2455520 2455519 2026-09-08T10:37:05Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2010.codfw.wmnet 2455520 wikitext text/x-wiki eypd4epuepe84x0knnyjkvkivrcwgir 2455521 2455520 2026-09-08T10:37:30Z Stashbot 7414 mvernon@cumin1003: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts aqs2010.codfw.wmnet 2455521 wikitext text/x-wiki 3q6ujcxhz84ddudtwi5bihvr4t9yeu7 2455522 2455521 2026-09-08T10:38:20Z Stashbot 7414 samtar@deploy1003: samtar: Continuing with deployment 2455522 wikitext text/x-wiki 6yzvzqvgvzoft8aidp24v7mdi5tc4x8 2455523 2455522 2026-09-08T10:39:18Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2010.codfw.wmnet 2455523 wikitext text/x-wiki m8smtx4k9mkxofis5mqo02bjpkke55b 2455524 2455523 2026-09-08T10:43:03Z Stashbot 7414 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1307862|IS/IS-labs: wgEnableWatchstarPopover default false, enable for enwiki beta (T431355)]] (duration: 10m 57s) 2455524 wikitext text/x-wiki 71hhjgva4nqfe1hrc7l0jdi1xwd57dt 2455525 2455524 2026-09-08T10:47:25Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs2009.codfw.wmnet with reason: host reimage 2455525 wikitext text/x-wiki hx9kocuepiwzvtg3rwfeldg7vnj7upg 2455526 2455525 2026-09-08T10:53:10Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs2009.codfw.wmnet with reason: host reimage 2455526 wikitext text/x-wiki mhqzmsvo9zimz8noiwopggwgft7j9mi 2455527 2455526 2026-09-08T10:56:44Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-liftwing-studio: sync 2455527 wikitext text/x-wiki ciiq2z83geaerkcwjl9wfa6lpay3dzo 2455528 2455527 2026-09-08T10:56:47Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-liftwing-studio: sync 2455528 wikitext text/x-wiki g1gvwupy3t3rpkhf3wh6dgh3z78jgex 2455529 2455528 2026-09-08T10:57:04Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-liftwing-studio: sync 2455529 wikitext text/x-wiki a1qctj0tag53lbisr51pcr5ei7k01nx 2455530 2455529 2026-09-08T10:57:06Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-liftwing-studio: sync 2455530 wikitext text/x-wiki idzhfj5ictfel93jv9zbd1svvhfvr2t 2455531 2455530 2026-09-08T11:10:56Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs2009.codfw.wmnet with OS bookworm 2455531 wikitext text/x-wiki h15ocg7vtq7nks3p9qxhi0e5rsqckh0 2455532 2455531 2026-09-08T11:12:17Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hosts.reboot-single for host aqs2010.codfw.wmnet 2455532 wikitext text/x-wiki n8xoxbypjp7ugtbvrtlmt2j3j7pzp3c 2455533 2455532 2026-09-08T11:20:09Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.provision for host aqs2010.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455533 wikitext text/x-wiki r0im1wz65bd1t073361gogn8l3qv4hk 2455534 2455533 2026-09-08T11:21:54Z Stashbot 7414 mvernon@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host aqs2010.codfw.wmnet 2455534 wikitext text/x-wiki 0dfx4lofm54jmpz617nj66fagy7709v 2455535 2455534 2026-09-08T11:21:56Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs2010.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2455535 wikitext text/x-wiki 7hbzpt5jjznq22392fx6x8nevryq05r 2455536 2455535 2026-09-08T11:21:57Z Stashbot 7414 mvernon@cumin1003: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts aqs2010.codfw.wmnet 2455536 wikitext text/x-wiki gnqoe0ajxdhmj28n5n79w8d8u57a395 2455537 2455536 2026-09-08T11:22:18Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host aqs2010.codfw.wmnet with OS bookworm 2455537 wikitext text/x-wiki 9fcda68611qy9ey3fh56w4bb36uqsjh 2455538 2455537 2026-09-08T11:23:23Z Stashbot 7414 mvernon@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts aqs2011.codfw.wmnet 2455538 wikitext text/x-wiki hm1g6le5ag7hqaiu83ag8stldkfzodi 2455539 2455538 2026-09-08T11:39:15Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs2010.codfw.wmnet with reason: host reimage 2455539 wikitext text/x-wiki d0c1dfxvmfb9uvb3ohtqjavjsd0llxj 2455540 2455539 2026-09-08T11:40:43Z Stashbot 7414 Amir1: dropping unneeded tables from x4 - db1260 (T437278) 2455540 wikitext text/x-wiki 7r913ygp0q1gm9gxajhzdcmawsdm4dm 2455541 2455540 2026-09-08T11:43:24Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs2010.codfw.wmnet with reason: host reimage 2455541 wikitext text/x-wiki pltq2rz1ke99srsmnpj2akzlpc7ibq3 2455542 2455541 2026-09-08T11:44:36Z Stashbot 7414 marostegui@dns1004: START - running authdns-update 2455542 wikitext text/x-wiki 9zfefrhd9pt4y9a83pl7cezeuq215eh 2455543 2455542 2026-09-08T11:46:53Z Stashbot 7414 marostegui@dns1004: END - running authdns-update 2455543 wikitext text/x-wiki 1c2gkqkwswsgohgl4g1fpx3fwqipjca 2455544 2455543 2026-09-08T11:56:36Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1337898|Skip DB read-only check for CentralAuthSessionManager (T437273)]], [[gerrit:1337899|Skip DB read-only check for CentralAuthSessionManager (T437273)]] 2455544 wikitext text/x-wiki tc3ilf770359nokvdl46dykdjw5jigo 498 9925 2455352 2454669 2026-09-07T17:55:22Z Stashbot 7414 Southparkfan: no disk space left on deployment-mediawiki14, cleared logs in /var/log/apache2/forensic to unbreak 2455352 wikitext text/x-wiki agvc0mqfabhaqjgqxlwrn66k7eea1e1 2455365 2455352 2026-09-07T19:03:46Z Stashbot 7414 Southparkfan: banhammer lots of ranges to get Beta Cluster back online; not sure it was very effective, but we seem to be out of the woods 2455365 wikitext text/x-wiki 6dn38fsbxm1um3dmoly49hxq2218l3x 2455367 2455365 2026-09-07T19:06:08Z Stashbot 7414 Southparkfan: 18:05 UTC: pool mediawiki15 and mediawiki16 (PHP 8.5) as replacements for 13 and 14 (PHP 8.3), smoke test - T435393 2455367 wikitext text/x-wiki qgtr3ko4any65eouu82b5ili7zc1jxk 2455374 2455367 2026-09-07T19:31:44Z Stashbot 7414 Southparkfan: 19:31 UTC: switch back from PHP 8.5 to PHP 8.3 hosts - T435393 2455374 wikitext text/x-wiki juhiqfo6ueenk9ql3b8lm79euz88d2r 0 17290 2455351 2454843 2026-09-07T17:55:21Z Stashbot 7414 Southparkfan: no disk space left on deployment-mediawiki14, cleared logs in /var/log/apache2/forensic to unbreak 2455351 wikitext text/x-wiki 5034ofp81fcw3ou6cb7yqbevfszd4j3 2455364 2455351 2026-09-07T19:03:45Z Stashbot 7414 Southparkfan: banhammer lots of ranges to get Beta Cluster back online; not sure it was very effective, but we seem to be out of the woods 2455364 wikitext text/x-wiki 6qoxilgqwtivee4zvt6m7i25sz2i7ir 2455366 2455364 2026-09-07T19:06:05Z Stashbot 7414 Southparkfan: 18:05 UTC: pool mediawiki15 and mediawiki16 (PHP 8.5) as replacements for 13 and 14 (PHP 8.3), smoke test - T435393 2455366 wikitext text/x-wiki bgk00y3f5qm1uq39wc3vyzftms9uguy 2455373 2455366 2026-09-07T19:31:42Z Stashbot 7414 Southparkfan: 19:31 UTC: switch back from PHP 8.5 to PHP 8.3 hosts - T435393 2455373 wikitext text/x-wiki 631u4njgg9wva5to20978iub6xz9rwx 0 19208 2455347 2435853 2026-09-07T16:28:22Z Martin Urbanec (WMF) 37730 phab1004 => phab1005 2455347 wikitext text/x-wiki b4jvzkpkgh9bxz5ip72dkqs465kx5jc 2455348 2455347 2026-09-07T16:30:22Z Martin Urbanec (WMF) 37730 phab2002 => phab2003 2455348 wikitext text/x-wiki q1garh2igxtv26azbpc7zwzosirxk9c 0 24248 2455410 2452285 2026-09-08T04:41:00Z ADenisse-WMF 31462 /* Installing plugins */ 2455410 wikitext text/x-wiki puoh3gingrnlrkdsd5bkt4ujmlwx34e 2455411 2455410 2026-09-08T04:41:40Z ADenisse-WMF 31462 /* Installing plugins */ 2455411 wikitext text/x-wiki dpfk87vqp9ai6rp35ah02ssavyzrus2 498 30942 2455204 2453971 2026-09-07T13:19:22Z Stashbot 7414 godog: re-apply cloudvirt role to moved hosts - T435921 2455204 wikitext text/x-wiki ar2g15x94bok7nxgius9wv2x2wcawhv 2455249 2455204 2026-09-07T14:02:02Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/349 2455249 wikitext text/x-wiki gnn87bjyfv47pg3tqq5nczon4jktza2 2455252 2455249 2026-09-07T14:02:24Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/349 2455252 wikitext text/x-wiki r56usudwwjk5o38idcazht6d9hqxd33 2455280 2455252 2026-09-07T14:22:54Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch 2455280 wikitext text/x-wiki br4u2chch8cweywxhtq9pfvb4o6gern 2455281 2455280 2026-09-07T14:23:18Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch 2455281 wikitext text/x-wiki 4huilnbdzro31neisvcadfvaorx2z75 12 441751 2455360 2455090 2026-09-07T18:39:19Z Taavi-WMF 41365 /* Extension databases */ link to News page 2455360 wikitext text/x-wiki 4rw10unwwfhfmzbovc09nh7kfyw6q31 0 442914 2455336 2455085 2026-09-07T15:50:18Z JJMC89 7474 Reverted edit by [[Special:Contributions/RAdin satoshi2008|RAdin satoshi2008]] ([[User talk:RAdin satoshi2008|talk]]) to last revision by [[User:Akd.2303|Akd.2303]] 2452780 wikitext text/x-wiki 05cxp45b5en1yxcv1reolomg0w6i2rq 498 443949 2455376 2452996 2026-09-07T19:49:34Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy d1b67a1 (Backfill page_creator / page_created_at in Dashboard Registry) for T435900 2455376 wikitext text/x-wiki im06e3hlzefw0zbftk76dluqtvz2z8q 498 447034 2455349 2452972 2026-09-07T17:14:35Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed 02470b6322 (l10n updates: he) 2455349 wikitext text/x-wiki jeoqnb40yy3limp9tg0i3ih3wu83qxp 0 449160 2455389 2454976 2026-09-07T20:40:49Z Dexbot 30554 Bot: Updating the report 2455389 wikitext text/x-wiki 785aqzd124e5fkcwdx6oex902rozto1 2455402 2455389 2026-09-08T00:02:13Z Dexbot 30554 Bot: Updating the report 2455402 wikitext text/x-wiki obxmm7cu8m3yyzm5ag3x0ikskkc4fx5 2455415 2455402 2026-09-08T07:12:05Z Dexbot 30554 Bot: Updating the report 2455415 wikitext text/x-wiki r2wsbhwviu29poypo3r5xb97xjlxgdc 2455420 2455415 2026-09-08T07:15:06Z Dexbot 30554 Bot: Updating the report 2455420 wikitext text/x-wiki j2arez75lw05ketbim6dw0zjz9tyn21 2455473 2455420 2026-09-08T09:06:20Z Dexbot 30554 Bot: Updating the report 2455473 wikitext text/x-wiki oupi3a1hupo4oyiaxq2m9n1t1w5d4xt 0 452153 2455159 2434964 2026-09-07T12:05:21Z DCaro (WMF) 18271 Added disabled tools ui 2455159 wikitext text/x-wiki ls5izv15eaivyy7tgowtiqtznjftwjk 0 452383 2455350 2453566 2026-09-07T17:18:56Z HNowlan (WMF) 15760 Add manager rotation 2455350 wikitext text/x-wiki jouycf9qcdudazazvnrcypjj52q6exv 12 452945 2455169 2394692 2026-09-07T12:49:03Z DCaro (WMF) 18271 /* Step 2: Create a basic PHP WSGI webservice for Apache */ updated to use jobs for publishing 2455169 wikitext text/x-wiki 4mqc2p5p684tagulcydoepml271pf3w 2455175 2455169 2026-09-07T12:57:54Z DCaro (WMF) 18271 /* How to create a basic PHP WSGI webservice */ 2455175 wikitext text/x-wiki ncgb11n75ggip3r7bl760uvf0l8w0vm 116 453906 2455329 2454946 2026-09-07T15:45:27Z Gitlabaccountapprovalbot 37332 @dvrted was approved. 2455329 wikitext text/x-wiki pwg7ato9y6itcazne2dqlpm4j10tmrp 2455406 2455329 2026-09-08T02:39:15Z Gitlabaccountapprovalbot 37332 tantomile was rejected. 2455406 wikitext text/x-wiki e38jv6tt2zqmsvvstb5zq0p7nyuygpd 1 460409 2455170 2451127 2026-09-07T12:51:32Z Peter F. Patel-Schneider 42591 /* status of label service queries */ Reply 2455170 wikitext text/x-wiki 821mo2nje3f9vs443cj5tpkh3zh5nye 0 460424 2455466 2453448 2026-09-08T08:59:49Z DMburugu (WMF) 25135 /* Operating Procedures */ add to operating procedures 2455466 wikitext text/x-wiki 7yyeklixdq08p7g5bkxfidwcpmxb7e4 2455468 2455466 2026-09-08T09:01:09Z DMburugu (WMF) 25135 /* Operating Procedures */ add to maintenance section of operating procedures 2455468 wikitext text/x-wiki hoymx4g6om0cwgffqu7oqt1td5akc8s 2455474 2455468 2026-09-08T09:07:29Z DMburugu (WMF) 25135 /* Release Confidence */ add to release confidence section 2455474 wikitext text/x-wiki 6yr3wtnmo10jie3exj2ks9dhcuuxgbk 2455477 2455474 2026-09-08T09:14:38Z DMburugu (WMF) 25135 /* Observability */ add to observability section 2455477 wikitext text/x-wiki im4rpiws6fumu4j7im3h2asenx7lxxn 2455478 2455477 2026-09-08T09:18:48Z DMburugu (WMF) 25135 /* Reliability and performance */ add to reliability and performance section 2455478 wikitext text/x-wiki t0rr4eyczfmfgsas3v6fvzbrfie8mci 0 460608 2455171 2454525 2026-09-07T12:53:47Z GGoncalves-WMF 42848 /* Client-facing */ Summarize what we heard in client interviews. 2455171 wikitext text/x-wiki 55f34qrai6i5sw6ozca40sdha3ubf99 498 460639 2455172 2026-09-07T12:54:36Z Stashbot 7414 wmbot~dcaro@tools-bastion-15: moved to push-to-deploy 2455172 wikitext text/x-wiki k6snt1fsqkxisahi5h3tesvj1exiiqw 3 460640 2455321 2026-09-07T15:32:08Z StrikerBot 8475 Welcome to Toolforge! 2455321 wikitext text/x-wiki airjabjxga494zagxda824norj4ariw 0 460641 2455359 2026-09-07T18:38:02Z Taavi-WMF 41365 init based on [[News/2025 Wikidata term store database split]] 2455359 wikitext text/x-wiki jreagfo1aha7pow8b0v0gqfg2yvw2pp 3 460642 2455403 2026-09-08T01:20:25Z StrikerBot 8475 Welcome to Toolforge! 2455403 wikitext text/x-wiki fzv2l0fur7n99qmgeafi4ho4cq7tl11