Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.14 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 0 4108 2445292 2445274 2026-08-10T15:03:16Z Superpes15 24252 /* {{Deployment_day|date=2026-08-10}} */ +Moving into the right section 2445292 wikitext text/x-wiki 68rfb7p31liadwxat4fhvx6bthdofh7 2445296 2445292 2026-08-10T21:20:19Z Superpes15 24252 +Moving 5 into the next session (no deployer available today) 2445296 wikitext text/x-wiki to6anm31q1c2px9f8250rxoow5eoc6z 0 7919 2445276 2445273 2026-08-10T13:05:24Z Stashbot 7414 wmde-fisch@deploy1003: Started scap sync-world: Backport for [[gerrit:1323939|Enable sub-references on more group2 wikis (batch3) (T432731)]] 2445276 wikitext text/x-wiki 6m5a9agvi2bwpmaw4ss1m7i9xd9snxp 2445277 2445276 2026-08-10T13:22:06Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch: Backport for [[gerrit:1323939|Enable sub-references on more group2 wikis (batch3) (T432731)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445277 wikitext text/x-wiki e4v7brsw4syo1cxbpbv634whpqj36f4 2445278 2445277 2026-08-10T13:25:38Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch: Continuing with deployment 2445278 wikitext text/x-wiki k4drvcemsv1n7bw8ly7uudfizd1fxgk 2445279 2445278 2026-08-10T13:38:45Z Stashbot 7414 wmde-fisch@deploy1003: Finished scap sync-world: Backport for [[gerrit:1323939|Enable sub-references on more group2 wikis (batch3) (T432731)]] (duration: 33m 21s) 2445279 wikitext text/x-wiki dbxc878ahzph9wjhwl2fvbbmcwocc77 2445280 2445279 2026-08-10T13:44:51Z Stashbot 7414 dani@deploy1003: helmfile [staging] START helmfile.d/services/miscweb: apply 2445280 wikitext text/x-wiki 8qw39i16tfli2exv7r5hq8d9o6wjv8l 2445281 2445280 2026-08-10T13:45:06Z Stashbot 7414 dani@deploy1003: helmfile [staging] DONE helmfile.d/services/miscweb: apply 2445281 wikitext text/x-wiki nyhiudlsoqfdoo0yj3o4q25joiees81 2445282 2445281 2026-08-10T13:45:08Z Stashbot 7414 dani@deploy1003: helmfile [eqiad] START helmfile.d/services/miscweb: apply 2445282 wikitext text/x-wiki ad4790cs57f053fnyn1foworibsdy5f 2445283 2445282 2026-08-10T13:45:25Z Stashbot 7414 dani@deploy1003: helmfile [eqiad] DONE helmfile.d/services/miscweb: apply 2445283 wikitext text/x-wiki tgmeckvc5mle33rpk6zrsyz78oy6fxk 2445284 2445283 2026-08-10T13:45:28Z Stashbot 7414 dani@deploy1003: helmfile [codfw] START helmfile.d/services/miscweb: apply 2445284 wikitext text/x-wiki 5c2ia4qm0qpb5328dvu7i290qn3airw 2445285 2445284 2026-08-10T13:45:48Z Stashbot 7414 dani@deploy1003: helmfile [codfw] DONE helmfile.d/services/miscweb: apply 2445285 wikitext text/x-wiki cy4882uj4p04a17zimk3v6jnk7ffejd 2445286 2445285 2026-08-10T13:57:00Z Stashbot 7414 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1323967|updateIsActiveFlagForMentees: Commit the final partial batch (T432959)]] 2445286 wikitext text/x-wiki eotrb8dcgbgqy6jk20mj0qd2xsdepf8 2445287 2445286 2026-08-10T14:07:33Z Stashbot 7414 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1323967|updateIsActiveFlagForMentees: Commit the final partial batch (T432959)]] (duration: 10m 33s) 2445287 wikitext text/x-wiki nga93n8z70mvpdep2aue7j23yu6vawf 2445288 2445287 2026-08-10T14:43:31Z Stashbot 7414 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1323973|mmv.bootstrap: Fix getUrlParam to account for TIFF lossy/lossless param (T434333)]] 2445288 wikitext text/x-wiki 8lturlml8ex8quthbowu0d39a3a2sck 2445289 2445288 2026-08-10T14:45:16Z Stashbot 7414 krinkle@deploy1003: krinkle: Backport for [[gerrit:1323973|mmv.bootstrap: Fix getUrlParam to account for TIFF lossy/lossless param (T434333)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445289 wikitext text/x-wiki av8d7tv0d3jqsvouwxs7qox69gs18fu 2445290 2445289 2026-08-10T14:50:49Z Stashbot 7414 krinkle@deploy1003: krinkle: Continuing with deployment 2445290 wikitext text/x-wiki 6dt83k370kbsz90tj5hdaun5g3t251i 2445291 2445290 2026-08-10T14:54:55Z Stashbot 7414 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1323973|mmv.bootstrap: Fix getUrlParam to account for TIFF lossy/lossless param (T434333)]] (duration: 11m 24s) 2445291 wikitext text/x-wiki h91dxbule19xmt9m6uq78stix4y9mxy 2445304 2445291 2026-08-11T02:00:45Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2445304 wikitext text/x-wiki 3xdkxcf595fowhyb722jo268wlwper1 2445305 2445304 2026-08-11T02:07:21Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 35s) 2445305 wikitext text/x-wiki 6oxzmkgym0bxe4e68tsrfsxaep1nuvn 2445306 2445305 2026-08-11T03:03:20Z Stashbot 7414 mwpresync@deploy1003: Started scap sync-world: testwikis to 1.47.0-wmf.15 refs T430834 2445306 wikitext text/x-wiki eyo9npxi99jdlbvlwzpgaww3ls4ri6i 2445310 2445306 2026-08-11T03:36:53Z Stashbot 7414 mwpresync@deploy1003: Finished scap sync-world: testwikis to 1.47.0-wmf.15 refs T430834 (duration: 33m 33s) 2445310 wikitext text/x-wiki oe5lk2pzz4y0g44tyg8ltg9qamle34j 2445311 2445310 2026-08-11T04:02:33Z Stashbot 7414 mwpresync@deploy1003: Pruned MediaWiki: 1.47.0-wmf.12 (duration: 02m 26s) 2445311 wikitext text/x-wiki 7y2fsbi4zoeorelydnpca2jelysptsx 2445312 2445311 2026-08-11T08:16:18Z Stashbot 7414 moritzm: imported jenkins 2.568.2 to thirdparty/jenkins for trixie-wikimedia 2445312 wikitext text/x-wiki ju8q9k1ralrqwpzyeimpnk47g1haiz3 2445313 2445312 2026-08-11T08:17:19Z Stashbot 7414 slyngshede@dns1004: START - running authdns-update 2445313 wikitext text/x-wiki 56tge715dphqm6mxtyrfwnhv0801l5w 2445314 2445313 2026-08-11T08:18:40Z Stashbot 7414 moritzm: installing openjdk-21 security updates 2445314 wikitext text/x-wiki 1c1lsx46ecvly07kqvjvd44gllrlmr7 2445315 2445314 2026-08-11T08:19:13Z Stashbot 7414 slyngshede@dns1004: END - running authdns-update 2445315 wikitext text/x-wiki et1hedridz9xlv2k379en9hdzjtln2e 2445316 2445315 2026-08-11T08:20:48Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1152: Switching over ms1 2445316 wikitext text/x-wiki 98qv4dsbsyag4pj89qwiy3jhuyr9bsd 2445317 2445316 2026-08-11T08:20:50Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.parsercache 2445317 wikitext text/x-wiki s38m3wi0vlgq3hft0dysdrj6r57xjdt 2445318 2445317 2026-08-11T08:20:51Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.parsercache (exit_code=99) 2445318 wikitext text/x-wiki gtv5gvoetk62r67s3p974t0d44s3pb0 2445319 2445318 2026-08-11T08:20:53Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1152: Switching over ms1 2445319 wikitext text/x-wiki hf0ozynpn82cfp3o9ja5m1pgmtzdyz3 2445320 2445319 2026-08-11T08:22:05Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool ms1 T434288', diff saved to https://phabricator.wikimedia.org/P95960 and previous config saved to /var/cache/conftool/dbconfig/20260811-082201-marostegui.json 2445320 wikitext text/x-wiki 1dxyviolfztkqj7ynv8p84fgbd69tvw 2445321 2445320 2026-08-11T08:22:37Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db2251.codfw.wmnet,db[1152,1267].eqiad.wmnet with reason: Switching over ms1 2445321 wikitext text/x-wiki 9cx9dvwub8damr2cznosz435u5y5qpy 2445323 2445321 2026-08-11T08:25:07Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db[2160,2232].codfw.wmnet,db[1164,1213,1217].eqiad.wmnet with reason: Primary switchover m1 T434043 2445323 wikitext text/x-wiki abwpqp08rtl96i9y3b6fn81nqhcbbk1 2445324 2445323 2026-08-11T08:30:00Z Stashbot 7414 marostegui: Failover m1 from db1213 to db1164 - T434043 2445324 wikitext text/x-wiki rgxt7pgp4lz5r6pdkhex1h0qbd32ne2 2445325 2445324 2026-08-11T08:40:58Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1267 to dbctl T434288', diff saved to https://phabricator.wikimedia.org/P95961 and previous config saved to /var/cache/conftool/dbconfig/20260811-084054-marostegui.json 2445325 wikitext text/x-wiki 5lhbzh53azk9dtrxeh17xx4dppsstpj 2445326 2445325 2026-08-11T08:48:08Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Repool ms1 T434288', diff saved to https://phabricator.wikimedia.org/P95962 and previous config saved to /var/cache/conftool/dbconfig/20260811-084804-marostegui.json 2445326 wikitext text/x-wiki qxq7j368ndy0f8yi47idf2p0sjzz0dn 2445327 2445326 2026-08-11T08:48:22Z Stashbot 7414 marostegui: Switchover ms1 master in eqiad T434288 2445327 wikitext text/x-wiki 1ncbldahmyfn0ugnf3vtvvyi131ox9c 2445328 2445327 2026-08-11T08:51:08Z Stashbot 7414 marostegui@dns1004: START - running authdns-update 2445328 wikitext text/x-wiki ev3x9ydrlqlfrjsmmy7y4s2ik77kd3v 2445329 2445328 2026-08-11T08:53:01Z Stashbot 7414 marostegui@dns1004: END - running authdns-update 2445329 wikitext text/x-wiki j451p86zn8qywum8waf080kvuhvqf2t 2445331 2445329 2026-08-11T09:05:04Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1213.eqiad.wmnet with reason: Reimage 2445331 wikitext text/x-wiki e8w4ai3239nrr3bbh784rqcba2ag6ij 2445332 2445331 2026-08-11T09:06:35Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.decommission 2445332 wikitext text/x-wiki kj2uqhaz48jqzhagmipoth4vilop6cj 2445333 2445332 2026-08-11T09:07:00Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1213.eqiad.wmnet with OS trixie 2445333 wikitext text/x-wiki jbhgb6ctanv9uynv9ksf7vehp2yyv05 2445334 2445333 2026-08-11T09:08:18Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.mysql.decommission (exit_code=99) 2445334 wikitext text/x-wiki 71tfnxajxbm10ds9387yth3h65eqje1 2445335 2445334 2026-08-11T09:08:38Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.decommission 2445335 wikitext text/x-wiki h81naz7w1qg4q3mom5kzoabqzi03iad 2445336 2445335 2026-08-11T09:08:42Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.depool depool db1177: Decommission 2445336 wikitext text/x-wiki dpidcxww9fmnyyl05a4dhouh5a9ib5j 2445337 2445336 2026-08-11T09:09:18Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1177: Decommission 2445337 wikitext text/x-wiki lhl7qxykied9bvnohr4qrrwo4mkcq2a 2445338 2445337 2026-08-11T09:22:50Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1213.eqiad.wmnet with reason: host reimage 2445338 wikitext text/x-wiki g23dsim2yc0s4fm7ciakfpwhl28phkz 2445339 2445338 2026-08-11T09:27:34Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1213.eqiad.wmnet with reason: host reimage 2445339 wikitext text/x-wiki cjfhaaz0eft4wowvh0igjkfz4ry8w4m 2445340 2445339 2026-08-11T09:28:24Z Stashbot 7414 moritzm: installing node-tar security updates 2445340 wikitext text/x-wiki 97io5a74x5pd9bpiy2vfwvrnsmkit2n 2445341 2445340 2026-08-11T09:35:42Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-coord1004.eqiad.wmnet with OS bookworm 2445341 wikitext text/x-wiki fp9azu5jkigxka0788xstqxqqpu1ezf 2445343 2445341 2026-08-11T09:40:48Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1006.eqiad.wmnet with OS bookworm 2445343 wikitext text/x-wiki i3e6v4n8ee1by618w4t2pax4hw05r2a 2445344 2445343 2026-08-11T09:41:19Z Stashbot 7414 moritzm: installing Linux 6.12.101 on Trixie hosts 2445344 wikitext text/x-wiki blso2jk1wjuyoiyefdyrxqc8q4be7ks 2445345 2445344 2026-08-11T09:49:44Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.decommission for hosts db1177.eqiad.wmnet 2445345 wikitext text/x-wiki 5w9hfcxo0vmxijqfzc72gd9kpdhksl1 2445346 2445345 2026-08-11T09:49:51Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1213.eqiad.wmnet with OS trixie 2445346 wikitext text/x-wiki l74tzl8b4afy5jfccfgk4uo6p7kuijq 2445347 2445346 2026-08-11T09:52:29Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-coord1004.eqiad.wmnet with reason: host reimage 2445347 wikitext text/x-wiki byfg3e9cva6fhrwcg6xtj5rrwtis9yq 2445348 2445347 2026-08-11T09:54:40Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db[2160,2232].codfw.wmnet,db[1164,1213,1217].eqiad.wmnet with reason: Primary switchover m1 T434493 2445348 wikitext text/x-wiki cfjsbcmn9kfhtgpc9s6kg97iirqoihf 2445349 2445348 2026-08-11T09:55:34Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2445349 wikitext text/x-wiki 209wef9iv0vv6f5crrb1wd35ffe13h0 2445350 2445349 2026-08-11T09:57:13Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-presto1006.eqiad.wmnet with reason: host reimage 2445350 wikitext text/x-wiki e2t9df7yvi93upl75yji1qszyqrbozr 2445351 2445350 2026-08-11T09:57:59Z Stashbot 7414 marostegui: Failover m1 from db1164 to db1213 - T434493 2445351 wikitext text/x-wiki g0g5hcp0h64jl4rudzs23f3cx2v3dl9 2445352 2445351 2026-08-11T09:58:07Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-coord1004.eqiad.wmnet with reason: host reimage 2445352 wikitext text/x-wiki a7ier7te51rptgrbzvsvbo7qyawitop 2445353 2445352 2026-08-11T10:00:50Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-presto1006.eqiad.wmnet with reason: host reimage 2445353 wikitext text/x-wiki ik2lmgn506xza50zdhhgxlra3go6yh3 2445354 2445353 2026-08-11T10:03:38Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db1177.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - fceratto@cumin1003" 2445354 wikitext text/x-wiki 96kil6yhe11cpt97vfzcvhj8c4xk3l8 2445355 2445354 2026-08-11T10:04:20Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db1177.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - fceratto@cumin1003" 2445355 wikitext text/x-wiki f9dhjs0oss89l4pt955io1h3c0ddzrm 2445356 2445355 2026-08-11T10:04:23Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2445356 wikitext text/x-wiki 5bil9c8p0uqaku6hl2r30f7phnnhhsn 2445357 2445356 2026-08-11T10:04:25Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db1177.eqiad.wmnet 2445357 wikitext text/x-wiki 79qdshpow6wcm6zv2b689rfoh7doew1 2445358 2445357 2026-08-11T10:04:30Z Stashbot 7414 fceratto@cumin1003: Removing db1177 from zarcillo T433474 2445358 wikitext text/x-wiki t76q8bwokivlo5tdaernj9s61yzuc89 2445359 2445358 2026-08-11T10:04:32Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.decommission (exit_code=0) 2445359 wikitext text/x-wiki 5kiewwdnnx0beiom5velqfuzi6225ss 2445360 2445359 2026-08-11T10:09:18Z Stashbot 7414 blake@deploy1003: Started scap sync-world: Non-deployment run to populate release values for T427668 2445360 wikitext text/x-wiki 0dg2o4ipg3bp3d3el6yq7upzur2g61n 2445361 2445360 2026-08-11T10:09:49Z Stashbot 7414 blake@deploy1003: Stopping before sync operations 2445361 wikitext text/x-wiki fj7cngxbt1svb382csk0067kw0rz79b 2445362 2445361 2026-08-11T10:14:32Z Stashbot 7414 blake@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-pretrain: apply 2445362 wikitext text/x-wiki kjsiaq8ky58owqakru5ifav9s1n462e 2445363 2445362 2026-08-11T10:17:36Z Stashbot 7414 blake@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-pretrain: apply 2445363 wikitext text/x-wiki skn9qv1qgekvab1w6cvp13iiw5gl85x 2445364 2445363 2026-08-11T10:17:51Z Stashbot 7414 blake@deploy1003: helmfile [codfw] START helmfile.d/services/mw-pretrain: apply 2445364 wikitext text/x-wiki evukuj0wg4vgo07id9mbdgb5fk96l0l 2445365 2445364 2026-08-11T10:18:07Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.clone of db1179.eqiad.wmnet onto db1278.eqiad.wmnet 2445365 wikitext text/x-wiki d7d89t10jirzpix6ga0diw1ixocfaow 2445366 2445365 2026-08-11T10:18:11Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1179: Depool db1179.eqiad.wmnet to then clone it to db1278.eqiad.wmnet - marostegui@cumin1003 2445366 wikitext text/x-wiki dev9wi8mdjswnt6ivc65jqmjhzpvnc6 2445367 2445366 2026-08-11T10:20:11Z Stashbot 7414 blake@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-pretrain: apply 2445367 wikitext text/x-wiki 1bonrn5at4qoxicxcf1x0h2xltyzkly 2445368 2445367 2026-08-11T10:21:11Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-presto1006.eqiad.wmnet with OS bookworm 2445368 wikitext text/x-wiki jufugg0devgu49evwwgxg7lef52dfis 2445369 2445368 2026-08-11T10:22:01Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1007.eqiad.wmnet with OS bookworm 2445369 wikitext text/x-wiki eb7ycdiluaesd6fr94kzsxxdiv2pbh7 2445370 2445369 2026-08-11T10:22:08Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1008.eqiad.wmnet with OS bookworm 2445370 wikitext text/x-wiki qjjrj3dyudwqy2jgvxk1l0m0cnhhs29 2445371 2445370 2026-08-11T10:25:31Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Remove db1177 T433474', diff saved to https://phabricator.wikimedia.org/P95964 and previous config saved to /var/cache/conftool/dbconfig/20260811-102527-fceratto.json 2445371 wikitext text/x-wiki pgetqff56el78z04amgaky3jwalz89h 2445372 2445371 2026-08-11T10:34:01Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-presto1008.eqiad.wmnet with OS bookworm 2445372 wikitext text/x-wiki oahq1qai9wn7x9bljft7mux4e5bmsko 2445373 2445372 2026-08-11T10:34:15Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1179: Depool db1179.eqiad.wmnet to then clone it to db1278.eqiad.wmnet - marostegui@cumin1003 2445373 wikitext text/x-wiki 2yjmx3ywta608geji5ktzpj05rr1rks 2445374 2445373 2026-08-11T10:35:52Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1008.eqiad.wmnet with OS bookworm 2445374 wikitext text/x-wiki b28m5ah95yt4zm8xevpkkak1hmxqdj6 2445375 2445374 2026-08-11T10:37:35Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-coord1004.eqiad.wmnet with OS bookworm 2445375 wikitext text/x-wiki 044ih5eiz015yau4s15659u2fknge5f 2445376 2445375 2026-08-11T10:38:15Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-presto1007.eqiad.wmnet with reason: host reimage 2445376 wikitext text/x-wiki 6qss623b3ta0o3p54nok274pvwdayv9 2445377 2445376 2026-08-11T10:43:50Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-presto1007.eqiad.wmnet with reason: host reimage 2445377 wikitext text/x-wiki h6brzubqq52lwl6lvw09v9qjh9brxvs 2445378 2445377 2026-08-11T10:45:12Z Stashbot 7414 tappof: bump space for prometheus k8s-dse in codfw 2445378 wikitext text/x-wiki sp9iehj0xpmpqpsowdtsihwh1jg342r 2445379 2445378 2026-08-11T11:11:48Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-presto1007.eqiad.wmnet with OS bookworm 2445379 wikitext text/x-wiki 1x6p89wtu6ogy4xdx31gx0mfngwlt1j 2445380 2445379 2026-08-11T11:20:13Z Stashbot 7414 moritzm: installing curl security updates 2445380 wikitext text/x-wiki tk39426myjj0hp7o4d967qf98weduyu 2445381 2445380 2026-08-11T11:20:26Z Stashbot 7414 btullis@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host an-presto1008.eqiad.wmnet with OS bookworm 2445381 wikitext text/x-wiki cs8nae1xvqgspaxrscjy8a3wiipy5h7 2445382 2445381 2026-08-11T11:21:24Z Stashbot 7414 btullis@dns1004: START - running authdns-update 2445382 wikitext text/x-wiki cxu5svq2kj6zpt3h2z1q3a2qcf9ax1n 2445383 2445382 2026-08-11T11:23:20Z Stashbot 7414 btullis@dns1004: END - running authdns-update 2445383 wikitext text/x-wiki 15l7inrro3uy0bvmrdpbor0ij4eg3j4 2445384 2445383 2026-08-11T11:26:48Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.provision for host an-presto1008.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2445384 wikitext text/x-wiki 9j7d2gyaqu7r5y6867womxtkay3j41f 2445385 2445384 2026-08-11T11:28:56Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-presto1008.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2445385 wikitext text/x-wiki 9h569aimrfz1veqfh9yxec3urolbl09 2445386 2445385 2026-08-11T11:29:23Z Stashbot 7414 moritzm: installing Python 3.11 security updates 2445386 wikitext text/x-wiki 4vzm7dyw45v9ahsnxsd2zed2y8v4hse 2445387 2445386 2026-08-11T11:34:33Z Stashbot 7414 cwilliams@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 25 hosts with reason: Primary switchover s8 T434514 2445387 wikitext text/x-wiki kpi9jkx5zscd8druu40j6aq86hv8s6i 2445388 2445387 2026-08-11T11:34:53Z Stashbot 7414 cwilliams@cumin1003: dbctl commit (dc=all): 'Set db2161 with weight 0 T434514', diff saved to https://phabricator.wikimedia.org/P95966 and previous config saved to /var/cache/conftool/dbconfig/20260811-113449-cwilliams.json 2445388 wikitext text/x-wiki qz39viq5g5h5nwp71mpgc89zckoq7i3 2445390 2445388 2026-08-11T11:36:42Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti-test2003.codfw.wmnet 2445390 wikitext text/x-wiki roise0mq475e2i4o2b6zhhx50e8212r 2445391 2445390 2026-08-11T11:36:53Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti-test2003.codfw.wmnet 2445391 wikitext text/x-wiki 0q73h3dasjdg07retc7bwxhpyrb0e27 2445392 2445391 2026-08-11T11:40:23Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1179: Pool db1179.eqiad.wmnet in after cloning 2445392 wikitext text/x-wiki hzzgg8pq3dn3iwritbau1f2fhd5ce3s 2445393 2445392 2026-08-11T11:40:48Z Stashbot 7414 cezmunsta: Starting s8 codfw failover from db2165 to db2161 - T434514 2445393 wikitext text/x-wiki iuyk0ycybo2kuhqfgj58ce6pops1680 2445394 2445393 2026-08-11T11:41:41Z Stashbot 7414 cwilliams@cumin1003: dbctl commit (dc=all): 'Promote db2161 to s8 primary T434514', diff saved to https://phabricator.wikimedia.org/P95968 and previous config saved to /var/cache/conftool/dbconfig/20260811-114136-cwilliams.json 2445394 wikitext text/x-wiki fv27yev48z3fe6taq3a1vamyca2m8fo 2445395 2445394 2026-08-11T11:42:54Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2003.codfw.wmnet 2445395 wikitext text/x-wiki 6euh9l9r7vd9tn3balucgfu0w1jedfp 2445396 2445395 2026-08-11T11:42:56Z Stashbot 7414 jmm@cumin2003: END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti-test2003.codfw.wmnet 2445396 wikitext text/x-wiki 43mwj2s4gfqq43ftwoc6avth0qhp9tn 2445397 2445396 2026-08-11T11:43:18Z Stashbot 7414 jnuche@deploy1003: Started deploy [releng/jenkins-deploy@050d19e] (releasing): test jenkins deploy for T434186 2445397 wikitext text/x-wiki jw8294i80ou3r7mi8bmradmh6tg9kgp 2445398 2445397 2026-08-11T11:43:55Z Stashbot 7414 cwilliams@cumin1003: dbctl commit (dc=all): 'Depool db2165 T434514', diff saved to https://phabricator.wikimedia.org/P95969 and previous config saved to /var/cache/conftool/dbconfig/20260811-114352-cwilliams.json 2445398 wikitext text/x-wiki dlizyfjymlynajzb0swike2nl3x8tiw 2445399 2445398 2026-08-11T11:44:15Z Stashbot 7414 jnuche@deploy1003: Finished deploy [releng/jenkins-deploy@050d19e] (releasing): test jenkins deploy for T434186 (duration: 01m 08s) 2445399 wikitext text/x-wiki m46rz8fnuwv02sunmpoq1973yk7hjwn 2445400 2445399 2026-08-11T11:47:37Z Stashbot 7414 jnuche@deploy1003: Started deploy [releng/jenkins-deploy@050d19e] (releasing): T434186 2445400 wikitext text/x-wiki 3jticmncmrsrdhtddybystl7um53amm 2445401 2445400 2026-08-11T11:48:44Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1324283|Use maximum compression level in SqlBlobStore and SqlBagOStuff (T428377)]] 2445401 wikitext text/x-wiki h45res13ior2jymxcu0384dauy9n8sa 2445402 2445401 2026-08-11T11:48:46Z Stashbot 7414 jnuche@deploy1003: Finished deploy [releng/jenkins-deploy@050d19e] (releasing): T434186 (duration: 01m 14s) 2445402 wikitext text/x-wiki 3y63hrd5ut5ct7o4tblpcr0v9a8nizq 2445403 2445402 2026-08-11T11:49:52Z Stashbot 7414 cwilliams@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2165.codfw.wmnet with reason: Maintenance 2445403 wikitext text/x-wiki 2x7k7k6tn56vsa04ns9z2nvr29t4b5x 2445404 2445403 2026-08-11T11:50:56Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti-test2002.codfw.wmnet 2445404 wikitext text/x-wiki 5xh8y3wo3s6w5yop7zp2px616wp3dao 2445405 2445404 2026-08-11T11:51:04Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet 2445405 wikitext text/x-wiki 8ciw96smz0aocqv8gda8mtq6wpcu6c9 2445406 2445405 2026-08-11T11:52:31Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1324283|Use maximum compression level in SqlBlobStore and SqlBagOStuff (T428377)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445406 wikitext text/x-wiki kd0oxvifdk5f7f5e8aqiijrvbmd9jmi 2445407 2445406 2026-08-11T11:54:20Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2445407 wikitext text/x-wiki fwsn4hs31e2qq4lgmkxshylmp4gz9yj 2445408 2445407 2026-08-11T11:56:45Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db2165: Security update 2445408 wikitext text/x-wiki q2wfh3634d98dxeqpoifu55fr5ezdd3 2445409 2445408 2026-08-11T11:57:09Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet 2445409 wikitext text/x-wiki gscwsr1i1u7hx2d7fm5jyeuphuwt386 2445410 2445409 2026-08-11T11:57:12Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti-test2002.codfw.wmnet 2445410 wikitext text/x-wiki me4ojoxy5nv5rps70ind18gacf12d65 0 449160 2445298 2445264 2026-08-11T00:00:28Z Dexbot 30554 Bot: Updating the report 2445298 wikitext text/x-wiki 9k1kmysx0fjiw5ma17fcvpdjgqo05uo 2445322 2445298 2026-08-11T08:22:40Z Dexbot 30554 Bot: Updating the report 2445322 wikitext text/x-wiki qqslx1ppcapzxeadsbjvghq8i65yzjh 2445389 2445322 2026-08-11T11:35:10Z Dexbot 30554 Bot: Updating the report 2445389 wikitext text/x-wiki 5p2sqztrssr8v8jtmtd454n46epc14o 116 453906 2445293 2445275 2026-08-10T17:18:35Z Gitlabaccountapprovalbot 37332 @mustafe was approved. 2445293 wikitext text/x-wiki 9o7c5s2rsj61ayeo2xf3y1xsgx6j6as 0 459697 2445330 2399100 2026-08-11T08:56:46Z Lucas Werkmeister (WMDE) 10037 /* wdqs-proxy */ link the actual repo 2445330 wikitext text/x-wiki lbs6xps5o7wbtec0jdvg67jfrl861xe 0 459699 2445297 2445269 2026-08-10T21:54:29Z Stashbot 7414 eileen: config revision changed from d4bac642 to bc943eff 2445297 wikitext text/x-wiki fgoo2xzg5rjv76osezab36wvzllkcsi 2 460540 2445294 2026-08-10T20:54:45Z Ottomata 88 Working on it! 2445294 wikitext text/x-wiki 3v3n5mes81u5ltq28h1ae1o4gq06tyh 2445295 2445294 2026-08-10T21:06:34Z Ottomata 88 2445295 wikitext text/x-wiki bvbzmttukjcxgaaje5xkc143mgjuvxh 2445299 2445295 2026-08-11T00:45:24Z Ottomata 88 link to https://phabricator.wikimedia.org/T351225#11682262 2445299 wikitext text/x-wiki 6wqzbmp4rzd880nc5c6ermvzijvn8ou 2445300 2445299 2026-08-11T01:05:19Z Ottomata 88 2445300 wikitext text/x-wiki nxhq047acj7vrb6cihnhn3wvdj8seov 2445301 2445300 2026-08-11T01:09:48Z Ottomata 88 2445301 wikitext text/x-wiki p5ztbiviqhiw5tdtuj4es2bymjbmt2q 2445302 2445301 2026-08-11T01:11:49Z Ottomata 88 2445302 wikitext text/x-wiki 7nnf6luhy51soshflxybrov2379qwcw 2445303 2445302 2026-08-11T01:16:14Z Ottomata 88 2445303 wikitext text/x-wiki a0sd8889nd6gl740tg0x4mahqu5s4tm 3 460541 2445307 2026-08-11T03:05:44Z StrikerBot 8475 Welcome to Toolforge! 2445307 wikitext text/x-wiki 5lwgxage0ptajo0jy0h2jmke5hs2izr 3 460542 2445308 2026-08-11T03:06:08Z StrikerBot 8475 Welcome to Toolforge! 2445308 wikitext text/x-wiki s968e02yiebq6k0nol5wrndghmogtfz 3 460543 2445309 2026-08-11T03:07:10Z StrikerBot 8475 Welcome to Toolforge! 2445309 wikitext text/x-wiki kmjwnznmia6zc5fq8fpswe124y3vyx7 0 460544 2445342 2026-08-11T09:38:45Z Lucas Werkmeister (WMDE) 10037 soft redirect to the documentation on mw.o for now (I expect at some point there should be on-wikitech documentation about this) 2445342 wikitext text/x-wiki itj6palu24ih5bv6f26e6xe1tl772rc