Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.12 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 0 4108 2440663 2440528 2026-07-24T12:43:17Z ScheduleDeploymentBot 37566 Add [[gerrit:1309727]] to Tuesday, July 28 UTC afternoon backport window 2440663 wikitext text/x-wiki t11ls9u4r3hfz6ag6sd7fh50w29ok58 2440670 2440663 2026-07-24T13:33:29Z ScheduleDeploymentBot 37566 Add [[gerrit:1315843]] to Monday, July 27 UTC afternoon backport window 2440670 wikitext text/x-wiki t6o9bsa3rybb6l572mb1ahfca4d5s7l 2440671 2440670 2026-07-24T13:34:51Z ScheduleDeploymentBot 37566 Add [[gerrit:1315842]] to Monday, July 27 UTC afternoon backport window 2440671 wikitext text/x-wiki aqr6fs5tvtmepu1yi01w9p0jt665bmp 2440778 2440671 2026-07-25T02:00:20Z DeploymentCalendarTool 20896 Remove Week of July 20 2440778 wikitext text/x-wiki tann3f34dv1mnnuy96rphd9hpwp95rz 0 7919 2440665 2440662 2026-07-24T13:10:54Z Stashbot 7414 fnegri@cumin1003: START - Cookbook sre.mysql.multiinstance_reboot for clouddb[1029-1031].eqiad.wmnet 2440665 wikitext text/x-wiki ompzarz2ha2q54s7rrilkurrr3hx3iy 2440666 2440665 2026-07-24T13:24:27Z Stashbot 7414 fnegri@cumin1003: END (PASS) - Cookbook sre.mysql.multiinstance_reboot (exit_code=0) for clouddb[1029-1031].eqiad.wmnet 2440666 wikitext text/x-wiki r8xclp26fy9eo1iezlu4g2pl33ryv24 2440669 2440666 2026-07-24T13:30:38Z Stashbot 7414 papaul: reboot mr1-eqsin for maintenance 2440669 wikitext text/x-wiki qj5e7bneaoqbdysbbk99svys3qkyp2q 2440672 2440669 2026-07-24T13:41:14Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.netbox 2440672 wikitext text/x-wiki ti7dhvkrh12dxey9mqlcnoqadzoxqo4 2440673 2440672 2026-07-24T13:42:26Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc-gp2006.codfw.wmnet with OS bookworm 2440673 wikitext text/x-wiki 0stlj20tjo983pe80cxrk0ku09nakdh 2440674 2440673 2026-07-24T13:44:30Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.wipe-cache mc-gp2006.codfw.wmnet on all recursors 2440674 wikitext text/x-wiki mnhm7dgkz9t2ynj8uno3nxrl1iaof0n 2440675 2440674 2026-07-24T13:44:34Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) mc-gp2006.codfw.wmnet on all recursors 2440675 wikitext text/x-wiki kwreea5w7d891h2jku8n496j7f3f171 2440676 2440675 2026-07-24T13:45:35Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: push new IPs for mc-gp2006 - cmooney@cumin1003" 2440676 wikitext text/x-wiki ot2ppugtl2bxm6jqcg4uewqdbx8pffg 2440677 2440676 2026-07-24T13:45:39Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: push new IPs for mc-gp2006 - cmooney@cumin1003" 2440677 wikitext text/x-wiki mt84l8hpsnzztwq8ch3rbey3fgsn7nq 2440678 2440677 2026-07-24T13:45:41Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2440678 wikitext text/x-wiki li8dklinhokgspbkgt2ntx7zjvofetq 2440679 2440678 2026-07-24T13:53:14Z Stashbot 7414 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1315130|Add missing normaliseParams() call to ThreeDHandler::doTransform (T432911)]] 2440679 wikitext text/x-wiki 4197ye9lqnx5jlbot26ryggmm3qyvgt 2440680 2440679 2026-07-24T13:55:21Z Stashbot 7414 krinkle@deploy1003: krinkle: Backport for [[gerrit:1315130|Add missing normaliseParams() call to ThreeDHandler::doTransform (T432911)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2440680 wikitext text/x-wiki 12esybv4o7ilw0li3qwnneo4ruiqddh 2440681 2440680 2026-07-24T13:57:36Z Stashbot 7414 ssastry@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2440681 wikitext text/x-wiki 9e94ml0s8buglrnjwav6dj78deb2v3s 2440682 2440681 2026-07-24T13:58:01Z Stashbot 7414 ssastry@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2440682 wikitext text/x-wiki qezx2chdywovg341th32p4mvpymgw4r 2440683 2440682 2026-07-24T13:58:03Z Stashbot 7414 ssastry@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2440683 wikitext text/x-wiki 5tmenpmnevusf7dck6iduto8nxund2i 2440684 2440683 2026-07-24T13:58:31Z Stashbot 7414 ssastry@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2440684 wikitext text/x-wiki audz9gg704mmvqo8y39zfkabz7k8em1 2440685 2440684 2026-07-24T14:09:19Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1071.eqiad.wmnet 2440685 wikitext text/x-wiki bce91ihp67htw42859y7ol0vbalic2j 2440686 2440685 2026-07-24T14:09:20Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1071.eqiad.wmnet 2440686 wikitext text/x-wiki bv2c4hc0sjla2gslh4jly7dqnteumbx 2440687 2440686 2026-07-24T14:09:31Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1071.eqiad.wmnet 2440687 wikitext text/x-wiki hlotfcmsgpx0tbk2hducqe6qdeqwz69 2440688 2440687 2026-07-24T14:09:32Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1071.eqiad.wmnet 2440688 wikitext text/x-wiki qmym3pwmqvhyxtvf3u3mhqrukndpw8l 2440689 2440688 2026-07-24T14:10:08Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1072.eqiad.wmnet 2440689 wikitext text/x-wiki c7icwcdlzwr3y0xwzxrlyzhanft766o 2440690 2440689 2026-07-24T14:10:10Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1072.eqiad.wmnet 2440690 wikitext text/x-wiki jaucxuhffb5aij0fwyq6ref44xyy42g 2440691 2440690 2026-07-24T14:10:19Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1072.eqiad.wmnet 2440691 wikitext text/x-wiki 0x0fxr9klbz1fthyv8axdafzgsgekj2 2440692 2440691 2026-07-24T14:10:21Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1072.eqiad.wmnet 2440692 wikitext text/x-wiki culyspwko10noua9rlou4j2qwerylus 2440693 2440692 2026-07-24T14:13:33Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.renumber-node Renumbering for host wikikube-worker1135.eqiad.wmnet 2440693 wikitext text/x-wiki tpr92gazxr207b0lyn5cglk4nsyxdgy 2440694 2440693 2026-07-24T14:13:36Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1135.eqiad.wmnet 2440694 wikitext text/x-wiki ptw0jv41rgoi3r15shdmsym2cn4mjhd 2440695 2440694 2026-07-24T14:14:10Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1135.eqiad.wmnet 2440695 wikitext text/x-wiki pv9artuelru9y10psxrncrfp0lsdtha 2440696 2440695 2026-07-24T14:14:47Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-worker1135.eqiad.wmnet with OS trixie 2440696 wikitext text/x-wiki 5dbfqa0x081n52p225pw5wh04uhp8ni 2440697 2440696 2026-07-24T14:15:14Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.move-vlan for host wikikube-worker1135 2440697 wikitext text/x-wiki qs9k1jamw0xagcn8wgobfmwug44j6d9 2440698 2440697 2026-07-24T14:26:37Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.dns.netbox 2440698 wikitext text/x-wiki ne2cjbe3gawbs5639eob13ff9ea1hye 2440699 2440698 2026-07-24T14:27:25Z Stashbot 7414 jiji@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host mc-gp2006.codfw.wmnet with OS bookworm 2440699 wikitext text/x-wiki 6rj6uq3isgnf0mby2ulj66hflqm79ec 2440700 2440699 2026-07-24T14:29:23Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc-gp2006.codfw.wmnet with OS bookworm 2440700 wikitext text/x-wiki 8a6qdki68h4p94bj8o9xussyqsntrl9 2440701 2440700 2026-07-24T14:29:52Z Stashbot 7414 krinkle@deploy1003: krinkle: Continuing with deployment 2440701 wikitext text/x-wiki ttqsp0hj7wcx9g7bpdgge0zvgmpdupx 2440702 2440701 2026-07-24T14:32:26Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1135 - jiji@cumin1003" 2440702 wikitext text/x-wiki a3nphncp7gh09401bhxysh1pfmxput3 2440703 2440702 2026-07-24T14:32:30Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1135 - jiji@cumin1003" 2440703 wikitext text/x-wiki auf1kpzxz4f6cc2h7rju6kkmosatzhq 2440704 2440703 2026-07-24T14:32:32Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2440704 wikitext text/x-wiki aby6zk83y47mhb0l52v2nqh6y2pul69 2440705 2440704 2026-07-24T14:32:33Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.dns.wipe-cache wikikube-worker1135.eqiad.wmnet 177.32.64.10.in-addr.arpa 7.7.1.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2440705 wikitext text/x-wiki lkasgx1j15vtsj98u8n78wjey6itn99 2440706 2440705 2026-07-24T14:32:35Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker1135.eqiad.wmnet 177.32.64.10.in-addr.arpa 7.7.1.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2440706 wikitext text/x-wiki c7a7lkooi8my4rydtpxjrkmofu6z5pn 2440707 2440706 2026-07-24T14:32:38Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker1135 2440707 wikitext text/x-wiki 6kojqvol8pgpfbcv2yxirfwp2ezza2i 2440708 2440707 2026-07-24T14:34:26Z Stashbot 7414 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1315130|Add missing normaliseParams() call to ThreeDHandler::doTransform (T432911)]] (duration: 41m 12s) 2440708 wikitext text/x-wiki pxx0mrir0qjv01w9f11u6457oxvjz81 2440710 2440708 2026-07-24T14:48:25Z Stashbot 7414 jiji@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host mc-gp2006.codfw.wmnet with OS bookworm 2440710 wikitext text/x-wiki qoshz9l1un9c7uob0nytkogzkf9rejy 2440712 2440710 2026-07-24T14:49:25Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.reimage for host mc-gp2006.codfw.wmnet with OS bookworm 2440712 wikitext text/x-wiki 7fmiipbcaghxvqrndia14n31wfc8m5v 2440713 2440712 2026-07-24T15:08:23Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on mc-gp2006.codfw.wmnet with reason: host reimage 2440713 wikitext text/x-wiki se8xu398gdcg3dwsb45npox7hytys2i 2440715 2440713 2026-07-24T15:13:16Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc-gp2006.codfw.wmnet with reason: host reimage 2440715 wikitext text/x-wiki ifq0c3a7po9pwmq8xco68xbeyrlcmgo 2440716 2440715 2026-07-24T15:15:52Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker1135 2440716 wikitext text/x-wiki rz52jw2wx2rm4wsxo5yynf35ozuzawi 2440717 2440716 2026-07-24T15:15:53Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker1135 2440717 wikitext text/x-wiki pi6t5uhs3erf8zz3xcnh7meqkanxywt 2440718 2440717 2026-07-24T15:30:30Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc-gp2006.codfw.wmnet with OS bookworm 2440718 wikitext text/x-wiki norjn8s4zhmds5gdwyf73dnt5hutcas 2440719 2440718 2026-07-24T15:32:23Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1135.eqiad.wmnet with reason: host reimage 2440719 wikitext text/x-wiki q8awfzezlf8qenk89nbxwjyro0kqe67 2440720 2440719 2026-07-24T15:33:08Z Stashbot 7414 cmooney@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 6 hosts with reason: upgrade lswtest-d8-eqiad 2440720 wikitext text/x-wiki bd47lbo95p138jqlbo5o98kxq36d2dq 2440721 2440720 2026-07-24T15:36:46Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1135.eqiad.wmnet with reason: host reimage 2440721 wikitext text/x-wiki 2te484bb8y7eeqzyy8aea7ehjcszbg3 2440722 2440721 2026-07-24T15:37:16Z Stashbot 7414 topranks: upgrade SR-Linux OS on lswtest-d8-eqiad 2440722 wikitext text/x-wiki 2c7nd8gzq4fd5n9170b0ssup8i9gp6t 2440723 2440722 2026-07-24T15:52:13Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.bmc-user-mgmt for 40 hosts 2440723 wikitext text/x-wiki 2n25qkg0ut8bxg7x1i3p6l5yaum6zk1 2440724 2440723 2026-07-24T15:54:14Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.bmc-user-mgmt (exit_code=0) for 40 hosts 2440724 wikitext text/x-wiki k6d378ianbdepfakirkdlg82djogs6h 2440725 2440724 2026-07-24T15:56:38Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1135.eqiad.wmnet with OS trixie 2440725 wikitext text/x-wiki pm87yznbs7n2refw19nby0sgwku4s6u 2440726 2440725 2026-07-24T16:03:37Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T430880, restore data on newly-reimaged host) xfer wdqs-all from wdqs1011.eqiad.wmnet -> wdqs1014.eqiad.wmnet, repooling source-only afterwards 2440726 wikitext text/x-wiki 6dvxffw3odi9m6rza9ua8ovrin0crdg 2440727 2440726 2026-07-24T16:03:54Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T430880, restore data on newly-reimaged host) xfer wdqs-all from wdqs2007.codfw.wmnet -> wdqs2008.codfw.wmnet, repooling source-only afterwards 2440727 wikitext text/x-wiki big3qnnhddmkknmdcsbyfx3m4zcdmce 2440728 2440727 2026-07-24T16:11:24Z Stashbot 7414 ssastry@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2440728 wikitext text/x-wiki p1unty7qfmqcenoqjkwxkeczuy4kjhc 2440729 2440728 2026-07-24T16:11:26Z Stashbot 7414 ssastry@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2440729 wikitext text/x-wiki efd0k3fres5ytkr20fs8evcr150iqgn 2440730 2440729 2026-07-24T16:11:28Z Stashbot 7414 ssastry@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2440730 wikitext text/x-wiki 8ypuk1m4mcf831gf7kmfhae1wlcwrgb 2440731 2440730 2026-07-24T16:11:31Z Stashbot 7414 ssastry@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2440731 wikitext text/x-wiki kadristm8mrpvvl7si5ye5vzws2yolv 2440733 2440731 2026-07-24T16:28:33Z Stashbot 7414 ssastry@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2440733 wikitext text/x-wiki d7sxfsffnz67yku5ueui1r34hlq1zfn 2440734 2440733 2026-07-24T16:28:36Z Stashbot 7414 ssastry@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2440734 wikitext text/x-wiki 1gpfv3ogrpyvgwv7nnkszwau91sau69 2440735 2440734 2026-07-24T16:28:37Z Stashbot 7414 ssastry@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2440735 wikitext text/x-wiki et4m35g4v1mcsukntw0f80w88z10sjt 2440736 2440735 2026-07-24T16:28:40Z Stashbot 7414 ssastry@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2440736 wikitext text/x-wiki ek5hy781h3ylb1k0ov03o0lx57x8v72 2440737 2440736 2026-07-24T16:33:16Z Stashbot 7414 bd808@deploy1003: helmfile [staging] START helmfile.d/services/developer-portal: apply 2440737 wikitext text/x-wiki a05t0ryfwxkp2fn0rwrprz1ndkbzubn 2440738 2440737 2026-07-24T16:33:44Z Stashbot 7414 bd808@deploy1003: helmfile [staging] DONE helmfile.d/services/developer-portal: apply 2440738 wikitext text/x-wiki hoft3o9y42vmfk08ns1l5e2xf1h6c3n 2440739 2440738 2026-07-24T16:34:04Z Stashbot 7414 bd808@deploy1003: helmfile [codfw] START helmfile.d/services/developer-portal: apply 2440739 wikitext text/x-wiki r8ww98cg5wzvj56gnj6jf6pxq5ezjze 2440740 2440739 2026-07-24T16:34:22Z Stashbot 7414 bd808@deploy1003: helmfile [codfw] DONE helmfile.d/services/developer-portal: apply 2440740 wikitext text/x-wiki 34g4ktfkvkry35v3t8xngnofg5w6yx3 2440741 2440740 2026-07-24T16:34:37Z Stashbot 7414 bd808@deploy1003: helmfile [eqiad] START helmfile.d/services/developer-portal: apply 2440741 wikitext text/x-wiki 7f4o9v71aptqlia0r81vs5j1dkkr00m 2440742 2440741 2026-07-24T16:34:53Z Stashbot 7414 bd808@deploy1003: helmfile [eqiad] DONE helmfile.d/services/developer-portal: apply 2440742 wikitext text/x-wiki omc3c2xulft475lj30mhd3a7l6ouk0i 2440743 2440742 2026-07-24T17:21:18Z Stashbot 7414 jiji@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1135.eqiad.wmnet 2440743 wikitext text/x-wiki kqb2w6yzanlhmwpe2tedgi0x8rtqych 2440744 2440743 2026-07-24T17:21:19Z Stashbot 7414 jiji@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1135.eqiad.wmnet 2440744 wikitext text/x-wiki 3w7exc6qif36yhsrjp9o82wkp4hwxls 2440745 2440744 2026-07-24T17:21:21Z Stashbot 7414 jiji@cumin1003: END (FAIL) - Cookbook sre.k8s.renumber-node (exit_code=1) Renumbering for host wikikube-worker1135.eqiad.wmnet 2440745 wikitext text/x-wiki twdcxcosra0z2d8cbnry5qef2zduq59 2440748 2440745 2026-07-24T21:38:22Z Stashbot 7414 jhathaway@cumin1003: START - Cookbook sre.hosts.reimage for host sretest2010.codfw.wmnet with OS trixie 2440748 wikitext text/x-wiki fn3l2g54pkk8pcslt4wu7v9abptjn0s 2440749 2440748 2026-07-24T21:39:24Z Stashbot 7414 jhathaway@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host sretest2010.codfw.wmnet with OS trixie 2440749 wikitext text/x-wiki nunrnisnl32wo7dol6k65ecuv379aku 2440750 2440749 2026-07-24T21:43:23Z Stashbot 7414 jhathaway@cumin1003: START - Cookbook sre.hosts.reimage for host sretest2010.codfw.wmnet with OS trixie 2440750 wikitext text/x-wiki ntojvg185aevpu7a736aff95krjl1dk 2440751 2440750 2026-07-24T21:47:39Z Stashbot 7414 jhathaway@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2010.codfw.wmnet with OS trixie 2440751 wikitext text/x-wiki d15coyqb8lphvbrjhfme03s5n467uc9 2440752 2440751 2026-07-24T21:51:07Z Stashbot 7414 jhathaway@cumin1003: START - Cookbook sre.hosts.reimage for host sretest2010.codfw.wmnet with OS trixie 2440752 wikitext text/x-wiki c5c726m06ojcjuj6avngp1c3n4r3e9r 2440753 2440752 2026-07-24T21:53:14Z Stashbot 7414 jhathaway@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2010.codfw.wmnet with OS trixie 2440753 wikitext text/x-wiki cgd4i4k3lt07cnr25vk2mex3bu7femp 2440754 2440753 2026-07-24T21:53:28Z Stashbot 7414 jhathaway@cumin1003: START - Cookbook sre.hosts.reimage for host sretest2010.codfw.wmnet with OS trixie 2440754 wikitext text/x-wiki hnci5w6lllmu0bt1skepfrv4wo7niv7 2440756 2440754 2026-07-24T22:00:33Z Stashbot 7414 jhathaway@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host sretest2010.codfw.wmnet with OS trixie 2440756 wikitext text/x-wiki 7um6mplqa5grocialkkveao6dvyi2q2 2440757 2440756 2026-07-24T22:13:29Z Stashbot 7414 ssastry@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2440757 wikitext text/x-wiki ksxgtp1ou29pjn0gierx2amido2pphf 2440758 2440757 2026-07-24T22:13:31Z Stashbot 7414 ssastry@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2440758 wikitext text/x-wiki 734pnfp4y0k8gbrco65j1g032z16663 2440759 2440758 2026-07-24T22:13:33Z Stashbot 7414 ssastry@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2440759 wikitext text/x-wiki 3wrjgeq6pyscoodx4fr4vu4ypgipihm 2440760 2440759 2026-07-24T22:13:36Z Stashbot 7414 ssastry@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2440760 wikitext text/x-wiki 2b3pt41aotufvnrhkd9smumu0dp7exy 2440761 2440760 2026-07-24T22:33:57Z Stashbot 7414 jhathaway@cumin1003: START - Cookbook sre.hosts.reimage for host sretest2010.codfw.wmnet with OS trixie 2440761 wikitext text/x-wiki gf3j140jqmi0sefupyqhow6s16ls3us 2440762 2440761 2026-07-24T23:03:36Z Stashbot 7414 jhathaway@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on sretest2010.codfw.wmnet with reason: host reimage 2440762 wikitext text/x-wiki 6xnf8ir9jx9lob5m1zxrav84w25zj8w 2440763 2440762 2026-07-24T23:08:53Z Stashbot 7414 jhathaway@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on sretest2010.codfw.wmnet with reason: host reimage 2440763 wikitext text/x-wiki m1gmdoowkfd6fcy4tkwr6cvkwha66mo 2440764 2440763 2026-07-24T23:43:29Z Stashbot 7414 jhathaway@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host sretest2010.codfw.wmnet with OS trixie 2440764 wikitext text/x-wiki 4802sjtvoun64gv3qdi8pf031xhg9hq 2440765 2440764 2026-07-24T23:54:42Z Stashbot 7414 ryankemper@cumin2003: conftool action : set/pooled=yes; selector: name=wdqs2008.codfw.wmnet 2440765 wikitext text/x-wiki c948evwj4bx3kse0yuavc3e0tu0kst9 2440766 2440765 2026-07-24T23:54:45Z Stashbot 7414 ryankemper@cumin2003: conftool action : set/pooled=yes; selector: name=wdqs1014.eqiad.wmnet 2440766 wikitext text/x-wiki q58fsleotfy146dl9mx49f3mraazdcb 2440768 2440766 2026-07-25T00:11:48Z Stashbot 7414 ryankemper: [WDQS] T430880 Repooled wdqs1014.eqiad.wmnet and wdqs2008.codfw.wmnet after Bookworm reimage, transfer, and postflight; wdqs2008 is serving, while wdqs1014 will remain outside of service until a pybal restart next monday 2440768 wikitext text/x-wiki ktrc2jmbkcd73nk2ifngffueovn0kc6 2440769 2440768 2026-07-25T00:24:11Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host wdqs2023.codfw.wmnet with OS bookworm 2440769 wikitext text/x-wiki pnu57v86c0weowsisq7m9h5t66ovqja 2440770 2440769 2026-07-25T00:24:30Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host wdqs1016.eqiad.wmnet with OS bookworm 2440770 wikitext text/x-wiki 0hfdw8md8j4wjhot5pq9yuc7b43cu7z 2440771 2440770 2026-07-25T00:27:42Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.move-vlan for host wdqs2023 2440771 wikitext text/x-wiki 15vjrwpq0i1exsiwk84or1i08y6epz3 2440772 2440771 2026-07-25T00:27:53Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.move-vlan for host wdqs1016 2440772 wikitext text/x-wiki cspnbe33gvjzvd4p5efcw4vqj9onb18 2440773 2440772 2026-07-25T00:27:55Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wdqs1016 2440773 wikitext text/x-wiki 32r0ust5v41dfriy2hh5mb0up55p9qp 2440774 2440773 2026-07-25T00:31:10Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.dns.netbox 2440774 wikitext text/x-wiki 4zhugbcwylakegauwvas3z2wirar8in 2440775 2440774 2026-07-25T00:43:55Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1016.eqiad.wmnet with reason: host reimage 2440775 wikitext text/x-wiki n10kso7zde3tz67nlg44h01rbltso6e 2440776 2440775 2026-07-25T00:49:22Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1016.eqiad.wmnet with reason: host reimage 2440776 wikitext text/x-wiki gzxy43103asfo3ar0mft6a6ap58uia3 2440777 2440776 2026-07-25T01:15:11Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wdqs1016.eqiad.wmnet with OS bookworm 2440777 wikitext text/x-wiki kqwo1rozy2nufjkw6qqho3yyin8tl0l 2440780 2440777 2026-07-25T02:00:45Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2440780 wikitext text/x-wiki kho7jwr2ekllr491lk08w4tu3ghzu47 2440781 2440780 2026-07-25T02:07:12Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 26s) 2440781 wikitext text/x-wiki bho46dhrzqqiggwyexvlpj4swaul74l 2440782 2440781 2026-07-25T02:30:03Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wdqs2023 - ryankemper@cumin2003" 2440782 wikitext text/x-wiki seqiwojttvzdwiia2nranvyx8qumoul 2440783 2440782 2026-07-25T02:30:08Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wdqs2023 - ryankemper@cumin2003" 2440783 wikitext text/x-wiki bjg5meco9zbf4y7k6qsxc4a1wyua46v 2440784 2440783 2026-07-25T02:30:09Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2440784 wikitext text/x-wiki 8edqsa4ji48du6nr0lfsv5t96a4sgn1 2440785 2440784 2026-07-25T02:30:10Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.dns.wipe-cache wdqs2023.codfw.wmnet 35.0.192.10.in-addr.arpa 5.3.0.0.0.0.0.0.2.9.1.0.0.1.0.0.1.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2440785 wikitext text/x-wiki 97wpitj32go4rskb6ghdubew429hh2i 2440786 2440785 2026-07-25T02:30:12Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wdqs2023.codfw.wmnet 35.0.192.10.in-addr.arpa 5.3.0.0.0.0.0.0.2.9.1.0.0.1.0.0.1.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2440786 wikitext text/x-wiki 107bghvba1bahu24urw1tzajlifkosk 2440787 2440786 2026-07-25T02:30:14Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.network.configure-switch-interfaces for host wdqs2023 2440787 wikitext text/x-wiki ci4vrk3mubuibjo483thmqsb1niq86k 2440788 2440787 2026-07-25T02:33:05Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wdqs2023 2440788 wikitext text/x-wiki 9daoew0yr0gl8hwuxffzfgpy6jvnfyd 2440789 2440788 2026-07-25T02:33:07Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wdqs2023 2440789 wikitext text/x-wiki b1wu59i9ktlikhl4izr8hbirbyo7k10 2440790 2440789 2026-07-25T02:56:04Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs2023.codfw.wmnet with reason: host reimage 2440790 wikitext text/x-wiki pbde014whqb3kaon1k6ynnfm7ktzg3x 2440791 2440790 2026-07-25T02:59:41Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs2023.codfw.wmnet with reason: host reimage 2440791 wikitext text/x-wiki 7g0pgma48y6gxfpv5onquvdfn84oita 2440792 2440791 2026-07-25T03:27:01Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wdqs2023.codfw.wmnet with OS bookworm 2440792 wikitext text/x-wiki bvp98jptvdzbpjipvwt1s37u3r9wgb8 2440795 2440792 2026-07-25T05:06:46Z Stashbot 7414 ryankemper@deploy1003: Started deploy [wdqs/wdqs@e8fb00c]: T430880 deploy to fresh wdqs host 2440795 wikitext text/x-wiki gco78dihb7x65fky7jjcuepqy59c3z9 2440796 2440795 2026-07-25T05:06:52Z Stashbot 7414 ryankemper@deploy1003: Finished deploy [wdqs/wdqs@e8fb00c]: T430880 deploy to fresh wdqs host (duration: 00m 06s) 2440796 wikitext text/x-wiki 3qbdhc5yf1ufy0ougbp5jxyy3s5jw0x 2440797 2440796 2026-07-25T05:07:02Z Stashbot 7414 ryankemper@deploy1003: Started deploy [wdqs/wdqs@e8fb00c]: T430880 deploy to fresh wdqs host 2440797 wikitext text/x-wiki 1c9z71fok8p49cdx19rserviubap0f1 2440798 2440797 2026-07-25T05:07:10Z Stashbot 7414 ryankemper@deploy1003: Finished deploy [wdqs/wdqs@e8fb00c]: T430880 deploy to fresh wdqs host (duration: 00m 07s) 2440798 wikitext text/x-wiki 71obbw2k1mreqdso3wm90bd278fpk8a 2440799 2440798 2026-07-25T05:19:57Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.wdqs.data-transfer (T430880, restore data on newly-reimaged host) xfer wdqs-all from wdqs2008.codfw.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards 2440799 wikitext text/x-wiki lz2so4aclugsivixuvbb6mwvrnb94q8 2440800 2440799 2026-07-25T05:20:29Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.wdqs.data-transfer (T430880, restore data on newly-reimaged host) xfer scholarly_articles from wdqs1024.eqiad.wmnet -> wdqs2023.codfw.wmnet, repooling source-only afterwards 2440800 wikitext text/x-wiki m8mg734as2uh3pe5ooduzh0571jeznl 2440801 2440800 2026-07-25T06:17:09Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T430880, restore data on newly-reimaged host) xfer scholarly_articles from wdqs1024.eqiad.wmnet -> wdqs2023.codfw.wmnet, repooling source-only afterwards 2440801 wikitext text/x-wiki kffogkveqcdwvainj558u357qj5hhun 2440802 2440801 2026-07-25T06:35:39Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T430880, restore data on newly-reimaged host) xfer wdqs-all from wdqs2008.codfw.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards 2440802 wikitext text/x-wiki gohl36tqx26cxpfme9650aasfomcabt 0 17290 2440746 2440420 2026-07-24T17:59:43Z Stashbot 7414 dancy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/1316011 2440746 wikitext text/x-wiki tdjqdvxl63gvjzkci2nqshfn0wudyg1 498 30942 2440711 2440335 2026-07-24T14:49:01Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.roll_reboot_osds 2440711 wikitext text/x-wiki 6d44t41539ygndx2gsq5sbj9kjrscye 2440714 2440711 2026-07-24T15:10:03Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.roll_reboot_osds (exit_code=0) 2440714 wikitext text/x-wiki ba7yk1fofssvop72kksrfm8xvlulqye 0 442914 2440755 2364655 2026-07-24T21:56:03Z Faizal04 57992 Undid revision [[Special:Diff/2364655|2364655]] by [[Special:Contributions/GGoncalves-WMF|GGoncalves-WMF]] ([[User talk:GGoncalves-WMF|talk]]) 2440755 wikitext text/x-wiki 9qovyxcm6nf426dged9k69kukem3e9t 3 443249 2440664 2440501 2026-07-24T12:57:46Z Dreamy Jazz 31583 /* Thanks! */ Reply 2440664 wikitext text/x-wiki 6fa2ylqwe5bpo1adonwliactalkzmhs 498 443949 2440667 2439888 2026-07-24T13:26:39Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 51b9ad1 (Add value-scoped reference columns (P123/Q456/S*, P123/?grouping/S*)) for T428637 2440667 wikitext text/x-wiki dys8dor4i1knhaw0b6ja9hox1vc05o4 2440668 2440667 2026-07-24T13:26:42Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy cefdfd6 (Bind ?grouping in drill-down queries for value-scoped columns) for T428637 2440668 wikitext text/x-wiki 03j385uxjxp05flq4h39dc04greiozi 2440814 2440668 2026-07-25T09:37:52Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 1b23f69 (Hide empty Top Properties header for column-less dashboards) 2440814 wikitext text/x-wiki bdha2c27trcjwl4hyje5196lhx14gci 0 444622 2440747 2439075 2026-07-24T20:11:01Z AndrewBogott (WMF) 51047 /* Create the upgrade tasks */ removed no-longer-relevant workgroup link 2440747 wikitext text/x-wiki stfk7htt6f0k04h5ahtm2jz0tq6tcd1 3 447329 2440793 1903145 2026-07-25T04:17:24Z SHB2000 24851 SHB2000 moved page [[User talk:Taiwania Justo]] to [[User talk:Sekidoki]]: Automatically moved page while renaming the user "[[Special:CentralAuth/Taiwania Justo|Taiwania Justo]]" to "[[Special:CentralAuth/Sekidoki|Sekidoki]]" 1903145 wikitext text/x-wiki iefgn2jdixj9lz6kf1dxjmsmlh7v70u 0 449160 2440767 2440556 2026-07-25T00:00:13Z Dexbot 30554 Bot: Updating the report 2440767 wikitext text/x-wiki lwqpc9yllw0bfudqs95jeu2l5l97s6i 0 454037 2440804 2361118 2026-07-25T09:25:00Z Dexbot 30554 Bot: Bot: Fixing double redirect to [[Test Kitchen/Archive/Deploy client libraries]] 2440804 wikitext text/x-wiki kg0nwnqrujqctwkce0a887gvirdtiux 0 454125 2440806 2361144 2026-07-25T09:25:03Z Dexbot 30554 Bot: Bot: Fixing double redirect to [[Test Kitchen/Archive/Deploy client libraries]] 2440806 wikitext text/x-wiki kg0nwnqrujqctwkce0a887gvirdtiux 0 456051 2440805 2361142 2026-07-25T09:25:02Z Dexbot 30554 Bot: Bot: Fixing double redirect to [[Test Kitchen/Archive/Deploy client libraries]] 2440805 wikitext text/x-wiki kg0nwnqrujqctwkce0a887gvirdtiux 3 457836 2440732 2271042 2026-07-24T16:19:47Z StrikerBot 8475 Welcome to Toolforge! 2440732 wikitext text/x-wiki 2w55hvh7xrwp6nismqm21xy8oaz6h7j 0 458940 2440807 2361477 2026-07-25T09:25:04Z Dexbot 30554 Bot: Bot: Fixing double redirect to [[Test Kitchen/Archive/Deploy client libraries]] 2440807 wikitext text/x-wiki kg0nwnqrujqctwkce0a887gvirdtiux 0 458948 2440808 2361485 2026-07-25T09:25:05Z Dexbot 30554 Bot: Bot: Fixing double redirect to [[Test Kitchen/Archive/Deploy client libraries]] 2440808 wikitext text/x-wiki kg0nwnqrujqctwkce0a887gvirdtiux 0 458961 2440809 2361498 2026-07-25T09:25:06Z Dexbot 30554 Bot: Bot: Fixing double redirect to [[Test Kitchen/Archive/Deploy client libraries]] 2440809 wikitext text/x-wiki kg0nwnqrujqctwkce0a887gvirdtiux 0 458962 2440810 2361499 2026-07-25T09:25:08Z Dexbot 30554 Bot: Bot: Fixing double redirect to [[Test Kitchen/Archive/Deploy client libraries]] 2440810 wikitext text/x-wiki kg0nwnqrujqctwkce0a887gvirdtiux 0 459044 2440709 2440513 2026-07-24T14:48:10Z DCausse (WMF) 5604 /* Alerts */ 2440709 wikitext text/x-wiki 4at9glqi0mx4oliduxh2e1ntxgmf5lw 0 459442 2440803 2361095 2026-07-25T09:24:59Z Dexbot 30554 Bot: Bot: Fixing double redirect to [[Test Kitchen/Archive/Deploy client libraries]] 2440803 wikitext text/x-wiki kg0nwnqrujqctwkce0a887gvirdtiux 0 459454 2440811 2361119 2026-07-25T09:25:10Z Dexbot 30554 Bot: Bot: Fixing double redirect to [[Test Kitchen/Archive/Deploy client libraries]] 2440811 wikitext text/x-wiki kg0nwnqrujqctwkce0a887gvirdtiux 0 459466 2440812 2361143 2026-07-25T09:25:11Z Dexbot 30554 Bot: Bot: Fixing double redirect to [[Test Kitchen/Archive/Deploy client libraries]] 2440812 wikitext text/x-wiki kg0nwnqrujqctwkce0a887gvirdtiux 0 459467 2440813 2361145 2026-07-25T09:25:12Z Dexbot 30554 Bot: Bot: Fixing double redirect to [[Test Kitchen/Archive/Deploy client libraries]] 2440813 wikitext text/x-wiki kg0nwnqrujqctwkce0a887gvirdtiux 0 460440 2440779 2438684 2026-07-25T02:00:30Z DeploymentCalendarTool 20896 Add last week 2440779 wikitext text/x-wiki aqn66a55b3fei9w21g5wp150ygtmsrl 3 460486 2440794 2026-07-25T04:17:24Z SHB2000 24851 SHB2000 moved page [[User talk:Taiwania Justo]] to [[User talk:Sekidoki]]: Automatically moved page while renaming the user "[[Special:CentralAuth/Taiwania Justo|Taiwania Justo]]" to "[[Special:CentralAuth/Sekidoki|Sekidoki]]" 2440794 wikitext text/x-wiki hjmmpmdoraqvo3sekcwqf1i173wub9i