Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.17 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 2161 2451764 2451470 2026-08-26T12:59:16Z Stashbot 7414 taavi: drop metricsinfra redis job T429930 2451764 wikitext text/x-wiki pb8j55aln015s9t8gr6tujj6wiyup3b 1 2515 2452097 2449944 2026-08-27T08:34:34Z MABot 10804 Bot: Archiving 1 thread (older than 30 days) to [[Talk:Main Page/Archive 2]] 2452097 wikitext text/x-wiki siv1uf1xtkisdnkqlv766k79ajz1z2r 498 3086 2451836 2451696 2026-08-26T14:48:49Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster tools upgrade to 1.33.13 (T433132) 2451836 wikitext text/x-wiki ijl5o0cuq1tbv2kupgvyrzeh90qoonc 2451837 2451836 2026-08-26T14:49:09Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=99) for cluster tools upgrade to 1.33.13 (T433132) 2451837 wikitext text/x-wiki m6b1p8nkavya6yg81pvhcu3lnkt59j6 2451839 2451837 2026-08-26T14:57:10Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster tools upgrade to 1.33.13 (T433132) 2451839 wikitext text/x-wiki 49zylyo86bqzw0cdzn84d5sheiuhgyz 2451846 2451839 2026-08-26T15:17:08Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=0) for cluster tools upgrade to 1.33.13 (T433132) 2451846 wikitext text/x-wiki 9z2z0n80xhoi9vd2vjhi7ik6w0pj42c 2451847 2451846 2026-08-26T15:19:14Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-7 from 1.32.13 to 1.33.13 (T433132) 2451847 wikitext text/x-wiki okhrwdzjl0b4to9x9969iwnw45bn29k 2451848 2451847 2026-08-26T15:20:23Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-control-7 from 1.32.13 to 1.33.13 (T433132) 2451848 wikitext text/x-wiki a4cd2q4s9mdon9evqef3p4igk4v0k8e 2451855 2451848 2026-08-26T15:28:08Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-7 from 1.32.13 to 1.33.13 (T433132) 2451855 wikitext text/x-wiki 2iys2mj9gu5xxof5pvi4gywe083y42e 2451857 2451855 2026-08-26T15:30:42Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-control-7 from 1.32.13 to 1.33.13 (T433132) 2451857 wikitext text/x-wiki ibf1qk1wxl416l4vll5hqztfmll6se3 2451859 2451857 2026-08-26T15:31:44Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-7 from 1.32.13 to 1.33.13 (T433132) 2451859 wikitext text/x-wiki ndqfddjlrolpwhq19q2m6anzx45ges4 2451868 2451859 2026-08-26T15:38:13Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-control-7 from 1.32.13 to 1.33.13 (T433132) 2451868 wikitext text/x-wiki 9ern5o0qxlvxbckoxzy54bnlk6qx78x 2451874 2451868 2026-08-26T15:42:12Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.restart_static_pods for tools-k8s-control-7 (T433132) 2451874 wikitext text/x-wiki hdtendlrgfx2f0qf02eginpy9aatit0 2451878 2451874 2026-08-26T15:48:49Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.restart_static_pods (exit_code=0) for tools-k8s-control-7 (T433132) 2451878 wikitext text/x-wiki mxls2l74h8ciku7ihs0edczqtq5vpu3 2451882 2451878 2026-08-26T16:19:49Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-8 from 1.32.13 to 1.33.13 (T433132) 2451882 wikitext text/x-wiki pvuxfi9z57sdoocvrqj8xppjbl5h59s 2451886 2451882 2026-08-26T16:24:34Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-8 from 1.32.13 to 1.33.13 (T433132) 2451886 wikitext text/x-wiki 3mkc3a42l1l3kxh8ls562lmpv1zz5do 2451888 2451886 2026-08-26T16:25:03Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-9 from 1.32.13 to 1.33.13 (T433132) 2451888 wikitext text/x-wiki abzxs4qlctnu5bykefkw7bh6i5uilqj 2451891 2451888 2026-08-26T16:31:00Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-9 from 1.32.13 to 1.33.13 (T433132) 2451891 wikitext text/x-wiki b8zojgo2nhqilj8gculemc8fh8ihyiq 2451892 2451891 2026-08-26T16:33:06Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers for tools-k8s-worker-102, tools-k8s-worker-103, tools-k8s-worker-105, tools-k8s-worker-106, tools-k8s-worker-107, tools-k8s-worker-108, tools-k8s-worker-109, tools-k8s-worker-110, tools-k8s-worker-111, tools-k8s-worker-112, tools-k8s-worker-113 (T433132) 2451892 wikitext text/x-wiki mng0c6q235hd4uas0nzawl700pcv8v6 2451894 2451892 2026-08-26T16:43:59Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-gateway-1 from 1.32.13 to 1.33.13 (T433132) 2451894 wikitext text/x-wiki sx1tcglngy4ht3fvq2hu7x8u39z2car 2451895 2451894 2026-08-26T16:44:52Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-gateway-1 from 1.32.13 to 1.33.13 (T433132) 2451895 wikitext text/x-wiki cvnvxl6q8siqax55fygk2i8zj9vrj0a 2451896 2451895 2026-08-26T16:46:17Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers (exit_code=0) for tools-k8s-worker-102, tools-k8s-worker-103, tools-k8s-worker-105, tools-k8s-worker-106, tools-k8s-worker-107, tools-k8s-worker-108, tools-k8s-worker-109, tools-k8s-worker-110, tools-k8s-worker-111, tools-k8s-worker-112, tools-k8s-worker-113 (T433132) 2451896 wikitext text/x-wiki 4cftoj1v4ok5sryrhjth0j9gqsa0lzc 2451897 2451896 2026-08-26T16:46:47Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-worker-nfs-23, tools- 2451897 wikitext text/x-wiki r4jyif5iwwnacur89rab89mbx34zvc8 2451898 2451897 2026-08-26T16:46:57Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-worker-nfs-23, tools- 2451898 wikitext text/x-wiki fa0db6q1mmcngx1csnfhcws3t9ay9h6 2451899 2451898 2026-08-26T16:47:06Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-worker-nfs-23, tools- 2451899 wikitext text/x-wiki iw8vs7bfxbgqucek8vyjl4g0qy8ijmx 2451901 2451899 2026-08-26T16:48:42Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-gateway-2 from 1.32.13 to 1.33.13 (T433132) 2451901 wikitext text/x-wiki mt6rkl4x9q7d7orfpw6c47z2z84lkt7 2451902 2451901 2026-08-26T16:49:43Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-gateway-2 from 1.32.13 to 1.33.13 (T433132) 2451902 wikitext text/x-wiki eiklav0vri9tpbtkb7cypjwa6mxopcx 2451903 2451902 2026-08-26T16:54:26Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-gateway-3 from 1.32.13 to 1.33.13 (T433132) 2451903 wikitext text/x-wiki b4in6um0qpico82ok5ha7vmg159q2dl 2451904 2451903 2026-08-26T16:55:21Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-gateway-3 from 1.32.13 to 1.33.13 (T433132) 2451904 wikitext text/x-wiki t4d9ku5rtjbbwtcaf8fi1raqnn5e6r6 2451908 2451904 2026-08-26T17:00:11Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade_bastions for tools-bastion-15.tools.eqiad1.wikimedia.cloud, tools-bastion-14.tools.eqiad1.wikimedia.cloud (T433132) 2451908 wikitext text/x-wiki 6fejk3gughdgq4zt799als7l6g7ae2i 2451910 2451908 2026-08-26T17:00:35Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade_bastions (exit_code=0) for tools-bastion-15.tools.eqiad1.wikimedia.cloud, tools-bastion-14.tools.eqiad1.wikimedia.cloud (T433132) 2451910 wikitext text/x-wiki hj7uhaf1ekk1biieu7co6g9rkv13jzs 2451915 2451910 2026-08-26T17:11:17Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers (exit_code=0) for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-wo 2451915 wikitext text/x-wiki 4bfklqbdxon668izt6uchajeg3ih36t 2451918 2451915 2026-08-26T17:12:04Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers (exit_code=99) for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-w 2451918 wikitext text/x-wiki t9m8nruxcez8qh3dcj1ix4k8o6kl37t 2451919 2451918 2026-08-26T17:13:06Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers (exit_code=0) for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-wo 2451919 wikitext text/x-wiki 7zyxizeph7wozlilg5a1igq0g21nzfv 2451922 2451919 2026-08-26T17:15:37Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-worker-nfs-23, tools- 2451922 wikitext text/x-wiki cfzsmij3jqh5jq7gtdvyzcgfhen20i2 2451924 2451922 2026-08-26T17:17:06Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers (exit_code=0) for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-wo 2451924 wikitext text/x-wiki c6yh7bzu7g39q3vntugpyb2xj5wro4b 2451925 2451924 2026-08-26T17:17:15Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-worker-nfs-23, tools- 2451925 wikitext text/x-wiki e45n64x3y9wx0tu1yuger23pbmi55ma 2451926 2451925 2026-08-26T17:17:47Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers (exit_code=0) for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-wo 2451926 wikitext text/x-wiki sedknir2ucp8ai5h4xeohlaszzud0du 2451927 2451926 2026-08-26T17:17:56Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-worker-nfs-23, tools- 2451927 wikitext text/x-wiki ogwm7akv4oc13xz7ezcssilwtvtvwvu 2451929 2451927 2026-08-26T17:18:27Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers (exit_code=0) for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-wo 2451929 wikitext text/x-wiki 03lu15m41uxkz331hpvgtt3642a72xk 2452102 2451929 2026-08-27T09:03:35Z Stashbot 7414 dhinus: restart harbor in tools-harbor-2 with "docker compose restart" 2452102 wikitext text/x-wiki s6i9m4w1pg7fmlsfl9ikfz6gh9xub9l 2452104 2452102 2026-08-27T10:03:18Z Stashbot 7414 fnegri@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor 2452104 wikitext text/x-wiki f5ef31cuf9rx8jsmx4hb58qu0eka1x9 2452105 2452104 2026-08-27T10:03:25Z Stashbot 7414 fnegri@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor 2452105 wikitext text/x-wiki 4yhaekyigoyflsqn6pegu1bjjefhced 2452110 2452105 2026-08-27T10:15:24Z Stashbot 7414 fnegri@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor 2452110 wikitext text/x-wiki 6omo3x9s9g6mk55bur3k8gmsd59par0 2452111 2452110 2026-08-27T10:15:31Z Stashbot 7414 fnegri@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor 2452111 wikitext text/x-wiki d1psbgj0xrqt1z47i4q7uxmsxqboqo4 2452112 2452111 2026-08-27T10:17:21Z Stashbot 7414 fnegri@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor 2452112 wikitext text/x-wiki le67r6uzkk3vuhrqbqov9izgr3p777w 2452113 2452112 2026-08-27T10:17:27Z Stashbot 7414 fnegri@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor 2452113 wikitext text/x-wiki toq6idqvqh69jdih5beosyrlgvujghb 2452117 2452113 2026-08-27T10:41:25Z Stashbot 7414 fnegri@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor 2452117 wikitext text/x-wiki jtnj8s5merzdj0pyjuqu0g1ag7r9vtn 2452118 2452117 2026-08-27T10:41:58Z Stashbot 7414 fnegri@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor 2452118 wikitext text/x-wiki ov6z5tvwanvds2fdfz53jtevv28zuwd 2452119 2452118 2026-08-27T10:43:13Z Stashbot 7414 fnegri@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor 2452119 wikitext text/x-wiki q3jyugvv1rmwlj75uuzdu6k8umo70ni 2452120 2452119 2026-08-27T10:43:42Z Stashbot 7414 fnegri@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor 2452120 wikitext text/x-wiki 8faueahe5qi0sy2rdo3xsoj6xox15p1 0 4108 2451784 2451753 2026-08-26T13:34:00Z KartikMistry 1115 Done. 2451784 wikitext text/x-wiki g3t6pewej93juoej66npnijorclazcw 2451917 2451784 2026-08-26T17:11:46Z ScheduleDeploymentBot 37566 Add [[gerrit:1329616]] to Wednesday, August 26 UTC late backport window 2451917 wikitext text/x-wiki 8xfm4sljcjw09a0owuq1uz62qx5uzhg 2452087 2451917 2026-08-27T07:12:36Z Christoph Jauera (WMDE) 3635 /* {{Deployment_day|date=2026-08-27}} */ Adding 1328590 2452087 wikitext text/x-wiki 82wdxu6to5voyikll8bddn06d98qqdb 2452116 2452087 2026-08-27T10:22:34Z ScheduleDeploymentBot 37566 Add [[gerrit:1329672]] to Thursday, August 27 UTC afternoon backport window 2452116 wikitext text/x-wiki reqt2p8rpbdm5r8941cvdrn5j4ipc4v 498 6726 2452114 2451686 2026-08-27T10:20:15Z Stashbot 7414 fnegri@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor 2452114 wikitext text/x-wiki n6g6qq4aokf4eknxkopcw5xq49au4n6 2452115 2452114 2026-08-27T10:22:09Z Stashbot 7414 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-harbor 2452115 wikitext text/x-wiki quromh37w0tponhhz9a2r313bj41imt 2452121 2452115 2026-08-27T10:54:08Z Stashbot 7414 fnegri@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor 2452121 wikitext text/x-wiki 7c5jy1m23yx01xcir2a0sgr3hxhl8ox 2452122 2452121 2026-08-27T10:54:14Z Stashbot 7414 fnegri@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor 2452122 wikitext text/x-wiki 45zljcuko5ppn3r5il93zcuglckrvdb 0 7814 2452095 2407261 2026-08-27T08:34:10Z Lucas Werkmeister (WMDE) 10037 /* How do I ... */ update for kafkacat→kcat rename (keep mentioning the old name so people can search for it) 2452095 wikitext text/x-wiki rw4zkas53ldfkxmctt041siq6kjoxw0 2452098 2452095 2026-08-27T08:35:03Z Lucas Werkmeister (WMDE) 10037 add kcat anchor (see previous edit) 2452098 wikitext text/x-wiki 5j9z9tdv0idgc7jjpxkzfp2shb8jjkz 0 7919 2451756 2451755 2026-08-26T12:02:54Z Stashbot 7414 mszwarc@deploy1003: Finished scap sync-world: Backport for [[gerrit:1329531|UserGroupsSpecialPage: Check in_array in strict mode (T435907)]], [[gerrit:1329530|UserGroupsSpecialPage: Check in_array in strict mode (T435907)]] (duration: 07m 27s) 2451756 wikitext text/x-wiki g1ec4dnu0zl4k615yt9054jmyv3hj1s 2451760 2451756 2026-08-26T12:31:40Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti3005.esams.wmnet with OS bookworm 2451760 wikitext text/x-wiki 2rkgdz1h5e07sm4t5dwb5z7o4d6p2o4 2451761 2451760 2026-08-26T12:38:43Z Stashbot 7414 aokoth@cumin1003: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1003.wikimedia.org with reason: Security Release - T436069 2451761 wikitext text/x-wiki azqp2zebn3yqf3tmlrl4rilhnjh0mxs 2451762 2451761 2026-08-26T12:49:31Z Stashbot 7414 aokoth@cumin1003: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1003.wikimedia.org with reason: Security Release - T436069 2451762 wikitext text/x-wiki 6ic7fxjjboo55ir93txssb57pi72a27 2451763 2451762 2026-08-26T12:49:48Z Stashbot 7414 Daimona: Running query from T436088#12256503 in x1.wikishared 2451763 wikitext text/x-wiki itdhew8m7n0tt5zneqd1tr31enqrui4 2451765 2451763 2026-08-26T13:01:06Z Stashbot 7414 brouberol@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1010.eqiad.wmnet with OS bookworm 2451765 wikitext text/x-wiki lx3wfvuf8ctlixpk2z5g5c9iwgjv7ea 2451766 2451765 2026-08-26T13:04:13Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.metamonitoring.downtime Downtime for 3:00:00 of prometheus/deadmanswitchnotified, prometheus/deadmanswitchonamdb, prometheus/extmon on 2 host(s) with reason: eqsin site rebuild 2451766 wikitext text/x-wiki nmoo82nptia7v1op0ji317e5grya570 2451767 2451766 2026-08-26T13:04:18Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.metamonitoring.downtime (exit_code=0) Downtime for 3:00:00 of prometheus/deadmanswitchnotified, prometheus/deadmanswitchonamdb, prometheus/extmon on 2 host(s) with reason: eqsin site rebuild 2451767 wikitext text/x-wiki 5rjyfu23sb79v9ohp8oyzjiizo0nech 2451768 2451767 2026-08-26T13:05:03Z Stashbot 7414 cmooney@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on 12 hosts with reason: site migration to new Nokia switches 2451768 wikitext text/x-wiki 86pgu3eekfafhke4i4golvpso7zq4os 2451769 2451768 2026-08-26T13:06:10Z Stashbot 7414 cmooney@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on 13 hosts with reason: site migration to new Nokia switches 2451769 wikitext text/x-wiki m3a6xfs2mcfckh2hww1q3r5pl6gms2q 2451770 2451769 2026-08-26T13:06:32Z Stashbot 7414 cmooney@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on 12 hosts with reason: site migration to new Nokia switches 2451770 wikitext text/x-wiki fuhhp171u87jkmmu6fbcza9hem8cnok 2451771 2451770 2026-08-26T13:07:43Z Stashbot 7414 aokoth@cumin1003: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab2002.wikimedia.org with reason: Security Release - T436069 2451771 wikitext text/x-wiki mp4wxlnver9y530ihn7i0vppw9wkaqh 2451772 2451771 2026-08-26T13:12:26Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db-test2002.codfw.wmnet decommissioned, removing all IPs except the asset tag one - fceratto@cumin1003" 2451772 wikitext text/x-wiki c8kj064uabyw1vsckgqhisndlfx8kms 2451773 2451772 2026-08-26T13:12:29Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2451773 wikitext text/x-wiki 061l3dizo3ud3wmu27tesntqh5gsgju 2451774 2451773 2026-08-26T13:12:32Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db-test2002.codfw.wmnet 2451774 wikitext text/x-wiki 6fkp5vtd92hd7hnnilj90xa568su3vy 2451775 2451774 2026-08-26T13:14:38Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti3005.esams.wmnet 2451775 wikitext text/x-wiki ke6a4qtd8n4zqgm7cc705yw1ee8zgxu 2451776 2451775 2026-08-26T13:16:10Z Stashbot 7414 brouberol@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-presto1010.eqiad.wmnet with reason: host reimage 2451776 wikitext text/x-wiki try7lq3kfnhn1m4i9gspvm5f21zl6m9 2451777 2451776 2026-08-26T13:16:25Z Stashbot 7414 kartik@deploy1003: Started scap sync-world: Backport for [[gerrit:1324031|ULS: Remove wgULSLanguageSelectorV2Enabled config (T429122)]] 2451777 wikitext text/x-wiki pd48b9h0fyp925qwzp4756m4pss2y6k 2451778 2451777 2026-08-26T13:18:25Z Stashbot 7414 aokoth@cumin1003: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab2002.wikimedia.org with reason: Security Release - T436069 2451778 wikitext text/x-wiki tgexnb9dehkhykpnuu2qgq4f55r3jto 2451779 2451778 2026-08-26T13:19:03Z Stashbot 7414 kartik@deploy1003: kartik, abi: Backport for [[gerrit:1324031|ULS: Remove wgULSLanguageSelectorV2Enabled config (T429122)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2451779 wikitext text/x-wiki 0o86565hvrhoz0sjdp2toeewhtv1jny 2451780 2451779 2026-08-26T13:21:18Z Stashbot 7414 brouberol@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on an-presto1010.eqiad.wmnet with reason: host reimage 2451780 wikitext text/x-wiki in9wvqb03tzmj41cxbjmqes32k99w4i 2451781 2451780 2026-08-26T13:22:48Z Stashbot 7414 kartik@deploy1003: kartik, abi: Continuing with deployment 2451781 wikitext text/x-wiki auro7fty3ao012cj5mlx6xrwwuijll4 2451782 2451781 2026-08-26T13:27:23Z Stashbot 7414 kartik@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324031|ULS: Remove wgULSLanguageSelectorV2Enabled config (T429122)]] (duration: 10m 57s) 2451782 wikitext text/x-wiki 4tdde4b4yhwwglnmx9nh89g8yn0lr9n 2451783 2451782 2026-08-26T13:32:10Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host clouddumps1002.wikimedia.org with OS trixie 2451783 wikitext text/x-wiki cbt75xbub0akmvum7vrsf1jhg76bf2e 2451785 2451783 2026-08-26T13:39:44Z Stashbot 7414 jmm@cumin2003: END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) for host ganeti3005.esams.wmnet 2451785 wikitext text/x-wiki 38cl51oqmt16dpdy8a53ojxgzk9ik38 2451787 2451785 2026-08-26T13:46:52Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on clouddumps1002.wikimedia.org with reason: host reimage 2451787 wikitext text/x-wiki h8796k3bx58g33mr18ajfwnhperyc9f 2451788 2451787 2026-08-26T13:53:12Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddumps1002.wikimedia.org with reason: host reimage 2451788 wikitext text/x-wiki sy6zok8pypfsind2g8cecsjxv56fb0l 2451789 2451788 2026-08-26T14:03:48Z Stashbot 7414 brouberol@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-presto1010.eqiad.wmnet with OS bookworm 2451789 wikitext text/x-wiki teazb9t8yr7omytgrajx39e52pepzj4 2451790 2451789 2026-08-26T14:05:00Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2451790 wikitext text/x-wiki juvgmvg751krlfxd121twow0q4bf4el 2451791 2451790 2026-08-26T14:07:47Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1329564|Switch eswiki to thumb.wikimedia.org (T427465)]] 2451791 wikitext text/x-wiki sxeiz50ngr6a8wt5h856a1eygwoie5k 2451792 2451791 2026-08-26T14:08:06Z Stashbot 7414 fceratto@cumin1003: Removing db-test2002 from zarcillo T435912 2451792 wikitext text/x-wiki 4rywc8xmdyfiiql56js8p5a5r01rsyu 2451793 2451792 2026-08-26T14:08:11Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.decommission (exit_code=0) 2451793 wikitext text/x-wiki ftn1sb49qjcq2s7338cy17rdxuqv1ve 2451794 2451793 2026-08-26T14:08:54Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2451794 wikitext text/x-wiki kiaxx8zuxdc8w3vf7l654re0q4p6tq4 2451795 2451794 2026-08-26T14:09:30Z Stashbot 7414 ecarg@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2451795 wikitext text/x-wiki sgjnz9bs2rbpz9p5igfaolqk7eyte23 2451796 2451795 2026-08-26T14:10:17Z Stashbot 7414 ecarg@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2451796 wikitext text/x-wiki rkmjxppbjm3sr99glfger6orlz2iv55 2451797 2451796 2026-08-26T14:10:21Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1329564|Switch eswiki to thumb.wikimedia.org (T427465)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2451797 wikitext text/x-wiki 14c3brhctdu349tvuvux2ulxnjp222w 2451798 2451797 2026-08-26T14:11:41Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2451798 wikitext text/x-wiki g9ubfiere5zcdexj7lva2g2mm7klno3 2451799 2451798 2026-08-26T14:11:54Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply 2451799 wikitext text/x-wiki 3kzbuxobfva5nnlvpkh2m3xjlrbgak0 2451800 2451799 2026-08-26T14:13:27Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.ganeti.makevm for new host krb1004.eqiad.wmnet 2451800 wikitext text/x-wiki cj6rp74olvweh9anl90jcjuc8zsj0dr 2451801 2451800 2026-08-26T14:13:30Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.dns.netbox 2451801 wikitext text/x-wiki 4owu1exphulbvbxq1igtgqo5zrr8qxs 2451802 2451801 2026-08-26T14:13:38Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2451802 wikitext text/x-wiki jm078howh796k4orzvmga1yn37a3igu 2451803 2451802 2026-08-26T14:15:42Z Stashbot 7414 moritzm: installing node-flatted security updates 2451803 wikitext text/x-wiki jm5tb5ju4eam06spu2qiyyvzv59uqzh 2451804 2451803 2026-08-26T14:17:59Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1329564|Switch eswiki to thumb.wikimedia.org (T427465)]] (duration: 10m 11s) 2451804 wikitext text/x-wiki 0u9d47ckd6om5vlbkouv62ehdiemjzv 2451805 2451804 2026-08-26T14:21:32Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM krb1004.eqiad.wmnet - bking@cumin2003" 2451805 wikitext text/x-wiki 32vtlgmb09s9shq3ifk02ktu0br62m4 2451806 2451805 2026-08-26T14:22:02Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2451806 wikitext text/x-wiki o1xbhoqus7rqchj0tvf4hn39hkk5psq 2451807 2451806 2026-08-26T14:23:05Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM krb1004.eqiad.wmnet - bking@cumin2003" 2451807 wikitext text/x-wiki iscfzjezivfd5gomlijrtxvma9r1mh4 2451808 2451807 2026-08-26T14:23:09Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2451808 wikitext text/x-wiki r5pe4sc8y8ygt81r4ywansv5yweetg6 2451809 2451808 2026-08-26T14:23:12Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.dns.wipe-cache krb1004.eqiad.wmnet on all recursors 2451809 wikitext text/x-wiki 7rwuk8p9jv0f5yggovaykiclk8g9pgc 2451810 2451809 2026-08-26T14:23:16Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2451810 wikitext text/x-wiki oifneq2y7ab2273rpbushrwqzyj5u2f 2451811 2451810 2026-08-26T14:23:19Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) krb1004.eqiad.wmnet on all recursors 2451811 wikitext text/x-wiki pjk5zqrblbljk119gr4sbmn7nt9cu45 2451812 2451811 2026-08-26T14:23:43Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM krb1004.eqiad.wmnet - bking@cumin2003" 2451812 wikitext text/x-wiki cebsq75ju3c88n5dlzbbduyrq6zauwz 2451813 2451812 2026-08-26T14:23:47Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM krb1004.eqiad.wmnet - bking@cumin2003" 2451813 wikitext text/x-wiki kt2na2sp1epng8wt6990421ouljtahl 2451814 2451813 2026-08-26T14:25:28Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.decommission 2451814 wikitext text/x-wiki 4l6i944aqb87079625nmx4z8h85a3th 2451815 2451814 2026-08-26T14:25:32Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.decommission for hosts db-test2001.codfw.wmnet 2451815 wikitext text/x-wiki aver9qxnap1xw5roap54suamztp1l6j 2451816 2451815 2026-08-26T14:25:59Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host krb1004.eqiad.wmnet with OS bookworm 2451816 wikitext text/x-wiki r0lcwwbbjb2x385ldr5856qfe8y09m3 2451817 2451816 2026-08-26T14:27:06Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.addnode for new host ganeti3005.esams.wmnet to cluster esams03 and group B 2451817 wikitext text/x-wiki nxx8919ndf8x3gf9nip4adntw2kg67y 2451818 2451817 2026-08-26T14:30:04Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.addnode (exit_code=0) for new host ganeti3005.esams.wmnet to cluster esams03 and group B 2451818 wikitext text/x-wiki ja1rzg189ebw9ndnsvtmx270ezp0wjp 2451819 2451818 2026-08-26T14:30:24Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2451819 wikitext text/x-wiki hlj0ihh5kmxy5a49wte0usxtdt3n7uo 2451820 2451819 2026-08-26T14:33:26Z Stashbot 7414 ecarg@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2451820 wikitext text/x-wiki plqzonzqzqcq4o99dbkt22511697ws2 2451821 2451820 2026-08-26T14:36:31Z Stashbot 7414 bking@cumin2003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host clouddumps1002.wikimedia.org with OS trixie 2451821 wikitext text/x-wiki p2mjpbhxhmvg5p7d9emeb2igg37nmy6 2451822 2451821 2026-08-26T14:36:52Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.reimage for host clouddumps1002.wikimedia.org with OS bookworm 2451822 wikitext text/x-wiki 9602023ogbz99e5ai576l9tw4v0r9xc 2451823 2451822 2026-08-26T14:38:19Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on krb1004.eqiad.wmnet with reason: host reimage 2451823 wikitext text/x-wiki hz8tiwlypr48bwyeekpfswh6pdae9g8 2451824 2451823 2026-08-26T14:39:23Z Stashbot 7414 ecarg@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2451824 wikitext text/x-wiki f000106bl7taxkgltr0ptov7ajix9y5 2451825 2451824 2026-08-26T14:40:13Z Stashbot 7414 ecarg@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2451825 wikitext text/x-wiki 9hgp6jp4qt1r7aqsa6nj74gcfcn39z2 2451826 2451825 2026-08-26T14:42:05Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db-test2001.codfw.wmnet decommissioned, removing all IPs except the asset tag one - fceratto@cumin1003" 2451826 wikitext text/x-wiki rr1xczrn3ohgwx79ahocghw1txcho4g 2451827 2451826 2026-08-26T14:42:28Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db-test2001.codfw.wmnet decommissioned, removing all IPs except the asset tag one - fceratto@cumin1003" 2451827 wikitext text/x-wiki ayxrjozj1tpn4xl6qlq4olg95vyxpr8 2451828 2451827 2026-08-26T14:42:31Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2451828 wikitext text/x-wiki kwezg7xwvx46r1ixd449ucf85tzq4wm 2451829 2451828 2026-08-26T14:42:33Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db-test2001.codfw.wmnet 2451829 wikitext text/x-wiki s9dvrz2icpdry9bcqjplphpj94j37dl 2451830 2451829 2026-08-26T14:45:01Z Stashbot 7414 bking@cumin2003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on krb1004.eqiad.wmnet with reason: host reimage 2451830 wikitext text/x-wiki asdjkwr7c1mgyunohwyh5uy1zznkhxe 2451832 2451830 2026-08-26T14:46:27Z Stashbot 7414 jelto@cumin1003: START - Cookbook sre.hosts.reboot-multiple Sequential unattended reboot of 1 host(s) [['A:owner-collaboration-services', 'P{etherpad2*}'], os=bookworm] 2451832 wikitext text/x-wiki elbi2zd4l6ya7e01tbfj4ejgc4zl704 2451833 2451832 2026-08-26T14:47:00Z Stashbot 7414 jelto@cumin1003: END (FAIL) - Cookbook sre.hosts.reboot-multiple (exit_code=99) Sequential unattended reboot of 1 host(s) [['A:owner-collaboration-services', 'P{etherpad2*}'], os=bookworm] 2451833 wikitext text/x-wiki 4czd0ntzigwxfhji8zzyb9ywqqdfhz5 2451834 2451833 2026-08-26T14:48:22Z Stashbot 7414 brouberol@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/aux-k8s-services/kafka-mirrormaker: apply 2451834 wikitext text/x-wiki 8o9yu8ex8pc3odtb40t63brher8b802 2451835 2451834 2026-08-26T14:48:31Z Stashbot 7414 brouberol@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/aux-k8s-services/kafka-mirrormaker: apply 2451835 wikitext text/x-wiki qg5oaoega0a4hh9yppqzlmh1hp7b2lq 2451838 2451835 2026-08-26T14:53:05Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on clouddumps1002.wikimedia.org with reason: host reimage 2451838 wikitext text/x-wiki qbhkryvlbpao4mw1o0zpg6bomsu0j24 2451840 2451838 2026-08-26T14:58:11Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddumps1002.wikimedia.org with reason: host reimage 2451840 wikitext text/x-wiki qgbd9slbcq7719b2o2d29hcva8lxn6r 2451842 2451840 2026-08-26T15:02:27Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host krb1004.eqiad.wmnet with OS bookworm 2451842 wikitext text/x-wiki lil38en6tg49gctumnmtbpiaohnguqs 2451843 2451842 2026-08-26T15:02:31Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host krb1004.eqiad.wmnet 2451843 wikitext text/x-wiki 10aaj1wurste0cs75qwmn4furj56v7t 2451844 2451843 2026-08-26T15:10:48Z Stashbot 7414 fceratto@cumin1003: Removing db-test2001 from zarcillo T435912 2451844 wikitext text/x-wiki 1r6kr0yhxte6x47r9vns2an69jceuhb 2451845 2451844 2026-08-26T15:10:52Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.mysql.decommission (exit_code=0) 2451845 wikitext text/x-wiki 80umh3lrbo6039rmt6b5ywun7gdg95d 2451849 2451845 2026-08-26T15:20:54Z Stashbot 7414 cdobbins@cumin1003: conftool action : set/pooled=no; selector: name=dns1004.* [reason: trixie upgrade] 2451849 wikitext text/x-wiki 7jdrufshtbp23idglmxsiz4xql6sbfg 2451850 2451849 2026-08-26T15:21:15Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.hosts.reimage for host dns1004.wikimedia.org with OS trixie 2451850 wikitext text/x-wiki 9j4cmszrt959bfypxeuuowiktenrkzl 2451852 2451850 2026-08-26T15:24:51Z Stashbot 7414 mpostoronca: Deployed patch for T432848 2451852 wikitext text/x-wiki i49tgmuuc2x1nz6gpgvccqu6m6h6t4j 2451853 2451852 2026-08-26T15:25:10Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.netbox 2451853 wikitext text/x-wiki hv98gqescxbima4a6tqui311a9ll7q0 2451854 2451853 2026-08-26T15:27:36Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2451854 wikitext text/x-wiki 8f29gdnqsvuuwgl7zn2rew1c6xzufkd 2451856 2451854 2026-08-26T15:28:11Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.netbox 2451856 wikitext text/x-wiki e5xfqjuy8ke4ool9y8xfq2aroiwholn 2451858 2451856 2026-08-26T15:31:14Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2451858 wikitext text/x-wiki rlcptz1okakeh7a47dt5jrd26p2i81q 2451860 2451858 2026-08-26T15:33:07Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: fix dns entries for cr2-eqiad transport interface IPs - cmooney@cumin1003" 2451860 wikitext text/x-wiki 9ckxu4i4phxxr5bx8jzqcym9y7mak68 2451861 2451860 2026-08-26T15:33:11Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: fix dns entries for cr2-eqiad transport interface IPs - cmooney@cumin1003" 2451861 wikitext text/x-wiki c1d4j34v06fxjkfjl8t9n5hb2f58gg9 2451862 2451861 2026-08-26T15:33:13Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2451862 wikitext text/x-wiki k0pwjdm79odku293xz1kgi9xqa10n2k 2451863 2451862 2026-08-26T15:33:38Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply 2451863 wikitext text/x-wiki 85n7mq92cogudt145c5ckesty00lztg 2451864 2451863 2026-08-26T15:35:19Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.netbox 2451864 wikitext text/x-wiki 3z49sbubeefkwx5hnzamhlsuhbkkow4 2451867 2451864 2026-08-26T15:38:00Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on dns1004.wikimedia.org with reason: host reimage 2451867 wikitext text/x-wiki 640vqmbe8l0raoewcjn7ngj41bmab61 2451872 2451867 2026-08-26T15:41:23Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: fix dns entries for cr2-eqiad transport interface IPs - cmooney@cumin1003" 2451872 wikitext text/x-wiki d0pa8g83ntkyvm7sq1phe2lz0f9hvr7 2451875 2451872 2026-08-26T15:44:18Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dns1004.wikimedia.org with reason: host reimage 2451875 wikitext text/x-wiki fd5eg6ff0z4x4p4c2fjv45yk04yk5r6 2451876 2451875 2026-08-26T15:44:44Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: fix dns entries for cr2-eqiad transport interface IPs - cmooney@cumin1003" 2451876 wikitext text/x-wiki nbbjo9ek6gi3inapsjuauy6vx14x6rx 2451877 2451876 2026-08-26T15:44:47Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2451877 wikitext text/x-wiki qughoaecd2nb6j2o1waexjarn0pv4bk 2451881 2451877 2026-08-26T16:18:45Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dns1004.wikimedia.org with OS trixie 2451881 wikitext text/x-wiki 5ok5j88ca4sbzohcxk678hr71wmdl5z 2451883 2451881 2026-08-26T16:20:26Z Stashbot 7414 cdobbins@cumin1003: conftool action : set/pooled=yes; selector: name=dns1004.* [reason: trixie upgrade] 2451883 wikitext text/x-wiki b0yn6leq9wkf827adwv2qjtm5fvygfn 2451884 2451883 2026-08-26T16:23:49Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.metamonitoring.downtime Downtime for 4:00:00 of prometheus/deadmanswitchnotified, prometheus/deadmanswitchonamdb, prometheus/extmon on 2 host(s) with reason: eqsin site rebuild 2451884 wikitext text/x-wiki 9u53liay0ubwfdtdk5tiuyitos94btp 2451885 2451884 2026-08-26T16:23:55Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.metamonitoring.downtime (exit_code=0) Downtime for 4:00:00 of prometheus/deadmanswitchnotified, prometheus/deadmanswitchonamdb, prometheus/extmon on 2 host(s) with reason: eqsin site rebuild 2451885 wikitext text/x-wiki mzz6ec272foyzzzmp2zct9tqe82wr1b 2451887 2451885 2026-08-26T16:24:40Z Stashbot 7414 cmooney@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on 12 hosts with reason: site migration to new Nokia switches 2451887 wikitext text/x-wiki 58fgj6l1z5dieek37051ke1iax0cem2 2451889 2451887 2026-08-26T16:25:12Z Stashbot 7414 cmooney@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on 13 hosts with reason: site migration to new Nokia switches 2451889 wikitext text/x-wiki kczpg91rz4aj4wexbgn393y1swl9yo2 2451890 2451889 2026-08-26T16:25:48Z Stashbot 7414 cmooney@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on 12 hosts with reason: site migration to new Nokia switches 2451890 wikitext text/x-wiki kvjhi43zkm997ily4f3nbxjfg1qozal 2451900 2451890 2026-08-26T16:47:24Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.netbox 2451900 wikitext text/x-wiki o3vjvfkqfkaqfeqdhzi57u8kxlcnsd1 2451905 2451900 2026-08-26T16:57:54Z Stashbot 7414 cmooney@dns3003: START - running authdns-update 2451905 wikitext text/x-wiki 6vv42oi0mfvxkv6dwpb1eijjtpaq5fj 2451906 2451905 2026-08-26T16:58:09Z Stashbot 7414 cdobbins@cumin1003: conftool action : set/pooled=no; selector: name=dns1005.* [reason: trixie upgrade] 2451906 wikitext text/x-wiki h4bm3hj4h3bcmcta18adq9rtvtdvetq 2451907 2451906 2026-08-26T16:58:33Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.hosts.reimage for host dns1005.wikimedia.org with OS trixie 2451907 wikitext text/x-wiki e04bfmnb7v0in3lauubqih5usk85hel 2451909 2451907 2026-08-26T17:00:22Z Stashbot 7414 cmooney@dns3003: FAIL - running authdns-update 2451909 wikitext text/x-wiki 6um4xgyavpt95a2zmrpa348d3npay56 2451911 2451909 2026-08-26T17:02:07Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: fix dns entries for cr2-eqiad transport interface IPs - cmooney@cumin1003" 2451911 wikitext text/x-wiki 6nphtf0ve5ggvejmhwinbfbps9xjknb 2451912 2451911 2026-08-26T17:02:29Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: fix dns entries for cr2-eqiad transport interface IPs - cmooney@cumin1003" 2451912 wikitext text/x-wiki 71optqz8bqsc0ppfpoerdif5a3oekw9 2451913 2451912 2026-08-26T17:02:32Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2451913 wikitext text/x-wiki nbgnzuld11en4h59r43mplrfznu975n 2451916 2451913 2026-08-26T17:11:21Z Stashbot 7414 kamila@deploy1003: kamila: Backport for [[gerrit:1328667|Add title-case mapping to support migration to PHP 8.5 (T432985)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2451916 wikitext text/x-wiki lu1nvnqfeab887ca1vmq127vsfai2hz 2451921 2451916 2026-08-26T17:14:54Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on dns1005.wikimedia.org with reason: host reimage 2451921 wikitext text/x-wiki dueosvxojsuhm6ardubvia1e40h2jk5 2451923 2451921 2026-08-26T17:16:01Z Stashbot 7414 kamila@deploy1003: kamila: Continuing with deployment 2451923 wikitext text/x-wiki sujw751axavwbaw9nxna66630k1k90q 2451928 2451923 2026-08-26T17:18:13Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dns1005.wikimedia.org with reason: host reimage 2451928 wikitext text/x-wiki dx21oqkr1g7xrj3af2s32ainu9cpeau 2451930 2451928 2026-08-26T17:20:23Z Stashbot 7414 kamila@deploy1003: Finished scap sync-world: Backport for [[gerrit:1328667|Add title-case mapping to support migration to PHP 8.5 (T432985)]] (duration: 11m 38s) 2451930 wikitext text/x-wiki jl4hqsaz3hxn7oa5sdy7javbx27mw4x 2451931 2451930 2026-08-26T17:22:13Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddumps1002.wikimedia.org with OS bookworm 2451931 wikitext text/x-wiki 9ynp5m3bz1u07tnks2mrirmsvs1i0mg 2451932 2451931 2026-08-26T17:22:18Z Stashbot 7414 taavi@cumin1003: conftool action : set/pooled=yes; selector: name=clouddumps1002.wikimedia.org,service=dumps-(rsync|https) 2451932 wikitext text/x-wiki f2q24dksftsbknh2lku43i0c15euh7p 2451939 2451932 2026-08-26T17:39:22Z Stashbot 7414 jasmine@dns2004: START - running authdns-update 2451939 wikitext text/x-wiki sv59it2bt74a78xx7fsge5p87sznebb 2451942 2451939 2026-08-26T17:41:55Z Stashbot 7414 jasmine@dns2004: END - running authdns-update 2451942 wikitext text/x-wiki 7k6pohxh3imx8uctmo11xmt0myd4igo 2451943 2451942 2026-08-26T17:42:50Z Stashbot 7414 jasmine@dns2004: START - running authdns-update 2451943 wikitext text/x-wiki hpghrelzl70p84eunhuyhh7r18ix2tx 2451944 2451943 2026-08-26T17:45:15Z Stashbot 7414 jasmine@dns2004: END - running authdns-update 2451944 wikitext text/x-wiki tf1nk00772bpe28xf8btmgbl9b618de 2451945 2451944 2026-08-26T17:46:38Z Stashbot 7414 catrope@deploy1003: Started scap sync-world: Backport for [[gerrit:1329624|Log failures to generate recovery codes in the account creation email (T436066)]] 2451945 wikitext text/x-wiki e4c3e2v4bjr5xpzyfo9qy14smk5hjht 2451947 2451945 2026-08-26T17:49:13Z Stashbot 7414 catrope@deploy1003: catrope: Backport for [[gerrit:1329624|Log failures to generate recovery codes in the account creation email (T436066)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2451947 wikitext text/x-wiki fskf70omrx1vn9y5h6jpba5ipkygjr0 2451948 2451947 2026-08-26T17:49:53Z Stashbot 7414 catrope@deploy1003: catrope: Continuing with deployment 2451948 wikitext text/x-wiki svxc8hs6s86pcznnngdhe0eow6jux9m 2451949 2451948 2026-08-26T17:54:14Z Stashbot 7414 catrope@deploy1003: Finished scap sync-world: Backport for [[gerrit:1329624|Log failures to generate recovery codes in the account creation email (T436066)]] (duration: 07m 35s) 2451949 wikitext text/x-wiki 2q2x3g0x7bgytyvg3s2v8hnv3scyb6g 2451951 2451949 2026-08-26T18:05:15Z Stashbot 7414 kamila@deploy1003: Started scap sync-world: Switch mw-debug/next to 8.5 - T432989 2451951 wikitext text/x-wiki j6u7t8cxbxehbnspf4hadch4rbsj1uu 2451955 2451951 2026-08-26T18:10:58Z Stashbot 7414 kamila@deploy1003: Finished scap sync-world: Switch mw-debug/next to 8.5 - T432989 (duration: 06m 00s) 2451955 wikitext text/x-wiki mlvd975uuknbek13s0adm66kfz9hqcv 2451956 2451955 2026-08-26T18:17:02Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dns1005.wikimedia.org with OS trixie 2451956 wikitext text/x-wiki qnof54meazvbaze7tkvi65dah6n8956 2451958 2451956 2026-08-26T18:26:11Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.remove-downtime for dns1005.wikimedia.org 2451958 wikitext text/x-wiki lkfnqddeir0tib0b6yh9cp2nk7yiyk5 2451959 2451958 2026-08-26T18:26:14Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for dns1005.wikimedia.org 2451959 wikitext text/x-wiki acrfiqhr6wqscqe5zasxyctmpabnmnm 2451960 2451959 2026-08-26T18:26:38Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=yes; selector: name=dns1005.wikimedia.org [reason: trixie upgrade completed] 2451960 wikitext text/x-wiki ts88c22dsdko9npj156zilqell957im 2451961 2451960 2026-08-26T18:27:03Z Stashbot 7414 sukhe@dns1004: START - running authdns-update 2451961 wikitext text/x-wiki 4uf2ypa2aw49orrl8x4eb82zwveklym 2451962 2451961 2026-08-26T18:29:20Z Stashbot 7414 sukhe@dns1004: END - running authdns-update 2451962 wikitext text/x-wiki c3j63bq5w0psd85a84ayxgjcgwwp9fm 2451965 2451962 2026-08-26T18:58:39Z Stashbot 7414 jasmine_: "homer 'lsw1-d8-codfw*' commit "enabled BGP on new wikikube-ctrl2006 host - T406596" 2451965 wikitext text/x-wiki 1zys8uwb72trmudxlm8i8h1ikbko8t1 2451966 2451965 2026-08-26T18:59:11Z Stashbot 7414 jasmine@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-ctrl2006.codfw.wmnet 2451966 wikitext text/x-wiki hsyxu6do2saz01icgzsghlg66st0xm0 2451967 2451966 2026-08-26T18:59:14Z Stashbot 7414 jasmine@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-ctrl2006.codfw.wmnet 2451967 wikitext text/x-wiki razgmdap3thx6uy64ai3tbnwm5anv2p 2451968 2451967 2026-08-26T19:44:56Z Stashbot 7414 dancy@deploy1003: Started deploy [gerrit/gerrit@637dbd8]: Deploying https://gerrit.wikimedia.org/r/c/operations/software/gerrit/+/1329648 2451968 wikitext text/x-wiki k4a12510u6gv73pmdg5wroe2e5we6ym 2451969 2451968 2026-08-26T19:45:10Z Stashbot 7414 dancy@deploy1003: Finished deploy [gerrit/gerrit@637dbd8]: Deploying https://gerrit.wikimedia.org/r/c/operations/software/gerrit/+/1329648 (duration: 00m 14s) 2451969 wikitext text/x-wiki 2zn5xfbv7p2virgxyhetpc8g87teioh 2451970 2451969 2026-08-26T19:45:25Z Stashbot 7414 brett@cumin2003: START - Cookbook sre.dns.admin DNS admin: pool eqsin [reason: no reason specified, no task ID specified] 2451970 wikitext text/x-wiki 6ckr3ih5oq4ar0gxorkh993ytv13jjh 2451971 2451970 2026-08-26T19:45:48Z Stashbot 7414 brett@cumin2003: END (FAIL) - Cookbook sre.dns.admin (exit_code=99) DNS admin: pool eqsin [reason: no reason specified, no task ID specified] 2451971 wikitext text/x-wiki rmtm3ekdmkmgh8lfesd8p46cytjweqy 2451972 2451971 2026-08-26T19:46:41Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=dns5.* 2451972 wikitext text/x-wiki oro3efi8b2k359fjvjn77r2r0f6meyv 2451973 2451972 2026-08-26T19:49:56Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.loadbalancer.upgrade restart A:liberica-eqsin and A:liberica 2451973 wikitext text/x-wiki e5otezqfo7u0h0ep5tqndvjvzu9vr86 2451974 2451973 2026-08-26T19:50:00Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.loadbalancer.admin depooling P{lvs5004.eqsin.wmnet} and A:liberica 2451974 wikitext text/x-wiki se9mjxngyux7epha4op7j0avjnpmv7f 2451975 2451974 2026-08-26T19:50:15Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) depooling P{lvs5004.eqsin.wmnet} and A:liberica 2451975 wikitext text/x-wiki 90fmuu5s7l2o470uynr7v3l8qj2scm3 2451976 2451975 2026-08-26T19:50:38Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.loadbalancer.admin pooling P{lvs5004.eqsin.wmnet} and A:liberica 2451976 wikitext text/x-wiki nb1mb1aautyiv4sd63u8hperoiv0a6d 2451977 2451976 2026-08-26T19:51:03Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) pooling P{lvs5004.eqsin.wmnet} and A:liberica 2451977 wikitext text/x-wiki hka6ral7urpq97uuyqog1vhjqaa1kcc 2451978 2451977 2026-08-26T19:51:09Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.loadbalancer.admin depooling P{lvs5005.eqsin.wmnet} and A:liberica 2451978 wikitext text/x-wiki d5w494p0i2v0gdbauuxp9ahg2qb87zx 2451979 2451978 2026-08-26T19:52:55Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) depooling P{lvs5005.eqsin.wmnet} and A:liberica 2451979 wikitext text/x-wiki refno0c3zoyrzebww4i80i2u561as2q 2451980 2451979 2026-08-26T19:53:18Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.loadbalancer.admin pooling P{lvs5005.eqsin.wmnet} and A:liberica 2451980 wikitext text/x-wiki 6vr3ncjfc6yh3s1g6f5s8nu813us2vv 2451981 2451980 2026-08-26T19:53:42Z Stashbot 7414 sukhe@dns1004: START - running authdns-update 2451981 wikitext text/x-wiki 0z0ubm4js70siqbf9l5huqwokq1wnva 2451982 2451981 2026-08-26T19:53:45Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) pooling P{lvs5005.eqsin.wmnet} and A:liberica 2451982 wikitext text/x-wiki e5fy2dhvjsc8r6vy6y4q4ukz5txrcpb 2451983 2451982 2026-08-26T19:53:50Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.loadbalancer.admin depooling P{lvs5006.eqsin.wmnet} and A:liberica 2451983 wikitext text/x-wiki l1r82aq1vaby141fc1e83rg6me6zxea 2451984 2451983 2026-08-26T19:54:06Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) depooling P{lvs5006.eqsin.wmnet} and A:liberica 2451984 wikitext text/x-wiki pn1v9y6rtll8ee4kicqrcoofmbsy70t 2451985 2451984 2026-08-26T19:54:28Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.loadbalancer.admin pooling P{lvs5006.eqsin.wmnet} and A:liberica 2451985 wikitext text/x-wiki a48xh84w1tx5yuac3nna7wtyip9h5u8 2451986 2451985 2026-08-26T19:54:57Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) pooling P{lvs5006.eqsin.wmnet} and A:liberica 2451986 wikitext text/x-wiki at8q8asq950sk3btv38zqmplek6xlts 2451987 2451986 2026-08-26T19:55:00Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.loadbalancer.upgrade (exit_code=0) restart A:liberica-eqsin and A:liberica 2451987 wikitext text/x-wiki 6ge0aj9hsooc6veuxvfbo2mwiybh3cy 2451988 2451987 2026-08-26T19:55:58Z Stashbot 7414 sukhe@dns1004: END - running authdns-update 2451988 wikitext text/x-wiki q6d2gvzkazns9i7ph6jsd73m9jvqxuz 2451989 2451988 2026-08-26T19:58:39Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=no; selector: name=cp5022.eqsin.wmnet 2451989 wikitext text/x-wiki czhpr0a3qqt2jlu9s19drjn39azb6ge 2451990 2451989 2026-08-26T20:06:24Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.dns.roll-restart-reboot-wikimedia-dns rolling restart_daemons on A:wikidough-eqsin 2451990 wikitext text/x-wiki efvi9lb6jz3xbbzh73qm9o3km9tundt 2451991 2451990 2026-08-26T20:08:20Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.dns.roll-restart-reboot-wikimedia-dns (exit_code=0) rolling restart_daemons on A:wikidough-eqsin 2451991 wikitext text/x-wiki lmas5e9b15fzu22v50hlgtitkgcl91r 2451992 2451991 2026-08-26T20:10:32Z Stashbot 7414 brett@cumin2003: START - Cookbook sre.dns.admin DNS admin: pool eqsin [reason: no reason specified, T435406] 2451992 wikitext text/x-wiki k6oavmrld4dhlswqvet216duptnt0s9 2451993 2451992 2026-08-26T20:10:38Z Stashbot 7414 brett@cumin2003: END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: pool eqsin [reason: no reason specified, T435406] 2451993 wikitext text/x-wiki djvkynxh0ipcqqiswros2n6sdizpllj 2451994 2451993 2026-08-26T20:24:21Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reimage for host cassandra-dev2002.codfw.wmnet with OS bookworm 2451994 wikitext text/x-wiki bgxmw9nxu3virre9384b94duhoarygk 2451995 2451994 2026-08-26T20:27:10Z Stashbot 7414 eevans@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cassandra-dev2002.codfw.wmnet with OS bookworm 2451995 wikitext text/x-wiki he88astiovrpwc2hohnzebwekvphqk9 2451996 2451995 2026-08-26T20:28:13Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.deploy.python-code homer to cumin2003.codfw.wmnet,cumin1003.eqiad.wmnet with reason: Homer - remove eqsin workaround from wmf-plugin for eqsin L3 switches - cmooney@cumin1003 2451996 wikitext text/x-wiki i5rxlrk0rmwsr6cpnnypil0jiaivkwo 2451997 2451996 2026-08-26T20:29:51Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2003.codfw.wmnet,cumin1003.eqiad.wmnet with reason: Homer - remove eqsin workaround from wmf-plugin for eqsin L3 switches - cmooney@cumin1003 2451997 wikitext text/x-wiki 085eu4gog8veswo57wnsm8qmipte6re 2451998 2451997 2026-08-26T20:48:34Z Stashbot 7414 cdobbins@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on cp5022.eqsin.wmnet with reason: needs repair 2451998 wikitext text/x-wiki qg0sb7vsl6nvy40jgqcvfbz6uwfnlo1 2451999 2451998 2026-08-26T20:57:24Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reimage for host cassandra-dev2002.codfw.wmnet with OS bookworm 2451999 wikitext text/x-wiki o1ky6achu67vj9jwkmmg4kw9ypy1t2t 2452000 2451999 2026-08-26T21:02:35Z Stashbot 7414 aaron@deploy1003: Started scap sync-world: Backport for [[gerrit:1329616|Rest: Rename "mode" key to "availability" in RestModuleOverrides (T433522)]] 2452000 wikitext text/x-wiki ppr9eas37fs2bh5s8tyyjkh5t2ejxth 2452001 2452000 2026-08-26T21:07:21Z Stashbot 7414 aaron@deploy1003: aaron: Backport for [[gerrit:1329616|Rest: Rename "mode" key to "availability" in RestModuleOverrides (T433522)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2452001 wikitext text/x-wiki 4dfx53sd0u15ngjvfleu44jipddzxn9 2452002 2452001 2026-08-26T21:09:20Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2452002 wikitext text/x-wiki 9ryzk7za3xvhykvigfbiy7gpxm73f44 2452003 2452002 2026-08-26T21:10:16Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2452003 wikitext text/x-wiki fcu2koez7euksd8z25b03sz5hbae7gx 2452004 2452003 2026-08-26T21:10:27Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2452004 wikitext text/x-wiki nezo8z3u7v36zyij2u1wi3dk5coq4ss 2452005 2452004 2026-08-26T21:10:56Z Stashbot 7414 aaron@deploy1003: aaron: Continuing with deployment 2452005 wikitext text/x-wiki begonkcjbwstw0433o0nhd8j9xsi1l5 2452006 2452005 2026-08-26T21:12:29Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2452006 wikitext text/x-wiki 9o26e59wj4uyqymbhbbjcc0pkzgfc2f 2452007 2452006 2026-08-26T21:12:46Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2452007 wikitext text/x-wiki rg3nhb2l6l85byzqmvi28l3moz3gcys 2452008 2452007 2026-08-26T21:14:17Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2452008 wikitext text/x-wiki qr4ysuvm078vhowznd9anvzyspk4i5y 2452009 2452008 2026-08-26T21:14:31Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2452009 wikitext text/x-wiki 59mfo0n3ak0g2gcyouqjaep0o9iw32j 2452010 2452009 2026-08-26T21:14:38Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2452010 wikitext text/x-wiki nppxrfyleazc69yy1gji1zngnjuu20x 2452011 2452010 2026-08-26T21:15:18Z Stashbot 7414 aaron@deploy1003: Finished scap sync-world: Backport for [[gerrit:1329616|Rest: Rename "mode" key to "availability" in RestModuleOverrides (T433522)]] (duration: 12m 43s) 2452011 wikitext text/x-wiki 6tf9tcp19sij4hl4bd2yq45cc78fn0j 2452012 2452011 2026-08-26T21:19:00Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2452012 wikitext text/x-wiki na37xqzyp0ge8dnb2xuelueu245u374 2452013 2452012 2026-08-26T21:19:18Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2452013 wikitext text/x-wiki 1cjc039let8kgcs0h4pgfddcua2y9c8 2452014 2452013 2026-08-26T21:19:29Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2452014 wikitext text/x-wiki 5p2wbbyo9ad8zfo1ugt82kzpb0o7lbc 2452015 2452014 2026-08-26T21:20:08Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2452015 wikitext text/x-wiki 9acnpziupd8n9gki24bfdq0vxn9774n 2452016 2452015 2026-08-26T21:20:15Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2452016 wikitext text/x-wiki 1ujt5jron8enq3bpg72aw1f4jvqls11 2452017 2452016 2026-08-26T21:20:48Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2452017 wikitext text/x-wiki fps763008kkzdqjx1pi6ega0keb7k1o 2452019 2452017 2026-08-26T21:31:00Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2452019 wikitext text/x-wiki ct1a855ifykp0he79685uhn9156i9ds 2452020 2452019 2026-08-26T21:31:19Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2452020 wikitext text/x-wiki kbiph3ptpvl9u2fxvprkm67hmeizkuf 2452022 2452020 2026-08-26T21:34:59Z Stashbot 7414 eevans@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cassandra-dev2002.codfw.wmnet with OS bookworm 2452022 wikitext text/x-wiki 1uz2tp6hsn235v9sqirl0be5ixwtsoj 2452023 2452022 2026-08-26T21:35:18Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reimage for host cassandra-dev2002.codfw.wmnet with OS bookworm 2452023 wikitext text/x-wiki ovmhhsfex6nv86h85tu55eic8w1yyrf 2452028 2452023 2026-08-26T21:47:20Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2452028 wikitext text/x-wiki jcnygdxf5ydt9bclwlx4m77rb7ytc8m 2452029 2452028 2026-08-26T21:48:08Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2452029 wikitext text/x-wiki 6w253vct829s9vkk78lulsu646hq37n 2452030 2452029 2026-08-26T21:50:48Z Stashbot 7414 jasmine@cumin1003: START - Cookbook sre.k8s.renumber-node Renumbering for host wikikube-worker1260.eqiad.wmnet 2452030 wikitext text/x-wiki 67el0i33t87y3cjgf7hdxz6iiqf3umm 2452031 2452030 2026-08-26T21:50:57Z Stashbot 7414 jasmine@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1260.eqiad.wmnet 2452031 wikitext text/x-wiki 1vaxiaf0pivq7l1uavvtrlhig8cp38a 2452032 2452031 2026-08-26T21:51:23Z Stashbot 7414 jasmine@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1260.eqiad.wmnet 2452032 wikitext text/x-wiki t4tuemk1b3oqt6ve04fu8pec94o4bnc 2452033 2452032 2026-08-26T21:51:54Z Stashbot 7414 jasmine@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-worker1260.eqiad.wmnet with OS trixie 2452033 wikitext text/x-wiki nrdroea6ts2i1plbz3nrmxmxuul93r2 2452034 2452033 2026-08-26T21:52:22Z Stashbot 7414 jasmine@cumin1003: START - Cookbook sre.hosts.move-vlan for host wikikube-worker1260 2452034 wikitext text/x-wiki ecf1l1elid7q2ckh6hf4kabeamsjhcy 2452035 2452034 2026-08-26T21:52:53Z Stashbot 7414 jasmine@cumin1003: START - Cookbook sre.dns.netbox 2452035 wikitext text/x-wiki g5ewprwkctcqrob642148t59942acmg 2452037 2452035 2026-08-26T21:55:11Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2452037 wikitext text/x-wiki jorc8g8x0t5avlj5vl3n2liez8s2g79 2452038 2452037 2026-08-26T21:55:51Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2452038 wikitext text/x-wiki od6fgkrl4mri8ccuzi20u2f7n3i6zg5 2452039 2452038 2026-08-26T21:58:11Z Stashbot 7414 jasmine@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1260 - jasmine@cumin1003" 2452039 wikitext text/x-wiki 8b2d95tde30l8861wtbkbir19wfwb54 2452040 2452039 2026-08-26T21:58:15Z Stashbot 7414 jasmine@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1260 - jasmine@cumin1003" 2452040 wikitext text/x-wiki t5dt8lzxd0b7vilh362kmna0w5np8vi 2452041 2452040 2026-08-26T21:58:19Z Stashbot 7414 jasmine@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2452041 wikitext text/x-wiki bqatf6gd0gmjo4w1spbjqmclgwn8xtn 2452042 2452041 2026-08-26T21:58:23Z Stashbot 7414 jasmine@cumin1003: START - Cookbook sre.dns.wipe-cache wikikube-worker1260.eqiad.wmnet 28.32.64.10.in-addr.arpa 8.2.0.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2452042 wikitext text/x-wiki tobi2tvlc84pr5sjtb9cjtqwssepudq 2452043 2452042 2026-08-26T21:58:26Z Stashbot 7414 jasmine@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker1260.eqiad.wmnet 28.32.64.10.in-addr.arpa 8.2.0.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2452043 wikitext text/x-wiki 4pmz4m157chdzif9ubale8ui97qe7tv 2452044 2452043 2026-08-26T21:58:30Z Stashbot 7414 jasmine@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker1260 2452044 wikitext text/x-wiki pkevadphnlx0vblceb180negdqqohsd 2452045 2452044 2026-08-26T21:59:01Z Stashbot 7414 jasmine@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker1260 2452045 wikitext text/x-wiki 87de6f14qvdkt7tzeykgfudbg418r4x 2452046 2452045 2026-08-26T21:59:04Z Stashbot 7414 jasmine@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker1260 2452046 wikitext text/x-wiki 0yssw52qr45x3c7aq2bsqx98350pxrh 2452047 2452046 2026-08-26T22:00:18Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2452047 wikitext text/x-wiki od16hiodmhk0tljxna4usiucngddkx8 2452048 2452047 2026-08-26T22:00:29Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2452048 wikitext text/x-wiki g0qyt9hqj75or16dy52ustzd9koc994 2452049 2452048 2026-08-26T22:00:59Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2452049 wikitext text/x-wiki gt0hwgbvbz6b832ir4z74svckhw01pv 2452050 2452049 2026-08-26T22:01:39Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2452050 wikitext text/x-wiki 6fzn9h8i2yua9wredojb0672qb6vogi 2452051 2452050 2026-08-26T22:01:52Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2452051 wikitext text/x-wiki qtvghbue3200afquwab9y7hhm1o0iln 2452052 2452051 2026-08-26T22:02:27Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2452052 wikitext text/x-wiki jofi167omt7o85zbtxkyxcwlq8fwerw 2452056 2452052 2026-08-26T22:20:18Z Stashbot 7414 jasmine@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1260.eqiad.wmnet with reason: host reimage 2452056 wikitext text/x-wiki nv6muw3mty8pf8fpfzwzqltcnk5niw0 2452057 2452056 2026-08-26T22:23:40Z Stashbot 7414 jasmine@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1260.eqiad.wmnet with reason: host reimage 2452057 wikitext text/x-wiki qd1prysbyx4rr6jbxpichcfafa31mgr 2452058 2452057 2026-08-26T22:24:38Z Stashbot 7414 lerickson@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2452058 wikitext text/x-wiki lbvva6pcfab170q2t4666p9np3tctew 2452059 2452058 2026-08-26T22:58:16Z Stashbot 7414 jasmine@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1260.eqiad.wmnet with OS trixie 2452059 wikitext text/x-wiki 1q7pd1vvqk3oxym9l4ma3j74ganb4qj 2452060 2452059 2026-08-26T23:10:52Z Stashbot 7414 jasmine@cumin1003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1260.eqiad.wmnet 2452060 wikitext text/x-wiki e88xujj9croffxace1w6gwrjbb7wod9 2452061 2452060 2026-08-26T23:10:55Z Stashbot 7414 jasmine@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1260.eqiad.wmnet 2452061 wikitext text/x-wiki awwvkbpr7jp43kgrdtulmkct9w786tx 2452062 2452061 2026-08-26T23:10:57Z Stashbot 7414 jasmine@cumin1003: END (FAIL) - Cookbook sre.k8s.renumber-node (exit_code=1) Renumbering for host wikikube-worker1260.eqiad.wmnet 2452062 wikitext text/x-wiki 6uyzmvqw4ng198mmpu6xqg0y04x337c 2452064 2452062 2026-08-26T23:34:42Z Stashbot 7414 catrope@deploy1003: Started scap sync-world: Backport for [[gerrit:1329692|Use IDBAccessObject::READ_LATEST in some cases (T436066)]] 2452064 wikitext text/x-wiki nwzhyvcgvtkle78au8osfs4dpgft0q1 2452065 2452064 2026-08-26T23:37:16Z Stashbot 7414 eevans@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cassandra-dev2002.codfw.wmnet with OS bookworm 2452065 wikitext text/x-wiki 5r507y7dlmke34s2pleotugh50v5hur 2452066 2452065 2026-08-26T23:39:46Z Stashbot 7414 catrope@deploy1003: catrope: Backport for [[gerrit:1329692|Use IDBAccessObject::READ_LATEST in some cases (T436066)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2452066 wikitext text/x-wiki 7hqn5dgm1r1c29s3am812ync385d6n2 2452067 2452066 2026-08-26T23:40:47Z Stashbot 7414 catrope@deploy1003: catrope: Continuing with deployment 2452067 wikitext text/x-wiki d2ixnan43adxsmh18ts2ad3ne13t729 2452068 2452067 2026-08-26T23:45:16Z Stashbot 7414 catrope@deploy1003: Finished scap sync-world: Backport for [[gerrit:1329692|Use IDBAccessObject::READ_LATEST in some cases (T436066)]] (duration: 10m 33s) 2452068 wikitext text/x-wiki nz9iusbsfn0tfgij37afwyck4mg8i0y 2452070 2452068 2026-08-27T00:18:24Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1329317|Allow linking to thumb.wikimedia.org in TemplateStyles (T435979)]] 2452070 wikitext text/x-wiki cbho5iyd5bpjn7wrprax5imxhnomk39 2452071 2452070 2026-08-27T00:22:52Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1329317|Allow linking to thumb.wikimedia.org in TemplateStyles (T435979)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2452071 wikitext text/x-wiki epxvz80qcf28zgqwye3hvt4j8675wl2 2452072 2452071 2026-08-27T00:32:47Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2452072 wikitext text/x-wiki jfunhizsuh50xzb5nixrpl3zdsoetpl 2452073 2452072 2026-08-27T00:37:07Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1329317|Allow linking to thumb.wikimedia.org in TemplateStyles (T435979)]] (duration: 18m 42s) 2452073 wikitext text/x-wiki sngmzj490rsbzfyu00opjper7768wby 2452076 2452073 2026-08-27T01:55:17Z Stashbot 7414 dmartin@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2452076 wikitext text/x-wiki 0n9qv4y60rxvdv38jwfordozn2bjouu 2452077 2452076 2026-08-27T01:57:53Z Stashbot 7414 dmartin@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2452077 wikitext text/x-wiki 3zn99wty85hm83c0byhxvong2a95uld 2452078 2452077 2026-08-27T02:00:42Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2452078 wikitext text/x-wiki 0ep5e46rnfdj0h01bipz93j1njlwb95 2452079 2452078 2026-08-27T02:08:43Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 08m 00s) 2452079 wikitext text/x-wiki 0p1td338ol5zuj93bcafk348cxk5y9m 2452081 2452079 2026-08-27T03:31:10Z Stashbot 7414 tstarling@deploy1003: Started scap sync-world: Backport for [[gerrit:1324965|Enable Produnto on testwiki (T421436)]] 2452081 wikitext text/x-wiki kow8l7n3ornofi9j8pfsey88czzxprw 2452082 2452081 2026-08-27T03:35:54Z Stashbot 7414 tstarling@deploy1003: tstarling: Backport for [[gerrit:1324965|Enable Produnto on testwiki (T421436)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2452082 wikitext text/x-wiki bq9s99f4vejkuhz1cypl11lng3tyvx8 2452083 2452082 2026-08-27T04:06:32Z Stashbot 7414 tstarling@deploy1003: tstarling: Continuing with deployment 2452083 wikitext text/x-wiki 4898t0qrlpd6cff856xkbgz07x3zg3f 2452084 2452083 2026-08-27T04:11:48Z Stashbot 7414 tstarling@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324965|Enable Produnto on testwiki (T421436)]] (duration: 40m 38s) 2452084 wikitext text/x-wiki je8iay9y2vzf8ha4syfal7el6u4v4ay 2452088 2452084 2026-08-27T07:21:58Z Stashbot 7414 wmde-fisch@deploy1003: Started scap sync-world: Backport for [[gerrit:1328590|Add feature flag to beta cluster and test wiki (T431544)]] 2452088 wikitext text/x-wiki 0lk2mjlqd0l8zgrji20tq3gnakr547n 2452089 2452088 2026-08-27T07:26:55Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch, mareikeheuer: Backport for [[gerrit:1328590|Add feature flag to beta cluster and test wiki (T431544)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2452089 wikitext text/x-wiki 8prmbrz9fr6o99o818p75dmzrny346k 2452090 2452089 2026-08-27T07:29:41Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch, mareikeheuer: Continuing with deployment 2452090 wikitext text/x-wiki 8g3flrquzw9y9t5nurz08to3s9lb1is 2452091 2452090 2026-08-27T07:34:05Z Stashbot 7414 wmde-fisch@deploy1003: Finished scap sync-world: Backport for [[gerrit:1328590|Add feature flag to beta cluster and test wiki (T431544)]] (duration: 12m 07s) 2452091 wikitext text/x-wiki dlo14ypsgilyk3w4lnkln74w7nnngfu 2452092 2452091 2026-08-27T08:11:19Z Stashbot 7414 filippo@cumin1003: conftool action : set/pooled=yes; selector: service=dumps-nfs,name=clouddumps1002.wikimedia.org 2452092 wikitext text/x-wiki cpvaxihbe0qjrxfm696a8p9pv6elr5l 2452093 2452092 2026-08-27T08:12:18Z Stashbot 7414 hashar@deploy1003: rebuilt and synchronized wikiversions files: group2 to 1.47.0-wmf.17 refs T430836 2452093 wikitext text/x-wiki 82o54reutcbrf7pxc9yaufdgniz7bz4 2452094 2452093 2026-08-27T08:27:52Z Stashbot 7414 gmodena@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2452094 wikitext text/x-wiki m0i6yu9jcg1c60ypk54eyzc7q2ylrml 2452100 2452094 2026-08-27T08:35:20Z Stashbot 7414 gmodena@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs-next: apply 2452100 wikitext text/x-wiki t2g82hj9s0agz9iufeqhr67e4bevczl 2452106 2452100 2026-08-27T10:09:22Z Stashbot 7414 blake@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-videoscaler: apply 2452106 wikitext text/x-wiki kh2o9o7jx0qijax7ks3mv60t4pevhlh 2452107 2452106 2026-08-27T10:12:38Z Stashbot 7414 blake@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-videoscaler: apply 2452107 wikitext text/x-wiki 5mi761t134qq4lqbsna28l7cfn54umo 2452108 2452107 2026-08-27T10:12:44Z Stashbot 7414 blake@deploy1003: helmfile [codfw] START helmfile.d/services/mw-videoscaler: apply 2452108 wikitext text/x-wiki 2blzxftu2le5j0ajt7lq7p29q24p4mk 2452109 2452108 2026-08-27T10:12:54Z Stashbot 7414 blake@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-videoscaler: apply 2452109 wikitext text/x-wiki 8606nq8qrnbn7mvpp5hwd2xclueb2nw 498 9925 2451953 2451589 2026-08-26T18:07:06Z Stashbot 7414 andrewbogott: resolving rebase conflicts in /srv/git/labs/private 2451953 wikitext text/x-wiki 551b3b1bcqvlmacjbdm6n0t209norz5 0 17290 2451831 2451610 2026-08-26T14:45:33Z Stashbot 7414 hashar: Updating Castor (0.4.3..0.4.5) on all Jenkins jobs | https://gerrit.wikimedia.org/r/1329577 2451831 wikitext text/x-wiki fdq8vojuvzor5ff8g10g4s41e1cq0u5 2451933 2451831 2026-08-26T17:24:28Z Stashbot 7414 dancy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/1329617 2451933 wikitext text/x-wiki 7rejoize8azwzi8ogudor5eu82w371c 2451952 2451933 2026-08-26T18:07:06Z Stashbot 7414 andrewbogott: resolving rebase conflicts in /srv/git/labs/private 2451952 wikitext text/x-wiki 2a3s28ydr6vv2a6av95cfh1jyv6t20u 0 20832 2451786 2430932 2026-08-26T13:44:17Z MLechvien-WMF 49197 Updating Serviceops intake process 2451786 wikitext text/x-wiki 3bcell510xdj3p9lfdzihbzlxp4os5v 0 23401 2452103 2300461 2026-08-27T10:00:41Z BJensen-WMF 48942 Typo fix. 2452103 wikitext text/x-wiki 34g0cfci9830jh37wbefgq51sh4c9oh 0 441912 2452101 2431169 2026-08-27T08:36:50Z Lucas Werkmeister (WMDE) 10037 /* Scripts */ kafkacat→kcat; also add -C to silence “auto-selecting Consumer mode” message 2452101 wikitext text/x-wiki 7mybn2d4auchjre2inx9p019zv3ggyg 0 444427 2451963 2451123 2026-08-26T18:52:09Z BKing (WMF) 29017 /* Current setup */ 2451963 wikitext text/x-wiki pe6jth6t5a51a44ajf0gtl0vqtf8594 2451964 2451963 2026-08-26T18:53:37Z BKing (WMF) 29017 /* Current setup */ 2451964 wikitext text/x-wiki l9mqwk7nyipqohify1ufjak6yb34rp3 498 444707 2451954 2440543 2026-08-26T18:07:50Z Stashbot 7414 andrewbogott: resolving rebase conflicts in /srv/git/labs/private 2451954 wikitext text/x-wiki 8rhruxs7ikz4ame710ugtr61myqeevl 0 446705 2452125 2425534 2026-08-27T11:22:58Z JMeybohm (WMF) 16709 /* Packaging */ Add instructions regarding the versioned kubernetes-client package 2452125 wikitext text/x-wiki 8ot5b0ubnis9jjy7b20i6vxkl2rq404 2452126 2452125 2026-08-27T11:35:47Z JMeybohm (WMF) 16709 /* Packaging */ 2452126 wikitext text/x-wiki su9oz87153jj3iohdg6pggnc8glhpyk 2452127 2452126 2026-08-27T11:43:29Z JMeybohm (WMF) 16709 /* Packaging */ 2452127 wikitext text/x-wiki 38xccfjbluza8fnzo4yute93avasq0o 0 446764 2451940 2451494 2026-08-26T17:40:31Z XCollazo-WMF 31846 /* Hosts and their roles */ Updating as per site.pp from puppet. 2451940 wikitext text/x-wiki a934iq8nv4vnf9ynpewt28o226h075j 2451941 2451940 2026-08-26T17:40:54Z XCollazo-WMF 31846 /* UI quicklinks */ Hue is long gone 2451941 wikitext text/x-wiki rnf75xicwz1d1kux0oh4cby1d80ardz 2451946 2451941 2026-08-26T17:49:13Z XCollazo-WMF 31846 Removing Oozie section althogether as that system is long gone. 2451946 wikitext text/x-wiki 6vq7hilsssui6urkinj9evbbl1anq7j 0 448386 2451841 2447717 2026-08-26T14:59:25Z JWodstrcil (WMF) 24826 fix typo in aux section 2451841 wikitext text/x-wiki rw3zvzjcy1xik54t50wfunsczugue0r 2 449053 2451873 2273830 2026-08-26T15:41:29Z TBurmeister (WMF) 27417 Add category 2451873 wikitext text/x-wiki 8avpkt6u4igt422l2p6lz2c3ifrltmh 0 449160 2452069 2451595 2026-08-27T00:01:46Z Dexbot 30554 Bot: Updating the report 2452069 wikitext text/x-wiki 4h98cmkdpc1a5tog7bequjf0x8ir2qs 13 449208 2452123 2326925 2026-08-27T11:10:21Z Saper 135 /* Warning: The running version of job 'XXX' is different from what was configured, please recreate or redeploy. */ new section 2452123 wikitext text/x-wiki 7s4u1iijreo2u2ttzqbnyr964zx1eii 2452124 2452123 2026-08-27T11:10:40Z Saper 135 /* Warning: The running version of job 'XXX' is different from what was configured, please recreate or redeploy. */ 2452124 wikitext text/x-wiki m51levi8xo2ny5uknwodcb2o5sqngrr 0 450684 2452021 2437717 2026-08-26T21:31:55Z AMastilovic-WMF 40067 /* Datasets config option */ 2452021 wikitext text/x-wiki jwh5zgw8macq7tfy9vxsrefeododqk9 498 452556 2451957 2195091 2026-08-26T18:18:41Z Stashbot 7414 andrewbogott: resolving rebase conflicts in /srv/git/operations/puppet on puppetserver-01.civicrm-prototype.eqiad1.wikimedia.cloud 2451957 wikitext text/x-wiki kzm7t8sh20b4io80bz7hrb88apzaok3 0 453101 2452036 2413592 2026-08-26T21:54:59Z DWisehaupt (WMF) 14911 Add infrastructure section for deps 2452036 wikitext text/x-wiki 0wgbsuehl3ofbbv1risj782twuqvm6f 116 453906 2452027 2451525 2026-08-26T21:42:35Z Gitlabaccountapprovalbot 37332 @mmns21 was approved. 2452027 wikitext text/x-wiki qcb7o29g81g6h41rj437kntad2xthj5 2452080 2452027 2026-08-27T02:18:38Z Gitlabaccountapprovalbot 37332 @madmax was approved. 2452080 wikitext text/x-wiki mcl7w7pgw3ojdicwf09mnfan65bs9df 0 454144 2451934 2451607 2026-08-26T17:31:26Z BKing (WMF) 29017 2451934 wikitext text/x-wiki 9iusd0a3jbc34auw301820e9wce4gi8 2451935 2451934 2026-08-26T17:32:43Z BKing (WMF) 29017 /* Reimaging */ 2451935 wikitext text/x-wiki 2op9ab5cfybm359kpe62534q1mhdn5b 2451936 2451935 2026-08-26T17:34:18Z Lucas Werkmeister 8093 /* Reimaging */ fix closing tag 2451936 wikitext text/x-wiki i4ia8e1f009kjaxmln1otqkw4s95yr0 2451937 2451936 2026-08-26T17:36:02Z Lucas Werkmeister 8093 /* Reimaging */ rearrange the wikitext so the syntaxhighlight and subsequent text stays inside the list item (IMHO the output is nicer this way, but the wikitext is rather uglier, so feel free to revert) 2451937 wikitext text/x-wiki rcllfr2luqcf50numbij40ai34h0y7l 2451938 2451937 2026-08-26T17:38:08Z BKing (WMF) 29017 /* Reimaging */ 2451938 wikitext text/x-wiki ie5o30j2jzhx3n9dzhuzh6upx4wx1ci 1 454185 2452096 2368123 2026-08-27T08:34:24Z MABot 10804 Bot: Archiving 1 thread from [[Talk:Main Page]] 2452096 wikitext text/x-wiki 6hpqz3mz9aifpdnzuly1z59iryba755 0 454372 2451757 2406126 2026-08-26T12:12:49Z Taavi-WMF 41365 update 2451757 wikitext text/x-wiki oj4gmqawt5t3mp8tgxjk7rs8ca1syc9 0 455989 2451893 2450561 2026-08-26T16:41:23Z RScout-WMF 56866 Small bug 2451893 wikitext text/x-wiki ssdd2mkuu46rd6ec6ge0qpwy1cpf8cw 2452024 2451893 2026-08-26T21:37:23Z SFaci-WMF 35637 /* Exposure logging */ 2452024 wikitext text/x-wiki 1dmoj0ny57kq2639rzros7q4tyi9foz 0 456039 2452025 2419923 2026-08-26T21:38:39Z SFaci-WMF 35637 /* Server-side instrumentation */ 2452025 wikitext text/x-wiki pv1m04ubk7sz2hem9lfy0844errxt3h 2452026 2452025 2026-08-26T21:38:52Z SFaci-WMF 35637 /* Server side */ 2452026 wikitext text/x-wiki ej8rvfhzjwipjikny1fp75dew9hgr64 0 458704 2451758 2296407 2026-08-26T12:13:54Z Taavi-WMF 41365 s/nginx/haproxy/g 2451758 wikitext text/x-wiki fin4tpf97wzdpd1xnq9nqfdd5d26tsj 0 458705 2451759 2296380 2026-08-26T12:15:07Z Taavi-WMF 41365 s/nginx/haproxy/g 2451759 wikitext text/x-wiki d94e6al6tozeppcj4i7bskuw82vkxlp 0 459697 2451869 2447748 2026-08-26T15:39:03Z TBurmeister (WMF) 27417 Add category 2451869 wikitext text/x-wiki 2g4zyf6vpm3m0thlwebqzp9frwmvbs8 0 459699 2451920 2451599 2026-08-26T17:13:36Z Stashbot 7414 dwisehaupt: started staging db refresh on frdb1006 2451920 wikitext text/x-wiki k67fx9zel0wbwkj70nl0y9trlbjxqmc 2452063 2451920 2026-08-26T23:18:00Z Stashbot 7414 larssandergreen: civicrm upgraded from 92c276d6 to 06ede270 2452063 wikitext text/x-wiki j47ffckewucb0brh373vyovmtzercmp 0 460222 2451871 2444694 2026-08-26T15:40:00Z TBurmeister (WMF) 27417 Add category 2451871 wikitext text/x-wiki opoywpyluggpteemwmkizzemqev0mf7 0 460223 2451866 2433213 2026-08-26T15:37:58Z TBurmeister (WMF) 27417 Add category 2451866 wikitext text/x-wiki envk4aqofvw7nc1mq0rsvo3i5fmd1f0 0 460366 2451879 2451188 2026-08-26T15:53:55Z TBurmeister (WMF) 27417 Add link to markdown file for GAS 2451879 wikitext text/x-wiki m8zdwkrmgmmqkx68zh16y5h1tfy0xgt 2451880 2451879 2026-08-26T15:57:12Z TBurmeister (WMF) 27417 More links 2451880 wikitext text/x-wiki g9vcf7nzazdcdm3pqu5r8c0xxyx32e7 0 460375 2451865 2451507 2026-08-26T15:35:37Z TBurmeister (WMF) 27417 Update links 2451865 wikitext text/x-wiki ml34qtyhk9o01po0slnq5l5sryymq3k 0 460538 2451870 2448741 2026-08-26T15:39:26Z TBurmeister (WMF) 27417 Add category 2451870 wikitext text/x-wiki 52xgyrdcj59re7bho5z7p7ci4pz24uu 0 460582 2451914 2451506 2026-08-26T17:04:16Z TBurmeister (WMF) 27417 /* wikibase:isSomeValue() */ Add link to mw page on blank node skolemization 2451914 wikitext text/x-wiki 3w4fcft472ua08dhbpaj58jak1i2mvf 2 460586 2451851 2451199 2026-08-26T15:23:50Z SSapaty (WMF) 28955 2451851 wikitext text/x-wiki 3f1xe6uu4u51dmach2x6bn56jyynoib 0 460590 2452018 2026-08-26T21:30:43Z AMastilovic-WMF 40067 Airflow datasets config guide 2452018 wikitext text/x-wiki 24kbv8rq65j2zw7b2h4hq4k5jbw4mad 0 460591 2452053 2026-08-26T22:09:19Z DWisehaupt (WMF) 14911 Initial page for fr-tech dependencies on production. 2452053 wikitext text/x-wiki ocf779yese2bzkxorvj4zmje9txfxk0 2452054 2452053 2026-08-26T22:10:01Z DWisehaupt (WMF) 14911 add community crm mail dep 2452054 wikitext text/x-wiki 96iusdd4vhosfpyx63fqr3drlo13tpj 2452055 2452054 2026-08-26T22:12:28Z DWisehaupt (WMF) 14911 add development deps 2452055 wikitext text/x-wiki 65t873yhzaebl6v7yrpdr5hlbncgnn1 2452074 2452055 2026-08-27T01:00:15Z DWisehaupt (WMF) 14911 add some more deps 2452074 wikitext text/x-wiki h4oi5c4uih3hc6yu222m50skfxki5m3 2452075 2452074 2026-08-27T01:01:20Z DWisehaupt (WMF) 14911 fix header level and add another dep 2452075 wikitext text/x-wiki ahbzsl07ftaw8gopm6gtvq7y0avohe3 0 460593 2452099 2026-08-27T08:35:20Z Lucas Werkmeister (WMDE) 10037 create redirect to match [[kafkacat]] 2452099 wikitext text/x-wiki 8v9z09w77hy2cv0k4gu84fvkglwu2pr 3 460594 2452128 2026-08-27T11:56:35Z StrikerBot 8475 Welcome to Toolforge! 2452128 wikitext text/x-wiki 1pccgb5d9xyrcaex92jrdtgzkzxb1dz 3 460595 2452129 2026-08-27T11:58:10Z StrikerBot 8475 Welcome to Toolforge! 2452129 wikitext text/x-wiki m7emqc1j3s0wve6uevzbq0r6fhkcqet