Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.19 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 0 4108 2456176 2455994 2026-09-09T19:41:16Z ScheduleDeploymentBot 37566 Add [[gerrit:1338278]] to Wednesday, September 09 UTC late backport window 2456176 wikitext text/x-wiki 7ytq99otex6gswe6q4dbf9gsqloo98v 2456179 2456176 2026-09-09T19:42:00Z ScheduleDeploymentBot 37566 Add [[gerrit:1338279]] to Wednesday, September 09 UTC late backport window 2456179 wikitext text/x-wiki 7ox03yu187hoicgakg4q9vm4k4h1ytv 2456180 2456179 2026-09-09T19:42:32Z ScheduleDeploymentBot 37566 Add [[gerrit:1338280]] to Wednesday, September 09 UTC late backport window 2456180 wikitext text/x-wiki notqgokntxrl7r3ob896fngsg50atzk 2456182 2456180 2026-09-09T19:42:46Z ScheduleDeploymentBot 37566 Add [[gerrit:1338281]] to Wednesday, September 09 UTC late backport window 2456182 wikitext text/x-wiki mgjj8x1ngwvxgpiqe6cbhqhkwcy5ou8 2456183 2456182 2026-09-09T19:43:00Z ScheduleDeploymentBot 37566 Add [[gerrit:1338282]] to Wednesday, September 09 UTC late backport window 2456183 wikitext text/x-wiki cybbn3lmc6c5e4hyn0zin38jv9raw4c 2456184 2456183 2026-09-09T19:43:10Z ScheduleDeploymentBot 37566 Add [[gerrit:1338283]] to Wednesday, September 09 UTC late backport window 2456184 wikitext text/x-wiki sh5dec5fgnfaxifo3yd1on3kkjz5dz0 0 7919 2456006 2456001 2026-09-09T12:16:21Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2456006 wikitext text/x-wiki lb3e5elkk1gz216hdujwrr1x8reoywz 2456007 2456006 2026-09-09T12:16:37Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2456007 wikitext text/x-wiki omzcqc4x2cvr7ih865kv56w3hapxgn6 2456008 2456007 2026-09-09T12:31:41Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1338151|core-Namespaces.php: Disallow indexing on talk namespaces on ukwiki (T437409)]] 2456008 wikitext text/x-wiki fk30a7djsxhqwxxeydzl6yl1nnq0194 2456009 2456008 2026-09-09T12:35:58Z Stashbot 7414 ladsgroup@deploy1003: tryvix1509, ladsgroup: Backport for [[gerrit:1338151|core-Namespaces.php: Disallow indexing on talk namespaces on ukwiki (T437409)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2456009 wikitext text/x-wiki l5d30kzstpjeengy99wxyhbwlfqrz71 2456010 2456009 2026-09-09T12:41:41Z Stashbot 7414 ladsgroup@deploy1003: tryvix1509, ladsgroup: Continuing with deployment 2456010 wikitext text/x-wiki 0tyvpcuib6i7h8zbcloxuc8tf6o6h37 2456011 2456010 2026-09-09T12:46:21Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1338151|core-Namespaces.php: Disallow indexing on talk namespaces on ukwiki (T437409)]] (duration: 14m 39s) 2456011 wikitext text/x-wiki es3y60fs4yrucd9ndg23llx643s52k5 2456012 2456011 2026-09-09T12:46:53Z Stashbot 7414 btullis@deploy1003: helmfile [ml-serve-codfw] START helmfile.d/admin 'apply'. 2456012 wikitext text/x-wiki t8y9mnkdb5kp3uayqxg2tkpa6udpkw2 2456013 2456012 2026-09-09T12:48:21Z Stashbot 7414 btullis@deploy1003: helmfile [ml-serve-codfw] DONE helmfile.d/admin 'apply'. 2456013 wikitext text/x-wiki ovcgfgnu838lrt8dvsgba5govx23yu1 2456015 2456013 2026-09-09T12:49:38Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.reimage for host cloudcephosd1055.eqiad.wmnet with OS trixie 2456015 wikitext text/x-wiki itkqhj8no6gza9pbqsl9tvg9pc6b0yr 2456018 2456015 2026-09-09T13:02:17Z Stashbot 7414 jmm@cumin2003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on ldap-rw[1001,2001].wikimedia.org with reason: work in progress 2456018 wikitext text/x-wiki jd8abhihcgvq0809bcg7tn7538uf7xg 2456019 2456018 2026-09-09T13:04:59Z Stashbot 7414 sbisson@deploy1003: Started scap sync-world: Backport for [[gerrit:1334945|ArticleGuidance: Add the redirect configuration keys (T434487)]], [[gerrit:1337983|Replace experiment with instrument and config-driven redirect (T434487)]] 2456019 wikitext text/x-wiki q4ilo0l14ut27d7anxrdcgfjbi8pg8t 2456020 2456019 2026-09-09T13:09:15Z Stashbot 7414 sbisson@deploy1003: sbisson: Backport for [[gerrit:1334945|ArticleGuidance: Add the redirect configuration keys (T434487)]], [[gerrit:1337983|Replace experiment with instrument and config-driven redirect (T434487)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2456020 wikitext text/x-wiki nzeyu5j2aj1kga5isabcqiiabsj5l3c 2456021 2456020 2026-09-09T13:10:14Z Stashbot 7414 sbisson@deploy1003: sbisson: Continuing with deployment 2456021 wikitext text/x-wiki 2u5o712xnqxzcl0a8o9j7x2dvtkx4nq 2456022 2456021 2026-09-09T13:11:24Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reimage for host sretest2013.codfw.wmnet with OS trixie 2456022 wikitext text/x-wiki 3j2yjuc7ziflybv38rwvy2igdgwbn5n 2456023 2456022 2026-09-09T13:14:35Z Stashbot 7414 elukey@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudcephosd1055.eqiad.wmnet with OS trixie 2456023 wikitext text/x-wiki bz6oj7tmwkdt89748vmf20pomlf6drj 2456024 2456023 2026-09-09T13:15:14Z Stashbot 7414 sbisson@deploy1003: Finished scap sync-world: Backport for [[gerrit:1334945|ArticleGuidance: Add the redirect configuration keys (T434487)]], [[gerrit:1337983|Replace experiment with instrument and config-driven redirect (T434487)]] (duration: 10m 15s) 2456024 wikitext text/x-wiki igolfj695b9xbekp9nlf27r5gqkyz5x 2456025 2456024 2026-09-09T13:17:11Z Stashbot 7414 cmooney@cumin1004: START - Cookbook sre.network.tls for network device lsw1-a4-eqiad 2456025 wikitext text/x-wiki tvtqt6qca8m9g4zobqrxsps04n7pkus 2456026 2456025 2026-09-09T13:17:46Z Stashbot 7414 cmooney@cumin1004: END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device lsw1-a4-eqiad 2456026 wikitext text/x-wiki qu143xxfs1zoguq8qflj7jo97uwc470 2456028 2456026 2026-09-09T13:22:10Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2456028 wikitext text/x-wiki gyhgfsi6g6t2p4yp54xf75wlh5l0y4v 2456029 2456028 2026-09-09T13:22:19Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2456029 wikitext text/x-wiki b6x00tckfmc0qdgekssqwikk2terj6j 2456030 2456029 2026-09-09T13:24:53Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on sretest2013.codfw.wmnet with reason: host reimage 2456030 wikitext text/x-wiki 79r0ovnih5hxraq2d0yioc3oe9apr94 2456031 2456030 2026-09-09T13:25:43Z Stashbot 7414 cmooney@cumin1004: START - Cookbook sre.network.tls for network device lsw1-b7-eqiad 2456031 wikitext text/x-wiki ixzojk5z4jud5aorizuh1e1jt8vfcr0 2456032 2456031 2026-09-09T13:26:08Z Stashbot 7414 cmooney@cumin1004: END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device lsw1-b7-eqiad 2456032 wikitext text/x-wiki 3hvn6dw24lcx1ljwei3og6rl238t6y2 2456033 2456032 2026-09-09T13:28:30Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on sretest2013.codfw.wmnet with reason: host reimage 2456033 wikitext text/x-wiki ghck73hn8m5d70jqjpa8r02fd4po6h3 2456034 2456033 2026-09-09T13:29:20Z Stashbot 7414 moritzm: pruned obsolete Bullseye image dispatch from the docker registry T416452 2456034 wikitext text/x-wiki 8ij1rgfhxwawq0lsv69dbm9waqanf0e 2456035 2456034 2026-09-09T13:42:37Z Stashbot 7414 btullis@deploy1003: helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. 2456035 wikitext text/x-wiki 1jskyipebae8g6dg1a106hyc2ydt3gx 2456036 2456035 2026-09-09T13:43:24Z Stashbot 7414 btullis@deploy1003: helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. 2456036 wikitext text/x-wiki tow0qljieqvk73zgtavnn946bquqc88 2456039 2456036 2026-09-09T13:53:34Z Stashbot 7414 btullis@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'apply'. 2456039 wikitext text/x-wiki h9brpliwprz5tmkfw8sf803yadlkyob 2456040 2456039 2026-09-09T13:55:04Z Stashbot 7414 btullis@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'apply'. 2456040 wikitext text/x-wiki 0je8kuelqatk7ljkn7a50pigzirj9wk 2456044 2456040 2026-09-09T14:06:24Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - sukhe@cumin1003" 2456044 wikitext text/x-wiki 4xz6vfqmke1iwnoigei2r7sxzmj9810 2456045 2456044 2026-09-09T14:07:14Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - sukhe@cumin1003" 2456045 wikitext text/x-wiki ophsdqoevts4tjgrby275pg02qarjby 2456046 2456045 2026-09-09T14:07:15Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host sretest2013.codfw.wmnet with OS trixie 2456046 wikitext text/x-wiki jwvkutkb4w74g4kxd9ytfslwpj2wlap 2456047 2456046 2026-09-09T14:07:16Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2456047 wikitext text/x-wiki j1ljhtv9cxea2yxxbw849miqj7n754r 2456048 2456047 2026-09-09T14:08:04Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2456048 wikitext text/x-wiki mqdpm1l1knpsxq8hx0ow95n71kl0ohu 2456049 2456048 2026-09-09T14:08:22Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2456049 wikitext text/x-wiki t1zcg14irhctdmzgkmp5ldihrxpn015 2456050 2456049 2026-09-09T14:10:20Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2456050 wikitext text/x-wiki syafld3uxjbpa4bllqyghaz4n93ajgx 2456051 2456050 2026-09-09T14:10:30Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2456051 wikitext text/x-wiki 6ac9x6ml0xog0yep7byq9fyuhpmr7qd 2456054 2456051 2026-09-09T14:12:17Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2456054 wikitext text/x-wiki o12cciv1ly6iej4gda9k1ctp74b9n24 2456056 2456054 2026-09-09T14:17:30Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2456056 wikitext text/x-wiki dxct94hgbe1d7ik3i874iw1wrsrf3hj 2456057 2456056 2026-09-09T14:17:47Z Stashbot 7414 jforrester@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2456057 wikitext text/x-wiki c1f5cf4rb98yafvvkfwel3nj91r37z2 2456058 2456057 2026-09-09T14:19:01Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2456058 wikitext text/x-wiki 46ll7yq02vnetve2kjqv8lyp9yw3moz 2456059 2456058 2026-09-09T14:19:41Z Stashbot 7414 jforrester@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2456059 wikitext text/x-wiki htgn5bvkulrs5b224uvy3ts26bi9g87 2456060 2456059 2026-09-09T14:19:48Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2456060 wikitext text/x-wiki 3ezkhqfhp71ho3boc3r9w1c4xtcsqn4 2456061 2456060 2026-09-09T14:20:23Z Stashbot 7414 jforrester@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2456061 wikitext text/x-wiki 959c8pd0nw6fw9j2du5lxpsrtiotci4 2456062 2456061 2026-09-09T14:32:36Z Stashbot 7414 kevinbazira@deploy1003: helmfile [ml-staging-codfw] 'sync' command on namespace 'tool-server' for release 'main' . 2456062 wikitext text/x-wiki c5o9jcqlrpjvlbj43hfzmz5f293aukk 2456063 2456062 2026-09-09T14:34:19Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-jobrunner: apply 2456063 wikitext text/x-wiki gf0j5scocogxx6lvnq842td6ojvhdf0 2456064 2456063 2026-09-09T14:35:03Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-jobrunner: apply 2456064 wikitext text/x-wiki 82lhmivkqubdi7kk1qijab98yao1rbf 2456065 2456064 2026-09-09T14:42:21Z Stashbot 7414 hnowlan: half concurrency for parsoidCachePrewarm RecordLintJob and refreshLinks in jobqueue, eqiad & codfw 2456065 wikitext text/x-wiki 56s4jpnzk0y2leexaz5as8m9p5grgsl 2456066 2456065 2026-09-09T15:01:45Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-jobrunner: apply 2456066 wikitext text/x-wiki 9u2jxndt6t2h1xoc6ogge7lwhmz9gl4 2456067 2456066 2026-09-09T15:04:43Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp6008.drmrs.wmnet 2456067 wikitext text/x-wiki ke081obqqt6y60ztajn2y75pagckaag 2456068 2456067 2026-09-09T15:06:36Z Stashbot 7414 moritzm: installing grub2 bugfix updates from Bookworm point release 2456068 wikitext text/x-wiki rvr20pl1jvdsjx61l7tpz5dfxtly4oa 2456069 2456068 2026-09-09T15:07:35Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-jobrunner: apply 2456069 wikitext text/x-wiki qhmrh2r06i4omqbb8sraf76k70dkgg6 2456070 2456069 2026-09-09T15:07:54Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-jobrunner: apply 2456070 wikitext text/x-wiki khqmh21hc6bxvb9yc6bpy83eh5wh5ku 2456071 2456070 2026-09-09T15:08:04Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-jobrunner: apply 2456071 wikitext text/x-wiki ricn6vyigwzggi7nhiort29ph0w8l3t 2456073 2456071 2026-09-09T15:17:46Z Stashbot 7414 urbanecm: Delete all running periodic jobs starting with `growthexperiments-refreshlinkrecommendations-*` (to pick up new configuration; T392944) 2456073 wikitext text/x-wiki n518c2pjor6jj6qhnij77kxtcbsvw80 2456075 2456073 2026-09-09T15:36:23Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1338213|Disable parsoidCachePrewarm jobs everywhere (T436001 T436206)]] 2456075 wikitext text/x-wiki hnt2vj6lzo513ju95p498qlkwnv375o 2456080 2456075 2026-09-09T15:40:56Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1338213|Disable parsoidCachePrewarm jobs everywhere (T436001 T436206)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2456080 wikitext text/x-wiki 84rzxrplhs2tncpclcrsgzy5ez5uh06 2456081 2456080 2026-09-09T15:41:28Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2456081 wikitext text/x-wiki pcexha3wrwz1bm8bqih93y7ocnnhgdz 2456082 2456081 2026-09-09T15:46:06Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1338213|Disable parsoidCachePrewarm jobs everywhere (T436001 T436206)]] (duration: 09m 43s) 2456082 wikitext text/x-wiki k9dyxd6yahghq0vfh8c3lcmye407k52 2456084 2456082 2026-09-09T15:54:35Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply 2456084 wikitext text/x-wiki 62zz14pbj7bkgeswydgp9ip220z0mg6 2456085 2456084 2026-09-09T15:55:47Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply 2456085 wikitext text/x-wiki 2sox45nt8d77ygz9ecywvagoqelux8q 2456092 2456085 2026-09-09T16:25:16Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: apply 2456092 wikitext text/x-wiki rzur2cvcdnz44fsawxbv9279pdfps5v 2456093 2456092 2026-09-09T16:26:35Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: apply 2456093 wikitext text/x-wiki tkspa6ru28cgyfem89j9ggb5sqiqo18 2456094 2456093 2026-09-09T16:27:00Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply 2456094 wikitext text/x-wiki 4afhod176x8nrj4w05q9ze7v31axaw9 2456095 2456094 2026-09-09T16:27:40Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply 2456095 wikitext text/x-wiki gby7ocz44czrq2vokq4jsjw3j0wng5o 2456096 2456095 2026-09-09T16:28:00Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: apply 2456096 wikitext text/x-wiki 1btyxrr8xnkcnhbu398ffhznuoxx1ez 2456097 2456096 2026-09-09T16:28:40Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: apply 2456097 wikitext text/x-wiki 0un000epq5i9z6njfgtbtdnxr81ab3k 2456100 2456097 2026-09-09T16:36:49Z Stashbot 7414 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1338173|testwiki: allow testing new AccountSetup experiment (T436872)]] 2456100 wikitext text/x-wiki brc2ooncdlnayufg0j24vkbpncrn1ew 2456101 2456100 2026-09-09T16:41:09Z Stashbot 7414 urbanecm@deploy1003: migr, urbanecm: Backport for [[gerrit:1338173|testwiki: allow testing new AccountSetup experiment (T436872)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2456101 wikitext text/x-wiki tep6jbxjyy1we5kmikyp9g9fjmyj73q 2456102 2456101 2026-09-09T16:41:45Z Stashbot 7414 urbanecm@deploy1003: migr, urbanecm: Continuing with deployment 2456102 wikitext text/x-wiki 6xqmsf3c7r8zijgesili2x42vzngxrg 2456103 2456102 2026-09-09T16:46:18Z Stashbot 7414 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1338173|testwiki: allow testing new AccountSetup experiment (T436872)]] (duration: 09m 28s) 2456103 wikitext text/x-wiki izz42l060arr3k51o6rhify9d5b3p6e 2456106 2456103 2026-09-09T17:04:50Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.provision for host aqs1026.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2456106 wikitext text/x-wiki 1e3wudf21vkzodqr7kdw3lu9kci03em 2456108 2456106 2026-09-09T17:12:19Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs1026.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2456108 wikitext text/x-wiki c90nswkbj7bbqfshznfwww8j22fsi3b 2456109 2456108 2026-09-09T17:23:06Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2456109 wikitext text/x-wiki bc2h1zyqly5b8qae1c5zb16bn9gy28u 2456110 2456109 2026-09-09T17:23:14Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2456110 wikitext text/x-wiki 5ri8axrisleensg39l9kqgypjbck4gy 2456111 2456110 2026-09-09T17:27:39Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] START helmfile.d/services/thumbor: apply 2456111 wikitext text/x-wiki bnibyfvkfiffap1muzf4onpjs6logp9 2456112 2456111 2026-09-09T17:29:45Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] DONE helmfile.d/services/thumbor: apply 2456112 wikitext text/x-wiki dcm969sbv8h40fk6m78yo028zvkjpfw 2456113 2456112 2026-09-09T17:31:24Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reimage for host aqs1026.eqiad.wmnet with OS bookworm 2456113 wikitext text/x-wiki eezueo8wtsoyl63yo2nu0rytw0g07re 2456114 2456113 2026-09-09T17:35:55Z Stashbot 7414 eevans@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host aqs1026.eqiad.wmnet with OS bookworm 2456114 wikitext text/x-wiki 31p49q33dp0lp4qfuhw2iklbf2prey9 2456115 2456114 2026-09-09T17:36:10Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reimage for host aqs1026.eqiad.wmnet with OS bookworm 2456115 wikitext text/x-wiki eemwfjj2kqqyqvz7c2z8de16258i9n3 2456119 2456115 2026-09-09T17:45:38Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs1026.eqiad.wmnet with reason: host reimage 2456119 wikitext text/x-wiki ejgfck6a2jn44rtu7t50q4shpv5mu3r 2456120 2456119 2026-09-09T17:49:30Z Stashbot 7414 rzl@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-cron: apply 2456120 wikitext text/x-wiki 46hx8lyrx5017izf1158or1v9oa7ku6 2456121 2456120 2026-09-09T17:49:37Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-product: apply 2456121 wikitext text/x-wiki 0dqsyedjt65qciri5q38raatd5te81t 2456122 2456121 2026-09-09T17:49:40Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1026.eqiad.wmnet with reason: host reimage 2456122 wikitext text/x-wiki kde31df4ox0hbdoqxo2et8y8vdxya6r 2456123 2456122 2026-09-09T17:49:57Z Stashbot 7414 rzl@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-cron: apply 2456123 wikitext text/x-wiki g00e34g189jmpbpzpmdxr44t1b542fd 2456124 2456123 2026-09-09T17:50:11Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-product: apply 2456124 wikitext text/x-wiki 8uj7vrqt49a03p2d72g95k2m7q1bee3 2456125 2456124 2026-09-09T17:50:18Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply 2456125 wikitext text/x-wiki jm8yb0lrjf1em2idx2lb86l3rswb4nn 2456126 2456125 2026-09-09T17:50:55Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply 2456126 wikitext text/x-wiki rbv46a7rbu96m3ff71gmpqekb3jcgy7 2456127 2456126 2026-09-09T17:51:12Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply 2456127 wikitext text/x-wiki 93nw8wzbnrypmmqqxp9rv2v41gl1w1e 2456128 2456127 2026-09-09T17:51:49Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply 2456128 wikitext text/x-wiki 2rx0d3jd5gbipcdt654a0pqgs65r1x1 2456129 2456128 2026-09-09T17:52:05Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-experiment-platform: apply 2456129 wikitext text/x-wiki p7gfgu5rrgg1d001c06m8tkk2bb0nsf 2456130 2456129 2026-09-09T17:52:28Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-experiment-platform: apply 2456130 wikitext text/x-wiki 97en0nj1zgbtr9c0a9lco0ybgip48q7 2456131 2456130 2026-09-09T17:53:24Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-fr-tech: apply 2456131 wikitext text/x-wiki ptectu1ug5b3i487qz56yl8161m4m1a 2456132 2456131 2026-09-09T17:53:49Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-fr-tech: apply 2456132 wikitext text/x-wiki mwkp11jk06juutpqhm7onpdg8fcdrd0 2456133 2456132 2026-09-09T17:54:01Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-main: apply 2456133 wikitext text/x-wiki rp1nigtubaotxl2d6ldv6hgvnpkdxmu 2456134 2456133 2026-09-09T17:54:10Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] START helmfile.d/services/thumbor: apply 2456134 wikitext text/x-wiki 8npnxdf7arvgw6jk9ah3qqpsjkgb2af 2456135 2456134 2026-09-09T17:54:34Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-main: apply 2456135 wikitext text/x-wiki 9t3llirgipwwy23yoov8wje0l3t8x32 2456136 2456135 2026-09-09T17:55:31Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-ml: apply 2456136 wikitext text/x-wiki hy3xnn00g8e7it0ujh8d93x6aglcvtz 2456137 2456136 2026-09-09T17:56:05Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-ml: apply 2456137 wikitext text/x-wiki 2p2vh3yh5jylf7tjhpmfzz7psirnkyd 2456138 2456137 2026-09-09T17:56:16Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] DONE helmfile.d/services/thumbor: apply 2456138 wikitext text/x-wiki 4nlo8ru63ic9awzwf41oa31utob81in 2456139 2456138 2026-09-09T17:56:28Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-platform-eng: apply 2456139 wikitext text/x-wiki dj11hcosxz8ez73w392s6rw3smt5n5n 2456140 2456139 2026-09-09T17:56:56Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-platform-eng: apply 2456140 wikitext text/x-wiki 18muglkxmpm875qe2t57is83pbhqzvq 2456141 2456140 2026-09-09T17:57:05Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-research: apply 2456141 wikitext text/x-wiki e3yv9g11o18j2hd7i80ndfdw18338am 2456142 2456141 2026-09-09T17:57:44Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-research: apply 2456142 wikitext text/x-wiki huc2uwhrtaxnonirvtf842kxogocxa1 2456143 2456142 2026-09-09T17:58:08Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-search: apply 2456143 wikitext text/x-wiki 1xqbk7gehzmt2s1yvw3xbgyxvkdo64a 2456144 2456143 2026-09-09T17:58:41Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-search: apply 2456144 wikitext text/x-wiki 5u2o3wvuvfl9riyvikat0llqcxa356g 2456145 2456144 2026-09-09T18:00:43Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-sre: apply 2456145 wikitext text/x-wiki ksidsxm2da8brvqbx9915uqj6kx3w0z 2456146 2456145 2026-09-09T18:01:25Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-sre: apply 2456146 wikitext text/x-wiki 6q000h6f7a7xllxthbjgagbvrm8lqeo 2456148 2456146 2026-09-09T18:01:51Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2456148 wikitext text/x-wiki 9wjc7u2czrz1j8ja1yrpx89uqds1tqm 2456149 2456148 2026-09-09T18:02:36Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply 2456149 wikitext text/x-wiki tltl6xlto81abjps37o0w3owvlrsrq7 2456150 2456149 2026-09-09T18:02:43Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wikidata: apply 2456150 wikitext text/x-wiki 70e5fcq06momn91fn70fmjcgwmfovdd 2456151 2456150 2026-09-09T18:03:07Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wikidata: apply 2456151 wikitext text/x-wiki 4m7oe898efakbrii9k8ppe4ntt3e0mg 2456152 2456151 2026-09-09T18:03:13Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wmde: apply 2456152 wikitext text/x-wiki t3kt0t2k96r1e1myodp9n7t3kz3f8zg 2456153 2456152 2026-09-09T18:03:48Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wmde: apply 2456153 wikitext text/x-wiki dsib24ac40tus11amwo4z2n6z9hyx4n 2456154 2456153 2026-09-09T18:06:57Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs1026.eqiad.wmnet with OS bookworm 2456154 wikitext text/x-wiki pjztqq41swuhi3br3rn02oka7kh2km9 2456159 2456154 2026-09-09T18:23:52Z Stashbot 7414 dduvall@deploy1003: rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.19 refs T430838 2456159 wikitext text/x-wiki m0fg4az4ftc68rw0kpypvvsl7oal49t 2456161 2456159 2026-09-09T18:59:58Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.dns.netbox 2456161 wikitext text/x-wiki 7vwsb5pf47x1k00vevoyc0wsps5vcpv 2456162 2456161 2026-09-09T19:03:37Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1075] - vriley@cumin1003" 2456162 wikitext text/x-wiki 4ja7lsjomukgta12nbka8bwtaywsfp5 2456163 2456162 2026-09-09T19:04:16Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1075] - vriley@cumin1003" 2456163 wikitext text/x-wiki 30zj10djpe76g5nuvwao9bnhox9j11a 2456164 2456163 2026-09-09T19:04:17Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2456164 wikitext text/x-wiki 8wj9i4b2g7nmbhl9y4hi2x7x644emus 2456165 2456164 2026-09-09T19:05:17Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1075 2456165 wikitext text/x-wiki 8cqlh7ie5q2gdq4vejsf0ewq7wmn74v 2456166 2456165 2026-09-09T19:05:18Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1075 2456166 wikitext text/x-wiki j36trnq0pwm174fulja1ie27745g4q6 2456167 2456166 2026-09-09T19:05:45Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1075.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2456167 wikitext text/x-wiki iea4oguydz2med0g0pjrn93tjbs0036 2456168 2456167 2026-09-09T19:05:57Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1075.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2456168 wikitext text/x-wiki n6linoqyrwyhdzqdzklj3ign4xt4vlu 2456169 2456168 2026-09-09T19:06:08Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1075.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2456169 wikitext text/x-wiki qw0y7djkqiywa3sx7pmlm4of3lt4smh 2456170 2456169 2026-09-09T19:19:19Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1075.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2456170 wikitext text/x-wiki kkrddddfxub320lszkjohl81b1xlnzv 2456171 2456170 2026-09-09T19:19:49Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1075.eqiad.wmnet with OS trixie 2456171 wikitext text/x-wiki 9s7jrik2jau29eoc5e21uwio1lbxt5f 2456172 2456171 2026-09-09T19:26:40Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1075.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2456172 wikitext text/x-wiki 46ax9t6n89l8ebvne4k1lkg1w6858yf 2456173 2456172 2026-09-09T19:28:51Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.provision for host aqs1027.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2456173 wikitext text/x-wiki sqrafvepjflt48lz12i7epc8hm6ixr3 2456174 2456173 2026-09-09T19:35:18Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1075.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2456174 wikitext text/x-wiki e65hn2txazthj7ai2r5gaibrjhquz7k 2456175 2456174 2026-09-09T19:36:17Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs1027.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART 2456175 wikitext text/x-wiki 8dcpupdv7tye9sp3pvd4w0pedoox81m 2456177 2456175 2026-09-09T19:41:34Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reimage for host aqs1027.eqiad.wmnet with OS bookworm 2456177 wikitext text/x-wiki 66shvj0bick92u32divt1vwfu6g73f9 2456187 2456177 2026-09-09T19:51:44Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on aqs1027.eqiad.wmnet with reason: host reimage 2456187 wikitext text/x-wiki ize48s2s851xnfx84semooy8fliqvkf 2456189 2456187 2026-09-09T19:57:50Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1027.eqiad.wmnet with reason: host reimage 2456189 wikitext text/x-wiki ivhk85bb4if198aq52kgnr41wnfhymk 2456196 2456189 2026-09-09T20:15:31Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs1027.eqiad.wmnet with OS bookworm 2456196 wikitext text/x-wiki 8xmian38vpde75ag4pfcblwsdyob90i 2456197 2456196 2026-09-09T20:22:57Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.k8s.renumber-node Renumbering for host wikikube-worker1342.eqiad.wmnet 2456197 wikitext text/x-wiki fr3uawts19vitnwgzdsl8jcctd0qfqs 2456198 2456197 2026-09-09T20:23:01Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1342.eqiad.wmnet 2456198 wikitext text/x-wiki 7w61xiujw8cvn3s3bp0jt8w3yfppg9p 2456199 2456198 2026-09-09T20:23:38Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1342.eqiad.wmnet 2456199 wikitext text/x-wiki e6i386re6m0mig3prhe12u3uczk7c8t 2456200 2456199 2026-09-09T20:23:56Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.hosts.reimage for host wikikube-worker1342.eqiad.wmnet with OS trixie 2456200 wikitext text/x-wiki n5goypxmc5um4ts47okds3coowx3va1 2456201 2456200 2026-09-09T20:24:26Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.hosts.move-vlan for host wikikube-worker1342 2456201 wikitext text/x-wiki lh6vp9shj2h4wyxgev3561jx8e2nw0l 2456202 2456201 2026-09-09T20:24:59Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.dns.netbox 2456202 wikitext text/x-wiki 66kmmy433tmd8hypuec50nu81479j5h 2456203 2456202 2026-09-09T20:28:59Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1342 - rzl@cumin2003" 2456203 wikitext text/x-wiki t154tjtomq31ca9e3vtu1oog54vje1x 2456204 2456203 2026-09-09T20:29:02Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1342 - rzl@cumin2003" 2456204 wikitext text/x-wiki k3husdjko9ug7tidi51tu5tznko62fn 2456205 2456204 2026-09-09T20:29:04Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2456205 wikitext text/x-wiki 7pyg9qcsfix8y4z38lfb13x3xmjarqv 2456206 2456205 2026-09-09T20:29:05Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.dns.wipe-cache wikikube-worker1342.eqiad.wmnet 160.32.64.10.in-addr.arpa 0.6.1.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2456206 wikitext text/x-wiki 0xj7vpojtwm5w7yidr0pkj96p6x0stm 2456207 2456206 2026-09-09T20:29:06Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker1342.eqiad.wmnet 160.32.64.10.in-addr.arpa 0.6.1.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2456207 wikitext text/x-wiki 65j06dil3ndexvleir3kdjbwb95xf5j 2456208 2456207 2026-09-09T20:29:07Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker1342 2456208 wikitext text/x-wiki j3wce57ce7t3c4v82hf5654k0zmjv69 2456209 2456208 2026-09-09T20:29:44Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker1342 2456209 wikitext text/x-wiki tck7jvqga05y6hylatjmjhqal3zfrem 2456210 2456209 2026-09-09T20:29:45Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker1342 2456210 wikitext text/x-wiki pvcq89q9i6cy7fjod5oo93sa710trmh 2456211 2456210 2026-09-09T20:34:41Z Stashbot 7414 sbassett@deploy1003: Started scap sync-world: Backport for [[gerrit:1338280|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338281|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338283|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338282|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338278|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338279|Revert^2 " 2456211 wikitext text/x-wiki 70c5puf6nyc1dzi9h6vmoctwuu9m61o 2456212 2456211 2026-09-09T20:35:23Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1075.eqiad.wmnet with OS trixie 2456212 wikitext text/x-wiki mruvo4n92z2095zjv55ayewfdedgyjs 2456213 2456212 2026-09-09T20:39:07Z Stashbot 7414 sbassett@deploy1003: aranyap, sbassett: Backport for [[gerrit:1338280|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338281|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338283|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338282|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338278|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338279|Revert^2 "Filter 2456213 wikitext text/x-wiki d2upgkr0z2fepzsba4gmlhxpkm3679n 2456214 2456213 2026-09-09T20:40:49Z Stashbot 7414 sbassett@deploy1003: aranyap, sbassett: Continuing with deployment 2456214 wikitext text/x-wiki evg09q8rm575ksapmz7lh2vwjl7yfn2 2456215 2456214 2026-09-09T20:41:51Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts sessionstore2004.codfw.wmnet 2456215 wikitext text/x-wiki 9l9bor3kjdqw3w2rf8hdntioc00rry3 2456216 2456215 2026-09-09T20:42:04Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1342.eqiad.wmnet with reason: host reimage 2456216 wikitext text/x-wiki p0sa4vmkpbw4e6cjc83b28wsbnf126h 2456217 2456216 2026-09-09T20:45:27Z Stashbot 7414 sbassett@deploy1003: Finished scap sync-world: Backport for [[gerrit:1338280|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338281|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338283|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338282|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338278|Revert^2 "Filter out non-http(s) license urls"]], [[gerrit:1338279|Revert^2 2456217 wikitext text/x-wiki 8z46zyzenyfyv2mezep5nn6e1pebf4p 2456218 2456217 2026-09-09T20:49:28Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1342.eqiad.wmnet with reason: host reimage 2456218 wikitext text/x-wiki lak19v33whj8t242i8856gwmxtzapgn 2456219 2456218 2026-09-09T20:50:09Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1075.eqiad.wmnet with reason: host reimage 2456219 wikitext text/x-wiki 436jbmnmq7mxyvmgc1kwqq39ypxnqhm 2456220 2456219 2026-09-09T20:53:12Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.presto.roll-restart-workers for Presto an-presto-test cluster: Roll restart of all Presto's jvm daemons. 2456220 wikitext text/x-wiki 62s7ojecfuinj7be1velz1z4641qp77 2456221 2456220 2026-09-09T20:53:30Z Stashbot 7414 bking@cumin2003: END (ERROR) - Cookbook sre.presto.roll-restart-workers (exit_code=97) for Presto an-presto-test cluster: Roll restart of all Presto's jvm daemons. 2456221 wikitext text/x-wiki 5pwaafaid079p52p3n7wsypi56kqv1d 2456222 2456221 2026-09-09T20:53:48Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1075.eqiad.wmnet with reason: host reimage 2456222 wikitext text/x-wiki k4q81qfaam8t74zx0cli4soz6snaefc 2456223 2456222 2026-09-09T20:54:13Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reboot-single for host sessionstore2004.codfw.wmnet 2456223 wikitext text/x-wiki ehid0anakty8tj4ec3ns25iar6twsqr 2456224 2456223 2026-09-09T20:54:50Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.presto.roll-restart-workers for Presto an-presto-test cluster: Roll restart of all Presto's jvm daemons. 2456224 wikitext text/x-wiki tdlv6so6agupu3x055a95i2zy7d3vdv 2456225 2456224 2026-09-09T20:57:00Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.presto.roll-restart-workers (exit_code=0) for Presto an-presto-test cluster: Roll restart of all Presto's jvm daemons. 2456225 wikitext text/x-wiki fzrrmgdnhgenqvcr71vsrer5u7ww6om 2456226 2456225 2026-09-09T20:57:11Z Stashbot 7414 bking@cumin2003: START - Cookbook sre.presto.roll-restart-workers for Presto an-presto-canary cluster: Roll restart of all Presto's jvm daemons. 2456226 wikitext text/x-wiki s6tqwnq6b57donhnyaebcv9877v5nlu 2456227 2456226 2026-09-09T20:59:32Z Stashbot 7414 bking@cumin2003: END (PASS) - Cookbook sre.presto.roll-restart-workers (exit_code=0) for Presto an-presto-canary cluster: Roll restart of all Presto's jvm daemons. 2456227 wikitext text/x-wiki 70z2wpjbrn7vybe4lg72spwogrel7xe 2456228 2456227 2026-09-09T21:05:00Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sessionstore2004.codfw.wmnet 2456228 wikitext text/x-wiki kq27cc1hzckngmwu4vnu13615jin57q 2456229 2456228 2026-09-09T21:05:02Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts sessionstore2004.codfw.wmnet 2456229 wikitext text/x-wiki c92oxfc7k4sodndsrj6zfwvbyfv7ak7 2456230 2456229 2026-09-09T21:06:52Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.provision for host sessionstore2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2456230 wikitext text/x-wiki lwerfg7wgeud745u06ymk7y9zutrggz 2456231 2456230 2026-09-09T21:07:31Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host sessionstore2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2456231 wikitext text/x-wiki r0yhco4g2ymtjdv4lpc9methgosfnb5 2456232 2456231 2026-09-09T21:07:32Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1342.eqiad.wmnet with OS trixie 2456232 wikitext text/x-wiki q2suq9diza6gqj7l9dub392959pg21i 2456233 2456232 2026-09-09T21:08:00Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reimage for host sessionstore2004.codfw.wmnet with OS bookworm 2456233 wikitext text/x-wiki 3gcr6ufsp3jplvlsw1pnljc22rot7oe 2456234 2456233 2026-09-09T21:09:42Z Stashbot 7414 jforrester@deploy1003: Started scap sync-world: Backport for [[gerrit:1338273|wikifunctions: Move client fragments to mainstash (T432849)]] 2456234 wikitext text/x-wiki bszwft113vfzlogo27a5uu2z51qk92i 2456235 2456234 2026-09-09T21:09:44Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - vriley@cumin1003" 2456235 wikitext text/x-wiki 1413s3my4w93m40hqb0xoks27ncyzj6 2456236 2456235 2026-09-09T21:14:02Z Stashbot 7414 jforrester@deploy1003: jforrester: Backport for [[gerrit:1338273|wikifunctions: Move client fragments to mainstash (T432849)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2456236 wikitext text/x-wiki 24lody48ik01xtr8cj0ukd8yfoag7fr 2456237 2456236 2026-09-09T21:15:33Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - vriley@cumin1003" 2456237 wikitext text/x-wiki mbk4x2cefpqi41sflqcgc230quibv57 2456238 2456237 2026-09-09T21:15:35Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1075.eqiad.wmnet with OS trixie 2456238 wikitext text/x-wiki 2zsf3h5ag9e8y24561rwnwbppsykz98 2456239 2456238 2026-09-09T21:17:33Z Stashbot 7414 jforrester@deploy1003: jforrester: Continuing with deployment 2456239 wikitext text/x-wiki s5xy874il8veeveukgkt8wowdfshpj7 2456240 2456239 2026-09-09T21:19:08Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1342.eqiad.wmnet 2456240 wikitext text/x-wiki 28qwj4c4zilcj5rlseuzqvve6942ims 2456241 2456240 2026-09-09T21:19:10Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1342.eqiad.wmnet 2456241 wikitext text/x-wiki 3wi2pm3yu7g1viyifpoi7c5xuzy8mpa 2456242 2456241 2026-09-09T21:19:11Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.k8s.renumber-node (exit_code=0) Renumbering for host wikikube-worker1342.eqiad.wmnet 2456242 wikitext text/x-wiki slvro19j8twh7x0pquph00qvwoelj0g 2456243 2456242 2026-09-09T21:20:50Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.k8s.renumber-node Renumbering for host wikikube-worker1343.eqiad.wmnet 2456243 wikitext text/x-wiki kegldx8rbjfjlxxqonwlkb2t0bv0eyt 2456244 2456243 2026-09-09T21:20:54Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1343.eqiad.wmnet 2456244 wikitext text/x-wiki p4ian0oi2rpxyin6ja3mj95t2vsbsl6 2456245 2456244 2026-09-09T21:21:28Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1343.eqiad.wmnet 2456245 wikitext text/x-wiki 0tlle1dx4yj100x9tml9eduh0f3hvux 2456246 2456245 2026-09-09T21:21:56Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.hosts.reimage for host wikikube-worker1343.eqiad.wmnet with OS trixie 2456246 wikitext text/x-wiki 3w7b6ppcrc5vk2qrrod8xt0ixnwpdkv 2456247 2456246 2026-09-09T21:22:12Z Stashbot 7414 jforrester@deploy1003: Finished scap sync-world: Backport for [[gerrit:1338273|wikifunctions: Move client fragments to mainstash (T432849)]] (duration: 12m 29s) 2456247 wikitext text/x-wiki so0p4y9gpoal1imljb59dr002pc7904 2456248 2456247 2026-09-09T21:22:25Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.hosts.move-vlan for host wikikube-worker1343 2456248 wikitext text/x-wiki 9ap2o4wsefvz5la9bbgqttcjmyp25yw 2456249 2456248 2026-09-09T21:23:02Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.dns.netbox 2456249 wikitext text/x-wiki qvlmeo16o2zdh7mnpxd27ucv1zpor7p 2456251 2456249 2026-09-09T21:27:27Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1343 - rzl@cumin2003" 2456251 wikitext text/x-wiki jcpiq20xx8y6dwyb8kpdezzarrzl7e2 2456252 2456251 2026-09-09T21:27:32Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1343 - rzl@cumin2003" 2456252 wikitext text/x-wiki b321pwb43lvgvntnjyrrewv3txhckrc 2456253 2456252 2026-09-09T21:27:33Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2456253 wikitext text/x-wiki j2br7ui3cis9qvki1nf6c3f2tjxnt41 2456254 2456253 2026-09-09T21:27:34Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.dns.wipe-cache wikikube-worker1343.eqiad.wmnet 193.48.64.10.in-addr.arpa 3.9.1.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2456254 wikitext text/x-wiki 2cviz124b1ly1f6hbhji7pt4uhtcr0d 2456255 2456254 2026-09-09T21:27:35Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker1343.eqiad.wmnet 193.48.64.10.in-addr.arpa 3.9.1.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2456255 wikitext text/x-wiki elvj9ut52q74r57pojxnenqk7apxstk 2456256 2456255 2026-09-09T21:27:36Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker1343 2456256 wikitext text/x-wiki g0lufhxt2101fw8zx5up24g7l8052zc 2456257 2456256 2026-09-09T21:28:31Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker1343 2456257 wikitext text/x-wiki pbso094cy5x2340yu0oda62sq5mvg0m 2456258 2456257 2026-09-09T21:28:32Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker1343 2456258 wikitext text/x-wiki 76lfyykvck77n8jpfl5y5z7aonq5xyw 2456259 2456258 2026-09-09T21:36:36Z Stashbot 7414 eevans@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sessionstore2004.codfw.wmnet with OS bookworm 2456259 wikitext text/x-wiki pus3aft8nktmlrryd59p7km748vjdnq 2456260 2456259 2026-09-09T21:36:52Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reimage for host sessionstore2004.codfw.wmnet with OS bookworm 2456260 wikitext text/x-wiki 7olkr4o6x5f635hnrv467eyz3loliby 2456261 2456260 2026-09-09T21:40:33Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1343.eqiad.wmnet with reason: host reimage 2456261 wikitext text/x-wiki g5l7695dpnuy6okyiztbqazfcvn5fdu 2456262 2456261 2026-09-09T21:44:06Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1343.eqiad.wmnet with reason: host reimage 2456262 wikitext text/x-wiki 45imeiggpbagdo7xn4q8gsv02q15kxd 2456263 2456262 2026-09-09T21:59:17Z Stashbot 7414 eevans@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sessionstore2004.codfw.wmnet with OS bookworm 2456263 wikitext text/x-wiki afbpw393isn0x25wzp2n4c7ktq8krkg 2456264 2456263 2026-09-09T22:00:08Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.reimage for host sessionstore2004.codfw.wmnet with OS bookworm 2456264 wikitext text/x-wiki 0fuivnl18mqj1tsolf9zn48o1u27fuy 2456265 2456264 2026-09-09T22:03:07Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1343.eqiad.wmnet with OS trixie 2456265 wikitext text/x-wiki d24s3wdc13exte07mizyazm7fj0onsc 2456266 2456265 2026-09-09T22:05:52Z Stashbot 7414 derenrich@deploy1003: Started scap sync-world: Backport for [[gerrit:1337999|Enable discord preview extension code on testwiki (T437344)]] 2456266 wikitext text/x-wiki 6nb0jw78g3707yu66h0qivhiu40ieh6 2456267 2456266 2026-09-09T22:10:12Z Stashbot 7414 derenrich@deploy1003: derenrich: Backport for [[gerrit:1337999|Enable discord preview extension code on testwiki (T437344)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2456267 wikitext text/x-wiki gs3is7jwylluo0ll7q01j4ftt3e4zri 2456268 2456267 2026-09-09T22:16:54Z Stashbot 7414 derenrich@deploy1003: derenrich: Rolling back deployment 2456268 wikitext text/x-wiki 6jl4mzi47m7b1ry55j8ww097d1v6f7h 2456269 2456268 2026-09-09T22:19:33Z Stashbot 7414 derenrich@deploy1003: Finished scap sync-world: Backport for [[gerrit:1337999|Enable discord preview extension code on testwiki (T437344)]] (duration: 13m 40s) 2456269 wikitext text/x-wiki cfd8zo2wmuhw3sclpxe983g0ee7worq 2456270 2456269 2026-09-09T22:20:26Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on sessionstore2004.codfw.wmnet with reason: host reimage 2456270 wikitext text/x-wiki bf2mahp3cz1zc2tic11o5rtxgujieis 2456271 2456270 2026-09-09T22:22:03Z Stashbot 7414 derenrich@deploy1003: Started scap sync-world: Backport for [[gerrit:1338303|Revert "Enable discord preview extension code on testwiki"]] 2456271 wikitext text/x-wiki tbv6gysdo0ickeyfbg4xqad1zrbjqls 2456272 2456271 2026-09-09T22:22:48Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1343.eqiad.wmnet 2456272 wikitext text/x-wiki 06t348k9a40f8h5a70qlmvdyef0cwug 2456273 2456272 2026-09-09T22:22:51Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1343.eqiad.wmnet 2456273 wikitext text/x-wiki piyiz6evg1u3bpw2zc0i9e00673h4qi 2456274 2456273 2026-09-09T22:22:53Z Stashbot 7414 rzl@cumin2003: END (FAIL) - Cookbook sre.k8s.renumber-node (exit_code=1) Renumbering for host wikikube-worker1343.eqiad.wmnet 2456274 wikitext text/x-wiki ri4r2bdmfv8zdhce3zuhbosul8dzgqz 2456275 2456274 2026-09-09T22:24:20Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on sessionstore2004.codfw.wmnet with reason: host reimage 2456275 wikitext text/x-wiki fpfhlzwlryxx93kkhd3dj6660xzje0v 2456276 2456275 2026-09-09T22:26:56Z Stashbot 7414 derenrich@deploy1003: derenrich, egardner: Backport for [[gerrit:1338303|Revert "Enable discord preview extension code on testwiki"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2456276 wikitext text/x-wiki mscvvefwqkakx9148ji6ley4ycwpzxm 2456277 2456276 2026-09-09T22:27:58Z Stashbot 7414 derenrich@deploy1003: derenrich, egardner: Continuing with deployment 2456277 wikitext text/x-wiki jkfsb2ztzjhefh0i2okvrbgg8sc6usu 2456279 2456277 2026-09-09T22:31:40Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.k8s.renumber-node Renumbering for host wikikube-worker1344.eqiad.wmnet 2456279 wikitext text/x-wiki fkxj4vp5vsvwvmhf68tzpxxe7a9lxl0 2456280 2456279 2026-09-09T22:31:44Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1344.eqiad.wmnet 2456280 wikitext text/x-wiki qiaj1o39nxbsum7tbnctnz4ppu3immt 2456281 2456280 2026-09-09T22:32:20Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1344.eqiad.wmnet 2456281 wikitext text/x-wiki hdwggwzxurvkav6v47jjbfr05f0hli9 2456284 2456281 2026-09-09T22:32:28Z Stashbot 7414 derenrich@deploy1003: Finished scap sync-world: Backport for [[gerrit:1338303|Revert "Enable discord preview extension code on testwiki"]] (duration: 10m 21s) 2456284 wikitext text/x-wiki fzra8mrcxbkdqi47cn2cgxss3k09d3j 2456285 2456284 2026-09-09T22:33:06Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.hosts.reimage for host wikikube-worker1344.eqiad.wmnet with OS trixie 2456285 wikitext text/x-wiki 5mt40aablewrlz02yxt3ofgdzqt2s95 2456286 2456285 2026-09-09T22:33:36Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.hosts.move-vlan for host wikikube-worker1344 2456286 wikitext text/x-wiki m5jtu17qrhi2xjhcefpzx3udr04cs95 2456287 2456286 2026-09-09T22:33:58Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.dns.netbox 2456287 wikitext text/x-wiki talfj2xrl1nc2sq1kjyen7pv4a462c8 2456291 2456287 2026-09-09T22:37:39Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1344 - rzl@cumin2003" 2456291 wikitext text/x-wiki f9cwolg7tcy2h3em9txf8h9wmerbd9q 2456292 2456291 2026-09-09T22:37:43Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1344 - rzl@cumin2003" 2456292 wikitext text/x-wiki ml9utna5qxzcpol8p6pyjwh5strulcu 2456293 2456292 2026-09-09T22:37:44Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2456293 wikitext text/x-wiki fus5067t4qs996x2u4gxrejuotc6fgi 2456294 2456293 2026-09-09T22:37:45Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.dns.wipe-cache wikikube-worker1344.eqiad.wmnet 194.48.64.10.in-addr.arpa 4.9.1.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2456294 wikitext text/x-wiki tsjvj8x5uprjcqzgggmj5afpgk83b88 2456295 2456294 2026-09-09T22:37:47Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker1344.eqiad.wmnet 194.48.64.10.in-addr.arpa 4.9.1.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2456295 wikitext text/x-wiki czd6a692s905jb7ovmv2ejo8kq3bbmq 2456296 2456295 2026-09-09T22:37:48Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker1344 2456296 wikitext text/x-wiki ekaojg0he2pjd6j7964d1bu6r5qt0ne 2456297 2456296 2026-09-09T22:39:05Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker1344 2456297 wikitext text/x-wiki 02ss9aolojjcmfdbjdbt8ya6ft5fa92 2456298 2456297 2026-09-09T22:39:12Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker1344 2456298 wikitext text/x-wiki aa7w009kex2ygwpbowbb0kxd2luz7m0 2456299 2456298 2026-09-09T22:45:13Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host sessionstore2004.codfw.wmnet with OS bookworm 2456299 wikitext text/x-wiki n30k5zibnauanwfhv64l84gkxnn406r 2456300 2456299 2026-09-09T22:51:31Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1344.eqiad.wmnet with reason: host reimage 2456300 wikitext text/x-wiki rycpsx7uiuvrhtwatm8cr9a0it7plnc 2456301 2456300 2026-09-09T22:57:58Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1344.eqiad.wmnet with reason: host reimage 2456301 wikitext text/x-wiki s0cwit5ovnw64xjaaevh3r4l25g79ji 2456302 2456301 2026-09-09T23:11:41Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1338142|Move testcommonswiki links tables to x4 (T398709)]] 2456302 wikitext text/x-wiki 0qym8b9auoibeydyzkubtgpdkizpyj0 2456303 2456302 2026-09-09T23:15:06Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1344.eqiad.wmnet with OS trixie 2456303 wikitext text/x-wiki n0ab7by3upqo0xt6lrs4vlv76cra84a 2456304 2456303 2026-09-09T23:16:01Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1338142|Move testcommonswiki links tables to x4 (T398709)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2456304 wikitext text/x-wiki j252hrr1psib5apaup2ckh3uk8vhhlc 2456305 2456304 2026-09-09T23:18:26Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2456305 wikitext text/x-wiki 7ezvl8e6klrdw9ogdbcyeemwfn5jse1 2456306 2456305 2026-09-09T23:22:56Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1338142|Move testcommonswiki links tables to x4 (T398709)]] (duration: 11m 15s) 2456306 wikitext text/x-wiki kbrhf6ifp20kw7e208hcquue6wwnl1b 2456309 2456306 2026-09-09T23:26:45Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1344.eqiad.wmnet 2456309 wikitext text/x-wiki 7wfnxhw27xvhoo1gebw8rnbohijjsyi 2456310 2456309 2026-09-09T23:26:47Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1344.eqiad.wmnet 2456310 wikitext text/x-wiki 50q6r2n0xm133pxjec59a8viqdacfel 2456311 2456310 2026-09-09T23:26:49Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.k8s.renumber-node (exit_code=0) Renumbering for host wikikube-worker1344.eqiad.wmnet 2456311 wikitext text/x-wiki amlek41mgbouys9kg9polqmvuatvnby 2456312 2456311 2026-09-09T23:27:23Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.k8s.renumber-node Renumbering for host wikikube-worker1345.eqiad.wmnet 2456312 wikitext text/x-wiki 3bq1zz9jfnr9fofgrvjk28qmc6dgrnp 2456313 2456312 2026-09-09T23:27:27Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1345.eqiad.wmnet 2456313 wikitext text/x-wiki l9i9n2wkgee4gwc44ngw59a0uwqhdlj 2456314 2456313 2026-09-09T23:28:02Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1345.eqiad.wmnet 2456314 wikitext text/x-wiki rj71s3d056v0cllz9e41ci5ubvrvk3w 2456315 2456314 2026-09-09T23:28:14Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.hosts.reimage for host wikikube-worker1345.eqiad.wmnet with OS trixie 2456315 wikitext text/x-wiki 9etj5qn1na6j3iqhd1nqo7dbdjgx463 2456316 2456315 2026-09-09T23:28:43Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.hosts.move-vlan for host wikikube-worker1345 2456316 wikitext text/x-wiki 8zyo9upzdenq6fyy65gru9t4xhzm29o 2456317 2456316 2026-09-09T23:29:03Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.dns.netbox 2456317 wikitext text/x-wiki 4nzj28v8ptpxdqmke6k2l9jznnis3r0 2456318 2456317 2026-09-09T23:33:00Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1345 - rzl@cumin2003" 2456318 wikitext text/x-wiki jr4c8keu2ogyyxcz9u4aw4410zi8twc 2456319 2456318 2026-09-09T23:33:04Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1345 - rzl@cumin2003" 2456319 wikitext text/x-wiki r4vhj4ppp3g21hbe9yl0dqpk2vpb6kc 2456320 2456319 2026-09-09T23:33:05Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2456320 wikitext text/x-wiki 50ml5zc58wmeht0r55k1ffp2rveaq0r 2456321 2456320 2026-09-09T23:33:06Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.dns.wipe-cache wikikube-worker1345.eqiad.wmnet 195.48.64.10.in-addr.arpa 5.9.1.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2456321 wikitext text/x-wiki qvzy4mvj52fwfzfnlddd5jje2byptc9 2456322 2456321 2026-09-09T23:33:08Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker1345.eqiad.wmnet 195.48.64.10.in-addr.arpa 5.9.1.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2456322 wikitext text/x-wiki 5oyumriay48dsrqmtp4web68cd7it1k 2456323 2456322 2026-09-09T23:33:09Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker1345 2456323 wikitext text/x-wiki ovoeob0yawkev3kcj139szfvtwjo28x 2456324 2456323 2026-09-09T23:33:40Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker1345 2456324 wikitext text/x-wiki 4bngagakp50we2h9lbdvaz1gjj237pn 2456325 2456324 2026-09-09T23:33:41Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker1345 2456325 wikitext text/x-wiki 07zlp0t9bays4r8t4cz8lmi8tkip7kz 2456326 2456325 2026-09-09T23:37:24Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply 2456326 wikitext text/x-wiki 0nst7v2dhbxrzo1hxtipr913s98jmeg 2456327 2456326 2026-09-09T23:37:36Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply 2456327 wikitext text/x-wiki n6pob8l6y9lok4xl9twuu1geihc94sn 2456328 2456327 2026-09-09T23:46:09Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1345.eqiad.wmnet with reason: host reimage 2456328 wikitext text/x-wiki 7t5j5jx5iqw80wlqpbo7m3fmak4jhml 2456329 2456328 2026-09-09T23:49:03Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1345.eqiad.wmnet with reason: host reimage 2456329 wikitext text/x-wiki 1oqvp2uanihsj3hc8fj5hk399hb78la 2456331 2456329 2026-09-10T00:08:51Z Stashbot 7414 dzahn@dns1004: START - running authdns-update 2456331 wikitext text/x-wiki hgtur14f25ovmpuz9mr6n0wnkkw5ff7 2456332 2456331 2026-09-10T00:08:58Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1345.eqiad.wmnet with OS trixie 2456332 wikitext text/x-wiki nq909qqnxpsx20lfwdnpgs1lyoq0v0q 2456333 2456332 2026-09-10T00:11:06Z Stashbot 7414 dzahn@dns1004: END - running authdns-update 2456333 wikitext text/x-wiki cju729hcsmcijbhkq9z5t5b3fil4t6w 2456334 2456333 2026-09-10T00:20:31Z Stashbot 7414 rzl@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1345.eqiad.wmnet 2456334 wikitext text/x-wiki o3mdzz74ncxudt1p47c06zqpjzh8h6i 2456335 2456334 2026-09-10T00:20:33Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1345.eqiad.wmnet 2456335 wikitext text/x-wiki idvunnk8qteyz198rk5h90gcysqqii6 2456336 2456335 2026-09-10T00:20:34Z Stashbot 7414 rzl@cumin2003: END (PASS) - Cookbook sre.k8s.renumber-node (exit_code=0) Renumbering for host wikikube-worker1345.eqiad.wmnet 2456336 wikitext text/x-wiki h2xsipnrczoyswmis98dtk057jnebb2 2456339 2456336 2026-09-10T02:00:30Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2456339 wikitext text/x-wiki 3i35m0vrize8gwk3dfbd5u8iv3d60in 2456340 2456339 2026-09-10T02:08:09Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 07m 38s) 2456340 wikitext text/x-wiki cdhxzgrrbxnq5m8n7ac3mgbh1k5537v 2456342 2456340 2026-09-10T05:03:41Z Stashbot 7414 kevinbazira@deploy1003: helmfile [ml-serve-eqiad] 'sync' command on namespace 'tool-server' for release 'main' . 2456342 wikitext text/x-wiki ibzhhsu0mwvbvxde8gadht3vzsn72bi 2456343 2456342 2026-09-10T05:04:31Z Stashbot 7414 kevinbazira@deploy1003: helmfile [ml-serve-codfw] 'sync' command on namespace 'tool-server' for release 'main' . 2456343 wikitext text/x-wiki i500bp0qkxyfj3fxipzovyrokaynhrr 2456347 2456343 2026-09-10T06:37:52Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.dns.netbox 2456347 wikitext text/x-wiki ehukff6snwnjyzgzvzuzxunsbh3jdtb 2456348 2456347 2026-09-10T06:38:50Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1075 2456348 wikitext text/x-wiki izyjhmgovy8ztqtwb0qtk020k9aezec 2456349 2456348 2026-09-10T06:39:27Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1075 2456349 wikitext text/x-wiki 62mb6noe9usr2gqvenpgmr4tgyyshiz 2456350 2456349 2026-09-10T06:42:06Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1075 cloud-private - filippo@cumin1003" 2456350 wikitext text/x-wiki qy5o6u8hx7ro2fombj5fx6klv91ucd8 2456351 2456350 2026-09-10T06:42:10Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1075 cloud-private - filippo@cumin1003" 2456351 wikitext text/x-wiki 7alr8596izkuj79vypb68xgfu66ttje 2456352 2456351 2026-09-10T06:42:11Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2456352 wikitext text/x-wiki d2wptstcgzyyivyeoxm9v8viayozsle 2456353 2456352 2026-09-10T06:43:30Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1065.eqiad.wmnet with OS trixie 2456353 wikitext text/x-wiki dz09sojsy7g4idtc2pfcznh4zg1n1t5 2456354 2456353 2026-09-10T06:59:27Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1065.eqiad.wmnet with reason: host reimage 2456354 wikitext text/x-wiki iorcs5ri3xi7k0jnl2zw0qvytny3gf4 2456355 2456354 2026-09-10T07:03:01Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1065.eqiad.wmnet with reason: host reimage 2456355 wikitext text/x-wiki b0rkdwywog7bfwoycjhd1hvwpz2xrms 2456356 2456355 2026-09-10T07:28:20Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2456356 wikitext text/x-wiki 0zp416fmu9pz50ts68lwuv0v6yf0rzw 2456357 2456356 2026-09-10T07:29:09Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2456357 wikitext text/x-wiki mzqh2qdadtu1ackxx826zrfai6na8gb 2456358 2456357 2026-09-10T07:32:11Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-dev: apply 2456358 wikitext text/x-wiki ppz0lpbwcge5vh4e4be0c09d18aff4z 2456359 2456358 2026-09-10T07:32:15Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-dev: apply 2456359 wikitext text/x-wiki crdnw3esicaqhhn87hgoxic9i67vr0n 2456360 2456359 2026-09-10T07:32:27Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-analytics-product: apply 2456360 wikitext text/x-wiki 5nusv7cljoc45jff55hwyt6yrltl13x 2456361 2456360 2026-09-10T07:32:31Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-analytics-product: apply 2456361 wikitext text/x-wiki gqde6eujnlz1j0s14a77putqe1a9gpp 2456362 2456361 2026-09-10T07:32:35Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-analytics-test: apply 2456362 wikitext text/x-wiki 2ghe0zb3cr43b0qv5qdz8ifx8q9kpll 2456363 2456362 2026-09-10T07:32:38Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-analytics-test: apply 2456363 wikitext text/x-wiki 5hbpqee0ldv965rhn7yvyc2mj0cbctz 2456364 2456363 2026-09-10T07:32:44Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-dumps: apply 2456364 wikitext text/x-wiki p024n4eb2sl52d9s53q4yk2ny6mrzyx 2456365 2456364 2026-09-10T07:32:47Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-dumps: apply 2456365 wikitext text/x-wiki c2qch7uhh2zg0rzx4n6igi5u99gzcyv 2456366 2456365 2026-09-10T07:33:02Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-experiment-platform: apply 2456366 wikitext text/x-wiki 2dp1qc7fbrb84uzdnpbxtcf12evl6ec 2456367 2456366 2026-09-10T07:33:05Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-experiment-platform: apply 2456367 wikitext text/x-wiki cbw57xttw41t564h3xp7rh1s7cj7huw 2456368 2456367 2026-09-10T07:33:11Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-fr-tech: apply 2456368 wikitext text/x-wiki 3ovke7pww6gmt7nnxcb2cqf3cflv1o8 2456369 2456368 2026-09-10T07:33:14Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-fr-tech: apply 2456369 wikitext text/x-wiki 3dqhpyo7b5o25mri58ct6mb947gz9j6 2456370 2456369 2026-09-10T07:33:18Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-main: apply 2456370 wikitext text/x-wiki 34m2j1oa4zdi5ut9a3rng7gon4ahnd8 2456371 2456370 2026-09-10T07:33:22Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-main: apply 2456371 wikitext text/x-wiki k75j2ztr7xkit06o0njzqivrrixn084 2456372 2456371 2026-09-10T07:33:25Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-ml: apply 2456372 wikitext text/x-wiki hoo2a1c5lppzh4943olmk47d1qkj1ob 2456373 2456372 2026-09-10T07:33:29Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-ml: apply 2456373 wikitext text/x-wiki 4ff3jwmzet21epeyl0yl758pggnpvfg 2456374 2456373 2026-09-10T07:33:32Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-platform-eng: apply 2456374 wikitext text/x-wiki 3hgg0ucq679lha3zcpglcyzrigifanl 2456375 2456374 2026-09-10T07:33:36Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-platform-eng: apply 2456375 wikitext text/x-wiki ls1lvwprazr4onf07h3dubq1b7qnzj9 2456376 2456375 2026-09-10T07:33:39Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-research: apply 2456376 wikitext text/x-wiki gt2ert76q4yq39g22jirouinexp4ccv 2456377 2456376 2026-09-10T07:33:43Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-research: apply 2456377 wikitext text/x-wiki rnmxcx8rhzgqil2khwy7bw8h5xn7v0h 2456378 2456377 2026-09-10T07:33:46Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-search: apply 2456378 wikitext text/x-wiki 655icpy2mmc28f2yqs875yz3k4xowk5 2456379 2456378 2026-09-10T07:33:50Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-search: apply 2456379 wikitext text/x-wiki k6lb20rzwic4ksansx5aqxvvsnnrm74 2456380 2456379 2026-09-10T07:33:53Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-sre: apply 2456380 wikitext text/x-wiki jmdfbcudf214zajz8m0dq7iim4wy167 2456381 2456380 2026-09-10T07:33:57Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-sre: apply 2456381 wikitext text/x-wiki bk3o7qi7bt15vmmqf4uvir53mkpww9e 2456382 2456381 2026-09-10T07:34:00Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-test-k8s: apply 2456382 wikitext text/x-wiki 4jcbw3g8bigyus1vx5bwsnroqr6dqjr 2456383 2456382 2026-09-10T07:34:07Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-test-k8s: apply 2456383 wikitext text/x-wiki osumx6okjmkxxtq5pfny0l234ugapju 2456384 2456383 2026-09-10T07:34:10Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-wikidata: apply 2456384 wikitext text/x-wiki a7w6tt6y7l2am7ztofgi7pia5gqlkqy 2456385 2456384 2026-09-10T07:34:12Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-wikidata: apply 2456385 wikitext text/x-wiki pbb3f8izi5rqo82g6t2vccgvxk242ls 2456386 2456385 2026-09-10T07:34:15Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-wmde: apply 2456386 wikitext text/x-wiki he5okcpyuxiilp4wxxbesxqicwo4ot3 2456387 2456386 2026-09-10T07:34:19Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-wmde: apply 2456387 wikitext text/x-wiki ezbb4wkkrzduvz05k0j6hglsm2hehvn 2456388 2456387 2026-09-10T07:34:22Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook: apply 2456388 wikitext text/x-wiki k6i45zrkz7lkht9pdzhxgwhnco8ali2 2456389 2456388 2026-09-10T07:34:26Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook: apply 2456389 wikitext text/x-wiki 1pbs6mtrnzmzsdzocsus9j89mis5pnb 2456390 2456389 2026-09-10T07:34:30Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2456390 wikitext text/x-wiki t0mlrcljniiarje7ynsob00af3215pb 2456391 2456390 2026-09-10T07:34:33Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply 2456391 wikitext text/x-wiki g2httd9od0e6qb38z6ec6ulfds6c8h6 2456392 2456391 2026-09-10T07:34:37Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-superset-metrics: apply 2456392 wikitext text/x-wiki enc5hnkt0ubynbb5e4re0cjr1hit5ij 2456393 2456392 2026-09-10T07:34:41Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-superset-metrics: apply 2456393 wikitext text/x-wiki 32x9ua3k7gquc0nl73kdimawhu7dv8j 2456394 2456393 2026-09-10T07:39:53Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1065.eqiad.wmnet with OS trixie 2456394 wikitext text/x-wiki bfggsmojtvhi5adtpq5rzke8wjxvtpl 2456395 2456394 2026-09-10T07:56:54Z Stashbot 7414 XioNoX: netflow2005:~$ sudo dpkg -i gnmic_0.48.0_Linux_x86_64.deb 2456395 wikitext text/x-wiki 20ygzvv8vg07rxicdgap2u2jtqlty2i 2456396 2456395 2026-09-10T07:58:27Z Stashbot 7414 XioNoX: netflow1004:~$ sudo dpkg -i gnmic_0.48.0_Linux_x86_64.deb 2456396 wikitext text/x-wiki 3ghd6ith62q6drmwciwghof04x7rzqr 2456397 2456396 2026-09-10T08:07:17Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1066.eqiad.wmnet with OS trixie 2456397 wikitext text/x-wiki 99bzm8uynwfnbgv5ptvi23sbkovzefb 2456398 2456397 2026-09-10T08:07:39Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1067.eqiad.wmnet with OS trixie 2456398 wikitext text/x-wiki 95lrzk2shvuab9b7s548drsfy9mqpnz 2456399 2456398 2026-09-10T08:09:56Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1072.eqiad.wmnet with OS trixie 2456399 wikitext text/x-wiki 8gc03s4g1pt1927rbshdgizfda7o6gf 2456400 2456399 2026-09-10T08:11:11Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1073.eqiad.wmnet with OS trixie 2456400 wikitext text/x-wiki qb26ho91uwiz0ltbme11cnssqmed1w5 2456401 2456400 2026-09-10T08:11:54Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1074.eqiad.wmnet with OS trixie 2456401 wikitext text/x-wiki 5pkp9m95z8d1yk2o7ge7d9do18vcv9k 2456402 2456401 2026-09-10T08:23:13Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1066.eqiad.wmnet with reason: host reimage 2456402 wikitext text/x-wiki ms0ymdd0ecmhgqw9bb6bacsxv0igja1 2456403 2456402 2026-09-10T08:23:42Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1067.eqiad.wmnet with reason: host reimage 2456403 wikitext text/x-wiki 38ad4yatjedi4tffb0wgofcsvh1crt1 2456404 2456403 2026-09-10T08:23:55Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/pageview-trending-relative-next: apply 2456404 wikitext text/x-wiki m10e2ir96877wg47jbo4gdpziozb2t0 2456405 2456404 2026-09-10T08:23:58Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/pageview-trending-relative-next: apply 2456405 wikitext text/x-wiki 0b7lmj9k0vlnw90to68o0b5fj3khbqm 2456406 2456405 2026-09-10T08:24:53Z Stashbot 7414 mszwarc@deploy1003: Started scap sync-world: Backport for [[gerrit:1338705|UIC: Fix getOpenContext when UserCardButton.vue is clicked (T435299)]] 2456406 wikitext text/x-wiki 31qdm3frginr258ws8tyjohy3bk7ngi 2456407 2456406 2026-09-10T08:25:57Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1072.eqiad.wmnet with reason: host reimage 2456407 wikitext text/x-wiki 1f1xdmq94dgj40en4548p690mowzs50 2456408 2456407 2026-09-10T08:27:27Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1073.eqiad.wmnet with reason: host reimage 2456408 wikitext text/x-wiki aina012wn30hzmwjwhaafnk9gyxyl0r 2456409 2456408 2026-09-10T08:27:50Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1075.eqiad.wmnet with OS trixie 2456409 wikitext text/x-wiki crgr922kbjk86q3yyhtw5da5k4wywls 2456410 2456409 2026-09-10T08:28:02Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1074.eqiad.wmnet with reason: host reimage 2456410 wikitext text/x-wiki degf4udqyk5humw5mbw9rngmd7gx0jm 2456411 2456410 2026-09-10T08:29:46Z Stashbot 7414 mszwarc@deploy1003: mszwarc: Backport for [[gerrit:1338705|UIC: Fix getOpenContext when UserCardButton.vue is clicked (T435299)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2456411 wikitext text/x-wiki m2ob11h0jdtudc3dlitvpb8zsmm0pg1 2456412 2456411 2026-09-10T08:29:49Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1066.eqiad.wmnet with reason: host reimage 2456412 wikitext text/x-wiki f0hri1v06bheyepqc9qgmj8oymty90z 2456413 2456412 2026-09-10T08:30:17Z Stashbot 7414 mszwarc@deploy1003: mszwarc: Continuing with deployment 2456413 wikitext text/x-wiki c8uo2ahkishzi67cnm35wd86qui29y8 2456414 2456413 2026-09-10T08:33:14Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1072.eqiad.wmnet with reason: host reimage 2456414 wikitext text/x-wiki mzpusmm87u29n8q9napmpzuddujuzn2 2456415 2456414 2026-09-10T08:34:50Z Stashbot 7414 mszwarc@deploy1003: Finished scap sync-world: Backport for [[gerrit:1338705|UIC: Fix getOpenContext when UserCardButton.vue is clicked (T435299)]] (duration: 09m 56s) 2456415 wikitext text/x-wiki tm2umawxlqocm2ed98inmk78ezjsoxh 2456416 2456415 2026-09-10T08:37:20Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1074.eqiad.wmnet with reason: host reimage 2456416 wikitext text/x-wiki egctxwrnzohpvjve0e2ea58owvryu1z 2456418 2456416 2026-09-10T08:41:36Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1073.eqiad.wmnet with reason: host reimage 2456418 wikitext text/x-wiki 0pacpayh9mo2uzelf6y3yllv6ovovq8 2456419 2456418 2026-09-10T08:43:45Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1075.eqiad.wmnet with reason: host reimage 2456419 wikitext text/x-wiki cql5fe43tsqilwsbh8hl7l8gbmxcvsu 2456420 2456419 2026-09-10T08:44:40Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1155.eqiad.wmnet with reason: Cloning x4 2456420 wikitext text/x-wiki 9aa98038ak5hunxczqog7nkd6cbc9vm 2456421 2456420 2026-09-10T08:44:47Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply 2456421 wikitext text/x-wiki htq2nug9z406hmu1aqki0rgdsneipnj 2456422 2456421 2026-09-10T08:45:02Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply 2456422 wikitext text/x-wiki 8jw8akkvxpdcxh654outocckv0ze7zw 2456424 2456422 2026-09-10T08:46:57Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1067.eqiad.wmnet with reason: host reimage 2456424 wikitext text/x-wiki 44ryg99wvyjmnzgps4s9veoxai4cfn0 2456426 2456424 2026-09-10T08:51:17Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1075.eqiad.wmnet with reason: host reimage 2456426 wikitext text/x-wiki pl0v1asflqi1p4tu4b5g2tkbbderatz 2456428 2456426 2026-09-10T09:04:59Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1067.eqiad.wmnet with OS trixie 2456428 wikitext text/x-wiki er7fyl6h0eis59ecrcdnrkzb758z0vr 2456429 2456428 2026-09-10T09:24:13Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen: apply 2456429 wikitext text/x-wiki ggp75w5dtmrpjmehb8k6cifkfa64lca 2456430 2456429 2026-09-10T09:24:24Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen: apply 2456430 wikitext text/x-wiki l5e5r57db2azmfgk5i7k90j7f4zpf1u 2456431 2456430 2026-09-10T09:37:10Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1074.eqiad.wmnet with OS trixie 2456431 wikitext text/x-wiki gyrtkfozjzdrm2behgvptvm4spcciag 2456432 2456431 2026-09-10T09:41:37Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1066.eqiad.wmnet with OS trixie 2456432 wikitext text/x-wiki 7uoihgvyk3mjs8eo70sg0mi9vp02tdn 2456433 2456432 2026-09-10T09:45:50Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1072.eqiad.wmnet with OS trixie 2456433 wikitext text/x-wiki mgbghh139msyjkeqkzwxbm2k1i6dy43 2456434 2456433 2026-09-10T09:53:22Z Stashbot 7414 marostegui@cumin1003: conftool action : set/pooled=no; selector: name=clouddb1024.eqiad.wmnet 2456434 wikitext text/x-wiki bc6hyl1nojarl3uym61j4fylk18uhue 2456435 2456434 2026-09-10T09:56:14Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1075.eqiad.wmnet with OS trixie 2456435 wikitext text/x-wiki mzryjtgyzp2y29ugq0y2v610kewue3h 2456436 2456435 2026-09-10T10:00:19Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1073.eqiad.wmnet with OS trixie 2456436 wikitext text/x-wiki 1t29ep9l6iy5wt6p5zg2s4jqpb1s4sk 2456437 2456436 2026-09-10T10:10:36Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb1024.eqiad.wmnet with reason: Cloning x4 2456437 wikitext text/x-wiki io7pg3ivk23fjmiy53bfwcj4ihw9b6b 2456438 2456437 2026-09-10T10:42:57Z Stashbot 7414 btullis@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on an-worker1199.eqiad.wmnet with reason: Upgrading RAID firmware 2456438 wikitext text/x-wiki 069rlj9gsxsmfr92m2nttp2q7zm1eps 2456439 2456438 2026-09-10T10:50:33Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen: apply 2456439 wikitext text/x-wiki na8xdw97d8emjie5yrgkf0wzy70hp4z 2456440 2456439 2026-09-10T10:50:58Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen: apply 2456440 wikitext text/x-wiki 8daday0ovjss1gcpbp0sqohx05hx75v 2456441 2456440 2026-09-10T11:03:59Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.remove-downtime for an-worker1199.eqiad.wmnet 2456441 wikitext text/x-wiki nihwpf7dbpo6asn2qx4f7fhn4kn6qrs 2456442 2456441 2026-09-10T11:04:00Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for an-worker1199.eqiad.wmnet 2456442 wikitext text/x-wiki n3gsby34vbvu78x5m0fqxz1g4gfo0ru 2456443 2456442 2026-09-10T11:04:23Z Stashbot 7414 btullis@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on an-worker1200.eqiad.wmnet with reason: Upgrading RAID firmware 2456443 wikitext text/x-wiki phdvcydj8bfobz5y6szg3ijbhb9i787 2456444 2456443 2026-09-10T11:16:04Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.remove-downtime for an-worker1200.eqiad.wmnet 2456444 wikitext text/x-wiki 9r5717k0h24shhf7vi2yc9k2rejjjer 2456445 2456444 2026-09-10T11:16:06Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for an-worker1200.eqiad.wmnet 2456445 wikitext text/x-wiki 3dnakknn39tg61vddecdfrmyoxe8emh 2456446 2456445 2026-09-10T11:24:14Z Stashbot 7414 btullis@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on an-worker1204.eqiad.wmnet with reason: Upgrading RAID firmware 2456446 wikitext text/x-wiki 227y6rbk9d27y9arcyrwku4z28bjeag 2456447 2456446 2026-09-10T11:25:55Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reboot-single for host dse-k8s-worker2004.codfw.wmnet 2456447 wikitext text/x-wiki bictj8bo8qzwnge2r7ss583f8yxijgy 2456448 2456447 2026-09-10T11:31:14Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-worker2004.codfw.wmnet 2456448 wikitext text/x-wiki 19nknpa1irtdob3r8c3t5n5v149rb8q 2456450 2456448 2026-09-10T11:49:54Z Stashbot 7414 cgoubert@dns1004: START - running authdns-update 2456450 wikitext text/x-wiki 1gkt8uqad2x74es49bg75gsgfkor4mh 2456451 2456450 2026-09-10T11:52:09Z Stashbot 7414 cgoubert@dns1004: END - running authdns-update 2456451 wikitext text/x-wiki cro9j339ldnnc5zx2vuqvfvcfu6l7ba 2456452 2456451 2026-09-10T11:54:41Z Stashbot 7414 btullis@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on an-worker1144.eqiad.wmnet with reason: Upgrading RAID firmware 2456452 wikitext text/x-wiki 0simc5gj4m3pu8vkt9wmqjmwcksgu9k 2456453 2456452 2026-09-10T11:55:53Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reboot-single for host dse-k8s-worker2005.codfw.wmnet 2456453 wikitext text/x-wiki hywd4mu80m5tziy8yfnhpl4ld88sbsp 498 9925 2456077 2456005 2026-09-09T15:40:06Z Stashbot 7414 komla@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase by 30 server-groups (T437117) 2456077 wikitext text/x-wiki 70qkcvvo9evmqqo6yg7ii2amoft3dxm 2456079 2456077 2026-09-09T15:40:12Z Stashbot 7414 komla@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) by 30 server-groups (T437117) 2456079 wikitext text/x-wiki bcmkl0zr9jjrsga420h37ojgg7xrplc 2456091 2456079 2026-09-09T16:12:14Z Stashbot 7414 Southparkfan: decommission deployment-docker-mathoid02 - T436465 2456091 wikitext text/x-wiki ecbe1428rwpx9qs5h42mbi0i1el961i 2456099 2456091 2026-09-09T16:32:31Z Stashbot 7414 Southparkfan: adjust cpjobqueue config to temporarily send jobs to deployment-jobrunner06 (PHP 8.5) - T435393 2456099 wikitext text/x-wiki ih0u25my3z19hm0iryuyt3dzjdpqlc5 2456105 2456099 2026-09-09T16:53:36Z Stashbot 7414 Southparkfan: cpjobqueue: switch back to deployment-jobrunner05 (PHP 8.3) - T435393 2456105 wikitext text/x-wiki 4zqp4dn0cnsbnhae7kazk3qotw8h9yf 2456283 2456105 2026-09-09T22:32:26Z Stashbot 7414 Southparkfan: set role::logging::mediawiki::udp2log::monitor: false to unbreak T436469 2456283 wikitext text/x-wiki ryk1c9uxui01hvdjuoatn9k3xj19qsx 2456289 2456283 2026-09-09T22:36:19Z Stashbot 7414 Southparkfan: point profile::rsyslog::udp_tee::destinations to both deployment-mwlog02 and deployment-mwlog03 8421/udp - T436469 2456289 wikitext text/x-wiki kru5hm2rcoo4k62uihpdm7ir8afqubd 0 17290 2456076 2456004 2026-09-09T15:40:03Z Stashbot 7414 komla@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase by 30 server-groups (T437117) 2456076 wikitext text/x-wiki q00lg3bapa53y9jixh7ykd9lj11mf48 2456078 2456076 2026-09-09T15:40:10Z Stashbot 7414 komla@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) by 30 server-groups (T437117) 2456078 wikitext text/x-wiki 7fp8rvt89r5no0co537ia0m0bb6wbih 2456089 2456078 2026-09-09T16:09:31Z Stashbot 7414 thcipriani: deployed some anti-scraper mitigations on beta 2456089 wikitext text/x-wiki 9zny4bfod0fkfttghavbfk1e9dr1bfd 2456090 2456089 2026-09-09T16:12:12Z Stashbot 7414 Southparkfan: decommission deployment-docker-mathoid02 - T436465 2456090 wikitext text/x-wiki 04ch2533xae9u5f9mx2l5sfw57s1aaq 2456098 2456090 2026-09-09T16:32:28Z Stashbot 7414 Southparkfan: adjust cpjobqueue config to temporarily send jobs to deployment-jobrunner06 (PHP 8.5) - T435393 2456098 wikitext text/x-wiki 3c647j9zk34f1i1k8lyfo5xtkektxw1 2456104 2456098 2026-09-09T16:53:34Z Stashbot 7414 Southparkfan: cpjobqueue: switch back to deployment-jobrunner05 (PHP 8.3) - T435393 2456104 wikitext text/x-wiki 1m547pi96tom5ue53svjfqmhn83w5v0 2456282 2456104 2026-09-09T22:32:24Z Stashbot 7414 Southparkfan: set role::logging::mediawiki::udp2log::monitor: false to unbreak T436469 2456282 wikitext text/x-wiki dpvfvk3nqsjz85twg6qia74sdhsjqs1 2456288 2456282 2026-09-09T22:36:17Z Stashbot 7414 Southparkfan: point profile::rsyslog::udp_tee::destinations to both deployment-mwlog02 and deployment-mwlog03 8421/udp - T436469 2456288 wikitext text/x-wiki gaoydl06eg0w9trx8c2418sd0xoec5j 2456344 2456288 2026-09-10T05:20:12Z Stashbot 7414 phedenskog: Updated 134 Quibble Jenkins jobs to drop the duplicated --reporting-url T323750 2456344 wikitext text/x-wiki 69fds4jbdvfft4cwxq8je6l5l9cpkxk 2456345 2456344 2026-09-10T05:49:07Z Stashbot 7414 phedenskog: Updated quibble-for-mediawiki-core postgres and sqlite jobs to run only the phpunit-database stage T437134 2456345 wikitext text/x-wiki bohn2ntv9g32565hn56ge4euquy01h5 12 18681 2456083 2444327 2026-09-09T15:51:05Z Taavi-WMF 41365 fix excess whitespace and cleanup spaces in visible links 2456083 wikitext text/x-wiki 7bxdgpplcsgw0b8dwm5l42nef3rea9b 0 24248 2456337 2455411 2026-09-10T01:10:59Z ADenisse-WMF 31462 Add instructions to upgrade the package using the GitLab CI pipeline. 2456337 wikitext text/x-wiki ktn0q0oajnc42tctjg9cq3pfj9q9rx0 2456346 2456337 2026-09-10T06:18:46Z ADenisse-WMF 31462 /* Installing plugins */ 2456346 wikitext text/x-wiki lui26mbtr5z452cjcnd274x0wzb7zk2 498 30942 2456417 2455786 2026-09-10T08:38:10Z Stashbot 7414 godog: reimage moved cloudvirt hosts - T435923 T435921 2456417 wikitext text/x-wiki otlpnvevcyul0wt2f1x27sv5pdzxhym 0 49472 2456038 2455767 2026-09-09T13:46:39Z BKing (WMF) 29017 /* Administration */ 2456038 wikitext text/x-wiki cudbjn3847pbbvpyg472ot632mtrvsw 0 56257 2456074 2451000 2026-09-09T15:21:09Z Lucas Werkmeister (WMDE) 10037 /* Upcoming Switches */ [[phabricator:T433364]] is at 14:00 UTC, not 15:00 (see also [[meta:Special:Diff/31008985]]) 2456074 wikitext text/x-wiki jgk8kdnkjehnbqzsixdarzp9st3vv94 0 442233 2456014 2455729 2026-09-09T12:48:52Z FNegri-WMF 32595 /* Current announcements */ Remove duplicate item 2456014 wikitext text/x-wiki ei1w7v21ydejqcu2aqngkymwqnw42ac 2456016 2456014 2026-09-09T12:51:39Z FNegri-WMF 32595 Update list of current and recently completed items 2456016 wikitext text/x-wiki 9uns5ii3ij6usnwavf141swb46hsgyu 0 447311 2456027 2403881 2026-09-09T13:22:08Z JMeybohm (WMF) 16709 /* Node (worker) */ 2456027 wikitext text/x-wiki 5ccrkiaviv94bzvfgnn7qjmxz976eco 0 447352 2456017 2455962 2026-09-09T12:59:32Z LToscano (WMF) 7191 2456017 wikitext text/x-wiki p9mvtumdsbej750kydvzgfoqquwu5ay 498 447659 2456341 2428906 2026-09-10T03:22:11Z Samwilson 1287 /* crontab summary */ crontab info now in https://gitlab.wikimedia.org/toolforge-repos/wsexport-config/-/blob/main/README.md 2456341 wikitext text/x-wiki hwb0845rnp0fm9qf5vp3pf1rvbjglt3 0 449160 2456330 2455933 2026-09-10T00:02:02Z Dexbot 30554 Bot: Updating the report 2456330 wikitext text/x-wiki jblrim33ec0wew47wkwl0c1ooemfa4o 0 451448 2456160 2436249 2026-09-09T18:54:14Z Jdforrester (WMF) 468 /* How to enable 'client mode' on a new wiki */ Drop the per-client DB table stuff, dropped from prod. 2456160 wikitext text/x-wiki jdstvpl48ha3367fnzgdcq0gangccut 12 452086 2456178 2450730 2026-09-09T19:41:35Z AndrewBogott (WMF) 51047 2456178 wikitext text/x-wiki iun2gz6kgdt3ryzuxgxswb91y9am1z8 2456181 2456178 2026-09-09T19:42:39Z AndrewBogott (WMF) 51047 /* Driver Selection */ 2456181 wikitext text/x-wiki 9e4v8zd9sfgwrrwcf01ilz251tb64hi 2456185 2456181 2026-09-09T19:47:03Z AndrewBogott (WMF) 51047 /* Deploy a cluster */ 2456185 wikitext text/x-wiki 37uqjlmphl1pg803frd5xs5yuwznzfg 2456186 2456185 2026-09-09T19:48:46Z AndrewBogott (WMF) 51047 /* Get kube config file */ 2456186 wikitext text/x-wiki gn6teo6eohv1jddj3asj6283wcsnwtm 2456188 2456186 2026-09-09T19:54:35Z AndrewBogott (WMF) 51047 /* Driver Selection */ 2456188 wikitext text/x-wiki 3f40eyjm8su4zp3e285es9algu1p1vk 2456190 2456188 2026-09-09T20:02:51Z AndrewBogott (WMF) 51047 /* Create a template */ 2456190 wikitext text/x-wiki 0bi5ho5rmt2f7pmpnjnlkacyefivzp3 2456191 2456190 2026-09-09T20:03:25Z AndrewBogott (WMF) 51047 /* Example Templates, Heat driver */ 2456191 wikitext text/x-wiki ey929iqhunwbjgwo3doqyurr8dbzt77 2456192 2456191 2026-09-09T20:03:49Z AndrewBogott (WMF) 51047 /* Example Templates, Cluster-api driver */ 2456192 wikitext text/x-wiki 19dxz23oyr8o27eoa7g97akza11k9vy 2456193 2456192 2026-09-09T20:04:19Z AndrewBogott (WMF) 51047 /* Example Template */ 2456193 wikitext text/x-wiki nm877iuemm9sz5d5hr9qz4p32hdfggw 2456194 2456193 2026-09-09T20:04:59Z AndrewBogott (WMF) 51047 /* Create a template */ 2456194 wikitext text/x-wiki h64k2i75g298ckc0ei3y5588mhbfrm5 0 452860 2456052 2443316 2026-09-09T14:10:50Z Krinkle 44 /* Phabricator */ 2456052 wikitext text/x-wiki 6gs5ap708z5w2f0z4joa28djn9vf6pl 12 452945 2456053 2455175 2026-09-09T14:11:06Z SSapaty (WMF) 28955 link to toolsadmin UI to get git repository url 2456053 wikitext text/x-wiki svar4cb06a6dz2qik0djdz7g37xkc25 116 453906 2456195 2455406 2026-09-09T20:12:13Z Gitlabaccountapprovalbot 37332 lufi2586 was rejected. 2456195 wikitext text/x-wiki jr6879aq8yd6p8rh37wjje9zf05d63w 2456338 2456195 2026-09-10T01:48:14Z Gitlabaccountapprovalbot 37332 tehkittycat was rejected. 2456338 wikitext text/x-wiki 7woy4jh6v0ifx1fjsk48ac22a3a3oil 2456427 2456338 2026-09-10T08:54:17Z Gitlabaccountapprovalbot 37332 nehap1706 was rejected. 2456427 wikitext text/x-wiki 07kveqkz4xvx7fblvho7mx5gk8laqo6 0 457955 2456041 2447712 2026-09-09T13:55:10Z DKertesz-WMF 59186 replace bast1003 with bast1004 2456041 wikitext text/x-wiki os7125gmnl3s8egaqxy5ml6xryqby2w 0 459087 2456116 2454095 2026-09-09T17:39:23Z GChoi-WMF 39250 /* Service Level Indicators (SLIs) */ 2456116 wikitext text/x-wiki szb9e08zrffh6qtlr0i9u84b94l3sfs 2456117 2456116 2026-09-09T17:40:10Z GChoi-WMF 39250 /* Service Level Indicators (SLIs) */ 2456117 wikitext text/x-wiki bot88869ihnr5iopm73w0uxqhom7yyf 2456155 2456117 2026-09-09T18:16:18Z GChoi-WMF 39250 /* Realistic targets */ 2456155 wikitext text/x-wiki hnh9kplk621o3qts7uq9s4ha1eprgmq 2456156 2456155 2026-09-09T18:17:17Z GChoi-WMF 39250 /* Ideal targets */ 2456156 wikitext text/x-wiki 3y642ll0iv0d2xts9tw6oyamq2kbmop 2456157 2456156 2026-09-09T18:18:17Z GChoi-WMF 39250 /* Reconciliation */ 2456157 wikitext text/x-wiki 4r0p9uezitndlmgtwv768uskkfhw01g 0 459560 2456055 2451465 2026-09-09T14:13:32Z DAdedoyin (WMF) 39150 Test 2456055 wikitext text/x-wiki gigu9yc7doykx9uilmdechhk5k01xht 0 459699 2456107 2455939 2026-09-09T17:06:58Z Stashbot 7414 ejegg: payments-wiki upgraded from af9803e8 to 59a9c9dc 2456107 wikitext text/x-wiki p64egq9wsrkpzw28mhif1473z4pxqyp 2456147 2456107 2026-09-09T18:01:44Z Stashbot 7414 ejegg: civicrm upgraded from 36a5aa98 to 8e39d645 2456147 wikitext text/x-wiki mjvfmvmmas8ii7r85290ap43n0z5620 2456250 2456147 2026-09-09T21:24:26Z Stashbot 7414 larssandergreen: payments-wiki upgraded from 59a9c9dc to 8c6757f8 2456250 wikitext text/x-wiki dxcyv37ibxo5eqz0h0hotlbtdge42m1 0 459966 2456423 2425850 2026-09-10T08:46:26Z Phuedx (WMF) 47850 WikimediaEvents now depends on TestKitchen 2456423 wikitext text/x-wiki rr8eyrlb20l7182qgxwgwybhchlyx6a 2456425 2456423 2026-09-10T08:48:59Z Phuedx (WMF) 47850 Promote the scenarios / examples to top-level sections as they aren't examples of CSS-only experiments per se 2456425 wikitext text/x-wiki 4tg3147vv6hlfq3rp6n2hirbvt4723t 0 460052 2456118 2450308 2026-09-09T17:43:32Z JVanderhoop-WMF 41069 typo fix 2456118 wikitext text/x-wiki s1nanoqqzepvg5kh1m8nie55a0ac1w4 0 460375 2456158 2452943 2026-09-09T18:19:40Z TBurmeister (WMF) 27417 Adjust intro 2456158 wikitext text/x-wiki hicbejko0fjz85q18vmo2n3y0kwo348 2456290 2456158 2026-09-09T22:37:15Z TBurmeister (WMF) 27417 Heavy first round of edits to restructure content, adjust formatting, add draft template because more edits needed 2456290 wikitext text/x-wiki d3vwd0r8lc4to9udylj6vgmfwmoddnr 0 460417 2456449 2453877 2026-09-10T11:31:23Z Kevin Bazira 14737 /* Endpoint */ add KServe predict endpoint 2456449 wikitext text/x-wiki 6fvc8yi3leufmdp1rj7hwf8wpnw1oea 0 460522 2456278 2445032 2026-09-09T22:29:18Z DLynch (WMF) 6438 Fill out information about deployment, dependencies, metrics 2456278 wikitext text/x-wiki pxhld63zkm9trhb0zv32gu8rrt92969 0 460590 2456307 2452018 2026-09-09T23:25:27Z AMastilovic-WMF 40067 /* Producers (produced_by) */ 2456307 wikitext text/x-wiki 9xw3318jq4n90rxx0a4360jsm1tilpt 2456308 2456307 2026-09-09T23:26:00Z AMastilovic-WMF 40067 /* How the sensor is chosen */ 2456308 wikitext text/x-wiki obx77ugj2x0fc35nfdrp8034080upoj 0 460646 2456037 2026-09-09T13:44:56Z BKing (WMF) 29017 Created page with "Placeholder" 2456037 wikitext text/x-wiki rp6ekfc3smhpzk3bym7bz0jugu65jzc 2 460647 2456042 2026-09-09T13:56:28Z Atsuko-WMF 54082 Created page with " == OIDC in K8S == https://phabricator.wikimedia.org/P96394 == OIDC in radosgw == * [https://github.com/ceph/ceph/blob/3d3b994d19cafbbb1d00e637bac58996c651fad5/src/rgw/rgw_rest_sts.cc#L290 Validation code] only uses audience * [https://docs.ceph.com/en/latest/radosgw/oidc/ Official doc] [https://github.com/ceph/ceph/blob/3d3b994d19cafbbb1d00e637bac58996c651fad5/doc/radosgw/oidc.rst Snapshot] == K8s + radosgw == * [https://github.com/fitbeard/radosgw-assume/blob/79f06a..." 2456042 wikitext text/x-wiki 66zgjq6x37gl4giooq8qlcaddmc606u 2456043 2456042 2026-09-09T14:03:58Z Atsuko-WMF 54082 /* OIDC in K8S */ 2456043 wikitext text/x-wiki ctfvr8nwipaw2gq98yhgxsxrpyrnvpf 498 460648 2456072 2026-09-09T15:16:08Z Labslogbot 55 Auto update of instance info. 2456072 wikitext text/x-wiki ldrca6nhwz9k41i4khnp7qzarx1rl6o 12 460649 2456086 2026-09-09T16:01:50Z Martin Urbanec (WMF) 37730 Created page with " +---------+---------+-----------------------------------------------------+ | Cipher | Algo | Fingerprint | +---------+---------+-----------------------------------------------------+ | RSA | SHA-256 | SHA256:yB8fNHP/PuUZAf7qeVb6m5c6sg1jsDXE3psF926HMPI | +---------+---------+-----------------------------------------------------+ | ECDSA | SHA-256 | SHA256:bqYTy5gYXsfnhHOy9lrFZg++BbFPjZmG7QZKqxK4M74 | +---------+-..." 2456086 wikitext text/x-wiki t5g6egure20pm49uwbzdskcagmqpakq 2456087 2456086 2026-09-09T16:02:03Z Martin Urbanec (WMF) 37730 Martin Urbanec (WMF) changed the content model of the page [[Help:SSH Fingerprints/mwlog1003.eqiad.wmnet]] from "wikitext" to "plain text" 2456087 text text/plain t5g6egure20pm49uwbzdskcagmqpakq 2456088 2456087 2026-09-09T16:02:07Z Martin Urbanec (WMF) 37730 Protected "[[Help:SSH Fingerprints/mwlog1003.eqiad.wmnet]]": Protecting key ([Edit=Allow only administrators] (indefinite) [Move=Allow only administrators] (indefinite)) 2456087 text text/plain t5g6egure20pm49uwbzdskcagmqpakq