Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.48.0-wmf.3 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 0 3818 2477255 2474964 2026-11-04T09:48:00Z FFurnari-WMF 36523 /* Volatile mount */ 2477255 wikitext text/x-wiki 4yfp1uplfsn3f0lh6wcr0ew071b0gcc 0 4039 2476848 2455824 2026-11-03T16:13:47Z SBassett (WMF) 13019 /* Guidelines for creating patches */ Add stricter guidance around i18n/unit tests within security patches. 2476848 wikitext text/x-wiki e3p1vewodatq8jnhkky5a7cd32g0jjq 2476893 2476848 2026-11-03T18:11:28Z WBrown (WMF) 37696 + 2476893 wikitext text/x-wiki 5s1p0e3qs413lix73hlng5le4bfi4bj 2476895 2476893 2026-11-03T18:14:19Z WBrown (WMF) 37696 more reasoning 2476895 wikitext text/x-wiki 4glwrcz0yojs4hnx5g9h56jkklzsurn 2477010 2476895 2026-11-03T22:43:21Z Jdforrester (WMF) 468 /* Guidelines for creating patches */ Add the consequence of not making your security patch pass CI. 2477010 wikitext text/x-wiki eb2pl3pf7jrpetebjw029n5qpmt3q3k 0 4108 2476770 2476704 2026-11-03T13:29:44Z ScheduleDeploymentBot 37566 Add [[gerrit:1361885]] to Tuesday, November 03 UTC afternoon backport window 2476770 wikitext text/x-wiki 9eaz9hvkdzjd6uu7k1tkmj5mkcq8euj 2476772 2476770 2026-11-03T13:30:02Z ScheduleDeploymentBot 37566 Add [[gerrit:1361888]] to Tuesday, November 03 UTC afternoon backport window 2476772 wikitext text/x-wiki jhmpiswpge0a7t1czoxjlu2aspen0kv 2476780 2476772 2026-11-03T13:34:30Z ScheduleDeploymentBot 37566 Add [[gerrit:1361887]] to Tuesday, November 03 UTC afternoon backport window 2476780 wikitext text/x-wiki tjhxh0rt44o8xvwd57c36snvhvz27dk 2476820 2476780 2026-11-03T15:28:11Z ScheduleDeploymentBot 37566 Add [[gerrit:1362080]] to Wednesday, November 04 UTC afternoon backport window 2476820 wikitext text/x-wiki 05gqbw5qddzyf4vd9ml801g7f85o36j 2476876 2476820 2026-11-03T17:05:00Z ScheduleDeploymentBot 37566 Add [[gerrit:1362118]] to Wednesday, November 04 UTC afternoon backport window 2476876 wikitext text/x-wiki boosq1wn3xgir41v7nmattzbuw0mcor 2476877 2476876 2026-11-03T17:05:33Z ScheduleDeploymentBot 37566 Add [[gerrit:1357358]] to Wednesday, November 04 UTC afternoon backport window 2476877 wikitext text/x-wiki fdbc25frl6snrgaxzz6c20xla04ojho 2476878 2476877 2026-11-03T17:06:32Z ScheduleDeploymentBot 37566 Add [[gerrit:1361447]] to Tuesday, November 03 UTC late backport window 2476878 wikitext text/x-wiki rpjrl19k54bb95lfsci99laene16ein 2476879 2476878 2026-11-03T17:06:42Z ScheduleDeploymentBot 37566 Add [[gerrit:1355485]] to Tuesday, November 03 UTC late backport window 2476879 wikitext text/x-wiki 0jw44kq8g57guhjbhdh27o0cej9xyvu 2476920 2476879 2026-11-03T18:29:19Z ScheduleDeploymentBot 37566 Add [[gerrit:1362144]] to Wednesday, November 04 UTC afternoon backport window 2476920 wikitext text/x-wiki moschtqqo6frfvdcr6hbdz6edlro0so 2476942 2476920 2026-11-03T19:08:49Z TCipriani (WMF) 4895 /* Week of November 02 */ 2476942 wikitext text/x-wiki oqvhahhg3wpoxipon2x0upk50iahkbr 2476980 2476942 2026-11-03T20:56:24Z ScheduleDeploymentBot 37566 Add [[gerrit:1362185]] to Tuesday, November 03 UTC late backport window 2476980 wikitext text/x-wiki sr3k4scgjcw15ez4c65sa5hqp4pv8v8 498 6726 2477299 2475461 2026-11-04T11:14:43Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component image-config 2477299 wikitext text/x-wiki qdv9kgk6qeu0nfmxt3ekmi2yy6cfk4c 2477307 2477299 2026-11-04T11:19:28Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component image-config 2477307 wikitext text/x-wiki rnfekk004f4ja6atmwu6o33do2406rj 2477308 2477307 2026-11-04T11:19:38Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component image-config 2477308 wikitext text/x-wiki nlvx0s30fb5i9292l9pivt18gds7fwk 2477315 2477308 2026-11-04T11:29:47Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component image-config 2477315 wikitext text/x-wiki 5xnu28bqej4zpgsw51ubxva6p0ignrs 0 7919 2476728 2476727 2026-11-03T12:02:21Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1233', diff saved to https://phabricator.wikimedia.org/P97122 and previous config saved to /var/cache/conftool/dbconfig/20261103-120219-ladsgroup.json 2476728 wikitext text/x-wiki 4jkln7vikpugw54v2sz6yelhsx3dgcd 2476735 2476728 2026-11-03T12:05:43Z Stashbot 7414 cmooney@cumin1004: START - Cookbook sre.hosts.dhcp for host sretest1006.eqiad.wmnet (remove config only) 2476735 wikitext text/x-wiki rsrenprwubqnv7s9q1dl2a6cvv96zk2 2476736 2476735 2026-11-03T12:05:47Z Stashbot 7414 cmooney@cumin1004: END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host sretest1006.eqiad.wmnet (remove config only) 2476736 wikitext text/x-wiki 1jns8uy0q56hn3q73yavsdy4458kefp 2476738 2476736 2026-11-03T12:07:58Z Stashbot 7414 cmooney@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on sretest1006.eqiad.wmnet with reason: host reimage 2476738 wikitext text/x-wiki 2ov1w45t23ymv334gsr5vayfwi2ed7r 2476741 2476738 2026-11-03T12:12:24Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1233', diff saved to https://phabricator.wikimedia.org/P97123 and previous config saved to /var/cache/conftool/dbconfig/20261103-121221-ladsgroup.json 2476741 wikitext text/x-wiki mrur4er2r31o8arh9doqfpwm72jcb1q 2476742 2476741 2026-11-03T12:12:52Z Stashbot 7414 cmooney@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on sretest1006.eqiad.wmnet with reason: host reimage 2476742 wikitext text/x-wiki 7r7sj7j1kac1t5qq4wzgw5mikmriquc 2476743 2476742 2026-11-03T12:22:26Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1233 (T441633)', diff saved to https://phabricator.wikimedia.org/P97124 and previous config saved to /var/cache/conftool/dbconfig/20261103-122223-ladsgroup.json 2476743 wikitext text/x-wiki lp2o8pppynepc0svuo4utuxsiw1nqip 2476744 2476743 2026-11-03T12:22:44Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1239.eqiad.wmnet with reason: Maintenance 2476744 wikitext text/x-wiki n7ukj7olmknnph5rzaa8fkse2fsle6z 2476745 2476744 2026-11-03T12:28:03Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1254.eqiad.wmnet with reason: Maintenance 2476745 wikitext text/x-wiki blcp5duvxuzv41kw8fxme01np0v1mdd 2476746 2476745 2026-11-03T12:28:06Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1254 (T441633)', diff saved to https://phabricator.wikimedia.org/P97125 and previous config saved to /var/cache/conftool/dbconfig/20261103-122803-ladsgroup.json 2476746 wikitext text/x-wiki p11bic56968zlihune5hqsd6remwn48 2476747 2476746 2026-11-03T12:28:52Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host irc1003.wikimedia.org 2476747 wikitext text/x-wiki er3h2ew79r5vaiwc0xaf8s9ako9v7zl 2476748 2476747 2026-11-03T12:31:01Z Stashbot 7414 cmooney@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host sretest1006.eqiad.wmnet with OS trixie 2476748 wikitext text/x-wiki am48173xqveol6t601ujmuj6z0dgo62 2476749 2476748 2026-11-03T12:32:43Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host irc1003.wikimedia.org 2476749 wikitext text/x-wiki hyb1zbyzbrouttz18aesfrocnfjh3xu 2476750 2476749 2026-11-03T12:34:06Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1254 (T441633)', diff saved to https://phabricator.wikimedia.org/P97126 and previous config saved to /var/cache/conftool/dbconfig/20261103-123403-ladsgroup.json 2476750 wikitext text/x-wiki 0bm696yah6z5dvoc0wx8hlr09ju6tne 2476751 2476750 2026-11-03T12:44:08Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1254', diff saved to https://phabricator.wikimedia.org/P97127 and previous config saved to /var/cache/conftool/dbconfig/20261103-124405-ladsgroup.json 2476751 wikitext text/x-wiki fj5rlidtma8ejmw56ffup8fxpnkpurc 2476752 2476751 2026-11-03T12:54:10Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1254', diff saved to https://phabricator.wikimedia.org/P97128 and previous config saved to /var/cache/conftool/dbconfig/20261103-125407-ladsgroup.json 2476752 wikitext text/x-wiki qr10ozmbnhu4045qnv6v4cxa51ofhfx 2476753 2476752 2026-11-03T13:04:12Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1254 (T441633)', diff saved to https://phabricator.wikimedia.org/P97129 and previous config saved to /var/cache/conftool/dbconfig/20261103-130409-ladsgroup.json 2476753 wikitext text/x-wiki rf6vj23ifioltsuvmxwcpt68vrbi1kn 2476754 2476753 2026-11-03T13:04:19Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1259.eqiad.wmnet with reason: Maintenance 2476754 wikitext text/x-wiki munw8v94t14lcogap9usnkfkcb604pz 2476755 2476754 2026-11-03T13:04:22Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1259 (T441633)', diff saved to https://phabricator.wikimedia.org/P97130 and previous config saved to /var/cache/conftool/dbconfig/20261103-130419-ladsgroup.json 2476755 wikitext text/x-wiki fs22dtx227hfb4oqwtinll0lsm4mirw 2476756 2476755 2026-11-03T13:10:32Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1259 (T441633)', diff saved to https://phabricator.wikimedia.org/P97131 and previous config saved to /var/cache/conftool/dbconfig/20261103-131029-ladsgroup.json 2476756 wikitext text/x-wiki hzp8pzob9yyu4ltlic0p6hu92z4prtd 2476758 2476756 2026-11-03T13:20:34Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1259', diff saved to https://phabricator.wikimedia.org/P97132 and previous config saved to /var/cache/conftool/dbconfig/20261103-132031-ladsgroup.json 2476758 wikitext text/x-wiki j14qqnt6pu2mj6u9kte9g4oezlwr4nw 2476759 2476758 2026-11-03T13:20:43Z Stashbot 7414 jayme@cumin1004: START - Cookbook sre.k8s.upgrade-apiservers-inplace Upgrade apiservers of staging-codfw to 1.34.12 2476759 wikitext text/x-wiki 8xgl65zfi38jufxvoj0t8vo9qbx7ccm 2476760 2476759 2026-11-03T13:20:53Z Stashbot 7414 jayme@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host kubestagemaster[2004-2005].codfw.wmnet 2476760 wikitext text/x-wiki qcb9vkcfqrqu7u2nvrm4ckd9jsz2ocx 2476761 2476760 2026-11-03T13:22:34Z Stashbot 7414 kamila@deploy2003: Started scap deploy-service shellbox: update to 8.5 + new image 2476761 wikitext text/x-wiki 1c0bo9tz0raqhtjcr0bhnxcpow9wsea 2476762 2476761 2026-11-03T13:22:37Z Stashbot 7414 kamila@deploy2003: scap deploy-service shellbox: [STAGING/DEFAULT] staging-eqiad Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2476762 wikitext text/x-wiki l7kbl82a6x027o230fkmr51i9za79bx 2476763 2476762 2026-11-03T13:23:43Z Stashbot 7414 kamila@deploy2003: scap deploy-service shellbox: [PRODUCTION/DEFAULT] eqiad Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2476763 wikitext text/x-wiki ibmapa5rhu62ty3bydvzpohc0gmi7po 2476764 2476763 2026-11-03T13:25:22Z Stashbot 7414 kamila@deploy2003: scap deploy-service shellbox: [PRODUCTION/DEFAULT] codfw Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2476764 wikitext text/x-wiki l2lq1j5toitcf6vkiuh1jgiv1k5l9ek 2476765 2476764 2026-11-03T13:27:19Z Stashbot 7414 bwojtowicz@deploy2003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . 2476765 wikitext text/x-wiki eyv5ie9srf59fn4dy821pft7wn8l3c5 2476766 2476765 2026-11-03T13:27:24Z Stashbot 7414 bwojtowicz@deploy2003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'llmisvc' . 2476766 wikitext text/x-wiki hf2kxh367xg582vkqehaxsdj403dy5e 2476767 2476766 2026-11-03T13:27:35Z Stashbot 7414 kamila@deploy2003: Finished scap deploy-service shellbox: update to 8.5 + new image (duration: 05m 00s) 2476767 wikitext text/x-wiki qfjy9fj823xpygun5ikk72ldej5949v 2476768 2476767 2026-11-03T13:27:52Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply 2476768 wikitext text/x-wiki kghb2hzq7wb6yowp84x7qbc40ryfvp6 2476769 2476768 2026-11-03T13:28:57Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply 2476769 wikitext text/x-wiki spfr3kpka46hh5tqorwstrsoqyv1p19 2476771 2476769 2026-11-03T13:29:53Z Stashbot 7414 jnuche@deploy2003: Started scap sync-world: testwikis to 1.48.0-wmf.3 refs T442221 2476771 wikitext text/x-wiki j877qmq20zt6tr654thfuoa8k1xx27k 2476773 2476771 2026-11-03T13:30:36Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1259', diff saved to https://phabricator.wikimedia.org/P97133 and previous config saved to /var/cache/conftool/dbconfig/20261103-133034-ladsgroup.json 2476773 wikitext text/x-wiki p7yzucapis2trjhga09xit23taufzi6 2476774 2476773 2026-11-03T13:31:18Z Stashbot 7414 jayme@cumin1004: END (ERROR) - Cookbook sre.k8s.pool-depool-node (exit_code=97) depool for host kubestagemaster[2004-2005].codfw.wmnet 2476774 wikitext text/x-wiki ge64qks0j0bqlijbquk3au8nqycgnk5 2476775 2476774 2026-11-03T13:31:26Z Stashbot 7414 jayme@cumin1004: END (FAIL) - Cookbook sre.k8s.upgrade-apiservers-inplace (exit_code=99) Upgrade apiservers of staging-codfw to 1.34.12 2476775 wikitext text/x-wiki fhq3zc5ks5rz55b0f9363iae2x21bul 2476776 2476775 2026-11-03T13:33:46Z Stashbot 7414 jayme@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host kubestagemaster[2004-2005].codfw.wmnet 2476776 wikitext text/x-wiki 984sqzncvgx5oion80gqmio1ncie44k 2476777 2476776 2026-11-03T13:33:48Z Stashbot 7414 jayme@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host kubestagemaster[2004-2005].codfw.wmnet 2476777 wikitext text/x-wiki onsxnqz5lglhv5fptg9dl5anq8auvc1 2476778 2476777 2026-11-03T13:34:08Z Stashbot 7414 jayme@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host kubestagemaster[2004-2005].codfw.wmnet 2476778 wikitext text/x-wiki gdga0bc3mx0ufptd1ivniy09kab8ovl 2476779 2476778 2026-11-03T13:34:10Z Stashbot 7414 jayme@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host kubestagemaster[2004-2005].codfw.wmnet 2476779 wikitext text/x-wiki 7kksf7r6lln9ctq45fwfrnk0e4d608i 2476781 2476779 2026-11-03T13:34:31Z Stashbot 7414 jayme@cumin1004: START - Cookbook sre.k8s.upgrade-apiservers-inplace Upgrade apiservers of staging-codfw to 1.34.12 2476781 wikitext text/x-wiki fxz57pb20ikfv8os3ujzzbsfoa68kas 2476782 2476781 2026-11-03T13:34:40Z Stashbot 7414 jayme@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host kubestagemaster[2004-2005].codfw.wmnet 2476782 wikitext text/x-wiki nvvu7k48b4o025z3s6kw0hxuklhz9rb 2476783 2476782 2026-11-03T13:34:43Z Stashbot 7414 jayme@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host kubestagemaster[2004-2005].codfw.wmnet 2476783 wikitext text/x-wiki d2a69kahnsvqaaqzz8ktjazak832jzx 2476784 2476783 2026-11-03T13:40:38Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1259 (T441633)', diff saved to https://phabricator.wikimedia.org/P97134 and previous config saved to /var/cache/conftool/dbconfig/20261103-134036-ladsgroup.json 2476784 wikitext text/x-wiki r8qmsmp1qa00m0f52m1lx4xvw2g6f93 2476785 2476784 2026-11-03T13:40:56Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1271.eqiad.wmnet with reason: Maintenance 2476785 wikitext text/x-wiki 30wgphojtkafhn6b9k8wysqwfxqlby2 2476786 2476785 2026-11-03T13:41:18Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on 7 hosts with reason: Maintenance 2476786 wikitext text/x-wiki bmg00p7fwwvs1tyw41e55ru2xodsz9d 2476787 2476786 2026-11-03T13:41:21Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1271 (T441633)', diff saved to https://phabricator.wikimedia.org/P97135 and previous config saved to /var/cache/conftool/dbconfig/20261103-134118-ladsgroup.json 2476787 wikitext text/x-wiki sy6cbpsk4q9sgwux4cikl4fa9ivanad 2476789 2476787 2026-11-03T13:42:52Z Stashbot 7414 jayme@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host kubestagemaster[2004-2005].codfw.wmnet 2476789 wikitext text/x-wiki 1260rgvfc3pa3j6mfowspxu5hekb1z1 2476790 2476789 2026-11-03T13:42:56Z Stashbot 7414 jayme@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host kubestagemaster[2004-2005].codfw.wmnet 2476790 wikitext text/x-wiki mo9uijdrg7ykhkuyt2w8vyu5wy2g468 2476791 2476790 2026-11-03T13:42:58Z Stashbot 7414 jayme@cumin1004: END (PASS) - Cookbook sre.k8s.upgrade-apiservers-inplace (exit_code=0) Upgrade apiservers of staging-codfw to 1.34.12 2476791 wikitext text/x-wiki 1x5ntjcox6g0jrdatmlmwovefq6zb5d 2476792 2476791 2026-11-03T13:45:23Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1271 (T441633)', diff saved to https://phabricator.wikimedia.org/P97136 and previous config saved to /var/cache/conftool/dbconfig/20261103-134519-ladsgroup.json 2476792 wikitext text/x-wiki d7jt38i9c9lpzx1gck42vqahgpfav3f 2476793 2476792 2026-11-03T13:47:30Z Stashbot 7414 dpogorzelski@deploy2003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'llm' . 2476793 wikitext text/x-wiki kwsowizzvb8q5olh4mvs66dztmc8nyq 2476794 2476793 2026-11-03T13:55:23Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1271', diff saved to https://phabricator.wikimedia.org/P97137 and previous config saved to /var/cache/conftool/dbconfig/20261103-135521-ladsgroup.json 2476794 wikitext text/x-wiki aqz6iirw7fiphdz3chehjxebhcbo0a6 2476796 2476794 2026-11-03T13:59:52Z Stashbot 7414 jelto@cumin1004: START - Cookbook sre.k8s.upgrade-kubelets-inplace Upgrade kubelet of staging-codfw to 1.34.12 2476796 wikitext text/x-wiki futg3nqwcrtr85uba96p1ke1kjbpcnv 2476797 2476796 2026-11-03T14:00:08Z Stashbot 7414 jelto@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host kubestage2001.codfw.wmnet 2476797 wikitext text/x-wiki 3e4xp3puc49ibfq00jcp4ptz4kxqlbn 2476798 2476797 2026-11-03T14:00:39Z Stashbot 7414 jelto@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host kubestage2001.codfw.wmnet 2476798 wikitext text/x-wiki n9xj1recl25g0ppa8rg4jdy3cuhkqbr 2476799 2476798 2026-11-03T14:59:36Z Stashbot 7414 samtar@deploy2003: Finished scap sync-world: Backport for [[gerrit:1355072|Add Draft/Draft_talk namespace aliases on wikis using ns 118 as draft namespace (T440179)]], [[gerrit:1361887|Add missing groupOverrides for sysop and bot on ltwiki. (T440216)]] (duration: 21m 43s) 2476799 wikitext text/x-wiki o4mpmc7d0xng32tecuiyxpbtj70y6qh 2476800 2476799 2026-11-03T15:00:14Z Stashbot 7414 klausman@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host ml-serve1013.eqiad.wmnet 2476800 wikitext text/x-wiki 7fdkadhyl029hoigkrndmpfuqhrrc82 2476802 2476800 2026-11-03T15:02:05Z Stashbot 7414 samtar@deploy2003: Started scap sync-world: Backport for Squashed diff to master ([[gerrit:1361885|1.48.0-wmf.1]], [[gerrit:1361888|1.48.0-wmf.3]]), [[gerrit:1362064|Revert "Enable NewUserMessage for viwikivoyage"]] 2476802 wikitext text/x-wiki i139ovpa6iav5rdsq5so35v7c45e8fr 2476804 2476802 2026-11-03T15:06:10Z Stashbot 7414 klausman@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host ml-serve1013.eqiad.wmnet 2476804 wikitext text/x-wiki 8fdbyzgijouttqaennh9tl6fpzrdki3 2476805 2476804 2026-11-03T15:06:13Z Stashbot 7414 klausman@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host ml-serve1013.eqiad.wmnet 2476805 wikitext text/x-wiki r2txv55kfj42w0h4124iout299ps9sy 2476806 2476805 2026-11-03T15:06:16Z Stashbot 7414 klausman@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host ml-serve1014.eqiad.wmnet 2476806 wikitext text/x-wiki sbtdctqmhn910jdeu7sbm1y3s9kz3gp 2476808 2476806 2026-11-03T15:07:31Z Stashbot 7414 cwilliams@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on es2035.codfw.wmnet with reason: Maintenance 2476808 wikitext text/x-wiki p5c8sy2dj64bogdgtphab7uu6zoa9r3 2476809 2476808 2026-11-03T15:09:53Z Stashbot 7414 slyngshede@cumin1004: END (PASS) - Cookbook sre.discovery.datacenter (exit_code=0) pool all active/active services in eqiad: Datacenter services switchover - T439676 2476809 wikitext text/x-wiki 84fdckulhntgndk02o9h348rl3t28qj 2476810 2476809 2026-11-03T15:10:42Z Stashbot 7414 cwilliams@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on es2036.codfw.wmnet with reason: Maintenance 2476810 wikitext text/x-wiki 10g099r7tz49ku8fuhq2hdtxtrxoyht 2476812 2476810 2026-11-03T15:12:21Z Stashbot 7414 cwilliams@cumin1004: START - Cookbook sre.hosts.remove-downtime for es2035.codfw.wmnet 2476812 wikitext text/x-wiki pxsbynwy63pvz2wi5rnq35l1y5s8p7e 2476813 2476812 2026-11-03T15:12:24Z Stashbot 7414 cwilliams@cumin1004: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for es2035.codfw.wmnet 2476813 wikitext text/x-wiki t5x8grymk61tf9tx2nkt10okm89i1c1 2476815 2476813 2026-11-03T15:16:21Z Stashbot 7414 klausman@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host ml-serve1014.eqiad.wmnet 2476815 wikitext text/x-wiki cm4yvc5tyxodwn89d5c5r4lz4bu089v 2476816 2476815 2026-11-03T15:19:36Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply 2476816 wikitext text/x-wiki noddicudmbsehxwk6itgo96bbcy2kky 2476817 2476816 2026-11-03T15:20:48Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply 2476817 wikitext text/x-wiki 2cprj75ti6kbdl1f8wghleixrssfyhy 2476818 2476817 2026-11-03T15:23:19Z Stashbot 7414 samtar@deploy2003: samtar, mlitn: Backport for Squashed diff to master ([[gerrit:1361885|1.48.0-wmf.1]], [[gerrit:1361888|1.48.0-wmf.3]]), [[gerrit:1362064|Revert "Enable NewUserMessage for viwikivoyage"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2476818 wikitext text/x-wiki 906xk067wygqsr1wgcbzown7kl10gww 2476819 2476818 2026-11-03T15:23:59Z Stashbot 7414 samtar@deploy2003: samtar, mlitn: Continuing with deployment 2476819 wikitext text/x-wiki 5y5wch05iv4e1gocpr5rdgny3qcfzep 2476821 2476819 2026-11-03T15:32:31Z Stashbot 7414 jelto@cumin1004: END (FAIL) - Cookbook sre.k8s.upgrade-kubelets-inplace (exit_code=1) Upgrade kubelet of staging-codfw to 1.34.12 2476821 wikitext text/x-wiki 3wf9v8ywex2g28fb5c3ar3v6kmkgazu 2476822 2476821 2026-11-03T15:35:52Z Stashbot 7414 samtar@deploy2003: Finished scap sync-world: Backport for Squashed diff to master ([[gerrit:1361885|1.48.0-wmf.1]], [[gerrit:1361888|1.48.0-wmf.3]]), [[gerrit:1362064|Revert "Enable NewUserMessage for viwikivoyage"]] (duration: 33m 48s) 2476822 wikitext text/x-wiki 53venk6qsa0krv1iu1934a9q4znnzc2 2476823 2476822 2026-11-03T15:35:57Z Stashbot 7414 jayme@cumin1004: START - Cookbook sre.k8s.upgrade-apiservers-inplace Upgrade apiservers of staging-codfw to 1.31.14 2476823 wikitext text/x-wiki 5qm48usasa0q8mnk1jdz2pjf5hgyv0q 2476824 2476823 2026-11-03T15:36:19Z Stashbot 7414 jayme@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host kubestagemaster[2004-2005].codfw.wmnet 2476824 wikitext text/x-wiki ggv5lutr3p3i54wo8swo54g5u0l7wog 2476825 2476824 2026-11-03T15:36:22Z Stashbot 7414 jayme@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host kubestagemaster[2004-2005].codfw.wmnet 2476825 wikitext text/x-wiki 43666xbllrurd1sam00d2ah1t3k8t3n 2476826 2476825 2026-11-03T15:36:48Z Stashbot 7414 dancy@deploy2003: Installing scap version "4.311.0" for 155 host(s) 2476826 wikitext text/x-wiki sf0xnfkrg0lunz817ozimdfyyrfqsbp 2476827 2476826 2026-11-03T15:39:48Z Stashbot 7414 jayme@cumin1004: END (FAIL) - Cookbook sre.k8s.upgrade-apiservers-inplace (exit_code=99) Upgrade apiservers of staging-codfw to 1.31.14 2476827 wikitext text/x-wiki j3qevke5g4m2sb22u72wm0jfp4p10st 2476828 2476827 2026-11-03T15:40:46Z Stashbot 7414 dancy@deploy2003: Installation of scap version "4.311.0" completed for 155 hosts 2476828 wikitext text/x-wiki hdtb79nrumxfxlg1hpaek8z1enx2k2b 2476829 2476828 2026-11-03T15:42:29Z Stashbot 7414 otto@deploy2003: Started scap deploy-service eventgate-analytics: v1.38.0 for T440968 2476829 wikitext text/x-wiki ah72mrms9h1im7dqymp27nucxbp5fbt 2476830 2476829 2026-11-03T15:42:32Z Stashbot 7414 otto@deploy2003: scap deploy-service eventgate-analytics: [STAGING/DEFAULT] staging-eqiad Deploying eventgate-analytics 2476830 wikitext text/x-wiki rszw7txaxk0awywl80ux3o3ph125ya9 2476831 2476830 2026-11-03T15:44:06Z Stashbot 7414 otto@deploy2003: scap deploy-service eventgate-analytics: [PRODUCTION/CANARY] eqiad Deploying eventgate-analytics 2476831 wikitext text/x-wiki ommgcp5v010ilz51c3vo4rv7o3mvnrj 2476832 2476831 2026-11-03T15:44:32Z Stashbot 7414 otto@deploy2003: scap deploy-service eventgate-analytics: [PRODUCTION/CANARY] codfw Deploying eventgate-analytics 2476832 wikitext text/x-wiki 0eass9s3kg39j1nec9kop0d3xlzslby 2476833 2476832 2026-11-03T15:44:56Z Stashbot 7414 otto@deploy2003: scap deploy-service eventgate-analytics: [PRODUCTION/DEFAULT] eqiad Deploying eventgate-analytics 2476833 wikitext text/x-wiki h09vlmqu3rt3lsiwy64xn3zyrtlgsp0 2476834 2476833 2026-11-03T15:46:14Z Stashbot 7414 jayme@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host kubestagemaster[2003-2005].codfw.wmnet 2476834 wikitext text/x-wiki 6yjyajrca0rbulvm8z8z1ri9tgeptbm 2476835 2476834 2026-11-03T15:46:17Z Stashbot 7414 jayme@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host kubestagemaster[2003-2005].codfw.wmnet 2476835 wikitext text/x-wiki 3w8zcjik708fftkcuya9ag05x0lmzgd 2476836 2476835 2026-11-03T15:53:05Z Stashbot 7414 otto@deploy2003: scap deploy-service eventgate-analytics: [PRODUCTION/DEFAULT] codfw Deploying eventgate-analytics 2476836 wikitext text/x-wiki 6u8wnz4ls67cwptmio6u30ab6jas94s 2476837 2476836 2026-11-03T15:53:54Z Stashbot 7414 otto@deploy2003: Finished scap deploy-service eventgate-analytics: v1.38.0 for T440968 (duration: 11m 24s) 2476837 wikitext text/x-wiki 4xl53mav7mhjljcw8j78ddykcgwvhyi 2476838 2476837 2026-11-03T16:05:01Z Stashbot 7414 klausman@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host ml-serve1014.eqiad.wmnet 2476838 wikitext text/x-wiki a5f41iuc9hu7k8pdivmjv244qzh2ow3 2476839 2476838 2026-11-03T16:05:05Z Stashbot 7414 klausman@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host ml-serve1014.eqiad.wmnet 2476839 wikitext text/x-wiki memzsj7aitpfw0nt9zzndedsrfkc7fo 2476840 2476839 2026-11-03T16:05:08Z Stashbot 7414 klausman@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host ml-serve1015.eqiad.wmnet 2476840 wikitext text/x-wiki adinn2sns6ee7mzpb71pdvaz09twzcf 2476841 2476840 2026-11-03T16:05:29Z Stashbot 7414 ozge@deploy2003: helmfile [staging] START helmfile.d/services/linkrecommendation: apply 2476841 wikitext text/x-wiki s421kvkg835cmcriqgxwoeyt2on0oyn 2476842 2476841 2026-11-03T16:06:39Z Stashbot 7414 ozge@deploy2003: helmfile [staging] DONE helmfile.d/services/linkrecommendation: apply 2476842 wikitext text/x-wiki ca6guqkgynkmyfmk86jtvp58nnzh6tv 2476843 2476842 2026-11-03T16:08:58Z Stashbot 7414 ozge@deploy2003: helmfile [eqiad] START helmfile.d/services/linkrecommendation: apply 2476843 wikitext text/x-wiki eri52vgry2iefiqefwghar2jdzacq95 2476844 2476843 2026-11-03T16:10:14Z Stashbot 7414 klausman@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host ml-serve1015.eqiad.wmnet 2476844 wikitext text/x-wiki t73l4l53gjva0cczklyke5ifqj54b4t 2476845 2476844 2026-11-03T16:10:37Z Stashbot 7414 ozge@deploy2003: helmfile [eqiad] DONE helmfile.d/services/linkrecommendation: apply 2476845 wikitext text/x-wiki njzq0amn362tpt47hoqj40pld04iklq 2476846 2476845 2026-11-03T16:11:59Z Stashbot 7414 ozge@deploy2003: helmfile [codfw] START helmfile.d/services/linkrecommendation: apply 2476846 wikitext text/x-wiki 2hpzjzory597urmqof9jirthhjc7sjq 2476847 2476846 2026-11-03T16:13:41Z Stashbot 7414 ozge@deploy2003: helmfile [codfw] DONE helmfile.d/services/linkrecommendation: apply 2476847 wikitext text/x-wiki 4b7wkhlvas8glj6ef7arglvvoph3czs 2476849 2476847 2026-11-03T16:16:13Z Stashbot 7414 klausman@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host ml-serve1015.eqiad.wmnet 2476849 wikitext text/x-wiki ap5w9373lcv5s8x7w29knq5guc3jnfk 2476850 2476849 2026-11-03T16:16:15Z Stashbot 7414 klausman@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host ml-serve1015.eqiad.wmnet 2476850 wikitext text/x-wiki d1fw51506yzkgrdfjxnk8473ahubtqp 2476851 2476850 2026-11-03T16:16:18Z Stashbot 7414 klausman@cumin1004: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on P{ml-serve101[2-5]*} and (A:ml-serve-master-eqiad or A:ml-serve-worker-eqiad) 2476851 wikitext text/x-wiki 95im4sj7gqsspihq89952vu1ptdf3ql 2476855 2476851 2026-11-03T16:24:39Z Stashbot 7414 otto@deploy2003: Started scap deploy-service eventgate-all: v1.38.0 for T440968 2476855 wikitext text/x-wiki gcckx664wkepje1az74sac5hdiaks0n 2476856 2476855 2026-11-03T16:24:42Z Stashbot 7414 otto@deploy2003: scap deploy-service eventgate-all: [STAGING/DEFAULT] staging-eqiad Deploying eventgate-analytics, eventgate-analytics-external, eventgate-logging-external, eventgate-main 2476856 wikitext text/x-wiki f984defx0rr9kd9jnlcasvsa7vv1z5c 2476857 2476856 2026-11-03T16:26:39Z Stashbot 7414 otto@deploy2003: scap deploy-service eventgate-all: [PRODUCTION/CANARY] eqiad Deploying eventgate-analytics, eventgate-analytics-external, eventgate-logging-external, eventgate-main 2476857 wikitext text/x-wiki glgejhkws0n4ew21usf559ibjdqu6jj 2476858 2476857 2026-11-03T16:26:56Z Stashbot 7414 dancy@deploy2003: Installing scap version "4.312.0" for 2 host(s) 2476858 wikitext text/x-wiki p78y5fcac43v8snguidxzdt3qnoorbf 2476859 2476858 2026-11-03T16:27:09Z Stashbot 7414 otto@deploy2003: scap deploy-service eventgate-all: [PRODUCTION/CANARY] codfw Deploying eventgate-analytics, eventgate-analytics-external, eventgate-logging-external, eventgate-main 2476859 wikitext text/x-wiki nrf2c4dwazqv4t64een0m0tlktfyj7m 2476860 2476859 2026-11-03T16:27:36Z Stashbot 7414 otto@deploy2003: scap deploy-service eventgate-all: [PRODUCTION/DEFAULT] eqiad Deploying eventgate-analytics, eventgate-analytics-external, eventgate-logging-external, eventgate-main 2476860 wikitext text/x-wiki 1ztc2e3xs5kiycu8a7s7qbjxjp1pa5i 2476861 2476860 2026-11-03T16:28:59Z Stashbot 7414 dancy@deploy2003: Installation of scap version "4.312.0" completed for 2 hosts 2476861 wikitext text/x-wiki hyvchd30g2e960kusax9eu7zck0jg0l 2476862 2476861 2026-11-03T16:36:27Z Stashbot 7414 otto@deploy2003: scap deploy-service eventgate-all: [PRODUCTION/DEFAULT] codfw Deploying eventgate-analytics, eventgate-analytics-external, eventgate-logging-external, eventgate-main 2476862 wikitext text/x-wiki kp17soqdbf717roekfn8sp668c0rhrl 2476863 2476862 2026-11-03T16:36:58Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host netbox-dev2003.codfw.wmnet 2476863 wikitext text/x-wiki 2genrxqqhlnbtq7e7zxv4t8tys0fwuw 2476865 2476863 2026-11-03T16:39:22Z Stashbot 7414 otto@deploy2003: Finished scap deploy-service eventgate-all: v1.38.0 for T440968 (duration: 14m 43s) 2476865 wikitext text/x-wiki ezdlbuixga2suwzl0lq3dayq9rj4cz8 2476868 2476865 2026-11-03T16:40:58Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netbox-dev2003.codfw.wmnet 2476868 wikitext text/x-wiki dblqaj666ko2uvhhkdd5s0fhee4mus7 2476869 2476868 2026-11-03T16:58:27Z Stashbot 7414 kamila@deploy2003: Started scap deploy-service shellbox: bump also shellbox score to php 8.5 2476869 wikitext text/x-wiki ffpzcssqvdfnky92xhbg22p42wpfv33 2476870 2476869 2026-11-03T16:58:30Z Stashbot 7414 kamila@deploy2003: scap deploy-service shellbox: [STAGING/DEFAULT] staging-eqiad Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2476870 wikitext text/x-wiki jvxuq5pc944h57m1bjogjera9jpotbb 2476872 2476870 2026-11-03T16:59:15Z Stashbot 7414 kamila@deploy2003: scap deploy-service shellbox: [PRODUCTION/DEFAULT] eqiad Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2476872 wikitext text/x-wiki 4u5zqjya0n7sarrnz33fdiz0kn0nkmx 2476874 2476872 2026-11-03T17:00:36Z Stashbot 7414 kamila@deploy2003: scap deploy-service shellbox: [PRODUCTION/DEFAULT] codfw Deploying shellbox, shellbox-constraints, shellbox-media, shellbox-syntaxhighlight, shellbox-timeline, shellbox-video 2476874 wikitext text/x-wiki qmklwbgl2ohvqpzqcuaqwmkor5fphk8 2476875 2476874 2026-11-03T17:03:18Z Stashbot 7414 kamila@deploy2003: Finished scap deploy-service shellbox: bump also shellbox score to php 8.5 (duration: 04m 51s) 2476875 wikitext text/x-wiki lt50rejmk1l3w9se576ezm0afgvfbd4 2476883 2476875 2026-11-03T17:46:24Z Stashbot 7414 tchanders@deploy2003: Started scap sync-world: Backport for [[gerrit:1362074|Enable link spam queue on Special:AbuseReview for alpha testing (T443432)]] 2476883 wikitext text/x-wiki swr61xo78gt31fx26uk3amn49xu9ukc 2476885 2476883 2026-11-03T17:50:40Z Stashbot 7414 tchanders@deploy2003: tchanders: Backport for [[gerrit:1362074|Enable link spam queue on Special:AbuseReview for alpha testing (T443432)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2476885 wikitext text/x-wiki j5xqx2o5zog3vxxw1v3ru2e3adjpkcg 2476887 2476885 2026-11-03T17:56:29Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1159.eqiad.wmnet with reason: Maintenance 2476887 wikitext text/x-wiki qmf0jrp9jbb5zbzue04n68s3irz1lla 2476888 2476887 2026-11-03T17:56:32Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1159 (T441633)', diff saved to https://phabricator.wikimedia.org/P97146 and previous config saved to /var/cache/conftool/dbconfig/20261103-175629-ladsgroup.json 2476888 wikitext text/x-wiki fy0mf7jox53glwro5k02jg0glzhcv95 2476890 2476888 2026-11-03T17:59:26Z Stashbot 7414 tchanders@deploy2003: tchanders: Continuing with deployment 2476890 wikitext text/x-wiki hy23jci31lc568r63zctsksrhqgg0um 2476891 2476890 2026-11-03T18:04:35Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1159 (T441633)', diff saved to https://phabricator.wikimedia.org/P97147 and previous config saved to /var/cache/conftool/dbconfig/20261103-180432-ladsgroup.json 2476891 wikitext text/x-wiki 7kf6scfirqemykldcjpwdhw897tzx8a 2476892 2476891 2026-11-03T18:05:23Z Stashbot 7414 tchanders@deploy2003: Finished scap sync-world: Backport for [[gerrit:1362074|Enable link spam queue on Special:AbuseReview for alpha testing (T443432)]] (duration: 19m 00s) 2476892 wikitext text/x-wiki ocawh35czqs3qr3tzxsealkg0lstbkt 2476894 2476892 2026-11-03T18:11:58Z Stashbot 7414 akhatun@deploy2003: Started scap sync-world: Backport for [[gerrit:1362126|revision feature counts: increase timeout 20s to 30s (T443482)]] 2476894 wikitext text/x-wiki 7o0r9sj1fz3rg5b1dkb27exwid75j46 2476897 2476894 2026-11-03T18:14:37Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1159', diff saved to https://phabricator.wikimedia.org/P97148 and previous config saved to /var/cache/conftool/dbconfig/20261103-181435-ladsgroup.json 2476897 wikitext text/x-wiki 4e1yo33np2wrnpcibo1lo4o3c7qjo3o 2476900 2476897 2026-11-03T18:16:03Z Stashbot 7414 akhatun@deploy2003: akhatun: Backport for [[gerrit:1362126|revision feature counts: increase timeout 20s to 30s (T443482)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2476900 wikitext text/x-wiki 5jvxq1sj3blvetmu3tliz9dw4rz4atu 2476901 2476900 2026-11-03T18:16:48Z Stashbot 7414 akhatun@deploy2003: akhatun: Continuing with deployment 2476901 wikitext text/x-wiki 4pk3157biq9ap1ak8pzz7ghoum4dk25 2476903 2476901 2026-11-03T18:17:56Z Stashbot 7414 cjming@deploy2003: Started scap deploy-service test-kitchen-next: deploy v2.3.0 to TK staging 2476903 wikitext text/x-wiki qouo3fvrxdynd1n91g18a8frah3gyvj 2476904 2476903 2026-11-03T18:17:59Z Stashbot 7414 cjming@deploy2003: scap deploy-service test-kitchen-next: [PRODUCTION/DEFAULT] dse-k8s-eqiad Deploying test-kitchen-next 2476904 wikitext text/x-wiki cgo3jbxm179rnqns20ljdu77ahg1oq9 2476906 2476904 2026-11-03T18:18:59Z Stashbot 7414 cjming@deploy2003: Finished scap deploy-service test-kitchen-next: deploy v2.3.0 to TK staging (duration: 01m 03s) 2476906 wikitext text/x-wiki omby22idnc79ob5plgvr53j3wa85pz6 2476909 2476906 2026-11-03T18:22:39Z Stashbot 7414 akhatun@deploy2003: Finished scap sync-world: Backport for [[gerrit:1362126|revision feature counts: increase timeout 20s to 30s (T443482)]] (duration: 10m 40s) 2476909 wikitext text/x-wiki g5bsft0mtv6vn0ei4dl6te7fnktdrmd 2476910 2476909 2026-11-03T18:23:14Z Stashbot 7414 kamila@cumin1004: START - Cookbook sre.hosts.reboot-single for host deploy2003.codfw.wmnet 2476910 wikitext text/x-wiki mgmcffu9ov6nc078bp145hoa3h02hf6 2476911 2476910 2026-11-03T18:24:39Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1159', diff saved to https://phabricator.wikimedia.org/P97149 and previous config saved to /var/cache/conftool/dbconfig/20261103-182437-ladsgroup.json 2476911 wikitext text/x-wiki 6poztlswskr1nhnjlth5ozsygi7gdjf 2476914 2476911 2026-11-03T18:27:26Z Stashbot 7414 eevans@deploy2003: helmfile [staging] START helmfile.d/services/linked-artifacts: apply 2476914 wikitext text/x-wiki mkjlok2ybcr4hofeh511bdandk8c9d3 2476915 2476914 2026-11-03T18:27:44Z Stashbot 7414 eevans@deploy2003: helmfile [staging] DONE helmfile.d/services/linked-artifacts: apply 2476915 wikitext text/x-wiki kjj8p7v08i0ugzlzj8qny713ltcl39m 2476916 2476915 2026-11-03T18:28:09Z Stashbot 7414 eevans@deploy2003: helmfile [codfw] START helmfile.d/services/linked-artifacts: apply 2476916 wikitext text/x-wiki pmpq9n0qjc9mr2n0ndsnoka1v2om528 2476917 2476916 2026-11-03T18:28:27Z Stashbot 7414 eevans@deploy2003: helmfile [codfw] DONE helmfile.d/services/linked-artifacts: apply 2476917 wikitext text/x-wiki 80h3164xw34zrnhi8ergseuiimpk7on 2476918 2476917 2026-11-03T18:28:35Z Stashbot 7414 eevans@deploy2003: helmfile [eqiad] START helmfile.d/services/linked-artifacts: apply 2476918 wikitext text/x-wiki d1p41e4va842o6rao5md8lt9y0ng904 2476919 2476918 2026-11-03T18:28:51Z Stashbot 7414 eevans@deploy2003: helmfile [eqiad] DONE helmfile.d/services/linked-artifacts: apply 2476919 wikitext text/x-wiki 1899hpj94fw0w8d0q219jmcvjxehxit 2476921 2476919 2026-11-03T18:29:44Z Stashbot 7414 cjming@deploy2003: Started scap deploy-service test-kitchen: deploy v2.3.0 to TK production 2476921 wikitext text/x-wiki 9ysyya9vh7fszr45wew9oukk4uxx531 2476922 2476921 2026-11-03T18:29:47Z Stashbot 7414 cjming@deploy2003: scap deploy-service test-kitchen: [PRODUCTION/DEFAULT] dse-k8s-eqiad Deploying test-kitchen 2476922 wikitext text/x-wiki tdvzd8hes3sw0xuwtajokqeg7sbo8gl 2476924 2476922 2026-11-03T18:31:57Z Stashbot 7414 cjming@deploy2003: Finished scap deploy-service test-kitchen: deploy v2.3.0 to TK production (duration: 02m 12s) 2476924 wikitext text/x-wiki 0solt1134dayvhkls3a8yf6bozvw8cg 2476927 2476924 2026-11-03T18:33:30Z Stashbot 7414 kamila@cumin1004: START - Cookbook sre.hosts.reboot-single for host deploy1003.eqiad.wmnet 2476927 wikitext text/x-wiki rafmicqnx8xsff7kp89mwsbxg0338t4 2476928 2476927 2026-11-03T18:34:32Z Stashbot 7414 kamila@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host deploy2003.codfw.wmnet 2476928 wikitext text/x-wiki gdvspvg6nvxoaymewius8xns3qfey8p 2476929 2476928 2026-11-03T18:34:41Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1159 (T441633)', diff saved to https://phabricator.wikimedia.org/P97150 and previous config saved to /var/cache/conftool/dbconfig/20261103-183439-ladsgroup.json 2476929 wikitext text/x-wiki qnmclvscfjfh21dqlxtb6bnvdqgsohi 2476930 2476929 2026-11-03T18:34:59Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1161.eqiad.wmnet with reason: Maintenance 2476930 wikitext text/x-wiki ecgq8n5he69udu8dk7lumqghyeq8c6g 2476931 2476930 2026-11-03T18:35:02Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1161 (T441633)', diff saved to https://phabricator.wikimedia.org/P97151 and previous config saved to /var/cache/conftool/dbconfig/20261103-183459-ladsgroup.json 2476931 wikitext text/x-wiki b70sccvx3klskkpp5v46kedpmts9ynd 2476934 2476931 2026-11-03T18:43:03Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1161 (T441633)', diff saved to https://phabricator.wikimedia.org/P97152 and previous config saved to /var/cache/conftool/dbconfig/20261103-184301-ladsgroup.json 2476934 wikitext text/x-wiki 05625bqf36axczstak687isnk3l4qs6 2476935 2476934 2026-11-03T18:46:40Z Stashbot 7414 kamila@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host deploy1003.eqiad.wmnet 2476935 wikitext text/x-wiki dbiq713d0zp0bbmaknq78yjq6e228vm 2476936 2476935 2026-11-03T18:49:22Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=no; selector: name=dns7002.* [reason: testing pdns-rec 5.4.9] 2476936 wikitext text/x-wiki kh16ztc9d7f8ecgq8c6y58pjglel13d 2476937 2476936 2026-11-03T18:49:27Z Stashbot 7414 atsuko@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply 2476937 wikitext text/x-wiki 7gqhhf19bqxtayu9lzwcpz6vkgsheoe 2476938 2476937 2026-11-03T18:50:14Z Stashbot 7414 atsuko@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply 2476938 wikitext text/x-wiki qfukycbnqkpeesj5hb1rqib9lbvgzce 2476939 2476938 2026-11-03T18:51:06Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=yes; selector: name=dns7002.* [reason: [end] testing pdns-rec 5.4.9] 2476939 wikitext text/x-wiki 5yn4e2dtq0bfsdazgv2lesi5y555jew 2476940 2476939 2026-11-03T18:53:05Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P97153 and previous config saved to /var/cache/conftool/dbconfig/20261103-185303-ladsgroup.json 2476940 wikitext text/x-wiki 347qivs3h7la0h21qfxrkbvcijv3rko 2476941 2476940 2026-11-03T19:03:07Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P97154 and previous config saved to /var/cache/conftool/dbconfig/20261103-190305-ladsgroup.json 2476941 wikitext text/x-wiki nkcg84je5twr229coag6ow2x98q9a5c 2476943 2476941 2026-11-03T19:09:07Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.wdqs.categories-reload reloading categories to wdqs2008.codfw.wmnet 2476943 wikitext text/x-wiki o0777on744qwo0nvz4dsz50jxy8mr7q 2476944 2476943 2026-11-03T19:11:03Z Stashbot 7414 dancy@deploy2003: Installing scap version "4.313.0" for 2 host(s) 2476944 wikitext text/x-wiki 7sjmzx8oyuszxv1ueu8ejeur00n31o3 2476945 2476944 2026-11-03T19:13:09Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1161 (T441633)', diff saved to https://phabricator.wikimedia.org/P97155 and previous config saved to /var/cache/conftool/dbconfig/20261103-191307-ladsgroup.json 2476945 wikitext text/x-wiki czp7bapa4090cnc59ove2d9apg9ljwh 2476946 2476945 2026-11-03T19:13:27Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1185.eqiad.wmnet with reason: Maintenance 2476946 wikitext text/x-wiki n7ucx4c48ee5ggwx27yzeo7y6o7ad2b 2476947 2476946 2026-11-03T19:13:30Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1185 (T441633)', diff saved to https://phabricator.wikimedia.org/P97156 and previous config saved to /var/cache/conftool/dbconfig/20261103-191327-ladsgroup.json 2476947 wikitext text/x-wiki i4mll1pauw0gvtdtwu028tda409d1qm 2476948 2476947 2026-11-03T19:14:29Z Stashbot 7414 dancy@deploy2003: Installing scap version "4.313.0" for 2 host(s) 2476948 wikitext text/x-wiki 6no6ytauupub04vh3rhxu5qir6crj7p 2476949 2476948 2026-11-03T19:16:26Z Stashbot 7414 dancy@deploy2003: Installing scap version "4.313.0" for 1 host(s) 2476949 wikitext text/x-wiki 86avvydja3s0e2dp5tmoqt9ygtdyp2d 2476950 2476949 2026-11-03T19:17:57Z Stashbot 7414 dancy@deploy2003: Installing scap version "4.313.0" for 1 host(s) 2476950 wikitext text/x-wiki 38vuvetc4mbnqai482ygs6h0bve4ykp 2476951 2476950 2026-11-03T19:18:43Z Stashbot 7414 atsuko@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply 2476951 wikitext text/x-wiki 9fy178j2yn2lqk5phc17zolf8su6f4z 2476952 2476951 2026-11-03T19:19:35Z Stashbot 7414 atsuko@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply 2476952 wikitext text/x-wiki n8z6r5sxo1tx2ur508f6kl3hfvs45e1 2476953 2476952 2026-11-03T19:20:19Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1185 (T441633)', diff saved to https://phabricator.wikimedia.org/P97157 and previous config saved to /var/cache/conftool/dbconfig/20261103-192017-ladsgroup.json 2476953 wikitext text/x-wiki a72zjzb1tu36rmqk1izutoqmh9ie9jc 2476954 2476953 2026-11-03T19:25:04Z Stashbot 7414 atsuko@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply 2476954 wikitext text/x-wiki cvdbkhdbuaf9uk9ty1c10qxdgk5jp6d 2476955 2476954 2026-11-03T19:25:21Z Stashbot 7414 atsuko@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply 2476955 wikitext text/x-wiki 52iex3ngr7n18r59j4wqwl9bca66zbl 2476956 2476955 2026-11-03T19:27:17Z Stashbot 7414 dancy@deploy2003: Installing scap version "4.313.0" for 2 host(s) 2476956 wikitext text/x-wiki tk1weaiq51rkefp7dch7209cdf8sbji 2476957 2476956 2026-11-03T19:29:01Z Stashbot 7414 dancy@deploy2003: Installation of scap version "4.313.0" completed for 2 hosts 2476957 wikitext text/x-wiki 8cipn56dq6691ogbror6v36id5ytp4o 2476958 2476957 2026-11-03T19:30:21Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1185', diff saved to https://phabricator.wikimedia.org/P97158 and previous config saved to /var/cache/conftool/dbconfig/20261103-193019-ladsgroup.json 2476958 wikitext text/x-wiki 5v7ru8tsloqj6ieqhq7wpcblcsk6q7e 2476959 2476958 2026-11-03T19:40:23Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1185', diff saved to https://phabricator.wikimedia.org/P97159 and previous config saved to /var/cache/conftool/dbconfig/20261103-194021-ladsgroup.json 2476959 wikitext text/x-wiki 3swekvv4nl49ukbn812d77a10o2ev36 2476961 2476959 2026-11-03T19:50:26Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1185 (T441633)', diff saved to https://phabricator.wikimedia.org/P97160 and previous config saved to /var/cache/conftool/dbconfig/20261103-195023-ladsgroup.json 2476961 wikitext text/x-wiki 1j35joo4vikg3a4dq5fawtvzgga3xaj 2476962 2476961 2026-11-03T19:50:43Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1200.eqiad.wmnet with reason: Maintenance 2476962 wikitext text/x-wiki 6edcunv5ag5jii7xy549i1f4crqx0hw 2476963 2476962 2026-11-03T19:50:46Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1200 (T441633)', diff saved to https://phabricator.wikimedia.org/P97161 and previous config saved to /var/cache/conftool/dbconfig/20261103-195043-ladsgroup.json 2476963 wikitext text/x-wiki pq6leq9p704njs7yz4csmjvwg66plct 2476964 2476963 2026-11-03T19:57:36Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1200 (T441633)', diff saved to https://phabricator.wikimedia.org/P97162 and previous config saved to /var/cache/conftool/dbconfig/20261103-195734-ladsgroup.json 2476964 wikitext text/x-wiki 3vszeqk6d0r0b042dxsr314ha9kicjj 2476965 2476964 2026-11-03T19:59:43Z Stashbot 7414 eevans@deploy2003: helmfile [staging] START helmfile.d/services/linked-artifacts: apply 2476965 wikitext text/x-wiki owbeoaagduyd6aru417myhs99v5r09l 2476966 2476965 2026-11-03T20:00:01Z Stashbot 7414 eevans@deploy2003: helmfile [staging] DONE helmfile.d/services/linked-artifacts: apply 2476966 wikitext text/x-wiki 7se36nejcxlpub9btjz9lq96s8j4jd9 2476967 2476966 2026-11-03T20:07:37Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1200', diff saved to https://phabricator.wikimedia.org/P97163 and previous config saved to /var/cache/conftool/dbconfig/20261103-200735-ladsgroup.json 2476967 wikitext text/x-wiki gj6aaud587zu22b7dtby3cggj0mla9j 2476968 2476967 2026-11-03T20:17:39Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1200', diff saved to https://phabricator.wikimedia.org/P97164 and previous config saved to /var/cache/conftool/dbconfig/20261103-201737-ladsgroup.json 2476968 wikitext text/x-wiki cyuwok0x6c8scn26ssm2wyuhrch3fq8 2476973 2476968 2026-11-03T20:27:41Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1200 (T441633)', diff saved to https://phabricator.wikimedia.org/P97165 and previous config saved to /var/cache/conftool/dbconfig/20261103-202739-ladsgroup.json 2476973 wikitext text/x-wiki 1exhs61x3kuq0ldmaj8leccwg0wyxrv 2476974 2476973 2026-11-03T20:27:49Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1207.eqiad.wmnet with reason: Maintenance 2476974 wikitext text/x-wiki bk5z3y8w7c1iiyh87echs2kdk2odqv8 2476975 2476974 2026-11-03T20:27:52Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1207 (T441633)', diff saved to https://phabricator.wikimedia.org/P97166 and previous config saved to /var/cache/conftool/dbconfig/20261103-202749-ladsgroup.json 2476975 wikitext text/x-wiki qk7bdk98uaoz2t8o3e5pov9aj5wf0d3 2476976 2476975 2026-11-03T20:34:32Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1207 (T441633)', diff saved to https://phabricator.wikimedia.org/P97167 and previous config saved to /var/cache/conftool/dbconfig/20261103-203430-ladsgroup.json 2476976 wikitext text/x-wiki 9dqn3mdx7riknpo5qxihswd909m2rlj 2476978 2476976 2026-11-03T20:44:33Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P97168 and previous config saved to /var/cache/conftool/dbconfig/20261103-204432-ladsgroup.json 2476978 wikitext text/x-wiki csugc4tcr1gpy0923qzh0v74spndk3u 2476979 2476978 2026-11-03T20:54:36Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P97169 and previous config saved to /var/cache/conftool/dbconfig/20261103-205434-ladsgroup.json 2476979 wikitext text/x-wiki rafx15t235bqc84g2ybgezhzxg6vfrk 2476981 2476979 2026-11-03T21:04:38Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1207 (T441633)', diff saved to https://phabricator.wikimedia.org/P97170 and previous config saved to /var/cache/conftool/dbconfig/20261103-210435-ladsgroup.json 2476981 wikitext text/x-wiki c147g5k5wmc3j9cule4nkb32ux8vr9f 2476982 2476981 2026-11-03T21:04:58Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1230.eqiad.wmnet with reason: Maintenance 2476982 wikitext text/x-wiki aa24a9jxltubjmiqj7nmjx8nvqcd063 2476983 2476982 2026-11-03T21:05:02Z Stashbot 7414 sbassett@deploy2003: Started scap sync-world: Backport for [[gerrit:1360542|Enable DBSC report-only mode on group0 and test2wiki (T393212)]] 2476983 wikitext text/x-wiki mghghw0lf2bal2ke7adfujloz0922py 2476984 2476983 2026-11-03T21:05:05Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1230 (T441633)', diff saved to https://phabricator.wikimedia.org/P97171 and previous config saved to /var/cache/conftool/dbconfig/20261103-210458-ladsgroup.json 2476984 wikitext text/x-wiki comjdvu2d9vzalt0bmtq4rglscir7yv 2476985 2476984 2026-11-03T21:09:17Z Stashbot 7414 sbassett@deploy2003: sbassett: Backport for [[gerrit:1360542|Enable DBSC report-only mode on group0 and test2wiki (T393212)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2476985 wikitext text/x-wiki 3z8gfu94kmaz6d7qp94a18z6ala5eap 2476986 2476985 2026-11-03T21:09:53Z Stashbot 7414 sbassett@deploy2003: sbassett: Continuing with deployment 2476986 wikitext text/x-wiki idgpfuek84l797x8ysx5ewli07y8cod 2476987 2476986 2026-11-03T21:11:43Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1230 (T441633)', diff saved to https://phabricator.wikimedia.org/P97172 and previous config saved to /var/cache/conftool/dbconfig/20261103-211141-ladsgroup.json 2476987 wikitext text/x-wiki 23e2aitfkpv9g2cgc3372go9qsbhw0v 2476988 2476987 2026-11-03T21:16:41Z Stashbot 7414 sbassett@deploy2003: Finished scap sync-world: Backport for [[gerrit:1360542|Enable DBSC report-only mode on group0 and test2wiki (T393212)]] (duration: 11m 41s) 2476988 wikitext text/x-wiki 3555aqdtgolp3o9ei44z89myl7635ok 2476989 2476988 2026-11-03T21:21:45Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1230', diff saved to https://phabricator.wikimedia.org/P97173 and previous config saved to /var/cache/conftool/dbconfig/20261103-212143-ladsgroup.json 2476989 wikitext text/x-wiki 80h567l70bt7q21na3h9sstkf284y70 2476990 2476989 2026-11-03T21:24:13Z Stashbot 7414 ebernhardson@deploy2003: Started scap sync-world: Backport for [[gerrit:1361843|eswiki: add editprotected to botadmin (T443272)]], [[gerrit:1361837|liwiktionary: Create Appendix namespace (T443269)]], [[gerrit:1361447|Stop exempting item/property creation from hCaptcha (T440832)]], [[gerrit:1355485|Remove WikibaseTmpMobileEditingUIBetaFeature config (T437054)]], [[gerrit:1362185|Revert "Reapply "cirrus: Send more_like 2476990 wikitext text/x-wiki 2cl5348hz7z9skubrzxqv434j6a9qq1 2476992 2476990 2026-11-03T21:28:41Z Stashbot 7414 ebernhardson@deploy2003: ebernhardson, ameisenigel, thiemowmde, mahmoud-abdelsattar: Backport for [[gerrit:1361843|eswiki: add editprotected to botadmin (T443272)]], [[gerrit:1361837|liwiktionary: Create Appendix namespace (T443269)]], [[gerrit:1361447|Stop exempting item/property creation from hCaptcha (T440832)]], [[gerrit:1355485|Remove WikibaseTmpMobileEditingUIBetaFeature config (T437054)]], [[gerrit:1362185|Reve 2476992 wikitext text/x-wiki six2kk14v0ar5bgnjvvfetyw4bobjzb 2476993 2476992 2026-11-03T21:31:46Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1230', diff saved to https://phabricator.wikimedia.org/P97174 and previous config saved to /var/cache/conftool/dbconfig/20261103-213145-ladsgroup.json 2476993 wikitext text/x-wiki g47k21ame6165let51a1q0s6idwjdjh 2476995 2476993 2026-11-03T21:36:15Z Stashbot 7414 ebernhardson@deploy2003: ebernhardson, ameisenigel, thiemowmde, mahmoud-abdelsattar: Continuing with deployment 2476995 wikitext text/x-wiki 0kygr7i8lqeckjazihe5cnfh0fhyy2p 2476996 2476995 2026-11-03T21:41:48Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1230 (T441633)', diff saved to https://phabricator.wikimedia.org/P97175 and previous config saved to /var/cache/conftool/dbconfig/20261103-214146-ladsgroup.json 2476996 wikitext text/x-wiki 76on9wy4qb36te0zsio1e0ismibkn9t 2476997 2476996 2026-11-03T21:42:06Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1274.eqiad.wmnet with reason: Maintenance 2476997 wikitext text/x-wiki t1i18nr0anwt0m8na31lh0gehwrdaxw 2476998 2476997 2026-11-03T21:42:09Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1274 (T441633)', diff saved to https://phabricator.wikimedia.org/P97176 and previous config saved to /var/cache/conftool/dbconfig/20261103-214206-ladsgroup.json 2476998 wikitext text/x-wiki il7qiwzi3zwnkkcs6ffv1qsow75gyq1 2476999 2476998 2026-11-03T21:42:52Z Stashbot 7414 ebernhardson@deploy2003: Finished scap sync-world: Backport for [[gerrit:1361843|eswiki: add editprotected to botadmin (T443272)]], [[gerrit:1361837|liwiktionary: Create Appendix namespace (T443269)]], [[gerrit:1361447|Stop exempting item/property creation from hCaptcha (T440832)]], [[gerrit:1355485|Remove WikibaseTmpMobileEditingUIBetaFeature config (T437054)]], [[gerrit:1362185|Revert "Reapply "cirrus: Send more_lik 2476999 wikitext text/x-wiki 8b6iyjzpa2h6i1wbcevasshhhwha83h 2477001 2476999 2026-11-03T22:14:09Z Stashbot 7414 jdlrobson@deploy2003: Started scap sync-world: Backport for [[gerrit:1362101|styles: Restore style to Special:MobileOptions (T363032)]] (1.48.0-wmf.3), [[gerrit:1361724|Revert "footer: Hide separator before a hidden last footer place" (T387797)]] (1.48.0-wmf.3), [[gerrit:1362134|downloadPageAction: Add the download link before the bookmark (T443387)]] (1.48.0-wmf.3) 2477001 wikitext text/x-wiki fgis95sysn3x0x1s425oatkscurjggc 2477002 2477001 2026-11-03T22:18:15Z Stashbot 7414 jdlrobson@deploy2003: jdlrobson: Backport for [[gerrit:1362101|styles: Restore style to Special:MobileOptions (T363032)]] (1.48.0-wmf.3), [[gerrit:1361724|Revert "footer: Hide separator before a hidden last footer place" (T387797)]] (1.48.0-wmf.3), [[gerrit:1362134|downloadPageAction: Add the download link before the bookmark (T443387)]] (1.48.0-wmf.3) synced to the testservers (see https://wikitech.wikimedia.org/wiki 2477002 wikitext text/x-wiki 052x5e5vlix39bl3ux06gdbuff9tdjk 2477004 2477002 2026-11-03T22:21:06Z Stashbot 7414 jdlrobson@deploy2003: jdlrobson: Continuing with deployment 2477004 wikitext text/x-wiki 3wy4bpg2am7ga3kly9iiq7vlpvweyb2 2477005 2477004 2026-11-03T22:27:34Z Stashbot 7414 jdlrobson@deploy2003: Finished scap sync-world: Backport for [[gerrit:1362101|styles: Restore style to Special:MobileOptions (T363032)]] (1.48.0-wmf.3), [[gerrit:1361724|Revert "footer: Hide separator before a hidden last footer place" (T387797)]] (1.48.0-wmf.3), [[gerrit:1362134|downloadPageAction: Add the download link before the bookmark (T443387)]] (1.48.0-wmf.3) (duration: 13m 25s) 2477005 wikitext text/x-wiki h42ddubihpva7aae0bfq4ds64ga976d 2477006 2477005 2026-11-03T22:33:36Z Stashbot 7414 kemayo@deploy2003: Started scap sync-world: Backport for [[gerrit:1362215|VisualEditor: Precompute edit suggestion counts on 5% of jawiki (T442767)]] 2477006 wikitext text/x-wiki 4u802hdctzt7n2br5oyx6vmg2b0ogk3 2477008 2477006 2026-11-03T22:38:05Z Stashbot 7414 kemayo@deploy2003: kemayo: Backport for [[gerrit:1362215|VisualEditor: Precompute edit suggestion counts on 5% of jawiki (T442767)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2477008 wikitext text/x-wiki c4h45ndshf1yvgru9bpsny72ckqj2ms 2477009 2477008 2026-11-03T22:38:38Z Stashbot 7414 kemayo@deploy2003: kemayo: Continuing with deployment 2477009 wikitext text/x-wiki hclvbm5cydl74jlhoohq67znmvzyf5q 2477011 2477009 2026-11-03T22:44:26Z Stashbot 7414 kemayo@deploy2003: Finished scap sync-world: Backport for [[gerrit:1362215|VisualEditor: Precompute edit suggestion counts on 5% of jawiki (T442767)]] (duration: 10m 49s) 2477011 wikitext text/x-wiki gylnts6towbmlq99vae7pluszhttqv1 2477012 2477011 2026-11-03T22:48:16Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1158.eqiad.wmnet with reason: Maintenance 2477012 wikitext text/x-wiki bbnj2cqo9mmupe9cb3rmqsw3v2ekcrw 2477013 2477012 2026-11-03T22:48:19Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1158 (T441633)', diff saved to https://phabricator.wikimedia.org/P97178 and previous config saved to /var/cache/conftool/dbconfig/20261103-224816-ladsgroup.json 2477013 wikitext text/x-wiki jqrelva1s76xs0a5budwlsdeqigi9p2 2477016 2477013 2026-11-03T22:53:38Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.wdqs.categories-reload (exit_code=0) reloading categories to wdqs2008.codfw.wmnet 2477016 wikitext text/x-wiki k5ypf29pflx5hbdnefg9nmej0a1ws25 2477017 2477016 2026-11-03T22:56:34Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1158 (T441633)', diff saved to https://phabricator.wikimedia.org/P97179 and previous config saved to /var/cache/conftool/dbconfig/20261103-225631-ladsgroup.json 2477017 wikitext text/x-wiki 1zdnx6mytj4t1nb2mf0y3e048jj6gal 2477019 2477017 2026-11-03T23:06:35Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1158', diff saved to https://phabricator.wikimedia.org/P97180 and previous config saved to /var/cache/conftool/dbconfig/20261103-230633-ladsgroup.json 2477019 wikitext text/x-wiki hw7g6f7fiymc2ktwzh578vaytn5paxg 2477021 2477019 2026-11-03T23:16:37Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1158', diff saved to https://phabricator.wikimedia.org/P97181 and previous config saved to /var/cache/conftool/dbconfig/20261103-231635-ladsgroup.json 2477021 wikitext text/x-wiki hhq5r4vvfyfbquitz3tx35kkirvcn98 2477022 2477021 2026-11-03T23:18:26Z Stashbot 7414 musikanimal@deploy2003: Started scap sync-world: Backport for [[gerrit:1352303|CommonSettings: remove redundant CodeMirror config settings (T440077)]], [[gerrit:1358261|metawiki: Disable wish submissions on 2026-11-09 (17:00 UTC) (T442015)]] 2477022 wikitext text/x-wiki ka3ewraabg7ursui9ntd111v83nef5t 2477023 2477022 2026-11-03T23:22:50Z Stashbot 7414 musikanimal@deploy2003: musikanimal, amdrel: Backport for [[gerrit:1352303|CommonSettings: remove redundant CodeMirror config settings (T440077)]], [[gerrit:1358261|metawiki: Disable wish submissions on 2026-11-09 (17:00 UTC) (T442015)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2477023 wikitext text/x-wiki 1ycd63h26vi3q9ofb5gvrffsrhcdgpd 2477024 2477023 2026-11-03T23:24:14Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/apertium: apply 2477024 wikitext text/x-wiki 3b3yunshup1muup5q0d4wcxbzpi1pel 2477025 2477024 2026-11-03T23:24:45Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/apertium: apply 2477025 wikitext text/x-wiki lvssfqzkpxtrji5eueoi7w6hf93zb5k 2477026 2477025 2026-11-03T23:26:29Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/chart-renderer: apply 2477026 wikitext text/x-wiki crbo2e68xnpkv8k3l6r2ilrup4zi8mx 2477027 2477026 2026-11-03T23:26:39Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1158 (T441633)', diff saved to https://phabricator.wikimedia.org/P97182 and previous config saved to /var/cache/conftool/dbconfig/20261103-232637-ladsgroup.json 2477027 wikitext text/x-wiki s3zhggajxchfdvol6nojiicfot7zcu7 2477028 2477027 2026-11-03T23:26:54Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/chart-renderer: apply 2477028 wikitext text/x-wiki 7i5lpqr039qpf7wxws5zebe8eqiwl62 2477029 2477028 2026-11-03T23:26:57Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1170.eqiad.wmnet with reason: Maintenance 2477029 wikitext text/x-wiki 9dqga4j21778yq5xjkmx84k6cc8jfb0 2477030 2477029 2026-11-03T23:27:01Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1170 (T441633)', diff saved to https://phabricator.wikimedia.org/P97183 and previous config saved to /var/cache/conftool/dbconfig/20261103-232657-ladsgroup.json 2477030 wikitext text/x-wiki bka58lnezj41dvjfq7wg1288z4q1jh3 2477031 2477030 2026-11-03T23:27:04Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply 2477031 wikitext text/x-wiki 781wlw9qqux3ku38br6ht0t3kgu1sci 2477032 2477031 2026-11-03T23:27:07Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply 2477032 wikitext text/x-wiki c6pti16ljl3i3rj45pnubpz3nfmb3kf 2477033 2477032 2026-11-03T23:27:09Z Stashbot 7414 musikanimal@deploy2003: musikanimal, amdrel: Continuing with deployment 2477033 wikitext text/x-wiki 87ptrbqli1sccq2f58851itju6p7jz6 2477034 2477033 2026-11-03T23:27:19Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/citoid: apply 2477034 wikitext text/x-wiki 2tmt17ta47lsno4julsqtz0gaujfky7 2477035 2477034 2026-11-03T23:27:35Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/citoid: apply 2477035 wikitext text/x-wiki 1v0cavlwglmot430uhi0pppq2qzl2c7 2477036 2477035 2026-11-03T23:27:45Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/commons-impact-analytics: apply 2477036 wikitext text/x-wiki 06dcsyncsxwrve7pkytl8a7wcrelfkl 2477037 2477036 2026-11-03T23:27:58Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/commons-impact-analytics: apply 2477037 wikitext text/x-wiki p068jj6lw1nse3u23wc6ycbf9wltkqd 2477038 2477037 2026-11-03T23:28:10Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/cxserver: apply 2477038 wikitext text/x-wiki 2jwky5f5yv6jmkow0njmbjehm7qki2i 2477039 2477038 2026-11-03T23:28:31Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/cxserver: apply 2477039 wikitext text/x-wiki 85t9elq2q42m9rbxl7bskyvex9jfumo 2477040 2477039 2026-11-03T23:28:41Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/data-gateway: apply 2477040 wikitext text/x-wiki 80hhupesydngxrefyxcvgd5rmqo4mnf 2477041 2477040 2026-11-03T23:28:57Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/data-gateway: apply 2477041 wikitext text/x-wiki l3zc44fg7ivk95jr8zt12ydsj3hor6p 2477042 2477041 2026-11-03T23:29:03Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/developer-portal: apply 2477042 wikitext text/x-wiki f9lg9ngtl4xy35tv2fqnonkw6p8ubrf 2477043 2477042 2026-11-03T23:29:30Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/developer-portal: apply 2477043 wikitext text/x-wiki s9ib500swo1fs2cf5umua42ezqc8vxd 2477044 2477043 2026-11-03T23:29:35Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/device-analytics: apply 2477044 wikitext text/x-wiki 30psd02m8mmzrn39ien9zgsoyg3h8ym 2477045 2477044 2026-11-03T23:29:57Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/device-analytics: apply 2477045 wikitext text/x-wiki pg9mbmo5luivjcsfufh06ilblj80tl5 2477046 2477045 2026-11-03T23:30:02Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/echostore: apply 2477046 wikitext text/x-wiki l6s90f1rikuqs0v3b5ydof7ifsqkegi 2477047 2477046 2026-11-03T23:30:23Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/echostore: apply 2477047 wikitext text/x-wiki kjm476v1cyoplh2cp1rkoglhy1lf72u 2477048 2477047 2026-11-03T23:30:36Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/edit-analytics: apply 2477048 wikitext text/x-wiki kgj8bz33w9udn34wnb4s0n3jpefvqc0 2477049 2477048 2026-11-03T23:30:55Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/edit-analytics: apply 2477049 wikitext text/x-wiki tmnyk8rg8324nb4h97utabqf20xw4ub 2477050 2477049 2026-11-03T23:31:02Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/editcheck-headless: apply 2477050 wikitext text/x-wiki 8wa3jfk1wk1a70n0xf3lf9eq9ki28ww 2477051 2477050 2026-11-03T23:31:57Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/editcheck-headless: apply 2477051 wikitext text/x-wiki 0yz451xrezwehl2v4ma7fsx4f9sci9l 2477052 2477051 2026-11-03T23:32:12Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/editor-analytics: apply 2477052 wikitext text/x-wiki mj3lg53kf5oll0ymj7ledmgd6wo3uts 2477053 2477052 2026-11-03T23:32:28Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/editor-analytics: apply 2477053 wikitext text/x-wiki 0sdaz6ubkuxdbr2wesw1dlbxqr1f3ve 2477054 2477053 2026-11-03T23:33:23Z Stashbot 7414 musikanimal@deploy2003: Finished scap sync-world: Backport for [[gerrit:1352303|CommonSettings: remove redundant CodeMirror config settings (T440077)]], [[gerrit:1358261|metawiki: Disable wish submissions on 2026-11-09 (17:00 UTC) (T442015)]] (duration: 14m 56s) 2477054 wikitext text/x-wiki 04lwayi4f0r789kpws5j4qsxkr84xjq 2477055 2477054 2026-11-03T23:33:29Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/eventstreams: apply 2477055 wikitext text/x-wiki m3fxy1ajsgu2n77l4a7mwa0uastufvh 2477056 2477055 2026-11-03T23:33:57Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/eventstreams: apply 2477056 wikitext text/x-wiki k21quf1zv4z8rjmi4bxu2sa9jegm79g 2477057 2477056 2026-11-03T23:34:02Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/geo-analytics: apply 2477057 wikitext text/x-wiki 81uozt9yc18qkphhszpik9yv0v41qxo 2477058 2477057 2026-11-03T23:34:13Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/geo-analytics: apply 2477058 wikitext text/x-wiki hlhk1ep96q1fynrhhuxdda8s6p30jv8 2477059 2477058 2026-11-03T23:34:37Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/kartotherian: apply 2477059 wikitext text/x-wiki 66jlv5gdjv7c0t3tq43b8htb18msebd 2477060 2477059 2026-11-03T23:35:38Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1170 (T441633)', diff saved to https://phabricator.wikimedia.org/P97184 and previous config saved to /var/cache/conftool/dbconfig/20261103-233536-ladsgroup.json 2477060 wikitext text/x-wiki bvtixxt2njisogbti6qx248t8ayt45e 2477063 2477060 2026-11-03T23:44:46Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/kartotherian: apply 2477063 wikitext text/x-wiki 6mt8aaeuusdfxus635odfmjyttrxhg0 2477064 2477063 2026-11-03T23:44:57Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/lac-lambdas: apply 2477064 wikitext text/x-wiki 7fctll4i8i34pm5p9dw60j9y7b7r9iq 2477065 2477064 2026-11-03T23:45:12Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/lac-lambdas: apply 2477065 wikitext text/x-wiki hicm90aj9u848b9fahaa2mqgm28zss8 2477066 2477065 2026-11-03T23:45:40Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1170', diff saved to https://phabricator.wikimedia.org/P97185 and previous config saved to /var/cache/conftool/dbconfig/20261103-234538-ladsgroup.json 2477066 wikitext text/x-wiki 3iustrkqdhzh795oa3vb1s5m4cbj70c 2477067 2477066 2026-11-03T23:46:31Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/machinetranslation: apply 2477067 wikitext text/x-wiki 5srv2lqu557mknhb97h523v693qil74 2477068 2477067 2026-11-03T23:52:56Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/machinetranslation: apply 2477068 wikitext text/x-wiki 8r3s8u6emxbehds036z6afujx97pdnp 2477069 2477068 2026-11-03T23:53:56Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/media-analytics: apply 2477069 wikitext text/x-wiki e1xcxcssngicvhvuedmaffr16kmu6be 2477070 2477069 2026-11-03T23:54:08Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/media-analytics: apply 2477070 wikitext text/x-wiki 5md85m4gpc7y7b1moqcpp0jxibyoxn0 2477071 2477070 2026-11-03T23:54:19Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/miscweb: apply 2477071 wikitext text/x-wiki nq0kp6ntv8ir5gk5ivrq5s97wb68a5k 2477072 2477071 2026-11-03T23:55:42Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1170', diff saved to https://phabricator.wikimedia.org/P97186 and previous config saved to /var/cache/conftool/dbconfig/20261103-235540-ladsgroup.json 2477072 wikitext text/x-wiki kumousmqiw5r34brrducjwmhsdjinj8 2477073 2477072 2026-11-03T23:56:30Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/miscweb: apply 2477073 wikitext text/x-wiki 5dg1pv8i4i0zm04k4rxgsv6152cz3fu 2477074 2477073 2026-11-03T23:56:36Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/mobileapps: apply 2477074 wikitext text/x-wiki oiuanfl5tpmn18ji2zr7kjxi8hvo2wd 2477075 2477074 2026-11-03T23:57:16Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/mobileapps: apply 2477075 wikitext text/x-wiki oge1i9e532xbszx71p9nseb8uirw821 2477076 2477075 2026-11-03T23:58:29Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/page-analytics: apply 2477076 wikitext text/x-wiki i7zo8fchpaxbrccg69wixg2zjdkvrxr 2477077 2477076 2026-11-03T23:58:42Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/page-analytics: apply 2477077 wikitext text/x-wiki nqgi3huv1y6rs50f8ixu0s92h7atsgw 2477078 2477077 2026-11-03T23:59:24Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/proton: apply 2477078 wikitext text/x-wiki fowzr08ls9muaw2k7kmmbmp5p3ndkbx 2477079 2477078 2026-11-04T00:00:08Z Stashbot 7414 root@deploy2003: helmfile [codfw] START helmfile.d/services/shellbox-timeline: sync 2477079 wikitext text/x-wiki norjga3szyfl6j8ju88yqpb8m2qqu6z 2477080 2477079 2026-11-04T00:00:19Z Stashbot 7414 root@deploy2003: helmfile [codfw] DONE helmfile.d/services/shellbox-timeline: sync 2477080 wikitext text/x-wiki gmeiydciv4lesoasuejqqyfefzuiyps 2477081 2477080 2026-11-04T00:00:30Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/proton: apply 2477081 wikitext text/x-wiki 0xobss7od21dtsd1mbm3dk2m55j3q4m 2477082 2477081 2026-11-04T00:01:05Z Stashbot 7414 ladsgroup@deploy2003: Started scap sync-world: Backport for [[gerrit:1362212|Avoid unnecessary/aborted requests in DASH mode (T442988)]] (1.48.0-wmf.3), [[gerrit:1362217|Enable chunked videos on testwiki (T373546)]] 2477082 wikitext text/x-wiki 6iujc6b7jl5cxe6acp2ggdokylf0wph 2477084 2477082 2026-11-04T00:05:40Z Stashbot 7414 ladsgroup@deploy2003: ladsgroup: Backport for [[gerrit:1362212|Avoid unnecessary/aborted requests in DASH mode (T442988)]] (1.48.0-wmf.3), [[gerrit:1362217|Enable chunked videos on testwiki (T373546)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2477084 wikitext text/x-wiki 8sg2o6z425a4rofkapo56zs2k6mwxir 2477085 2477084 2026-11-04T00:05:44Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1170 (T441633)', diff saved to https://phabricator.wikimedia.org/P97187 and previous config saved to /var/cache/conftool/dbconfig/20261104-000542-ladsgroup.json 2477085 wikitext text/x-wiki 8ycsg5im54m8ej24g7fff8l416zmj24 2477086 2477085 2026-11-04T00:06:02Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1174.eqiad.wmnet with reason: Maintenance 2477086 wikitext text/x-wiki bj2njiz67vqbx5hc39728xccrccvvgg 2477088 2477086 2026-11-04T00:07:49Z Stashbot 7414 ladsgroup@deploy2003: ladsgroup: Continuing with deployment 2477088 wikitext text/x-wiki 65vbph8rhkyje1zjyunxad45qmqnhkh 2477089 2477088 2026-11-04T00:08:37Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/push-notifications: apply 2477089 wikitext text/x-wiki 9sjgnbnzj289r3oeyuui80hoykohjur 2477090 2477089 2026-11-04T00:09:05Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/push-notifications: apply 2477090 wikitext text/x-wiki 9e3dlbd3dpid1uwyyv4bztb49luchpf 2477091 2477090 2026-11-04T00:09:43Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/recommendation-api: apply 2477091 wikitext text/x-wiki n6j2cmhwdxrepm3o7b9ro3qy3sjtohk 2477092 2477091 2026-11-04T00:10:20Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/recommendation-api: apply 2477092 wikitext text/x-wiki 4378jz4cn6unbxf0a5xb6o62d8ccqha 2477093 2477092 2026-11-04T00:10:57Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/sessionstore: apply 2477093 wikitext text/x-wiki r8eo3csm8mg15iknh0h9n3z2do6kb7e 2477094 2477093 2026-11-04T00:11:20Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1181.eqiad.wmnet with reason: Maintenance 2477094 wikitext text/x-wiki tl0yll1kf3pbe7xl8j1kskqn28dyhf7 2477095 2477094 2026-11-04T00:11:23Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/sessionstore: apply 2477095 wikitext text/x-wiki dgzs16ikfj3s8tmp71acs9u58wu3jcd 2477096 2477095 2026-11-04T00:11:27Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1181 (T441633)', diff saved to https://phabricator.wikimedia.org/P97188 and previous config saved to /var/cache/conftool/dbconfig/20261104-001120-ladsgroup.json 2477096 wikitext text/x-wiki 8ademh578mbcwpgddhr7ne038zmwrfh 2477097 2477096 2026-11-04T00:12:06Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/tegola-vector-tiles: apply 2477097 wikitext text/x-wiki exzuy0jpbtgemxa6f1n4l147mt2zckt 2477098 2477097 2026-11-04T00:12:54Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/tegola-vector-tiles: apply 2477098 wikitext text/x-wiki niwjc04hxrfwcw1kvpp3coo7eh9gsx8 2477099 2477098 2026-11-04T00:13:08Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/termbox: apply 2477099 wikitext text/x-wiki ipg3cwmcyg9sisl7isjq47lg437f9bq 2477100 2477099 2026-11-04T00:13:50Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/termbox: apply 2477100 wikitext text/x-wiki srzsia88gu4vfpv04yw9osk3uiivnve 2477101 2477100 2026-11-04T00:13:55Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/toolhub: apply 2477101 wikitext text/x-wiki 90vcxj1fsl7vuemdlygzkllcri02a8s 2477102 2477101 2026-11-04T00:14:06Z Stashbot 7414 ladsgroup@deploy2003: Finished scap sync-world: Backport for [[gerrit:1362212|Avoid unnecessary/aborted requests in DASH mode (T442988)]] (1.48.0-wmf.3), [[gerrit:1362217|Enable chunked videos on testwiki (T373546)]] (duration: 13m 02s) 2477102 wikitext text/x-wiki lz56093eny3uve5l93ozy9ilxd2s76a 2477103 2477102 2026-11-04T00:14:36Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/toolhub: apply 2477103 wikitext text/x-wiki 5x1nswytc8oo1jtl99564kanyozkhpp 2477104 2477103 2026-11-04T00:14:48Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/wikidata-query-gui: apply 2477104 wikitext text/x-wiki ncaei4qupujc32h177c7tcmp9xf36nr 2477105 2477104 2026-11-04T00:15:10Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/wikidata-query-gui: apply 2477105 wikitext text/x-wiki 598jtfdpp5qhiguv6m0f0r9oos4m1pu 2477106 2477105 2026-11-04T00:15:40Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/wikifeeds: apply 2477106 wikitext text/x-wiki s9vfejr0zfye35n2wlgt4811186bv5e 2477107 2477106 2026-11-04T00:15:54Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/wikifeeds: apply 2477107 wikitext text/x-wiki 8zb02dds54hiltey4zmsx7jnsj51g2j 2477108 2477107 2026-11-04T00:15:59Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2477108 wikitext text/x-wiki lmyfdl1kob78q6o0lomf68pwgjanf1c 2477112 2477108 2026-11-04T00:19:22Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1181 (T441633)', diff saved to https://phabricator.wikimedia.org/P97189 and previous config saved to /var/cache/conftool/dbconfig/20261104-001920-ladsgroup.json 2477112 wikitext text/x-wiki p58hk7glxtakcmupcnd9tvewyctf2l0 2477113 2477112 2026-11-04T00:26:08Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2477113 wikitext text/x-wiki 3v9t703s5kps1kzfy9c4w208ti4f82q 2477114 2477113 2026-11-04T00:27:39Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2477114 wikitext text/x-wiki l6oofs3avzvdfabejecqre98jzupdpx 2477115 2477114 2026-11-04T00:27:49Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2477115 wikitext text/x-wiki 9eje3qbnp8bktfts1vmfr1ieypw9qn8 2477116 2477115 2026-11-04T00:28:09Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/zotero: apply 2477116 wikitext text/x-wiki pte9neg34crxiz99ikkfjv7ptp1gp5s 2477117 2477116 2026-11-04T00:28:28Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/zotero: apply 2477117 wikitext text/x-wiki o4b0xse988fiqhn8hg12lu6nmqk9rf0 2477118 2477117 2026-11-04T00:29:24Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1181', diff saved to https://phabricator.wikimedia.org/P97190 and previous config saved to /var/cache/conftool/dbconfig/20261104-002922-ladsgroup.json 2477118 wikitext text/x-wiki 5y401vxqcmm313rj2jgai6ckf640xe6 2477119 2477118 2026-11-04T00:30:53Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] START helmfile.d/services/kartotherian: apply 2477119 wikitext text/x-wiki 35l3zjp2ogwuw78quqru7jd8hhdvdpc 2477120 2477119 2026-11-04T00:32:01Z Stashbot 7414 rzl@deploy2003: helmfile [codfw] DONE helmfile.d/services/kartotherian: apply 2477120 wikitext text/x-wiki qseibc3hd5zm09tuo3a507zb1wk37j0 2477121 2477120 2026-11-04T00:39:26Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1181', diff saved to https://phabricator.wikimedia.org/P97191 and previous config saved to /var/cache/conftool/dbconfig/20261104-003924-ladsgroup.json 2477121 wikitext text/x-wiki jsgq4emt681ipgzwynpq13e6ovhc6e1 2477124 2477121 2026-11-04T00:49:28Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1181 (T441633)', diff saved to https://phabricator.wikimedia.org/P97192 and previous config saved to /var/cache/conftool/dbconfig/20261104-004926-ladsgroup.json 2477124 wikitext text/x-wiki 8q57g906jw9tm1ys9ef48zzey3iisv1 2477125 2477124 2026-11-04T00:49:36Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1190.eqiad.wmnet with reason: Maintenance 2477125 wikitext text/x-wiki tar61dqe07k5okpp34ukohv4kfo445s 2477126 2477125 2026-11-04T00:49:39Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1190 (T441633)', diff saved to https://phabricator.wikimedia.org/P97193 and previous config saved to /var/cache/conftool/dbconfig/20261104-004935-ladsgroup.json 2477126 wikitext text/x-wiki 3ovl2s4dwryv3fnorsbt3gjdu5ka5er 2477127 2477126 2026-11-04T00:58:05Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1190 (T441633)', diff saved to https://phabricator.wikimedia.org/P97194 and previous config saved to /var/cache/conftool/dbconfig/20261104-005803-ladsgroup.json 2477127 wikitext text/x-wiki amyyzuqtsmqfdgtqjfpv3ixq3kxao8k 2477128 2477127 2026-11-04T01:08:07Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1190', diff saved to https://phabricator.wikimedia.org/P97195 and previous config saved to /var/cache/conftool/dbconfig/20261104-010805-ladsgroup.json 2477128 wikitext text/x-wiki qdoruku74lbni1m7jiz43f7qixaig5y 2477129 2477128 2026-11-04T01:18:09Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1190', diff saved to https://phabricator.wikimedia.org/P97196 and previous config saved to /var/cache/conftool/dbconfig/20261104-011807-ladsgroup.json 2477129 wikitext text/x-wiki 8u3qf9gqt69n3gp198cxlhele9rhn9z 2477130 2477129 2026-11-04T01:28:11Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1190 (T441633)', diff saved to https://phabricator.wikimedia.org/P97197 and previous config saved to /var/cache/conftool/dbconfig/20261104-012809-ladsgroup.json 2477130 wikitext text/x-wiki b2v1f9iy9n5yi62pqwc3rq39am3gdsv 2477131 2477130 2026-11-04T01:28:29Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1191.eqiad.wmnet with reason: Maintenance 2477131 wikitext text/x-wiki o7yskk33cp8j0ogqnbrtx5e9cbnwiaf 2477132 2477131 2026-11-04T01:28:32Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1191 (T441633)', diff saved to https://phabricator.wikimedia.org/P97198 and previous config saved to /var/cache/conftool/dbconfig/20261104-012829-ladsgroup.json 2477132 wikitext text/x-wiki sltt48n5pfcwn0yvpl5a18vuu01blzt 2477133 2477132 2026-11-04T01:35:21Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1191 (T441633)', diff saved to https://phabricator.wikimedia.org/P97199 and previous config saved to /var/cache/conftool/dbconfig/20261104-013519-ladsgroup.json 2477133 wikitext text/x-wiki ggjipqdwiz4wskev8ldvecij7uxvzio 2477137 2477133 2026-11-04T01:45:23Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1191', diff saved to https://phabricator.wikimedia.org/P97200 and previous config saved to /var/cache/conftool/dbconfig/20261104-014521-ladsgroup.json 2477137 wikitext text/x-wiki f9fc0e05137px1i050i2yesefvzdz8i 2477138 2477137 2026-11-04T01:55:25Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1191', diff saved to https://phabricator.wikimedia.org/P97201 and previous config saved to /var/cache/conftool/dbconfig/20261104-015523-ladsgroup.json 2477138 wikitext text/x-wiki dvrx983clm036mwm9cn88q43f1rm6ah 2477139 2477138 2026-11-04T02:00:51Z Stashbot 7414 mwpresync@deploy2003: Started scap build-images: Publishing wmf/next image 2477139 wikitext text/x-wiki lw9wp8eniyrhl2m5i5w82z8v831ethx 2477140 2477139 2026-11-04T02:05:29Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1191 (T441633)', diff saved to https://phabricator.wikimedia.org/P97202 and previous config saved to /var/cache/conftool/dbconfig/20261104-020525-ladsgroup.json 2477140 wikitext text/x-wiki 1wki6m9wcgd1b5z88eqwl26i80jx30r 2477141 2477140 2026-11-04T02:05:45Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1194.eqiad.wmnet with reason: Maintenance 2477141 wikitext text/x-wiki c5fq9q9oxzxfk56lmen2nfabxqjzecc 2477142 2477141 2026-11-04T02:05:48Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1194 (T441633)', diff saved to https://phabricator.wikimedia.org/P97203 and previous config saved to /var/cache/conftool/dbconfig/20261104-020545-ladsgroup.json 2477142 wikitext text/x-wiki 2xkjgvcbw5cxxjt8xpck8eee5rp9gwo 2477143 2477142 2026-11-04T02:08:25Z Stashbot 7414 mwpresync@deploy2003: Finished scap build-images: Publishing wmf/next image (duration: 07m 33s) 2477143 wikitext text/x-wiki qd8qqkqlcdjuqkkxo3pzynb4tye8hmr 2477144 2477143 2026-11-04T02:13:58Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1194 (T441633)', diff saved to https://phabricator.wikimedia.org/P97204 and previous config saved to /var/cache/conftool/dbconfig/20261104-021356-ladsgroup.json 2477144 wikitext text/x-wiki ph8xn2p2ividcob2cnpoim9bxmx09wh 2477145 2477144 2026-11-04T02:24:00Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1194', diff saved to https://phabricator.wikimedia.org/P97205 and previous config saved to /var/cache/conftool/dbconfig/20261104-022358-ladsgroup.json 2477145 wikitext text/x-wiki b6jyjvtb368reyxxpm9y5obbrz2jjjv 2477147 2477145 2026-11-04T02:34:02Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1194', diff saved to https://phabricator.wikimedia.org/P97206 and previous config saved to /var/cache/conftool/dbconfig/20261104-023401-ladsgroup.json 2477147 wikitext text/x-wiki sy6ldxwmftun310cmjjuuic1w0m17u9 2477149 2477147 2026-11-04T02:44:05Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1194 (T441633)', diff saved to https://phabricator.wikimedia.org/P97207 and previous config saved to /var/cache/conftool/dbconfig/20261104-024403-ladsgroup.json 2477149 wikitext text/x-wiki ntge5xlhszer778r3882dqf3b54q1tl 2477151 2477149 2026-11-04T02:44:23Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1202.eqiad.wmnet with reason: Maintenance 2477151 wikitext text/x-wiki o68yal7ybigk4xpqotvi2wkuk0jdfi5 2477152 2477151 2026-11-04T02:44:25Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1202 (T441633)', diff saved to https://phabricator.wikimedia.org/P97208 and previous config saved to /var/cache/conftool/dbconfig/20261104-024423-ladsgroup.json 2477152 wikitext text/x-wiki 12pfty3hus44g7llndviw5z6uxe1inh 2477153 2477152 2026-11-04T02:51:25Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1202 (T441633)', diff saved to https://phabricator.wikimedia.org/P97209 and previous config saved to /var/cache/conftool/dbconfig/20261104-025123-ladsgroup.json 2477153 wikitext text/x-wiki 7awhaocwk7la2ym76i4f5rak1b61mny 2477158 2477153 2026-11-04T03:01:27Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1202', diff saved to https://phabricator.wikimedia.org/P97210 and previous config saved to /var/cache/conftool/dbconfig/20261104-030125-ladsgroup.json 2477158 wikitext text/x-wiki e28mjqwhhlhxc2qyuwrpu2rtnqgqrqv 2477167 2477158 2026-11-04T03:11:29Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1202', diff saved to https://phabricator.wikimedia.org/P97211 and previous config saved to /var/cache/conftool/dbconfig/20261104-031127-ladsgroup.json 2477167 wikitext text/x-wiki 3rywnbkfs5wi8j132u7m3cto7v23it6 2477168 2477167 2026-11-04T03:21:31Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1202 (T441633)', diff saved to https://phabricator.wikimedia.org/P97212 and previous config saved to /var/cache/conftool/dbconfig/20261104-032129-ladsgroup.json 2477168 wikitext text/x-wiki ci2v25z0wlesv1z11jrgez5gvfb4g2a 2477169 2477168 2026-11-04T03:21:49Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1227.eqiad.wmnet with reason: Maintenance 2477169 wikitext text/x-wiki 92n4eap0dojqslzzhyku121a6qd1rqi 2477170 2477169 2026-11-04T03:21:51Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1227 (T441633)', diff saved to https://phabricator.wikimedia.org/P97213 and previous config saved to /var/cache/conftool/dbconfig/20261104-032149-ladsgroup.json 2477170 wikitext text/x-wiki hgbkf5zgs3ev8cygun81l9ynwbrhonm 2477171 2477170 2026-11-04T03:28:57Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1227 (T441633)', diff saved to https://phabricator.wikimedia.org/P97214 and previous config saved to /var/cache/conftool/dbconfig/20261104-032855-ladsgroup.json 2477171 wikitext text/x-wiki 4n0aeejwphrclcydpionb85zqnukpib 2477172 2477171 2026-11-04T03:38:59Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1227', diff saved to https://phabricator.wikimedia.org/P97215 and previous config saved to /var/cache/conftool/dbconfig/20261104-033857-ladsgroup.json 2477172 wikitext text/x-wiki mh9pczw5ushfnfu6ff3l10j3nycecjj 2477173 2477172 2026-11-04T03:49:01Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1227', diff saved to https://phabricator.wikimedia.org/P97216 and previous config saved to /var/cache/conftool/dbconfig/20261104-034859-ladsgroup.json 2477173 wikitext text/x-wiki hxwqoqcf5nts0k72atxcpzly9bbubsk 2477174 2477173 2026-11-04T03:59:03Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1227 (T441633)', diff saved to https://phabricator.wikimedia.org/P97217 and previous config saved to /var/cache/conftool/dbconfig/20261104-035901-ladsgroup.json 2477174 wikitext text/x-wiki rz2xxh8l6jm3kk3j1nvnddpzdvk2a7p 2477175 2477174 2026-11-04T03:59:22Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1231.eqiad.wmnet with reason: Maintenance 2477175 wikitext text/x-wiki 44xrvwqiq1s3ppthq8rptn58k1x8gho 2477176 2477175 2026-11-04T03:59:25Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1231 (T441633)', diff saved to https://phabricator.wikimedia.org/P97218 and previous config saved to /var/cache/conftool/dbconfig/20261104-035922-ladsgroup.json 2477176 wikitext text/x-wiki an5kvmul1kuknzhq3w26jtm2s978owm 2477178 2477176 2026-11-04T04:06:15Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1231 (T441633)', diff saved to https://phabricator.wikimedia.org/P97219 and previous config saved to /var/cache/conftool/dbconfig/20261104-040613-ladsgroup.json 2477178 wikitext text/x-wiki h89bno2dasz9pblk8t661nw4q2xjfnj 2477180 2477178 2026-11-04T04:16:17Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1231', diff saved to https://phabricator.wikimedia.org/P97220 and previous config saved to /var/cache/conftool/dbconfig/20261104-041615-ladsgroup.json 2477180 wikitext text/x-wiki c5l70y2xnhr7t33ny7mcuan3s2qpsta 2477182 2477180 2026-11-04T04:26:19Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1231', diff saved to https://phabricator.wikimedia.org/P97221 and previous config saved to /var/cache/conftool/dbconfig/20261104-042617-ladsgroup.json 2477182 wikitext text/x-wiki tshvvd8mek1qo96z5ixew77q5zo7z2q 2477186 2477182 2026-11-04T04:36:21Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1231 (T441633)', diff saved to https://phabricator.wikimedia.org/P97222 and previous config saved to /var/cache/conftool/dbconfig/20261104-043619-ladsgroup.json 2477186 wikitext text/x-wiki n0wj5pj6w1g8ubid3e7tt0ard9qtchk 2477187 2477186 2026-11-04T04:36:39Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1253.eqiad.wmnet with reason: Maintenance 2477187 wikitext text/x-wiki fv76f1bc26gow7sv4vp0k4ogqwhsgx6 2477188 2477187 2026-11-04T04:36:42Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1253 (T441633)', diff saved to https://phabricator.wikimedia.org/P97223 and previous config saved to /var/cache/conftool/dbconfig/20261104-043639-ladsgroup.json 2477188 wikitext text/x-wiki 2ebiv5mcpkdyefxyjoln33z5sa3wrps 2477189 2477188 2026-11-04T04:43:47Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1253 (T441633)', diff saved to https://phabricator.wikimedia.org/P97224 and previous config saved to /var/cache/conftool/dbconfig/20261104-044345-ladsgroup.json 2477189 wikitext text/x-wiki fbfi1d1e8ouvnfxovsk67ijhzqrq665 2477190 2477189 2026-11-04T04:53:49Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1253', diff saved to https://phabricator.wikimedia.org/P97225 and previous config saved to /var/cache/conftool/dbconfig/20261104-045347-ladsgroup.json 2477190 wikitext text/x-wiki 99btz75bqzbddyao6iu9537bvy5fqiq 2477191 2477190 2026-11-04T05:03:51Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1253', diff saved to https://phabricator.wikimedia.org/P97226 and previous config saved to /var/cache/conftool/dbconfig/20261104-050349-ladsgroup.json 2477191 wikitext text/x-wiki sp7gayuxiimdkr9xf9jcezosgbujdu5 2477192 2477191 2026-11-04T05:13:54Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1253 (T441633)', diff saved to https://phabricator.wikimedia.org/P97227 and previous config saved to /var/cache/conftool/dbconfig/20261104-051351-ladsgroup.json 2477192 wikitext text/x-wiki 9r95yys4qq98ysingh3aafvdbrbfpy3 2477193 2477192 2026-11-04T05:14:12Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1273.eqiad.wmnet with reason: Maintenance 2477193 wikitext text/x-wiki tm0zd7mob5k9ufde3n96snurpydnppf 2477194 2477193 2026-11-04T05:14:24Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on 7 hosts with reason: Maintenance 2477194 wikitext text/x-wiki 3zw1c9hi5ik5fjbjvmrua41ewob7kqe 2477195 2477194 2026-11-04T05:14:26Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1273 (T441633)', diff saved to https://phabricator.wikimedia.org/P97228 and previous config saved to /var/cache/conftool/dbconfig/20261104-051423-ladsgroup.json 2477195 wikitext text/x-wiki f278cm8lvf2jurgbx26ydlm26roxxgl 2477197 2477195 2026-11-04T05:21:38Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1273 (T441633)', diff saved to https://phabricator.wikimedia.org/P97229 and previous config saved to /var/cache/conftool/dbconfig/20261104-052135-ladsgroup.json 2477197 wikitext text/x-wiki egau950w6gtd9np0yx5strn31f01v42 2477198 2477197 2026-11-04T05:31:39Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1273', diff saved to https://phabricator.wikimedia.org/P97230 and previous config saved to /var/cache/conftool/dbconfig/20261104-053137-ladsgroup.json 2477198 wikitext text/x-wiki nu9xu7a95bzrbb53bi8li61gu08ttj2 2477199 2477198 2026-11-04T05:41:41Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1273', diff saved to https://phabricator.wikimedia.org/P97231 and previous config saved to /var/cache/conftool/dbconfig/20261104-054139-ladsgroup.json 2477199 wikitext text/x-wiki 3n5yupaffzh7x32w1imr8zxsecvdgxs 2477200 2477199 2026-11-04T05:51:43Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1273 (T441633)', diff saved to https://phabricator.wikimedia.org/P97232 and previous config saved to /var/cache/conftool/dbconfig/20261104-055141-ladsgroup.json 2477200 wikitext text/x-wiki fofndowwdynn3b60jsxksdaitlqt1b6 2477201 2477200 2026-11-04T05:52:01Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1284.eqiad.wmnet with reason: Maintenance 2477201 wikitext text/x-wiki q2fdk56n3fzw6nz8mzi3xhe9i7prstb 2477202 2477201 2026-11-04T05:52:04Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1284 (T441633)', diff saved to https://phabricator.wikimedia.org/P97233 and previous config saved to /var/cache/conftool/dbconfig/20261104-055201-ladsgroup.json 2477202 wikitext text/x-wiki qdrk9l8e9k623z2kk96ltn4vz9en6dl 2477203 2477202 2026-11-04T05:57:08Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1284 (T441633)', diff saved to https://phabricator.wikimedia.org/P97234 and previous config saved to /var/cache/conftool/dbconfig/20261104-055706-ladsgroup.json 2477203 wikitext text/x-wiki 2bfwrn8z5pg7edxbqhycfp5arx3rvse 2477204 2477203 2026-11-04T06:07:10Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1284', diff saved to https://phabricator.wikimedia.org/P97235 and previous config saved to /var/cache/conftool/dbconfig/20261104-060708-ladsgroup.json 2477204 wikitext text/x-wiki kociemrto6ifk0mckzfdib6227nnna1 2477205 2477204 2026-11-04T06:17:12Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1284', diff saved to https://phabricator.wikimedia.org/P97236 and previous config saved to /var/cache/conftool/dbconfig/20261104-061710-ladsgroup.json 2477205 wikitext text/x-wiki 12co1kn1f3ojstd27lmiqkosrln98nc 2477206 2477205 2026-11-04T06:27:14Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1284 (T441633)', diff saved to https://phabricator.wikimedia.org/P97237 and previous config saved to /var/cache/conftool/dbconfig/20261104-062712-ladsgroup.json 2477206 wikitext text/x-wiki 05qxmdshjegmkbgvtge233sysmmh9jc 2477207 2477206 2026-11-04T06:27:31Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1285.eqiad.wmnet with reason: Maintenance 2477207 wikitext text/x-wiki fkbm6ju23aa17zgozkukemlc2k69okf 2477208 2477207 2026-11-04T06:30:54Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1288.eqiad.wmnet with reason: Maintenance 2477208 wikitext text/x-wiki qq2p39wkykwy0l0m7hyva3chvvnriv4 2477209 2477208 2026-11-04T06:30:57Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db1288 (T441633)', diff saved to https://phabricator.wikimedia.org/P97238 and previous config saved to /var/cache/conftool/dbconfig/20261104-063055-ladsgroup.json 2477209 wikitext text/x-wiki cnnpw3dvj13hf4bxbq2obx7ojgx1g0m 2477210 2477209 2026-11-04T06:35:57Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1288 (T441633)', diff saved to https://phabricator.wikimedia.org/P97239 and previous config saved to /var/cache/conftool/dbconfig/20261104-063554-ladsgroup.json 2477210 wikitext text/x-wiki 4zjq61pv9ulli1be6ikyo71gj4z60eb 2477213 2477210 2026-11-04T06:45:58Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1288', diff saved to https://phabricator.wikimedia.org/P97240 and previous config saved to /var/cache/conftool/dbconfig/20261104-064556-ladsgroup.json 2477213 wikitext text/x-wiki 5nmm2e02tzb8v8v36bda3tb029dj53i 2477214 2477213 2026-11-04T06:55:52Z Stashbot 7414 marostegui@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1274.eqiad.wmnet with reason: Server didn't come back 2477214 wikitext text/x-wiki n4dyl0me9l03mngr991jxyfzmlr1rox 2477215 2477214 2026-11-04T06:56:00Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1288', diff saved to https://phabricator.wikimedia.org/P97241 and previous config saved to /var/cache/conftool/dbconfig/20261104-065558-ladsgroup.json 2477215 wikitext text/x-wiki rk792csn3h4m1l8ssej6wxdn8heywul 2477216 2477215 2026-11-04T07:06:02Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db1288 (T441633)', diff saved to https://phabricator.wikimedia.org/P97242 and previous config saved to /var/cache/conftool/dbconfig/20261104-070600-ladsgroup.json 2477216 wikitext text/x-wiki 7ttx2f829r2pg8d4pe6f09rmvgoqk0o 2477217 2477216 2026-11-04T07:06:21Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance 2477217 wikitext text/x-wiki sbzpqq1qnpziuztdp08j17squjq4wzx 2477218 2477217 2026-11-04T07:15:24Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.mysql.pool pool db1274: after reboot 2477218 wikitext text/x-wiki 68xxdw1bc9krjc19ix4mngop2cb6bph 2477219 2477218 2026-11-04T07:30:19Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.hosts.reimage for host dbprov2005.codfw.wmnet with OS trixie 2477219 wikitext text/x-wiki 0ghbwhqib54322upin40x6v23j2h1fy 2477220 2477219 2026-11-04T07:31:15Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.hosts.dhcp for host dbprov2005.codfw.wmnet (push config only) 2477220 wikitext text/x-wiki gfyi4oyye94meagkelpfy268otkgebz 2477221 2477220 2026-11-04T07:31:20Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host dbprov2005.codfw.wmnet (push config only) 2477221 wikitext text/x-wiki 00dr5eqvtwlnsxc2rvxaxsu7m0okevy 2477222 2477221 2026-11-04T07:45:18Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.hosts.dhcp for host dbprov2005.codfw.wmnet (remove config only) 2477222 wikitext text/x-wiki kryq0k0p8zo0s5kam589zudlhudbbdx 2477223 2477222 2026-11-04T07:45:23Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host dbprov2005.codfw.wmnet (remove config only) 2477223 wikitext text/x-wiki cm61erl5pv8wqmexnd9ov8f637ed5ln 2477224 2477223 2026-11-04T07:47:52Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on dbprov2005.codfw.wmnet with reason: host reimage 2477224 wikitext text/x-wiki j3jdqg26cu54zvgubh7myvt62mb8jic 2477225 2477224 2026-11-04T07:53:35Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbprov2005.codfw.wmnet with reason: host reimage 2477225 wikitext text/x-wiki 48ke04jnqktaarud0cyfb6fbpzuk882 2477226 2477225 2026-11-04T08:00:28Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1274: after reboot 2477226 wikitext text/x-wiki rf46y2jpd4p7ovhb58prdfxg5okk2x9 2477228 2477226 2026-11-04T08:06:49Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host bast3007.wikimedia.org 2477228 wikitext text/x-wiki 16s0edizxvrisxyuyahiepbt3qsuqos 2477229 2477228 2026-11-04T08:10:55Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast3007.wikimedia.org 2477229 wikitext text/x-wiki aaba0yfqn9gy3blqx90kwopkcrmjuxs 2477230 2477229 2026-11-04T08:11:23Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dbprov2005.codfw.wmnet with OS trixie 2477230 wikitext text/x-wiki h9v1u8030j9hz041qlxld9cno3xyjg8 2477232 2477230 2026-11-04T08:15:58Z Stashbot 7414 Tran: Deployed patch for T441775 2477232 wikitext text/x-wiki crpdypvwgfy6v4nisb414h189eaz52l 2477234 2477232 2026-11-04T08:50:35Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.hosts.reimage for host dbprov2006.codfw.wmnet with OS trixie 2477234 wikitext text/x-wiki tw68geydz291lng4meaar7uhoya06xg 2477235 2477234 2026-11-04T08:51:32Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.hosts.dhcp for host dbprov2006.codfw.wmnet (push config only) 2477235 wikitext text/x-wiki jw951ooy2eaa46stimrvj1t5slfbhrk 2477236 2477235 2026-11-04T08:51:36Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host dbprov2006.codfw.wmnet (push config only) 2477236 wikitext text/x-wiki p524zy4z5mwkw8bj6qyovmw6k18nzvl 2477237 2477236 2026-11-04T09:05:02Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.hosts.dhcp for host dbprov2006.codfw.wmnet (remove config only) 2477237 wikitext text/x-wiki bejh319r3gqg9a3kgwldlqzax0rorwy 2477238 2477237 2026-11-04T09:05:08Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host dbprov2006.codfw.wmnet (remove config only) 2477238 wikitext text/x-wiki 5wk9fhnppojndx9g8wysotstu6pq5sz 2477240 2477238 2026-11-04T09:07:15Z Stashbot 7414 arnaudb@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on gerrit2002.wikimedia.org with reason: T440177 offline reindex test 2477240 wikitext text/x-wiki rpxnaf80g473ysrs5sptzj4d7je9uh2 2477241 2477240 2026-11-04T09:07:30Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on dbprov2006.codfw.wmnet with reason: host reimage 2477241 wikitext text/x-wiki iyo5q5jszkwj2lgch8yyzt0qpqg74f7 2477242 2477241 2026-11-04T09:08:20Z Stashbot 7414 cwilliams@cumin1004: START - Cookbook sre.mysql.pool pool es2036: Maintenance 2477242 wikitext text/x-wiki nzgd37fgcjb925c9bvg12tmnuamr1tn 2477243 2477242 2026-11-04T09:12:42Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbprov2006.codfw.wmnet with reason: host reimage 2477243 wikitext text/x-wiki ry2dbkb6f78lep1ppvyjfhjnnsphzsf 2477244 2477243 2026-11-04T09:13:54Z Stashbot 7414 jnuche@deploy2003: rebuilt and synchronized wikiversions files: group1 to 1.48.0-wmf.3 refs T442221 2477244 wikitext text/x-wiki 148cxfdtdm4i6jrr2frrr8xtvezu169 2477245 2477244 2026-11-04T09:14:02Z Stashbot 7414 brouberol@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook: apply 2477245 wikitext text/x-wiki qerd7gafm0nf9dxvwo5alwmu8rscn9c 2477246 2477245 2026-11-04T09:14:08Z Stashbot 7414 brouberol@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook: apply 2477246 wikitext text/x-wiki blbfxm7iu0ud7f7dee8tngbusurvw4l 2477247 2477246 2026-11-04T09:21:42Z Stashbot 7414 brouberol@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook: apply 2477247 wikitext text/x-wiki b3u94uvkxbnzqylklzebmiriykypqyk 2477248 2477247 2026-11-04T09:21:49Z Stashbot 7414 brouberol@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook: apply 2477248 wikitext text/x-wiki 4eat94sbk2ykcqvvbq5jj4f11a8yufp 2477250 2477248 2026-11-04T09:31:05Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dbprov2006.codfw.wmnet with OS trixie 2477250 wikitext text/x-wiki c2vsgvbmzvqdif2yy14g8jq7ir7i2b5 2477251 2477250 2026-11-04T09:44:22Z Stashbot 7414 brouberol@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/growthbook: apply 2477251 wikitext text/x-wiki 9q8iv4zjvpe344207rwl4n4s79clgv2 2477252 2477251 2026-11-04T09:45:18Z Stashbot 7414 brouberol@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/growthbook: apply 2477252 wikitext text/x-wiki 48ynyp3eaq39rkhciit1rlls6rfxej5 2477253 2477252 2026-11-04T09:45:26Z Stashbot 7414 brouberol@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/ferretdb-growthbook: apply 2477253 wikitext text/x-wiki 8xxwv7ec5um45cgcapw25jgm50e8gbo 2477254 2477253 2026-11-04T09:45:29Z Stashbot 7414 brouberol@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/ferretdb-growthbook: apply 2477254 wikitext text/x-wiki e89w26jdabvdio70vhypxlm802x7ex3 2477256 2477254 2026-11-04T09:48:50Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.hosts.reimage for host dbprov2004.codfw.wmnet with OS trixie 2477256 wikitext text/x-wiki joeygiyxu8giwytqiv9q91axwby9sj4 2477257 2477256 2026-11-04T09:49:46Z Stashbot 7414 brouberol@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/growthbook: apply 2477257 wikitext text/x-wiki 8hjev6u4pt3yda7wr0hn9gmzzaj6mna 2477258 2477257 2026-11-04T09:49:49Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.hosts.dhcp for host dbprov2004.codfw.wmnet (push config only) 2477258 wikitext text/x-wiki bn5coyxn2bn4usloool2y4sgafqar7b 2477259 2477258 2026-11-04T09:49:52Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host dbprov2004.codfw.wmnet (push config only) 2477259 wikitext text/x-wiki ou5eurw931rmrwobwxvsr5u9trbxnzf 2477260 2477259 2026-11-04T09:50:41Z Stashbot 7414 brouberol@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/growthbook: apply 2477260 wikitext text/x-wiki b7e17ygpw9oqkdquz1axrdtzrowphvj 2477261 2477260 2026-11-04T09:53:30Z Stashbot 7414 cwilliams@cumin1004: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es2036: Maintenance 2477261 wikitext text/x-wiki o47kwb0xflr8wi1hf603p3tgmw19lqb 2477262 2477261 2026-11-04T09:54:03Z Stashbot 7414 brouberol@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook: apply 2477262 wikitext text/x-wiki t8orcdu2vvjdassfszjvdajsdy3h02i 2477263 2477262 2026-11-04T09:54:09Z Stashbot 7414 brouberol@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook: apply 2477263 wikitext text/x-wiki quzcz0qiwagc1rvxa2vs88p34kiv2kg 2477265 2477263 2026-11-04T09:58:40Z Stashbot 7414 klausman@cumin1004: START - Cookbook sre.hadoop.check-user-leftovers Checking for leftover data for user akosiaris 2477265 wikitext text/x-wiki nyfbd11wg9p0u1fn6ulyot397laz6x1 2477266 2477265 2026-11-04T09:58:50Z Stashbot 7414 klausman@cumin1004: END (PASS) - Cookbook sre.hadoop.check-user-leftovers (exit_code=0) Checking for leftover data for user akosiaris 2477266 wikitext text/x-wiki cobyurwqmbbi7bt51rm915613ae7opo 2477267 2477266 2026-11-04T09:59:02Z Stashbot 7414 klausman@cumin1004: START - Cookbook sre.hadoop.check-user-leftovers Checking for leftover data for user akosiaris 2477267 wikitext text/x-wiki 7p9rptquolggu35frdnnm0oihj6gltf 2477268 2477267 2026-11-04T09:59:11Z Stashbot 7414 klausman@cumin1004: END (PASS) - Cookbook sre.hadoop.check-user-leftovers (exit_code=0) Checking for leftover data for user akosiaris 2477268 wikitext text/x-wiki 6m2dbz2f698xxwxc7n9xf5ek9us9lsd 2477269 2477268 2026-11-04T10:03:34Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.hosts.dhcp for host dbprov2004.codfw.wmnet (remove config only) 2477269 wikitext text/x-wiki grt8g6lm0gcvg4h7fs3i13pv427qt8p 2477270 2477269 2026-11-04T10:03:39Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host dbprov2004.codfw.wmnet (remove config only) 2477270 wikitext text/x-wiki qkxahmjfol4wnx8mysz40emqofvs1zr 2477271 2477270 2026-11-04T10:06:14Z Stashbot 7414 marostegui@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on dbprov2004.codfw.wmnet with reason: host reimage 2477271 wikitext text/x-wiki 08c6z3k68zpsodgh2le0md951sq05pt 2477272 2477271 2026-11-04T10:09:41Z Stashbot 7414 blake@deploy2003: helmfile [staging] START helmfile.d/services/changeprop-jobqueue: apply 2477272 wikitext text/x-wiki me9ndm7vl7till540r5x6zc1c04ahh1 2477273 2477272 2026-11-04T10:10:33Z Stashbot 7414 blake@deploy2003: helmfile [staging] DONE helmfile.d/services/changeprop-jobqueue: apply 2477273 wikitext text/x-wiki pu5pr2mu8tzseu0njlqxc3aokj6n5iu 2477274 2477273 2026-11-04T10:10:52Z Stashbot 7414 blake@deploy2003: helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply 2477274 wikitext text/x-wiki lscai44ljmnb93opjcoahh3c5wr05og 2477276 2477274 2026-11-04T10:12:01Z Stashbot 7414 blake@deploy2003: helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply 2477276 wikitext text/x-wiki tcv3j0mg5pv76iuvdwdlg5xrtdytyxc 2477277 2477276 2026-11-04T10:12:21Z Stashbot 7414 cmooney@cumin1004: START - Cookbook sre.hosts.reboot-single for host lvs1020.eqiad.wmnet 2477277 wikitext text/x-wiki 14y9aokmlytwm02uuqz8vst3fs7l3z7 2477278 2477277 2026-11-04T10:14:07Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbprov2004.codfw.wmnet with reason: host reimage 2477278 wikitext text/x-wiki jl21ncza4wm2jlsjf9wdw48kynv17c8 2477279 2477278 2026-11-04T10:17:31Z Stashbot 7414 blake@deploy2003: helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: apply 2477279 wikitext text/x-wiki 98e0jh0909vda5gzylw1dciuhmgqaba 2477280 2477279 2026-11-04T10:18:30Z Stashbot 7414 blake@deploy2003: helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: apply 2477280 wikitext text/x-wiki s80zjpe18skltsiblgtfbtzjd4tgtv8 2477281 2477280 2026-11-04T10:19:47Z Stashbot 7414 gkyziridis@deploy2003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . 2477281 wikitext text/x-wiki cux0zusy6lr3se8uqxxi2fjkbnctubw 2477282 2477281 2026-11-04T10:19:53Z Stashbot 7414 gkyziridis@deploy2003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'llmisvc' . 2477282 wikitext text/x-wiki ljq1trnrmojy55oem70ks8x89ludmnc 2477283 2477282 2026-11-04T10:27:20Z Stashbot 7414 klausman@cumin1004: START - Cookbook sre.hadoop.check-user-leftovers Checking for leftover data for user akosiaris 2477283 wikitext text/x-wiki ezugyxyveoz9tdowxgt8mc0u4sxi67v 2477284 2477283 2026-11-04T10:27:29Z Stashbot 7414 klausman@cumin1004: END (PASS) - Cookbook sre.hadoop.check-user-leftovers (exit_code=0) Checking for leftover data for user akosiaris 2477284 wikitext text/x-wiki lpujcwmpcmu4nvzdgr8ja4n0i6qrgk4 2477285 2477284 2026-11-04T10:28:01Z Stashbot 7414 klausman@cumin1004: START - Cookbook sre.hadoop.check-user-leftovers Checking for leftover data for user akosiaris 2477285 wikitext text/x-wiki 74iym5ssi4wlbffsveslou0hg4jy0zi 2477286 2477285 2026-11-04T10:28:05Z Stashbot 7414 klausman@cumin1004: END (ERROR) - Cookbook sre.hadoop.check-user-leftovers (exit_code=97) Checking for leftover data for user akosiaris 2477286 wikitext text/x-wiki jvs4k84dvaeim61ecplegus2bp64wzp 2477287 2477286 2026-11-04T10:28:07Z Stashbot 7414 klausman@cumin1004: START - Cookbook sre.hadoop.check-user-leftovers Checking for leftover data for user akosiaris 2477287 wikitext text/x-wiki t84sc3quc2kic30dl6rmmkhug3f5gyf 2477288 2477287 2026-11-04T10:28:15Z Stashbot 7414 klausman@cumin1004: END (PASS) - Cookbook sre.hadoop.check-user-leftovers (exit_code=0) Checking for leftover data for user akosiaris 2477288 wikitext text/x-wiki bxaq9sl2vakf2vwumrb4sngdd6hbho5 2477289 2477288 2026-11-04T10:32:37Z Stashbot 7414 marostegui@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dbprov2004.codfw.wmnet with OS trixie 2477289 wikitext text/x-wiki fhj8ogh0cd6rvrw1wnbrrgengbox566 2477291 2477289 2026-11-04T10:34:22Z Stashbot 7414 cmooney@cumin1004: END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) for host lvs1020.eqiad.wmnet 2477291 wikitext text/x-wiki 1bwzi03llfzsefzs2sz04b9uf1zh433 2477292 2477291 2026-11-04T10:49:02Z Stashbot 7414 moritzm: installing vim security updates 2477292 wikitext text/x-wiki rqluo5zmkbxlzik2n02c0nfz1tnb7a2 2477293 2477292 2026-11-04T10:52:43Z Stashbot 7414 fabfur: temporary disabling puppet on A:cp to fix https://gerrit.wikimedia.org/r/c/operations/puppet/+/1328543 2477293 wikitext text/x-wiki kskryoeo6t84fbiym0mnnr7qhz635gy 2477294 2477293 2026-11-04T10:59:18Z Stashbot 7414 ladsgroup@deploy2003: Started scap sync-world: Backport for [[gerrit:1363009|Better browser support for DASH (T442988)]] (1.48.0-wmf.3) 2477294 wikitext text/x-wiki qbs1ubxbvn2u2tle3flecrqcv9hrqaz 2477295 2477294 2026-11-04T11:02:08Z Stashbot 7414 fabfur: enabling puppet again on A:cp to apply fix https://gerrit.wikimedia.org/r/c/operations/puppet/+/1363067 2477295 wikitext text/x-wiki b7z2j96hvquk9bto2iz47v3wh5v5ctu 2477296 2477295 2026-11-04T11:04:02Z Stashbot 7414 ladsgroup@deploy2003: ladsgroup: Backport for [[gerrit:1363009|Better browser support for DASH (T442988)]] (1.48.0-wmf.3) synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2477296 wikitext text/x-wiki ffdpbwctfqg7xmcmk29y6y2zhdb5o35 2477297 2477296 2026-11-04T11:06:13Z Stashbot 7414 ladsgroup@deploy2003: ladsgroup: Continuing with deployment 2477297 wikitext text/x-wiki p3qnmvx6xhy7wntfm2q8yfeq37jhtxu 2477298 2477297 2026-11-04T11:12:09Z Stashbot 7414 ladsgroup@deploy2003: Finished scap sync-world: Backport for [[gerrit:1363009|Better browser support for DASH (T442988)]] (1.48.0-wmf.3) (duration: 12m 51s) 2477298 wikitext text/x-wiki 4y7vti5u1grif3lbm5fjljr0fm2mtow 2477300 2477298 2026-11-04T11:14:48Z Stashbot 7414 claime: Starting poolcounter reboot sequence 2477300 wikitext text/x-wiki dijbsfgmsymb9iynfmn3bxx7tyoht52 2477301 2477300 2026-11-04T11:15:05Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reboot-single for host an-mariadb1001.eqiad.wmnet 2477301 wikitext text/x-wiki 7m551vx22h60yowyd9pqe3q8ya4wzg5 2477302 2477301 2026-11-04T11:15:47Z Stashbot 7414 atsuko@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply 2477302 wikitext text/x-wiki dm5r34rr042ai6a0z0y1e0dbfd0z8az 2477303 2477302 2026-11-04T11:16:11Z Stashbot 7414 cgoubert@deploy2003: Started scap sync-world: Backport for [[gerrit:1363062|ProductionServices: Depool poolcounter1006 for reboot (#1/5) (T441653)]] 2477303 wikitext text/x-wiki ppmw4h7o70grfe3972a3waal6d5ky8r 2477304 2477303 2026-11-04T11:17:02Z Stashbot 7414 atsuko@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply 2477304 wikitext text/x-wiki tbv91wrmx478y06koyszmx7y7jx4gqj 2477305 2477304 2026-11-04T11:18:31Z Stashbot 7414 cgoubert@deploy2003: cgoubert: Backport for [[gerrit:1363062|ProductionServices: Depool poolcounter1006 for reboot (#1/5) (T441653)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2477305 wikitext text/x-wiki f4cxqn6g2u2w00zcsm2bkhay35ggj0d 2477306 2477305 2026-11-04T11:19:02Z Stashbot 7414 cgoubert@deploy2003: cgoubert: Continuing with deployment 2477306 wikitext text/x-wiki 8e8j1mhytkobcanpzhclrgt4exm4i2t 2477309 2477306 2026-11-04T11:20:04Z Stashbot 7414 marostegui@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1262.eqiad.wmnet with reason: Reboot 2477309 wikitext text/x-wiki adlts6dfxirjbgfh5tq9n1aene9526v 2477310 2477309 2026-11-04T11:21:41Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host an-mariadb1001.eqiad.wmnet 2477310 wikitext text/x-wiki fao0eg3pl1nbvgc8jbwdk1u3cypfphc 2477311 2477310 2026-11-04T11:24:50Z Stashbot 7414 cgoubert@deploy2003: Finished scap sync-world: Backport for [[gerrit:1363062|ProductionServices: Depool poolcounter1006 for reboot (#1/5) (T441653)]] (duration: 08m 39s) 2477311 wikitext text/x-wiki oou679ic0xzg9mrob4rj2r0ejreupr8 2477312 2477311 2026-11-04T11:25:02Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.hosts.reboot-single for host poolcounter1006.eqiad.wmnet 2477312 wikitext text/x-wiki 8zfxldp3cp1a969v63wq36uvz4mgzh4 2477313 2477312 2026-11-04T11:28:40Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host poolcounter1006.eqiad.wmnet 2477313 wikitext text/x-wiki m6ghcbtz6mefx4voel3lhyf7fvnvvlz 2477314 2477313 2026-11-04T11:28:48Z Stashbot 7414 marostegui@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1262.eqiad.wmnet with reason: Reboot 2477314 wikitext text/x-wiki e8tqwbnxzffxs312v9g9r1r6r678p3g 2477316 2477314 2026-11-04T11:35:17Z Stashbot 7414 ladsgroup@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2175.codfw.wmnet with reason: Maintenance 2477316 wikitext text/x-wiki f6tbo6h3y1im0akqcuumdzstz2x0c9s 2477317 2477316 2026-11-04T11:35:21Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Depooling db2175 (T441633)', diff saved to https://phabricator.wikimedia.org/P97256 and previous config saved to /var/cache/conftool/dbconfig/20261104-113517-ladsgroup.json 2477317 wikitext text/x-wiki cs5irpb3tki42umwe5j79sbthf006vn 2477318 2477317 2026-11-04T11:35:39Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reboot-single for host an-mariadb1002.eqiad.wmnet 2477318 wikitext text/x-wiki 7fog52wkjc809olor7bsfrlvcb5t7i5 2477320 2477318 2026-11-04T11:37:42Z Stashbot 7414 cgoubert@deploy2003: Started scap sync-world: Backport for [[gerrit:1363102|ProductionServices: Depool poolcounter1006 for reboot (#1/5) (T441653)]], [[gerrit:1363103|ProductionServices: Repool poolcounter1006, depool poolcounter1007 (#2/5) (T441653)]] 2477320 wikitext text/x-wiki 88wy7p3p8juxli3kmkf9efq1nl4w2bc 2477321 2477320 2026-11-04T11:41:53Z Stashbot 7414 cgoubert@deploy2003: cgoubert: Backport for [[gerrit:1363102|ProductionServices: Depool poolcounter1006 for reboot (#1/5) (T441653)]], [[gerrit:1363103|ProductionServices: Repool poolcounter1006, depool poolcounter1007 (#2/5) (T441653)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2477321 wikitext text/x-wiki ekb6v8vzz7jjobbnumyj1qxknbtcyew 2477322 2477321 2026-11-04T11:42:05Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host an-mariadb1002.eqiad.wmnet 2477322 wikitext text/x-wiki gbn9lbli3park9ef912a5713yhmh23k 2477323 2477322 2026-11-04T11:42:22Z Stashbot 7414 cgoubert@deploy2003: cgoubert: Continuing with deployment 2477323 wikitext text/x-wiki 9fmuxj1ubkslmvz1mwtklz19gzlzao0 2477324 2477323 2026-11-04T11:42:53Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db2175 (T441633)', diff saved to https://phabricator.wikimedia.org/P97257 and previous config saved to /var/cache/conftool/dbconfig/20261104-114251-ladsgroup.json 2477324 wikitext text/x-wiki 7ubm1uv9p0eye2hayzq4iyghw9fp091 2477325 2477324 2026-11-04T11:44:31Z Stashbot 7414 tappof: putting metamonitoring checks into "maintenance mode without sending a notification" to update the metamonitoring script T443406 2477325 wikitext text/x-wiki ibbtc447zdmiq9yfhea6auiayxzjjar 2477326 2477325 2026-11-04T11:48:06Z Stashbot 7414 cgoubert@deploy2003: Finished scap sync-world: Backport for [[gerrit:1363102|ProductionServices: Depool poolcounter1006 for reboot (#1/5) (T441653)]], [[gerrit:1363103|ProductionServices: Repool poolcounter1006, depool poolcounter1007 (#2/5) (T441653)]] (duration: 10m 24s) 2477326 wikitext text/x-wiki gcykwkagn32xpucoh25qb1v2xdcxy6q 2477327 2477326 2026-11-04T11:49:15Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.hosts.reboot-single for host poolcounter1007.eqiad.wmnet 2477327 wikitext text/x-wiki 7wkev6qv7ovj5l3j71qljhcsqrwek81 2477328 2477327 2026-11-04T11:52:20Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.reboot-nodes rolling reboot on A:dse-k8s-worker-codfw 2477328 wikitext text/x-wiki 93jqogviyl05qmy7zxlzx7pg0rbahnp 2477329 2477328 2026-11-04T11:52:23Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-wdqs2001.codfw.wmnet 2477329 wikitext text/x-wiki a23f567y3khynto72vqeh8s9l2028n0 2477330 2477329 2026-11-04T11:52:54Z Stashbot 7414 ladsgroup@cumin1004: dbctl commit (dc=all): 'Repooling after maintenance db2175', diff saved to https://phabricator.wikimedia.org/P97258 and previous config saved to /var/cache/conftool/dbconfig/20261104-115252-ladsgroup.json 2477330 wikitext text/x-wiki slhb660qva00gpcakasnfzietaxptab 2477331 2477330 2026-11-04T11:52:59Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-wdqs2001.codfw.wmnet 2477331 wikitext text/x-wiki aodoa8ts6jdqyhrsvvksq6llasqkv7s 2477332 2477331 2026-11-04T11:53:15Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host poolcounter1007.eqiad.wmnet 2477332 wikitext text/x-wiki 6aai8pfxcflqatwl2f7cserrrrt3nx0 2477333 2477332 2026-11-04T11:54:47Z Stashbot 7414 cgoubert@deploy2003: Started scap sync-world: Backport for [[gerrit:1363104|ProductionServices: Repool poolcounter1007, depool poolcounter2005 (#3/5) (T441653)]] 2477333 wikitext text/x-wiki nul0j00nnr4ioy77t2wqxk7aumc2irj 2477334 2477333 2026-11-04T11:55:03Z Stashbot 7414 blake@cumin1004: START - Cookbook sre.k8s.reboot-nodes rolling reboot on A:wikikube-staging-worker-codfw 2477334 wikitext text/x-wiki tbd7z7tebv27i126q2uay9vpo2yof2u 2477335 2477334 2026-11-04T11:55:06Z Stashbot 7414 blake@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host kubestage2001.codfw.wmnet 2477335 wikitext text/x-wiki 43b9z2u81ppl5f8dmae30kcbbbk40nm 2477336 2477335 2026-11-04T11:55:10Z Stashbot 7414 blake@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host kubestage2001.codfw.wmnet 2477336 wikitext text/x-wiki 2b2ua0a0s01mjqv0pxhhp5w0pxjr85s 2477337 2477336 2026-11-04T11:57:01Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-wdqs2001.codfw.wmnet 2477337 wikitext text/x-wiki jbrj6hgatwkoza0bqscki7xxjlkfavz 2477338 2477337 2026-11-04T11:57:04Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-wdqs2001.codfw.wmnet 2477338 wikitext text/x-wiki mp99q0de8xc035t8ixzq0yuomctzal0 2477339 2477338 2026-11-04T11:57:10Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-wdqs2002.codfw.wmnet 2477339 wikitext text/x-wiki rz9larpsl1clrjaiqdt2fmfb95oof7q 2477340 2477339 2026-11-04T11:57:45Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-wdqs2002.codfw.wmnet 2477340 wikitext text/x-wiki mnhkqn1ysmc25mys9iml4rif5uqcks1 0 17290 2476853 2476468 2026-11-03T16:20:33Z Stashbot 7414 bd808: Another try at deleting deployment-ms-be07 (T436476) 2476853 wikitext text/x-wiki 920bvyzxtt8mmwrmq43npyzw5fltmwo 2476854 2476853 2026-11-03T16:21:52Z Stashbot 7414 bd808: Another try at deleting deployment-ms-be08 (T436476) 2476854 wikitext text/x-wiki 9xy02fwr16ktji9ysz8jyi7lgy75mdx 2476864 2476854 2026-11-03T16:38:28Z Stashbot 7414 dduvall: pruning all images and restarting containerd/docker on integration-agent-docker-1095 to deploy T440670 2476864 wikitext text/x-wiki 9nm8gjeulgqk8akxl22ru73p4jnpokv 2476866 2476864 2026-11-03T16:39:25Z Stashbot 7414 dduvall: enabling integration-agent-docker-1095 jenkins agent 2476866 wikitext text/x-wiki dn0vcokkh3isvovg3v73s7n439jgpvi 2476960 2476866 2026-11-03T19:48:30Z Stashbot 7414 hashar: Updating Jenkins jobs for ci-bookworm update https://gerrit.wikimedia.org/r/c/integration/config/+/1362098 2476960 wikitext text/x-wiki d50kk1yquxowiilsgchd1pzvqowu88x 2477003 2476960 2026-11-03T22:20:05Z Stashbot 7414 Southparkfan: deployment-prep: Set profile::tlsproxy::envoy::cfssl_label: discovery2026 in project hiera - T436476, T424549 2477003 wikitext text/x-wiki 3ltjo5rasw4u2g815wumnb1btoggeos 2477014 2477003 2026-11-03T22:48:25Z Stashbot 7414 dancy: Reconfigured jobs for T439116 experiment: mediawiki-node26 quibble-vendor-mysql-php83 2477014 wikitext text/x-wiki miuyyhfjak91g0tz3thx5uinvh4o6xl 2477018 2477014 2026-11-03T23:00:09Z Stashbot 7414 dancy: Revert configuration of jobs for T439116 experiment: mediawiki-node26 quibble-vendor-mysql-php83 2477018 wikitext text/x-wiki lgboxz5ncbnvrybhtifu4xbl7rfn4xi 2477020 2477018 2026-11-03T23:07:49Z Stashbot 7414 Krinkle: Updating docker-pkg files on contint primary for https://gerrit.wikimedia.org/r/1362221 2477020 wikitext text/x-wiki jd7n8ir5arby12cq6nd2iu1dyctmbgo 2477061 2477020 2026-11-03T23:38:23Z Stashbot 7414 James_F: Docker: Provide civicrm-trixie, for T439483 2477061 wikitext text/x-wiki tvl0e0em48usgrj9y2rrrd9lfejbcx1 2477062 2477061 2026-11-03T23:42:15Z Stashbot 7414 James_F: Zuul: Add civicrm-trixie-based experimental jobs, for T439483 2477062 wikitext text/x-wiki ko4mhgma0nas20lg2tu3ezh1uh0plq1 2 18608 2476788 1877298 2026-11-03T13:41:35Z Yug 1705 2476788 wikitext text/x-wiki i76otsxoafcjc9cobbsvsag4qpmgq8e 498 30942 2476933 2476036 2026-11-03T18:38:29Z Stashbot 7414 volans: starting job files logs truncate on NFS as per https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolsNfsAlmostFull 2476933 wikitext text/x-wiki sd4tqtml819zb3s8ql4lu1g8v5qr2f7 2477227 2476933 2026-11-04T08:03:02Z Stashbot 7414 filippo@cloudcumin1001: START - Cookbook wmcs.openstack.roll_reboot_cloudnets 2477227 wikitext text/x-wiki 4fg6c4p2h1jo0doz26eitpgznwwczj9 2477231 2477227 2026-11-04T08:14:39Z Stashbot 7414 filippo@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.roll_reboot_cloudnets (exit_code=0) 2477231 wikitext text/x-wiki 6jytayu58rve0ouo4nn4ab1f6pkovco 2477233 2477231 2026-11-04T08:28:14Z Stashbot 7414 godog: roll-reboot cloudgws 2477233 wikitext text/x-wiki gkw6bd5v2rmi8rrfmzd99benegez850 2477275 2477233 2026-11-04T10:11:55Z Stashbot 7414 volans: truncating large uwsgi log files on Toolforge NFS as it is getting too full. Leaving the last 10k lines in each of them. 2477275 wikitext text/x-wiki bw0j4i5sppzbetje4s56dd8evv7aasf 498 383664 2477007 2473420 2026-11-03T22:34:08Z Stashbot 7414 addshore: T443531 setup possibly temporary route to reach the freenode logs see https://phabricator.wikimedia.org/P97177 2477007 wikitext text/x-wiki hqwoccv9ixjzk9drjcc77joxvozzqcq 498 443628 2476740 2476625 2026-11-03T12:08:38Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41632616293 (https://github.com/cluebotng/component-configs/commits/f601a120fa49f05bf11c44e48bfeac7b3047b828) 2476740 wikitext text/x-wiki j5ivelvpndovrl0sxoagq8g8pkphbbd 2476925 2476740 2026-11-03T18:32:07Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41693580770 (https://github.com/cluebotng/component-configs/commits/9ff9bae00f7241eef432f91586a0cc47e459c4e7) 2476925 wikitext text/x-wiki gclxfweavgs2o1dnorjgxg86p1471mr 2477110 2476925 2026-11-04T00:17:33Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41745119511 (https://github.com/cluebotng/component-configs/commits/1ec1f58d43747a66c7dced1860e116768b3953e6) 2477110 wikitext text/x-wiki nuz5z29q82ra51s68qu91njtqf6cog1 2477123 2477110 2026-11-04T00:41:35Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41748064641 (https://github.com/cluebotng/component-configs/commits/cff94477a84651d691f840e8ded909cba65ec7a2) 2477123 wikitext text/x-wiki gfgrgx1ih6583d5gpun7whimxgvq6xq 2477135 2477123 2026-11-04T01:41:30Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41755274447 (https://github.com/cluebotng/component-configs/commits/c68d0cf2fead4e93381cc05f1756241a916a24d6) 2477135 wikitext text/x-wiki ebvxm2ajf7qiufy3odperua8n2dolsp 2477163 2477135 2026-11-04T03:04:29Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41764208829 (https://github.com/cluebotng/component-configs/commits/85b5ce73ff9dbeca337eb18aa04b06b92e36e12f) 2477163 wikitext text/x-wiki 0duwot28nsks1b3uy71mqbdxnteqs2k 2477185 2477163 2026-11-04T04:31:08Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41773053080 (https://github.com/cluebotng/component-configs/commits/da4b9b8d6a0e7e83007541e61941093c6c9db0b3) 2477185 wikitext text/x-wiki 2c2s3apn4qaa85i3o2i9b12khg711j2 2477212 2477185 2026-11-04T06:36:55Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41786107372 (https://github.com/cluebotng/component-configs/commits/c8dca63c87878642bde2b1434d3dba8287dae06a) 2477212 wikitext text/x-wiki 771v4wgo0txwve2zfkv9qepjwaeerd8 0 446051 2476803 2450201 2026-11-03T15:05:15Z Ottomata 88 Added test kitchen decision 2476803 wikitext text/x-wiki jzjbtcw1roj8fvmxspywf8ud7kvp4gc 2476807 2476803 2026-11-03T15:07:03Z Ottomata 88 linked Test Kitchen note to detailed section 2476807 wikitext text/x-wiki 65boreuaacs83uwh0kuvmlh1ycn019h 2476811 2476807 2026-11-03T15:11:13Z Ottomata 88 /* Test Kitchen analytics or Event Data Product? */ 2476811 wikitext text/x-wiki cv31ypuwf13yitrmv010rtct59i3tit 498 446211 2476729 2476609 2026-11-03T12:05:07Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41632616290 (https://github.com/cluebotng/component-configs/commits/f601a120fa49f05bf11c44e48bfeac7b3047b828) 2476729 wikitext text/x-wiki 3gz2fv7v291n3f2wgcczj4yl3z35gi7 2477157 2476729 2026-11-04T03:00:22Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41764208889 (https://github.com/cluebotng/component-configs/commits/85b5ce73ff9dbeca337eb18aa04b06b92e36e12f) 2477157 wikitext text/x-wiki 2ofgorm7wenwk0l9ppu1lv2gcakxyvb 0 447343 2476757 2447582 2026-11-03T13:16:57Z ABran-WMF 37196 /* Emergency access */ 2476757 wikitext text/x-wiki 2ujyuh42yq5etmosalcndpqkn65fm8g 0 449160 2476889 2476637 2026-11-03T17:56:44Z Dexbot 30554 Bot: Updating the report 2476889 wikitext text/x-wiki njlaa5n9t592yp109xddhfpazyk6lbm 2477015 2476889 2026-11-03T22:48:55Z Dexbot 30554 Bot: Updating the report 2477015 wikitext text/x-wiki h23c89okcnrg1fx3ksdaawiyj4r3fiv 2477083 2477015 2026-11-04T00:01:13Z Dexbot 30554 Bot: Updating the report 2477083 wikitext text/x-wiki bgygvuqrouuh3420g37lyjehz633w7q 2477087 2477083 2026-11-04T00:07:15Z Dexbot 30554 Bot: Updating the report 2477087 wikitext text/x-wiki jqw2ghyrgn3nily1v021h4yeuihcrbc 2477319 2477087 2026-11-04T11:36:59Z Dexbot 30554 Bot: Updating the report 2477319 wikitext text/x-wiki 0sacng1yqv7cs08cvz04a2uez2qk0lo 0 451069 2476972 2038941 2026-11-03T20:22:17Z WFan (WMF) 38087 2476972 wikitext text/x-wiki ift5j4tzvw7h8emu3wja43pkbxv8e69 0 451075 2476970 2470930 2026-11-03T20:18:40Z DAdedoyin (WMF) 39150 Sprint U - 2026 Rotation 2476970 wikitext text/x-wiki qxjqvg2miotovjx047msqec996ubggc 0 451676 2476971 2044493 2026-11-03T20:21:38Z WFan (WMF) 38087 2476971 wikitext text/x-wiki g8bsray305enceiwqlpnfcx24oanuu8 498 452890 2476733 2476616 2026-11-03T12:05:16Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41632616142 (https://github.com/cluebotng/component-configs/commits/f601a120fa49f05bf11c44e48bfeac7b3047b828) 2476733 wikitext text/x-wiki axwsqxorya2newmjoojg8bummcbdpgx 2476926 2476733 2026-11-03T18:32:35Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41693690557 (https://github.com/cluebotng/component-configs/commits/2350c7e21a6306b68d1249b6635e7820e90f8d65) 2476926 wikitext text/x-wiki irmjx2p2hks14w8fglynd3r8u8ljqgs 2476932 2476926 2026-11-03T18:38:06Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41694203768 (https://github.com/cluebotng/component-configs/commits/50df9beed114de84489c4353abb75acc5cc118bd) 2476932 wikitext text/x-wiki 2yi7z36fokrx33epgxj2eipd5zc2liu 2477111 2476932 2026-11-04T00:18:02Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41745119500 (https://github.com/cluebotng/component-configs/commits/1ec1f58d43747a66c7dced1860e116768b3953e6) 2477111 wikitext text/x-wiki kwv7wpqssk83kmju86h67jrmp4h4a0m 2477136 2477111 2026-11-04T01:42:06Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41755274459 (https://github.com/cluebotng/component-configs/commits/c68d0cf2fead4e93381cc05f1756241a916a24d6) 2477136 wikitext text/x-wiki ljg4esacuo2cb7ouil378uxwnagw4b5 2477165 2477136 2026-11-04T03:04:56Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41764208843 (https://github.com/cluebotng/component-configs/commits/85b5ce73ff9dbeca337eb18aa04b06b92e36e12f) 2477165 wikitext text/x-wiki 3zrz0i6jpyzy7qav9r0rbrp5m5poa82 2477184 2477165 2026-11-04T04:29:29Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41773053027 (https://github.com/cluebotng/component-configs/commits/da4b9b8d6a0e7e83007541e61941093c6c9db0b3) 2477184 wikitext text/x-wiki pu2987ok7voxvm4s5djf1calsyuy9bo 2477196 2477184 2026-11-04T05:15:23Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41777591788 (https://github.com/cluebotng/component-configs/commits/e816c7dcfa9f55dfe3171cd83dcb396d239b99eb) 2477196 wikitext text/x-wiki 3a6zrupny6h3jez1g2tazpsb174j21d 0 453662 2476905 2452144 2026-11-03T18:18:06Z TChin (WMF) 25137 /* NodeJS service-runner */ Add thread arena info 2476905 wikitext text/x-wiki 2shuoo5y049kgzejszdubon5e1pkjm6 116 453906 2476977 2475971 2026-11-03T20:39:23Z Gitlabaccountapprovalbot 37332 @mfchris84 was approved. 2476977 wikitext text/x-wiki 9jn5isj1xwiwcnhi5dg24fa0fd2wtp3 0 454000 2476873 2436574 2026-11-03T17:00:27Z BCornwall-WMF 31521 /* acme-chief */ reword terrible pyyaml sentences 2476873 wikitext text/x-wiki bm9hhjlsequaw6j7ygndw5qpoqyxad5 0 458648 2477249 2460395 2026-11-04T09:24:21Z BRouberol-WMF 39141 /* Performing a Point In Time Recovery (PITR) */ 2477249 wikitext text/x-wiki ss6ju716kpngyf7qe0bm68kki3ti0fk 2477264 2477249 2026-11-04T09:56:40Z BRouberol-WMF 39141 /* Performing a Point In Time Recovery (PITR) */ 2477264 wikitext text/x-wiki o59p8elbdo1brdzffs6ib35fk26k0bd 0 458675 2477290 2468070 2026-11-04T10:33:06Z CMooney (WMF) 24288 /* Show route to a given destination IP */ 2477290 wikitext text/x-wiki 122wxlnnm0af4b3oddjtrr0wy9wpk7e 498 458787 2476734 2476624 2026-11-03T12:05:17Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41632616232 (https://github.com/cluebotng/component-configs/commits/f601a120fa49f05bf11c44e48bfeac7b3047b828) 2476734 wikitext text/x-wiki c7zm145mjoo4bx7gk2b3r4bj0jcsi73 2476923 2476734 2026-11-03T18:30:04Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41693580691 (https://github.com/cluebotng/component-configs/commits/9ff9bae00f7241eef432f91586a0cc47e459c4e7) 2476923 wikitext text/x-wiki 6rg4bykgs76phq8crbw5jhtv5ih0e7z 2477122 2476923 2026-11-04T00:40:27Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41748064637 (https://github.com/cluebotng/component-configs/commits/cff94477a84651d691f840e8ded909cba65ec7a2) 2477122 wikitext text/x-wiki 2lrnfjki0jqoyybujldq2qxy0anglgw 2477134 2477122 2026-11-04T01:40:43Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41755274531 (https://github.com/cluebotng/component-configs/commits/c68d0cf2fead4e93381cc05f1756241a916a24d6) 2477134 wikitext text/x-wiki bmtyfehr2woexvy9ixhyzpmtjrodrge 2477162 2477134 2026-11-04T03:03:18Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41764208797 (https://github.com/cluebotng/component-configs/commits/85b5ce73ff9dbeca337eb18aa04b06b92e36e12f) 2477162 wikitext text/x-wiki p9t27xqe600ywhbihnibew86vsbziz2 2477211 2477162 2026-11-04T06:36:12Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41786107416 (https://github.com/cluebotng/component-configs/commits/c8dca63c87878642bde2b1434d3dba8287dae06a) 2477211 wikitext text/x-wiki 6w8mzj8yzscmdf8h5z9suuc1ze4ueij 0 459044 2476994 2445066 2026-11-03T21:34:10Z BKing (WMF) 29017 2476994 wikitext text/x-wiki lfdj56jg4thfr9nh0vp253qoj17fmqm 498 459243 2476739 2476693 2026-11-03T12:08:25Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41632616200 (https://github.com/cluebotng/component-configs/commits/f601a120fa49f05bf11c44e48bfeac7b3047b828) 2476739 wikitext text/x-wiki bebtowwnzz9lhase7hzg1sj7qsekoov 2476886 2476739 2026-11-03T17:56:22Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41688162297 (https://github.com/cluebotng/component-configs/commits/7f183bdf8d9d62f3092978941b91e04a9a5531c6) 2476886 wikitext text/x-wiki rwbh483i7fjjfv9eqdsvkbttfwz1kcr 2477146 2476886 2026-11-04T02:25:27Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41760170800 (https://github.com/cluebotng/component-configs/commits/ee3a8262f4918cfcf2e22ccfe0d728ebf8143df2) 2477146 wikitext text/x-wiki o8xhb3na6sxj5gu6jaog6z9iibms8mq 2477148 2477146 2026-11-04T02:39:06Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41761692440 (https://github.com/cluebotng/component-configs/commits/3cdc869413ae6a77faaac98309e3b59dab578a53) 2477148 wikitext text/x-wiki 3ohzn32subg82kwjf1jmkoqh6xkpfry 2477150 2477148 2026-11-04T02:44:10Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41761998436 (https://github.com/cluebotng/component-configs/commits/c16528be19dec3e78a0c5b289774e8fd56d8ab32) 2477150 wikitext text/x-wiki 9jwgvogeowc57mxl0eq5vkg6edj3e5d 2477154 2477150 2026-11-04T02:54:47Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41763262663 (https://github.com/cluebotng/component-configs/commits/d15c2921f9532c4723d817aa328532a85783a46c) 2477154 wikitext text/x-wiki 8fveu31q55njktk7w51k9pt91k7a642 2477155 2477154 2026-11-04T02:59:26Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41763724637 (https://github.com/cluebotng/component-configs/commits/1681819283fff40cec7eedcf2e2ddd6b6e555828) 2477155 wikitext text/x-wiki 4ok1j83tcx59ebvhw87nbl0jik6zclt 2477164 2477155 2026-11-04T03:04:40Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41764208847 (https://github.com/cluebotng/component-configs/commits/85b5ce73ff9dbeca337eb18aa04b06b92e36e12f) 2477164 wikitext text/x-wiki fnz7ovrm2uq8n329ykzmj46idad1x37 498 459244 2476732 2476607 2026-11-03T12:05:14Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41632616245 (https://github.com/cluebotng/component-configs/commits/f601a120fa49f05bf11c44e48bfeac7b3047b828) 2476732 wikitext text/x-wiki kotpnsb5pw25mvsrk9duxi08oapklbt 2476899 2476732 2026-11-03T18:15:17Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41691194741 (https://github.com/cluebotng/component-configs/commits/7c77c0270dd527e186687a6d6eb0aff11d0ad80c) 2476899 wikitext text/x-wiki j5xgg9y22obqmq9gs57dkg28e7epsp4 2476908 2476899 2026-11-03T18:19:57Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41691894047 (https://github.com/cluebotng/component-configs/commits/4d3b69effc2827247ca575cf060dce20483d3103) 2476908 wikitext text/x-wiki ssauco673is4wffdv5f281doczn0v77 2476913 2476908 2026-11-03T18:27:15Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41693130037 (https://github.com/cluebotng/component-configs/commits/0d19841535066d42c60bc747e02829434852f914) 2476913 wikitext text/x-wiki qau28gvlvz51c4tz6tbqr8mswvc28pw 2477109 2476913 2026-11-04T00:16:47Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41745119449 (https://github.com/cluebotng/component-configs/commits/1ec1f58d43747a66c7dced1860e116768b3953e6) 2477109 wikitext text/x-wiki qchxa49y5nq7j1g1nozfw6lhs26v177 2477161 2477109 2026-11-04T03:03:11Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41764161174 (https://github.com/cluebotng/component-configs/commits/efdda3a013777f4e684be77dc6e05704036d4f0a) 2477161 wikitext text/x-wiki bstf9dejychog77p4y6ezo8aplav6ip 2477166 2477161 2026-11-04T03:08:15Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41764208826 (https://github.com/cluebotng/component-configs/commits/85b5ce73ff9dbeca337eb18aa04b06b92e36e12f) 2477166 wikitext text/x-wiki mkdow83i8k67b34uujnvydc1ojnqb5b 498 459255 2476731 2476610 2026-11-03T12:05:11Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41632616324 (https://github.com/cluebotng/component-configs/commits/f601a120fa49f05bf11c44e48bfeac7b3047b828) 2476731 wikitext text/x-wiki gaa4y244c2xfkwcex5aznw7yphj6ep9 2476898 2476731 2026-11-03T18:14:48Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41691194793 (https://github.com/cluebotng/component-configs/commits/7c77c0270dd527e186687a6d6eb0aff11d0ad80c) 2476898 wikitext text/x-wiki mytkutzbip4nk3qsc1uytrqao44q7uq 2476907 2476898 2026-11-03T18:19:24Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41691894105 (https://github.com/cluebotng/component-configs/commits/4d3b69effc2827247ca575cf060dce20483d3103) 2476907 wikitext text/x-wiki q1anlj6r4pxcbnu02jwn6pv046hcn4f 2476912 2476907 2026-11-03T18:26:52Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41693129959 (https://github.com/cluebotng/component-configs/commits/0d19841535066d42c60bc747e02829434852f914) 2476912 wikitext text/x-wiki 2ulhlcfw1p8qisgx0dzh61amsc414op 2477160 2476912 2026-11-04T03:02:40Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41764208884 (https://github.com/cluebotng/component-configs/commits/85b5ce73ff9dbeca337eb18aa04b06b92e36e12f) 2477160 wikitext text/x-wiki 9x42pnasqkv1ghvbxflymzywv0abpfk 498 459263 2476730 2476606 2026-11-03T12:05:08Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41632616265 (https://github.com/cluebotng/component-configs/commits/f601a120fa49f05bf11c44e48bfeac7b3047b828) 2476730 wikitext text/x-wiki 7a2bp6p107tok3yzcsyuw4g47a3fivh 2477156 2476730 2026-11-04T03:00:14Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41764208830 (https://github.com/cluebotng/component-configs/commits/85b5ce73ff9dbeca337eb18aa04b06b92e36e12f) 2477156 wikitext text/x-wiki d7sp6uusd9huzajxm1f8fn7gx3thk1i 0 459290 2476801 2474317 2026-11-03T15:00:29Z KCVelaga 26729 for some reason the deployment didn't happen, moving up 2476801 wikitext text/x-wiki 6hr4c03hqeyb965wrpugon9hxfmtbrq 498 459362 2476737 2476608 2026-11-03T12:07:18Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41632616438 (https://github.com/cluebotng/component-configs/commits/f601a120fa49f05bf11c44e48bfeac7b3047b828) 2476737 wikitext text/x-wiki saehvfeze75e1splzromiu40s07ydri 2477159 2476737 2026-11-04T03:02:35Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41764208828 (https://github.com/cluebotng/component-configs/commits/85b5ce73ff9dbeca337eb18aa04b06b92e36e12f) 2477159 wikitext text/x-wiki m7cq40380z8cl40q9klxbo7orv6b9wr 0 459699 2476969 2476591 2026-11-03T20:18:00Z Stashbot 7414 eileen: civicrm upgraded from 6212f766 to 6fc4ee60 2476969 wikitext text/x-wiki p2q2t59aswp25oq2mbopxxxndnsknr1 2477179 2476969 2026-11-04T04:08:11Z Stashbot 7414 cstone: payments-wiki upgraded from 3d53577c to e801b49c 2477179 wikitext text/x-wiki h1dzpo2fyjx4yyqwqz473fmo6s4zrl4 2477183 2477179 2026-11-04T04:29:11Z Stashbot 7414 eileen: civicrm upgraded from 6fc4ee60 to e09e0b2b 2477183 wikitext text/x-wiki 4r0p4wcknyapjnhz4lxq6w4gqt5u60g 0 460559 2476867 2476342 2026-11-03T16:40:12Z AStein-WMF 39485 2476867 wikitext text/x-wiki lcd8tqsuux1o0wv4abz5is7hb7r2o4n 0 460616 2476814 2453701 2026-11-03T15:12:30Z GLavagetto (WMF) 2905 /* What you should do when asking for a code review */ Clarified the ways a ping can happen; also added a short notice on LLM usage. 2476814 wikitext text/x-wiki gcih6wm0rcxnxwo07rwjn5gff7vdfnv 0 460627 2477177 2476491 2026-11-04T04:01:50Z AKhatun (WMF) 50461 Doc update: timeout and new alerts 2477177 wikitext text/x-wiki 8fwdwjpqhig6aozxnrfyvdcmj4r7552 2477181 2477177 2026-11-04T04:17:29Z AKhatun (WMF) 50461 2477181 wikitext text/x-wiki hsb6buadht92qeoabfr2nthp2bw2smw 10 460755 2477000 2462776 2026-11-03T21:48:40Z Protolyte 55939 Replaced empty heading with "----" to resolve lint error 2477000 wikitext text/x-wiki g1pjd7upwbiwszqmafrx0y4565zzj9e 116 460821 2476991 2469844 2026-11-03T21:26:30Z Protolyte 55939 Removed redundant image size parameter since it's already included in the template 2476991 wikitext text/x-wiki fuq4qrekekwzccchxy1w058oibc5fjc 0 460894 2476880 2475883 2026-11-03T17:26:38Z FNegri-WMF 32595 update list and add command used to generate it 2476880 wikitext text/x-wiki ssl5j8wt1yoltll24ogr64fq9avrqcp 2476881 2476880 2026-11-03T17:27:40Z FNegri-WMF 32595 Improve formatting 2476881 wikitext text/x-wiki lq7q8dvoav8vosyvopw1ukb70yrxa8m 2476882 2476881 2026-11-03T17:40:10Z FNegri-WMF 32595 Regenerated the list using a more comprehensive command that also checks deployments and cronjobs 2476882 wikitext text/x-wiki 9v7736m9rzlmik2j5mo5om4q0q4n9d8 2476884 2476882 2026-11-03T17:48:18Z FNegri-WMF 32595 Fix links and generate them in the command 2476884 wikitext text/x-wiki cnek0s6zv301p1lp7gtm7iferpmz1v2 2476896 2476884 2026-11-03T18:14:22Z FNegri-WMF 32595 Update list, using a modified command that excludes leftovers (finished pods, completed or failed jobs) 2476896 wikitext text/x-wiki jpzg3u6i0xe57nv1pklzd16si2fqzf8 1 460921 2476902 2475994 2026-11-03T18:17:33Z FNegri-WMF 32595 /* Update */ Reply 2476902 wikitext text/x-wiki kk1jaiwx3hlt9uwv4re7yw5z0a9sadr 2 460928 2476852 2476581 2026-11-03T16:19:53Z BCornwall-WMF 31521 2476852 wikitext text/x-wiki f6c2p04qmyaq6jm2nul5578q7srq67a 3 460931 2476795 2026-11-03T13:57:13Z StrikerBot 8475 Welcome to Toolforge! 2476795 wikitext text/x-wiki k29dxtxjqoafe7k4mdnla2g3v89a0f6 498 460932 2476871 2026-11-03T16:59:06Z Labslogbot 55 Auto update of instance info. 2476871 wikitext text/x-wiki 51vzokjmiohdcat0zezq77vfj3nlsdn