Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.46.0-wmf.26 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 3086 2409239 2406777 2026-05-05T01:35:23Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2409239 wikitext text/x-wiki 85le27cjkp5n30ss7jxfeovh6pfkqqb 2409240 2409239 2026-05-05T01:45:34Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-api 2409240 wikitext text/x-wiki j1zuopbwwzn7p5qy080ait5zoewa836 2409249 2409240 2026-05-05T02:24:12Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2409249 wikitext text/x-wiki kcjw5ikgkzyr78opqlzxcdjqetxhj0d 2409250 2409249 2026-05-05T02:29:14Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api 2409250 wikitext text/x-wiki gki5soeyndxr1o6bxyg85gb0snz6egu 2409251 2409250 2026-05-05T02:35:59Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer 2409251 wikitext text/x-wiki mmhgg9tj0wkduiswhxnan7onstl29s4 2409252 2409251 2026-05-05T02:54:13Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer 2409252 wikitext text/x-wiki iosmcmmy5nmm3p84s1btw1siqij3bav 2409255 2409252 2026-05-05T03:11:03Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2409255 wikitext text/x-wiki pxql107liwn9c02ivfx12iglj080vuc 2409256 2409255 2026-05-05T03:23:15Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2409256 wikitext text/x-wiki abl8kq4om55inszu7qp33g0w7vricpv 2409259 2409256 2026-05-05T03:50:17Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2409259 wikitext text/x-wiki k9jl9cociqjg16unw28rfp44olij0g7 2409260 2409259 2026-05-05T04:03:15Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2409260 wikitext text/x-wiki bmwb9adfsde6q9khu8l24l9711iafkb 2409264 2409260 2026-05-05T04:35:30Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2409264 wikitext text/x-wiki kaondxvu2hc5ax0aay51d2dkq0b9vxo 2409265 2409264 2026-05-05T04:49:18Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2409265 wikitext text/x-wiki plgdxrwhpuxubxyiywfm01pbdlyz2q9 0 4108 2409053 2408876 2026-05-04T13:51:55Z ScheduleDeploymentBot 37566 Add [[gerrit:1281967]] to Tuesday, May 05 UTC morning backport window 2409053 wikitext text/x-wiki rq8lhfbanm5gu5i58am5onejlej0fk4 2409118 2409053 2026-05-04T15:13:26Z ScheduleDeploymentBot 37566 Add [[gerrit:1281501]] to Monday, May 04 UTC late backport window 2409118 wikitext text/x-wiki fbqei1haujtu44bbcfbyq0u1x4zdbsu 2409125 2409118 2026-05-04T15:21:09Z ScheduleDeploymentBot 37566 Add [[gerrit:1282385]] to Monday, May 04 UTC late backport window 2409125 wikitext text/x-wiki 5806x8gqjp72nmlf5rx0wq2flbh1bho 2409168 2409125 2026-05-04T18:25:22Z ScheduleDeploymentBot 37566 Add [[gerrit:1276432]] to Monday, May 04 UTC late backport window 2409168 wikitext text/x-wiki ehita8qehyal4yzm1av7qyb9vw4axgr 2409277 2409168 2026-05-05T06:01:31Z ScheduleDeploymentBot 37566 Add [[gerrit:1282723]] to Tuesday, May 05 UTC morning backport window 2409277 wikitext text/x-wiki fyfyoi9u9nycweyn7u86qvdiyok08hs 2409314 2409277 2026-05-05T07:56:19Z ScheduleDeploymentBot 37566 Add [[gerrit:1282850]] to Tuesday, May 05 UTC afternoon backport window 2409314 wikitext text/x-wiki ozkqb6gt9npac2a9h2b79xw5jib6jal 2409329 2409314 2026-05-05T08:11:49Z Adam Wight (WMDE) 3064 mark deployments done 2409329 wikitext text/x-wiki tuwspbg84xk1cfg70gu23c7rgrq344j 2409411 2409329 2026-05-05T09:40:59Z ScheduleDeploymentBot 37566 Add [[gerrit:1281964]] to Tuesday, May 05 UTC afternoon backport window 2409411 wikitext text/x-wiki sd4s69r4nyyb113uq3vybkmyubjrdt2 2409414 2409411 2026-05-05T09:49:33Z ScheduleDeploymentBot 37566 Add [[gerrit:1282931]] to Tuesday, May 05 UTC afternoon backport window 2409414 wikitext text/x-wiki 3twe4uheobzj5ad15oefmp1fygiusxb 2 6404 2409196 2100382 2026-05-04T20:00:55Z Koavf 1382 {{DISPLAYTITLE:User:<span style="color:grey">''ko'''a'''</span><span style="color:black">v</span><span style="color:grey">f''</span>}} 2409196 wikitext text/x-wiki 6enrtxte1ff93t6vsf5wh45z9iqmq2a 498 6726 2409237 2407040 2026-05-05T01:25:53Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2409237 wikitext text/x-wiki cgvwttts88y6dckxmwt8r7gox05nu04 2409238 2409237 2026-05-05T01:35:13Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-api 2409238 wikitext text/x-wiki tkg9dvp67njsz219qgo9pj1b6m83rbi 2409241 2409238 2026-05-05T01:48:47Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer 2409241 wikitext text/x-wiki md6r2isop014nc1h2i7up1pmw9t70vj 2409242 2409241 2026-05-05T01:49:13Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component jobs-emailer 2409242 wikitext text/x-wiki bqg103d53yje3vvj902xhtokj47cfui 2409243 2409242 2026-05-05T01:49:34Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer 2409243 wikitext text/x-wiki 3r31nth4u48mkhk11yn8b2jmdfpm0vc 2409245 2409243 2026-05-05T02:04:00Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer 2409245 wikitext text/x-wiki 42q6bnsqwz9l9oked7oecw37mymef4h 2409247 2409245 2026-05-05T02:09:43Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2409247 wikitext text/x-wiki 3x72h0662g731xiygoincyiielhaoao 2409248 2409247 2026-05-05T02:14:13Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api 2409248 wikitext text/x-wiki l7tbrv3hu2fq7q1o8hsba04fhzdt4qz 2409253 2409248 2026-05-05T02:57:16Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2409253 wikitext text/x-wiki k2q8ti87ymef6xavsmfkg5pjx3fc140 2409254 2409253 2026-05-05T03:09:13Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2409254 wikitext text/x-wiki kfy8o906xz9663bdwqcb9b95gle6ju4 2409257 2409254 2026-05-05T03:36:44Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2409257 wikitext text/x-wiki o67dd3bn97moe53zitkrsd6xrfcgsxy 2409258 2409257 2026-05-05T03:48:14Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2409258 wikitext text/x-wiki f8nqpdwbhbzg4e7svya4c8l5w9g79dc 2409262 2409258 2026-05-05T04:20:41Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2409262 wikitext text/x-wiki hq6so502fy0b8ap45g7zb6m5fh3peq9 2409263 2409262 2026-05-05T04:31:14Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2409263 wikitext text/x-wiki oln0jh0tjldl62aw0uoycg19lah9czt 0 7919 2409015 2409014 2026-05-04T12:03:34Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5004.eqsin.wmnet with OS bookworm 2409015 wikitext text/x-wiki b4fkobd647qjp749pu7tf904lenlumr 2409016 2409015 2026-05-04T12:04:17Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P92198 and previous config saved to /var/cache/conftool/dbconfig/20260504-120416-fceratto.json 2409016 wikitext text/x-wiki 7psxp6p1qp8iz2wxgvqhy3nx6i9dgxr 2409017 2409016 2026-05-04T12:14:25Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 (T419961)', diff saved to https://phabricator.wikimedia.org/P92199 and previous config saved to /var/cache/conftool/dbconfig/20260504-121424-fceratto.json 2409017 wikitext text/x-wiki 0spk9e7ik6vzf35qcx5vubxjx6668wj 2409018 2409017 2026-05-04T12:14:34Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1203.eqiad.wmnet with reason: Maintenance 2409018 wikitext text/x-wiki iam99k0zk2ilsny7k96qqi6rhdyicwx 2409019 2409018 2026-05-04T12:14:43Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1203 (T419961)', diff saved to https://phabricator.wikimedia.org/P92200 and previous config saved to /var/cache/conftool/dbconfig/20260504-121441-fceratto.json 2409019 wikitext text/x-wiki 1l3xmbavha9mu5smjbxy04no6lqfmf5 2409020 2409019 2026-05-04T12:21:56Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1203 (T419961)', diff saved to https://phabricator.wikimedia.org/P92201 and previous config saved to /var/cache/conftool/dbconfig/20260504-122155-fceratto.json 2409020 wikitext text/x-wiki ob2jvlenba2by3f70lqujoashi0121v 2409021 2409020 2026-05-04T12:32:04Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1203', diff saved to https://phabricator.wikimedia.org/P92202 and previous config saved to /var/cache/conftool/dbconfig/20260504-123203-fceratto.json 2409021 wikitext text/x-wiki bc970fnwusmr1kztvxf16frntpdfkb5 2409022 2409021 2026-05-04T12:42:11Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1203', diff saved to https://phabricator.wikimedia.org/P92203 and previous config saved to /var/cache/conftool/dbconfig/20260504-124210-fceratto.json 2409022 wikitext text/x-wiki 0sszct3ayls1smlzrqcl7pl5si07akh 2409023 2409022 2026-05-04T12:45:39Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5004.eqsin.wmnet with reason: host reimage 2409023 wikitext text/x-wiki r142cr1q1ma0oqz5b4zepbgwn676fdk 2409024 2409023 2026-05-04T12:50:15Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on durum5004.eqsin.wmnet with reason: host reimage 2409024 wikitext text/x-wiki 2nfx93lxxn502koaweihhnx99tmbrte 2409025 2409024 2026-05-04T12:51:08Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2213.codfw.wmnet with reason: Maintenance 2409025 wikitext text/x-wiki nnuqnc51schn87gb9ctfem6ma2a7ocx 2409026 2409025 2026-05-04T12:52:20Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1203 (T419961)', diff saved to https://phabricator.wikimedia.org/P92204 and previous config saved to /var/cache/conftool/dbconfig/20260504-125219-fceratto.json 2409026 wikitext text/x-wiki igi00gaqqhxbtw07ht1rrof7ay6e5l6 2409027 2409026 2026-05-04T12:52:40Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1214.eqiad.wmnet with reason: Maintenance 2409027 wikitext text/x-wiki c2m2ctufgbgi6qv9lm3am7kkg5x7yqf 2409028 2409027 2026-05-04T12:52:49Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1214 (T419961)', diff saved to https://phabricator.wikimedia.org/P92205 and previous config saved to /var/cache/conftool/dbconfig/20260504-125247-fceratto.json 2409028 wikitext text/x-wiki 0bwi7w5gypo708ksvzw1841d74yqbrp 2409029 2409028 2026-05-04T12:59:22Z Stashbot 7414 dcausse: T425301: resuming writes on cloudelastic 2409029 wikitext text/x-wiki m0mf5cnjfoop04jto7lp2jln6kl3w42 2409030 2409029 2026-05-04T12:59:24Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply 2409030 wikitext text/x-wiki e5q4d5bavziee9udatqcrjkuhraoxay 2409031 2409030 2026-05-04T12:59:29Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply 2409031 wikitext text/x-wiki ouohyu9hy3f4u7fbzpvloxjwmy0pf4r 2409032 2409031 2026-05-04T12:59:47Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1214 (T419961)', diff saved to https://phabricator.wikimedia.org/P92206 and previous config saved to /var/cache/conftool/dbconfig/20260504-125945-fceratto.json 2409032 wikitext text/x-wiki nrfgyrb9nccnkniwdnczj32ybyq7k5c 2409033 2409032 2026-05-04T13:09:55Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1214', diff saved to https://phabricator.wikimedia.org/P92207 and previous config saved to /var/cache/conftool/dbconfig/20260504-130953-fceratto.json 2409033 wikitext text/x-wiki t4m13fzf2hzdagdzm0rptt2ujjhi1bd 2409034 2409033 2026-05-04T13:10:09Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5004.eqsin.wmnet with OS bookworm 2409034 wikitext text/x-wiki s1wkf1znrdmjdppyyhpq2hga4ut9clg 2409035 2409034 2026-05-04T13:10:10Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum5004.eqsin.wmnet 2409035 wikitext text/x-wiki i3u1jw8p7pjxf7e7cmsr25r5q6ac7u7 2409037 2409035 2026-05-04T13:13:00Z Stashbot 7414 moritzm: installing jaraco.context security updates 2409037 wikitext text/x-wiki k5wkaegxe1fhfwvy18hv12wuldqy7gr 2409039 2409037 2026-05-04T13:20:03Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1214', diff saved to https://phabricator.wikimedia.org/P92208 and previous config saved to /var/cache/conftool/dbconfig/20260504-132002-fceratto.json 2409039 wikitext text/x-wiki pjuiizfahlkdc5mptee9thfdou34o01 2409040 2409039 2026-05-04T13:23:25Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host kafka-logging1007.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409040 wikitext text/x-wiki 3a2b6az8ooq491cbiscgzqukmo8ntsl 2409041 2409040 2026-05-04T13:23:59Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host kafka-logging1007.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409041 wikitext text/x-wiki 26h8trmm2btgnfnqbp7evcy61jip8q5 2409042 2409041 2026-05-04T13:29:35Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host kafka-logging1007.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409042 wikitext text/x-wiki pcu35p6jfgpr56dq2sfzizkgy8horo1 2409043 2409042 2026-05-04T13:30:11Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1214 (T419961)', diff saved to https://phabricator.wikimedia.org/P92209 and previous config saved to /var/cache/conftool/dbconfig/20260504-133010-fceratto.json 2409043 wikitext text/x-wiki 5si42g4xj3dzp5uj9jd0es14bxkkaco 2409044 2409043 2026-05-04T13:30:32Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1226.eqiad.wmnet with reason: Maintenance 2409044 wikitext text/x-wiki eucx63kb5qe5yd6u2fwmbo687o7ijyc 2409045 2409044 2026-05-04T13:30:40Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1226 (T419961)', diff saved to https://phabricator.wikimedia.org/P92210 and previous config saved to /var/cache/conftool/dbconfig/20260504-133039-fceratto.json 2409045 wikitext text/x-wiki e3q0mgcfkmdj4uzf8s2mr76p1u2iemp 2409046 2409045 2026-05-04T13:40:16Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host kafka-logging1007.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409046 wikitext text/x-wiki 0a0wq91umfscgq01ubljk5outk10sjt 2409047 2409046 2026-05-04T13:40:49Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1226 (T419961)', diff saved to https://phabricator.wikimedia.org/P92211 and previous config saved to /var/cache/conftool/dbconfig/20260504-134048-fceratto.json 2409047 wikitext text/x-wiki g4y7x1oqvkmsn2sq7cd4qzuc45z2nf0 2409048 2409047 2026-05-04T13:43:27Z Stashbot 7414 sbisson@deploy1003: Started scap sync-world: Backport for [[gerrit:1281965|zhwikinews: (1/2) revert 20th anniversary logo change (config) (T420165)]] 2409048 wikitext text/x-wiki f9huakpe0buqcjy5blrydm48st4g96m 2409049 2409048 2026-05-04T13:45:09Z Stashbot 7414 sbisson@deploy1003: 1f616emo, sbisson: Backport for [[gerrit:1281965|zhwikinews: (1/2) revert 20th anniversary logo change (config) (T420165)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409049 wikitext text/x-wiki 4twvrbbo3y0pa788364wunaao0oanms 2409050 2409049 2026-05-04T13:46:38Z Stashbot 7414 sbisson@deploy1003: 1f616emo, sbisson: Continuing with deployment 2409050 wikitext text/x-wiki lw89nqrfh5ahn9a3qmjfc24jo6rib8f 2409051 2409050 2026-05-04T13:50:57Z Stashbot 7414 sbisson@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281965|zhwikinews: (1/2) revert 20th anniversary logo change (config) (T420165)]] (duration: 07m 30s) 2409051 wikitext text/x-wiki qy3ax2dtfom2vwdb9pbss4knx1msuf4 2409052 2409051 2026-05-04T13:50:59Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1226', diff saved to https://phabricator.wikimedia.org/P92212 and previous config saved to /var/cache/conftool/dbconfig/20260504-135056-fceratto.json 2409052 wikitext text/x-wiki 3unvbuexm0zm7vqs3vxs86j4w9gunb5 2409054 2409052 2026-05-04T13:52:21Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host install5004.wikimedia.org 2409054 wikitext text/x-wiki lnfr07u6m0dd4hes84kx4lqyqylzksp 2409055 2409054 2026-05-04T13:52:23Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2409055 wikitext text/x-wiki axzw4ztsdn52aqyfoqhdhuzm3yh4wte 2409056 2409055 2026-05-04T13:53:20Z Stashbot 7414 sbisson@deploy1003: Started scap sync-world: Backport for [[gerrit:1282354|ArticleGuidance: enable on simple english (T425351)]] 2409056 wikitext text/x-wiki g4ssepqii869dq5q00ud7qy7qfejwya 2409057 2409056 2026-05-04T13:54:06Z Stashbot 7414 dcausse: T425301: stopping writes again on cloudelastic, cluster unstable 2409057 wikitext text/x-wiki troqbm33p2z7mshns52hh6fwy1w1ino 2409058 2409057 2026-05-04T13:55:02Z Stashbot 7414 sbisson@deploy1003: sbisson: Backport for [[gerrit:1282354|ArticleGuidance: enable on simple english (T425351)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409058 wikitext text/x-wiki phb2j8iqfps5oq60uju3dtsetlx5h3k 2409059 2409058 2026-05-04T13:55:34Z Stashbot 7414 sbisson@deploy1003: sbisson: Continuing with deployment 2409059 wikitext text/x-wiki n4mpmk6ihria9gbigi2f4b1vbp3yz3t 2409060 2409059 2026-05-04T13:55:47Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply 2409060 wikitext text/x-wiki squdca04qisv7htdkj6up2yjhn9fyay 2409061 2409060 2026-05-04T13:55:59Z Stashbot 7414 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply 2409061 wikitext text/x-wiki db9kc8fw2hgb5bkk4nibs7qzu486ky5 2409062 2409061 2026-05-04T13:56:07Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM install5004.wikimedia.org - jmm@cumin2002" 2409062 wikitext text/x-wiki hzeu6ln71n72tedkr84oboqj4m58ilo 2409063 2409062 2026-05-04T13:56:44Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM install5004.wikimedia.org - jmm@cumin2002" 2409063 wikitext text/x-wiki tssv6naouwzdk6rbkubjxdqhe8wxai2 2409064 2409063 2026-05-04T13:56:45Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2409064 wikitext text/x-wiki prrwny3vz1go3p9q6f8etz9zla07qdl 2409065 2409064 2026-05-04T13:56:46Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache install5004.wikimedia.org on all recursors 2409065 wikitext text/x-wiki r70ri6n7vr84lik2t8qtl13n0lpda0c 2409066 2409065 2026-05-04T13:56:48Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) install5004.wikimedia.org on all recursors 2409066 wikitext text/x-wiki bxqtat77nyurdk8283piae4xxs0k5ue 2409067 2409066 2026-05-04T13:57:06Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2409067 wikitext text/x-wiki 532waj1hfw7szympwpyk1l9taebh38b 2409068 2409067 2026-05-04T13:59:42Z Stashbot 7414 sbisson@deploy1003: Finished scap sync-world: Backport for [[gerrit:1282354|ArticleGuidance: enable on simple english (T425351)]] (duration: 06m 22s) 2409068 wikitext text/x-wiki jcp8xtiwabm3bpqz00vjjc8ctd8npzi 2409069 2409068 2026-05-04T14:00:16Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.dns.admin DNS admin: depool ulsfo [reason: New switch configuration, T408892] 2409069 wikitext text/x-wiki kgvjudpzxjwo57bx56xw3wd1g25x7ot 2409070 2409069 2026-05-04T14:00:22Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: depool ulsfo [reason: New switch configuration, T408892] 2409070 wikitext text/x-wiki 6pwk1h5dinwanl5iqdadxww7nilyc38 2409071 2409070 2026-05-04T14:00:46Z Stashbot 7414 slyngshede@cumin1003: conftool action : set/pooled=no; selector: cluster=dnsbox,dc=ulsfo [reason: ulsfo switch refresh T408892] 2409071 wikitext text/x-wiki 0i4nqs3rvktu9madc5lfxfbkyjbvnq9 2409072 2409071 2026-05-04T14:01:06Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1226', diff saved to https://phabricator.wikimedia.org/P92213 and previous config saved to /var/cache/conftool/dbconfig/20260504-140105-fceratto.json 2409072 wikitext text/x-wiki 3yy2i8al6ht8pphzd1gtmr04cnuc27k 2409073 2409072 2026-05-04T14:04:24Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2001.codfw.wmnet with OS trixie 2409073 wikitext text/x-wiki ih4yi75d7wpowmbs0715fa5pscohxi2 2409074 2409073 2026-05-04T14:04:53Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2001 2409074 wikitext text/x-wiki 4e0bqxmqxdb52i1haoxz2xzv24vufq5 2409075 2409074 2026-05-04T14:07:55Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.dns.netbox 2409075 wikitext text/x-wiki 7f42xmhjepmtrwwi9bjpy891efkkvwy 2409076 2409075 2026-05-04T14:11:14Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1226 (T419961)', diff saved to https://phabricator.wikimedia.org/P92214 and previous config saved to /var/cache/conftool/dbconfig/20260504-141113-fceratto.json 2409076 wikitext text/x-wiki e1vljixeyvn1dsqur160wqr1v5xuev3 2409077 2409076 2026-05-04T14:13:16Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2001 - herron@cumin1003" 2409077 wikitext text/x-wiki os4mkbbhhulimkx8p161xfiql168hr1 2409078 2409077 2026-05-04T14:13:23Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2001 - herron@cumin1003" 2409078 wikitext text/x-wiki lzgk7airrv4mc7b1y6e5aqybgcqr1cn 2409079 2409078 2026-05-04T14:13:23Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2409079 wikitext text/x-wiki 6qrn1upmxfka39ir7nilu3ucn53b796 2409080 2409079 2026-05-04T14:13:24Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2001.codfw.wmnet 94.0.192.10.in-addr.arpa 4.9.0.0.0.0.0.0.2.9.1.0.0.1.0.0.1.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2409080 wikitext text/x-wiki 8mfn7byralpcmy4ui43d5dnyyzdjm12 2409081 2409080 2026-05-04T14:13:26Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2001.codfw.wmnet 94.0.192.10.in-addr.arpa 4.9.0.0.0.0.0.0.2.9.1.0.0.1.0.0.1.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2409081 wikitext text/x-wiki 5ji68wm4wvgz3ntl7y6npmry3r2m3o7 2409082 2409081 2026-05-04T14:13:27Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2001 2409082 wikitext text/x-wiki 7ajfe2kelxyowi81hmrvtvek6bmwb0g 2409083 2409082 2026-05-04T14:16:36Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2001 2409083 wikitext text/x-wiki 0knabbiyy8y9digitw4l3wijaiw3t7i 2409084 2409083 2026-05-04T14:16:37Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2001 2409084 wikitext text/x-wiki 64eb2rfgrvo7pcghcstq6u8mkrh25e4 2409085 2409084 2026-05-04T14:25:31Z Stashbot 7414 pt1979@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on asw2-ulsfo,cr[3-4]-ulsfo,mr1-ulsfo with reason: switch refresh 2409085 wikitext text/x-wiki ozeefpgu5vinpjiod1j9qxm1231f5v5 2409086 2409085 2026-05-04T14:28:45Z Stashbot 7414 pt1979@cumin1003: DONE (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 4:00:00 on cr[3-4]-ulsfo IPV6,cr[3-4]-ulsfo.mgmt,mr1-ulsfo IPV6 with reason: switch refresh 2409086 wikitext text/x-wiki 2kvhegtk6wyjqx3ae0thv37e6k75jt6 2409087 2409086 2026-05-04T14:30:37Z Stashbot 7414 pt1979@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on cr[3-4]-ulsfo IPv6,cr[3-4]-ulsfo.mgmt,mr1-ulsfo IPv6 with reason: switch refresh 2409087 wikitext text/x-wiki 0ug2m68gcrmml3gk2ajvjfoawggy1uf 2409088 2409087 2026-05-04T14:33:27Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2229.codfw.wmnet with reason: Maintenance 2409088 wikitext text/x-wiki rif84z880pwyu96tb831hro8twu0fut 2409089 2409088 2026-05-04T14:33:36Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db2229 (T419635)', diff saved to https://phabricator.wikimedia.org/P92215 and previous config saved to /var/cache/conftool/dbconfig/20260504-143334-fceratto.json 2409089 wikitext text/x-wiki e7vipw298omvq47xd6wblemsop1p79f 2409091 2409089 2026-05-04T14:34:44Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2001.codfw.wmnet with reason: host reimage 2409091 wikitext text/x-wiki pls11vtzzdwjgzcni5gtyp698u327zd 2409092 2409091 2026-05-04T14:39:36Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2001.codfw.wmnet with reason: host reimage 2409092 wikitext text/x-wiki 7q9xrrjhe6ehoaofo1ax6h4gz911lq3 2409093 2409092 2026-05-04T14:41:17Z Stashbot 7414 pt1979@cumin1003: START - Cookbook sre.hosts.remove-downtime for 7 hosts 2409093 wikitext text/x-wiki k6aidgynsybbiog5i9w0ae6boxc1thm 2409094 2409093 2026-05-04T14:41:22Z Stashbot 7414 pt1979@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for 7 hosts 2409094 wikitext text/x-wiki rn4yb48iachpq0ayb56wqs3pgcw659p 2409095 2409094 2026-05-04T14:42:16Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2229 (T419635)', diff saved to https://phabricator.wikimedia.org/P92216 and previous config saved to /var/cache/conftool/dbconfig/20260504-144213-fceratto.json 2409095 wikitext text/x-wiki cr65wk81x79bkfrlom81fdrb6iswyvq 2409100 2409095 2026-05-04T14:52:23Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2229', diff saved to https://phabricator.wikimedia.org/P92217 and previous config saved to /var/cache/conftool/dbconfig/20260504-145222-fceratto.json 2409100 wikitext text/x-wiki lhpj72j79hxdwdmdel18c0fpw4ix4cf 2409103 2409100 2026-05-04T14:58:41Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2001.codfw.wmnet with OS trixie 2409103 wikitext text/x-wiki afrtb0xxxu9ugzx68cfnuwscko1tvq5 2409104 2409103 2026-05-04T14:58:52Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1282381|Close Gun Wikinews (T421796)]] 2409104 wikitext text/x-wiki 2zlfn335rpe0xpi3wox8ibrj2679cwb 2409105 2409104 2026-05-04T15:00:36Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1282381|Close Gun Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409105 wikitext text/x-wiki oe1t9vwy9r2her3f8fnasxx711vyvc2 2409106 2409105 2026-05-04T15:01:22Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2409106 wikitext text/x-wiki 03t3kbrxoznij34sjk7rezmeh9cjesm 2409107 2409106 2026-05-04T15:02:31Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2229', diff saved to https://phabricator.wikimedia.org/P92218 and previous config saved to /var/cache/conftool/dbconfig/20260504-150230-fceratto.json 2409107 wikitext text/x-wiki ff3sax5ludeisgb6dk3gml8tjqcfj8y 2409108 2409107 2026-05-04T15:05:38Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1282381|Close Gun Wikinews (T421796)]] (duration: 06m 45s) 2409108 wikitext text/x-wiki agpe75dhxwq8hz5c247bior3jk0c0rw 2409109 2409108 2026-05-04T15:06:44Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409109 wikitext text/x-wiki l7kvgbkmwxo5kyxnr91fsar4f6r0isn 2409110 2409109 2026-05-04T15:10:03Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) 2409110 wikitext text/x-wiki ks5wswtmota6r7n0y1nnh1w79nosx05 2409111 2409110 2026-05-04T15:10:13Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2409111 wikitext text/x-wiki n8alfr8q645x1iu818mo5iw3eze89si 2409112 2409111 2026-05-04T15:10:57Z Stashbot 7414 papaul: ongoing switch refresh in ULSFO 2409112 wikitext text/x-wiki sryvcw9p068hrp87y9woqu66wsx66zt 2409113 2409112 2026-05-04T15:12:40Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2229 (T419635)', diff saved to https://phabricator.wikimedia.org/P92219 and previous config saved to /var/cache/conftool/dbconfig/20260504-151238-fceratto.json 2409113 wikitext text/x-wiki o4i7g0sxeqhek71viucxgpvyzq69j0s 2409114 2409113 2026-05-04T15:13:02Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2409114 wikitext text/x-wiki 26rn2j6p7vxqbwkt7so8ot7wyc4r65v 2409115 2409114 2026-05-04T15:13:03Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache install5004.wikimedia.org on all recursors 2409115 wikitext text/x-wiki 6n1dk3rbaaqw0tanpjbauuenn9lqn0d 2409116 2409115 2026-05-04T15:13:06Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) install5004.wikimedia.org on all recursors 2409116 wikitext text/x-wiki hnkguil5461k3svd885fntbgb8g1x7l 2409117 2409116 2026-05-04T15:13:13Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host install5004.wikimedia.org 2409117 wikitext text/x-wiki bwa2citmfzp7wtzcqa8ajypl1m80180 2409119 2409117 2026-05-04T15:15:06Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409119 wikitext text/x-wiki sy0vpdt87raywg4c7s92nrib5tqycup 2409120 2409119 2026-05-04T15:16:05Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409120 wikitext text/x-wiki nxg71qsns4x4gjb9qh59weg1l0fxq17 2409121 2409120 2026-05-04T15:17:26Z Stashbot 7414 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'. 2409121 wikitext text/x-wiki t3r2l6c7i4vw830rtbok07eja0dadde 2409122 2409121 2026-05-04T15:17:59Z Stashbot 7414 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'. 2409122 wikitext text/x-wiki snlt1v5wbgj45myemkrdaowewnhxqv3 2409124 2409122 2026-05-04T15:20:08Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409124 wikitext text/x-wiki a8vsjj0z8fg4793376k26hpwr4dj7rj 2409126 2409124 2026-05-04T15:22:31Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2229.codfw.wmnet with reason: Maintenance 2409126 wikitext text/x-wiki ct47byxevfrx8rmjwgra8ulatdy9i7g 2409127 2409126 2026-05-04T15:22:41Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db2229 (T419635)', diff saved to https://phabricator.wikimedia.org/P92220 and previous config saved to /var/cache/conftool/dbconfig/20260504-152238-fceratto.json 2409127 wikitext text/x-wiki 7f0xfws4gbbuno6m1f02aegmyhdpz01 2409128 2409127 2026-05-04T15:24:52Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2229 (T419635)', diff saved to https://phabricator.wikimedia.org/P92221 and previous config saved to /var/cache/conftool/dbconfig/20260504-152449-fceratto.json 2409128 wikitext text/x-wiki 75eknedyscoiqj3bpn4a0wgrudhcbxy 2409129 2409128 2026-05-04T15:31:29Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1282060|Make errorpages responsive]] 2409129 wikitext text/x-wiki 5zrf18p9fwavbnd23p0szc0waylcuyc 2409130 2409129 2026-05-04T15:32:13Z Stashbot 7414 elukey@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-logging-external: sync 2409130 wikitext text/x-wiki kuzq6mvdk9hji9gc4q3gwrltjsrw9yo 2409131 2409130 2026-05-04T15:32:36Z Stashbot 7414 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-logging-external: sync 2409131 wikitext text/x-wiki is1lpwi93yd5xvb1mo0fbe8uins8j3g 2409132 2409131 2026-05-04T15:33:11Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup, chlod: Backport for [[gerrit:1282060|Make errorpages responsive]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409132 wikitext text/x-wiki jkb1yx4vav73secramnt047hv0j428u 2409133 2409132 2026-05-04T15:33:30Z Stashbot 7414 ayounsi@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 20:00:00 on 39 hosts with reason: switches replacement 2409133 wikitext text/x-wiki h7z5i5xdnc6yi64l8nxurfaaccwxfsw 2409134 2409133 2026-05-04T15:34:16Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup, chlod: Continuing with deployment 2409134 wikitext text/x-wiki kurf000zttpy677oz1enx9n9bn92h0h 2409135 2409134 2026-05-04T15:34:59Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2229', diff saved to https://phabricator.wikimedia.org/P92222 and previous config saved to /var/cache/conftool/dbconfig/20260504-153458-fceratto.json 2409135 wikitext text/x-wiki avunop72xd2f2cmsztpq5mguq1yetmh 2409136 2409135 2026-05-04T15:38:29Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1282060|Make errorpages responsive]] (duration: 06m 59s) 2409136 wikitext text/x-wiki rz4evf4281frm4rm7exe93gl9pntqoo 2409137 2409136 2026-05-04T15:45:08Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2229', diff saved to https://phabricator.wikimedia.org/P92223 and previous config saved to /var/cache/conftool/dbconfig/20260504-154506-fceratto.json 2409137 wikitext text/x-wiki 5h9xuwmsqpxylq8y8o21lqina4zgq2r 2409138 2409137 2026-05-04T15:55:17Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2229 (T419635)', diff saved to https://phabricator.wikimedia.org/P92224 and previous config saved to /var/cache/conftool/dbconfig/20260504-155514-fceratto.json 2409138 wikitext text/x-wiki n5qumf59ixpn7q6dgqf79640d7tfm9f 2409139 2409138 2026-05-04T15:58:25Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1282384|Close Greek Wikinews (T421796)]] 2409139 wikitext text/x-wiki fj57nk6wjjfo73k0wq08d8rqtm2rqv1 2409140 2409139 2026-05-04T16:00:07Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1282384|Close Greek Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409140 wikitext text/x-wiki aar8apvxr48dk5ul6wi2xozk6a4jdwd 2409141 2409140 2026-05-04T16:00:29Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2409141 wikitext text/x-wiki ktmh5084t66fw7y14zq5j7tv7owyw9s 2409142 2409141 2026-05-04T16:04:45Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1282384|Close Greek Wikinews (T421796)]] (duration: 06m 19s) 2409142 wikitext text/x-wiki 7h38r1p1zhx8dxerzgqb6nji6stvqnt 2409143 2409142 2026-05-04T16:33:32Z Stashbot 7414 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply 2409143 wikitext text/x-wiki 1rnlktznymh007h4lvxynr6on63zgfp 2409144 2409143 2026-05-04T16:33:45Z Stashbot 7414 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply 2409144 wikitext text/x-wiki pn3mgy00482pi7ev11yi2wqzvgzigx7 2409145 2409144 2026-05-04T16:33:50Z Stashbot 7414 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply 2409145 wikitext text/x-wiki 0kbe88yi07h9a4m1b7060yx9xka0upg 2409146 2409145 2026-05-04T16:34:53Z Stashbot 7414 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply 2409146 wikitext text/x-wiki lg5fwjk9eogoq8skqdnh4ck5q9243rc 2409147 2409146 2026-05-04T16:39:50Z Stashbot 7414 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply 2409147 wikitext text/x-wiki s04447f06dswanqixasc39kauzm25tl 2409148 2409147 2026-05-04T16:40:06Z Stashbot 7414 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply 2409148 wikitext text/x-wiki 49r0d1imi6ls94n6wx0xbjorx2i4cpn 2409150 2409148 2026-05-04T17:31:38Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409150 wikitext text/x-wiki cqe9o3m4lb4qh5kbxyfmeobxlvurbtb 2409151 2409150 2026-05-04T17:38:02Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409151 wikitext text/x-wiki 9v36kq14161w9ei9i47li9vh0ti1r9z 2409152 2409151 2026-05-04T17:38:04Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409152 wikitext text/x-wiki 1hs2zvj617aymunmha5knzg2ur5o1xt 2409153 2409152 2026-05-04T17:38:06Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409153 wikitext text/x-wiki 6bcv4jt2pl9bny3c105vhjsrhig5fkh 2409155 2409153 2026-05-04T17:41:16Z Stashbot 7414 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409155 wikitext text/x-wiki ipovyfpe39a1ga73u4ifeia3g5rod3i 2409157 2409155 2026-05-04T17:47:20Z Stashbot 7414 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409157 wikitext text/x-wiki mhmocvuej9h5npmt50lzhz83nkyrvnl 2409158 2409157 2026-05-04T17:47:39Z Stashbot 7414 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409158 wikitext text/x-wiki 7aewujs8t62xaxw5fshek4antikcy4v 2409159 2409158 2026-05-04T17:49:14Z Stashbot 7414 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2409159 wikitext text/x-wiki 2edhbrlpyulkmo2ebuzww8y71leh3ku 2409160 2409159 2026-05-04T18:06:52Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.260.0" for 2 host(s) 2409160 wikitext text/x-wiki c1keyqbe8z4u3sf93voeirgbgzlch6t 2409161 2409160 2026-05-04T18:08:45Z Stashbot 7414 dancy@deploy1003: Installation of scap version "4.260.0" completed for 2 hosts 2409161 wikitext text/x-wiki cntqikh2gu4q6ovst7b7wt382n792ny 2409162 2409161 2026-05-04T18:09:20Z Stashbot 7414 dancy@deploy1003: Started scap sync-world: testing 2409162 wikitext text/x-wiki 6ib8ldv52svpk8lv55tsuayslvt84fq 2409163 2409162 2026-05-04T18:10:31Z Stashbot 7414 dancy@deploy1003: dancy: testing synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409163 wikitext text/x-wiki m66oo2batvi3e97wyryw6lx68zb5wt4 2409164 2409163 2026-05-04T18:11:00Z Stashbot 7414 dancy@deploy1003: dancy: Rolling back deployment 2409164 wikitext text/x-wiki byd0pe6wic77l0lwwgd513dj7dl6q5d 2409165 2409164 2026-05-04T18:11:24Z Stashbot 7414 dancy@deploy1003: Finished scap sync-world: testing (duration: 02m 04s) 2409165 wikitext text/x-wiki mk1wz4c1pj0see46ubgfsztsrjcuil6 2409166 2409165 2026-05-04T18:22:13Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1282405|Close Albanian Wikinews (T421796)]] 2409166 wikitext text/x-wiki 038g85d9xv7v5immxkx3bzukqszexmi 2409167 2409166 2026-05-04T18:23:55Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1282405|Close Albanian Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409167 wikitext text/x-wiki aoui9b2ljj9jbbve9ofotd07jkmsnth 2409169 2409167 2026-05-04T18:27:09Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2409169 wikitext text/x-wiki swqf3l2ix0qq6lnngh0vvc1lhw4fioi 2409170 2409169 2026-05-04T18:31:30Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1282405|Close Albanian Wikinews (T421796)]] (duration: 09m 17s) 2409170 wikitext text/x-wiki irsubg4gdeysyb4jiriw27hhvj9oeqq 2409171 2409170 2026-05-04T18:53:37Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1282407|Close Limburgish Wikinews (T421796)]] 2409171 wikitext text/x-wiki iwcgyrtgcydxvj9m169g9yq936ibypk 2409172 2409171 2026-05-04T18:55:19Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1282407|Close Limburgish Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409172 wikitext text/x-wiki dek44w3jjwa22l6lncrzs4401zxgdhd 2409173 2409172 2026-05-04T18:55:41Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2409173 wikitext text/x-wiki iyk2th7gr5m924xm4olhc1fage83ure 2409175 2409173 2026-05-04T18:59:53Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1282407|Close Limburgish Wikinews (T421796)]] (duration: 06m 16s) 2409175 wikitext text/x-wiki 0wi844qc2q1p7rweo31hqetqfr7zee0 2409176 2409175 2026-05-04T19:06:18Z Stashbot 7414 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply 2409176 wikitext text/x-wiki 9d0955t4gtm7up5ad7j9zr5gtgqphz4 2409177 2409176 2026-05-04T19:06:26Z Stashbot 7414 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply 2409177 wikitext text/x-wiki d7q2lytfgpq7gyuw7ay7vrbatael9lm 2409178 2409177 2026-05-04T19:23:10Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: remove privatemounts to see if it helps - bking@cumin2002 - T424852 2409178 wikitext text/x-wiki glyvw3ugotb7ili4lnv880u8uyqutg8 2409179 2409178 2026-05-04T19:23:26Z Stashbot 7414 root@deploy1003: helmfile [eqiad] START helmfile.d/admin 'sync'. 2409179 wikitext text/x-wiki 1w83ofwme8oaz2ehvi7pvrrm4zhideq 2409180 2409179 2026-05-04T19:23:38Z Stashbot 7414 bking@cumin2002: END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: remove privatemounts to see if it helps - bking@cumin2002 - T424852 2409180 wikitext text/x-wiki d7zdn112yikrnzbrupgw2pc6dbv7cv5 2409181 2409180 2026-05-04T19:23:41Z Stashbot 7414 root@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'sync'. 2409181 wikitext text/x-wiki 0uedrrh9bh8ofu2csqz4jl2t3iskvy9 2409182 2409181 2026-05-04T19:27:22Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging1005.eqiad.wmnet with OS trixie 2409182 wikitext text/x-wiki 9t6mfi9harf277ab4hebp0dwjr8rw0h 2409183 2409182 2026-05-04T19:27:43Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging1005 2409183 wikitext text/x-wiki 3mstv74888ene3nnftcuvecvz6ovip4 2409184 2409183 2026-05-04T19:27:44Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging1005 2409184 wikitext text/x-wiki n114bzhu7ellutexvcsuqju6x6qqv41 2409185 2409184 2026-05-04T19:28:46Z Stashbot 7414 bking@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on 6 hosts with reason: ongoing troubleshooting 2409185 wikitext text/x-wiki srcxfjpki7cvxw19igy6iip7b1oktmn 2409186 2409185 2026-05-04T19:37:09Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: remove privatemounts to see if it helps - bking@cumin2002 - T424852 2409186 wikitext text/x-wiki i56zn3b4k6elzea74a50533zr8f92j6 2409187 2409186 2026-05-04T19:40:26Z Stashbot 7414 bking@cumin2002: END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: remove privatemounts to see if it helps - bking@cumin2002 - T424852 2409187 wikitext text/x-wiki h42al6dosy8pwx51zwiir4te86kqz84 2409188 2409187 2026-05-04T19:42:18Z Stashbot 7414 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging1005.eqiad.wmnet with reason: host reimage 2409188 wikitext text/x-wiki 7vbzgw3bt6s9zixrrpo427m8l9zb27i 2409189 2409188 2026-05-04T19:44:48Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.dns.netbox 2409189 wikitext text/x-wiki dlidl6xdkm977tq66tg8fkmlfd70yqf 2409190 2409189 2026-05-04T19:48:53Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging1005.eqiad.wmnet with reason: host reimage 2409190 wikitext text/x-wiki o6xwlm19ahb68ddh3d6q5sokj2hmj9h 2409191 2409190 2026-05-04T19:49:32Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: asw1-22-ulsfo - ayounsi@cumin1003" 2409191 wikitext text/x-wiki fex7jebpjslm5saxr0lkeq6m1k9n5np 2409192 2409191 2026-05-04T19:49:37Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: asw1-22-ulsfo - ayounsi@cumin1003" 2409192 wikitext text/x-wiki jr8gl9loo0snwevptmqz3ubp2e0qz8l 2409193 2409192 2026-05-04T19:49:38Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2409193 wikitext text/x-wiki b5nt88mtxeja3rnwp824hnextsdgwhg 2409194 2409193 2026-05-04T19:50:58Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.dns.wipe-cache asw1-22-ulsfo.wikimedia.org on all recursors 2409194 wikitext text/x-wiki 39ic09ak8jgkr3oljsjtlb69gs7ztca 2409195 2409194 2026-05-04T19:51:19Z Stashbot 7414 ayounsi@cumin1003: END (FAIL) - Cookbook sre.dns.wipe-cache (exit_code=99) asw1-22-ulsfo.wikimedia.org on all recursors 2409195 wikitext text/x-wiki m4e1y62vi1syq66ytfs11puqfyp78f1 2409197 2409195 2026-05-04T20:03:44Z Stashbot 7414 toyofuku@deploy1003: Started scap sync-world: Backport for [[gerrit:1277667|Enable the reading list beta feature survey on all wikipedias (T421776)]] 2409197 wikitext text/x-wiki 0ldl006rqzje1kh3guy4t20ic06qxz7 2409198 2409197 2026-05-04T20:05:26Z Stashbot 7414 toyofuku@deploy1003: toyofuku: Backport for [[gerrit:1277667|Enable the reading list beta feature survey on all wikipedias (T421776)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409198 wikitext text/x-wiki r0uaiutje8jj8g7om66uf5p88n1zp9c 2409199 2409198 2026-05-04T20:06:51Z Stashbot 7414 toyofuku@deploy1003: toyofuku: Continuing with deployment 2409199 wikitext text/x-wiki 9hcyk16xpt02r8y2hbdd4igrh21oiny 2409200 2409199 2026-05-04T20:07:14Z Stashbot 7414 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging1005.eqiad.wmnet with OS trixie 2409200 wikitext text/x-wiki dfdbf7cj7ld668fcs4kiyjvsu3d8fqd 2409201 2409200 2026-05-04T20:11:05Z Stashbot 7414 toyofuku@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277667|Enable the reading list beta feature survey on all wikipedias (T421776)]] (duration: 07m 21s) 2409201 wikitext text/x-wiki fhqrre1ktqdbrw4kmvpwv50luocu8pu 2409202 2409201 2026-05-04T20:16:26Z Stashbot 7414 cjming@deploy1003: Started scap sync-world: Backport for [[gerrit:1282385|Revert^2 "Use js promise for email confirmation banner"]] 2409202 wikitext text/x-wiki 81qc3re78o9b02xeqc0zbp3xj5up4hh 2409203 2409202 2026-05-04T20:18:07Z Stashbot 7414 cjming@deploy1003: mmartorana, cjming: Backport for [[gerrit:1282385|Revert^2 "Use js promise for email confirmation banner"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409203 wikitext text/x-wiki 7m7qo2ldju2k3u6m91fvje6ib1rzfqq 2409204 2409203 2026-05-04T20:34:37Z Stashbot 7414 cjming@deploy1003: mmartorana, cjming: Continuing with deployment 2409204 wikitext text/x-wiki 856qzgyfrhk6az3aftugjf2abjzp0zg 2409205 2409204 2026-05-04T20:38:46Z Stashbot 7414 cjming@deploy1003: Finished scap sync-world: Backport for [[gerrit:1282385|Revert^2 "Use js promise for email confirmation banner"]] (duration: 22m 19s) 2409205 wikitext text/x-wiki tme2zok79yasxfbmu5gpg1yqfcvv6t8 2409208 2409205 2026-05-04T21:09:09Z Stashbot 7414 cjming@deploy1003: Started scap sync-world: Backport for [[gerrit:1276432|Enable Hebrew keyboard DWIM for namespace resolution on hewikis (T412468)]] 2409208 wikitext text/x-wiki 3o04k9eothxqk2lmpjac1jnf2l3myio 2409209 2409208 2026-05-04T21:10:50Z Stashbot 7414 cjming@deploy1003: cjming, neriah: Backport for [[gerrit:1276432|Enable Hebrew keyboard DWIM for namespace resolution on hewikis (T412468)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409209 wikitext text/x-wiki 0ryj5gdgjq7t3junh4u3m0w3zfcd1cs 2409210 2409209 2026-05-04T21:16:19Z Stashbot 7414 cjming@deploy1003: cjming, neriah: Continuing with deployment 2409210 wikitext text/x-wiki 3j4v7p47knqq10mmodyxv8rv8le3twk 2409211 2409210 2026-05-04T21:20:31Z Stashbot 7414 cjming@deploy1003: Finished scap sync-world: Backport for [[gerrit:1276432|Enable Hebrew keyboard DWIM for namespace resolution on hewikis (T412468)]] (duration: 11m 20s) 2409211 wikitext text/x-wiki kwc3u7l8nsmviuu13eq7wjrxbjqn1rg 2409212 2409211 2026-05-04T21:32:24Z Stashbot 7414 cwhite@deploy1003: Started deploy [statsv/statsv@152de49]: fix logging 2409212 wikitext text/x-wiki eviiy0v28zemg6dqcxdk4z1a3e68j5g 2409213 2409212 2026-05-04T21:32:36Z Stashbot 7414 cwhite@deploy1003: Finished deploy [statsv/statsv@152de49]: fix logging (duration: 00m 11s) 2409213 wikitext text/x-wiki gixy6g72eczmdtfu9upuysfcx4hd01p 2409214 2409213 2026-05-04T21:42:52Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply 2409214 wikitext text/x-wiki s1kevfyeirg667qcxe74b2kwo3svbs9 2409216 2409214 2026-05-04T21:43:15Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply 2409216 wikitext text/x-wiki rrk3to72hukgm8x1dnw356x4ax8ww99 2409218 2409216 2026-05-04T22:06:46Z Stashbot 7414 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply 2409218 wikitext text/x-wiki 8spa6zuqsdyb3n4jkcf4taknrqmrkct 2409219 2409218 2026-05-04T22:06:50Z Stashbot 7414 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply 2409219 wikitext text/x-wiki cr9sqvybhoc88smcwd3z7rwggp3lmmb 2409220 2409219 2026-05-04T22:06:55Z Stashbot 7414 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply 2409220 wikitext text/x-wiki 9rszio9x2eps5vcofxoualushn4451l 2409221 2409220 2026-05-04T22:08:47Z Stashbot 7414 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply 2409221 wikitext text/x-wiki j6hr0mrem3g7hsam3xs84mk54qlzirl 2409222 2409221 2026-05-04T23:07:56Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1282410|Close Hebrew Wikinews (T421796)]] 2409222 wikitext text/x-wiki ja0eeerwps65yk9kjqka7o3ki2no4v9 2409223 2409222 2026-05-04T23:09:45Z Stashbot 7414 ladsgroup@deploy1003: neriah, ladsgroup: Backport for [[gerrit:1282410|Close Hebrew Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409223 wikitext text/x-wiki c7kr1sul0aurmkxyuewpe8v1hxtr0tk 2409224 2409223 2026-05-04T23:10:22Z Stashbot 7414 ladsgroup@deploy1003: neriah, ladsgroup: Continuing with deployment 2409224 wikitext text/x-wiki mugak2i16rg19rytv65n82qznbapecp 2409225 2409224 2026-05-04T23:14:36Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1282410|Close Hebrew Wikinews (T421796)]] (duration: 06m 45s) 2409225 wikitext text/x-wiki 29nw6lld8gt7rmuawvpmyy1s26jihyh 2409226 2409225 2026-05-04T23:46:44Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1282432|Close Bosnian Wikinews (T421796)]] 2409226 wikitext text/x-wiki f9fif4zrmph4xo29e8c7csin9zrcm5u 2409227 2409226 2026-05-04T23:48:26Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1282432|Close Bosnian Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409227 wikitext text/x-wiki 4quyq7vq05kd4wz3k96xs153i1ozjyd 2409229 2409227 2026-05-05T00:09:14Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1282434|Close Catalan Wikinews (T421796)]] 2409229 wikitext text/x-wiki 19egrpqg39fjdexwei7d47ytml74tj6 2409230 2409229 2026-05-05T00:10:58Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1282434|Close Catalan Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409230 wikitext text/x-wiki 4kgi8iam2d343u7s8gvc4t7j9542huu 2409231 2409230 2026-05-05T00:11:54Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2409231 wikitext text/x-wiki a4c4toc3n9mzkyesnhihmzoat22mu6b 2409232 2409231 2026-05-05T00:16:04Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1282434|Close Catalan Wikinews (T421796)]] (duration: 06m 50s) 2409232 wikitext text/x-wiki 36xundxlxr7ny2m4n6lm315s0zlzc6n 2409233 2409232 2026-05-05T01:16:34Z Stashbot 7414 pt1979@cumin2002: START - Cookbook sre.dns.netbox 2409233 wikitext text/x-wiki h2n5batehus6w30b5td6j7n5xxtmhee 2409234 2409233 2026-05-05T01:21:12Z Stashbot 7414 pt1979@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add dns infor for new switches - pt1979@cumin2002" 2409234 wikitext text/x-wiki 7jmzluguvzdx8zk5sr98s60ouj2owjt 2409235 2409234 2026-05-05T01:21:34Z Stashbot 7414 pt1979@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add dns infor for new switches - pt1979@cumin2002" 2409235 wikitext text/x-wiki 7oc1hrwhjh6dpkdii57wr23xjbnzf12 2409236 2409235 2026-05-05T01:21:35Z Stashbot 7414 pt1979@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2409236 wikitext text/x-wiki nv93rx3kmxtk90xcf36nuro7ck5ew2n 2409244 2409236 2026-05-05T02:00:50Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2409244 wikitext text/x-wiki ntwr2zw85vzfhs53brivt4hl4hdhkb8 2409246 2409244 2026-05-05T02:07:29Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 39s) 2409246 wikitext text/x-wiki b3nb36co1l2ta1j3c4box8l7trjrk8p 2409261 2409246 2026-05-05T04:03:16Z Stashbot 7414 mwpresync@deploy1003: Pruned MediaWiki: 1.46.0-wmf.23 (duration: 03m 12s) 2409261 wikitext text/x-wiki t9v83c8cmrbv6qicbya00ajtr8bekjd 2409266 2409261 2026-05-05T05:28:52Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s2 master: reimage to Debian Trixie 2409266 wikitext text/x-wiki rhkq5gf229xrolgbl73zb58dq0hqx8s 2409267 2409266 2026-05-05T05:30:10Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1156.eqiad.wmnet with reason: Reimage to Trixie 2409267 wikitext text/x-wiki jfbh86ezxcgsbe096uuuchhcvn0soy1 2409268 2409267 2026-05-05T05:30:15Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1156: Reimage to Trixie 2409268 wikitext text/x-wiki ao31jnqwljysce0elv0al93dmoprrp0 2409269 2409268 2026-05-05T05:31:13Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1156: Reimage to Trixie 2409269 wikitext text/x-wiki rkcp4bzxvf400xamg5qss45mi9g687a 2409270 2409269 2026-05-05T05:33:15Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1156.eqiad.wmnet with OS trixie 2409270 wikitext text/x-wiki 0y61gjnktwmoko7m2x06gyjawjoybd3 2409271 2409270 2026-05-05T05:42:51Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "patterns_as_inline_patterns - oblivian@cumin1003" 2409271 wikitext text/x-wiki 4ue0jciehdti7zkhu9eapo8mm80wxue 2409272 2409271 2026-05-05T05:42:53Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: patterns_as_inline_patterns - oblivian@cumin1003 2409272 wikitext text/x-wiki 17ica700i5s59mga2so4llsk27028c3 2409273 2409272 2026-05-05T05:43:46Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: patterns_as_inline_patterns - oblivian@cumin1003 2409273 wikitext text/x-wiki fteqvjjbkffgy68v0gx2ex9vq3dhmyy 2409274 2409273 2026-05-05T05:43:48Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.hiddenparma (exit_code=0) Hiddenparma deployment to the alerting hosts with reason: "patterns_as_inline_patterns - oblivian@cumin1003" 2409274 wikitext text/x-wiki 207oh6mjjtiw6rsxvamuusme584yyg0 2409275 2409274 2026-05-05T05:46:10Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1156.eqiad.wmnet with reason: host reimage 2409275 wikitext text/x-wiki rjmef3lrq0i7px9zjsmal9ndyeixxe5 2409276 2409275 2026-05-05T05:49:43Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1156.eqiad.wmnet with reason: host reimage 2409276 wikitext text/x-wiki tbhu9cp18ldyre4wcuq4q6u56d9pnv7 2409278 2409276 2026-05-05T06:11:12Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1156.eqiad.wmnet with OS trixie 2409278 wikitext text/x-wiki 8mr4p44ebd09s2rk43a5j9u5o0125pa 2409279 2409278 2026-05-05T06:14:43Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1156: after reimage to trixie 2409279 wikitext text/x-wiki 9w9me4lodyyu6545ludjgopvhtq20sh 2409280 2409279 2026-05-05T06:42:43Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2222.codfw.wmnet with reason: Reimage to Trixie 2409280 wikitext text/x-wiki q6q1m1tv2a6zan8sstk8ir7h3yptm0t 2409281 2409280 2026-05-05T06:42:49Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2222: Reimage to Trixie 2409281 wikitext text/x-wiki fbi852mbh6x0xo938knenwyes1t9akw 2409282 2409281 2026-05-05T06:42:52Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1170.eqiad.wmnet with reason: Reimage to Trixie 2409282 wikitext text/x-wiki trd4o03c0eywbkso5jol7pjxy996zy0 2409283 2409282 2026-05-05T06:42:57Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1170: Reimage to Trixie 2409283 wikitext text/x-wiki 1qkb1rcjmgdzrmkbnvstjccv88g6uq7 2409284 2409283 2026-05-05T06:43:07Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2222: Reimage to Trixie 2409284 wikitext text/x-wiki o7fin7i5xoyld8d2ef3t9orohl31rs5 2409285 2409284 2026-05-05T06:43:25Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1170: Reimage to Trixie 2409285 wikitext text/x-wiki f8drheqyqm26ro2weqkq8vfe3g6nhrw 2409286 2409285 2026-05-05T06:44:29Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2222.codfw.wmnet with OS trixie 2409286 wikitext text/x-wiki fp38cvi5co722b9q2n2l190id7k1252 2409287 2409286 2026-05-05T06:44:40Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1170.eqiad.wmnet with OS trixie 2409287 wikitext text/x-wiki l7dxlgr3uvtivuzsi5af5d34qt3astv 2409288 2409287 2026-05-05T06:58:36Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1170.eqiad.wmnet with reason: host reimage 2409288 wikitext text/x-wiki jheju4zdfsuoh5tflrj41qym4m6kk8t 2409289 2409288 2026-05-05T06:58:48Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host install5004.wikimedia.org 2409289 wikitext text/x-wiki jc1n1mavusp1u4i71papgmq1zcsmpvu 2409290 2409289 2026-05-05T06:58:51Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2409290 wikitext text/x-wiki syycdvghlxiq6a9z8w5ybusso73j8ty 2409291 2409290 2026-05-05T07:00:08Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1156: after reimage to trixie 2409291 wikitext text/x-wiki cer1qk44wvfdu1u7ysoyimdgp4bxgd5 2409292 2409291 2026-05-05T07:00:36Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2222.codfw.wmnet with reason: host reimage 2409292 wikitext text/x-wiki lx7f8z50dttc9ubzv5yo7qjursld0zk 2409293 2409292 2026-05-05T07:03:35Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) 2409293 wikitext text/x-wiki k699tgrstzt03gtgjdlizgrl8k86ps6 2409294 2409293 2026-05-05T07:03:41Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host install5004.wikimedia.org 2409294 wikitext text/x-wiki sk5jbuv4xkkyyanzix8fnpu88z518l7 2409295 2409294 2026-05-05T07:03:44Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1170.eqiad.wmnet with reason: host reimage 2409295 wikitext text/x-wiki rmjgv91gzb67icd6r627s2qiu9cp16s 2409296 2409295 2026-05-05T07:05:02Z Stashbot 7414 awight@deploy1003: Started scap sync-world: Backport for [[gerrit:1281967|zhwikinews: (2/2) revert 20th anniversary logo change (assets) (T420165)]] 2409296 wikitext text/x-wiki 1vhempf8mlubfrnf52exs496pq7ch4t 2409297 2409296 2026-05-05T07:06:49Z Stashbot 7414 awight@deploy1003: awight, 1f616emo: Backport for [[gerrit:1281967|zhwikinews: (2/2) revert 20th anniversary logo change (assets) (T420165)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409297 wikitext text/x-wiki lgtst60k6iuzbjvf44jhhieep645bc9 2409298 2409297 2026-05-05T07:07:30Z Stashbot 7414 awight@deploy1003: awight, 1f616emo: Continuing with deployment 2409298 wikitext text/x-wiki 1gq4uj1qjscska1gf9ayleex34xu906 2409299 2409298 2026-05-05T07:07:52Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2222.codfw.wmnet with reason: host reimage 2409299 wikitext text/x-wiki feinjcnf6df8t7v52ghj3rlzhk2ye4l 2409300 2409299 2026-05-05T07:11:46Z Stashbot 7414 awight@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281967|zhwikinews: (2/2) revert 20th anniversary logo change (assets) (T420165)]] (duration: 06m 43s) 2409300 wikitext text/x-wiki sh1i2qblk63daivu3zvmwncp8pcvws1 2409301 2409300 2026-05-05T07:26:10Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1170.eqiad.wmnet with OS trixie 2409301 wikitext text/x-wiki b3veb410r3jd3cpt8dctwf812f9cooa 2409302 2409301 2026-05-05T07:30:43Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1170: after reimage to trixie 2409302 wikitext text/x-wiki 0zbcmbirwa5o3m7tf3bxxug0wywvkrh 2409303 2409302 2026-05-05T07:31:11Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2222.codfw.wmnet with OS trixie 2409303 wikitext text/x-wiki 3nez2gmg5mg4bdclh9n1mil2au19jtp 2409304 2409303 2026-05-05T07:36:17Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2222: after reimage to trixie 2409304 wikitext text/x-wiki 6uffktjo8yhst8qj1d5s5qtjlmzab5l 2409305 2409304 2026-05-05T07:45:46Z Stashbot 7414 zabe: zabe@deploy1003:~$ mwscript namespaceDupes.php scnwiki --fix # T425378 2409305 wikitext text/x-wiki ajof7rosxrv23cmjkzcbwybu4xybvds 2409306 2409305 2026-05-05T07:50:40Z Stashbot 7414 zabe: zabe@deploy1003:~$ foreachwiki refreshImageMetadata --force --mediatype AUDIO --mime audio/midi # T414645 2409306 wikitext text/x-wiki sumlo0b5bos7dn14s4p2du26xxligue 2409307 2409306 2026-05-05T07:50:45Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 25 hosts with reason: Primary switchover s3 T424864 2409307 wikitext text/x-wiki o6g1lexz7beykfmk22lcrx6le4xw0x0 2409308 2409307 2026-05-05T07:51:59Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Set db2205 with weight 0 T424864', diff saved to https://phabricator.wikimedia.org/P92239 and previous config saved to /var/cache/conftool/dbconfig/20260505-075156-marostegui.json 2409308 wikitext text/x-wiki f922f87b2mn0odtl40lgcdm231vwxd8 2409309 2409308 2026-05-05T07:52:52Z Stashbot 7414 marostegui: Starting s3 codfw failover from db2209 to db2205 - T424864 2409309 wikitext text/x-wiki 1m4fjdiq7tb0dewclf289mvad4qhjw8 2409310 2409309 2026-05-05T07:54:09Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2155.codfw.wmnet with reason: Maintenance 2409310 wikitext text/x-wiki 0j2zagkhnav9rv6o4dusj85q6wz6sg3 2409311 2409310 2026-05-05T07:54:17Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db2155 (T419961)', diff saved to https://phabricator.wikimedia.org/P92241 and previous config saved to /var/cache/conftool/dbconfig/20260505-075416-fceratto.json 2409311 wikitext text/x-wiki 7lw2ii6oklypef27fqkwvawpbz7bkoe 2409313 2409311 2026-05-05T07:55:18Z Stashbot 7414 awight: EU morning deployment was fun 2409313 wikitext text/x-wiki ft9uumdiveiyul8r2v35gruxpd4vvyj 2409315 2409313 2026-05-05T07:56:57Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Promote db2205 to s3 primary T424864', diff saved to https://phabricator.wikimedia.org/P92242 and previous config saved to /var/cache/conftool/dbconfig/20260505-075654-marostegui.json 2409315 wikitext text/x-wiki 6vnb2mtdd7xpbrruldywcqtobdffcwy 2409316 2409315 2026-05-05T07:57:49Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool db2209 T424864', diff saved to https://phabricator.wikimedia.org/P92243 and previous config saved to /var/cache/conftool/dbconfig/20260505-075746-marostegui.json 2409316 wikitext text/x-wiki nm6u72gzj3gf30zlz4zzd27z76u2rv5 2409317 2409316 2026-05-05T07:58:34Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.dns.netbox 2409317 wikitext text/x-wiki izyz46xr35ocahbrn1uhn8nhv5elj7s 2409318 2409317 2026-05-05T08:01:06Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2209.codfw.wmnet with reason: Reimage to Trixie 2409318 wikitext text/x-wiki mg27qfxenhy2as9mxly6lptoaajx4xn 2409319 2409318 2026-05-05T08:01:12Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2209: Reimage to Trixie 2409319 wikitext text/x-wiki fz7wl4ukwru3396defm57sd2yxucjx1 2409320 2409319 2026-05-05T08:01:20Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2209: Reimage to Trixie 2409320 wikitext text/x-wiki 5eeunif46vkss7qukqdp5jd2bwkoeio 2409321 2409320 2026-05-05T08:01:34Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: ulsfo includes - ayounsi@cumin1003" 2409321 wikitext text/x-wiki r0jpbfm1l35jeqetzhvd6mlgp8zb7jn 2409322 2409321 2026-05-05T08:01:59Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: ulsfo includes - ayounsi@cumin1003" 2409322 wikitext text/x-wiki 9calgbgf7iyl3e44216mvabmpqz9kbe 2409323 2409322 2026-05-05T08:01:59Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2409323 wikitext text/x-wiki liq66ov8uxmi6b8prcmlu4cdvo6sqip 2409324 2409323 2026-05-05T08:02:39Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2209.codfw.wmnet with OS trixie 2409324 wikitext text/x-wiki iapip4ye8owy2si1wiy5ywyzb2r2lir 2409325 2409324 2026-05-05T08:03:03Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2155 (T419961)', diff saved to https://phabricator.wikimedia.org/P92245 and previous config saved to /var/cache/conftool/dbconfig/20260505-080301-fceratto.json 2409325 wikitext text/x-wiki 1kkjzop9mho0azr3zv1b8idme6qjltn 2409326 2409325 2026-05-05T08:03:13Z Stashbot 7414 ayounsi@dns1004: START - running authdns-update 2409326 wikitext text/x-wiki covsp9168cog87ypbpegjwg927d2sze 2409327 2409326 2026-05-05T08:05:30Z Stashbot 7414 ayounsi@dns1004: END - running authdns-update 2409327 wikitext text/x-wiki f7fq14iiuok0pca6nzijckae9uz1746 2409328 2409327 2026-05-05T08:08:38Z Stashbot 7414 zabe: zabe@deploy1003:~$ foreachwiki refreshImageMetadata --broken-only --mediatype AUDIO --mime audio/flac # T414641 2409328 wikitext text/x-wiki 62g1neoubizsul2jw2hticlbx9hhgoz 2409330 2409328 2026-05-05T08:13:10Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P92247 and previous config saved to /var/cache/conftool/dbconfig/20260505-081309-fceratto.json 2409330 wikitext text/x-wiki dtdbulvkymej16pt2l69f4tc3jzbrkj 2409331 2409330 2026-05-05T08:14:19Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host install5004.wikimedia.org 2409331 wikitext text/x-wiki 8qmbrdue6dz8p0s3ok2v6sp14koctlt 2409332 2409331 2026-05-05T08:14:22Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2409332 wikitext text/x-wiki 71j3hsiqc09ys5nhtpc9xz5gpo90lpr 2409333 2409332 2026-05-05T08:16:08Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1170: after reimage to trixie 2409333 wikitext text/x-wiki rbur9nfzdffzjg9757oouad6yax92gc 2409334 2409333 2026-05-05T08:16:40Z Stashbot 7414 zabe: zabe@deploy1003:~$ foreachwiki refreshImageMetadata --broken-only --mediatype AUDIO --mime audio/x-flac # T414641 2409334 wikitext text/x-wiki igmsxsgt8k9wga50cog16pww6ag3hx3 2409335 2409334 2026-05-05T08:22:01Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2209.codfw.wmnet with reason: host reimage 2409335 wikitext text/x-wiki pedc3tvl0d479doi4cwtpnjykpyyggx 2409336 2409335 2026-05-05T08:22:45Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2222: after reimage to trixie 2409336 wikitext text/x-wiki tw0cbshb5f765qboo3jetsbciy0absz 2409337 2409336 2026-05-05T08:23:19Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P92250 and previous config saved to /var/cache/conftool/dbconfig/20260505-082318-fceratto.json 2409337 wikitext text/x-wiki i4l882bvt4l0qzgat64p5ep4og9vm8t 2409338 2409337 2026-05-05T08:24:23Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . 2409338 wikitext text/x-wiki ly8v3fq8dxx63qc9txf5yc1bvxf6idc 2409339 2409338 2026-05-05T08:28:50Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM install5004.wikimedia.org - jmm@cumin2002" 2409339 wikitext text/x-wiki 1oaqvqxzp33hegho5fkqssjr1xdo4nf 2409340 2409339 2026-05-05T08:28:55Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM install5004.wikimedia.org - jmm@cumin2002" 2409340 wikitext text/x-wiki lgnpegmy0f3qadziwybaczojew47opc 2409341 2409340 2026-05-05T08:28:56Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2409341 wikitext text/x-wiki roger4idb2mxwsifb4b5qonnw4d2vyz 2409342 2409341 2026-05-05T08:28:57Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache install5004.wikimedia.org on all recursors 2409342 wikitext text/x-wiki rv8pmo2niwz3v56675lhy89fsp2jexz 2409343 2409342 2026-05-05T08:29:17Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.dns.wipe-cache (exit_code=99) install5004.wikimedia.org on all recursors 2409343 wikitext text/x-wiki t94b68iv8gl2gp1ac36n6axp40lfbcf 2409344 2409343 2026-05-05T08:29:45Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2209.codfw.wmnet with reason: host reimage 2409344 wikitext text/x-wiki 6gslejvprrxgx8vapt1zy71bmz6zhz4 2409345 2409344 2026-05-05T08:32:22Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/services/miscweb: apply 2409345 wikitext text/x-wiki 8ch8k8tjvqhernwgmwwp8hi37a9kbvx 2409346 2409345 2026-05-05T08:32:53Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/services/miscweb: apply 2409346 wikitext text/x-wiki nk9r1c5bo75yaa8h5tbkybcyraamvws 2409347 2409346 2026-05-05T08:32:57Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2409347 wikitext text/x-wiki skhjgtmkji9s82tiil3tzpwrrkr060q 2409348 2409347 2026-05-05T08:33:27Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2155 (T419961)', diff saved to https://phabricator.wikimedia.org/P92251 and previous config saved to /var/cache/conftool/dbconfig/20260505-083326-fceratto.json 2409348 wikitext text/x-wiki s9gurd5nbn9lypovr91p38ucxjqgvex 2409349 2409348 2026-05-05T08:33:49Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2172.codfw.wmnet with reason: Maintenance 2409349 wikitext text/x-wiki mqw06tdnzd3y5yoo6m93tl248aimje7 2409350 2409349 2026-05-05T08:33:57Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db2172 (T419961)', diff saved to https://phabricator.wikimedia.org/P92252 and previous config saved to /var/cache/conftool/dbconfig/20260505-083356-fceratto.json 2409350 wikitext text/x-wiki soha5cpdb3ulu8y7jhhu4c4nsvmi8xb 2409351 2409350 2026-05-05T08:34:03Z Stashbot 7414 ayounsi@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 20:00:00 on 13 hosts with reason: switches replacement 2409351 wikitext text/x-wiki nsghyxxiwql3pdjd63sjk9li9ry6t4k 2409352 2409351 2026-05-05T08:34:33Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/services/miscweb: apply 2409352 wikitext text/x-wiki 2wmkd17c7fn5ixct0cmm38gfx2jcfwj 2409353 2409352 2026-05-05T08:34:53Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/services/miscweb: apply 2409353 wikitext text/x-wiki kn93r19q1l82pi09tpv8yaae2fj43io 2409354 2409353 2026-05-05T08:35:18Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . 2409354 wikitext text/x-wiki aquk7xn9anq8xw181i8yh3iwibur14e 2409355 2409354 2026-05-05T08:37:03Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . 2409355 wikitext text/x-wiki i8j5stl2a47e4g5an43jbew9wbx9oz6 2409356 2409355 2026-05-05T08:37:12Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) 2409356 wikitext text/x-wiki firsphzjzgfrtilpk6dv4e6l9frjet4 2409357 2409356 2026-05-05T08:37:24Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.netbox 2409357 wikitext text/x-wiki 6mm6tygwbrscg77gshj03p99ra9845u 2409358 2409357 2026-05-05T08:38:28Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . 2409358 wikitext text/x-wiki qwxleswq65m4rqerubza0ctznoaf9c0 2409359 2409358 2026-05-05T08:40:02Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . 2409359 wikitext text/x-wiki j66jdy3pmgbadw9p3zlpnpz7rx90vfh 2409360 2409359 2026-05-05T08:41:09Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . 2409360 wikitext text/x-wiki 29b4dk1zpr0cu3hkqtjab35yaxc2j09 2409361 2409360 2026-05-05T08:42:32Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2172 (T419961)', diff saved to https://phabricator.wikimedia.org/P92253 and previous config saved to /var/cache/conftool/dbconfig/20260505-084231-fceratto.json 2409361 wikitext text/x-wiki 6hffharxhtl9qrt3wooje3vsr9ypd7g 2409362 2409361 2026-05-05T08:42:41Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . 2409362 wikitext text/x-wiki nwyp088ipracky3gc364ux9931zxe8e 2409363 2409362 2026-05-05T08:46:09Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2213.codfw.wmnet with reason: Maintenance 2409363 wikitext text/x-wiki 6ulr5qnfv1bnwmn3ohi2rfh08qsqmcu 2409364 2409363 2026-05-05T08:46:18Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db2213 (T419635)', diff saved to https://phabricator.wikimedia.org/P92254 and previous config saved to /var/cache/conftool/dbconfig/20260505-084616-fceratto.json 2409364 wikitext text/x-wiki a1ymb3289f5ytzsivc3z6qpgdyln4wz 2409366 2409364 2026-05-05T08:48:40Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Remove records for VM install5004.wikimedia.org - jmm@cumin2002" 2409366 wikitext text/x-wiki bgp7he4eou5wpmgltkmsh8nlaeuc6ae 2409367 2409366 2026-05-05T08:48:45Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Remove records for VM install5004.wikimedia.org - jmm@cumin2002" 2409367 wikitext text/x-wiki ftst65o25xixx7d0spfcjui7gxkzi00 2409368 2409367 2026-05-05T08:48:46Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2409368 wikitext text/x-wiki 17w1s8zclk0mmx4ngtsu9qcxycytb7f 2409369 2409368 2026-05-05T08:48:47Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache install5004.wikimedia.org on all recursors 2409369 wikitext text/x-wiki a3f9qjbgqfll5v7dgtz2zc9miwexke1 2409370 2409369 2026-05-05T08:49:06Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.dns.wipe-cache (exit_code=99) install5004.wikimedia.org on all recursors 2409370 wikitext text/x-wiki icx9fdv3ch6ayfitdmui5rocabjkvaq 2409371 2409370 2026-05-05T08:50:17Z Stashbot 7414 moritzm: installing augeas security updates 2409371 wikitext text/x-wiki c8kezt3teiqw3g3fwgelj5bbvwetw3q 2409372 2409371 2026-05-05T08:50:42Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host install5004.wikimedia.org 2409372 wikitext text/x-wiki kj2j7ay5euvr1g4crg45hhkdsalqx1q 2409373 2409372 2026-05-05T08:52:39Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2172', diff saved to https://phabricator.wikimedia.org/P92255 and previous config saved to /var/cache/conftool/dbconfig/20260505-085238-fceratto.json 2409373 wikitext text/x-wiki tmztv7ty3lbsr30xdtilaraj622of94 2409374 2409373 2026-05-05T08:52:46Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2209.codfw.wmnet with OS trixie 2409374 wikitext text/x-wiki or0d597guudtrsi2lpb0ede0qqgaw1z 2409375 2409374 2026-05-05T08:54:09Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2213 (T419635)', diff saved to https://phabricator.wikimedia.org/P92256 and previous config saved to /var/cache/conftool/dbconfig/20260505-085407-fceratto.json 2409375 wikitext text/x-wiki 27bk2tno4x5y4uc3wtw8jw5nm6vrw1j 2409376 2409375 2026-05-05T08:58:02Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2209: after reimage to trixie 2409376 wikitext text/x-wiki oao2pqt7qkwspwv7b8lgxvx9elnrs0h 2409377 2409376 2026-05-05T09:02:47Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2172', diff saved to https://phabricator.wikimedia.org/P92258 and previous config saved to /var/cache/conftool/dbconfig/20260505-090246-fceratto.json 2409377 wikitext text/x-wiki jknwu1kjksiwtrh79mfmxowqueyxo9t 2409378 2409377 2026-05-05T09:04:16Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2213', diff saved to https://phabricator.wikimedia.org/P92259 and previous config saved to /var/cache/conftool/dbconfig/20260505-090415-fceratto.json 2409378 wikitext text/x-wiki qh2h5jcslm25sbj5kquic6obskpmvea 2409379 2409378 2026-05-05T09:12:56Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2172 (T419961)', diff saved to https://phabricator.wikimedia.org/P92260 and previous config saved to /var/cache/conftool/dbconfig/20260505-091254-fceratto.json 2409379 wikitext text/x-wiki f2w94tj0tozbnave8uuch6pyugc1wj7 2409380 2409379 2026-05-05T09:13:18Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2199.codfw.wmnet with reason: Maintenance 2409380 wikitext text/x-wiki 2wrf2yvx9oau56ajqd4wwgygwbvhhx7 2409381 2409380 2026-05-05T09:14:24Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2213', diff saved to https://phabricator.wikimedia.org/P92262 and previous config saved to /var/cache/conftool/dbconfig/20260505-091423-fceratto.json 2409381 wikitext text/x-wiki 3o4m1t7hhw34i1aawmvtincbuul2ua0 2409382 2409381 2026-05-05T09:18:01Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2206.codfw.wmnet with reason: Maintenance 2409382 wikitext text/x-wiki ozvxb2268s23jyi9z2br9c7fk3juubg 2409383 2409382 2026-05-05T09:18:09Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db2206 (T419961)', diff saved to https://phabricator.wikimedia.org/P92263 and previous config saved to /var/cache/conftool/dbconfig/20260505-091808-fceratto.json 2409383 wikitext text/x-wiki 8dfoaea8v75zb5dbjatjnpwl44blcgy 2409384 2409383 2026-05-05T09:24:34Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2213 (T419635)', diff saved to https://phabricator.wikimedia.org/P92264 and previous config saved to /var/cache/conftool/dbconfig/20260505-092431-fceratto.json 2409384 wikitext text/x-wiki sc1hpvko2q7egiwfehsghtr0j1h5guh 2409385 2409384 2026-05-05T09:25:15Z Stashbot 7414 jelto@deploy1003: helmfile [staging] START helmfile.d/services/miscweb: apply 2409385 wikitext text/x-wiki dgubtw92o2wrcda3bdg55yiso1qoqes 2409386 2409385 2026-05-05T09:26:24Z Stashbot 7414 jelto@deploy1003: helmfile [staging] DONE helmfile.d/services/miscweb: apply 2409386 wikitext text/x-wiki mtgx55br349mntuoaahlxpaq4bzvi1z 2409387 2409386 2026-05-05T09:26:56Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2206 (T419961)', diff saved to https://phabricator.wikimedia.org/P92265 and previous config saved to /var/cache/conftool/dbconfig/20260505-092654-fceratto.json 2409387 wikitext text/x-wiki rjodq4jezxvyw9msfq3b6chu6n10n7u 2409388 2409387 2026-05-05T09:28:49Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2221.codfw.wmnet with reason: Reimage to Trixie 2409388 wikitext text/x-wiki 06v3663vxo2ljphu21ppr15m62n8wb7 2409389 2409388 2026-05-05T09:28:50Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . 2409389 wikitext text/x-wiki haig8fo9bmzmwy1w5i3a4k7q0i35tzs 2409390 2409389 2026-05-05T09:28:53Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1174.eqiad.wmnet with reason: Reimage to Trixie 2409390 wikitext text/x-wiki l6wllucd5214gqupe5mpmhxn97uqj3y 2409391 2409390 2026-05-05T09:28:54Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2221: Reimage to Trixie 2409391 wikitext text/x-wiki awgubm1ud6u49ynze5j7vf50vw3k65l 2409392 2409391 2026-05-05T09:28:58Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1174: Reimage to Trixie 2409392 wikitext text/x-wiki 3ol56qs9i7bhulhc73vurqjmbmlrckz 2409393 2409392 2026-05-05T09:29:10Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/services/miscweb: apply 2409393 wikitext text/x-wiki n6hbxu655zsui1gxpje0ieno5a2ofdq 2409394 2409393 2026-05-05T09:29:13Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2221: Reimage to Trixie 2409394 wikitext text/x-wiki beubbgiu6vycenwc9os6azujrh61oyv 2409395 2409394 2026-05-05T09:29:36Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1174: Reimage to Trixie 2409395 wikitext text/x-wiki azrzdb24wgzoy9b082bg4pnbjgowxei 2409396 2409395 2026-05-05T09:29:45Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/services/miscweb: apply 2409396 wikitext text/x-wiki l9xis1lbcewjpv5xaj7kr69qwdrq8yq 2409397 2409396 2026-05-05T09:30:05Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/services/miscweb: apply 2409397 wikitext text/x-wiki 2d1e0ftdycw7iw4vfu712sf90kb1ztk 2409398 2409397 2026-05-05T09:30:24Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/services/miscweb: apply 2409398 wikitext text/x-wiki 5rygvrc01tlvisrc61rc7pqvfq8stv0 2409399 2409398 2026-05-05T09:30:25Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2221.codfw.wmnet with OS trixie 2409399 wikitext text/x-wiki gk3jbcdvhmn2uptzfs1p1udufd2e5wj 2409400 2409399 2026-05-05T09:30:41Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1174.eqiad.wmnet with OS trixie 2409400 wikitext text/x-wiki nwcmc9q0731u8pmbkv11yx715di2gyq 2409401 2409400 2026-05-05T09:30:42Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2213.codfw.wmnet with reason: Maintenance 2409401 wikitext text/x-wiki ghzgniy0boz1qtzojbxnh8i9g41a9mh 2409402 2409401 2026-05-05T09:32:58Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1173.eqiad.wmnet with reason: Maintenance 2409402 wikitext text/x-wiki qp3k5bf4oy15w9lwmnt5xx83xu1vqnm 2409403 2409402 2026-05-05T09:33:07Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1173 (T419635)', diff saved to https://phabricator.wikimedia.org/P92269 and previous config saved to /var/cache/conftool/dbconfig/20260505-093305-fceratto.json 2409403 wikitext text/x-wiki 3i8c1nrrtf5cri3jtz21gu0cvp3p9mi 2409405 2409403 2026-05-05T09:35:35Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . 2409405 wikitext text/x-wiki fi2qa4toxdupzlsh64a285f3c9ipm1j 2409406 2409405 2026-05-05T09:36:13Z Stashbot 7414 jelto@deploy1003: helmfile [eqiad] START helmfile.d/services/miscweb: apply 2409406 wikitext text/x-wiki 3s8fdjusp29ay7wirm35yj20wadvqut 2409407 2409406 2026-05-05T09:36:22Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1173 (T419635)', diff saved to https://phabricator.wikimedia.org/P92270 and previous config saved to /var/cache/conftool/dbconfig/20260505-093619-fceratto.json 2409407 wikitext text/x-wiki 7lornxtlf622h5qt2q538nrmn7ekbtz 2409408 2409407 2026-05-05T09:37:04Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2206', diff saved to https://phabricator.wikimedia.org/P92271 and previous config saved to /var/cache/conftool/dbconfig/20260505-093703-fceratto.json 2409408 wikitext text/x-wiki 6yqngyi5qf52qkx25im0oanbojoiehx 2409409 2409408 2026-05-05T09:38:10Z Stashbot 7414 jelto@deploy1003: helmfile [eqiad] DONE helmfile.d/services/miscweb: apply 2409409 wikitext text/x-wiki 162rdednqy1nqdrucdppkytaqnxd7k8 2409410 2409409 2026-05-05T09:39:13Z Stashbot 7414 jelto@deploy1003: helmfile [codfw] START helmfile.d/services/miscweb: apply 2409410 wikitext text/x-wiki m0905txevedqrm85dxu3r4lx423rpnm 2409412 2409410 2026-05-05T09:41:35Z Stashbot 7414 jelto@deploy1003: helmfile [codfw] DONE helmfile.d/services/miscweb: apply 2409412 wikitext text/x-wiki d5qh7dx8apz7dhacqiiqwqgx18tlakv 2409413 2409412 2026-05-05T09:42:04Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . 2409413 wikitext text/x-wiki asjunvwureh3bupgsitlmp8igib92j1 2409415 2409413 2026-05-05T10:14:36Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1174: after reimage to trixie 2409415 wikitext text/x-wiki as3hdcsd3udl75te45l5fzi1rhfnskv 2409416 2409415 2026-05-05T10:16:17Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2210', diff saved to https://phabricator.wikimedia.org/P92281 and previous config saved to /var/cache/conftool/dbconfig/20260505-101616-fceratto.json 2409416 wikitext text/x-wiki gbruwlibbap6egglsf8ka3wvbtiblgy 2409417 2409416 2026-05-05T10:17:31Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . 2409417 wikitext text/x-wiki k0x05ned51ncjlqobis7mjmdodl3wle 2409418 2409417 2026-05-05T10:19:55Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2221.codfw.wmnet with OS trixie 2409418 wikitext text/x-wiki hfm78hg44l07vzik5q8t8sjpq4sueba 2409420 2409418 2026-05-05T10:22:37Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . 2409420 wikitext text/x-wiki 7bsnf1mwocfag3sb8cqbz9hoh3921b7 2409421 2409420 2026-05-05T10:23:26Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/services/miscweb: apply 2409421 wikitext text/x-wiki bjggmyrq4yfe1w4pl5o2fsvqvcdos1e 2409422 2409421 2026-05-05T10:23:28Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . 2409422 wikitext text/x-wiki lma78evhcw4ths0mdq0qizlryaa831h 2409423 2409422 2026-05-05T10:23:53Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/services/miscweb: apply 2409423 wikitext text/x-wiki jzxcmfqhccesc1ppn882s1in20i4jyg 2409424 2409423 2026-05-05T10:24:04Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/services/miscweb: apply 2409424 wikitext text/x-wiki 4kwqqquj4q65jne80sdn1zl1737wlcl 2409425 2409424 2026-05-05T10:24:04Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2221: after reimage to trixie 2409425 wikitext text/x-wiki r7mug9iv3skpkt4hw8a484vej82jijb 2409426 2409425 2026-05-05T10:24:48Z Stashbot 7414 jelto@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/services/miscweb: apply 2409426 wikitext text/x-wiki l8xm50smck58aq5g07pz7e4a7u5bcsq 2409427 2409426 2026-05-05T10:26:25Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2210', diff saved to https://phabricator.wikimedia.org/P92283 and previous config saved to /var/cache/conftool/dbconfig/20260505-102623-fceratto.json 2409427 wikitext text/x-wiki 19f6c7znpc5ifmi34xc5fm4tvn0ij0g 2409428 2409427 2026-05-05T10:29:16Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' . 2409428 wikitext text/x-wiki 4x1hpcyxgexqr4mu95frw9ggllnw0xy 2409429 2409428 2026-05-05T10:32:48Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' . 2409429 wikitext text/x-wiki cvnb0dp995wtj2w8j16ae38g826acgo 2409430 2409429 2026-05-05T10:36:33Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2210 (T419961)', diff saved to https://phabricator.wikimedia.org/P92285 and previous config saved to /var/cache/conftool/dbconfig/20260505-103632-fceratto.json 2409430 wikitext text/x-wiki 3lsymakixukfoy8sbjwldj400h4dron 2409431 2409430 2026-05-05T10:36:55Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2219.codfw.wmnet with reason: Maintenance 2409431 wikitext text/x-wiki jnwh29iiaiwjb9z62oetnww3t13fx4d 2409432 2409431 2026-05-05T10:37:03Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db2219 (T419961)', diff saved to https://phabricator.wikimedia.org/P92286 and previous config saved to /var/cache/conftool/dbconfig/20260505-103702-fceratto.json 2409432 wikitext text/x-wiki 6c407o4yshlh042hozoz9ry891lza6t 2409433 2409432 2026-05-05T10:40:25Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1227.eqiad.wmnet with reason: Maintenance 2409433 wikitext text/x-wiki t7losn4g0o1fv34q3sen1p0s1jhzguy 2409434 2409433 2026-05-05T10:40:35Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1227 (T419635)', diff saved to https://phabricator.wikimedia.org/P92288 and previous config saved to /var/cache/conftool/dbconfig/20260505-104032-fceratto.json 2409434 wikitext text/x-wiki 1e2edifm752elqcn987ikjy8wmdlwfh 2409436 2409434 2026-05-05T10:45:23Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2219 (T419961)', diff saved to https://phabricator.wikimedia.org/P92290 and previous config saved to /var/cache/conftool/dbconfig/20260505-104521-fceratto.json 2409436 wikitext text/x-wiki lsla2xjz8b4wtot0h4g11tc9flx1cd1 2409437 2409436 2026-05-05T10:49:41Z Stashbot 7414 elukey@deploy1003: helmfile [staging-codfw] START helmfile.d/admin 'sync'. 2409437 wikitext text/x-wiki o56d9mtwbrpv1tiss9k9xb88dwc0fli 2409438 2409437 2026-05-05T10:49:46Z Stashbot 7414 elukey@deploy1003: helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. 2409438 wikitext text/x-wiki 4rm0e4h1uh8rcffawu1k6d1zwllq1kc 2409439 2409438 2026-05-05T10:50:06Z Stashbot 7414 elukey@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'sync'. 2409439 wikitext text/x-wiki oiqv0hm2q2uqtsigblcgovl1tb18rh6 2409440 2409439 2026-05-05T10:50:09Z Stashbot 7414 elukey@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'sync'. 2409440 wikitext text/x-wiki 01hw3igl9zzkkrp9je97qvkyvp95xof 2409441 2409440 2026-05-05T10:50:37Z Stashbot 7414 elukey@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'sync'. 2409441 wikitext text/x-wiki 859z0rdesybpz25p6z9di00f1eks0yh 2409442 2409441 2026-05-05T10:50:37Z Stashbot 7414 elukey@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'sync'. 2409442 wikitext text/x-wiki mvuhpxz9nr02oqr63n06zen3vld9flt 2409443 2409442 2026-05-05T10:54:23Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1227 (T419635)', diff saved to https://phabricator.wikimedia.org/P92291 and previous config saved to /var/cache/conftool/dbconfig/20260505-105419-fceratto.json 2409443 wikitext text/x-wiki si9nvrmc5jna70rzzhf622ympdsa4ce 2409444 2409443 2026-05-05T10:55:30Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2219', diff saved to https://phabricator.wikimedia.org/P92293 and previous config saved to /var/cache/conftool/dbconfig/20260505-105529-fceratto.json 2409444 wikitext text/x-wiki n9n9aj8izhd2k6k0j0ccme74rvrk1os 2409445 2409444 2026-05-05T11:00:04Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1174: after reimage to trixie 2409445 wikitext text/x-wiki morjfp04sexvad42ghvq5dzj666l6iy 2409446 2409445 2026-05-05T11:04:28Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1227', diff saved to https://phabricator.wikimedia.org/P92295 and previous config saved to /var/cache/conftool/dbconfig/20260505-110427-fceratto.json 2409446 wikitext text/x-wiki ehiuxlvg1t7niga6jges4586iom1ykn 2409447 2409446 2026-05-05T11:05:24Z Stashbot 7414 ayounsi@cumin1003: START - Cookbook sre.loadbalancer.admin config_reloading P{lvs4009*} and A:liberica 2409447 wikitext text/x-wiki 3dm651ehbalpm0tncx140zguyfces6o 2409448 2409447 2026-05-05T11:05:32Z Stashbot 7414 ayounsi@cumin1003: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) config_reloading P{lvs4009*} and A:liberica 2409448 wikitext text/x-wiki 29qdw83oalivurjoghqzewlmczb2adi 2409449 2409448 2026-05-05T11:05:38Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2219', diff saved to https://phabricator.wikimedia.org/P92296 and previous config saved to /var/cache/conftool/dbconfig/20260505-110537-fceratto.json 2409449 wikitext text/x-wiki 9pat9y1x4mqnvvvdzbcugfkmmt4mazw 2409450 2409449 2026-05-05T11:07:24Z Stashbot 7414 moritzm: installing multipart bugfix updates from bookworm point release 2409450 wikitext text/x-wiki nx3wv578s8vpcwvmkav7juzx8kew20r 2409451 2409450 2026-05-05T11:09:30Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2221: after reimage to trixie 2409451 wikitext text/x-wiki qt6klc5u4hj85q5qfs74243tdw9oetm 2409452 2409451 2026-05-05T11:10:35Z Stashbot 7414 moritzm: installing ca-certificates updates from bookworm point release 2409452 wikitext text/x-wiki hqlwt9cp6iqjjkpt0k83xb003aoce41 2409453 2409452 2026-05-05T11:14:36Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1227', diff saved to https://phabricator.wikimedia.org/P92298 and previous config saved to /var/cache/conftool/dbconfig/20260505-111435-fceratto.json 2409453 wikitext text/x-wiki h6i20fdv3hm9l1pftf6ocli3xnpw293 2409454 2409453 2026-05-05T11:15:47Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2219 (T419961)', diff saved to https://phabricator.wikimedia.org/P92299 and previous config saved to /var/cache/conftool/dbconfig/20260505-111545-fceratto.json 2409454 wikitext text/x-wiki 079dcqf2j4595nyrbqhqd0widusaif4 2409455 2409454 2026-05-05T11:16:09Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2236.codfw.wmnet with reason: Maintenance 2409455 wikitext text/x-wiki 79zzp960h80pvavpk9uvpp99m7h3w6g 2409456 2409455 2026-05-05T11:16:17Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db2236 (T419961)', diff saved to https://phabricator.wikimedia.org/P92300 and previous config saved to /var/cache/conftool/dbconfig/20260505-111616-fceratto.json 2409456 wikitext text/x-wiki 3wdcp0bq8csfdzoi5zrqihcwf6p0yei 2409457 2409456 2026-05-05T11:24:40Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2236 (T419961)', diff saved to https://phabricator.wikimedia.org/P92301 and previous config saved to /var/cache/conftool/dbconfig/20260505-112438-fceratto.json 2409457 wikitext text/x-wiki khdsrujhwkfy4y6qzanzrungvk640p6 2409458 2409457 2026-05-05T11:24:52Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1227 (T419635)', diff saved to https://phabricator.wikimedia.org/P92302 and previous config saved to /var/cache/conftool/dbconfig/20260505-112449-fceratto.json 2409458 wikitext text/x-wiki bydrasj4s6kxl2pjb8hj8er5ksua4b6 2409459 2409458 2026-05-05T11:34:48Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2236', diff saved to https://phabricator.wikimedia.org/P92303 and previous config saved to /var/cache/conftool/dbconfig/20260505-113446-fceratto.json 2409459 wikitext text/x-wiki p2jw01lp56c3zeqk2g6296vxdb9bh00 2409460 2409459 2026-05-05T11:38:15Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1282955|Close Norwegian Wikinews (T421796)]] 2409460 wikitext text/x-wiki mgr1k18zhky1qe5pty7jzpqebsa05g2 2409461 2409460 2026-05-05T11:38:23Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.reboot-vm for VM aux-k8s-etcd2003.codfw.wmnet 2409461 wikitext text/x-wiki au4njmmh4d45opl7hvuvdft54sg2z1b 2409462 2409461 2026-05-05T11:39:56Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1282955|Close Norwegian Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409462 wikitext text/x-wiki 7rg8gp3zpmto42jw1sk849a759l0qqf 2409463 2409462 2026-05-05T11:42:26Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.reboot-vm (exit_code=0) for VM aux-k8s-etcd2003.codfw.wmnet 2409463 wikitext text/x-wiki coul3ukex9j57jrzfdzwoiwt6tl8wid 2409464 2409463 2026-05-05T11:43:31Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.reboot-vm for VM aux-k8s-etcd2004.codfw.wmnet 2409464 wikitext text/x-wiki mprplojv67wo0fbydzonyegzzhmxjo0 2409465 2409464 2026-05-05T11:43:31Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2409465 wikitext text/x-wiki iueuc2dq1g8x5wcoukjva1n3hz35cv6 2409466 2409465 2026-05-05T11:44:56Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2236', diff saved to https://phabricator.wikimedia.org/P92304 and previous config saved to /var/cache/conftool/dbconfig/20260505-114455-fceratto.json 2409466 wikitext text/x-wiki tn362nrwdjv4jaghlesaubvh4to81d3 2409467 2409466 2026-05-05T11:47:25Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.reboot-vm (exit_code=0) for VM aux-k8s-etcd2004.codfw.wmnet 2409467 wikitext text/x-wiki 6rl40w7vuwxdtvz3wockisd8502wwal 2409468 2409467 2026-05-05T11:47:36Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1282955|Close Norwegian Wikinews (T421796)]] (duration: 09m 21s) 2409468 wikitext text/x-wiki la6hvzfraz6w70h7gtug3jvv82ioypd 2409469 2409468 2026-05-05T11:51:20Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1282960|Close Shan Wikinews (T421796)]] 2409469 wikitext text/x-wiki powiey6140xfa507rgat11t1blw8g4z 2409470 2409469 2026-05-05T11:52:01Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.reboot-vm for VM aux-k8s-etcd2005.codfw.wmnet 2409470 wikitext text/x-wiki d8d0g1vekmfezubwgc4en25vhh0dez2 2409471 2409470 2026-05-05T11:53:01Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1282960|Close Shan Wikinews (T421796)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2409471 wikitext text/x-wiki oov6g8m0j4diet7ntri2bef227grzcd 2409472 2409471 2026-05-05T11:53:28Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2409472 wikitext text/x-wiki e6ty3kkt9qsug8uwmjvxmvdnzw1bbj3 2409473 2409472 2026-05-05T11:55:05Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db2236 (T419961)', diff saved to https://phabricator.wikimedia.org/P92305 and previous config saved to /var/cache/conftool/dbconfig/20260505-115503-fceratto.json 2409473 wikitext text/x-wiki m0nv3l40e7ybywy3xeeoyfm41v70y2o 2409474 2409473 2026-05-05T11:55:28Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2237.codfw.wmnet with reason: Maintenance 2409474 wikitext text/x-wiki ofj28o5dhj20p7w5v196jb899te708t 2409475 2409474 2026-05-05T11:55:36Z Stashbot 7414 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db2237 (T419961)', diff saved to https://phabricator.wikimedia.org/P92306 and previous config saved to /var/cache/conftool/dbconfig/20260505-115535-fceratto.json 2409475 wikitext text/x-wiki 72fetyq58e56e7cdcmsaboa4avik0n2 2409476 2409475 2026-05-05T11:56:01Z Stashbot 7414 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.reboot-vm (exit_code=0) for VM aux-k8s-etcd2005.codfw.wmnet 2409476 wikitext text/x-wiki iq1gxzagg0f9t30htfikspnomv5s0c9 2409477 2409476 2026-05-05T11:57:34Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1282960|Close Shan Wikinews (T421796)]] (duration: 06m 13s) 2409477 wikitext text/x-wiki khgfj22jw8hezvzbb6dfwu4gfsmzfp5 0 17290 2409097 2408984 2026-05-04T14:51:29Z Stashbot 7414 dancy: Upgrading gitlab cloud runners (staging) from 1.35.1-do.3 to 1.35.1-do.5 2409097 wikitext text/x-wiki 9bnamdzh3xaxhz846t1se8r4lizm24z 2409123 2409097 2026-05-04T15:19:43Z Stashbot 7414 dancy: Upgrading gitlab cloud runners (prod) from 1.35.1-do.3 to 1.35.1-do.5 2409123 wikitext text/x-wiki 907wdo8nvar0vk6p16tfrazdjzzh3e9 498 30942 2409419 2408657 2026-05-05T10:20:34Z Stashbot 7414 taavi: taavi@cloudcontrol1007 ~ $ sudo wmcs-enc-cli --openstack-project admin delete_project wikilabels # T416588 2409419 wikitext text/x-wiki 6l930r4ii9u6bf8q56e2l62kahav8e8 498 73236 2409156 2371256 2026-05-04T17:45:44Z Stashbot 7414 wmbot~anticomposite@tools-bastion-15: stewardbots/StewardBot/manage.sh restart # Disconnected 2409156 wikitext text/x-wiki e32zdy6qi6w2vt94180etxsl8iv211k 0 447017 2409217 2382274 2026-05-04T21:52:56Z RLazarus (WMF) 15215 2409217 wikitext text/x-wiki dpcfh5tt0om6dmnz9qijqmundwfnz64 0 449160 2409090 2408919 2026-05-04T14:34:20Z Dexbot 30554 Bot: Updating the report 2409090 wikitext text/x-wiki loma93vj0chaw3pzfrsvfr0tj1pramr 2409228 2409090 2026-05-05T00:02:27Z Dexbot 30554 Bot: Updating the report 2409228 wikitext text/x-wiki fvyq8kfvn26ywoef58isl9z179myitg 2409312 2409228 2026-05-05T07:54:24Z Dexbot 30554 Bot: Updating the report 2409312 wikitext text/x-wiki 12nos048nevdiui8lloqpunljpfjy9b 2409365 2409312 2026-05-05T08:48:32Z Dexbot 30554 Bot: Updating the report 2409365 wikitext text/x-wiki rqc8fzlwuhkr7etqoiasenkec9udfk7 2409404 2409365 2026-05-05T09:33:38Z Dexbot 30554 Bot: Updating the report 2409404 wikitext text/x-wiki 0fgzermg4k1mmdti811z8126vn2l5v9 2409435 2409404 2026-05-05T10:42:46Z Dexbot 30554 Bot: Updating the report 2409435 wikitext text/x-wiki qy3busihglicrev1mzp7ew7a39cwy81 0 451475 2409215 2314904 2026-05-04T21:43:04Z RLazarus (WMF) 15215 belatedly changing "draft" to "approved" following prior discussion 2409215 wikitext text/x-wiki 4dusisdce1e1xi9021ourx3fsa4pxzr 0 455928 2409101 2408017 2026-05-04T14:54:16Z SFaci-WMF 35637 /* Experiments */ 2409101 wikitext text/x-wiki hxcv590yk4qc9uplldce47rnw02z4pz 0 455989 2409096 2406929 2026-05-04T14:51:14Z SFaci-WMF 35637 /* Instrumentation */ 2409096 wikitext text/x-wiki d6stnrqceldd4mlrl9i6rgkyzgx5vp0 2409098 2409096 2026-05-04T14:51:43Z SFaci-WMF 35637 /* Instrumentation */ 2409098 wikitext text/x-wiki ibnalfyw1q74mgy31roe4d48uacydjm 2409099 2409098 2026-05-04T14:52:11Z SFaci-WMF 35637 /* Stream configuration */ 2409099 wikitext text/x-wiki 7fev6jyfpyj9148wozvjohglmtfvxjv 2409174 2409099 2026-05-04T18:59:07Z KReid-WMF 49588 /* Experiment design: user traffic per wiki */ Adds clarification that the sampling rate note only applies if the experiment has been started. 2409174 wikitext text/x-wiki em061hmepwquo0i9f4m8z0407ksbuiv 0 456039 2409102 2384860 2026-05-04T14:55:39Z SFaci-WMF 35637 /* Stream configuration */ 2409102 wikitext text/x-wiki 52oc4qtc4abno590yz0d5wcazzqr1lf 0 459699 2409154 2408818 2026-05-04T17:39:40Z Stashbot 7414 ejegg: standalone (IPN listener) SmashPig upgraded from 1be60746 to 4201ef56 2409154 wikitext text/x-wiki ikru5vkoljiq3y0cen8x6ao5ly23k90 0 459743 2409207 2408384 2026-05-04T20:45:34Z LErickson-WMF 51715 2409207 wikitext text/x-wiki 6om11werfbnaafzz0ndlq5cfxsbx5gs 2 460122 2409036 2408346 2026-05-04T13:12:20Z GModena (WMF) 20194 2409036 wikitext text/x-wiki kx27xlv2mny25myqm1k1txvzs1p8vy9 2409038 2409036 2026-05-04T13:15:28Z GModena (WMF) 20194 2409038 wikitext text/x-wiki d8ibfhltmhwc9gmxqpaee0svcfndurj 3 460136 2409149 2026-05-04T16:45:15Z StrikerBot 8475 Welcome to Toolforge! 2409149 wikitext text/x-wiki ke066zbj7yrw0hzj2ccpp192g2k56ia 0 460137 2409206 2026-05-04T20:41:40Z LErickson-WMF 51715 Created page with "The Wikidata Platform team populates several tables with data about queries processed by WDQS. == Individual queries == * [https://datahub.wikimedia.org/dataset/urn:li:dataset:(urn:li:dataPlatform:hive,event.wdqs_external_sparql_query,PROD)/Schema?is_lineage_mode=false&schemaFilter= event.wdqs_external_sparql_query]: raw logs for all queries to WDQS's external endpoints * [http://event.wdqs_internal_sparql_queryhttps://datahub.wikimedia.org/dataset/urn:li:dataset:(urn:..." 2409206 wikitext text/x-wiki qgz2oosjwryu2d4axtn1136pymbx5sx