Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.20 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 2161 2458392 2454443 2026-09-17T12:46:19Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'proxy' 2458392 wikitext text/x-wiki im4ndwhk15r8myj7whvr8nmbc0th1gn 2458393 2458392 2026-09-17T12:52:04Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'proxy' 2458393 wikitext text/x-wiki syov2d6kgnub77rwfytfp97sdyd758a 2458395 2458393 2026-09-17T12:53:57Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'proxy' 2458395 wikitext text/x-wiki eo45tusf3i39nf7q5sxcp682hi3m9gj 2458396 2458395 2026-09-17T12:54:04Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on proxy-7.project-proxy.eqiad1.wikimedia.cloud 2458396 wikitext text/x-wiki lvlsfgsuwfxsbatcascx6m4qnokyx8c 2458398 2458396 2026-09-17T12:56:39Z Stashbot 7414 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on proxy-7.project-proxy.eqiad1.wikimedia.cloud 2458398 wikitext text/x-wiki mpmarmhm23vk9g286xbc80yva1r23ao 2458400 2458398 2026-09-17T13:02:00Z Stashbot 7414 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=99) with prefix 'proxy' 2458400 wikitext text/x-wiki goffng80w37jn341h4y3om6czxy9kqo 0 4108 2458421 2458303 2026-09-17T13:36:16Z ScheduleDeploymentBot 37566 Add [[gerrit:1342678]] to Thursday, September 17 UTC afternoon backport window 2458421 wikitext text/x-wiki 0dztstipmtc7e9i03taqlzy548eknvz 2458422 2458421 2026-09-17T13:36:26Z ScheduleDeploymentBot 37566 Add [[gerrit:1342362]] to Thursday, September 17 UTC afternoon backport window 2458422 wikitext text/x-wiki 6wr6mk0h67q3qu7fjrmed9xla16ok7b 2458444 2458422 2026-09-17T14:42:04Z Martin Urbanec (WMF) 37730 +1342224 2458444 wikitext text/x-wiki gm3n3u9mqepn0tiw8j5h2gmxu7fyrw2 2458549 2458444 2026-09-17T18:06:53Z ScheduleDeploymentBot 37566 Add [[gerrit:1342774]] to Thursday, September 17 UTC late backport window 2458549 wikitext text/x-wiki fmhiipjyruzf47ys1recnq4te6wyms2 2458553 2458549 2026-09-17T18:46:05Z ScheduleDeploymentBot 37566 Add [[gerrit:1342781]] to Thursday, September 17 UTC late backport window 2458553 wikitext text/x-wiki 78v9588tptt4857drrxn7n6l6cy2jcl 2458561 2458553 2026-09-17T19:55:47Z ScheduleDeploymentBot 37566 Add [[gerrit:1342798]] to Thursday, September 17 UTC late backport window 2458561 wikitext text/x-wiki 51jmcuv3y81sos7w6hgszt5syw29sve 2458589 2458561 2026-09-18T07:47:30Z ScheduleDeploymentBot 37566 Add [[gerrit:1342825]] to Monday, September 21 UTC afternoon backport window 2458589 wikitext text/x-wiki 9pb3s19vji5pqn1jlmpa2opmiee4z9q 0 7919 2458377 2458368 2026-09-17T12:05:59Z Stashbot 7414 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1342626|MathMathML: Simplify Mathoid fallback/a11y class logic (T436026)]], [[gerrit:1342627|ext.math.mathjax: Implement mwe-math-mathml-a11y for client-side MathJax (T436026)]] 2458377 wikitext text/x-wiki qvlqsvverko4q8jiqtmbisgp7tkf5xx 2458378 2458377 2026-09-17T12:10:35Z Stashbot 7414 krinkle@deploy1003: krinkle: Backport for [[gerrit:1342626|MathMathML: Simplify Mathoid fallback/a11y class logic (T436026)]], [[gerrit:1342627|ext.math.mathjax: Implement mwe-math-mathml-a11y for client-side MathJax (T436026)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2458378 wikitext text/x-wiki 1mp9higv01dupvsymuzqg7b4wu4ujlt 2458379 2458378 2026-09-17T12:14:31Z Stashbot 7414 krinkle@deploy1003: krinkle: Continuing with deployment 2458379 wikitext text/x-wiki 99wn5bvx57wy6majmcp5q201xo6qlc5 2458381 2458379 2026-09-17T12:19:04Z Stashbot 7414 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1342626|MathMathML: Simplify Mathoid fallback/a11y class logic (T436026)]], [[gerrit:1342627|ext.math.mathjax: Implement mwe-math-mathml-a11y for client-side MathJax (T436026)]] (duration: 13m 04s) 2458381 wikitext text/x-wiki pcha3f1093fhb48x1wlfp8cflstnxko 2458383 2458381 2026-09-17T12:35:42Z Stashbot 7414 ozge@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . 2458383 wikitext text/x-wiki g4snyqxf65mn1pyaophl2mdds0z9y7a 2458385 2458383 2026-09-17T12:40:58Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2458385 wikitext text/x-wiki h0arw0nt0jaqfztjgc49cag5kg4lmhv 2458386 2458385 2026-09-17T12:41:00Z Stashbot 7414 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2458386 wikitext text/x-wiki qmjfd3mow3e3m5kc5c56wcrbcsu4vm5 2458388 2458386 2026-09-17T12:42:33Z Stashbot 7414 jclark@cumin1004: START - Cookbook sre.hosts.provision for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2458388 wikitext text/x-wiki qzkd775133q8cz7mlwn5dc2il84jtzy 2458394 2458388 2026-09-17T12:53:41Z Stashbot 7414 ozge@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . 2458394 wikitext text/x-wiki iktd2igf7r5tgnalaiduindd6g1w78n 2458397 2458394 2026-09-17T12:55:49Z Stashbot 7414 jclark@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2458397 wikitext text/x-wiki 1n7ukye7h9ser0bcwwksj4qdt6bcmgw 2458403 2458397 2026-09-17T13:09:56Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2458403 wikitext text/x-wiki nbs20ysg9lyq8qr79g4s6iahscg4eou 2458404 2458403 2026-09-17T13:09:58Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2458404 wikitext text/x-wiki pkxwihhubngghh5u5wgwek05ramq18q 2458408 2458404 2026-09-17T13:11:55Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2458408 wikitext text/x-wiki qpdy4imqs4sl14d1arruenttd49v3e9 2458409 2458408 2026-09-17T13:11:58Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2458409 wikitext text/x-wiki 2wx0e4of9cmhsgrnvd4l3e1bat1fhqx 2458413 2458409 2026-09-17T13:24:40Z Stashbot 7414 jclark@cumin1004: START - Cookbook sre.network.configure-switch-interfaces for host ml-serve1016 2458413 wikitext text/x-wiki av2u5sjydxxqljcktt70wfokxmzqy25 2458414 2458413 2026-09-17T13:24:49Z Stashbot 7414 jclark@cumin1004: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ml-serve1016 2458414 wikitext text/x-wiki tnqi4d19ggaw6lq7heiv54kxop13o2d 2458415 2458414 2026-09-17T13:29:39Z Stashbot 7414 Emperor: apus - radosgw-admin quota set --quota-scope=user --uid=docker-registry --max-size=5T T438339 2458415 wikitext text/x-wiki cuc5luqcdto17y27zurz3f16onf8032 2458416 2458415 2026-09-17T13:30:52Z Stashbot 7414 elukey@cumin1004: START - Cookbook sre.hosts.provision for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2458416 wikitext text/x-wiki soblev0nemo75d7clx4y9xkojeinepr 2458417 2458416 2026-09-17T13:31:01Z Stashbot 7414 elukey@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2458417 wikitext text/x-wiki 2pm5h16lenqt5ubkrer3k0fttmqxvs8 2458419 2458417 2026-09-17T13:33:55Z Stashbot 7414 elukey@cumin1004: START - Cookbook sre.hosts.provision for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2458419 wikitext text/x-wiki 70gf1lktws66g5n9idhbop16qa5x65g 2458420 2458419 2026-09-17T13:34:02Z Stashbot 7414 elukey@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2458420 wikitext text/x-wiki cls3utk9cz7l7m62aqaqhe2o95adplk 2458423 2458420 2026-09-17T13:37:46Z Stashbot 7414 ozge@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . 2458423 wikitext text/x-wiki 368fs15g7addjnnkfnypq4zqstaaatd 2458424 2458423 2026-09-17T13:46:09Z Stashbot 7414 caro@deploy1003: Started scap sync-world: Backport for [[gerrit:1342678|enwiki desktop VE: add education popup for switching to source editor (T434249)]], [[gerrit:1342362|Make VE the default editor on enwiki desktop (T436574)]] 2458424 wikitext text/x-wiki 3vnh090o3ea63g4rzbrry3oz7p44a32 2458425 2458424 2026-09-17T14:06:23Z Stashbot 7414 caro@deploy1003: caro: Backport for [[gerrit:1342678|enwiki desktop VE: add education popup for switching to source editor (T434249)]], [[gerrit:1342362|Make VE the default editor on enwiki desktop (T436574)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2458425 wikitext text/x-wiki g5hrwgm3mgyaxloxp4s3xxypa22mgai 2458427 2458425 2026-09-17T14:07:50Z Stashbot 7414 caro@deploy1003: caro: Continuing with deployment 2458427 wikitext text/x-wiki lluo9xct7qkmvz9wsiq0rh91tu4djyu 2458431 2458427 2026-09-17T14:20:01Z Stashbot 7414 caro@deploy1003: Finished scap sync-world: Backport for [[gerrit:1342678|enwiki desktop VE: add education popup for switching to source editor (T434249)]], [[gerrit:1342362|Make VE the default editor on enwiki desktop (T436574)]] (duration: 33m 52s) 2458431 wikitext text/x-wiki kaozy830dbbcvqsw5gmqfe6eclwn2oz 2458436 2458431 2026-09-17T14:26:54Z Stashbot 7414 cscott@deploy1003: Started scap sync-world: Backport for [[gerrit:1331830|Keep Balinese Palm Leaf variants enabled on wikisource (T436398)]], [[gerrit:1340216|Turn on variant conversion for PageAssessments (T328012)]], [[gerrit:1341949|Parsoid Read Views: Enable on 61 wikiquote wikis (T437917)]] 2458436 wikitext text/x-wiki ktzgori1h5q1ca010ltrqlpgebzygel 2458438 2458436 2026-09-17T14:32:59Z Stashbot 7414 cscott@deploy1003: ssastry, cscott: Backport for [[gerrit:1331830|Keep Balinese Palm Leaf variants enabled on wikisource (T436398)]], [[gerrit:1340216|Turn on variant conversion for PageAssessments (T328012)]], [[gerrit:1341949|Parsoid Read Views: Enable on 61 wikiquote wikis (T437917)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2458438 wikitext text/x-wiki jx5h13zywj6uelb3gvuj7wzt0k1507g 2458439 2458438 2026-09-17T14:33:41Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.reimage for host ncredir4003.ulsfo.wmnet with OS trixie 2458439 wikitext text/x-wiki prvco1u35dp4kq38uycsncuq82z6chv 2458441 2458439 2026-09-17T14:35:44Z Stashbot 7414 cscott@deploy1003: ssastry, cscott: Continuing with deployment 2458441 wikitext text/x-wiki qlls5dmy5xl0xfwe17rl57tsf94x5r5 2458443 2458441 2026-09-17T14:39:44Z Stashbot 7414 elukey@cumin1004: START - Cookbook sre.hosts.reimage for host registry1005.eqiad.wmnet with OS trixie 2458443 wikitext text/x-wiki 4q624vazia6ph4t7tmvmjauog0gze1l 2458445 2458443 2026-09-17T14:42:25Z Stashbot 7414 cscott@deploy1003: Finished scap sync-world: Backport for [[gerrit:1331830|Keep Balinese Palm Leaf variants enabled on wikisource (T436398)]], [[gerrit:1340216|Turn on variant conversion for PageAssessments (T328012)]], [[gerrit:1341949|Parsoid Read Views: Enable on 61 wikiquote wikis (T437917)]] (duration: 15m 31s) 2458445 wikitext text/x-wiki 84jl5ve7062tqi647dcwt3x68xgreox 2458446 2458445 2026-09-17T14:47:53Z Stashbot 7414 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1342265|JsonSchemaBuilder: Cache the root schema in the process (T437588)]], [[gerrit:1342264|JsonSchemaBuilder: Cache the root schema in the process (T437588)]], [[gerrit:1342684|SI: Preserve the username filter when switching queues (T438308)]] 2458446 wikitext text/x-wiki l9r93rmzho5b7izhq9qiocx5sq4hoqn 2458447 2458446 2026-09-17T14:50:14Z Stashbot 7414 aokoth@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on phab2003.codfw.wmnet with reason: Reimage 2458447 wikitext text/x-wiki kfacbyn82x4k849dmo6owopj1s4x864 2458448 2458447 2026-09-17T14:51:35Z Stashbot 7414 aokoth@cumin1004: START - Cookbook sre.hosts.reimage for host phab2003.codfw.wmnet with OS trixie 2458448 wikitext text/x-wiki jxtpy0evz51q0y2rgbt426edqec5len 2458449 2458448 2026-09-17T14:51:50Z Stashbot 7414 urbanecm@deploy1003: mszwarc, urbanecm: Backport for [[gerrit:1342265|JsonSchemaBuilder: Cache the root schema in the process (T437588)]], [[gerrit:1342264|JsonSchemaBuilder: Cache the root schema in the process (T437588)]], [[gerrit:1342684|SI: Preserve the username filter when switching queues (T438308)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2458449 wikitext text/x-wiki 8a0ujm7w2p3c277mpafhj3cx1ii5upu 2458450 2458449 2026-09-17T14:55:55Z Stashbot 7414 urbanecm@deploy1003: mszwarc, urbanecm: Continuing with deployment 2458450 wikitext text/x-wiki k0eas0rzgghsjflnjlqi0mclmewxxkr 2458451 2458450 2026-09-17T14:58:00Z Stashbot 7414 elukey@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on registry1005.eqiad.wmnet with reason: host reimage 2458451 wikitext text/x-wiki e7avm0mrfxtf33wrzhfmrn0m8dd45r2 2458452 2458451 2026-09-17T14:58:48Z Stashbot 7414 cdobbins@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ncredir4003.ulsfo.wmnet with reason: host reimage 2458452 wikitext text/x-wiki 1llakfy0168xpgu6j05k9elvsue9ix4 2458453 2458452 2026-09-17T15:00:08Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.ganeti.makevm for new host ceph-admin1001.eqiad.wmnet 2458453 wikitext text/x-wiki j9fkzjozsyfb60ten0371xdgf0qhee2 2458454 2458453 2026-09-17T15:00:10Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.dns.netbox 2458454 wikitext text/x-wiki 2sen8qa7v6vbhtmsru9gmyac4skmx8e 2458456 2458454 2026-09-17T15:00:27Z Stashbot 7414 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1342265|JsonSchemaBuilder: Cache the root schema in the process (T437588)]], [[gerrit:1342264|JsonSchemaBuilder: Cache the root schema in the process (T437588)]], [[gerrit:1342684|SI: Preserve the username filter when switching queues (T438308)]] (duration: 12m 34s) 2458456 wikitext text/x-wiki dffblwxfcilk33nxra8vxu0msaj92c5 2458457 2458456 2026-09-17T15:01:10Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.ganeti.makevm for new host ceph-admin2001.codfw.wmnet 2458457 wikitext text/x-wiki 8ghx4hrwm91nhc7f8zhr556r18y1d50 2458458 2458457 2026-09-17T15:02:12Z Stashbot 7414 elukey@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on registry1005.eqiad.wmnet with reason: host reimage 2458458 wikitext text/x-wiki 943p4ga7ys9nqff3h7f10p20p8igdws 2458459 2458458 2026-09-17T15:04:13Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ceph-admin1001.eqiad.wmnet - btullis@cumin1004" 2458459 wikitext text/x-wiki mvrzhrdau5xzojbom9dw84hlj7ryl01 2458460 2458459 2026-09-17T15:04:17Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ceph-admin1001.eqiad.wmnet - btullis@cumin1004" 2458460 wikitext text/x-wiki jj18qwpgdpmts43cndcjlfdxc6emetx 2458461 2458460 2026-09-17T15:04:18Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2458461 wikitext text/x-wiki 75tfcunfnsy1sst48fual0m2qgktidu 2458462 2458461 2026-09-17T15:04:19Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.dns.wipe-cache ceph-admin1001.eqiad.wmnet on all recursors 2458462 wikitext text/x-wiki r210wp8ywsrigphpn4ff0k2nnmw1k22 2458463 2458462 2026-09-17T15:04:20Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) ceph-admin1001.eqiad.wmnet on all recursors 2458463 wikitext text/x-wiki a7wppza7hhefmzu81i34ivgcgeyugtk 2458465 2458463 2026-09-17T15:04:49Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ceph-admin1001.eqiad.wmnet - btullis@cumin1004" 2458465 wikitext text/x-wiki tiuo9wve4ripvngd5w56f1t5fh1u4vv 2458466 2458465 2026-09-17T15:04:52Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ceph-admin1001.eqiad.wmnet - btullis@cumin1004" 2458466 wikitext text/x-wiki 0o13s8rz1hlk15fh0tjg7yldyook6j0 2458467 2458466 2026-09-17T15:04:57Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.dns.netbox 2458467 wikitext text/x-wiki f3o1dak2f2gyrcu9uco564kc5gp6ohh 2458468 2458467 2026-09-17T15:04:59Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reimage for host ceph-admin1001.eqiad.wmnet with OS bookworm 2458468 wikitext text/x-wiki 1kykcbc0930al67ipg6x5s4izxx4l8t 2458469 2458468 2026-09-17T15:05:15Z Stashbot 7414 aokoth@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on phab2003.codfw.wmnet with reason: host reimage 2458469 wikitext text/x-wiki euxcvm9cnu268ch6xym5jgz04bf6nql 2458470 2458469 2026-09-17T15:05:46Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ncredir4003.ulsfo.wmnet with reason: host reimage 2458470 wikitext text/x-wiki b06rs1xfb2w7e3vg17o70qzltyor9gl 2458471 2458470 2026-09-17T15:06:27Z Stashbot 7414 Msz2001: Deployed private code changes to Suggestedinvestigations 2458471 wikitext text/x-wiki 3ogyb62o57tzo9x6f4m8knt6b1c5z5h 2458472 2458471 2026-09-17T15:08:58Z Stashbot 7414 aokoth@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on phab2003.codfw.wmnet with reason: host reimage 2458472 wikitext text/x-wiki ropw7x15ba7y9nri3il23s2g1uo3qcc 2458474 2458472 2026-09-17T15:15:19Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ceph-admin2001.codfw.wmnet - btullis@cumin1004" 2458474 wikitext text/x-wiki tc7z5c697i6zofxmre0wb0oyyi427qg 2458475 2458474 2026-09-17T15:15:24Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ceph-admin2001.codfw.wmnet - btullis@cumin1004" 2458475 wikitext text/x-wiki lsea5l1lvdj1ncqv9x82rqqz8t2r71d 2458476 2458475 2026-09-17T15:15:25Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2458476 wikitext text/x-wiki 3r57zt4xv8b7a4k83h9dn3ps42py51q 2458477 2458476 2026-09-17T15:15:27Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.dns.wipe-cache ceph-admin2001.codfw.wmnet on all recursors 2458477 wikitext text/x-wiki bkqrkwyajs6dv121utyztarox8020na 2458478 2458477 2026-09-17T15:15:28Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) ceph-admin2001.codfw.wmnet on all recursors 2458478 wikitext text/x-wiki 2yfutjyj7gievsfvnyjltl65y85qmp7 2458479 2458478 2026-09-17T15:16:08Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ceph-admin2001.codfw.wmnet - btullis@cumin1004" 2458479 wikitext text/x-wiki 50nxop2quaejjnjzbhs0d8tundoy7ij 2458480 2458479 2026-09-17T15:16:10Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ceph-admin2001.codfw.wmnet - btullis@cumin1004" 2458480 wikitext text/x-wiki 6stb2uqlroxh3k2mlxmxe46ky0pixx6 2458481 2458480 2026-09-17T15:16:15Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reimage for host ceph-admin2001.codfw.wmnet with OS bookworm 2458481 wikitext text/x-wiki n74u9750c6lhjaq3mwgs1u8zdrefw35 2458482 2458481 2026-09-17T15:17:51Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ceph-admin1001.eqiad.wmnet with reason: host reimage 2458482 wikitext text/x-wiki peh4wi3ijhf3tf5285h8l0vxtkxsx9r 2458483 2458482 2026-09-17T15:19:02Z Stashbot 7414 elukey@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host registry1005.eqiad.wmnet with OS trixie 2458483 wikitext text/x-wiki ihiu2i33nufpkikuxhqcuqf72rakwyo 2458484 2458483 2026-09-17T15:23:33Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ceph-admin1001.eqiad.wmnet with reason: host reimage 2458484 wikitext text/x-wiki sybsou3trok8qshrbku5rrfbb5138p5 2458485 2458484 2026-09-17T15:25:50Z Stashbot 7414 aokoth@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host phab2003.codfw.wmnet with OS trixie 2458485 wikitext text/x-wiki gymo1zx9ytx2gusx6qvgt4y87m86a78 2458486 2458485 2026-09-17T15:26:47Z Stashbot 7414 cdobbins@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ncredir4003.ulsfo.wmnet with OS trixie 2458486 wikitext text/x-wiki 4uhurwr4fxqj0qvpbf640w3zdjmwowy 2458487 2458486 2026-09-17T15:37:56Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ceph-admin2001.codfw.wmnet with reason: host reimage 2458487 wikitext text/x-wiki 7iuy9kon2pygikqlmvolcsz9ma6kmg1 2458488 2458487 2026-09-17T15:39:45Z Stashbot 7414 cdobbins@cumin1004: conftool action : set/pooled=yes; selector: name=ncredir4003.* 2458488 wikitext text/x-wiki 8ud17g4oxenobjif2aygitaf8n0ypy1 2458489 2458488 2026-09-17T15:40:02Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ceph-admin1001.eqiad.wmnet with OS bookworm 2458489 wikitext text/x-wiki s3joyeg0xzrjmv0to5uf6ioplqj9zqg 2458490 2458489 2026-09-17T15:40:04Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host ceph-admin1001.eqiad.wmnet 2458490 wikitext text/x-wiki 3z5p013a43qoyjk4fbnpr2ciqeufbc2 2458491 2458490 2026-09-17T15:44:02Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ceph-admin2001.codfw.wmnet with reason: host reimage 2458491 wikitext text/x-wiki 8ak549cgkceboqfqscbwdraqjic02zs 2458492 2458491 2026-09-17T15:48:43Z Stashbot 7414 moritzm: installing libde265 security updates 2458492 wikitext text/x-wiki 3tcmnpyf41afeicuvd3pssj9090xbi3 2458493 2458492 2026-09-17T15:51:21Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.hosts.reimage for host ms-be1099.eqiad.wmnet with OS trixie 2458493 wikitext text/x-wiki mena9qqrr0uck929izhmib2snddjv5p 2458494 2458493 2026-09-17T15:52:24Z Stashbot 7414 rzl@deploy1003: helmfile [staging-codfw] START helmfile.d/admin 'apply'. 2458494 wikitext text/x-wiki 4vavlu0heaa037dox53r9401rvkayl8 2458495 2458494 2026-09-17T15:53:17Z Stashbot 7414 rzl@deploy1003: helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. 2458495 wikitext text/x-wiki lcpifs75nmsafvqkwnhpmb8y7e9kbxa 2458497 2458495 2026-09-17T15:53:48Z Stashbot 7414 rzl@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'apply'. 2458497 wikitext text/x-wiki 2y5qjqeev5ovnw4q7gx5es86t4x0e0t 2458498 2458497 2026-09-17T15:54:05Z Stashbot 7414 rzl@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. 2458498 wikitext text/x-wiki dlfzhcbun1zlrkrgncnlixsuzvr3rbz 2458499 2458498 2026-09-17T15:54:31Z Stashbot 7414 rzl@deploy1003: helmfile [codfw] START helmfile.d/admin 'apply'. 2458499 wikitext text/x-wiki i2mdq2bugpfdldsrexbqfpyyjilfddf 2458500 2458499 2026-09-17T15:54:50Z Stashbot 7414 rzl@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'apply'. 2458500 wikitext text/x-wiki j7ndq6ncsgnkj9amscakwnyg8f7256f 2458501 2458500 2026-09-17T15:55:14Z Stashbot 7414 rzl@deploy1003: helmfile [eqiad] START helmfile.d/admin 'apply'. 2458501 wikitext text/x-wiki by4fynt4df4hzq1m0634es5ir3j0jfo 2458502 2458501 2026-09-17T15:55:31Z Stashbot 7414 rzl@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'apply'. 2458502 wikitext text/x-wiki 9yydpb8r522wrjboohu36gd0j9ksiih 2458504 2458502 2026-09-17T16:01:33Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ceph-admin2001.codfw.wmnet with OS bookworm 2458504 wikitext text/x-wiki 4xudyhmn3yqrugffvm07k872ml0wchv 2458505 2458504 2026-09-17T16:01:35Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host ceph-admin2001.codfw.wmnet 2458505 wikitext text/x-wiki mh8gjo52bcpe2gs39ndb0fcctc2d5bf 2458506 2458505 2026-09-17T16:01:45Z Stashbot 7414 moritzm: installing aom security updates 2458506 wikitext text/x-wiki o4z92qvl4wndyhfdkpz9fg0llswjsom 2458508 2458506 2026-09-17T16:06:23Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be1099.eqiad.wmnet with reason: host reimage 2458508 wikitext text/x-wiki b3xbw8thzqqckznguli5ezsqxsocz8d 2458509 2458508 2026-09-17T16:07:59Z Stashbot 7414 rzl@deploy1003: helmfile [staging] START helmfile.d/services/editcheck-headless: apply 2458509 wikitext text/x-wiki dt9wmnz6nvy0wciwhv2ncd6ermxj7yt 2458510 2458509 2026-09-17T16:08:58Z Stashbot 7414 rzl@deploy1003: helmfile [staging] DONE helmfile.d/services/editcheck-headless: apply 2458510 wikitext text/x-wiki 1mkaj2vvmpb49c9e0028m907fyv5kd9 2458512 2458510 2026-09-17T16:11:04Z Stashbot 7414 dbrant@deploy1003: helmfile [staging] START helmfile.d/services/wikifeeds: apply 2458512 wikitext text/x-wiki h8bps72kjn9rqaldpp8vr5wv0ajti27 2458513 2458512 2026-09-17T16:11:28Z Stashbot 7414 dbrant@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifeeds: apply 2458513 wikitext text/x-wiki 83pqjqkot7pmdxe0s0t04sb4yynf088 2458514 2458513 2026-09-17T16:12:07Z Stashbot 7414 vriley@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be1099.eqiad.wmnet with reason: host reimage 2458514 wikitext text/x-wiki gmy66yx46wace33v8fzz9ww3wm70aka 2458515 2458514 2026-09-17T16:12:12Z Stashbot 7414 moritzm: installing libapache-mod-auth-oidc security updates 2458515 wikitext text/x-wiki 89z0kfnizj1b5u2tddrc28lhlfximrh 2458516 2458515 2026-09-17T16:14:27Z Stashbot 7414 dbrant@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifeeds: apply 2458516 wikitext text/x-wiki 3pnx50lmlb6t0rim1jovxmsz693dqmd 2458517 2458516 2026-09-17T16:14:54Z Stashbot 7414 dbrant@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifeeds: apply 2458517 wikitext text/x-wiki 9pl0zj7hgrsjbgzyqaa72be66y40mmc 2458518 2458517 2026-09-17T16:15:09Z Stashbot 7414 dbrant@deploy1003: helmfile [codfw] START helmfile.d/services/wikifeeds: apply 2458518 wikitext text/x-wiki h55wma4u7golbbucqxvinet0r4hbecg 2458519 2458518 2026-09-17T16:15:29Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.dns.netbox 2458519 wikitext text/x-wiki as2n7v134chvygpihm1o7p5hn7m1sqy 2458520 2458519 2026-09-17T16:15:37Z Stashbot 7414 dbrant@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifeeds: apply 2458520 wikitext text/x-wiki 1fl6k1piq6fefhlqt3uyytc3e4ph0r5 2458523 2458520 2026-09-17T16:19:27Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [ms-be1100] - vriley@cumin1004" 2458523 wikitext text/x-wiki rvkl5c2snq8lx493f4d6cqfko32jamm 2458524 2458523 2026-09-17T16:19:44Z Stashbot 7414 vriley@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [ms-be1100] - vriley@cumin1004" 2458524 wikitext text/x-wiki lpaap2cqrctzetz2mi5msbrtv8nv0i7 2458525 2458524 2026-09-17T16:19:45Z Stashbot 7414 vriley@cumin1004: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2458525 wikitext text/x-wiki s6ks1ajk6wfjegg85bybu1mxr6vi57d 2458526 2458525 2026-09-17T16:20:28Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.network.configure-switch-interfaces for host ms-be1100 2458526 wikitext text/x-wiki 6k173ivksd2q1deh4oxxduv42wk8z8f 2458527 2458526 2026-09-17T16:20:53Z Stashbot 7414 vriley@cumin1004: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ms-be1100 2458527 wikitext text/x-wiki tl7nphrs9c7900gn5ve6bqa703obi2l 2458528 2458527 2026-09-17T16:21:30Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.hosts.provision for host ms-be1100.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2458528 wikitext text/x-wiki m3sg7ficwbop95x6i9uz8c5e4v7gw2i 2458533 2458528 2026-09-17T16:31:04Z Stashbot 7414 aokoth@deploy1003: Started deploy [phabricator/deployment@c386249]: Deploy Phab 2458533 wikitext text/x-wiki 01a7mwgfa5ydgv04qh7oma09gvog9fy 2458534 2458533 2026-09-17T16:31:21Z Stashbot 7414 aokoth@deploy1003: Finished deploy [phabricator/deployment@c386249]: Deploy Phab (duration: 00m 19s) 2458534 wikitext text/x-wiki dndrdc0miijs80f6fkmi19no699j7v8 2458536 2458534 2026-09-17T16:35:46Z Stashbot 7414 vriley@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ms-be1100.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2458536 wikitext text/x-wiki loth2p8e7lltst0192te62vpg6hb2jn 2458537 2458536 2026-09-17T16:42:18Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - vriley@cumin1004" 2458537 wikitext text/x-wiki 04lvmrv8flmmdm8pzm35fnvaaij7pyd 2458538 2458537 2026-09-17T16:42:50Z Stashbot 7414 vriley@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - vriley@cumin1004" 2458538 wikitext text/x-wiki qlb0y868wj4486y51mudya8touq6q6l 2458539 2458538 2026-09-17T16:42:51Z Stashbot 7414 vriley@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be1099.eqiad.wmnet with OS trixie 2458539 wikitext text/x-wiki o2z9a0tc33hy1wp6mz5f1k5e80yqxzr 2458540 2458539 2026-09-17T16:52:27Z Stashbot 7414 aokoth@deploy1003: Started deploy [phabricator/deployment@c386249]: Deploy Phab 2458540 wikitext text/x-wiki 84f1d784r2d6z575gbo37twu8l4n3o2 2458541 2458540 2026-09-17T16:52:39Z Stashbot 7414 aokoth@deploy1003: Finished deploy [phabricator/deployment@c386249]: Deploy Phab (duration: 00m 12s) 2458541 wikitext text/x-wiki 0z9vfmry9cjal370g5vjkozy8mjqw2d 2458542 2458541 2026-09-17T17:23:05Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.hosts.reimage for host ms-be1100.eqiad.wmnet with OS trixie 2458542 wikitext text/x-wiki cfsx3udpvwxoftircqq6wedbnkvdp9m 2458543 2458542 2026-09-17T17:37:44Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.hosts.provision for host ms-be1100.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2458543 wikitext text/x-wiki 9gx40mr2429wftrxf8547de65g64xr0 2458544 2458543 2026-09-17T17:38:08Z Stashbot 7414 vriley@cumin1004: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ms-be1100.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2458544 wikitext text/x-wiki b1m3fkmuoix51cjyj7pg6dzac0jwhlf 2458545 2458544 2026-09-17T17:38:32Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.hosts.provision for host ms-be1100.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2458545 wikitext text/x-wiki dreoigqfvrzbpnrhqj7fgjp29z4ft6i 2458548 2458545 2026-09-17T17:47:08Z Stashbot 7414 vriley@cumin1004: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ms-be1100.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2458548 wikitext text/x-wiki 3ph3k0vcjww6jzvvahk8z0wdx4yyg5m 2458551 2458548 2026-09-17T18:20:49Z Stashbot 7414 urbanecm: Deploy a security fix for T438389 2458551 wikitext text/x-wiki q9p33wdxty2augx1ojx37bvfejqfcot 2458552 2458551 2026-09-17T18:32:58Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.hosts.reimage for host ms-be1100.eqiad.wmnet with OS trixie 2458552 wikitext text/x-wiki e6mu7izolbuzga0ozyqky0a50s4u22q 2458554 2458552 2026-09-17T18:48:11Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be1100.eqiad.wmnet with reason: host reimage 2458554 wikitext text/x-wiki faydwvpwzkb876msf2f0gnv99tj1zn8 2458555 2458554 2026-09-17T18:52:34Z Stashbot 7414 vriley@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be1100.eqiad.wmnet with reason: host reimage 2458555 wikitext text/x-wiki is9io65kdjb7uk4iw38e8j83ne31qb5 2458556 2458555 2026-09-17T19:10:06Z Stashbot 7414 vriley@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - vriley@cumin1004" 2458556 wikitext text/x-wiki 7p2k12v45tz3j801z8igr7mwqgz4bib 2458557 2458556 2026-09-17T19:15:16Z Stashbot 7414 vriley@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - vriley@cumin1004" 2458557 wikitext text/x-wiki 9oja2ekuzaz33ab0sx0temseqxwgwhs 2458558 2458557 2026-09-17T19:15:17Z Stashbot 7414 vriley@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be1100.eqiad.wmnet with OS trixie 2458558 wikitext text/x-wiki jezq14627yohk4god8s5au2g1n56c44 2458562 2458558 2026-09-17T20:11:47Z Stashbot 7414 jhuneidi@deploy1003: Started scap sync-world: Backport for [[gerrit:1342774|Worklist Promotion test kitchen - Enable flag in production (T434513)]], [[gerrit:1342781|Exclude returntoapp query from app interception on iOS (T438395)]], [[gerrit:1342798|Revert "Update wikimania wordmark for 2026"]] 2458562 wikitext text/x-wiki rb78e6emhyfxkb50peo17y4jcu3tlm6 2458563 2458562 2026-09-17T20:31:46Z Stashbot 7414 jhuneidi@deploy1003: robertsky, jhuneidi, cmelo, tsev: Backport for [[gerrit:1342774|Worklist Promotion test kitchen - Enable flag in production (T434513)]], [[gerrit:1342781|Exclude returntoapp query from app interception on iOS (T438395)]], [[gerrit:1342798|Revert "Update wikimania wordmark for 2026"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2458563 wikitext text/x-wiki 7wkopsmt12uz7dvkwqvo5zd69znagbs 2458564 2458563 2026-09-17T20:35:29Z Stashbot 7414 jhuneidi@deploy1003: robertsky, jhuneidi, cmelo, tsev: Continuing with deployment 2458564 wikitext text/x-wiki qs08dsketwe84rbdu8jqb55wcr1vy2y 2458565 2458564 2026-09-17T20:47:45Z Stashbot 7414 jhuneidi@deploy1003: Finished scap sync-world: Backport for [[gerrit:1342774|Worklist Promotion test kitchen - Enable flag in production (T434513)]], [[gerrit:1342781|Exclude returntoapp query from app interception on iOS (T438395)]], [[gerrit:1342798|Revert "Update wikimania wordmark for 2026"]] (duration: 35m 59s) 2458565 wikitext text/x-wiki m41s3we12z6uvaus7fxmkrdx5pitfn9 2458566 2458565 2026-09-17T20:55:48Z Stashbot 7414 tsev@deploy1003: mwscript-k8s job started: purgeList.php # T438395 2458566 wikitext text/x-wiki 96p7ruiseno11nm2uh9umm95scnbhx9 2458567 2458566 2026-09-17T21:04:00Z Stashbot 7414 tsev@deploy1003: mwscript-k8s job started: purgeList.php # T438395 2458567 wikitext text/x-wiki 7xgnrngbypywl585jmwhdguk7z7i62s 2458575 2458567 2026-09-18T02:00:56Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2458575 wikitext text/x-wiki oq47z53r058wzo74cjx3x1wecreul3z 2458576 2458575 2026-09-18T02:08:54Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 07m 57s) 2458576 wikitext text/x-wiki nz9qvw5d4cp32nk490wr3valvya9kyi 2458590 2458576 2026-09-18T08:21:32Z Stashbot 7414 brouberol@dns1004: START - running authdns-update 2458590 wikitext text/x-wiki 7oonc9cf2k61dl3ga32dsr74g52rs7y 2458591 2458590 2026-09-18T08:23:52Z Stashbot 7414 brouberol@dns1004: END - running authdns-update 2458591 wikitext text/x-wiki bphpl30nd30g4frdvwe7yma1xvipkz5 2458596 2458591 2026-09-18T09:08:55Z Stashbot 7414 brouberol@deploy1003: helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. 2458596 wikitext text/x-wiki 77811b2669e52s16ty4achxseewhlam 2458597 2458596 2026-09-18T09:09:40Z Stashbot 7414 brouberol@deploy1003: helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. 2458597 wikitext text/x-wiki tdqk132zj2xn6lb5eoy4d12x6fr3fme 2458598 2458597 2026-09-18T09:18:52Z Stashbot 7414 marostegui: clone db1270:x4 from db1155:x4 lag will appear on x4 2458598 wikitext text/x-wiki ktyzd8fwjinvcqj5mkz22z8daex0xhf 2458599 2458598 2026-09-18T09:18:58Z Stashbot 7414 marostegui@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on 21 hosts with reason: cloning db1270 2458599 wikitext text/x-wiki elzo10xfgqnooaykmpa86p35v5oxvyq 2458600 2458599 2026-09-18T09:19:28Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2458600 wikitext text/x-wiki pq4fqvysiblhesizu2d46ya19paksiw 2458601 2458600 2026-09-18T09:20:19Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2458601 wikitext text/x-wiki h5o9z2jfcfhxktp8vffmv6m3vuixctw 2458602 2458601 2026-09-18T09:22:54Z Stashbot 7414 marostegui@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on an-redacteddb1001.eqiad.wmnet with reason: cloning 2458602 wikitext text/x-wiki a21y6ah8sn72192l7ktdvqodomf5ura 2458603 2458602 2026-09-18T09:30:04Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2458603 wikitext text/x-wiki egbj4vpqufh6p93804iiyx528x3mpjb 2458604 2458603 2026-09-18T09:30:15Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/pageview-trending-relative: apply 2458604 wikitext text/x-wiki s9d9syii09nnlrgljyuv51ssiz94rcg 498 9925 2458531 2457681 2026-09-17T16:24:07Z Stashbot 7414 brett: delete deployment-cache, deployment-cache-text, and deployment-cache-upload prefixes - T436468 2458531 wikitext text/x-wiki 09xsp0qmxkcinwep56kv3mjcnudc6ba 0 17290 2458410 2458310 2026-09-17T13:13:49Z Stashbot 7414 hashar: Updated plugins on the CI Jenkins 2458410 wikitext text/x-wiki 9qgcr2rlidh6bbtvv6ulev0xlik0dst 2458455 2458410 2026-09-17T15:00:13Z Stashbot 7414 hashar: gerrit: converting REL1_44 branches to tags to formally EOL REL1_44 | T428911 2458455 wikitext text/x-wiki liarqjo6kh3s5p283mc8lkfedpg31gz 2458473 2458455 2026-09-17T15:09:56Z Stashbot 7414 hashar: gerrit: convert old REL branches on VisualEditor/VisualEditor ( T380841 T428864 T428911 ) using: for branch in REL1_25 REL1_26 REL1_27 REL1_28 REL1_29 REL1_30 REL1_31 REL1_32 REL1_33 REL1_34 REL1_35 REL1_36 REL1_37 REL1_38 REL1_39 REL1_40 REL1_41 REL1_42 REL1_44; do ./.tox/make-release/bin/python ./make-release/branch.py --delete --abandon --bundle ve $branch; done; 2458473 wikitext text/x-wiki 7wi8h54qkunbi49pd88pfclbzmtx1p3 2458503 2458473 2026-09-17T15:59:43Z Stashbot 7414 hashar: gerrit: manually deleted old wmf branches from VisualEditor/VisualEditor # T438373 2458503 wikitext text/x-wiki hgw0bmtmyau88sd9nv5vhj7am8tf0ep 2458530 2458503 2026-09-17T16:24:04Z Stashbot 7414 brett: delete deployment-cache, deployment-cache-text, and deployment-cache-upload prefixes - T436468 2458530 wikitext text/x-wiki ppk8egq989favbltqp8h478gw4iq93y 2458568 2458530 2026-09-17T21:27:56Z Stashbot 7414 hashar: Re running `postmerge` for https://gerrit.wikimedia.org/r/c/mediawiki/services/wikifeeds/+/1342700 for T438379 # `zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/services/wikifeeds --change 1342700,1` 2458568 wikitext text/x-wiki kfoer0ym82g7fkydz8kcphfd7dwnm81 0 442343 2458532 2429222 2026-09-17T16:24:13Z Xaosflux 38028 /* Canonical domain names */ note on scope of what is not on this list 2458532 wikitext text/x-wiki snti3b7n1bpj6a0kpgx5sd6hgreoir1 498 443261 2458390 2458376 2026-09-17T12:44:49Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'proxy' 2458390 wikitext text/x-wiki 0dss25lx052xnf0gc6xj0iq59qacqn2 2458391 2458390 2026-09-17T12:44:56Z Stashbot 7414 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=99) with prefix 'proxy' 2458391 wikitext text/x-wiki 6rzx1txdhij0i5wsx9px3hl6b44ph4x 2458401 2458391 2026-09-17T13:09:42Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'cloudinfra-backup' 2458401 wikitext text/x-wiki di2z5d951t0197a5ebq3lfnv4gqptut 2458402 2458401 2026-09-17T13:09:46Z Stashbot 7414 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=99) with prefix 'cloudinfra-backup' 2458402 wikitext text/x-wiki opsmrmbiihdignudmmviwa8zsyeyxog 2458405 2458402 2026-09-17T13:10:30Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase by 10 server-groups 2458405 wikitext text/x-wiki c1d473gmm5qr2e5tbyil2yja3dz3z9n 2458406 2458405 2026-09-17T13:10:37Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) by 10 server-groups 2458406 wikitext text/x-wiki 3593czvo8ab6yf14gfko0vbzig89du9 2458407 2458406 2026-09-17T13:11:18Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'cloudinfra-backup' 2458407 wikitext text/x-wiki a1hzx1yv77prb52mc6o584o3jphjkx1 2458411 2458407 2026-09-17T13:18:24Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'cloudinfra-backup' 2458411 wikitext text/x-wiki m8jfc3qoqefva4xe6bb374f45pjy0c3 498 443946 2458559 2452977 2026-09-17T19:35:09Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed 4efbe9d1ac (l10n updates: ne) [actually deployed 558d00f12b, temporary T431146 branch rebased on top of that] 2458559 wikitext text/x-wiki jyiza8kzj1cxlck5jm9iqwpo28u29br 498 443949 2458440 2456507 2026-09-17T14:34:59Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Install new requirement (rdflib) in Python 3.11 virtual environment using 'toolforge webservice python3.11 shell' / '/data/project/integraality/www/python/venv/bin/pip install -r /data/project/integraality/integraality/requirements.txt' 2458440 wikitext text/x-wiki 1a0nqakz0ws8q2msfpo0s439b5l0d8i 2458560 2458440 2026-09-17T19:38:32Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 1a7f6a7 (Treat a None WDQS response as a transient backend failure) for T278156 2458560 wikitext text/x-wiki mqvjq2a78ec5zct5i50zsfchsodzqor 2458569 2458560 2026-09-17T21:56:32Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy b76be3f (Log the full traceback when a dashboard run fails) 2458569 wikitext text/x-wiki pv4t8ohuwnqgc8f0tyibi4zuo1tvdkb 2458586 2458569 2026-09-18T07:06:06Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 209dee5 (Reject non-Wikimedia target URLs) for T385749 2458586 wikitext text/x-wiki 0s1v20fwud3lsuisbeusueoisvrj3kj 2458587 2458586 2026-09-18T07:17:32Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy fc09d4c (Reject non-Wikimedia target URLs) for T385749 2458587 wikitext text/x-wiki ker4iwgeeerd0oruwfn2b0anab5q85v 2458588 2458587 2026-09-18T07:17:33Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy d7ffdd1 (Return proper HTTP status codes on /update and /queries errors) 2458588 wikitext text/x-wiki syau05q02b2vm35l7d2chzqmh8d0q9h 2458592 2458588 2026-09-18T08:33:39Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 812bdd0 (Detach the SSE log handler even if worker startup fails) 2458592 wikitext text/x-wiki fa6u2vzafv9dn513158em1vze6vo563 2458593 2458592 2026-09-18T08:33:41Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy d3d88fe (Catch maxlag/login timeout during site construction in process_one_page) 2458593 wikitext text/x-wiki mafbyfuhixja11hnyys6mrvo687dgvd 2458594 2458593 2026-09-18T08:33:43Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy e37d7e7 (Add uWSGI harakiri and worker lifetime backstops) 2458594 wikitext text/x-wiki f5u4abqvyoyohz34rzsnyn0cgmgk6u5 0 445919 2458432 2458097 2026-09-17T14:20:09Z MVernon (WMF) 25893 /* Caveats and limitations */ correct path to auto.yaml 2458432 wikitext text/x-wiki ma6oby05ezshp09r7tilkpdr20jmpo6 0 447352 2458442 2456017 2026-09-17T14:37:28Z LToscano (WMF) 7191 /* Storage backends */ 2458442 wikitext text/x-wiki ckgykv7c93obovs0uvv6hqij7uvciza 0 449160 2458572 2458351 2026-09-18T00:00:07Z Dexbot 30554 Bot: Updating the report 2458572 wikitext text/x-wiki 6hozg9ownnrvdqy6dn6nr7mo4b0i1ek 3 451525 2458546 2387967 2026-09-17T17:46:39Z روتانا 48757 روتانا moved page [[User talk:Lhoussine Bentayfaste]] to [[User talk:Lhoussine BENTAYFASTE]]: Automatically moved page while renaming the user "[[Special:CentralAuth/Lhoussine Bentayfaste|Lhoussine Bentayfaste]]" to "[[Special:CentralAuth/Lhoussine BENTAYFASTE|Lhoussine BENTAYFASTE]]" 2030086 wikitext text/x-wiki 5gyye5i1it9n682aci25nasjxo4lgqa 0 452153 2458550 2455159 2026-09-17T18:07:12Z BLiviero-WMF 53603 Magnum service level upgraded to 2 stars 2458550 wikitext text/x-wiki jtxnr5gi2v6l49uf7t385vmssdz8fa1 0 454144 2458389 2455945 2026-09-17T12:44:39Z Filippo Giunchedi 2939 Add troubleshooting section 2458389 wikitext text/x-wiki t236vpwjiyh38le4h0uriras1p0stle 0 454372 2458399 2451757 2026-09-17T12:59:14Z Taavi-WMF 41365 docs on adding proxy nodes 2458399 wikitext text/x-wiki lazei7dmob6hc1wiadhiqzckdgtvo78 2458595 2458399 2026-09-18T09:08:47Z Taavi-WMF 41365 /* Add a new proxy node */ add missing security group step 2458595 wikitext text/x-wiki n7p8i2d4culnb6mv6g3n32h6m5qdpxp 1 456479 2458529 2458367 2026-09-17T16:23:19Z Xaosflux 38028 /* wiki.gives */ Reply 2458529 wikitext text/x-wiki ssh7cvlxbgl8hvka558imte1qwek78q 0 458676 2458418 2306063 2026-09-17T13:31:56Z MVernon (WMF) 25893 docker-registry quota bumped to 5T (T438339) 2458418 wikitext text/x-wiki cugfvr7cry3ij7y36jno80yp6apsbrg 12 458860 2458464 2456512 2026-09-17T15:04:27Z DCaro (WMF) 18271 /* Creating your tool configuration */ add the yaml schema comment everywhere 2458464 wikitext text/x-wiki b1j75xibfzj8e00siphpwf36hzcvgpg 0 459290 2458507 2458363 2026-09-17T16:06:10Z JAllemandou (WMF) 37709 Add patch to future deploy 2458507 wikitext text/x-wiki ieh3i8aoztuv4sgp2wrc2lr1fbpg83j 2458511 2458507 2026-09-17T16:10:55Z SNwachukwu (WMF) 39521 Add an item to the Next deployment train to be deployed 2458511 wikitext text/x-wiki 6v0sbv0el9tk4e0lufy0hyslxz5c1lc 2458605 2458511 2026-09-18T09:33:56Z JAllemandou (WMF) 37709 Added lines to deploy to next deploy 2458605 wikitext text/x-wiki av43v5zmidsnhvhq7yx2qn2v60t3psu 0 459699 2458573 2458281 2026-09-18T00:36:01Z Stashbot 7414 ejegg: stopped recurring charge jobs 2458573 wikitext text/x-wiki luoqixnccb4yjjxy99f7sebe9jheqyi 2458574 2458573 2026-09-18T01:38:41Z Stashbot 7414 ejegg: started recurring charge jobs again 2458574 wikitext text/x-wiki gg9k6svo650a5i6mu65ko31fnwir747 2 460039 2458606 2450813 2026-09-18T10:02:07Z Atsuko-WMF 54082 Add Patch Link Button to Gitlab Userscript 2458606 wikitext text/x-wiki tqkoys39be2kdf2v4hngqp01rsjmyob 0 460089 2458496 2458227 2026-09-17T15:53:21Z EEvans (WMF) 5035 /* Provisioning storage */ reference a phab when creating tables 2458496 wikitext text/x-wiki krlfej40ue62pwijkgd1rq55ko1tzzq 3 460392 2458577 2431453 2026-09-18T02:37:18Z Max 37491 Max moved page [[User talk:SHB2000]] to [[User talk:SHB]]: Automatically moved page while renaming the user "[[Special:CentralAuth/SHB2000|SHB2000]]" to "[[Special:CentralAuth/SHB|SHB]]" 2431453 wikitext text/x-wiki 6c7u98aheqlzkijrb8zr9xzhwsz9gy3 0 460574 2458429 2453913 2026-09-17T14:18:38Z JGiannelos (WMF) 17083 JGiannelos (WMF) moved page [[SLO/Wikipedia/Read view]] to [[Jgiannelos-tmp-SLO-Wikipedia-Read view]] 2453913 wikitext text/x-wiki cyfvbok63prsdk48dqto0jvcnlw0dya 1 460617 2458426 2457714 2026-09-17T14:07:30Z GLavagetto (WMF) 2905 /* Do not comment on a patch without casting a vote */ Reply 2458426 wikitext text/x-wiki rwgspv84d6btr0675ishth1wsotdjc5 2458535 2458426 2026-09-17T16:32:06Z BCornwall-WMF 31521 /* Do not comment on a patch without casting a vote */ Reply 2458535 wikitext text/x-wiki 2eq054w5fqzjv9nhrgk2i53kgk97649 2458579 2458535 2026-09-18T05:16:22Z GLavagetto (WMF) 2905 /* Pick one reviewer and ping them */ Reply 2458579 wikitext text/x-wiki 12k8gyhk3iqhf2qfm6a3n6zyei2qu7p 2458580 2458579 2026-09-18T05:22:15Z GLavagetto (WMF) 2905 /* Do not comment on a patch without casting a vote */ Reply 2458580 wikitext text/x-wiki 0gcjzda1jgreres7ctf7j7xkq3w3a7s 2458581 2458580 2026-09-18T05:24:06Z GLavagetto (WMF) 2905 /* Guidelines, not rules */ Reply 2458581 wikitext text/x-wiki ae0madarudxa9e4mmv6y7viigxrz9j8 2458582 2458581 2026-09-18T05:30:51Z GLavagetto (WMF) 2905 /* LLM Code reviews */ Reply 2458582 wikitext text/x-wiki qhtii3dljj9khi56m7m6a236z6guk25 2458583 2458582 2026-09-18T05:32:15Z GLavagetto (WMF) 2905 /* Write a clear commit message */ Reply 2458583 wikitext text/x-wiki ptwitujzfwqxd43iw0addqf4md6xxq7 2458584 2458583 2026-09-18T05:35:05Z GLavagetto (WMF) 2905 /* Can we change how we do +1/+2 while we're at it? */ Reply 2458584 wikitext text/x-wiki ktax7uht0a4a7pyxapgb0rsa34m1vv9 2458585 2458584 2026-09-18T05:36:32Z GLavagetto (WMF) 2905 /* Ping them actively to get their attention once you’re ready for their review, to ensure they are available */ Reply 2458585 wikitext text/x-wiki qr1pw0i02hu7u1c9irfnvkn768anadi 0 460621 2458384 2458373 2026-09-17T12:40:33Z JGiannelos (WMF) 17083 2458384 wikitext text/x-wiki m9hgyqfi5j35jdq90eloz2e0s4s4ceb 2458387 2458384 2026-09-17T12:41:07Z JGiannelos (WMF) 17083 2458387 wikitext text/x-wiki jc8se25x9kn4avt1eyjh5d2l1jbsp20 2458433 2458387 2026-09-17T14:23:12Z JGiannelos (WMF) 17083 /* Realistic targets */ 2458433 wikitext text/x-wiki oz9g05bni1py2nnomqpuopdb72693zk 2458434 2458433 2026-09-17T14:24:19Z JGiannelos (WMF) 17083 /* Note */ 2458434 wikitext text/x-wiki 3d7ndu2iqgwvgcddr9kyau4mv7zzbiz 2458437 2458434 2026-09-17T14:29:35Z JGiannelos (WMF) 17083 /* Realistic targets */ 2458437 wikitext text/x-wiki q61jaya8lyjsf7dmv44bvggn7lr920f 2 460671 2458428 2457468 2026-09-17T14:14:13Z CKlimas-WMF 59190 2458428 wikitext text/x-wiki l73ynxzdc8ivmpa2wrlczyqu8d1gxqj 0 460682 2458521 2458225 2026-09-17T16:16:11Z AndrewBogott (WMF) 51047 2458521 wikitext text/x-wiki 5gfbj7em2o9of1q0gikiy08us7ggjts 2458522 2458521 2026-09-17T16:16:31Z AndrewBogott (WMF) 51047 /* Fixing */ 2458522 wikitext text/x-wiki c41oeb938rkr6hp6dyybd71zubbolrj 498 460689 2458380 2026-09-17T12:16:26Z Labslogbot 55 Auto update of instance info. 2458380 wikitext text/x-wiki 73mpb13who5jsjw5ormnm58d4mixh17 3 460690 2458382 2026-09-17T12:21:51Z StrikerBot 8475 Welcome to Toolforge! 2458382 wikitext text/x-wiki oe6gz4rlevmxal8k2kdyhytpxk7oj2h 0 460692 2458430 2026-09-17T14:18:38Z JGiannelos (WMF) 17083 JGiannelos (WMF) moved page [[SLO/Wikipedia/Read view]] to [[Jgiannelos-tmp-SLO-Wikipedia-Read view]] 2458430 wikitext text/x-wiki 6w4ij0yohn90ixtr602e1sf89lr4osw 3 460694 2458547 2026-09-17T17:46:39Z روتانا 48757 روتانا moved page [[User talk:Lhoussine Bentayfaste]] to [[User talk:Lhoussine BENTAYFASTE]]: Automatically moved page while renaming the user "[[Special:CentralAuth/Lhoussine Bentayfaste|Lhoussine Bentayfaste]]" to "[[Special:CentralAuth/Lhoussine BENTAYFASTE|Lhoussine BENTAYFASTE]]" 2458547 wikitext text/x-wiki pw14t8k72bdfvlrnykeikyw94m5m0k3 3 460696 2458578 2026-09-18T02:37:18Z Max 37491 Max moved page [[User talk:SHB2000]] to [[User talk:SHB]]: Automatically moved page while renaming the user "[[Special:CentralAuth/SHB2000|SHB2000]]" to "[[Special:CentralAuth/SHB|SHB]]" 2458578 wikitext text/x-wiki dgionzejbwpmne0gol3e840676zou8k