Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.48.0-wmf.3 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 1516 2479317 2454895 2026-11-10T03:44:19Z Stashbot 7414 AntiComposite: kill all bots - did not rejoin after netsplit 2479317 wikitext text/x-wiki t9fk99h84cvqkn0pgzko5sfkx1gpbwh 498 3086 2479351 2479142 2026-11-10T10:11:21Z Stashbot 7414 fnegri@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor 2479351 wikitext text/x-wiki fboqw97kmbb2ydsshp2qkptfabdfkok 2479352 2479351 2026-11-10T10:14:33Z Stashbot 7414 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-harbor 2479352 wikitext text/x-wiki t7to5543f5uegv8yaq38vvh2so04aps 0 4108 2479259 2479132 2026-11-09T18:16:51Z ScheduleDeploymentBot 37566 Add [[gerrit:1366869]] to Thursday, November 12 UTC late backport window 2479259 wikitext text/x-wiki t7gaqro4c7rc19c2c28d8lhshvbtra2 2479280 2479259 2026-11-09T21:16:59Z ScheduleDeploymentBot 37566 Add [[gerrit:1367950]] to Tuesday, November 10 UTC morning backport window 2479280 wikitext text/x-wiki o9qi7n66q02z7smzlgoilha1gmkouf5 2479315 2479280 2026-11-10T03:01:53Z ScheduleDeploymentBot 37566 Add [[gerrit:1367999]] to Tuesday, November 10 UTC morning backport window 2479315 wikitext text/x-wiki b88fsq7u8rozlyslytseefjdtutl83j 2479316 2479315 2026-11-10T03:02:39Z ScheduleDeploymentBot 37566 Add [[gerrit:1367997]] to Tuesday, November 10 UTC morning backport window 2479316 wikitext text/x-wiki i22hz942lt2quf6j9hla1gda86dpsxi 2479326 2479316 2026-11-10T08:07:42Z Mmullie (WMF) 291 /* {{Deployment_day|date=2026-11-10}} */ 2479326 wikitext text/x-wiki q2jeurbgktx78u1iyx1aaf4au3kvug7 0 7919 2479150 2479149 2026-11-09T11:59:12Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dse-k8s-worker1049.eqiad.wmnet with reason: host reimage 2479150 wikitext text/x-wiki 8nkmffkkr60xpcm9kr71dxxst135uvw 2479151 2479150 2026-11-09T12:00:05Z Stashbot 7414 root@deploy2003: helmfile [staging] START helmfile.d/services/shellbox: sync 2479151 wikitext text/x-wiki h87tf0gwcp06t8hc70bheqce9lc0wp5 2479152 2479151 2026-11-09T12:00:15Z Stashbot 7414 root@deploy2003: helmfile [staging] DONE helmfile.d/services/shellbox: sync 2479152 wikitext text/x-wiki kfht6odevqwp98iktfppcosbc99jvpx 2479153 2479152 2026-11-09T12:00:19Z Stashbot 7414 root@deploy2003: helmfile [staging] START helmfile.d/services/shellbox-timeline: sync 2479153 wikitext text/x-wiki s7znuex6isaknrfeyg81uo89qhmwv9l 2479154 2479153 2026-11-09T12:00:29Z Stashbot 7414 root@deploy2003: helmfile [staging] DONE helmfile.d/services/shellbox-timeline: sync 2479154 wikitext text/x-wiki opdz31w63u8z5n9w7v077nzvn10k3in 2479155 2479154 2026-11-09T12:00:32Z Stashbot 7414 root@deploy2003: helmfile [codfw] START helmfile.d/services/shellbox-timeline: sync 2479155 wikitext text/x-wiki 9oa8z04lnkmcsmwf4g40g35oyooqd1x 2479156 2479155 2026-11-09T12:00:39Z Stashbot 7414 root@deploy2003: helmfile [codfw] DONE helmfile.d/services/shellbox-timeline: sync 2479156 wikitext text/x-wiki hiyieyy08w0zl93a9ivmlvd6kgpb24f 2479157 2479156 2026-11-09T12:08:17Z Stashbot 7414 Emperor: stop swift on thanos-be2006 to VACUUM 27G container DB T443888 2479157 wikitext text/x-wiki elklfitm75hkrucsdlwpcpaw1vfsggz 2479159 2479157 2026-11-09T12:11:18Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dse-k8s-worker2012.codfw.wmnet with OS bookworm 2479159 wikitext text/x-wiki cex2gl9xvnzvlxtf8393zxqh5hg9125 2479160 2479159 2026-11-09T12:17:19Z Stashbot 7414 Emperor: VACUUM 27G container database on thanos-be2008 T443888 2479160 wikitext text/x-wiki mopjz7d5q5x8c6jzxej0yfp01ji6oyo 2479161 2479160 2026-11-09T12:19:57Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dse-k8s-worker1049.eqiad.wmnet with OS bookworm 2479161 wikitext text/x-wiki 1k7yqmmvqsfph4j8n178qnnrd2v4w1u 2479165 2479161 2026-11-09T14:01:22Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reboot-single for host dse-k8s-worker2012.codfw.wmnet 2479165 wikitext text/x-wiki 5camw73g7wr9rne7a6qxbkejjf0uuip 2479166 2479165 2026-11-09T14:01:32Z Stashbot 7414 btullis@cumin1004: START - Cookbook sre.hosts.reboot-single for host dse-k8s-worker1049.eqiad.wmnet 2479166 wikitext text/x-wiki tandvluzz3fn1oe2xwohfsdp4fm8dwn 2479167 2479166 2026-11-09T14:04:37Z Stashbot 7414 jdrewniak@deploy2003: Started scap sync-world: Backport for [[gerrit:1364782|Enable ReadingLists custom lists on all Wikipedias. (T441372)]] 2479167 wikitext text/x-wiki 2hh7xgt5qqps8v3hbas9wmqd0qajfji 2479169 2479167 2026-11-09T14:07:28Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-worker1049.eqiad.wmnet 2479169 wikitext text/x-wiki lb24pfokynbtv2qfy0f5cxjslqa7ssq 2479170 2479169 2026-11-09T14:08:28Z Stashbot 7414 btullis@cumin1004: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-worker2012.codfw.wmnet 2479170 wikitext text/x-wiki 29k2n9aabc6d7vjvq54v5ea0h54arvo 2479171 2479170 2026-11-09T14:11:02Z Stashbot 7414 jdrewniak@deploy2003: jdrewniak: Backport for [[gerrit:1364782|Enable ReadingLists custom lists on all Wikipedias. (T441372)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2479171 wikitext text/x-wiki h28u6vglp94do0s93oxbl1jayidibeg 2479173 2479171 2026-11-09T14:16:50Z Stashbot 7414 jdrewniak@deploy2003: jdrewniak: Continuing with deployment 2479173 wikitext text/x-wiki mwcxfuu28t6pchsipos71cpuobri2rm 2479175 2479173 2026-11-09T14:22:56Z Stashbot 7414 jdrewniak@deploy2003: Finished scap sync-world: Backport for [[gerrit:1364782|Enable ReadingLists custom lists on all Wikipedias. (T441372)]] (duration: 18m 18s) 2479175 wikitext text/x-wiki hivtdnujco155oi2wu63yykjbkgs5mf 2479176 2479175 2026-11-09T14:30:38Z Stashbot 7414 krinkle@deploy2003: Started scap sync-world: Backport for [[gerrit:1367786|RawAction: Enable full CDN caching for canonical CSS/JSON/JS URLs (T279120)]] (1.48.0-wmf.3), [[gerrit:1367752|Always return bundled from getLexers, remove "fetch_lexers" shellbox call (T292663)]] (1.48.0-wmf.3) 2479176 wikitext text/x-wiki khq1lsxola2j1hro6gch5j9wscsi02i 2479177 2479176 2026-11-09T14:34:39Z Stashbot 7414 krinkle@deploy2003: krinkle: Backport for [[gerrit:1367786|RawAction: Enable full CDN caching for canonical CSS/JSON/JS URLs (T279120)]] (1.48.0-wmf.3), [[gerrit:1367752|Always return bundled from getLexers, remove "fetch_lexers" shellbox call (T292663)]] (1.48.0-wmf.3) synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2479177 wikitext text/x-wiki tl6ai9kkpm41rkzwynx2ud2qoi1jvnl 2479179 2479177 2026-11-09T14:38:27Z Stashbot 7414 krinkle@deploy2003: krinkle: Continuing with deployment 2479179 wikitext text/x-wiki p2fn0dzkg3xkg8sajg54w0j0hryv82u 2479180 2479179 2026-11-09T14:38:47Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/weight=10; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-worker2012.codfw.wmnet 2479180 wikitext text/x-wiki j022hxhj1bqxs8l6jmdknx5zaxdr3pu 2479181 2479180 2026-11-09T14:38:53Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/pooled=yes; selector: service=kubesvc,cluster=dse-k8s,dc=codfw,name=dse-k8s-worker2012.codfw.wmnet 2479181 wikitext text/x-wiki dhv5mznc21npigz6owxuyh6t9j222ys 2479182 2479181 2026-11-09T14:39:13Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/weight=10; selector: service=kubesvc,cluster=dse-k8s,dc=eqiad,name=dse-k8s-worker1049.eqiad.wmnet 2479182 wikitext text/x-wiki 921b5o99n917vd5yzu7xec7mghg8nl7 2479183 2479182 2026-11-09T14:39:21Z Stashbot 7414 btullis@puppetserver1001: conftool action : set/pooled=yes; selector: service=kubesvc,cluster=dse-k8s,dc=eqiad,name=dse-k8s-worker1049.eqiad.wmnet 2479183 wikitext text/x-wiki sx14n70fn7ckj6d4uqesdk67sp9lvl5 2479184 2479183 2026-11-09T14:44:28Z Stashbot 7414 krinkle@deploy2003: Finished scap sync-world: Backport for [[gerrit:1367786|RawAction: Enable full CDN caching for canonical CSS/JSON/JS URLs (T279120)]] (1.48.0-wmf.3), [[gerrit:1367752|Always return bundled from getLexers, remove "fetch_lexers" shellbox call (T292663)]] (1.48.0-wmf.3) (duration: 13m 51s) 2479184 wikitext text/x-wiki 5n91j4p9obdgbfb1j8fiwa97ea2raqx 2479186 2479184 2026-11-09T14:46:27Z Stashbot 7414 krinkle@deploy2003: Started scap sync-world: Backport for [[gerrit:1367786|RawAction: Enable full CDN caching for canonical CSS/JSON/JS URLs (T279120)]] (1.48.0-wmf.3) 2479186 wikitext text/x-wiki k5phsy2qx85pt8fg09tqabr81abxxsd 2479187 2479186 2026-11-09T14:47:09Z Stashbot 7414 krinkle@deploy2003: sync-world aborted: Backport for [[gerrit:1367786|RawAction: Enable full CDN caching for canonical CSS/JSON/JS URLs (T279120)]] (1.48.0-wmf.3) (duration: 00m 41s) 2479187 wikitext text/x-wiki m4cixsbsqxb9egofb56qmls61h0m0bz 2479188 2479187 2026-11-09T14:48:22Z Stashbot 7414 krinkle@deploy2003: Started scap sync-world: Backport for [[gerrit:1354558|Remove unused "TitleBlacklist-cache" channel (T85428 T441284)]] 2479188 wikitext text/x-wiki r4dfhni8qfk9zzcplmsuqulokyk16ow 2479189 2479188 2026-11-09T14:52:26Z Stashbot 7414 krinkle@deploy2003: krinkle: Backport for [[gerrit:1354558|Remove unused "TitleBlacklist-cache" channel (T85428 T441284)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2479189 wikitext text/x-wiki nsuexz4emi16aly1ylgxvjrfusu9wqa 2479190 2479189 2026-11-09T14:53:43Z Stashbot 7414 krinkle@deploy2003: krinkle: Continuing with deployment 2479190 wikitext text/x-wiki eet7h4yo7zwi6de11n3id6eiej0a4va 2479191 2479190 2026-11-09T14:59:42Z Stashbot 7414 krinkle@deploy2003: Finished scap sync-world: Backport for [[gerrit:1354558|Remove unused "TitleBlacklist-cache" channel (T85428 T441284)]] (duration: 11m 20s) 2479191 wikitext text/x-wiki b0y5o67r0civewrxem4mwg9qd7j2zm1 2479197 2479191 2026-11-09T15:07:17Z Stashbot 7414 otto@deploy2003: helmfile [staging] START helmfile.d/services/changeprop-jobqueue: apply 2479197 wikitext text/x-wiki 8j9gffihda0b8cr6bga0065dadhtw1d 2479198 2479197 2026-11-09T15:07:39Z Stashbot 7414 ottomata: deploying changeprop-jobqueue to pick up new queue for T443442 2479198 wikitext text/x-wiki pvl14wa1vsbf48p9a429qpwn0c4w7d6 2479199 2479198 2026-11-09T15:07:52Z Stashbot 7414 otto@deploy2003: helmfile [staging] DONE helmfile.d/services/changeprop-jobqueue: apply 2479199 wikitext text/x-wiki so8dz2g0s4m3gvv9w4up2mxdsrie6er 2479200 2479199 2026-11-09T15:08:28Z Stashbot 7414 otto@deploy2003: helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply 2479200 wikitext text/x-wiki jhxb3k0cbbr8g0opi5pv6xgw86h6sno 2479201 2479200 2026-11-09T15:10:03Z Stashbot 7414 otto@deploy2003: helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply 2479201 wikitext text/x-wiki c3uggah8no7dgkicw7nou5kyztvdlbg 2479202 2479201 2026-11-09T15:10:25Z Stashbot 7414 otto@deploy2003: helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: apply 2479202 wikitext text/x-wiki b9wsyhznfbrbye95p908ojq3pvti7tc 2479203 2479202 2026-11-09T15:11:42Z Stashbot 7414 otto@deploy2003: helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: apply 2479203 wikitext text/x-wiki homr8t2pnieggjibwd9lv2so9jz1gvs 2479204 2479203 2026-11-09T15:14:29Z Stashbot 7414 otto@deploy2003: Started scap sync-world: Backport for [[gerrit:1362093|Prefetch edit_suggestions_counts with EventBus LinkedArtifacts (T443442)]] 2479204 wikitext text/x-wiki jwrqm3kgcg8m8zk94q3o037fevkjxwp 2479205 2479204 2026-11-09T15:16:34Z Stashbot 7414 otto@deploy2003: otto: Backport for [[gerrit:1362093|Prefetch edit_suggestions_counts with EventBus LinkedArtifacts (T443442)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2479205 wikitext text/x-wiki e70x6fh6s1mfedkz2w6l622wp1y3e0g 2479206 2479205 2026-11-09T15:17:05Z Stashbot 7414 otto@deploy2003: otto: Continuing with deployment 2479206 wikitext text/x-wiki gpd1vuqtt8z784x2d1fwixsenwykh6f 2479207 2479206 2026-11-09T15:22:34Z Stashbot 7414 klausman@deploy2003: helmfile [ml-staging-codfw] START helmfile.d/admin 'apply'. 2479207 wikitext text/x-wiki fu0ejbgwutg3gz7i3f533id0f0melaj 2479208 2479207 2026-11-09T15:23:09Z Stashbot 7414 otto@deploy2003: Finished scap sync-world: Backport for [[gerrit:1362093|Prefetch edit_suggestions_counts with EventBus LinkedArtifacts (T443442)]] (duration: 08m 40s) 2479208 wikitext text/x-wiki gy09uk5yoev6ql4ekrbtvw5dxskqrx4 2479209 2479208 2026-11-09T15:26:30Z Stashbot 7414 ladsgroup@deploy2003: Started scap sync-world: Backport for [[gerrit:1367862|Enable MPEG-DASH everywhere (T373546)]] 2479209 wikitext text/x-wiki ka30bdhkwtynihm632g30wwhqtx9029 2479210 2479209 2026-11-09T15:27:07Z Stashbot 7414 dbrant@deploy2003: helmfile [staging] START helmfile.d/services/wikifeeds: apply 2479210 wikitext text/x-wiki 4nxnbtjpay2p3gcp39ltplbvrapjfye 2479211 2479210 2026-11-09T15:27:20Z Stashbot 7414 klausman@deploy2003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'apply'. 2479211 wikitext text/x-wiki oi1lxz5myhy3e3zpt6ixq0frnnxeqcb 2479212 2479211 2026-11-09T15:27:30Z Stashbot 7414 dbrant@deploy2003: helmfile [staging] DONE helmfile.d/services/wikifeeds: apply 2479212 wikitext text/x-wiki s86su3tlzx4ixrypmlh319wrf64m8ll 2479213 2479212 2026-11-09T15:27:40Z Stashbot 7414 dbrant@deploy2003: helmfile [eqiad] START helmfile.d/services/wikifeeds: apply 2479213 wikitext text/x-wiki a6sid084q3wrlvsb7w8b3d2vslb4v0k 2479214 2479213 2026-11-09T15:28:05Z Stashbot 7414 dbrant@deploy2003: helmfile [eqiad] DONE helmfile.d/services/wikifeeds: apply 2479214 wikitext text/x-wiki fx476h38rkmi51reipqiy6n5gh9qtw5 2479215 2479214 2026-11-09T15:28:13Z Stashbot 7414 dbrant@deploy2003: helmfile [codfw] START helmfile.d/services/wikifeeds: apply 2479215 wikitext text/x-wiki rlrgsqampbw1zrg1ivkcy8qinibafyi 2479216 2479215 2026-11-09T15:28:34Z Stashbot 7414 ladsgroup@deploy2003: ladsgroup: Backport for [[gerrit:1367862|Enable MPEG-DASH everywhere (T373546)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2479216 wikitext text/x-wiki sev674bn0xc2t99zoo71syp4j4w7fsn 2479217 2479216 2026-11-09T15:28:37Z Stashbot 7414 dbrant@deploy2003: helmfile [codfw] DONE helmfile.d/services/wikifeeds: apply 2479217 wikitext text/x-wiki 6oqndoexzgzvgm639rybs4k762m8n4p 2479218 2479217 2026-11-09T15:28:51Z Stashbot 7414 ayounsi@cumin1004: START - Cookbook sre.network.peering with action 'email' for AS: 267201 2479218 wikitext text/x-wiki 7fl3bo90pcsh48l87j5ixei27vphyri 2479219 2479218 2026-11-09T15:29:08Z Stashbot 7414 ayounsi@cumin1004: END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 267201 2479219 wikitext text/x-wiki 3fl5u7dxdti1oz21gdc7c17euh2qxle 2479220 2479219 2026-11-09T15:29:19Z Stashbot 7414 ladsgroup@deploy2003: ladsgroup: Continuing with deployment 2479220 wikitext text/x-wiki pgfkig8q63lfbzk3j0d04c0b8i2xqmx 2479221 2479220 2026-11-09T15:32:50Z Stashbot 7414 btullis@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply 2479221 wikitext text/x-wiki it3c6u6z9pfvgjfi4eoz2zpgeylirtr 2479222 2479221 2026-11-09T15:33:31Z Stashbot 7414 btullis@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply 2479222 wikitext text/x-wiki 5mlr7b01i7w6i31xiz63q9cipozrje2 2479223 2479222 2026-11-09T15:35:17Z Stashbot 7414 ladsgroup@deploy2003: Finished scap sync-world: Backport for [[gerrit:1367862|Enable MPEG-DASH everywhere (T373546)]] (duration: 08m 48s) 2479223 wikitext text/x-wiki 34kqfud7gf0t6kohxfkc371qk1wgsdr 2479225 2479223 2026-11-09T15:48:37Z Stashbot 7414 btullis@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-dumps: apply 2479225 wikitext text/x-wiki 6i1ambpwi2axpnwnb89e1t9ggp1600p 2479226 2479225 2026-11-09T15:48:47Z Stashbot 7414 btullis@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-dumps: apply 2479226 wikitext text/x-wiki 5mtz9tazp80g4ob5k0yl2ydfkmbt63x 2479227 2479226 2026-11-09T15:55:06Z Stashbot 7414 otto@deploy2003: Started scap sync-world: Backport for [[gerrit:1357611|[Beta EventBus] Add revision_tags_change and log_tags_change dev0 streams, enable in beta (T434143)]] 2479227 wikitext text/x-wiki 6wbdxyil6sbhxtqq8j8b5gza8f02zjl 2479228 2479227 2026-11-09T15:59:21Z Stashbot 7414 otto@deploy2003: tchin, otto: Backport for [[gerrit:1357611|[Beta EventBus] Add revision_tags_change and log_tags_change dev0 streams, enable in beta (T434143)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2479228 wikitext text/x-wiki 0er1ubwp3oz0ir7kwshsgd3b7rue0h3 2479229 2479228 2026-11-09T16:00:49Z Stashbot 7414 otto@deploy2003: tchin, otto: Continuing with deployment 2479229 wikitext text/x-wiki 9vhfk2iql6d20zic2hl5b2gx3chkd27 2479230 2479229 2026-11-09T16:01:49Z Stashbot 7414 klausman@deploy2003: helmfile [ml-staging-codfw] START helmfile.d/admin 'apply'. 2479230 wikitext text/x-wiki lz5dyytqe7qqregtofy6a41541be0jo 2479231 2479230 2026-11-09T16:02:08Z Stashbot 7414 klausman@deploy2003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'apply'. 2479231 wikitext text/x-wiki r9kav81t004bj5mnz5hbeg7t6zh8v7f 2479232 2479231 2026-11-09T16:03:01Z Stashbot 7414 trueg@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2479232 wikitext text/x-wiki d28v8kgc8c49ylhqi2mrc3clpq4x3nd 2479234 2479232 2026-11-09T16:05:36Z Stashbot 7414 jelto@deploy2003: helmfile [aux-k8s-eqiad] START helmfile.d/aux-k8s-services/etherpad: apply 2479234 wikitext text/x-wiki oqwj60hvvyn6qm5tmda1ve2e00m5dk4 2479235 2479234 2026-11-09T16:06:06Z Stashbot 7414 trueg@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply 2479235 wikitext text/x-wiki 7ecidyqav8k7ran6gfu8fpk1h9pygom 2479236 2479235 2026-11-09T16:07:05Z Stashbot 7414 otto@deploy2003: Finished scap sync-world: Backport for [[gerrit:1357611|[Beta EventBus] Add revision_tags_change and log_tags_change dev0 streams, enable in beta (T434143)]] (duration: 11m 59s) 2479236 wikitext text/x-wiki 65gs5jqllxd59rugkswmmhlihak5akd 2479237 2479236 2026-11-09T16:08:56Z Stashbot 7414 klausman@deploy2003: helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. 2479237 wikitext text/x-wiki onmxhx9yj4axxyau4s5heq13gpgno4p 2479238 2479237 2026-11-09T16:10:51Z Stashbot 7414 klausman@deploy2003: helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. 2479238 wikitext text/x-wiki n0j8e8x0r64snot8lq3rmh3jl58t94w 2479239 2479238 2026-11-09T16:15:59Z Stashbot 7414 jelto@deploy2003: helmfile [aux-k8s-eqiad] DONE helmfile.d/aux-k8s-services/etherpad: apply 2479239 wikitext text/x-wiki hxp77mn9hfkymkblqzg6csnlsmrv24o 2479240 2479239 2026-11-09T16:43:42Z Stashbot 7414 klausman@deploy2003: helmfile [ml-serve-codfw] START helmfile.d/admin 'apply'. 2479240 wikitext text/x-wiki 2d2mm4amb67b27q6otbvwkyie3mrg5s 2479241 2479240 2026-11-09T16:45:41Z Stashbot 7414 klausman@deploy2003: helmfile [ml-serve-codfw] DONE helmfile.d/admin 'apply'. 2479241 wikitext text/x-wiki ddxd4lb9vf88tja3mm7jox2oo09bya2 2479242 2479241 2026-11-09T17:08:53Z Stashbot 7414 trueg@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2479242 wikitext text/x-wiki qkhgzi7wc1tpj6i61p0xi6oog7yt6wp 2479244 2479242 2026-11-09T17:23:30Z Stashbot 7414 trueg@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply 2479244 wikitext text/x-wiki 8keoq15cjdpgybd1ogxd888fmyt7qta 2479245 2479244 2026-11-09T17:25:04Z Stashbot 7414 trueg@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply 2479245 wikitext text/x-wiki o7s19yyzhrbptvn344mbj13llc0smtc 2479246 2479245 2026-11-09T17:28:23Z Stashbot 7414 ebernhardson: T444015 start full-cluster reindex in eqiad 2479246 wikitext text/x-wiki jnfq9jix66uo4qbd54wqdjpz4xx59kv 2479247 2479246 2026-11-09T17:35:13Z Stashbot 7414 trueg@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply 2479247 wikitext text/x-wiki tcoc4pv9f6qehrwpmk8pgwxvpp61nc5 2479248 2479247 2026-11-09T17:36:06Z Stashbot 7414 trueg@deploy2003: helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs-next: apply 2479248 wikitext text/x-wiki to9ul95din1bbuoa9k58mhdwjwmh3sv 2479249 2479248 2026-11-09T17:40:32Z Stashbot 7414 ebernhardson: T444015 start cirrussearch full-cluster reindex in codfw and cloudelastic 2479249 wikitext text/x-wiki 38rr19of9287m85b58tnd74mkxfzja7 2479250 2479249 2026-11-09T17:45:42Z Stashbot 7414 trueg@deploy2003: helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs-next: apply 2479250 wikitext text/x-wiki 5b6dt3hxziclh3whu5y9hm1foeyf7xi 2479251 2479250 2026-11-09T17:48:59Z Stashbot 7414 cgoubert@deploy2003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply 2479251 wikitext text/x-wiki ki5jm037xwjwgjcuyyxncnrd9l9095r 2479252 2479251 2026-11-09T17:49:32Z Stashbot 7414 cgoubert@deploy2003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply 2479252 wikitext text/x-wiki h1mt0v3et7yhl4v8ps5eln45gbhmke8 2479253 2479252 2026-11-09T17:54:32Z Stashbot 7414 cgoubert@deploy2003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2479253 wikitext text/x-wiki hkkqyz48u65gf28otgm1idz8bxmhldv 2479254 2479253 2026-11-09T17:54:45Z Stashbot 7414 cgoubert@deploy2003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2479254 wikitext text/x-wiki oeqlegw1tbmcu3caysw78uj2qd5upqi 2479255 2479254 2026-11-09T18:06:02Z Stashbot 7414 cgoubert@deploy2003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2479255 wikitext text/x-wiki jb16przs702e3h0r245i68oz3bsdwc8 2479256 2479255 2026-11-09T18:06:13Z Stashbot 7414 cgoubert@deploy2003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2479256 wikitext text/x-wiki a7bs5yt60d11zo6373hvyaie2uz03b3 2479257 2479256 2026-11-09T18:06:35Z Stashbot 7414 cgoubert@deploy2003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply 2479257 wikitext text/x-wiki kc0yp5pgma3f333o8v8htu9c65kjy5j 2479258 2479257 2026-11-09T18:06:48Z Stashbot 7414 cgoubert@deploy2003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply 2479258 wikitext text/x-wiki o6o0pk196dxu5dc7qkpe7vozd7u79vm 2479262 2479258 2026-11-09T19:00:13Z Stashbot 7414 btullis@deploy2003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply 2479262 wikitext text/x-wiki 7ovq39kxi0ly1wbozczqt13juplyim5 2479263 2479262 2026-11-09T19:01:34Z Stashbot 7414 btullis@deploy2003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply 2479263 wikitext text/x-wiki 3y2mbvr7ql0s1q7uph22k2oam48a3gc 2479265 2479263 2026-11-09T19:44:38Z Stashbot 7414 gmodena@deploy2003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2479265 wikitext text/x-wiki 8ryjwxo1h2flm6f170yuomzf7cnbl9y 2479266 2479265 2026-11-09T19:44:49Z Stashbot 7414 gmodena@deploy2003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2479266 wikitext text/x-wiki mf5sm2lhbpbqd3gvd4aa6hs57ncnqsf 2479270 2479266 2026-11-09T20:19:18Z Stashbot 7414 gmodena@deploy2003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2479270 wikitext text/x-wiki 4w1l71zjaoojgo8t314b7bou6qv6ub9 2479271 2479270 2026-11-09T20:19:29Z Stashbot 7414 gmodena@deploy2003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2479271 wikitext text/x-wiki bc1lmh7fp3bnmyr05856f3nq98jahfo 2479276 2479271 2026-11-09T21:05:35Z Stashbot 7414 egardner@deploy2003: Started scap sync-world: Backport for [[gerrit:1358854|Enable the ShareCard feature on all Wikipedias (Minerva only) (T439162)]] 2479276 wikitext text/x-wiki 12cm32uonnrpu8aji8zgo0ov62y2upa 2479277 2479276 2026-11-09T21:11:40Z Stashbot 7414 egardner@deploy2003: egardner: Backport for [[gerrit:1358854|Enable the ShareCard feature on all Wikipedias (Minerva only) (T439162)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2479277 wikitext text/x-wiki ksaesl5qb5d9x0354vlm5gpz6xwruus 2479278 2479277 2026-11-09T21:12:18Z Stashbot 7414 egardner@deploy2003: egardner: Continuing with deployment 2479278 wikitext text/x-wiki q9rlamvg6e4w689rr2lpritaetcasfk 2479279 2479278 2026-11-09T21:16:52Z Stashbot 7414 akhatun@deploy2003: helmfile [staging] START helmfile.d/services/lac-lambdas: apply 2479279 wikitext text/x-wiki 1f7pdq5lm5vmdb42o8n65azz6bbtdlc 2479281 2479279 2026-11-09T21:17:11Z Stashbot 7414 akhatun@deploy2003: helmfile [staging] DONE helmfile.d/services/lac-lambdas: apply 2479281 wikitext text/x-wiki anfml5gimvnve1f671ciq63qfkxe8xj 2479282 2479281 2026-11-09T21:18:23Z Stashbot 7414 egardner@deploy2003: Finished scap sync-world: Backport for [[gerrit:1358854|Enable the ShareCard feature on all Wikipedias (Minerva only) (T439162)]] (duration: 12m 48s) 2479282 wikitext text/x-wiki h3xx0zql4vin4595ost81on9vvqccpl 2479283 2479282 2026-11-09T21:19:24Z Stashbot 7414 akhatun@deploy2003: helmfile [eqiad] START helmfile.d/services/lac-lambdas: apply 2479283 wikitext text/x-wiki qnc7jxjlztb0srhq2w5s9rdi6ka06ig 2479284 2479283 2026-11-09T21:19:46Z Stashbot 7414 akhatun@deploy2003: helmfile [eqiad] DONE helmfile.d/services/lac-lambdas: apply 2479284 wikitext text/x-wiki nw8wf9v7e94krerl3gzuzryu4cfuvpe 2479285 2479284 2026-11-09T21:22:36Z Stashbot 7414 akhatun@deploy2003: helmfile [codfw] START helmfile.d/services/lac-lambdas: apply 2479285 wikitext text/x-wiki 8z57i3ygfic58xacz9c6qh7wdq2rm0v 2479286 2479285 2026-11-09T21:23:02Z Stashbot 7414 akhatun@deploy2003: helmfile [codfw] DONE helmfile.d/services/lac-lambdas: apply 2479286 wikitext text/x-wiki 6f2pu87226whozvv1b3rsj400i88l90 2479293 2479286 2026-11-09T23:11:02Z Stashbot 7414 maryum: Deployed security fix for T443539, T440392, T435882, T442600 2479293 wikitext text/x-wiki 4wvisug0kkcxtbpk0t76pgw1opn51wd 2479295 2479293 2026-11-10T00:00:06Z Stashbot 7414 root@deploy2003: helmfile [staging] START helmfile.d/services/shellbox-timeline: sync 2479295 wikitext text/x-wiki 3bvopgxhd1fiom03uoph6337wl4c8cn 2479296 2479295 2026-11-10T00:00:16Z Stashbot 7414 root@deploy2003: helmfile [staging] DONE helmfile.d/services/shellbox-timeline: sync 2479296 wikitext text/x-wiki kyzkfojh0vxa5ui6b0lndtgwiwj42xl 2479297 2479296 2026-11-10T00:00:19Z Stashbot 7414 root@deploy2003: helmfile [codfw] START helmfile.d/services/shellbox-timeline: sync 2479297 wikitext text/x-wiki rv7ujqg2wkffauhhba636vuudqj5ls4 2479298 2479297 2026-11-10T00:00:26Z Stashbot 7414 root@deploy2003: helmfile [codfw] DONE helmfile.d/services/shellbox-timeline: sync 2479298 wikitext text/x-wiki gavz7whhhwed8ydbsx48hxqznybjf5a 2479301 2479298 2026-11-10T00:07:17Z Stashbot 7414 zabe@deploy2003: Started scap sync-world: Backport for [[gerrit:1360928|Disable automatic user page watching on loginwiki (T442779)]] 2479301 wikitext text/x-wiki 220nqjh9vsl4p294jjj9o1dda4rla1x 2479302 2479301 2026-11-10T00:15:24Z Stashbot 7414 zabe@deploy2003: zabe: Backport for [[gerrit:1360928|Disable automatic user page watching on loginwiki (T442779)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2479302 wikitext text/x-wiki j8lnj883nu7cs2dlvo2tno9bom5uj63 2479303 2479302 2026-11-10T00:16:51Z Stashbot 7414 zabe@deploy2003: zabe: Continuing with deployment 2479303 wikitext text/x-wiki gbw465yw27uol37hq303ar83picww5g 2479304 2479303 2026-11-10T00:25:42Z Stashbot 7414 zabe@deploy2003: Finished scap sync-world: Backport for [[gerrit:1360928|Disable automatic user page watching on loginwiki (T442779)]] (duration: 18m 24s) 2479304 wikitext text/x-wiki sm7bnazjgkjcy0m6p0ihgs6uzjs0r1s 2479305 2479304 2026-11-10T00:43:00Z Stashbot 7414 zabe@deploy2003: Started scap sync-world: Backport for [[gerrit:1361059|Move reads/writes of wbc_entity_usage to x1 for enwiki (T439753)]] 2479305 wikitext text/x-wiki bqddtbx48s03ypy0i6n1lat7o3qxhm3 2479306 2479305 2026-11-10T00:47:07Z Stashbot 7414 zabe@deploy2003: zabe: Backport for [[gerrit:1361059|Move reads/writes of wbc_entity_usage to x1 for enwiki (T439753)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2479306 wikitext text/x-wiki iqn7ba9uu3k8xp62qw675rnw7b1jgbv 2479307 2479306 2026-11-10T00:47:36Z Stashbot 7414 zabe@deploy2003: zabe: Continuing with deployment 2479307 wikitext text/x-wiki 9i55xrwqny0luw29x9b01yyylpqvqy3 2479308 2479307 2026-11-10T00:54:47Z Stashbot 7414 zabe@deploy2003: Finished scap sync-world: Backport for [[gerrit:1361059|Move reads/writes of wbc_entity_usage to x1 for enwiki (T439753)]] (duration: 11m 47s) 2479308 wikitext text/x-wiki nsvmvpzpwn40tlzz9vg6xyxmvmkqbwk 2479309 2479308 2026-11-10T01:06:50Z Stashbot 7414 zabe: run populateLuAttachmentMethod.php # T442322 2479309 wikitext text/x-wiki kt789v2qwdsskwmk3wf1ddpqlvwjffw 2479311 2479309 2026-11-10T02:00:52Z Stashbot 7414 mwpresync@deploy2003: Started scap build-images: Publishing wmf/next image 2479311 wikitext text/x-wiki gn39fw4i0joons9x1fuw8wfh6rlo6o5 2479312 2479311 2026-11-10T02:08:28Z Stashbot 7414 mwpresync@deploy2003: Finished scap build-images: Publishing wmf/next image (duration: 07m 35s) 2479312 wikitext text/x-wiki aujzr8ed1kt7geut7oph1a3ogknnaps 2479318 2479312 2026-11-10T04:03:23Z Stashbot 7414 mwpresync@deploy2003: Started scap sync-world: testwikis to 1.48.0-wmf.4 refs T442222 2479318 wikitext text/x-wiki 4nhf9l5qqj8sjhd9p3ge4k91okmi2yg 2479319 2479318 2026-11-10T04:42:41Z Stashbot 7414 mwpresync@deploy2003: Finished scap sync-world: testwikis to 1.48.0-wmf.4 refs T442222 (duration: 39m 18s) 2479319 wikitext text/x-wiki n0sumz3c5zau9f5pqnfk9lbdgkkptyl 2479320 2479319 2026-11-10T05:02:28Z Stashbot 7414 mwpresync@deploy2003: Pruned MediaWiki: 1.47.0-wmf.24 (duration: 02m 26s) 2479320 wikitext text/x-wiki tsdxxk88vwvazhyi5nc6l8gzn2jaxmx 2479323 2479320 2026-11-10T07:56:55Z Stashbot 7414 jdrewniak@deploy2003: Started scap sync-world: Backport for [[gerrit:1367950|Bumping portals to master (T128546)]] 2479323 wikitext text/x-wiki 6g7pzetsa3l4y3olft1ee1mm1tshoox 2479324 2479323 2026-11-10T08:03:28Z Stashbot 7414 jdrewniak@deploy2003: jdrewniak: Backport for [[gerrit:1367950|Bumping portals to master (T128546)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2479324 wikitext text/x-wiki q5ot7g0nss0op1m8tbyi55xn7elztc9 2479325 2479324 2026-11-10T08:04:25Z Stashbot 7414 jdrewniak@deploy2003: jdrewniak: Continuing with deployment 2479325 wikitext text/x-wiki db4da064wh6nwtw4hp23igheakfyrt8 2479327 2479325 2026-11-10T08:15:21Z Stashbot 7414 jdrewniak@deploy2003: Finished scap sync-world: Backport for [[gerrit:1367950|Bumping portals to master (T128546)]] (duration: 18m 26s) 2479327 wikitext text/x-wiki qdjgmg9ffmbt8qzcpkxezft9hlf3jzu 2479328 2479327 2026-11-10T08:28:17Z Stashbot 7414 chlod@deploy2003: Started scap sync-world: Backport for [[gerrit:1367997|tcywiki: revert 10 anniversary logo (T434176)]], [[gerrit:1367999|bjnwiktionary: update logos (T444085)]] 2479328 wikitext text/x-wiki h4nfer8m454nuy3cwebligl29oecp4e 2479329 2479328 2026-11-10T08:35:29Z Stashbot 7414 chlod@deploy2003: chlod, anzx: Backport for [[gerrit:1367997|tcywiki: revert 10 anniversary logo (T434176)]], [[gerrit:1367999|bjnwiktionary: update logos (T444085)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2479329 wikitext text/x-wiki 38xtp0frrjmowb4i8vm6kep2w1rc4a9 2479330 2479329 2026-11-10T08:36:47Z Stashbot 7414 chlod@deploy2003: chlod, anzx: Continuing with deployment 2479330 wikitext text/x-wiki geay21am3mt8qc3uiuz85xhq7kv6ja2 2479331 2479330 2026-11-10T08:41:14Z Stashbot 7414 jgiannelos@deploy2003: helmfile [staging] START helmfile.d/services/thumbor: apply 2479331 wikitext text/x-wiki e8rngcv9lkjobkzn2lpx63oi4ugytgm 2479332 2479331 2026-11-10T08:41:25Z Stashbot 7414 jgiannelos@deploy2003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2479332 wikitext text/x-wiki 1sqhyqxbrd7lzw892bve9mcr3z1o5wc 2479333 2479332 2026-11-10T08:42:42Z Stashbot 7414 ayounsi@cumin1004: START - Cookbook sre.dns.admin DNS admin: depool esams [reason: network upgrade, T440349] 2479333 wikitext text/x-wiki 7d8jz0725lc1avtzscbewqg1ab6qs76 2479334 2479333 2026-11-10T08:42:50Z Stashbot 7414 ayounsi@cumin1004: END (FAIL) - Cookbook sre.dns.admin (exit_code=99) DNS admin: depool esams [reason: network upgrade, T440349] 2479334 wikitext text/x-wiki 19nlrkwg2u32ezfjkl1vmcm573wgium 2479335 2479334 2026-11-10T08:43:41Z Stashbot 7414 ayounsi@cumin1004: START - Cookbook sre.dns.admin DNS admin: depool esams [reason: network upgrade, T440349] 2479335 wikitext text/x-wiki f8w8g4oufrgkhj2anz2txktvzdpnhqr 2479336 2479335 2026-11-10T08:43:47Z Stashbot 7414 ayounsi@cumin1004: END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: depool esams [reason: network upgrade, T440349] 2479336 wikitext text/x-wiki lcrnn97803y889npdyae7u7pnjdsa4h 2479337 2479336 2026-11-10T08:44:00Z Stashbot 7414 chlod@deploy2003: Finished scap sync-world: Backport for [[gerrit:1367997|tcywiki: revert 10 anniversary logo (T434176)]], [[gerrit:1367999|bjnwiktionary: update logos (T444085)]] (duration: 15m 43s) 2479337 wikitext text/x-wiki 7zsuz7mw4czhbh0zacqy1ywlgxn1zl6 2479339 2479337 2026-11-10T08:48:47Z Stashbot 7414 mlitn@deploy2003: Started scap sync-world: Backport for [[gerrit:1368165|Squashed diff to master]] (1.48.0-wmf.3) 2479339 wikitext text/x-wiki r1bjns67rjonuqo6p6qmys4earem38b 2479340 2479339 2026-11-10T08:55:08Z Stashbot 7414 mlitn@deploy2003: mlitn: Backport for [[gerrit:1368165|Squashed diff to master]] (1.48.0-wmf.3) synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2479340 wikitext text/x-wiki i1gpzl60ch5lrt8aoxln1724p1b0l5x 2479341 2479340 2026-11-10T08:55:53Z Stashbot 7414 mlitn@deploy2003: mlitn: Continuing with deployment 2479341 wikitext text/x-wiki n8d39imklrtlfitr6oim5wq6r9kvibr 2479342 2479341 2026-11-10T09:01:53Z Stashbot 7414 mlitn@deploy2003: Finished scap sync-world: Backport for [[gerrit:1368165|Squashed diff to master]] (1.48.0-wmf.3) (duration: 13m 06s) 2479342 wikitext text/x-wiki npuyvwkggima2xtsh0kmibsvq37rbxw 2479343 2479342 2026-11-10T09:02:52Z Stashbot 7414 matthiasmullie: UTC morning backport window done 2479343 wikitext text/x-wiki 1x5bwdkd5tbvn1rsw147f180ob04pf5 2479346 2479343 2026-11-10T09:22:12Z Stashbot 7414 XioNoX: set graceful shutdown on cr1-esams transit/ix bgp - T440349 2479346 wikitext text/x-wiki lqa2js5jpg4uvtncyndmj806cm4t0oh 2479347 2479346 2026-11-10T09:28:57Z Stashbot 7414 aklapper@deploy2003: rebuilt and synchronized wikiversions files: group0 to 1.48.0-wmf.4 refs T442222 2479347 wikitext text/x-wiki hr001psiqaca33fpo3abssfq8tp533e 2479353 2479347 2026-11-10T10:35:56Z Stashbot 7414 ayounsi@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on 39 hosts with reason: Switch maintenance 2479353 wikitext text/x-wiki igpj8yb91d2tlai5mgx74jto17q7ygj 2479354 2479353 2026-11-10T10:37:29Z Stashbot 7414 ayounsi@cumin1004: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on asw1-bw27-esams,asw1-bw27-esams IPv6,asw1-bw27-esams.mgmt with reason: Switch maintenance 2479354 wikitext text/x-wiki asqktiwmsgy3l4hd1sxnru6d5wmnp1a 2479355 2479354 2026-11-10T10:54:41Z Stashbot 7414 ayounsi@cumin1004: START - Cookbook sre.dns.netbox 2479355 wikitext text/x-wiki s0duej5dleu36gu4hqbkod9drnk1x7m 2479356 2479355 2026-11-10T10:57:20Z Stashbot 7414 ayounsi@cumin1004: END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) 2479356 wikitext text/x-wiki knwi4w4pr9mui83xvt8drfxijp6q33u 2479357 2479356 2026-11-10T11:35:57Z Stashbot 7414 ayounsi@cumin1004: START - Cookbook sre.dns.netbox 2479357 wikitext text/x-wiki 5vmejscm54kv7g91ehe8fpbaobn5vpa 2479358 2479357 2026-11-10T11:41:19Z Stashbot 7414 ayounsi@cumin1004: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: new cr1-esams - ayounsi@cumin1004" 2479358 wikitext text/x-wiki 9dps8zm77lwdfzvy82i7jlqd5gmq3ub 2479359 2479358 2026-11-10T11:42:41Z Stashbot 7414 ayounsi@cumin1004: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: new cr1-esams - ayounsi@cumin1004" 2479359 wikitext text/x-wiki ehkh84vu2wnses07181szgbo6vlra87 2479360 2479359 2026-11-10T11:42:43Z Stashbot 7414 ayounsi@cumin1004: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2479360 wikitext text/x-wiki lddshud562omvtb6j34vuotb42ou3hq 2479361 2479360 2026-11-10T11:49:00Z Stashbot 7414 jhancock@cumin2003: START - Cookbook sre.hosts.provision for host cp3074.mgmt.esams.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2479361 wikitext text/x-wiki k3xtjgtppj01grcg9x7von3jgwgk28t 2479362 2479361 2026-11-10T11:55:48Z Stashbot 7414 gmodena@deploy2003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2479362 wikitext text/x-wiki 6pvc9stkb3pnso7yhgzitzatg0s9gop 2479363 2479362 2026-11-10T11:56:00Z Stashbot 7414 gmodena@deploy2003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2479363 wikitext text/x-wiki qrnk7gzaonan83pc2pm4s2hm8lxhunc 0 17290 2479269 2479062 2026-11-09T20:19:10Z Stashbot 7414 jeena: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/1367888 2479269 wikitext text/x-wiki 78x3xpadh21uvx9f1ljbp1f8xvb63gj 2479287 2479269 2026-11-09T21:30:39Z Stashbot 7414 dancy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1367952 2479287 wikitext text/x-wiki izwxfp9p8pkna9tv9k321wi3xi1xi9u 2479292 2479287 2026-11-09T22:55:52Z Stashbot 7414 thcipriani: taking integration-castor06 down for iops upgrade 2479292 wikitext text/x-wiki 4rh9f24e7yne6ztevnsor7uzplqjkur 2479294 2479292 2026-11-09T23:21:52Z Stashbot 7414 thcipriani: castor back at 23:03 with 4x-iops volume 2479294 wikitext text/x-wiki q0l51st8r7yodo8shfanagjtjkhnkbd 13 311299 2479314 2479028 2026-11-10T02:27:12Z EggRoll97 37347 /* Nominations */ + 2479314 wikitext text/x-wiki h32uny4elgz3apxh4u3tso34sordlbo 2479338 2479314 2026-11-10T08:45:45Z JJMC89 7474 /* November 2026 committee nominations */ counter 2479338 wikitext text/x-wiki 5hgohc1xg56xldvtk5vgncbctwfl5mt 498 443628 2479291 2479052 2026-11-09T22:45:07Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42714216808 (https://github.com/cluebotng/component-configs/commits/162553adae765260d7a2ab19fba838f5c2818b17) 2479291 wikitext text/x-wiki lw4uh96p8xybuvy6tucg37sjhw5ee0k 2479350 2479291 2026-11-10T09:58:39Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42794675366 (https://github.com/cluebotng/component-configs/commits/7ddb6d22ce0087e9b0810178168e9b131183f6ce) 2479350 wikitext text/x-wiki joqg3nbcz3s3r05ob22otz6xkb31ss3 498 443949 2479163 2479069 2026-11-09T12:42:53Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy d891c0a (Log save outcome as SSE step with diff link) 2479163 wikitext text/x-wiki bgethzuovxlpsfbrc1lxfgww6agkcgd 2479164 2479163 2026-11-09T13:57:14Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy aa5a5a7 (Cap pywikibot max_retries to 5 on web-triggered updates) 2479164 wikitext text/x-wiki glf4m3xsbb66qq80ugje4o6vox0sbv9 2479168 2479164 2026-11-09T14:05:29Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 573ba56 (Configure uwsgi to not log /healthz health-check endpoint) 2479168 wikitext text/x-wiki ksyn5wj53c0wzrvouymwggml7251wrz 2479192 2479168 2026-11-09T15:06:30Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 9978e50 (Add get_previous_grouping_counts to DashboardRegistry) for T328534 2479192 wikitext text/x-wiki k51jd8jhwb3qhl9n9lqajdovd1jl3xz 2479193 2479192 2026-11-09T15:06:32Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 526c485 (Add format_diff_cell and format_position_cell to Grouping) for T328534 2479193 wikitext text/x-wiki qq3h6bo2z0of5mt4e8qij6t7gi2cebq 2479194 2479193 2026-11-09T15:06:35Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy daa1424 (Update ResultsFormatter: add Pos and Δ columns when diffs present) for T328534 2479194 wikitext text/x-wiki mdvqo5pfzsnl3qw9kvhm7rpq09was4z 2479195 2479194 2026-11-09T15:06:37Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy e476365 (Fetch previous counts in process_page, compute diff/rank maps) for T328534 2479195 wikitext text/x-wiki g81wdm60541z6pxgx82oci1p7qbq3pr 2479196 2479195 2026-11-09T15:06:40Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 0546c11 (Test that ReportParser is transparent to Pos and Δ columns) for T328534 2479196 wikitext text/x-wiki agvjvtbfti9saji3sfi85w1g6ofbgwx 2479260 2479196 2026-11-09T18:23:38Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 8f7638e (Log the wikitext to be saved as a collapsible fold in the update stream) 2479260 wikitext text/x-wiki lv65prgdpyfft53sstbm2rwp9la8ijv 2479267 2479260 2026-11-09T19:49:43Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 47dedf7 (Propagate pywikibot save errors instead of swallowing them) 2479267 wikitext text/x-wiki lgcoq2hcfql7b2qlgd5gik8un5of0m7 2479275 2479267 2026-11-09T20:57:44Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 6037e1d (Show +N delta for new groupings (★ entries)) for T328534 2479275 wikitext text/x-wiki 9xvn34gmcpto4fnq1xtxrbrrpeajd22 0 445760 2479162 2465893 2026-11-09T12:35:30Z HNowlan (WMF) 15760 /* What the jobs do */ Attempt to document jobs that create other jobs 2479162 wikitext text/x-wiki 8uk9r4j3omsskfbnqxsc7pkqrqext4y 498 446211 2479290 2479048 2026-11-09T22:44:06Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42714216923 (https://github.com/cluebotng/component-configs/commits/162553adae765260d7a2ab19fba838f5c2818b17) 2479290 wikitext text/x-wiki flojyhbei0bwxpz029zf93ywgk9g88e 2479349 2479290 2026-11-10T09:57:17Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42794587529 (https://github.com/cluebotng/component-configs/commits/cb08182a83bfbb1987b8a4b271e6d9e57dd58c5c) 2479349 wikitext text/x-wiki 9petw87rl63oc3tx63ta6r29bvor7x2 0 449079 2479268 2440084 2026-11-09T20:15:08Z EEggleston (WMF) 41072 /* CiviCRM */ update install options, add troubleshooting section. 2479268 wikitext text/x-wiki i7r5tipxcvbfsb4e6nn1vq4pqfh6anj 2479272 2479268 2026-11-09T20:22:51Z EEggleston (WMF) 41072 /* CiviCRM */ 2479272 wikitext text/x-wiki 0ojxx9mtttpw9nkc0zjxiwgy8g207bi 0 449160 2479299 2479076 2026-11-10T00:02:19Z Dexbot 30554 Bot: Updating the report 2479299 wikitext text/x-wiki 6yiyljicmq5uqkyue1q3rov9cew6z32 12 452086 2479243 2478925 2026-11-09T17:11:51Z AndrewBogott (WMF) 51047 /* Driver Selection */ 2479243 wikitext text/x-wiki 80vo95ynj78vgevpy1u2xhwa7m817kp 116 453906 2479300 2479007 2026-11-10T00:03:14Z Gitlabaccountapprovalbot 37332 fl0per was rejected. 2479300 wikitext text/x-wiki b8jpmxnbidfgywcklo6usinhmx62dnb 2479345 2479300 2026-11-10T09:15:45Z Gitlabaccountapprovalbot 37332 @fsesini was approved. 2479345 wikitext text/x-wiki bide70gmu3a8u36053c3w96k0h2fwuu 0 456374 2479185 2450500 2026-11-09T14:46:19Z SFaci-WMF 35637 2479185 wikitext text/x-wiki fx989fumqio981cug3yumtild0eucd0 498 459244 2479174 2479061 2026-11-09T14:20:39Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42505001795 (https://github.com/cluebotng/component-configs/commits/3d315d295fbd9ddb713c5f5abe3e6e921e02f904) 2479174 wikitext text/x-wiki ofg20rb678nmlabb6ev37ke8oagzfdc 0 459699 2479233 2478560 2026-11-09T16:03:30Z Stashbot 7414 larssandergreen: civicrm upgraded from 5ba9d8e4 to 16578cc3 2479233 wikitext text/x-wiki 2mbfktxgynrqeb04zqotdmkz0nvqwtu 2479273 2479233 2026-11-09T20:38:24Z Stashbot 7414 eileen: civicrm upgraded from 16578cc3 to 7dcfeead 2479273 wikitext text/x-wiki n4gfgxe411ikg0kbtyqpc0u9i9vuvm7 2479274 2479273 2026-11-09T20:40:20Z Stashbot 7414 eileen: civicrm upgraded from 16578cc3 to 7dcfeead 2479274 wikitext text/x-wiki smrljzaoc8vzdph1tfaa007mt2w29k2 2479310 2479274 2026-11-10T01:22:44Z Stashbot 7414 dwisehaupt: now running the new slander code which should have fewer chances for edge case crashes. 2479310 wikitext text/x-wiki ps6rmb73jo1fgqlk51aeb8j24ptricj 2479313 2479310 2026-11-10T02:15:22Z Stashbot 7414 eileen: civicrm upgraded from 7dcfeead to 59f73d65 2479313 wikitext text/x-wiki ijjyz1gl66anjrbdffualhegpkx271f 2479321 2479313 2026-11-10T05:32:08Z Stashbot 7414 eileen: civicrm upgraded from 59f73d65 to 5a020f06 2479321 wikitext text/x-wiki 51hhfbjdibl99t0sbitwmrzzz05dcen 2479322 2479321 2026-11-10T06:22:10Z Stashbot 7414 eileen: civicrm upgraded from 5a020f06 to 3360fb3f 2479322 wikitext text/x-wiki cdhd1cctfhr4apzef6y20jlax2cv4cr 0 460232 2479172 2461450 2026-11-09T14:16:50Z MPopov (WMF) 5891 Unlisting dimensions that are not available anymore 2479172 wikitext text/x-wiki lmf84hiya71swamh7r1lvbu1ptuht1x 0 460608 2479158 2479083 2026-11-09T12:08:33Z APizzata-WMF 47941 2479158 wikitext text/x-wiki 81fo8koauw5qpb6fpoui9ipxdj8e5wu 1 460842 2479261 2478926 2026-11-09T18:28:01Z B.Zsolt 40247 /* AWB error */ new section 2479261 wikitext text/x-wiki 7wol4u1yr3ulqvlhkxearabpgaphexa 2 460954 2479178 2026-11-09T14:35:18Z Krinkle 44 Created page with "{{See also|CDN/History|Data centers}} == 2016: Status quo == Stack: * Edge: Varnish frontend * Edge: Varnish backend * Primary DC: Varnish backend tier 2 * MediaWiki: Primary DC Input: * 4 edge DCs (eqiad, codfw, esams, ulsfo) * x 10 cache_text nodes * = 40 Reduction: * /10=4: Varnish frontend to Varnish backend by URL hash * /4=1: Edge DC to primary DC by tier 2 Varnish backend Changes this year (no impact on capacity or load): * The <code>cache_mobile</code> cl..." 2479178 wikitext text/x-wiki o3yqqrscpog45y8h9pdirjai06zumlk 3 460955 2479224 2026-11-09T15:42:16Z StrikerBot 8475 Welcome to Toolforge! 2479224 wikitext text/x-wiki 699avufzu6a1zhscu4a0s78t7unamdg 498 460956 2479264 2026-11-09T19:10:59Z Stashbot 7414 wmbot~nokibsarkar@tools-bastion-15: webservice restart 2479264 wikitext text/x-wiki jllhbv8xl6z394pq34ihtn4aoppq4fj 0 460957 2479288 2026-11-09T21:59:12Z GModena (WMF) 20194 Created page with "initial change" 2479288 wikitext text/x-wiki ds7tdiyh8wkz0fuatvhnnwdrwjq6hqk 2479289 2479288 2026-11-09T22:14:44Z GModena (WMF) 20194 Add alert runbook 2479289 wikitext text/x-wiki dz07jlgaq2o0zo1p2gqkcmbgw9fnamu 3 460958 2479344 2026-11-10T09:03:43Z StrikerBot 8475 Welcome to Toolforge! 2479344 wikitext text/x-wiki 87xfvju9e1oxqr47tcxmo5ihofxa1hg 498 460959 2479348 2026-11-10T09:49:25Z Labslogbot 55 Auto update of instance info. 2479348 wikitext text/x-wiki 1dws6q4l56sesti2c7q0d66aeokd1az