Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.14 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 0 4108 2445463 2445296 2026-08-11T13:19:16Z ScheduleDeploymentBot 37566 Add [[gerrit:1313985]] to Tuesday, August 11 UTC afternoon backport window 2445463 wikitext text/x-wiki 9hnd5p1j954c5lo55dy5xmtjdoqxmpy 2445517 2445463 2026-08-11T14:10:04Z Superpes15 24252 /* {{Deployment_day|date=2026-08-17}} */ +1 2445517 wikitext text/x-wiki 02i3pwr1o3c5pk07d1q26sua0q8kdlt 2445536 2445517 2026-08-11T14:32:06Z ScheduleDeploymentBot 37566 Add [[gerrit:1275938]] to Wednesday, August 12 UTC afternoon backport window 2445536 wikitext text/x-wiki tig2prknim0tlik6nfbe6nbkwzz9nrv 2445668 2445536 2026-08-11T20:06:54Z ScheduleDeploymentBot 37566 Add [[gerrit:1324384]] to Tuesday, August 11 UTC late backport window 2445668 wikitext text/x-wiki k4b4i884n2qimdv5ebtc5bvhyegkgg2 2445669 2445668 2026-08-11T20:06:59Z ScheduleDeploymentBot 37566 Add [[gerrit:1324385]] to Tuesday, August 11 UTC late backport window 2445669 wikitext text/x-wiki 38cs7r5q5bulsver4pd2g2r20ipcyun 2445711 2445669 2026-08-12T00:30:20Z ScheduleDeploymentBot 37566 Add [[gerrit:1321224]] to Wednesday, August 12 UTC afternoon backport window 2445711 wikitext text/x-wiki e2epy9x2h77k6h1ip1yg3yngeipwiyq 2445712 2445711 2026-08-12T00:31:16Z Daimona Eaytoy 11462 /* {{Deployment_day|date=2026-08-12}} */ +data migration 2445712 wikitext text/x-wiki kdxz6qskvytv9lm5vffyi3iq73kee2u 498 6726 2445888 2445026 2026-08-12T11:55:05Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-cli 2445888 wikitext text/x-wiki q7n11mga3hpmb940p4yhwgmyrnx84zt 2445889 2445888 2026-08-12T11:55:10Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-cli 2445889 wikitext text/x-wiki 14ro9u0r0qxx0stzzfgpzqqr7jn7cbs 0 7919 2445411 2445410 2026-08-11T12:00:22Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324283|Use maximum compression level in SqlBlobStore and SqlBagOStuff (T428377)]] (duration: 11m 37s) 2445411 wikitext text/x-wiki ofxpd5jpved402lh8ft361o4f8no32v 2445412 2445411 2026-08-11T12:03:17Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4005.ulsfo.wmnet 2445412 wikitext text/x-wiki kxobr57as83dl6wr7fljhxto0tjceub 2445413 2445412 2026-08-11T12:03:35Z Stashbot 7414 jmm@cumin2003: END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti4005.ulsfo.wmnet 2445413 wikitext text/x-wiki mxf026i4ebtlqrsjkec2vwdne5hikr1 2445414 2445413 2026-08-11T12:04:06Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2033.codfw.wmnet 2445414 wikitext text/x-wiki 7r5s7y4634vbfcb7hwz4o6gwtbv7i4w 2445416 2445414 2026-08-11T12:07:39Z Stashbot 7414 moritzm: failover ganeti master in ganeti/test to ganeti-test2003 2445416 wikitext text/x-wiki frx2h9nmnw1k28yhtnd4l42lymd6437 2445417 2445416 2026-08-11T12:14:09Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2033.codfw.wmnet 2445417 wikitext text/x-wiki gjwvt9jswci3yqt27ly7l3lz1n183j8 2445418 2445417 2026-08-11T12:18:21Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1008.eqiad.wmnet with OS bookworm 2445418 wikitext text/x-wiki r8i55huvdnqk5vsilrzqojw1epuziud 2445419 2445418 2026-08-11T12:18:39Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti-test2001.codfw.wmnet 2445419 wikitext text/x-wiki szq4egk047fkn7u8bbyw1nijjhrw3xz 2445420 2445419 2026-08-11T12:18:45Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1009.eqiad.wmnet with OS bookworm 2445420 wikitext text/x-wiki 5ho5fa5kc5qps4ir3xo8hqehir7pnqk 2445421 2445420 2026-08-11T12:19:07Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet 2445421 wikitext text/x-wiki png1v5pl388uot3pt1rsa3kkoniy957 2445422 2445421 2026-08-11T12:22:13Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2033.codfw.wmnet 2445422 wikitext text/x-wiki 7cr0v0ja9p5ru11siagcy6yr96r1mau 2445423 2445422 2026-08-11T12:22:16Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2033.codfw.wmnet 2445423 wikitext text/x-wiki 9dqi7z33ohk3x0f5db15uxr85x2ecc0 2445424 2445423 2026-08-11T12:22:53Z Stashbot 7414 moritzm: failover ganeti master in codfw/routed to ganeti2033 2445424 wikitext text/x-wiki ctn4emb2fn5x4bq6x8xfnz9rch7urn0 2445425 2445424 2026-08-11T12:23:42Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-coord1003.eqiad.wmnet with OS bookworm 2445425 wikitext text/x-wiki 11rl4ro07cecjcw552ln1fx7bzxhtvq 2445426 2445425 2026-08-11T12:25:36Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1179: Pool db1179.eqiad.wmnet in after cloning 2445426 wikitext text/x-wiki av01vph08aat6wxweu4j2e681drcg6u 2445427 2445426 2026-08-11T12:25:51Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1179.eqiad.wmnet onto db1278.eqiad.wmnet 2445427 wikitext text/x-wiki c3j6s5sclsj4f3v5qgzjpz42fijb53l 2445428 2445427 2026-08-11T12:30:25Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) for host ganeti-test2001.codfw.wmnet 2445428 wikitext text/x-wiki euz42xjx5ncuyw3nnjrrwct1fmg4ukm 2445429 2445428 2026-08-11T12:30:28Z Stashbot 7414 jmm@cumin2002: END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti-test2001.codfw.wmnet 2445429 wikitext text/x-wiki duxqz8qrq5nw6n5agi1gt1r8egr28og 2445430 2445429 2026-08-11T12:34:38Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2034.codfw.wmnet 2445430 wikitext text/x-wiki s58ly6qqv1jsqia7pvtjicpi0i07so7 2445431 2445430 2026-08-11T12:37:48Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-staging-codfw] 'sync' command on namespace 'liftwing-openapi-server' for release 'main' . 2445431 wikitext text/x-wiki 71qafiroim2hcau67d715qz8ui0p3bo 2445432 2445431 2026-08-11T12:38:14Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-serve-eqiad] 'sync' command on namespace 'liftwing-openapi-server' for release 'main' . 2445432 wikitext text/x-wiki boxuzczgml8aj8rto6uzxdx780ifzd6 2445433 2445432 2026-08-11T12:38:55Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-serve-codfw] 'sync' command on namespace 'liftwing-openapi-server' for release 'main' . 2445433 wikitext text/x-wiki 0c7hu4mx3w5p1fuxtpv8ewjc6qtzn9u 2445434 2445433 2026-08-11T12:39:50Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-staging-codfw] 'sync' command on namespace 'ores-legacy' for release 'main' . 2445434 wikitext text/x-wiki 7ob1f5w03usg9jvd9mvpraa6ledsj5e 2445435 2445434 2026-08-11T12:40:37Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-coord1003.eqiad.wmnet with reason: host reimage 2445435 wikitext text/x-wiki awyu1yfzl1hyr3eyf9ilit0b2rtij12 2445436 2445435 2026-08-11T12:42:07Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2165: Security update 2445436 wikitext text/x-wiki 4tdqxlqj28n2017hqep9g8rjqyimyie 2445437 2445436 2026-08-11T12:42:15Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-serve-eqiad] 'sync' command on namespace 'ores-legacy' for release 'main' . 2445437 wikitext text/x-wiki swsz0vpr1vca1d7v2vzkkkct0kyzqyv 2445438 2445437 2026-08-11T12:43:11Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-serve-codfw] 'sync' command on namespace 'ores-legacy' for release 'main' . 2445438 wikitext text/x-wiki 11q5axd79r8r46uy0wy18pk6gc6o03h 2445439 2445438 2026-08-11T12:43:42Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-coord1003.eqiad.wmnet with reason: host reimage 2445439 wikitext text/x-wiki 2l84z48gk1vdq37xj5jh52xv4xm03zx 2445440 2445439 2026-08-11T12:43:58Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-staging-codfw] 'sync' command on namespace 'recommendation-api-ng' for release 'main' . 2445440 wikitext text/x-wiki mamkxicov1exm8d3h8elr61a53cl6g2 2445441 2445440 2026-08-11T12:44:30Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-serve-eqiad] 'sync' command on namespace 'recommendation-api-ng' for release 'main' . 2445441 wikitext text/x-wiki qmhqi37xs31j21i06u3m1nzck5wpsqa 2445442 2445441 2026-08-11T12:45:00Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-serve-codfw] 'sync' command on namespace 'recommendation-api-ng' for release 'main' . 2445442 wikitext text/x-wiki ss1vsq4i07do5tnqsdsxman1ib11bs5 2445443 2445442 2026-08-11T12:45:32Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-staging-codfw] 'sync' command on namespace 'tts-section-generator' for release 'main' . 2445443 wikitext text/x-wiki 53sb0bailoii6ldrdqrn6ufu3bt69xo 2445444 2445443 2026-08-11T12:46:16Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-serve-eqiad] 'sync' command on namespace 'tts-section-generator' for release 'main' . 2445444 wikitext text/x-wiki kh7ovptaukfqd753av874x8ge7qph70 2445445 2445444 2026-08-11T12:47:22Z Stashbot 7414 bwojtowicz@deploy1003: helmfile [ml-serve-codfw] 'sync' command on namespace 'tts-section-generator' for release 'main' . 2445445 wikitext text/x-wiki qrf2lzxzl2zi578j9js2nwq8ai6tohh 2445446 2445445 2026-08-11T12:51:32Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool ms2 T434288', diff saved to https://phabricator.wikimedia.org/P95977 and previous config saved to /var/cache/conftool/dbconfig/20260811-125129-marostegui.json 2445446 wikitext text/x-wiki 4aaqu01x28j6zaf8g4nuj8m0n2vjyt0 2445448 2445446 2026-08-11T12:53:33Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db1157: Using as clone source 2445448 wikitext text/x-wiki bcuh5jptwmczjcwijw19dokk8n0b3hw 2445449 2445448 2026-08-11T12:54:11Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1157: Using as clone source 2445449 wikitext text/x-wiki nj1s3m2kcg0wt5af8rtiwz1pgtft9d4 2445450 2445449 2026-08-11T12:59:02Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db2253.codfw.wmnet,db[1151,1266].eqiad.wmnet with reason: Switching over ms2 2445450 wikitext text/x-wiki 0xdeeqpyo4f1imsp23vodotlno75z0v 2445451 2445450 2026-08-11T13:04:17Z Stashbot 7414 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1323779|[ukwiki] Remove reviewer usergroup (T434252)]], [[gerrit:1323312|[frwiktionary] Add new Schème namespace and its talk (T415716)]] 2445451 wikitext text/x-wiki hj66zj54tyaa6ymvzfe2d9n3k6nayl3 2445452 2445451 2026-08-11T13:06:16Z Stashbot 7414 urbanecm@deploy1003: urbanecm, superpes: Backport for [[gerrit:1323779|[ukwiki] Remove reviewer usergroup (T434252)]], [[gerrit:1323312|[frwiktionary] Add new Schème namespace and its talk (T415716)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445452 wikitext text/x-wiki l4et46tn2rajyuu28babanrtc1zgb1f 2445453 2445452 2026-08-11T13:06:30Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1266 to dbctl T434288', diff saved to https://phabricator.wikimedia.org/P95979 and previous config saved to /var/cache/conftool/dbconfig/20260811-130627-marostegui.json 2445453 wikitext text/x-wiki 6o8fh2a1buyh0rwdhqfjp82rh4qm19e 2445454 2445453 2026-08-11T13:06:59Z Stashbot 7414 urbanecm@deploy1003: urbanecm, superpes: Continuing with deployment 2445454 wikitext text/x-wiki 7x8z997j9qr4lgmks5wcusn7z6l000j 2445455 2445454 2026-08-11T13:07:28Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Repool ms2 T434288', diff saved to https://phabricator.wikimedia.org/P95980 and previous config saved to /var/cache/conftool/dbconfig/20260811-130725-marostegui.json 2445455 wikitext text/x-wiki 114mv8o31dogshiy6g2l1sfcu5cb1sj 2445456 2445455 2026-08-11T13:08:22Z Stashbot 7414 marostegui@dns1004: START - running authdns-update 2445456 wikitext text/x-wiki icdhjbnn6o6amqjkyk6o33pg1ahi75o 2445457 2445456 2026-08-11T13:10:15Z Stashbot 7414 marostegui@dns1004: END - running authdns-update 2445457 wikitext text/x-wiki i5vpjroz6vf97lw81fxjrtak3zsbbjq 2445458 2445457 2026-08-11T13:11:07Z Stashbot 7414 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1323779|[ukwiki] Remove reviewer usergroup (T434252)]], [[gerrit:1323312|[frwiktionary] Add new Schème namespace and its talk (T415716)]] (duration: 06m 49s) 2445458 wikitext text/x-wiki 06fvai0u7jyon7pnsjfwpsmgoj47w5a 2445459 2445458 2026-08-11T13:11:32Z Stashbot 7414 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1323350|[tgwiki] Remove temporary logo for Wikipedia 25 (Vector 2022 + Vector Legacy) (T415307)]], [[gerrit:1322961|[slwiki] Revert temporary logo for Wikipedia 25 (Vector legacy + Vector 2022) (T414265)]], [[gerrit:1323827|[itwiki] Remove temporary logo for Wikipedia 25 (Vector 2022 + Vector Legacy) (T414320)]] 2445459 wikitext text/x-wiki a698bmu5dx7f8ff9gukt2izynor6oki 2445460 2445459 2026-08-11T13:13:36Z Stashbot 7414 urbanecm@deploy1003: urbanecm, superpes: Backport for [[gerrit:1323350|[tgwiki] Remove temporary logo for Wikipedia 25 (Vector 2022 + Vector Legacy) (T415307)]], [[gerrit:1322961|[slwiki] Revert temporary logo for Wikipedia 25 (Vector legacy + Vector 2022) (T414265)]], [[gerrit:1323827|[itwiki] Remove temporary logo for Wikipedia 25 (Vector 2022 + Vector Legacy) (T414320)]] synced to the testservers (see https://wiki 2445460 wikitext text/x-wiki 2r25de9uzue9bz76xbvl1jz50aww9un 2445461 2445460 2026-08-11T13:15:25Z Stashbot 7414 urbanecm@deploy1003: urbanecm, superpes: Continuing with deployment 2445461 wikitext text/x-wiki adu7nghvzxeeodhre4e1dneh42msvi6 2445462 2445461 2026-08-11T13:19:07Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-coord1003.eqiad.wmnet with OS bookworm 2445462 wikitext text/x-wiki t3o87n9wvp3kkwj02ztpqguhvwmaamf 2445464 2445462 2026-08-11T13:19:31Z Stashbot 7414 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1323350|[tgwiki] Remove temporary logo for Wikipedia 25 (Vector 2022 + Vector Legacy) (T415307)]], [[gerrit:1322961|[slwiki] Revert temporary logo for Wikipedia 25 (Vector legacy + Vector 2022) (T414265)]], [[gerrit:1323827|[itwiki] Remove temporary logo for Wikipedia 25 (Vector 2022 + Vector Legacy) (T414320)]] (duration: 08m 00s) 2445464 wikitext text/x-wiki 1e0pwph4xffe3etwa3bw2tq7zsds1n4 2445465 2445464 2026-08-11T13:20:59Z Stashbot 7414 urbanecm@deploy1003: mwscript-k8s job started: namespaceDupes.php --wiki=frwiktionary # T415716 2445465 wikitext text/x-wiki 7f7jxrspb3ni9xo5wiqiwtci4tj7gm5 2445466 2445465 2026-08-11T13:21:08Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2034.codfw.wmnet 2445466 wikitext text/x-wiki ounxabbux7yhs58ifyoewjss7bvdsw6 2445467 2445466 2026-08-11T13:21:14Z Stashbot 7414 urbanecm@deploy1003: mwscript-k8s job started: namespaceDupes.php --wiki=frwiktionary --fix # T415716 2445467 wikitext text/x-wiki 8rt41ydo0g3reoqautfu0zk2odj8sq3 2445468 2445467 2026-08-11T13:25:30Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.clone of db1157.eqiad.wmnet onto db1272.eqiad.wmnet 2445468 wikitext text/x-wiki cy9iq5hhdg8dl5h8q5slvtp9uty9l5d 2445469 2445468 2026-08-11T13:25:38Z Stashbot 7414 cwilliams@cumin1003: END (FAIL) - Cookbook sre.mysql.clone (exit_code=99) of db1157.eqiad.wmnet onto db1272.eqiad.wmnet 2445469 wikitext text/x-wiki qxnmzidvpyspfefn8u5gahrkgphb8j6 2445470 2445469 2026-08-11T13:29:08Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2034.codfw.wmnet 2445470 wikitext text/x-wiki n6d161ga4x6q5bx6uy6fbziexq57vk7 2445471 2445470 2026-08-11T13:29:15Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2034.codfw.wmnet 2445471 wikitext text/x-wiki 19tavshj5qmsk3hp2utqfok5nuxcak0 2445473 2445471 2026-08-11T13:34:24Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1313985|wmf-config/ProductionServices: set URL for urldownloader to service record (T429175)]] 2445473 wikitext text/x-wiki i086bpvb91g5o8nmxh2q4741ir3vq7n 2445474 2445473 2026-08-11T13:36:23Z Stashbot 7414 kharlan@deploy1003: kharlan, sukhe: Backport for [[gerrit:1313985|wmf-config/ProductionServices: set URL for urldownloader to service record (T429175)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445474 wikitext text/x-wiki 0nzrnxopesa993d38awzgui0um4nwr0 2445475 2445474 2026-08-11T13:37:01Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4005.ulsfo.wmnet 2445475 wikitext text/x-wiki 30hrfr29qzebhovvxvwwyeb9sdfp8cg 2445476 2445475 2026-08-11T13:38:26Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-presto1008.eqiad.wmnet with OS bookworm 2445476 wikitext text/x-wiki es6nbdls18bfdcingq0ag1ke3n1auhc 2445477 2445476 2026-08-11T13:38:49Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reimage for host dns7001.wikimedia.org with OS trixie 2445477 wikitext text/x-wiki 855keogwm56adngejw4qjrliiq0wmrl 2445478 2445477 2026-08-11T13:39:25Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reimage for host dns4003.wikimedia.org with OS trixie 2445478 wikitext text/x-wiki roihl1kov7ziov52wap75o025xo55av 2445479 2445478 2026-08-11T13:40:36Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1008.eqiad.wmnet with OS bookworm 2445479 wikitext text/x-wiki olxcjlugtgbcnlwv2d8ptfatd84u2fm 2445481 2445479 2026-08-11T13:44:53Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti4005.ulsfo.wmnet 2445481 wikitext text/x-wiki 1700n7ifi5o9ewe4zb5scwpbmycod5y 2445482 2445481 2026-08-11T13:48:30Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti4005.ulsfo.wmnet 2445482 wikitext text/x-wiki dbyz39eyysdcfi41o2pqcc5f7jnd5fz 2445483 2445482 2026-08-11T13:48:55Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4005.ulsfo.wmnet 2445483 wikitext text/x-wiki 5xnad3dx0b9u3dj0lejdcr2fzb92zy0 2445484 2445483 2026-08-11T13:51:53Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4006.ulsfo.wmnet 2445484 wikitext text/x-wiki i58ml3xzxn0p7ye9ibqkauziqzepxqo 2445485 2445484 2026-08-11T13:52:53Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7001.magru.wmnet 2445485 wikitext text/x-wiki ts7jt8qp9ueo12ysx39mpbbivx2vrvf 2445486 2445485 2026-08-11T13:52:57Z Stashbot 7414 marostegui: Failover ms1 T434288 2445486 wikitext text/x-wiki ql8hzkxhp8j3emib6akhn23ensze7to 2445487 2445486 2026-08-11T13:53:01Z Stashbot 7414 marostegui: Failover ms2 T434288 2445487 wikitext text/x-wiki hhlnsgrs0fpd887vrjbmatiorv1vn7d 2445489 2445487 2026-08-11T13:55:23Z Stashbot 7414 kharlan@deploy1003: kharlan, sukhe: Continuing with deployment 2445489 wikitext text/x-wiki g39kqx5rxw4usrbitd1s0n42uwbyjta 2445490 2445489 2026-08-11T13:55:52Z Stashbot 7414 jayme@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'sync'. 2445490 wikitext text/x-wiki kucnr5c3gp0loaxwl7sp1hv1p785i0q 2445491 2445490 2026-08-11T13:55:57Z Stashbot 7414 jayme@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'sync'. 2445491 wikitext text/x-wiki e8whjfydyz1cacos0rer9h88pyy3l4s 2445492 2445491 2026-08-11T13:56:00Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-presto1008.eqiad.wmnet with reason: host reimage 2445492 wikitext text/x-wiki gig8edajj0qzcrmjxvooqopyp5ctydp 2445493 2445492 2026-08-11T13:56:26Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7001.magru.wmnet 2445493 wikitext text/x-wiki m79v3skk4q3bp47hjzg7nbwgh7v8ucm 2445494 2445493 2026-08-11T13:56:52Z Stashbot 7414 jayme@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'sync'. 2445494 wikitext text/x-wiki 6b9f1z88b0zueoo2cvf24ohrdhdc3gs 2445495 2445494 2026-08-11T13:57:07Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on dns4003.wikimedia.org with reason: host reimage 2445495 wikitext text/x-wiki 0rmkgp37rmb8f0kueyuow5h6zeta41o 2445497 2445495 2026-08-11T13:58:02Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti4006.ulsfo.wmnet 2445497 wikitext text/x-wiki 849fhuebb2o88fwesdwg8hi2qenro66 2445498 2445497 2026-08-11T13:58:05Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7002.magru.wmnet 2445498 wikitext text/x-wiki mf5c37x0c258l85olgrqubwwf6j6ljp 2445499 2445498 2026-08-11T13:59:30Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1313985|wmf-config/ProductionServices: set URL for urldownloader to service record (T429175)]] (duration: 25m 06s) 2445499 wikitext text/x-wiki 21symggdvb6u82g4ugibeuiumrazu8u 2445500 2445499 2026-08-11T13:59:37Z Stashbot 7414 jayme@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'sync'. 2445500 wikitext text/x-wiki jxia2bkavcr8gwwn52yq95mudsnimkq 2445501 2445500 2026-08-11T13:59:59Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-presto1008.eqiad.wmnet with reason: host reimage 2445501 wikitext text/x-wiki dg2gz3ybbf5t1xes8yl8oki65xqjytj 2445502 2445501 2026-08-11T14:00:07Z Stashbot 7414 btullis@dns1004: START - running authdns-update 2445502 wikitext text/x-wiki tsm1qgkp11uagiojclj1vl9w68f8y3e 2445503 2445502 2026-08-11T14:02:04Z Stashbot 7414 btullis@dns1004: FAIL - running authdns-update 2445503 wikitext text/x-wiki qgpv3j918vk2g0q531qjs04tabdwd5c 2445504 2445503 2026-08-11T14:02:40Z Stashbot 7414 kharlan@deploy1003: Started scap sync-world: Backport for [[gerrit:1324318|Revert "wmf-config/ProductionServices: set URL for urldownloader to service record" (T429175)]] 2445504 wikitext text/x-wiki kca23nx795fh0qsy905gxeglnth7bpe 2445505 2445504 2026-08-11T14:02:42Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti7002.magru.wmnet 2445505 wikitext text/x-wiki msro1t4fgcmd2rp9hpjpqn0f2q33w0h 2445506 2445505 2026-08-11T14:03:21Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on dns7001.wikimedia.org with reason: host reimage 2445506 wikitext text/x-wiki m8lqlnj5lyje2cpn8a6ycizaukpm8ql 2445507 2445506 2026-08-11T14:03:52Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dns4003.wikimedia.org with reason: host reimage 2445507 wikitext text/x-wiki 6qs3kyit22m2qavm8dt07jatmxc7q4g 2445508 2445507 2026-08-11T14:04:46Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti4006.ulsfo.wmnet 2445508 wikitext text/x-wiki le7q8ref59xrab90r8dwme74zfi20bd 2445509 2445508 2026-08-11T14:04:50Z Stashbot 7414 kharlan@deploy1003: kharlan: Backport for [[gerrit:1324318|Revert "wmf-config/ProductionServices: set URL for urldownloader to service record" (T429175)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445509 wikitext text/x-wiki llg1ikqtbywcfumr4srtbwn3cokvo5l 2445510 2445509 2026-08-11T14:04:54Z Stashbot 7414 jayme: updated calico to v3.30.7 on staging-eqiad - T427400 2445510 wikitext text/x-wiki rz4ucjzwahqkpetc9keex5lt40lvhkz 2445511 2445510 2026-08-11T14:05:12Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4006.ulsfo.wmnet 2445511 wikitext text/x-wiki 5z597aayabcgrgm5gyqln9opb9pag77 2445512 2445511 2026-08-11T14:05:22Z Stashbot 7414 kharlan@deploy1003: kharlan: Continuing with deployment 2445512 wikitext text/x-wiki ml8bi5ptj25evs41wxdotxbxxcg8gwx 2445513 2445512 2026-08-11T14:06:38Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4007.ulsfo.wmnet 2445513 wikitext text/x-wiki d2r03py3nsy0poksg8u17j1afa51ed5 2445514 2445513 2026-08-11T14:07:17Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dns7001.wikimedia.org with reason: host reimage 2445514 wikitext text/x-wiki ax87wdn7i0erhtvp73wo05gbeudk2p1 2445515 2445514 2026-08-11T14:09:13Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-master1004.eqiad.wmnet with OS bookworm 2445515 wikitext text/x-wiki ttqhv8p2ycej73uyxv4uciojihc50xz 2445516 2445515 2026-08-11T14:09:27Z Stashbot 7414 kharlan@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324318|Revert "wmf-config/ProductionServices: set URL for urldownloader to service record" (T429175)]] (duration: 06m 46s) 2445516 wikitext text/x-wiki gf02166fdknk6ggya2ww1qbniyyltht 2445518 2445516 2026-08-11T14:11:02Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti7002.magru.wmnet 2445518 wikitext text/x-wiki k8fdfgb62rwrf52jvgzvnrgjcrsz12k 2445519 2445518 2026-08-11T14:11:08Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7002.magru.wmnet 2445519 wikitext text/x-wiki i8cfe6edzqrv4hkhqvzl56ory9agmjp 2445520 2445519 2026-08-11T14:12:42Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti4007.ulsfo.wmnet 2445520 wikitext text/x-wiki 0qm4iweiubsze4xvkbrt4yv2hbwkwjs 2445521 2445520 2026-08-11T14:12:45Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7003.magru.wmnet 2445521 wikitext text/x-wiki 86ztd66ymdjyq6goz9o7ajszz0eg2fh 2445522 2445521 2026-08-11T14:15:10Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti7003.magru.wmnet 2445522 wikitext text/x-wiki 7fe1k0lzs2sae2atawy6x0rv7taertz 2445524 2445522 2026-08-11T14:17:49Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-presto1008.eqiad.wmnet with OS bookworm 2445524 wikitext text/x-wiki ndqx6fnarnqrqmuzkfgshnm6jrwduii 2445525 2445524 2026-08-11T14:18:22Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1010.eqiad.wmnet with OS bookworm 2445525 wikitext text/x-wiki 5mijqtxqywgf2w7wplz3bfj14qlc6y5 2445526 2445525 2026-08-11T14:19:28Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti4007.ulsfo.wmnet 2445526 wikitext text/x-wiki rfwr1ls5c71qyl5t6gb12m35jjp6qn8 2445527 2445526 2026-08-11T14:19:52Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4007.ulsfo.wmnet 2445527 wikitext text/x-wiki chs8zstoqxq77tji23uavighkooz2sa 2445528 2445527 2026-08-11T14:21:38Z Stashbot 7414 btullis@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host an-master1004.eqiad.wmnet with OS bookworm 2445528 wikitext text/x-wiki 040x1430xdmldm17cf9nrjl2cv9itw7 2445529 2445528 2026-08-11T14:22:27Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.provision for host an-master1004.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2445529 wikitext text/x-wiki 4neoch3pgylnwwipd4qhee82qgrkqn1 2445530 2445529 2026-08-11T14:23:19Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-master1004.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2445530 wikitext text/x-wiki o6ddlmsxwkkpd3pa17hfaaibtujwkyd 2445531 2445530 2026-08-11T14:23:22Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti7003.magru.wmnet 2445531 wikitext text/x-wiki kkkef3xhcprctiui2g0pfai9u1gmcrq 2445532 2445531 2026-08-11T14:23:27Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7003.magru.wmnet 2445532 wikitext text/x-wiki lqkl9ygqllquh60qcmzap8ziyrw2mzm 2445533 2445532 2026-08-11T14:28:50Z Stashbot 7414 moritzm: failover ganeti master in ulsfo to ganeti4005 2445533 wikitext text/x-wiki 1l5y2plouj4i5zx74752l5e3x6bg4jf 2445534 2445533 2026-08-11T14:30:13Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7004.magru.wmnet 2445534 wikitext text/x-wiki iagsx3tim96to88doea8jnsjbdjeaay 2445535 2445534 2026-08-11T14:31:15Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.clone of db1157.eqiad.wmnet onto db1272.eqiad.wmnet 2445535 wikitext text/x-wiki ld87c8m1be7xydlsglz9au4x5hdoggl 2445537 2445535 2026-08-11T14:32:11Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-master1004.eqiad.wmnet with OS bookworm 2445537 wikitext text/x-wiki 5fg8ok0exy4yia8agp4w6pc7tuhy4i9 2445538 2445537 2026-08-11T14:32:41Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-presto1009.eqiad.wmnet with OS bookworm 2445538 wikitext text/x-wiki d4q0fd0n4n01isnj1dkgnsl4uxdq6bg 2445539 2445538 2026-08-11T14:36:42Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.provision for host an-presto1009.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2445539 wikitext text/x-wiki 092o3gg3x3q7i0bue7j0ytz4krap4f2 2445540 2445539 2026-08-11T14:39:10Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-presto1009.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2445540 wikitext text/x-wiki oqp7prrdz00mr02yr2vkiykkwc20pit 2445541 2445540 2026-08-11T14:39:31Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1009.eqiad.wmnet with OS bookworm 2445541 wikitext text/x-wiki 8cpe51eaxgrcpi38m71ttgwc4myb0xq 2445542 2445541 2026-08-11T14:40:36Z Stashbot 7414 btullis@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host an-presto1010.eqiad.wmnet with OS bookworm 2445542 wikitext text/x-wiki nlqkb8ho1rk49eqxy9zbcwk895hyk8s 2445543 2445542 2026-08-11T14:41:30Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.provision for host an-presto1010.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2445543 wikitext text/x-wiki n1r453930a0k2epbuzoewa21gppyy29 2445544 2445543 2026-08-11T14:44:22Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-presto1010.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL 2445544 wikitext text/x-wiki 25thz3093i51lm0ngcj7zftebqd97zr 2445545 2445544 2026-08-11T14:47:21Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dns7001.wikimedia.org with OS trixie 2445545 wikitext text/x-wiki sdrbw8d89ylja0xv5y2bvtxcbbr5s3b 2445546 2445545 2026-08-11T14:47:35Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1010.eqiad.wmnet with OS bookworm 2445546 wikitext text/x-wiki e9njo7h1ljoej2w7njpht9thg379lxa 2445547 2445546 2026-08-11T14:49:23Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dns4003.wikimedia.org with OS trixie 2445547 wikitext text/x-wiki 1m66gbgghttlymi9hha2km5xcr1kx4e 2445548 2445547 2026-08-11T14:55:49Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-presto1010.eqiad.wmnet with OS bookworm 2445548 wikitext text/x-wiki 347j1gwgnbpdhiedeiq0ca0uwy50wy1 2445550 2445548 2026-08-11T15:02:40Z Stashbot 7414 cwilliams@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 28 hosts with reason: Primary switchover s2 T434565 2445550 wikitext text/x-wiki rzwha4yvet4px2bj096fp2zj5uikeio 2445551 2445550 2026-08-11T15:04:06Z Stashbot 7414 cwilliams@cumin1003: dbctl commit (dc=all): 'Set db2207 with weight 0 T434565', diff saved to https://phabricator.wikimedia.org/P95982 and previous config saved to /var/cache/conftool/dbconfig/20260811-150402-cwilliams.json 2445551 wikitext text/x-wiki o0jhfq03wy29zjdcfg2z0x08drne7l5 2445553 2445551 2026-08-11T15:08:26Z Stashbot 7414 cezmunsta: Starting s2 codfw failover from db2204 to db2207 - T434565 2445553 wikitext text/x-wiki s8po84h3u02ncedo4y4hpxxkopthtut 2445554 2445553 2026-08-11T15:09:08Z Stashbot 7414 cwilliams@cumin1003: dbctl commit (dc=all): 'Promote db2207 to s2 primary T434565', diff saved to https://phabricator.wikimedia.org/P95983 and previous config saved to /var/cache/conftool/dbconfig/20260811-150905-cwilliams.json 2445554 wikitext text/x-wiki si8lp40jc4ea5giau5pczahko6og5bd 2445555 2445554 2026-08-11T15:10:03Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti7004.magru.wmnet 2445555 wikitext text/x-wiki n9nzx5y9ab07xszglqzhovhcgq0vvrc 2445556 2445555 2026-08-11T15:10:39Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4008.ulsfo.wmnet 2445556 wikitext text/x-wiki 8qrxb1zsqirhqqzqqzwgq1722bzbjty 2445557 2445556 2026-08-11T15:11:30Z Stashbot 7414 cwilliams@cumin1003: dbctl commit (dc=all): 'Depool db2204 T434565', diff saved to https://phabricator.wikimedia.org/P95984 and previous config saved to /var/cache/conftool/dbconfig/20260811-151126-cwilliams.json 2445557 wikitext text/x-wiki 20f6wvq575wjfpaftkemzibzf7a72dm 2445558 2445557 2026-08-11T15:11:38Z Stashbot 7414 hashar: Restarting CI Jenkins on contint1003 due to Java upgrade. 2445558 wikitext text/x-wiki dl2kg56o8w38lglu7q0znsb1w4696j0 2445559 2445558 2026-08-11T15:11:59Z Stashbot 7414 btullis@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host an-master1004.eqiad.wmnet with OS bookworm 2445559 wikitext text/x-wiki nz5volprvx3v0hd6dcmm68x7ffmzi2w 2445560 2445559 2026-08-11T15:12:37Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-master1004.eqiad.wmnet with OS bookworm 2445560 wikitext text/x-wiki cmcyuyklplet5xy1q5m37apsz12d4ur 2445561 2445560 2026-08-11T15:14:49Z Stashbot 7414 cwilliams@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2204.codfw.wmnet with reason: Maintenance 2445561 wikitext text/x-wiki 7chi3qqp1wirsx1vudy4lgoihdisnsh 2445562 2445561 2026-08-11T15:16:50Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.278.0" for 3 host(s) 2445562 wikitext text/x-wiki 9ahe9mwd55ob0ux0p37ejtzhphw03by 2445563 2445562 2026-08-11T15:18:13Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti7004.magru.wmnet 2445563 wikitext text/x-wiki oyplkzioor6iixa7vn7obe6gkigzws9 2445564 2445563 2026-08-11T15:18:19Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7004.magru.wmnet 2445564 wikitext text/x-wiki 3zcoe00h11lwfrev8hf333r02x0bna7 2445565 2445564 2026-08-11T15:18:47Z Stashbot 7414 dancy@deploy1003: Installation of scap version "4.278.0" completed for 3 hosts 2445565 wikitext text/x-wiki l242solv5o97wiv55ukl8x1a6jbkhj2 2445566 2445565 2026-08-11T15:20:48Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db2204: Security update 2445566 wikitext text/x-wiki myejl4t4h2tqgw3yx2ludelr23nal0x 2445568 2445566 2026-08-11T15:25:16Z Stashbot 7414 dancy@deploy1003: Started scap sync-world: Backport for [[gerrit:1324339|Add /w/deployment-info.php entrypoint]] 2445568 wikitext text/x-wiki 7ivciu9wwdofwaexakehm8rdw5h3i8f 2445569 2445568 2026-08-11T15:27:19Z Stashbot 7414 dancy@deploy1003: dancy: Backport for [[gerrit:1324339|Add /w/deployment-info.php entrypoint]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445569 wikitext text/x-wiki gl40xu74xpldi7tf5urgsm9nl7zpgo1 2445570 2445569 2026-08-11T15:28:23Z Stashbot 7414 tappof: remove 2026-05 swift log archives from centrallog to free some space (T434502) 2445570 wikitext text/x-wiki reoy7uwkduomy6vz2r9utad0swhoi26 2445571 2445570 2026-08-11T15:28:39Z Stashbot 7414 dancy@deploy1003: dancy: Continuing with deployment 2445571 wikitext text/x-wiki 0w58fhfn05m7w9rs1u6ga1vbcxra84u 2445572 2445571 2026-08-11T15:29:14Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti4008.ulsfo.wmnet 2445572 wikitext text/x-wiki lg3efugz9f405gg1plr26pjbt9xf2ta 2445573 2445572 2026-08-11T15:30:04Z Stashbot 7414 moritzm: failover ganeti master in magru to ganeti7004 2445573 wikitext text/x-wiki mzdnnqmljngjb8u13haffa3wqwmi34a 2445574 2445573 2026-08-11T15:32:41Z Stashbot 7414 dancy@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324339|Add /w/deployment-info.php entrypoint]] (duration: 07m 25s) 2445574 wikitext text/x-wiki 86uxe0sigxbo3103n4m7j4yskxo84ut 2445575 2445574 2026-08-11T15:32:45Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.279.0" for 3 host(s) 2445575 wikitext text/x-wiki 1jm4wgug7j7g8yuxtiyuho1q25zoj16 2445576 2445575 2026-08-11T15:33:52Z Stashbot 7414 btullis@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host an-master1004.eqiad.wmnet with OS bookworm 2445576 wikitext text/x-wiki owj86r4fdipykx63fcvf0h6nn5a0b1a 2445577 2445576 2026-08-11T15:34:19Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-master1004.eqiad.wmnet with OS bookworm 2445577 wikitext text/x-wiki imyxp8rpwvx5zt848w0by8acampwysf 2445578 2445577 2026-08-11T15:34:23Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1004.eqiad.wmnet with OS bookworm 2445578 wikitext text/x-wiki 9q821v22myln0c2ju243zrgo2ii0ub9 2445579 2445578 2026-08-11T15:34:40Z Stashbot 7414 dancy@deploy1003: Installation of scap version "4.279.0" completed for 3 hosts 2445579 wikitext text/x-wiki q7qmwx6kilj8n2qaka77jpfp8nijtm7 2445580 2445579 2026-08-11T15:34:57Z Stashbot 7414 dancy@deploy1003: Started scap sync-world: Testing 2445580 wikitext text/x-wiki owyp5duo9xkxn3ado0i75y3216d9zsp 2445581 2445580 2026-08-11T15:35:34Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti4008.ulsfo.wmnet 2445581 wikitext text/x-wiki cfgrh89s3722dhglacp2q46g4rg54qi 2445582 2445581 2026-08-11T15:35:39Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4008.ulsfo.wmnet 2445582 wikitext text/x-wiki 5a6zlyvunhmjfd0ygbqkzdhlf38tkn2 2445583 2445582 2026-08-11T15:40:20Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-master1004.eqiad.wmnet with OS bookworm 2445583 wikitext text/x-wiki t2nsdp69am3fkwvj96xbe20677vhsxn 2445584 2445583 2026-08-11T15:40:23Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1004.eqiad.wmnet with OS bookworm 2445584 wikitext text/x-wiki 97sl6jwjyl328oz4pl1y4l332hgd430 2445585 2445584 2026-08-11T15:41:25Z Stashbot 7414 dancy@deploy1003: Finished scap sync-world: Testing (duration: 06m 28s) 2445585 wikitext text/x-wiki 2vwcd6nua1idtc78ec1ykzyrwasocwd 2445587 2445585 2026-08-11T15:54:05Z Stashbot 7414 jhancock@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-ctrl2006.codfw.wmnet with OS trixie 2445587 wikitext text/x-wiki 0i93nd8ec260zoekeks7bvu6aghloll 2445588 2445587 2026-08-11T15:54:44Z Stashbot 7414 jhancock@cumin1003: START - Cookbook sre.hosts.provision for host wikikube-ctrl2006.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2445588 wikitext text/x-wiki 3yj7zixjotyhkx1f1ysrt2irb2lm0yi 2445589 2445588 2026-08-11T15:55:13Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1157: Pool db1157.eqiad.wmnet in after cloning 2445589 wikitext text/x-wiki 5xw3ykexcplexyz47it6tl7l7v9m0ff 2445591 2445589 2026-08-11T16:01:27Z Stashbot 7414 jhancock@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-ctrl2006.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2445591 wikitext text/x-wiki mgjqck14by3ad6m6i0jn64x8k5ul3f5 2445593 2445591 2026-08-11T16:02:24Z Stashbot 7414 reedy@deploy1003: Started scap sync-world: Backport for [[gerrit:1324335|InitialiseSettings: Enable 2FA enforcement on various private wikis (T428103)]] 2445593 wikitext text/x-wiki l8uoq9imw0n1tjigrg37lpl9xmos0md 2445594 2445593 2026-08-11T16:02:58Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7001.magru.wmnet 2445594 wikitext text/x-wiki nydukh4aarg8oo8358vfjh5n4mf76tg 2445595 2445594 2026-08-11T16:04:38Z Stashbot 7414 reedy@deploy1003: reedy: Backport for [[gerrit:1324335|InitialiseSettings: Enable 2FA enforcement on various private wikis (T428103)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445595 wikitext text/x-wiki iitz2wyvfc8a26vhlti878ut7ky2rqc 2445596 2445595 2026-08-11T16:04:58Z Stashbot 7414 reedy@deploy1003: reedy: Continuing with deployment 2445596 wikitext text/x-wiki 7xzxwn3glkv0uw45vdnxcds0kotljps 2445597 2445596 2026-08-11T16:06:11Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2204: Security update 2445597 wikitext text/x-wiki meukfpkexzj2ycwba3wqhqr50h8q2w3 2445598 2445597 2026-08-11T16:08:06Z Stashbot 7414 jhancock@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-ctrl2006.codfw.wmnet with OS trixie 2445598 wikitext text/x-wiki ixhn35q2oc4jgmaj4n5ay0o1qj84s31 2445599 2445598 2026-08-11T16:09:05Z Stashbot 7414 reedy@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324335|InitialiseSettings: Enable 2FA enforcement on various private wikis (T428103)]] (duration: 06m 40s) 2445599 wikitext text/x-wiki 80qgpuky5mgosf0fi8o1mo03ahznljj 2445602 2445599 2026-08-11T16:13:16Z Stashbot 7414 sukhe: restart ntpsec.serviceon dns7001 2445602 wikitext text/x-wiki cqnf7l7n5hdv24x0exoykbxwnimlm9b 2445605 2445602 2026-08-11T16:24:24Z Stashbot 7414 reedy@deploy1003: Started scap sync-world: Backport for [[gerrit:1324356|InitialiseSettings: Fix wgOATHAuthEnforce2FAForAll]] 2445605 wikitext text/x-wiki fh3qi1sc2163awqq7fq43s6mqnfku5m 2445606 2445605 2026-08-11T16:26:28Z Stashbot 7414 reedy@deploy1003: reedy: Backport for [[gerrit:1324356|InitialiseSettings: Fix wgOATHAuthEnforce2FAForAll]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445606 wikitext text/x-wiki 6j8q93x00sfx0l22jnu94p7rupi079e 2445607 2445606 2026-08-11T16:27:14Z Stashbot 7414 reedy@deploy1003: reedy: Continuing with deployment 2445607 wikitext text/x-wiki g6jsosafkpzcm1s91ah7tw91bss3ox3 2445608 2445607 2026-08-11T16:28:13Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-master1004.eqiad.wmnet with OS bookworm 2445608 wikitext text/x-wiki 2btitk9pt18q4fpsey3b2vv005qgloa 2445609 2445608 2026-08-11T16:30:38Z Stashbot 7414 jhancock@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-ctrl2006.codfw.wmnet with OS trixie 2445609 wikitext text/x-wiki 8b4apdrmc66rw67i110k1o7f7dp941u 2445610 2445609 2026-08-11T16:31:16Z Stashbot 7414 reedy@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324356|InitialiseSettings: Fix wgOATHAuthEnforce2FAForAll]] (duration: 06m 52s) 2445610 wikitext text/x-wiki d9n4fgjtvl3tirtu4h72mvam7obxw70 2445611 2445610 2026-08-11T16:39:00Z Stashbot 7414 jhancock@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-ctrl2006.codfw.wmnet with OS trixie 2445611 wikitext text/x-wiki dbpfgky5btld25tmep6wseguxbxm1hv 2445612 2445611 2026-08-11T16:40:26Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1157: Pool db1157.eqiad.wmnet in after cloning 2445612 wikitext text/x-wiki apm4vcvindodtv7en4af4n90lr753le 2445613 2445612 2026-08-11T16:41:41Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1157.eqiad.wmnet onto db1272.eqiad.wmnet 2445613 wikitext text/x-wiki 8rtf4xvcftkk3exrw9w99pm14hqlese 2445614 2445613 2026-08-11T16:44:25Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-master1004.eqiad.wmnet with reason: host reimage 2445614 wikitext text/x-wiki q14p0w5oht3m9ria3bhqgpzxj4zddvf 2445615 2445614 2026-08-11T16:48:29Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-master1004.eqiad.wmnet with reason: host reimage 2445615 wikitext text/x-wiki 2wyczwbp5fvzc01k2md8ie630yabbp9 2445616 2445615 2026-08-11T16:51:12Z Stashbot 7414 sukhe@dns1004: START - running authdns-update 2445616 wikitext text/x-wiki 82ablu7v8x0arp7an13qxb3eemwzoho 2445617 2445616 2026-08-11T16:53:01Z Stashbot 7414 sukhe@dns1004: FAIL - running authdns-update 2445617 wikitext text/x-wiki 0ayp35535o9r4ies1nccbrtce0yqf7d 2445618 2445617 2026-08-11T16:53:09Z Stashbot 7414 sukhe@dns1004: START - running authdns-update 2445618 wikitext text/x-wiki pioriga0p6vknhysmuoy4vvugox1xlx 2445619 2445618 2026-08-11T16:54:59Z Stashbot 7414 sukhe@dns1004: END - running authdns-update 2445619 wikitext text/x-wiki sibhefl0grhdl1tptu6wy062osjtj0o 2445620 2445619 2026-08-11T17:02:14Z Stashbot 7414 sukhe: sudo cumin "A:cp-magru" "disable-puppet 'merging CR 1324355'" 2445620 wikitext text/x-wiki 4o0altrnfzc9kf8ntknzebo01zdcrs2 2445621 2445620 2026-08-11T17:11:44Z Stashbot 7414 sukhe: sukhe@cp7005:~$ sudo puppet agent -tv 2445621 wikitext text/x-wiki dlbms354tds6dd62wxpylij92tiddj6 2445622 2445621 2026-08-11T17:12:00Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-master1004.eqiad.wmnet with OS bookworm 2445622 wikitext text/x-wiki 2su4so6xjprztm2tdanojqlcli42tfw 2445623 2445622 2026-08-11T17:18:44Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1324361|Move FR config from IS.php to a dedicated file]], [[gerrit:1324363|Remove $wmg = $wg hacks in CentralAuth (T119117)]] 2445623 wikitext text/x-wiki e48wdn4ml65cnoa73xtb3z0tyu3g63m 2445624 2445623 2026-08-11T17:19:33Z Stashbot 7414 sukhe: sudo cumin "A:cp-magru" "run-puppet-agent --enable 'merging CR 1324355'" 2445624 wikitext text/x-wiki 47cpxlhkpya52lwf4vmcmn82f2hfqqs 2445625 2445624 2026-08-11T17:23:12Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1324361|Move FR config from IS.php to a dedicated file]], [[gerrit:1324363|Remove $wmg = $wg hacks in CentralAuth (T119117)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445625 wikitext text/x-wiki 6jyyyhxsoef74fob88kx7hsxeznt8r8 2445626 2445625 2026-08-11T17:26:59Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2445626 wikitext text/x-wiki qxdv8vec6mp187hs4cucpbbt8a96tkg 2445627 2445626 2026-08-11T17:31:07Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324361|Move FR config from IS.php to a dedicated file]], [[gerrit:1324363|Remove $wmg = $wg hacks in CentralAuth (T119117)]] (duration: 12m 23s) 2445627 wikitext text/x-wiki i6ixr7k1z8uoga0se1hqzrpo85ni1gh 2445628 2445627 2026-08-11T17:34:44Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reimage for host dns6001.wikimedia.org with OS trixie 2445628 wikitext text/x-wiki l456dcclwdv05ksrt4mootl5il5wyn2 2445629 2445628 2026-08-11T17:36:02Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reimage for host dns5003.wikimedia.org with OS trixie 2445629 wikitext text/x-wiki immrjxwh3jlnbvsa31d35p67nynxdpg 2445630 2445629 2026-08-11T17:39:07Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti7001.magru.wmnet 2445630 wikitext text/x-wiki bq9mw5etwuzh6xj01ylldrqyg0ev7ex 2445631 2445630 2026-08-11T17:45:34Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1324369|Enable desktop lazy loading on group0 (T148047)]] 2445631 wikitext text/x-wiki 6g6hryyxcnxv61kdkoftpmjttj21mvz 2445632 2445631 2026-08-11T17:47:35Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1324369|Enable desktop lazy loading on group0 (T148047)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445632 wikitext text/x-wiki 2ei1rodwioeh7o4qppdmq8grqnoc3wr 2445633 2445632 2026-08-11T17:48:57Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2445633 wikitext text/x-wiki mor9a1llz6ux3pyqectzg0t9npimavd 2445634 2445633 2026-08-11T17:49:17Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti7001.magru.wmnet 2445634 wikitext text/x-wiki 9p4cciv79mxavkjh616l7cdfzrb2v80 2445635 2445634 2026-08-11T17:49:22Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7001.magru.wmnet 2445635 wikitext text/x-wiki 5q05l1ywkxzgaprrdvkm8plppcy5uqb 2445636 2445635 2026-08-11T17:49:47Z Stashbot 7414 ladsgroup@cumin1003: START - Cookbook sre.mysql.sanitarium_restart 2445636 wikitext text/x-wiki sjp6f5krnnbtel617ah9xmck0h10ivw 2445637 2445636 2026-08-11T17:49:49Z Stashbot 7414 ladsgroup@cumin1003: END (FAIL) - Cookbook sre.mysql.sanitarium_restart (exit_code=99) 2445637 wikitext text/x-wiki 3cq7pvmltqi0h59hp4vf7y8udfa0dqz 2445638 2445637 2026-08-11T17:49:57Z Stashbot 7414 ladsgroup@cumin1003: START - Cookbook sre.mysql.sanitarium_restart 2445638 wikitext text/x-wiki h1jt7apvj1qbdja3e3jg92r86c3m49p 2445639 2445638 2026-08-11T17:52:34Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on dns6001.wikimedia.org with reason: host reimage 2445639 wikitext text/x-wiki 26iinrpafwabl33o5hgfifccflr83yi 2445640 2445639 2026-08-11T17:53:06Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324369|Enable desktop lazy loading on group0 (T148047)]] (duration: 07m 31s) 2445640 wikitext text/x-wiki slo5vk9wkgzwrw4fehegy3cnog7024u 2445641 2445640 2026-08-11T17:58:16Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dns6001.wikimedia.org with reason: host reimage 2445641 wikitext text/x-wiki cfugh664fvsi4qkrqnbkfa68m8uht94 2445642 2445641 2026-08-11T18:01:22Z Stashbot 7414 ladsgroup@cumin1003: END (PASS) - Cookbook sre.mysql.sanitarium_restart (exit_code=0) 2445642 wikitext text/x-wiki 5gux5m08qrnioyv1ijkwf3azb8lxlzd 2445643 2445642 2026-08-11T18:05:29Z Stashbot 7414 brett: import trafficserver-9.2.15~deb13+wmf1 into trixie-wikimedia (T434478) 2445643 wikitext text/x-wiki pcj218z7s1y8d4aqccpqegqcfh84705 2445644 2445643 2026-08-11T18:05:33Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on dns5003.wikimedia.org with reason: host reimage 2445644 wikitext text/x-wiki 6huazy8mhu660noz9ilyjpulthc9a6x 2445645 2445644 2026-08-11T18:06:38Z Stashbot 7414 brennen: 1.47.0-wmf.15 train status (T430834) - no current blockers, rolling to group0 2445645 wikitext text/x-wiki ooqz03lo6gz3mj3gm92btilq3ikj9yp 2445646 2445645 2026-08-11T18:06:40Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.cdn.roll-upgrade-ats Rolling upgrade of ATS on P{cp7001.magru.wmnet} and A:cp - 9.2.15 Upgrade () 2445646 wikitext text/x-wiki 6krlyynzv65n65riklmzdang96oh35u 2445648 2445646 2026-08-11T18:09:22Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dns5003.wikimedia.org with reason: host reimage 2445648 wikitext text/x-wiki ap4muk02b4roj69r8u0fkmchl3fqbhd 2445649 2445648 2026-08-11T18:12:44Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.cdn.roll-upgrade-ats (exit_code=0) Rolling upgrade of ATS on P{cp7001.magru.wmnet} and A:cp - 9.2.15 Upgrade () 2445649 wikitext text/x-wiki tohbjxjysb1mdljrd57vm5rfjpmfpy2 2445650 2445649 2026-08-11T18:13:57Z Stashbot 7414 brennen@deploy1003: rebuilt and synchronized wikiversions files: group0 to 1.47.0-wmf.15 refs T430834 2445650 wikitext text/x-wiki 4ppzazid4ag4ss5yr2ile9rx8p0xfpp 2445651 2445650 2026-08-11T18:14:20Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.cdn.roll-upgrade-ats Rolling upgrade of ATS on P{cp7009.magru.wmnet} and A:cp - 9.2.15 Upgrade () 2445651 wikitext text/x-wiki 5qwp8iambv8221dlognuifmjpkepwcr 2445652 2445651 2026-08-11T18:19:52Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.cdn.roll-upgrade-ats (exit_code=0) Rolling upgrade of ATS on P{cp7009.magru.wmnet} and A:cp - 9.2.15 Upgrade () 2445652 wikitext text/x-wiki ehvcnwk4gcdz7dx2xd7a0ogngrw3hbe 2445656 2445652 2026-08-11T18:55:08Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dns6001.wikimedia.org with OS trixie 2445656 wikitext text/x-wiki a0amxhrmbyuvw6zsrsi5bpti2c8l2mh 2445658 2445656 2026-08-11T18:59:30Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dns5003.wikimedia.org with OS trixie 2445658 wikitext text/x-wiki 93yww8h64ychx6njf6250j3ga4c9kg6 2445659 2445658 2026-08-11T19:03:28Z Stashbot 7414 brennen@deploy1003: Started scap sync-world: testwikis to 1.47.0-wmf.15 refs T430834 2445659 wikitext text/x-wiki b8ql9lafz1y9xwg06131pw5bfb4qvq2 2445660 2445659 2026-08-11T19:04:14Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.remove-downtime for dns5003.wikimedia.org 2445660 wikitext text/x-wiki 48jrxohme9t6rztk1zvpg54wchckix4 2445661 2445660 2026-08-11T19:04:16Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for dns5003.wikimedia.org 2445661 wikitext text/x-wiki 4nbduygncmj73kg1l725gcm8kjhuceg 2445662 2445661 2026-08-11T19:04:42Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.remove-downtime for dns6001.wikimedia.org 2445662 wikitext text/x-wiki f808m8agj6wrxmitkd9bgbtxeqaa5ik 2445663 2445662 2026-08-11T19:04:44Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for dns6001.wikimedia.org 2445663 wikitext text/x-wiki bns1egg4rphsnx22qq1auw6fhc7627p 2445664 2445663 2026-08-11T19:10:10Z Stashbot 7414 brennen@deploy1003: Finished scap sync-world: testwikis to 1.47.0-wmf.15 refs T430834 (duration: 06m 41s) 2445664 wikitext text/x-wiki bn79gfrsqy5wyr3w4oxs3ydcurhcanw 2445666 2445664 2026-08-11T19:54:02Z Stashbot 7414 jhancock@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-ctrl2006.codfw.wmnet with reason: host reimage 2445666 wikitext text/x-wiki itgvsahpb5kwm9kig9vsor3924qammk 2445667 2445666 2026-08-11T19:59:05Z Stashbot 7414 jhancock@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-ctrl2006.codfw.wmnet with reason: host reimage 2445667 wikitext text/x-wiki 4st4fjak78zydxxbl4vi869h536pfmr 2445670 2445667 2026-08-11T20:14:47Z Stashbot 7414 jhancock@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin1003" 2445670 wikitext text/x-wiki k6t7plnnfu9sxzlvjyhb906zyemuw13 2445671 2445670 2026-08-11T20:15:33Z Stashbot 7414 jhancock@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin1003" 2445671 wikitext text/x-wiki 1t9ot2782foqkfy0aixexudf7kdaanz 2445672 2445671 2026-08-11T20:15:35Z Stashbot 7414 jhancock@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-ctrl2006.codfw.wmnet with OS trixie 2445672 wikitext text/x-wiki pv2iej3j9s1mnapgd2w4h62c5d1weuh 2445673 2445672 2026-08-11T20:18:29Z Stashbot 7414 jhancock@cumin1003: START - Cookbook sre.hosts.reimage for host wikikube-ctrl2006.codfw.wmnet with OS trixie 2445673 wikitext text/x-wiki su3gyal8r3ekpljdyot31i54xix0aeg 2445676 2445673 2026-08-11T20:35:02Z Stashbot 7414 kemayo@deploy1003: Started scap sync-world: Backport for [[gerrit:1324386|build: Updating js-yaml to 3.15.1, 4.3.1]] 2445676 wikitext text/x-wiki jew0p55l279xqk6ua8embtcqgw3d6vf 2445677 2445676 2026-08-11T20:37:04Z Stashbot 7414 kemayo@deploy1003: kemayo: Backport for [[gerrit:1324386|build: Updating js-yaml to 3.15.1, 4.3.1]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445677 wikitext text/x-wiki 5jjkccxfv2ycx4sjeih159kirqn1nlz 2445678 2445677 2026-08-11T20:37:48Z Stashbot 7414 jhancock@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-ctrl2006.codfw.wmnet with reason: host reimage 2445678 wikitext text/x-wiki plq4r8tbbwprob3va0gy7uukx7uji57 2445679 2445678 2026-08-11T20:38:32Z Stashbot 7414 kemayo@deploy1003: kemayo: Continuing with deployment 2445679 wikitext text/x-wiki i6oplxn3kz9ntkgu7wixcaifag2a6yv 2445681 2445679 2026-08-11T20:42:39Z Stashbot 7414 kemayo@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324386|build: Updating js-yaml to 3.15.1, 4.3.1]] (duration: 07m 36s) 2445681 wikitext text/x-wiki 6gla332zw6vpthth6c9r1pn5sonnh60 2445682 2445681 2026-08-11T20:42:58Z Stashbot 7414 jhancock@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-ctrl2006.codfw.wmnet with reason: host reimage 2445682 wikitext text/x-wiki ocxiqtgtg6obr47ekok7uwzmf3bszna 2445683 2445682 2026-08-11T20:43:24Z Stashbot 7414 kemayo@deploy1003: Started scap sync-world: Backport for [[gerrit:1324384|Editcheck: ecenable causing errors when editCheckTagging is active (T434403)]], [[gerrit:1324385|Editcheck: ecenable causing errors when editCheckTagging is active (T434403)]] 2445683 wikitext text/x-wiki jkcpc8hq8xy9g5f35bmz2jvczeqkjq3 2445684 2445683 2026-08-11T20:45:28Z Stashbot 7414 kemayo@deploy1003: kemayo: Backport for [[gerrit:1324384|Editcheck: ecenable causing errors when editCheckTagging is active (T434403)]], [[gerrit:1324385|Editcheck: ecenable causing errors when editCheckTagging is active (T434403)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445684 wikitext text/x-wiki 113x0ha43y90yv4omxmw5r8vrn995kf 2445685 2445684 2026-08-11T20:46:18Z Stashbot 7414 kemayo@deploy1003: kemayo: Continuing with deployment 2445685 wikitext text/x-wiki 868c7q5w02xohgttwutgany5lrb1ybt 2445686 2445685 2026-08-11T20:50:22Z Stashbot 7414 kemayo@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324384|Editcheck: ecenable causing errors when editCheckTagging is active (T434403)]], [[gerrit:1324385|Editcheck: ecenable causing errors when editCheckTagging is active (T434403)]] (duration: 06m 58s) 2445686 wikitext text/x-wiki 9j6hcqhwtizqj6dvvux5xnuqhaicrzr 2445687 2445686 2026-08-11T20:59:14Z Stashbot 7414 jhancock@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-ctrl2006.codfw.wmnet with OS trixie 2445687 wikitext text/x-wiki 32ivrv1omg6vzt76pbrtq4kgoo0boz7 2445690 2445687 2026-08-11T21:03:47Z Stashbot 7414 jdlrobson@deploy1003: Started scap sync-world: Backport for [[gerrit:1324370|Phase 4 of legal footer deployment (T432796)]], [[gerrit:1319804|Disable wgMFCustomSiteModules on English Wikipedia (T375538)]] 2445690 wikitext text/x-wiki ahwerybgd39jj6ip4ftfb6nmn92dq6o 2445691 2445690 2026-08-11T21:06:01Z Stashbot 7414 jdlrobson@deploy1003: jdlrobson: Backport for [[gerrit:1324370|Phase 4 of legal footer deployment (T432796)]], [[gerrit:1319804|Disable wgMFCustomSiteModules on English Wikipedia (T375538)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445691 wikitext text/x-wiki hofdpiw46o39rv5dgc0c40mkisz721t 2445692 2445691 2026-08-11T21:15:06Z Stashbot 7414 jdlrobson@deploy1003: jdlrobson: Continuing with deployment 2445692 wikitext text/x-wiki 5l7y36hs6yeakwu2zde5gyt708gv8gl 2445694 2445692 2026-08-11T21:19:14Z Stashbot 7414 jdlrobson@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324370|Phase 4 of legal footer deployment (T432796)]], [[gerrit:1319804|Disable wgMFCustomSiteModules on English Wikipedia (T375538)]] (duration: 15m 26s) 2445694 wikitext text/x-wiki iy41q90znwpleipnwiofttnmjz464ad 2445696 2445694 2026-08-11T21:29:18Z Stashbot 7414 sbassett: Deployed security fix for T434521 (wmf.14) 2445696 wikitext text/x-wiki oo5m5heq5lzsidrlshggap6ykj1u0kv 2445698 2445696 2026-08-11T21:37:12Z Stashbot 7414 sbassett: Deployed security fix for T434521 (wmf.15) 2445698 wikitext text/x-wiki 4pvrtfu7c443rwt2tdndsaog5z4nk17 2445706 2445698 2026-08-12T00:17:39Z Stashbot 7414 kemayo@deploy1003: Started scap sync-world: Backport for [[gerrit:1324427|ve.ui.CodeMirrorAction: load ::highlight() styles only when used (T434403)]], [[gerrit:1324429|Bump mediawiki/mediawiki-codesniffer to v52.0.0]], [[gerrit:1324428|ve.ui.CodeMirrorAction: load ::highlight() styles only when used (T434403)]] 2445706 wikitext text/x-wiki 6hraaucydruesss6apilcav5kzacfsi 2445707 2445706 2026-08-12T00:19:42Z Stashbot 7414 kemayo@deploy1003: kemayo: Backport for [[gerrit:1324427|ve.ui.CodeMirrorAction: load ::highlight() styles only when used (T434403)]], [[gerrit:1324429|Bump mediawiki/mediawiki-codesniffer to v52.0.0]], [[gerrit:1324428|ve.ui.CodeMirrorAction: load ::highlight() styles only when used (T434403)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445707 wikitext text/x-wiki t0hjgajjlu90re644bdbaea4x6fymkn 2445708 2445707 2026-08-12T00:21:23Z Stashbot 7414 kemayo@deploy1003: kemayo: Continuing with deployment 2445708 wikitext text/x-wiki 6zxl7d2o84f62ecqn3osfxrcbiobjy7 2445709 2445708 2026-08-12T00:25:34Z Stashbot 7414 kemayo@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324427|ve.ui.CodeMirrorAction: load ::highlight() styles only when used (T434403)]], [[gerrit:1324429|Bump mediawiki/mediawiki-codesniffer to v52.0.0]], [[gerrit:1324428|ve.ui.CodeMirrorAction: load ::highlight() styles only when used (T434403)]] (duration: 07m 55s) 2445709 wikitext text/x-wiki pqgk84o06fx5l0mhxaxta50mc3uvvr8 2445713 2445709 2026-08-12T02:00:37Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2445713 wikitext text/x-wiki 2u0l0fts2lp8ev1igcjzsxg9hzop57z 2445715 2445713 2026-08-12T02:07:22Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 45s) 2445715 wikitext text/x-wiki qfvplo5yj98hhmdt9hb99hv5u468hp7 2445717 2445715 2026-08-12T03:41:14Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-tool1008.eqiad.wmnet with OS bookworm 2445717 wikitext text/x-wiki h7g2w8ggw5i9phl7jg94m7elo5f4e2z 2445718 2445717 2026-08-12T03:53:04Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-tool1008.eqiad.wmnet with reason: host reimage 2445718 wikitext text/x-wiki qfuls5vc7k9w0hkr79025wubmc5mhn1 2445719 2445718 2026-08-12T03:58:25Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-tool1008.eqiad.wmnet with reason: host reimage 2445719 wikitext text/x-wiki 19nzxaetkk1fo6gy1c3hg0nrlkpj5o2 2445720 2445719 2026-08-12T04:16:25Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-tool1008.eqiad.wmnet with OS bookworm 2445720 wikitext text/x-wiki 63hpz4knjso0y3745gc8mg7c27smfnk 2445721 2445720 2026-08-12T04:40:46Z Stashbot 7414 ryankemper: T434494 reimaged `an-tool1008.eqiad.wmnet` to bookworm; yarn.wikimedia.org is back up 2445721 wikitext text/x-wiki hnw1oboz6t1kn7x7yfg4o57lz9d8kjx 2445722 2445721 2026-08-12T05:53:51Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5004.eqsin.wmnet 2445722 wikitext text/x-wiki mwgmfhiw9vzcn95l5jnvkzmik9ok581 2445723 2445722 2026-08-12T05:55:07Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti5004.eqsin.wmnet 2445723 wikitext text/x-wiki 5px46ebdanmiw263s8kgf9gsv16ayug 2445724 2445723 2026-08-12T06:03:29Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti5004.eqsin.wmnet 2445724 wikitext text/x-wiki hs2o14dfdvjmxqa5cw41iz8y23cqpwm 2445725 2445724 2026-08-12T06:03:36Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti5004.eqsin.wmnet 2445725 wikitext text/x-wiki 28il391fbw2gogzpcy88q1c6kuodadi 2445726 2445725 2026-08-12T06:06:10Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5005.eqsin.wmnet 2445726 wikitext text/x-wiki fnkegdaw7p4wdsdfmnszb4acqpafwq2 2445727 2445726 2026-08-12T06:12:13Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti5005.eqsin.wmnet 2445727 wikitext text/x-wiki rt2rkq4a9irgiiut2z0zvw5kn833xxl 2445728 2445727 2026-08-12T06:20:19Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti5005.eqsin.wmnet 2445728 wikitext text/x-wiki mm1yd1h91skzqspbe8vsh305maxtnjg 2445729 2445728 2026-08-12T06:20:32Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti5005.eqsin.wmnet 2445729 wikitext text/x-wiki tnn6kh94tjtnpg7e1i8etmpp4dfo2kx 2445730 2445729 2026-08-12T06:23:02Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5006.eqsin.wmnet 2445730 wikitext text/x-wiki 2wkx9wlb7jpw7itfc45alipz84l9f1n 2445731 2445730 2026-08-12T06:28:04Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti5006.eqsin.wmnet 2445731 wikitext text/x-wiki 74tzl5st3gpihxj309v8nyq06xkqn8g 2445732 2445731 2026-08-12T06:36:23Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti5006.eqsin.wmnet 2445732 wikitext text/x-wiki jb02r10x4l7zuznj9u1c805e2jhdijg 2445733 2445732 2026-08-12T06:36:30Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti5006.eqsin.wmnet 2445733 wikitext text/x-wiki rak60dfs694x7si6bbpo2wjwp39w7el 2445734 2445733 2026-08-12T06:38:38Z Stashbot 7414 moritzm: failover ganeti master in eqsin to ganeti5004 2445734 wikitext text/x-wiki a6qs2meonip2l8l7v2hx9o1z67r2mlh 2445735 2445734 2026-08-12T06:54:14Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5007.eqsin.wmnet 2445735 wikitext text/x-wiki 0mjz5zop9gvwewxvqyupm1c300024y4 2445736 2445735 2026-08-12T06:58:58Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti5007.eqsin.wmnet 2445736 wikitext text/x-wiki hhywxvnvbblrvizhhmpakm3srlwzaj1 2445737 2445736 2026-08-12T07:05:22Z Stashbot 7414 arnaudb@dns1006: START - running authdns-update 2445737 wikitext text/x-wiki o6b8e1rea434bcjqslqlhxe3w4hahzs 2445738 2445737 2026-08-12T07:07:09Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti5007.eqsin.wmnet 2445738 wikitext text/x-wiki 2pgg50xunqlvu3s1mtvm4yt8an151lb 2445739 2445738 2026-08-12T07:07:16Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti5007.eqsin.wmnet 2445739 wikitext text/x-wiki hyn3e1xgnkl3t3lsygjtyt3qgmq61ws 2445740 2445739 2026-08-12T07:07:22Z Stashbot 7414 arnaudb@dns1006: END - running authdns-update 2445740 wikitext text/x-wiki s3azy8chcwjki9n41xqwa7d15xx6bfa 2445741 2445740 2026-08-12T07:11:12Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti3008.esams.wmnet 2445741 wikitext text/x-wiki thhh0pgpdvjc8v4gq1223p4fwwtyzzi 2445742 2445741 2026-08-12T07:12:54Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts an-master1003.eqiad.wmnet 2445742 wikitext text/x-wiki 0nn2593it7xqe34wj0qh7xfble1bdsw 2445743 2445742 2026-08-12T07:13:33Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts an-master1003.eqiad.wmnet 2445743 wikitext text/x-wiki 5ptvka9nkv4bream1yc2pv0yrpofxqj 2445744 2445743 2026-08-12T07:13:44Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts an-master1003.eqiad.wmnet 2445744 wikitext text/x-wiki mme6s02emysv5oyuq6jiv2ro9b9nyz2 2445745 2445744 2026-08-12T07:14:25Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reboot-single for host an-master1003.eqiad.wmnet 2445745 wikitext text/x-wiki 5ocs6x52qmcavaqaaipdn8rnkrquqc8 2445746 2445745 2026-08-12T07:15:35Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-presto1009.eqiad.wmnet with OS bookworm 2445746 wikitext text/x-wiki 1y22xebbtw4n42dn6htle9ztiv86fcy 2445747 2445746 2026-08-12T07:16:36Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti3008.esams.wmnet 2445747 wikitext text/x-wiki e9dup7ozzgopb4pxkcsqg2jtwj3pr8v 2445748 2445747 2026-08-12T07:16:45Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1009.eqiad.wmnet with OS bookworm 2445748 wikitext text/x-wiki 24lm8vn2oa5cnj90g35hfjngjz8x6ux 2445749 2445748 2026-08-12T07:17:41Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1011.eqiad.wmnet with OS bookworm 2445749 wikitext text/x-wiki sy8amdfzn2b5as84ydh84vgvd487ppj 2445750 2445749 2026-08-12T07:18:53Z Stashbot 7414 btullis@cumin1003: END (ERROR) - Cookbook sre.hosts.reboot-single (exit_code=97) for host an-master1003.eqiad.wmnet 2445750 wikitext text/x-wiki 52e2dri5kbbligy2zbmybetddsgyceh 2445751 2445750 2026-08-12T07:18:58Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts an-master1003.eqiad.wmnet 2445751 wikitext text/x-wiki h1r8ktakg84gdkmi1widkl56kmxc4qz 2445752 2445751 2026-08-12T07:19:23Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-master1003.eqiad.wmnet with OS bookworm 2445752 wikitext text/x-wiki af5eq15ah285w1zyivf0tholzuh9bko 2445753 2445752 2026-08-12T07:22:58Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-web1001.eqiad.wmnet with OS bookworm 2445753 wikitext text/x-wiki qvo5qt444zcc2dweho1lke0519la8tj 2445754 2445753 2026-08-12T07:24:50Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti3008.esams.wmnet 2445754 wikitext text/x-wiki bheziku2n02i7atru99vf8bhffvc8ob 2445755 2445754 2026-08-12T07:25:14Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti3008.esams.wmnet 2445755 wikitext text/x-wiki say4q7x0lo6n4acu25289vm3bfews5d 2445756 2445755 2026-08-12T07:25:24Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti3007.esams.wmnet 2445756 wikitext text/x-wiki sfdfqnbhyuk0u0xtddnkwut5flyt3z6 2445757 2445756 2026-08-12T07:27:35Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti3007.esams.wmnet 2445757 wikitext text/x-wiki d47zkc40jbzrif73uiih290ha8n87nj 2445758 2445757 2026-08-12T07:33:44Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-presto1011.eqiad.wmnet with reason: host reimage 2445758 wikitext text/x-wiki 7int4w88ptzvq8o2p74zxhtt0a3aokm 2445759 2445758 2026-08-12T07:35:48Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti3007.esams.wmnet 2445759 wikitext text/x-wiki becxrtlep9n5vjd5ub65slpkpxq47gx 2445760 2445759 2026-08-12T07:35:54Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti3007.esams.wmnet 2445760 wikitext text/x-wiki m6e3of1imtwwayntc56vlmb20nkt7mt 2445761 2445760 2026-08-12T07:38:11Z Stashbot 7414 btullis@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host an-master1003.eqiad.wmnet with OS bookworm 2445761 wikitext text/x-wiki o3nqffhmxthi36r8o1o9j9k41ba5dk1 2445762 2445761 2026-08-12T07:38:14Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-presto1011.eqiad.wmnet with reason: host reimage 2445762 wikitext text/x-wiki 1lzgam859otnieulpoj7ydsvdk5a7en 2445763 2445762 2026-08-12T07:50:35Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-master1003.eqiad.wmnet with OS bookworm 2445763 wikitext text/x-wiki my914ierkw3y4zumoaa98lbboo7tmdm 2445764 2445763 2026-08-12T07:58:52Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-web1001.eqiad.wmnet with reason: host reimage 2445764 wikitext text/x-wiki qj6uetpetf3aghqe4mth3ymwli83dox 2445766 2445764 2026-08-12T08:04:11Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-web1001.eqiad.wmnet with reason: host reimage 2445766 wikitext text/x-wiki tm5j7z37zyuzpvu9pbcaolmqfrb7ciq 2445767 2445766 2026-08-12T08:07:09Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-master1003.eqiad.wmnet with reason: host reimage 2445767 wikitext text/x-wiki itlqb0y7uvm52indm3u5ogzgyl5un5t 2445768 2445767 2026-08-12T08:13:16Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-master1003.eqiad.wmnet with reason: host reimage 2445768 wikitext text/x-wiki dicw307prl0eqaeumrhc54o5oowdecw 2445769 2445768 2026-08-12T08:25:16Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-presto1011.eqiad.wmnet with OS bookworm 2445769 wikitext text/x-wiki eehdi5ir24g5d7ro7jw3f1t5n4kg2ch 2445770 2445769 2026-08-12T08:25:53Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1012.eqiad.wmnet with OS bookworm 2445770 wikitext text/x-wiki gmib0pmz9hg2084nvv97v2jmfc8gko1 2445772 2445770 2026-08-12T08:28:56Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool ms3 T434288', diff saved to https://phabricator.wikimedia.org/P95993 and previous config saved to /var/cache/conftool/dbconfig/20260812-082852-marostegui.json 2445772 wikitext text/x-wiki msqvp6kec2psc5slga19g6spokrkusf 2445774 2445772 2026-08-12T08:32:09Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db2252.codfw.wmnet,db[1153,1268].eqiad.wmnet with reason: Switching over ms3 2445774 wikitext text/x-wiki jyhgdkq0ne3zh4eu5igm1rfxt2a0fkg 2445775 2445774 2026-08-12T08:35:52Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-web1001.eqiad.wmnet with OS bookworm 2445775 wikitext text/x-wiki sgab1ylyy5rkx896mdswuct84fefncz 2445776 2445775 2026-08-12T08:37:27Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1268 to dbctl T434288', diff saved to https://phabricator.wikimedia.org/P95994 and previous config saved to /var/cache/conftool/dbconfig/20260812-083722-marostegui.json 2445776 wikitext text/x-wiki op4n21hjmyz1ewpjc8s4ghyze7dm1w5 2445777 2445776 2026-08-12T08:37:37Z Stashbot 7414 marostegui: Failover ms3 T434288 2445777 wikitext text/x-wiki k9vq5o1874obltax32y4l3cvuexm3m0 2445778 2445777 2026-08-12T08:38:06Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-master1003.eqiad.wmnet with OS bookworm 2445778 wikitext text/x-wiki 2mrt34vewr8f8a0abkfq8ghvkb6ex0t 2445779 2445778 2026-08-12T08:38:18Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Pool back ms3', diff saved to https://phabricator.wikimedia.org/P95995 and previous config saved to /var/cache/conftool/dbconfig/20260812-083816-marostegui.json 2445779 wikitext text/x-wiki t690savdth0b7484ctoqrfl7n5yg9ce 2445780 2445779 2026-08-12T08:40:25Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1013.eqiad.wmnet with OS bookworm 2445780 wikitext text/x-wiki r74eeyd2e0gypjt6nchb3rc5mcxj58w 2445781 2445780 2026-08-12T08:41:38Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-launcher1003.eqiad.wmnet with OS bookworm 2445781 wikitext text/x-wiki syatlnfmhhcvkvt5n4kxgwf35888ukk 2445782 2445781 2026-08-12T08:42:17Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-presto1012.eqiad.wmnet with reason: host reimage 2445782 wikitext text/x-wiki 08c77060acrdpo5nu070z0lrtxhnjph 2445783 2445782 2026-08-12T08:43:05Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-presto1009.eqiad.wmnet with OS bookworm 2445783 wikitext text/x-wiki nf4uk798bi5xxu8a4ggqxw5fe9z431i 2445784 2445783 2026-08-12T08:45:09Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.sanitize-wiki Managing sanitization for wikis testwiki in section s3 2445784 wikitext text/x-wiki da5ayje5dqe8ume5qx5tfg9lcvfhb1m 2445785 2445784 2026-08-12T08:51:17Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-presto1012.eqiad.wmnet with reason: host reimage 2445785 wikitext text/x-wiki r9a98i2zn26i39i21fi7v0f8wic3orq 2445786 2445785 2026-08-12T08:55:27Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1278 to dbctl T434288', diff saved to https://phabricator.wikimedia.org/P95996 and previous config saved to /var/cache/conftool/dbconfig/20260812-085521-marostegui.json 2445786 wikitext text/x-wiki 2n03g75d6qshc4jy1qshb3rp6l11zew 2445787 2445786 2026-08-12T08:55:57Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1278: Pool in x1 2445787 wikitext text/x-wiki 8nq7henyceuq3a5goarrgs0qkixurip 2445788 2445787 2026-08-12T08:58:04Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-launcher1003.eqiad.wmnet with reason: host reimage 2445788 wikitext text/x-wiki tscgd8qy32kegpfne3twi2w86panyeg 2445789 2445788 2026-08-12T09:03:13Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-launcher1003.eqiad.wmnet with reason: host reimage 2445789 wikitext text/x-wiki drk7aovy4yw6jdvfmcdjsacnoxbmx72 2445790 2445789 2026-08-12T09:13:07Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "Turnilo import: fix regexp escaping bug - oblivian@cumin1003" 2445790 wikitext text/x-wiki ik9c8apnvittnzyrhzb7jte434t7usl 2445791 2445790 2026-08-12T09:13:10Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: Turnilo import: fix regexp escaping bug - oblivian@cumin1003 2445791 wikitext text/x-wiki qde7pmgwcm3a9wbm3afh4ab9spxtzyx 2445792 2445791 2026-08-12T09:14:02Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: Turnilo import: fix regexp escaping bug - oblivian@cumin1003 2445792 wikitext text/x-wiki 1z4axxf9p2tmd2j75x06478il9wlrbb 2445793 2445792 2026-08-12T09:14:04Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.hiddenparma (exit_code=0) Hiddenparma deployment to the alerting hosts with reason: "Turnilo import: fix regexp escaping bug - oblivian@cumin1003" 2445793 wikitext text/x-wiki kduf8i6g3uq8yt9fz9fycgq1cmq2m8u 2445794 2445793 2026-08-12T09:16:03Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti3006.esams.wmnet 2445794 wikitext text/x-wiki koya7nwgttpcqoi5tj1bz8g37hcunxh 2445795 2445794 2026-08-12T09:18:04Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti3006.esams.wmnet 2445795 wikitext text/x-wiki mq2tqh5ilg2hbti0w8k9efam5jyrcej 2445796 2445795 2026-08-12T09:23:08Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1009.eqiad.wmnet with OS bookworm 2445796 wikitext text/x-wiki sy9f3kj2w4i6mgsva03c2oyvjp3fuac 2445797 2445796 2026-08-12T09:23:26Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-presto1009.eqiad.wmnet with OS bookworm 2445797 wikitext text/x-wiki ftj77r9xvbfke0ebq6udrs0uldfot3s 2445798 2445797 2026-08-12T09:24:13Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-presto1012.eqiad.wmnet with OS bookworm 2445798 wikitext text/x-wiki ebbpbhlf6dsnsplosxc7rnvisayckr9 2445799 2445798 2026-08-12T09:26:09Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti3006.esams.wmnet 2445799 wikitext text/x-wiki a8m99apwevyvehsvguf8fem0ovxc7mh 2445800 2445799 2026-08-12T09:26:15Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti3006.esams.wmnet 2445800 wikitext text/x-wiki arbzoyooj5h73ueor34355f46fa8f7z 2445802 2445800 2026-08-12T09:29:41Z Stashbot 7414 moritzm: failover ganeti master in esams to ganeti3008 2445802 wikitext text/x-wiki 9x4da1ej9er17sszvgvu7z3f0lym9x1 2445803 2445802 2026-08-12T09:33:37Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1014.eqiad.wmnet with OS bookworm 2445803 wikitext text/x-wiki grxbe2y3g4t4rbj7xhe9hpk2jddb991 2445804 2445803 2026-08-12T09:34:15Z Stashbot 7414 btullis@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host an-presto1013.eqiad.wmnet with OS bookworm 2445804 wikitext text/x-wiki gqrtv3dn39r3rjw0xccu01ijb2x9yik 2445805 2445804 2026-08-12T09:37:27Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1013.eqiad.wmnet with OS bookworm 2445805 wikitext text/x-wiki pxa4hetjrb4upq98gds7p3gq87ps32v 2445806 2445805 2026-08-12T09:37:33Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti3005.esams.wmnet 2445806 wikitext text/x-wiki kcm9darpdwmnyig2vgs6fswcm9qpj88 2445807 2445806 2026-08-12T09:41:04Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-launcher1003.eqiad.wmnet with OS bookworm 2445807 wikitext text/x-wiki 0uob5gbh9hpzp4p1wyeiwmn2icwbhtv 2445808 2445807 2026-08-12T09:41:06Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1278: Pool in x1 2445808 wikitext text/x-wiki ns5ldab9nx3kg6nqvm8fqpmjhpnxuhe 2445809 2445808 2026-08-12T09:49:48Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-presto1014.eqiad.wmnet with reason: host reimage 2445809 wikitext text/x-wiki lychv50y6naozgobw8hg36i9xrpf7g7 2445810 2445809 2026-08-12T09:50:21Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti3005.esams.wmnet 2445810 wikitext text/x-wiki ochwpihphx4wbpxofpvz0kjcajbn7jw 2445811 2445810 2026-08-12T09:50:38Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1009.eqiad.wmnet with OS bookworm 2445811 wikitext text/x-wiki cgjabaei2c3nu05npqzlimppqr41kav 2445812 2445811 2026-08-12T09:53:30Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-presto1014.eqiad.wmnet with reason: host reimage 2445812 wikitext text/x-wiki fv73tdik7hogrgwmb23uqk2rt6c7wp4 2445814 2445812 2026-08-12T10:01:01Z Stashbot 7414 cwilliams@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 32 hosts with reason: Primary switchover s1 T434644 2445814 wikitext text/x-wiki a3jv3ho27zfdzkg97kf1xf93ijnmran 2445815 2445814 2026-08-12T10:01:37Z Stashbot 7414 cwilliams@cumin1003: dbctl commit (dc=all): 'Set db2203 with weight 0 T434644', diff saved to https://phabricator.wikimedia.org/P96001 and previous config saved to /var/cache/conftool/dbconfig/20260812-100134-cwilliams.json 2445815 wikitext text/x-wiki avkjkk66wnxi0453fh4zybhbydcj1yk 2445816 2445815 2026-08-12T10:02:18Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-presto1013.eqiad.wmnet with OS bookworm 2445816 wikitext text/x-wiki ip2kwn9qg3dggk5z90n2ylkjxjax9tb 2445818 2445816 2026-08-12T10:03:56Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1013.eqiad.wmnet with OS bookworm 2445818 wikitext text/x-wiki fr65ixfu23qodgu77vlq2lo4f0enzdn 2445819 2445818 2026-08-12T10:08:04Z Stashbot 7414 cezmunsta: Starting s1 codfw failover from db2212 to db2203 - T434644 2445819 wikitext text/x-wiki rhb8nr0k9698d6t9oy58zr0tw8vmfsx 2445820 2445819 2026-08-12T10:08:52Z Stashbot 7414 cwilliams@cumin1003: dbctl commit (dc=all): 'Promote db2203 to s1 primary T434644', diff saved to https://phabricator.wikimedia.org/P96002 and previous config saved to /var/cache/conftool/dbconfig/20260812-100849-cwilliams.json 2445820 wikitext text/x-wiki bpvfecx1ft6mrehjaa5595fkzdjn12j 2445821 2445820 2026-08-12T10:09:43Z Stashbot 7414 moritzm: powercycle ganeti3005 2445821 wikitext text/x-wiki fouu9zs8919numi0jsjsm6or3ff487c 2445822 2445821 2026-08-12T10:10:56Z Stashbot 7414 cwilliams@cumin1003: dbctl commit (dc=all): 'Depool db2212 T434644', diff saved to https://phabricator.wikimedia.org/P96003 and previous config saved to /var/cache/conftool/dbconfig/20260812-101053-cwilliams.json 2445822 wikitext text/x-wiki qy3jujq5z73e61k5eevai7sf2t6eaiz 2445823 2445822 2026-08-12T10:11:55Z Stashbot 7414 blake@deploy1003: Started scap sync-world: Non-deployment scap run to populate new release values for T427668 2445823 wikitext text/x-wiki gn1ta46dd0v9hswustnx4n9u54wnms7 2445824 2445823 2026-08-12T10:12:27Z Stashbot 7414 blake@deploy1003: Stopping before sync operations 2445824 wikitext text/x-wiki 62pcfkvmmqjlnxp16hgumgt83y3bcg6 2445825 2445824 2026-08-12T10:12:54Z Stashbot 7414 blake@deploy1003: helmfile [codfw] START helmfile.d/services/mw-pretrain: apply 2445825 wikitext text/x-wiki ipd6wa5utbhzckj9nveop3f7lk9vhfw 2445826 2445825 2026-08-12T10:13:07Z Stashbot 7414 blake@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-pretrain: apply 2445826 wikitext text/x-wiki c9apb8vz9x347sjfn2lg7er85j1qq2l 2445827 2445826 2026-08-12T10:13:53Z Stashbot 7414 cwilliams@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2212.codfw.wmnet with reason: Maintenance 2445827 wikitext text/x-wiki 19ncku1oqsgr75yfgs3f6mjqwju2t9w 2445828 2445827 2026-08-12T10:18:22Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-presto1013.eqiad.wmnet with reason: host reimage 2445828 wikitext text/x-wiki rtp92mjcdt6qhlkbwp5kxgvae097m34 2445829 2445828 2026-08-12T10:23:34Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-presto1013.eqiad.wmnet with reason: host reimage 2445829 wikitext text/x-wiki hn36keej7f1iu6o0wkpjyajg3jjkz4r 2445830 2445829 2026-08-12T10:24:53Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db1159: Clone source for db1274 2445830 wikitext text/x-wiki 8tydr5jx5epboithx70kumq2l9dz99d 2445831 2445830 2026-08-12T10:25:22Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1159: Clone source for db1274 2445831 wikitext text/x-wiki 7k97mm1ah8tdv1as2ki4hcad7eediow 2445832 2445831 2026-08-12T10:29:48Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.network.peering with action 'configure' for AS: 396993 2445832 wikitext text/x-wiki jcd3e5j33ol95lm4fxlvxyxgwemkin6 2445833 2445832 2026-08-12T10:31:01Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 396993 2445833 wikitext text/x-wiki ilf8x1p04cq6k03lj8im4elqj4fqu3o 2445834 2445833 2026-08-12T10:33:11Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.multiinstance_reboot for db1274.eqiad.wmnet 2445834 wikitext text/x-wiki g194qm6aqwwkzpxtdhymotgrumqg7rd 2445835 2445834 2026-08-12T10:33:35Z Stashbot 7414 cwilliams@cumin1003: END (FAIL) - Cookbook sre.mysql.multiinstance_reboot (exit_code=99) for db1274.eqiad.wmnet 2445835 wikitext text/x-wiki 1r12rddmmfnv9oxqn8u1fd8v2jyu3uh 2445836 2445835 2026-08-12T10:38:05Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.clone of db1159.eqiad.wmnet onto db1274.eqiad.wmnet 2445836 wikitext text/x-wiki 2qr86enf12sgls7n77qojjjfchkcb8z 2445837 2445836 2026-08-12T10:41:53Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-presto1014.eqiad.wmnet with OS bookworm 2445837 wikitext text/x-wiki a2am4kmxva9q86473p5qu3efzz143pw 2445838 2445837 2026-08-12T10:43:12Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-presto1013.eqiad.wmnet with OS bookworm 2445838 wikitext text/x-wiki 57oijt2djya4bdk0t0xt9alyam2wb7b 2445840 2445838 2026-08-12T10:45:19Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.mysql.sanitize-wiki (exit_code=99) Managing sanitization for wikis testwiki in section s3 2445840 wikitext text/x-wiki 0kg08ujwrlt2iupvn0k2xfwido2ztay 2445841 2445840 2026-08-12T10:48:32Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db2212: Security update 2445841 wikitext text/x-wiki rvh4b6x1e6e2k961t9carmbgoyqy1m4 2445842 2445841 2026-08-12T10:50:16Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1016.eqiad.wmnet with OS bookworm 2445842 wikitext text/x-wiki c5szkco8d0ufi72bjelyuppj3ggzgqt 2445843 2445842 2026-08-12T10:51:05Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1015.eqiad.wmnet with OS bookworm 2445843 wikitext text/x-wiki itcs0d7r1pl3k1hcbch9iooa2vmw8ej 2445844 2445843 2026-08-12T10:54:40Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-presto1009.eqiad.wmnet with OS bookworm 2445844 wikitext text/x-wiki 5tqupfzewllxzfioppxhu5gmr0tczdx 2445845 2445844 2026-08-12T11:00:42Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.sanitize-wiki Managing sanitization for wikis testwiki in section s3 2445845 wikitext text/x-wiki splcaizd3o5dh6uddou17k2gv80u72w 2445846 2445845 2026-08-12T11:02:37Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti6003.drmrs.wmnet 2445846 wikitext text/x-wiki 30kxeokzvbeky92ajmhtmw1uunrja2o 2445847 2445846 2026-08-12T11:04:05Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on 20 hosts with reason: Cloning 2445847 wikitext text/x-wiki e3ksvglwo8c6me2hc69lgpjr6qi716r 2445848 2445847 2026-08-12T11:05:11Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1159: Pool db1159.eqiad.wmnet in after cloning 2445848 wikitext text/x-wiki 5caeqsfk4mncz57dwgih49a7w8y7ppm 2445849 2445848 2026-08-12T11:05:41Z Stashbot 7414 mvolz@deploy1003: helmfile [staging] START helmfile.d/services/citoid: apply 2445849 wikitext text/x-wiki hcp0l5rqtkswa0m0g2bdhjedid5b60p 2445850 2445849 2026-08-12T11:06:01Z Stashbot 7414 mvolz@deploy1003: helmfile [staging] DONE helmfile.d/services/citoid: apply 2445850 wikitext text/x-wiki odn3wfzsx82mk1jj5rtk3pwlsuodntt 2445851 2445850 2026-08-12T11:07:02Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.clone of db1158.eqiad.wmnet onto db1273.eqiad.wmnet 2445851 wikitext text/x-wiki ins6wxv36lncj4xd1kj14jwuxhi0tfb 2445852 2445851 2026-08-12T11:07:05Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1158: Depool db1158.eqiad.wmnet to then clone it to db1273.eqiad.wmnet - marostegui@cumin1003 2445852 wikitext text/x-wiki g4ywk94k31l38pr0q32nid2n88xi3ng 2445853 2445852 2026-08-12T11:07:16Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-presto1016.eqiad.wmnet with reason: host reimage 2445853 wikitext text/x-wiki t3f9ae7znjubswg16wmechgvwpjq47f 2445854 2445853 2026-08-12T11:08:51Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1158: Depool db1158.eqiad.wmnet to then clone it to db1273.eqiad.wmnet - marostegui@cumin1003 2445854 wikitext text/x-wiki pm0n3prly36fw6dqaz6wqehxb8nf4fk 2445855 2445854 2026-08-12T11:09:09Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1009.eqiad.wmnet with OS bookworm 2445855 wikitext text/x-wiki gxf2urtqzsy851njsm6d2zdr3cyav8w 2445856 2445855 2026-08-12T11:10:35Z Stashbot 7414 jmm@cumin2003: END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host ganeti3005.esams.wmnet 2445856 wikitext text/x-wiki hcfllye5vf5b8iuladky63noc7wbcu2 2445857 2445856 2026-08-12T11:10:37Z Stashbot 7414 jmm@cumin2003: END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti3005.esams.wmnet 2445857 wikitext text/x-wiki ceonp00jn73tn7gauzc7oowe755iko7 2445858 2445857 2026-08-12T11:11:12Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-presto1009.eqiad.wmnet with reason: host reimage 2445858 wikitext text/x-wiki 5bsjcmxdqenxbxajkoo70xd36sqvqgo 2445859 2445858 2026-08-12T11:11:29Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti6003.drmrs.wmnet 2445859 wikitext text/x-wiki noawsy4s2ky0vbnsp8h6xwy9s5kq9h9 2445860 2445859 2026-08-12T11:14:24Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-presto1016.eqiad.wmnet with reason: host reimage 2445860 wikitext text/x-wiki czc4zrwklohcrneeafavv6wc1m21ya2 2445861 2445860 2026-08-12T11:16:51Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] START helmfile.d/services/citoid: apply 2445861 wikitext text/x-wiki 646tbes9m1zh534qglcurrqngn0hz7x 2445862 2445861 2026-08-12T11:17:20Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] DONE helmfile.d/services/citoid: apply 2445862 wikitext text/x-wiki bsph8772sd1h3x5tldb1iyxrb6fkyyn 2445863 2445862 2026-08-12T11:17:27Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-presto1009.eqiad.wmnet with reason: host reimage 2445863 wikitext text/x-wiki cpigpy4i7gz9uynu4kgz26hkof0lfo6 2445864 2445863 2026-08-12T11:17:38Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti6003.drmrs.wmnet 2445864 wikitext text/x-wiki t0c8ywwhbf5ijthe5bgx59iw9byoexp 2445865 2445864 2026-08-12T11:17:42Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti6003.drmrs.wmnet 2445865 wikitext text/x-wiki 36cfcfbykvhlzb5ga713et9vf54i5ve 2445866 2445865 2026-08-12T11:18:18Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on 22 hosts with reason: Cloning 2445866 wikitext text/x-wiki ixpzpbti8ngc3gn9tq0649nmggwm72y 2445868 2445866 2026-08-12T11:20:19Z Stashbot 7414 mvolz@deploy1003: helmfile [eqiad] START helmfile.d/services/citoid: apply 2445868 wikitext text/x-wiki 2yapq0zxnfgxc87xdz20a3ztwyrpgb9 2445869 2445868 2026-08-12T11:20:49Z Stashbot 7414 mvolz@deploy1003: helmfile [eqiad] DONE helmfile.d/services/citoid: apply 2445869 wikitext text/x-wiki 9azaogo7g9ztggt8mcivetbj5kb6jg1 2445870 2445869 2026-08-12T11:22:55Z Stashbot 7414 moritzm: failover ganeti master in drmrs01 to ganeti6003 2445870 wikitext text/x-wiki 3wi246018d4z38948xmltp1upls2ayx 2445871 2445870 2026-08-12T11:32:45Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1324306|WikimediaAntiAbuse: Enable personal info for enwiki with no display (T431292)]] 2445871 wikitext text/x-wiki gbk6c44yo1qckt5wom7hf0lfc8cm4cw 2445872 2445871 2026-08-12T11:33:38Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti6001.drmrs.wmnet 2445872 wikitext text/x-wiki mym7riutx9178a91ezrqn9l4om7eu6k 2445873 2445872 2026-08-12T11:33:42Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2212: Security update 2445873 wikitext text/x-wiki 0olb0xco4sntg0kcmsyduzqsur333ip 2445874 2445873 2026-08-12T11:34:50Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1324306|WikimediaAntiAbuse: Enable personal info for enwiki with no display (T431292)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2445874 wikitext text/x-wiki kyvac8mzig58ruapbuvd7128jko4cg3 2445875 2445874 2026-08-12T11:38:49Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti6001.drmrs.wmnet 2445875 wikitext text/x-wiki s5bjkm5e7pdrivaw93979s620a26ne7 2445876 2445875 2026-08-12T11:39:02Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2445876 wikitext text/x-wiki kidhz8g1t5r5k3esszc3c1qarunwq20 2445877 2445876 2026-08-12T11:42:16Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-presto1015.eqiad.wmnet with OS bookworm 2445877 wikitext text/x-wiki pg2kyqironyqllapwjl1hj5bj5sx6pc 2445878 2445877 2026-08-12T11:43:11Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324306|WikimediaAntiAbuse: Enable personal info for enwiki with no display (T431292)]] (duration: 10m 26s) 2445878 wikitext text/x-wiki g51lni7p7e0ir6fjsv6yo0c8pn5jr13 2445879 2445878 2026-08-12T11:45:05Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti6001.drmrs.wmnet 2445879 wikitext text/x-wiki 6jllwsiipxp8h8qnwyheqgdskw4436b 2445880 2445879 2026-08-12T11:45:22Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti6001.drmrs.wmnet 2445880 wikitext text/x-wiki 71rsq90mmx1961mbjyzq5cvvvka8aqn 2445881 2445880 2026-08-12T11:46:43Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti6004.drmrs.wmnet 2445881 wikitext text/x-wiki 701hki45mf6gwseg8v3t98v1m5wl1gg 2445882 2445881 2026-08-12T11:48:02Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-presto1016.eqiad.wmnet with OS bookworm 2445882 wikitext text/x-wiki 5yy3y60vjov0lx7zgls9ezzivr5mz0y 2445884 2445882 2026-08-12T11:50:24Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1159: Pool db1159.eqiad.wmnet in after cloning 2445884 wikitext text/x-wiki 7tire0jyuy7f26oy935iykax5wq1sns 2445885 2445884 2026-08-12T11:50:40Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1159.eqiad.wmnet onto db1274.eqiad.wmnet 2445885 wikitext text/x-wiki 4l0fxcnceswvb1nxgqmv3a5rghsjhp9 2445886 2445885 2026-08-12T11:53:27Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-presto1015.eqiad.wmnet with OS bookworm 2445886 wikitext text/x-wiki rtyttbnc3g2j27yfo8un98ntlvpg29y 2445887 2445886 2026-08-12T11:53:43Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti6004.drmrs.wmnet 2445887 wikitext text/x-wiki 9a4t75dtrvmgpv7991ad36yqljrthrb 0 22777 2445867 2434109 2026-08-12T11:19:37Z CGoubert-WMF 32775 /* Assign a hostname */ Formatting 2445867 wikitext text/x-wiki k4mp5h6177cn4nz2a5e2d6ibfgyw5p0 0 23732 2445480 2427841 2026-08-11T13:40:55Z Jynus 38174 /* Dashboard */ naming it also so it can be easily searchable 2445480 wikitext text/x-wiki k659o8il6wo9il7z3tkexjgcx2qnsce 498 30942 2445549 2445260 2026-08-11T14:58:03Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.roll_restart_osd_daemons (1324319) 2445549 wikitext text/x-wiki icsmtxdtm318u26sx5x7mccvb2snmx4 2445567 2445549 2026-08-11T15:20:54Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.roll_restart_osd_daemons (exit_code=0) (1324319) 2445567 wikitext text/x-wiki 5okmtdzd3lnchzp7rrz2b7q7kr2m006 0 49472 2445654 2363616 2026-08-11T18:41:13Z BKing (WMF) 29017 /* Upgrade Matomo */ 2445654 wikitext text/x-wiki 2mjkvrr2hu4g2g3tpcj1abn3ygwmvcj 2445688 2445654 2026-08-11T21:00:16Z BKing (WMF) 29017 /* Upgrade Matomo */ 2445688 wikitext text/x-wiki jxrc67fa4bbxowre0xnzieqooedqql2 2445689 2445688 2026-08-11T21:02:34Z BKing (WMF) 29017 /* Upgrade Matomo */ 2445689 wikitext text/x-wiki gq9j5d0bobqh6rd3io9m7nts3lhhdrm 2445695 2445689 2026-08-11T21:24:39Z BKing (WMF) 29017 /* Upgrade Matomo */ 2445695 wikitext text/x-wiki j6a0wemw0um17f05vdilpkmvb3i2ray 2445697 2445695 2026-08-11T21:30:37Z BKing (WMF) 29017 /* Updating the Plugins */ 2445697 wikitext text/x-wiki o2cs613afmj5pbtq64zqtmxhkql8wn0 2445765 2445697 2026-08-12T07:58:57Z BTullis (WMF) 25295 Added details about data retention and visitor privacy 2445765 wikitext text/x-wiki dt8u306mnfmetyq742602ictv22i51k 0 350482 2445600 2439108 2026-08-11T16:09:06Z BCornwall-WMF 31521 /* Packaging */ better wording for dgit/debian changes 2445600 wikitext text/x-wiki eft2v3y1bw786zis9k55268p1g9ncr4 2445601 2445600 2026-08-11T16:10:00Z BCornwall-WMF 31521 /* Packaging */ mention that no debian changes may have occured 2445601 wikitext text/x-wiki 8bqcq4wcspy3l5zkx661vh43655j46z 2445603 2445601 2026-08-11T16:16:49Z BCornwall-WMF 31521 /* Packaging */ fix git-diff version ordering 2445603 wikitext text/x-wiki d0b77c1idj18lcd301l449yyi0jqbdr 0 441958 2445883 2435258 2026-08-12T11:49:38Z Filippo Giunchedi 2939 /* Remove hypervisor */ Add resource provider delete 2445883 wikitext text/x-wiki a42vuy06y5y75t66izbla34jw2g76pn 498 443713 2445592 2444206 2026-08-11T16:01:38Z Stashbot 7414 wmbot~valeriobozzolan@tools-bastion-15: deletionbot: released latest changes - up to commit https://gitlab.wikimedia.org/valeriobozzolan/itwiki-deletions-info-bot/-/commit/bafcd1b47a083e8f79f877572f45c90525e730ec 2445592 wikitext text/x-wiki 414q19exydwjuew4lqo9x92vfypi207 0 446419 2445590 1875489 2026-08-11T15:59:56Z Effie Mouzeli (WMF) 12880 Creating a new article for MW 2445590 wikitext text/x-wiki kd1620ukml0ssm2lahetsf8y997ay3c 0 447901 2445496 2394237 2026-08-11T13:58:00Z TBurmeister (WMF) 27417 Make developer guide link more visible 2445496 wikitext text/x-wiki ird58jgk7wk8tv3xhylzjqb2pkjq9nf 0 449160 2445447 2445389 2026-08-11T12:53:23Z Dexbot 30554 Bot: Updating the report 2445447 wikitext text/x-wiki eww3uqesl0h2s0osd2s38wmcxjmmz20 2445552 2445447 2026-08-11T15:05:45Z Dexbot 30554 Bot: Updating the report 2445552 wikitext text/x-wiki 6xwg5s7ovtenvx0pqx2f50k1zerkdvd 2445705 2445552 2026-08-12T00:01:04Z Dexbot 30554 Bot: Updating the report 2445705 wikitext text/x-wiki iyz5tv9ktrdmvpi1g0iy2majfzlioj0 2445773 2445705 2026-08-12T08:29:16Z Dexbot 30554 Bot: Updating the report 2445773 wikitext text/x-wiki boh7fia44b0nxat4i5wz8lfgnmqtgz4 2445817 2445773 2026-08-12T10:02:30Z Dexbot 30554 Bot: Updating the report 2445817 wikitext text/x-wiki m8n3imjlaodn6wdwvk21v9ze1stkvck 116 453906 2445523 2445293 2026-08-11T14:15:30Z Gitlabaccountapprovalbot 37332 @mmilenkovicwmf was approved. 2445523 wikitext text/x-wiki mqxp9l57p6vs8nzsi4q5iplf8tnxm1t 0 454559 2445839 2434734 2026-08-12T10:43:17Z MMuhlenhoff (WMF) 5268 /* How to get notified of security issues */ Add a note on how to track Juniper issues 2445839 wikitext text/x-wiki n8heofsd67mrz4vt0gx9m3ee2rlvbqe 498 458781 2445702 2401958 2026-08-11T23:27:24Z BryanDavis 1604 /* Requests from your IP have been blocked */ add a bit about where to find your IPv4 2445702 wikitext text/x-wiki aud2d4e61k4llecoo6a1bgg5xdtzuyy 0 459326 2445488 2353233 2026-08-11T13:54:45Z TBurmeister (WMF) 27417 Add link to end user Airflow dev guide 2445488 wikitext text/x-wiki 7oeth7ybfeolkxnrrhehf9hkhiby8uy 0 459370 2445701 2376169 2026-08-11T22:50:16Z WFan (WMF) 38087 2445701 wikitext text/x-wiki 45lczhlrib76jbbgc1bbljyse12cxm3 0 459560 2445674 2441953 2026-08-11T20:20:52Z DAdedoyin (WMF) 39150 Test 2445674 wikitext text/x-wiki dya6dgxxktz2mwefawwfd71v83pnwn6 0 459699 2445604 2445297 2026-08-11T16:22:20Z Stashbot 7414 damilare: payments-wiki upgraded from 5482106f to fff3c1de 2445604 wikitext text/x-wiki gzazab0282emy3tz14se8lebm6bqmji 2445675 2445604 2026-08-11T20:34:53Z Stashbot 7414 eileen: civicrm upgraded from e701061b to 7659a901 2445675 wikitext text/x-wiki 472ywswcme5tu8ju3qgxf2jmjllp4na 2445680 2445675 2026-08-11T20:39:33Z Stashbot 7414 ejegg: standalone (IPN listener) SmashPig upgraded from aee58123 to 3acbe469 2445680 wikitext text/x-wiki 71stpfccjgkqaowaoqmf2599n8fmuwm 2445693 2445680 2026-08-11T21:17:21Z Stashbot 7414 eileen: civicrm upgraded from 7659a901 to 816627cd 2445693 wikitext text/x-wiki tpa7mv78ylet8xjbzyhddzhbzv2fn5e 2445699 2445693 2026-08-11T22:38:29Z Stashbot 7414 eileen: config revision changed from 4610d391 to de01935f 2445699 wikitext text/x-wiki gsua97czg9d9kwgw08ejaerxpgzcdmq 2445700 2445699 2026-08-11T22:41:13Z Stashbot 7414 eileen: civicrm upgraded from 816627cd to dcd4bd86 2445700 wikitext text/x-wiki goa4tx56xjn1f7xw4azwnwypjxogvg9 2445703 2445700 2026-08-11T23:42:44Z Stashbot 7414 eileen: civicrm upgraded from dcd4bd86 to 2d09ad88 2445703 wikitext text/x-wiki 9jnnjf48dvlbyojm0y0ka0o1btftvkj 2445704 2445703 2026-08-11T23:46:23Z Stashbot 7414 eileen: civicrm upgraded from 2d09ad88 to 5bf16724 2445704 wikitext text/x-wiki 0l7jba4y3nk6jkt4qnqgzrf650gsfvi 2445710 2445704 2026-08-12T00:27:56Z Stashbot 7414 eileen: civicrm upgraded from 5bf16724 to e0e88ae9 2445710 wikitext text/x-wiki s81cax4i6q43apae08bli5dy5s0ok5a 2445714 2445710 2026-08-12T02:03:42Z Stashbot 7414 eileen: civicrm upgraded from e0e88ae9 to 0d8a3fcd 2445714 wikitext text/x-wiki d3hn7b66tl0713ickrq4lowo4xph6ct 2445716 2445714 2026-08-12T02:55:43Z Stashbot 7414 cstone: tools upgraded from 93a9d78d to 9f2c4da9 2445716 wikitext text/x-wiki ixz7lchdfsvfe6bpd8vsyggglzbkxo1 0 460161 2445771 2428378 2026-08-12T08:26:13Z T Arrow 1475 Remove reference to label service rewriting after slack discussion with Wikidata Platform Team which is now not part of the architecture 2445771 wikitext text/x-wiki kzi8tqyqv8uxm1ilxa944a1dshims07 116 460408 2445586 2435552 2026-08-11T15:43:39Z Ancient9983 52221 /* Developer documentation */ Add some useful links to developer documentation. 2445586 wikitext text/x-wiki gd3bi1o2lge5ibx93g4mg917i24i2sc 0 460417 2445801 2444007 2026-08-12T09:26:43Z AChou-WMF 15621 Add Data & privacy section 2445801 wikitext text/x-wiki 1vb3hilgq345lklx60sd3iyd4x05ge4 0 460460 2445813 2444616 2026-08-12T09:56:48Z AChou-WMF 15621 Add FAQ 2445813 wikitext text/x-wiki h15v84k7pzojp5zsw8zlepwcoi8icv7 2 460540 2445472 2445303 2026-08-11T13:33:34Z Ottomata 88 2445472 wikitext text/x-wiki 4znft7prgq9gsiykzcbd0za1exr62et 2445647 2445472 2026-08-11T18:07:04Z Isaac (WMF) 13091 /* Article Features */ in summary table: cleaned up links, added documentation for navboxes and removed headings (duplicate) and templates (largely a wikitext feature) 2445647 wikitext text/x-wiki gnwk7j95izlim22w08ngc1lxl69rfk0 2445653 2445647 2026-08-11T18:34:17Z Ottomata 88 2445653 wikitext text/x-wiki ne68dbj5a261run63vmlpy7eqfz35j6 2445655 2445653 2026-08-11T18:52:19Z Ottomata 88 slack thread consultations 2445655 wikitext text/x-wiki k7te3bcgu3aglpsl2vmarrkmgdpwv50 2445657 2445655 2026-08-11T18:55:48Z Ottomata 88 /* Slack thread consultations */ 2445657 wikitext text/x-wiki 12r2p5bjgboirh0g3cfzxwqrp2l0vhu 2445665 2445657 2026-08-11T19:40:51Z Ottomata 88 /* Slack thread consultations */ 2445665 wikitext text/x-wiki evp4dihpmu1gtqawjcr1hljfbackg8v 0 460545 2445415 2026-08-11T12:05:05Z JMonton-WMF 47887 Created page with "Placeholder for `webrequest.pageview.v1` enrichment." 2445415 wikitext text/x-wiki bintdi6be3gajo7xageah39t9u07kre