Change #1071064 had a related patch set uploaded (by Krinkle; author: Krinkle):
[mediawiki/extensions/BounceHandler@master] BounceHandlerActions: Simplify and clean up code
https://gerrit.wikimedia.org/r/1071064
Change #1071064 had a related patch set uploaded (by Krinkle; author: Krinkle):
[mediawiki/extensions/BounceHandler@master] BounceHandlerActions: Simplify and clean up code
https://gerrit.wikimedia.org/r/1071064
I've documented the following on Wikitech: https://wikitech.wikimedia.org/wiki/BounceHandler
BounceHandler has signs of life again. Thanks @Joe!
I'm finally seeing bounces get processed in logstash https://logstash.wikimedia.org/goto/3d34190bb82088f19669b0c66331d7c9
Mentioned in SAL (#wikimedia-operations) [2024-09-05T09:37:20Z] <oblivian@deploy1003> Finished scap sync-world: Backport for [[gerrit:1070879|BounceHandler: add IPs for the new mx servers (T338761)]] (duration: 11m 16s)
Mentioned in SAL (#wikimedia-operations) [2024-09-05T09:32:39Z] <oblivian@deploy1003> oblivian: Backport for [[gerrit:1070879|BounceHandler: add IPs for the new mx servers (T338761)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)
Mentioned in SAL (#wikimedia-operations) [2024-09-05T09:26:04Z] <oblivian@deploy1003> Started scap sync-world: Backport for [[gerrit:1070879|BounceHandler: add IPs for the new mx servers (T338761)]]
Change #1070879 merged by jenkins-bot:
[operations/mediawiki-config@master] BounceHandler: add IPs for the new mx servers
https://gerrit.wikimedia.org/r/1070879
Change #1070879 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):
[operations/mediawiki-config@master] BounceHandler: add IPs for the new mx servers
https://gerrit.wikimedia.org/r/1070879
Change #1070858 merged by Giuseppe Lavagetto:
[operations/puppet@production] postfix: add X-Client-IP to VERP handler
https://gerrit.wikimedia.org/r/1070858
Change #1070858 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):
[operations/puppet@production] postfix: add X-Client-IP to VERP handler
https://gerrit.wikimedia.org/r/1070858
I still see the errors in the logs, and it's baffling. In fact, I've tried the command now listed in exim's configuration:
Change #1070805 merged by Giuseppe Lavagetto:
[operations/puppet@production] exim: fix VERP handling for mediawiki
https://gerrit.wikimedia.org/r/1070805
Change #1070805 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):
[operations/puppet@production] exim: fix VERP handling for mediawiki
https://gerrit.wikimedia.org/r/1070805
The check is done using Webrequest::getIP() which uses REMOTE_ADDR as a source for the address, and then overrides that using X-Forwarded-For only for trusted proxies.
Quick query against puppet found this change to mx around the same time: https://gerrit.wikimedia.org/r/c/operations/puppet/+/766571
Well, it think it's official. It's down.
I'm not aware of a way to escalate unowned code other than via MediaWiki-Engineering intake. This isn't great, but it does need to be looked at. Tagging here to start, acknowleding that we may end up delegating it elsewhere in the org.
Thank you @Amdrel
This extension doesn't need updating. None of the results from the code search look like they need updating and temporary accounts cannot have emails associated with them in the first place anyway.
BounceHandler logs in the last 90 days are all POST received from restricted IP <internal appserver ip>. Some process wants to process bounces, but cannot because it's not an mx server.
Still broken.
Change #1010165 merged by jenkins-bot:
[mediawiki/extensions/BounceHandler@master] Migrate addQuotes to expression builder
https://gerrit.wikimedia.org/r/1010165
Change 1010165 had a related patch set uploaded (by Atieno; author: Atieno):
[mediawiki/extensions/BounceHandler@master] Migrate addQuotes to expression builder
https://gerrit.wikimedia.org/r/1010165
Change 960210 merged by jenkins-bot:
[mediawiki/extensions/BounceHandler@master] Use HookHandlers for core hooks
Change 960210 had a related patch set uploaded (by Umherirrender; author: Umherirrender):
[mediawiki/extensions/BounceHandler@master] Use HookHandlers for core hooks
Was added with d8eebdb205f3f12cd1086635a23441f9e8a41886, so very old (REL1_27) and no need for backporting.
Change 919466 merged by jenkins-bot:
[mediawiki/extensions/BounceHandler@master] Call SpecialPage::getTitleFor with the internal special page name
Change 919466 had a related patch set uploaded (by Umherirrender; author: Umherirrender):
[mediawiki/extensions/BounceHandler@master] Call SpecialPage::getTitleFor with the internal special page name
This is done
Change 883506 merged by Marostegui:
[operations/dns@master] wmnet: Update x1-master alias
Mentioned in SAL (#wikimedia-operations) [2023-01-26T07:01:59Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1120 T327861', diff saved to https://phabricator.wikimedia.org/P43352 and previous config saved to /var/cache/conftool/dbconfig/20230126-070158-root.json
Mentioned in SAL (#wikimedia-operations) [2023-01-26T07:00:36Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Promote db1103 to x1 primary and set section read-write T327861', diff saved to https://phabricator.wikimedia.org/P43351 and previous config saved to /var/cache/conftool/dbconfig/20230126-070035-marostegui.json
Mentioned in SAL (#wikimedia-operations) [2023-01-26T07:00:08Z] <marostegui> Starting x1 eqiad failover from db1120 to db1103 - T327861
Change 883694 merged by Marostegui:
[operations/puppet@production] db1120: Disable notifications
Change 883694 had a related patch set uploaded (by Marostegui; author: Marostegui):
[operations/puppet@production] db1120: Disable notifications
Change 882785 merged by Marostegui:
[operations/puppet@production] mariadb: Promote db1103 to x1 master
Mentioned in SAL (#wikimedia-operations) [2023-01-26T06:17:52Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Set db1103 with weight 0 T327861', diff saved to https://phabricator.wikimedia.org/P43350 and previous config saved to /var/cache/conftool/dbconfig/20230126-061751-root.json
Mentioned in SAL (#wikimedia-operations) [2023-01-26T06:17:03Z] <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 10 hosts with reason: Primary switchover x1 T327861
Mentioned in SAL (#wikimedia-operations) [2023-01-26T06:16:44Z] <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on 10 hosts with reason: Primary switchover x1 T327861
All done
Mentioned in SAL (#wikimedia-operations) [2022-09-06T06:08:15Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1103 T316745', diff saved to https://phabricator.wikimedia.org/P33831 and previous config saved to /var/cache/conftool/dbconfig/20220906-060815-root.json
Change 828469 merged by Marostegui:
[operations/dns@master] wmnet: Update x1-master alias