Change #1071064 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/extensions/BounceHandler@master] BounceHandlerActions: Simplify and clean up code

https://gerrit.wikimedia.org/r/1071064

I've documented the following on Wikitech: https://wikitech.wikimedia.org/wiki/BounceHandler

BounceHandler has signs of life again. Thanks @Joe!

I'm finally seeing bounces get processed in logstash https://logstash.wikimedia.org/goto/3d34190bb82088f19669b0c66331d7c9

Mentioned in SAL (#wikimedia-operations) [2024-09-05T09:37:20Z] <oblivian@deploy1003> Finished scap sync-world: Backport for [[gerrit:1070879|BounceHandler: add IPs for the new mx servers (T338761)]] (duration: 11m 16s)

Mentioned in SAL (#wikimedia-operations) [2024-09-05T09:32:39Z] <oblivian@deploy1003> oblivian: Backport for [[gerrit:1070879|BounceHandler: add IPs for the new mx servers (T338761)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-09-05T09:26:04Z] <oblivian@deploy1003> Started scap sync-world: Backport for [[gerrit:1070879|BounceHandler: add IPs for the new mx servers (T338761)]]

Change #1070879 merged by jenkins-bot:

[operations/mediawiki-config@master] BounceHandler: add IPs for the new mx servers

https://gerrit.wikimedia.org/r/1070879

Change #1070879 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/mediawiki-config@master] BounceHandler: add IPs for the new mx servers

https://gerrit.wikimedia.org/r/1070879

Change #1070858 merged by Giuseppe Lavagetto:

[operations/puppet@production] postfix: add X-Client-IP to VERP handler

https://gerrit.wikimedia.org/r/1070858

Change #1070858 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/puppet@production] postfix: add X-Client-IP to VERP handler

https://gerrit.wikimedia.org/r/1070858

I still see the errors in the logs, and it's baffling. In fact, I've tried the command now listed in exim's configuration:

Change #1070805 merged by Giuseppe Lavagetto:

[operations/puppet@production] exim: fix VERP handling for mediawiki

https://gerrit.wikimedia.org/r/1070805

Change #1070805 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/puppet@production] exim: fix VERP handling for mediawiki

https://gerrit.wikimedia.org/r/1070805

The check is done using Webrequest::getIP() which uses REMOTE_ADDR as a source for the address, and then overrides that using X-Forwarded-For only for trusted proxies.

Quick query against puppet found this change to mx around the same time: https://gerrit.wikimedia.org/r/c/operations/puppet/+/766571

Well, it think it's official. It's down.

I'm not aware of a way to escalate unowned code other than via MediaWiki-Engineering intake. This isn't great, but it does need to be looked at. Tagging here to start, acknowleding that we may end up delegating it elsewhere in the org.

Thank you @Amdrel

This extension doesn't need updating. None of the results from the code search look like they need updating and temporary accounts cannot have emails associated with them in the first place anyway.

BounceHandler logs in the last 90 days are all POST received from restricted IP <internal appserver ip>. Some process wants to process bounces, but cannot because it's not an mx server.

Still broken.

Done in 5928a49d33b4a7d1f6576e2ad846118b273735c3

Change #1010165 merged by jenkins-bot:

[mediawiki/extensions/BounceHandler@master] Migrate addQuotes to expression builder

https://gerrit.wikimedia.org/r/1010165

Change 1010165 had a related patch set uploaded (by Atieno; author: Atieno):

[mediawiki/extensions/BounceHandler@master] Migrate addQuotes to expression builder

https://gerrit.wikimedia.org/r/1010165

Change 960210 merged by jenkins-bot:

[mediawiki/extensions/BounceHandler@master] Use HookHandlers for core hooks

https://gerrit.wikimedia.org/r/960210

Change 960210 had a related patch set uploaded (by Umherirrender; author: Umherirrender):

[mediawiki/extensions/BounceHandler@master] Use HookHandlers for core hooks

https://gerrit.wikimedia.org/r/960210

Was added with d8eebdb205f3f12cd1086635a23441f9e8a41886, so very old (REL1_27) and no need for backporting.

Change 919466 merged by jenkins-bot:

[mediawiki/extensions/BounceHandler@master] Call SpecialPage::getTitleFor with the internal special page name

https://gerrit.wikimedia.org/r/919466

Change 919466 had a related patch set uploaded (by Umherirrender; author: Umherirrender):

[mediawiki/extensions/BounceHandler@master] Call SpecialPage::getTitleFor with the internal special page name

https://gerrit.wikimedia.org/r/919466

This is done

Change 883506 merged by Marostegui:

[operations/dns@master] wmnet: Update x1-master alias

https://gerrit.wikimedia.org/r/883506

Mentioned in SAL (#wikimedia-operations) [2023-01-26T07:01:59Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1120 T327861', diff saved to https://phabricator.wikimedia.org/P43352 and previous config saved to /var/cache/conftool/dbconfig/20230126-070158-root.json

Mentioned in SAL (#wikimedia-operations) [2023-01-26T07:00:36Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Promote db1103 to x1 primary and set section read-write T327861', diff saved to https://phabricator.wikimedia.org/P43351 and previous config saved to /var/cache/conftool/dbconfig/20230126-070035-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2023-01-26T07:00:08Z] <marostegui> Starting x1 eqiad failover from db1120 to db1103 - T327861

Change 883694 merged by Marostegui:

[operations/puppet@production] db1120: Disable notifications

https://gerrit.wikimedia.org/r/883694

Change 883694 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] db1120: Disable notifications

https://gerrit.wikimedia.org/r/883694

Change 882785 merged by Marostegui:

[operations/puppet@production] mariadb: Promote db1103 to x1 master

https://gerrit.wikimedia.org/r/882785

Mentioned in SAL (#wikimedia-operations) [2023-01-26T06:17:52Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Set db1103 with weight 0 T327861', diff saved to https://phabricator.wikimedia.org/P43350 and previous config saved to /var/cache/conftool/dbconfig/20230126-061751-root.json

Mentioned in SAL (#wikimedia-operations) [2023-01-26T06:17:03Z] <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 10 hosts with reason: Primary switchover x1 T327861

Mentioned in SAL (#wikimedia-operations) [2023-01-26T06:16:44Z] <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on 10 hosts with reason: Primary switchover x1 T327861

All done

Mentioned in SAL (#wikimedia-operations) [2022-09-06T06:08:15Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1103 T316745', diff saved to https://phabricator.wikimedia.org/P33831 and previous config saved to /var/cache/conftool/dbconfig/20220906-060815-root.json

Change 828469 merged by Marostegui:

[operations/dns@master] wmnet: Update x1-master alias

https://gerrit.wikimedia.org/r/828469