survey: adjust the survey prompt probability computation

[hyangah]

The current logic is biased and favors users who open editors for the first time near the end of the month. Our requirements are:

• Prompting should be approximately uniformly distributed throughout the month and year. As much as possible, we should avoid cyclical behavior.

• The sample of prompted users should approximate our 28d user base. That is to say that any 28d user is approximately equally likely to be prompted throughout the month. Obviously for users that only use VS Code once a month, this means they are more likely to be prompted in each individual session.

• We should aim for 200-300 responses per month. Currently we're around 300.

cc @pjweinb @findleyr @fflewddur

[pjweinb]

I propose just choosing a random day in the following 28 days. The code would be fairly simple, and there are good statistical tests for the uniform distribution. The exact number 28 isn't critical. It's about a month, and it's big enough so that if no one starts vscode on Sundays, the distribution is still fairly flat.

…

On Tue, Nov 22, 2022 at 2:53 PM Hyang-Ah Hana Kim ***@***.***> wrote: Assigned #2545 <#2545> to @pjweinb <https://github.com/pjweinb>. — Reply to this email directly, view it on GitHub <#2545 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJIAI5YNG62RW2WNMY74ITWJUQDDANCNFSM6AAAAAASIGHNU4> . You are receiving this because you were assigned.Message ID: ***@***.***>

[fflewddur]

This sounds great to me. Very elegant solution, thanks @pjweinb!

[hyangah]

Closed with go.dev/cl/457475.
Thanks @pjweinb