Wikipedia:Dispute Resolution Improvement Project/June report

Hi all! I thought it was time I provided an update on my work so far, so the community has an understanding of what I've been doing and what I've learned, along with what I am currently working on and my plans for the future.

Early results of the survey

Over the past month, I have been working on compiling the results of the dispute resolution survey that was offered in April 2012, and preparing these results for the community.

The purpose of this survey was to get Wikipedians, in their own words, to explain how they felt about dispute resolution on the English Wikipedia and their experiences with it. Demographic information and feedback on the processes was gathered, as well as first-hand accounts of their experiences. The respondents detailed processes that they think are effective, but also highlighted issues that they have experienced that made things difficult. The survey was offered to members of the community based on their activity within dispute resolution - editors who had more than ten edits to a range of dispute resolution forums in the period of February 2010 – February 2012, that were not indefinitely blocked at the time, were eligible. 1952 users were eligible to take the survey, and it was offered to the top 1060 active editors (1100 originally, however 40 were blocked). The survey received 238 valid responses (a response rate of just over 22%, and the results were compared with the m:Editor Survey 2011.

The full survey results are collated, however I am still in the process of writing up detailed findings. Some highlights of the report include:

Over half of all respondents (and 80% of females) were older than 40. Only 8.4% of respondents were female, roughly in-line with the results of the Editor survey at 9%.
94% had requested assistance from a dispute resolution forum at some point, and were generally unhappy with their experiences in dispute resolution – only one in five were satisfied – however despite this 62% had participated in dispute resolution within the last year.
Requests for Comment is the most used dispute resolution forum, with 60% participating in the last year. Mediation Committee proceedings were used the least, only 10% were involved in proceedings.
Over half of the respondents offered assistance in a dispute within the last year – however some forums had more assistance than others.
Opinions of dispute resolution were overall relatively negative - Arbitration was rated as the best dispute resolution forum by respondents – with one in three respondents rating it as good or better. In contrast, Wikiquette assistance was rated the worst – only 1 in 12 rated it as satisfactory.
Dispute resolution volunteers do so because they felt the process was critical to Wikipedia functioning, liked helping people or as payback for previous assistance.
Some respondents haven’t volunteered due to the unpleasantness of disputes, the prolonged nature of dispute resolution, or due to poor past experiences or a lack of knowledge in resolving disputes.
When asked about their personal experiences with dispute resolution, positive aspects were that their dispute was resolved, the examples set by volunteers and the positive behavior of their fellow participants, while negative aspects included the time it takes to resolve a dispute, and the potential for the the processes to become unfair - many citing the source of this unfairness as as administrators that became involved in the process.
Dispute resolution is most effective at resolving issues over policy and its interpretation, and issues with reliable sources, according to 2 in 5 respondents
The main problems with dispute resolution are its complexity, its inaccessibility, and the there are too many resolution processes and not enough volunteers.
Respondents want stricter action taken against problematic editors, a simplified, more accessible process where closure can be bought to a dispute quickly
Possible technical changes proposed include the ability to block editors from editing specific pages, a bot which could detect disputes and the creation of a filter that could warn someone before they breach the three-revert rule
A little over half of the respondents were interested in participating in further discussions on improving dispute resolution, and a third were interested in learning how to resolve disputes, or teach others.

Findings based on the results

From the results, there are some positives and negatives. When disputes are attended to by experienced volunteers with co-operative fellow editors, respondents were generally satisfied with their experience - but complex, time consuming processes, inexperienced or insufficient volunteers or uncooperative fellow editors make the experience an unpleasant one. 70% of respondents have volunteered with dispute resolution at some point, however only 40% did so in the month of March 2012 - many of these to talk pages only, and this may be due the complexity of the processes or lack of understanding on how to resolve disputes.

Based on the results of the survey, my recommendations to the community are to:

Find ways to make the process more straightforward and easy to use – potentially by amalgamating the many existing dispute resolution processes into a few consolidated processes – making them more inviting for new volunteers and less complex for people that need assistance
Have existing volunteers in dispute resolution help develop a how-to guide with tips and tricks that can be used to resolve disputes – giving new volunteers the confidence to assist in dispute resolution– and possibly giving participants ways they can resolve a dispute without requiring assistance.
Discuss technical changes such as page-specific blocking, as well as policy changes such as time-fixed binding resolution of contentious content disputes, or by giving dispute resolution “teeth” – making the process more binding and consequential

Activity report and dispute resolution manual

Following up the results of this survey, an analysis of existing dispute resolution techniques will be undertaken to measure the effectiveness of the processes. A more detailed methodology can be found here, however the basic metrics will be sought after:

Number of disputes that were filed in a month - to assess usage of dispute resolution.
Number of editors participating in dispute resolution - likewise to assess usage.
Response time, number of volunteers and resolution time- to assess the impact of volunteers. I want to see how long it takes on average for a dispute to be responded to, how many are responding to disputes, and how long on average it takes to resolve a dispute.
Success rate - how many disputes filed vs how many resolved - if the dispute has continued on another dispute resolution forum or the talk page, it will be deemed unresolved. I also will compare the success rate with the resolution time to see if there is a link between the time it takes to resolve a dispute compared to how many are successfully resolved.

Solutions this project will focus on

From the results of the activity analysis, determine which dispute resolution forums may be closed or merged with others to make the process simpler. Develop an easy to use navigation page where editors can go to find the best place to resolve their dispute.
A how-to manual for volunteers to guide them on how to recognise various forms of disputes, as well as techniques they can use to resolve them.
Discussing technical changes with developers and the community, along with policy changes to give dispute resolution more “teeth”.

As always, if you have any questions feel free to ask me via email or my talk page. Regards, Steven Zhang ^{Get involved in DR!} 07:42, 2 July 2012 (UTC)[reply]