MediaWiki

Wikimedia Apps/Team/Android/荒らし対策

< Wikimedia Apps | Team | Android
This page is a translated version of the page Wikimedia Apps/Team/Android/Anti Vandalism and the translation is 17% complete.
Outdated translations are marked like this.
Languages:

背景

Android チームではさまざまな言語版のコミュニティから、アプリ版を使った編集の品質を改善してほしいと要請を受けています。 言語版やウィキの枠を越えてコミュニティ参加者の皆さんから公平で偏重のない仲裁用のツールがほしいという声が寄せられています。 当チームの目標は仲裁の公平性を適度に確保しながら既存のワークフロー（作業手順）を強化し、例は2022年コミュニティ要望調査のこちらになります。

アプリによってネーティブなウォッチリスト、投稿履歴、編集履歴などの最近の更新、また差分画面に元に戻すボタンとロールバック・ボタンの追加ばかりでなく、さらにアプリ内で仲裁の解決策を作成する機会も得られます。解決策は既存のモバイル用マイクロ貢献ワークフローに基づいて構築されます。

当チームでは、さまざまな言語版で経験豊富なモデレーターのお役に立つ破壊行為対策ツール（V1）を開発する予定です。さらに、経験豊富でも、これまでに巡回を行ったことがない編集者には、巡回の方法を習得する研修タスクを導入する予定です。これはコミュニティと緊密に協力して実施します。またプラットフォーム横断型の利用者にシームレスな体験を確保することを目指して、ORES などのツール改善は研究チームと協力、さらに、この問題についてモバイル・ウェブ版のツールに取り組むモデレーター・ツール・チームと協調していきます。

製品の要件：

  • 編集者はさまざまな編集の差分を確認できる
  • 編集者には、提案された変更を「元に戻す」または「スキップ」する選択肢がある
  • 最近の編集について、さまざまな言語の内容を表示できる
  • 編集者は警告メッセージを作成して保存、変更を元に戻すときに使用できる。
  • 経験豊富な編集者にはバージョン（V2）を待ち、アプリ経由の巡回を紹介予定。

技術的な制限

現状ではフィード1回で複数言語を扱えませんが、利用者は言語間の切り替えができるはずです。

仮説と検証

Hypothesis & Assumptions

We believe that simplifying the patrolling tasks and structuring them as advanced-level tasks would be beneficial. Experienced editors without sysops or rollback rights could be assigned easier patrolling tasks or a training task, and once they reach a threshold of initial patrolling activities, they can be promoted to higher levels. To achieve this, we plan to start with a proof of concept for general patrolling of edits based on users with rollback rights.

We understand that patrolling recent edits is a priority and more suitable for mobile than tasks like NPP (New Page Patrol), AFD (Articles for Deletion), and AFC (Articles for Creation). Additionally, we expect that there will be community consensus on implementing a training task across different languages.

検証

  • Key Indicator 1: 65% of target mature audiences using the tool find it helpful for maintaining the quality of wikis and would recommend it to other patrollers
  • Key Indicator 2: Increase in edits made by mature audiences by 5%
  • Key Indicator 3: 10% of target mature audiences engage with filter preferences
  • Key Indicator 4: 65% of target mature audiences engage with the tool at least three times in a thirty-day window
  • Guardrail: Experienced users without rollback rights, both those who have and have not used alternative patrolling tools, should equally understand the workflow
  • Guardrail: There should be no reports of the tool being used to negatively target underrepresented content or contributors based on in-app reporting mechanisms

Things that would be helpful to know

  •   私たちのこのツールの仕様と他の巡回ツールを比べた場合、MediaWiki のタグ類はどのように対照するか（SWViewer、Huggle、Twinkle）
  •   導入後、取り消し／巻き戻し／感謝の発生は増えたか
  •   その他のおすすめ編集タスクと比べ、この対象の監修の間でこのタスクはどのくらい知名度があるか
  •   この機能で最も多用されるのはどの捜査か

Risk management

Risk Cause Level Response Response Action Trigger Contingency Plan
Tool causes a decrease in new editor retention on wikis where it's available Editors use the tool to increase the volume of edits they're able to revert, but do not leave talk page messages to explain the revert, leaving new editors confused and unmotivated. Medium Monitor Measure the retention rate of editors who had a revert from the patroller task. If they got a talk message, did that increase their retention? Qualitative complaints If a user has not left a talk page message after using undo or rollback and attempts to move to next edit, add a prompt asking/requiring them to leave a Talk Page message
Policy disagreements between existing patrollers and newer patrollers using Edit Patrol / New patrollers use the tool a way that confuses other patrollers. Behavior and/or communication from App patrollers is seen as insufficient or lacking by other patrollers. A new group of editors are able to start patrolling at scale using Edit Patrol, but they are not aware of existing policies or norms. Medium Mitigate Post updates, and invite to office hours patrolling groups (CVU, recent changes patrol, small wiki monitoring team, stewards, global sysops, global rollbackers) before full release. Share Example Messages with communities. Qualitative complaints from patrollers Add more policy and template onboarding to tool, and revisit Example Messages after Community Configuration 2.0 is released. Continue work on building training task.
An increase in edit wars Patrollers reviewing the same feed of edits go back and forth reverting each other's changes because revisions or rollbacks made in the tool appear in the feed for the next person (if no filters are applied). Low Accept - Qualitative complaints Add more onboarding about policies surrounding Edit Wars. Add in-app warning for edit-war behavior.
The tool is being used to negatively target underrepresented content or contributors. Patrollers are systematically reverting good faith edits for particular users or languages, harassing, or mistreating contributors through the tool. Low Mitigate Unless otherwise requested per wiki, the tool available to those with rollback rights. Qualitative complaints, in-app reporting mechanisms, guardrail analysis by Product Analyst Add universal code of conduct onboarding to tool.
The tool causes an increase in vandalism, and increase in amount of work for existing patrollers Vandals use the streamlined interface to quickly revert or rollback large quantities of good faith edits. Low (Dormant) Mitigate Unless otherwise requested per wiki, the tool available to those with rollback rights. Qualitative complaints and anomalies surfaced in data reporting Implement a maximum number of undos/rollbacks that can be done in a range of time.
Someone is exposed to vandalism or there is a copyright violation in Example Messages provided in-app A vandal has edited the Example Messages or pasted copyright material in using Community Configuration 2.0 or Translate.wiki, and they automatically appear in everyone's app Low (Dormant) Mitigate Once community configurable, ensure permissions are set for editing with Growth team. Qualitative complaints Inrease necessary permissions to edit Example Messages.

How to Follow Along

We have created T322083 as our Phabricator Epic to track this work. We invite you to collaborate with us there or on our Talk Page.

このページでは、進捗のたび情報更新をしていく予定です。改訂版設計フィードバックのセッション参加に興味がある人から、当チームのトークページにてご連絡を受け次第、こちらから手順などをお返事します。

We have the following roll out strategy:

2023–2024 time period Consultation Released to rollbackers
2023年10月 インドネシア語版ウィキペディア
2023年11月 インドネシア語版ウィキペディア インドネシア語版ウィキペディア
2024年1月 スペイン語版ウィキペディア, フランス語版ウィキペディア
2024年2月 スペイン語版ウィキペディア, フランス語版ウィキペディア
2024年3月 中国語版ウィキペディア, 英語版ウィキペディア, イボ語版ウィキペディア スペイン語版ウィキペディア, フランス語版ウィキペディア
2024年4月 英語版ウィキペディア, イボ語版ウィキペディア, Global Sysops and Rollbackers 英語版ウィキペディア, イボ語版ウィキペディア
2024年5月 Available for all wikis


技術面の解説文書

この機能がどのように展開されるか関心がある皆さんは、ぜひ詳細を巡回者タスク開発の記録で確認してください。

更新情報

2024年9月

  • After conversations with Igbo Wikipedia admins, we changed the availability of Edit Patrol for users on Igbo Wikipedia, and it’s now available to users with 500 edits and 30 days of tenure. T371442

2024年8月

  • Inspired by this feature, we also added an entry point to the talk page from the Watchlist Diff page. T370697

2024年7月 - Results after 30 days

We have results to share 30 days after the release of Edit Patrol to all Wikipedias within the Android App. Overall, we met or almost met 3/5 of our Key Indicators, and 2/2 of our guardrails.

Validation

  • KR 1.1: 65% of Target mature audiences that use the tool say they find it helpful for maintaining the quality of wikis and would recommend it to other patrollers
    • Almost met: In an in-app feedback about satisfaction with the feature, 63.6% of users who gave feedback selected Satisfied, 22.7% selected Neutral, 13.6% selected Unsatisfied
    • Qualitatively: an Indonesian Wikipedia editor mentioned they found accessing the talk page templates through the feature helpful, and that it would make them more efficient. They mentioned this feature is the only option for them to check Recent Changes on Indonesian Wikipedia. Another Indonesian Wikipedia editor found the tool helpful for patrolling without needing to carry a laptop. 2 other editors mentioned they are excited to download the app to try the Edit Patrol feature, one mentioned it might make them more active in patrolling.
  • KR 1.2: 10% of target mature audiences interacted with another user by sending thanks or a talk page message.
    • Not met: 3.2% of all users sent Thanks, no users sent talk page messages through the feature. We heard from patrollers about the importance of communication along with undo and rollback actions especially for newcomers. However we did not see users sending Talk Page messages along with edits, even though ‘Talk’ was an option in the toolbar. Iterative improvements to motivate users to send talk page messages should be considered for future work.
  • KR 1.3: 10% of target mature audiences engage with filter for preferences
    • Met: 23.8% of users engaged with Filters
  • KR 1.4.1: 30% of users engage with the tool more than once in a 30 day window
    • Met: 38.1% of users returned on more than 1 day
  • KR 1.4.2: 65% of Target mature audiences (potential sysop/rollback Android editors) engage with the tool once in a thirty day window
    • Not met: 30.6% of all potential users engaged with the tool once. This could be due to a few reasons: many users with extended rights already have established workflows on Desktop or Mobile Web, or users with extended rights who have the the Android App are not aware of the feature.

Guardrails

  • Guardrail 1: Experienced users without rollback rights, users that have and have not used alternative patrolling tools equally understand the workflow
    • Met: We conducted interviews with 6 Wikimedians who had a variety of experience patrolling with V1 of the feature. Some were admins, and others were experienced editors. 2 interviewees had not used alternative patrolling tools.
      • 6/6 of participants thought that the location of the tool is intuitive.
      • 6/6 could easily understand the 'List of edits' page displayed
      • 6/6 of the participants thought that the information provided through the feature looked fine.
      • 2/6 of the participants didn't understand the title 'ORES score' (they knew good vs bad faith edits but didn't relate that to the ORES score), and we updated the way the scores were presented in response.
  • Guardrail 2: We do not receive reports of tool being used to negatively target underrepresented content or contributors based on in-app reporting mechanisms
    • Met: We have not received reports through in app reporting.

Curiosities

  • Curiosity 1: How does use of our tool compare to other patrolling tools when looking at MediaWiki Tags (SWViewer, Huggle, and Twinkle)
    • One of our goals in building this tool was that it could work across language wikipedias.
    • In terms of Edits: We saw that 93.1% of Android Patroller edits were made on wikis other than English, comparable to SWViewer where 88.4% of edits were made on language wikipedias other than English. This is markedly different from Huggle and Twinkle, where the majority of edits are made on English Wikipedia
Rollback/Undo Edits by Tool on English vs Other Wikis
Tool Total Rollback/Undo Edits % English Wikipedia Edits % Other Wiki Edits
Huggle 9249 88.6% 11.4%
Twinkle 17133 89.9% 10.1%
SWViewer 3758 11.6% 88.4%
Android App Patrollers 331 6.9% 93.1%
    • In terms of unique Editors, 60% of unique editors on the Android Patrolling tool were editing on a language other than English, comparable to SWViewer where 71% of editors are editing on a language other than English. Huggle and Twinkle’s unique editors/bots were primarily editing English Wikipedia, with only 10% of Huggle’s unique editors editing a language other than English, and 14% of Huggle unique editors.
Unique Rollback/undo Editors by Tool on English vs Other Wikis
Tool Total Uniques % English Wikipedia Editors/Bots % Other Wikipedia Editors/Bots
Huggle 75 89.33% 10.67%
Twinkle 829 85.77% 14.23%
SWViewer 64 28.13% 71.88%
Android App Patrollers (All non bot editors) 23 39.13% 60.87%
  • Curiosity 2: Do we see an increase in Undo/Rollback/Thank events
    • We saw an average 7.9% increase in rollback/undo counts compared to previous monthly events. We did not see an increase or change in Thank events.
  • Curiosity 3: How popular is this task with our target audience relative to other Suggested Edits task?
    • Edit patrol had 331 edits in a 30 day time period, making up 2% of all suggested edits. Adding image captions has a similar usage rate, at 3.1% of all suggested edits, and 517 edits in a 30 day time period. Edit patrol has a key difference in availability: it is only available to a much smaller audience: editors with rollback rights (other suggested edits are available to users with 50+ edits).
  • Curiosity 4: What actions are most popular in the feature?
    • All editors who entered the feature opened an edit from the list of edits, and swiped through edits. 25.4% of all editors completed undo or rollback actions.
Action % of All Users Completing Action
Swiping through edits 100.0%
Clicking on an Edit 100.0%
Completing an Undo 23.8%
Completing a Rollback 12.7%
  • Curiosity 5: How are users interacting with Saved Messages and Example Messages?
    • We have not seen engagement with Saved Messages so far.
  • Curiosity 6: How are users interacting with template search and filing out template data?
    • We have not seen engagement with Templates so far.

Lessons Learned

  • 30% of our eligible audience engaged with the tool in a 30-day window. When deciding how to release the feature, we listened to community requests to only make it available to users with rollback rights. This resulted in a smaller eligible audience, and we did not have a baseline for frequency of use in a given period. In the future, we should continue to explore how the feature could be made available to more users, especially on small to medium-sized Wikipedias. If community configuration capabilities are added to this tool in the future, it could allow for expanded access: communities could choose to allow editors with a certain edit threshold to have access to this tool.
  • We heard from patrollers about the importance of communication along with undo and rollback actions, and we heard requests to add support for Templates. However, we did not see users sending talk page messages along with edits, or using templates within the feature. The feature could have benefitted from an in-app feedback mechanism triggered after a user completed a rollback or undo function, but did not send a message.

2024年6月

  • We connected with Moderator Tools on the release of the RevertRisk model to RecentChanges, and agreed that we'll wait to confirm model stability and effectiveness with Automoderator on production wikis before assessing the work needed to make the RevertRisk model available via RecentChanges. In addition, we are still monitoring the timing of needing to switch to RevertRisk model as a result of anticipated degradation of the GoodFaith and Damage scores from ORES legacy.

2024年5月

  • Edit Patrol is now available on production for all Wikis! T350513
  • Community ambassador, Bona, shared the feature at the ESEAP 2024 conference, and we heard the following feedback:
    • One Indonesian Wikipedia editor mentioned they found the talk page templates helpful, and it makes them more efficient. They mentioned this feature is the only option for them to check Recent Changes on Indonesian Wikipedia.
    • Another Indonesian editor mentioned it was confusing at first to change their primary language to view the tool (indicating that we could improve the entry point). They found the tool helpful for patrolling without needing to carry a laptop.
    • 2 other editors mentioned they are excited to download the app to try the Edit Patrol feature, one mentioned it might make them more active in patrolling.
  • We have preliminary analysis to share after Edit Patrol’s release (acknowledging this was a phased release, we will share numbers 30-days after its release to all wikipedias) T363353
    • Key Indicator 1: 65% of Target mature audiences that use the tool say they find it helpful for maintaining the quality of wikis and would recommend it to other patrollers
      • Actual: 57.1% of users who gave feedback selected Satisfied, 42.9% selected Neutral
    • Key Indicator 2: 10% of target mature audiences interacted with another user by sending thanks or a talk page message.
      • 0% of users sent thanks or talk page messages. We had a discrepancy in our data instrumentation that caused us to over-report the number of messages that were sent through the tool. This was updated to the corrected percent on 10 July 2024.
    • Key Indicator 3: 10% of target mature audiences engage with filter for preferences
      • We saw 1.6% of users engage with Filters
    • Key Indicator 4: 65% of Target mature audiences engage with the tool at least once in a thirty day window
      • We saw 87.5% of those who opened the feature completed successful engagements with the tool.
  • We learned we may be able to offer blocking as an action, as requested by some users
  • We’ve documented multiple ways that community configuration could benefit this tool in the future.
  • Fixed some small issues in Edit Patrol:
    • Make “Next” button active for Saved / Example Messages in Talk flow, even if user has not made change to saved message T362994
    • Navigating back from editing a saved message now returns you to the tab you started on T362996

2024年4月

The team released “V2” of Edit Patrol to target wikis, which includes Saved messages functionality and support for Templates to production, T356774 & T355141.

  • Edit patrol is now available on production for Igbo and Chinese Wikipedia, it will be available in production for all Wikis by mid-May.T350512
  • We received feedback that we should change “warn” to “talk” to encourage more positive behavior, and we finished updating this throughout the feature: T360449 & T361648
  • Outreach:
    • We met with admins from the Igbo community to share the feature and receive feedback on how it lands with a smaller wiki.
    • We shared release information and consultation hours information with English Wikipedia, and on discussion pages of patrolling audiences: Stewards, Global Rollbackers, Global Sysops, Recent Changes Patrol, Small wiki monitoring team, and Coutnervandalism unit. Our consultation hour was held on Friday, April 26.
    • We coordinated with WMFR to share a short async presentation and demo in the French Admins meeting on April 13.
  • Learnings:
    • We learned from one English Wikipedia admin that they would value quick access to Blocking users through this tool, and would expect to find it under the user overflow menu. They would also like to see a prompt to quickly add the appropriate talk page message to accompany the block. T363422
    • A global rollbacker shared that they would like to have access to Advanced Reporting and Vetting (ARV), which is supported through Twinkle.
    • We heard another request for the Example Messages provided to be community configurable, highlighting the importance of this work once Community Config supports the apps T358265
    • We heard feedback that reaffirms our choice to make this tool only available to rollbackers at the start, but suggested a threshold to be considered in the future could be 500+ edits and 30 days of tenure. We see another opportunity for Community Config here, supporting wikis in changing permission levels for this feature.T358265

2024年4月, Special Mid Month Update

  • Edit patrol has been released to all wikis in the Beta version of the app! To view the feature, download or update the Wikipedia Beta app. Open the Edits tab, and look for Edit Patrol under Suggested Edits. The feature is available to users with rollback rights, and will display if you have rollback rights on the language that is set as your Primary in the App.
Edit patrol demonstration on test.wiki

2024年3月

  • We released the Edit Patrol suggested edit to French and Spanish Wikipedias. T350511
  • Development continued for Saved Messages for Edit Patrol (Example messages available in the App) and access to templates.
  • While completing development on Saved Messages, we learned that it wasn’t feasible to allow users to edit, delete, and rearrange the prewritten "Example Messages" from within the app in the same way that users can edit their own custom made messages. We updated the designs for Saved Messages to add visual separation between “Your Messages” and “Example Messages” T359209
 
  • We learned we will be able to use the Growth team’s community config to make Example messages available for modification in the app.
  • We solicited feedback about how to handle the links to policies within the example messages: we heard from one global admin (a staff member), who recommended against linking to English Policies as the fall-back. We heard from a member of Igbo community that they may choose to keep the links to English policies, and eventually update them. We decided to modify the wording of the English example messages, adding "this wikipedia" in a few places so that the sentences make sense without links. Translation instructions on translatewiki will say, “Please include the relevant language version of the link(s). If no link is found, please do not include a link. Editors may add a link(s) to another language wiki's policy if desired.” We will continue to get more input as we scale to more languages and iterate on how we are treating saved messages for small wikis until community config is available. The example message texts are on a Media.wiki page with links to translations.
  • We received feedback that we should change “warn” to “talk” to encourage more positive behavior, and began updating this throughout the feature. T360449
  • In our meeting with Spanish Wikipedians, they reaffirmed the choice of Edit Patrol only being available to rollbackers.
  • We are sharing the feature to Igbo Wikipedia April 3rd and will have office hours with English Wikipedia April 26nd.

2024年2月

  • We hosted two synchronous community consultations & recorded a third where we shared the Edit Patrol workflow, and showed designs for Saved messages and templates:
  • We've started development on Saved Messages T356774 (example messages available in the App) and access to templates T355141, which was well received by French Wikipedia. Draft saved messages are posted here, and we welcome feedback on the discussion page.
  • Another community member raised interest in the example warning messages being configurable via the community config.(T358265#9573393), and we are coordinating with the Growth team on this.
  • French Wikipedians reaffirmed the choice of Edit Patrol only being available to rollbackers.

2024年1月

  • Indonesian Wikipedia requested consideration for using Edit Patrol in the upcoming election to fight disinformation and vandalism. Our team had the feature reviewed by our legal team, and volunteers were cleared to use the tool for patrolling during their upcoming election.
  • This month we learned that some communities appreciate being able to have speedy deletion as an option in patrolling tools, which we will consider for future iterations of this feature.
  • Additionally, our team met with the Growth, Moderator Tools and Machine Learning teams to discuss adding the revert risk model to recent changes filtering. We may be able to switch over from using Good Faith and Damaging ORES scores to the Revert Risk model as we scale to Spanish, French and Chinese Wikipedias in the coming months. This may open the door for an experimental training task for experienced editors without rollback rights.
  • We have established demo sessions for our next set of pilot wikis:
    • French Moderators - scheduled for 21 February 2024 from 16-17 UTC
    • Spanish Moderators - scheduled for 28 February 2024 from 16-17 UTC
    • Chinese Moderators - recorded presentation to be shared asynchronous
  • Our designer worked on the flow and UI for the saved warning messages and templates, which will be released in the coming months.
  •  
    Choose Talk
  •  
    Select message from list of saved messages
  •  
    Select from list of example messages
  •  
    Create new message
  •  
    Review message and make changes
  •  
    Save as new message or update saved message
  •  
    Review preview and publish (Web view)
  •  
    Edit list of messages by swiping left or right
  •  
    Reorder or delete multiple messages
  •  
    Saved messages are always accessible from the overflow menu
  • Template Flow: T355141: The team is working to add support for templates, so patrollers can utilize existing templates in their messages
  •  
    Templates will be accessible from the Wikitext editing toolbar
  •  
    In Saved Messages (Edit Patrol), create a new message
  •  
    Compose message, and open Template icon
  •  
    Search for templates by name, recently used are displayed
  •  
    Fill out template form and insert
  •  
    Message with template inserted. Then follow same steps to, save, preview, publish

2023年12月

Patrolling Pain Points on French Wikipedia

We met with a few language communities to get feedback about Edit patrol. The Indonesian Wikipedia community approved the release of the feature for their community and shared which templates are used for warnings, which we will incorporate in a future release. For the month of December we are enabling users to save warning messages on their device (T343841). In January we will begin outreach to Spanish and Chinese Wikipedia to scale to more communities. We’ve already begun outreach to French Wikipedia via the French Wikimedia ED, he shared some pain points identified by French patrollers as it relates to RC Live and SWViewer and other tools as it relates specifically to the French Wikipedia community that we are taking into consideration for Edit Patrol. We are hoping to establish office hours sometime in February with French Wikipedia to discuss Edit patrol.

2023年11月

Release to Indonesian Wikipedia

Edit Patrol was released to Indonesian and Test Wikipedia. In order to see the feature in Suggested Edits you must have rollback or sysops rights.

Through this first release, rollbackers are able to:

  • Onboarding to the task with an explanation of when to use Thank, Watch, Warn, Undo or Rollback, as well as an explanation of ORES scores. (T343242)
  •  
    Onboarding modal 1
  •  
    Onboarding modal 2
  •  
    Onboarding modal 3
  •  
    patrol edit 1
  •  
    patrol edit 2
  • Compose, save, and publish first warning messages. (T344842)
  •  
    Compose new warn message
  •  
    Edit new message
  •  
    Tapping back
  •  
    Empty subject
  •  
    Empty Message
  •  
    Save changes
  •  
    Message published
  •  
    Try publishing without message title
  •  
    Add message title
  •  
    Message saved and published
  • Select, edit, save and publish existing warning message (T344842)
  •  
    Add message
  •  
    Choose message
  •  
    Message selected
  •  
    Edit message
  •  
    Preview message
  •  
    Publish, Save
  •  
    Save new message Empty Message title
  •  
    Preview message
  •  
    Save changes message
  •  
    Message saved and published
  • View information about other editors including their tenure as an editor and their edit count (T343835)
  •  
    Menu-list of edits
  •  
    User information modal
  • Filters for the recent changes feed include language wiki, registration status, experience level, quality score prediction, user intent prediction, automated contributions and significance of edit. The filter page also allows sorting by latest revision. It is also possible to search through edits.
 
Filter with no results
  • An ability to report issues with the filter (T343834)
 
App bar menu-problem with the feature dialog

We welcome feedback about the feature as we prepare to reach out to additional wikis to scale the feature.

September 2023: Advancing V1 of Patroller Tasks: Unveiling New Features to the Indonesian Wikipedia Community

We continued development on V1 of Patroller tasks and announced the feature to Indonesian Wikipedia. Our engineers demoed the filters for Patroller task V1:

  •  
    No results
  •  
    Reset filters
  •  
    Default filters

2023年8月：プロジェクトの情報更新と技術面の考慮事項

  • We shared updates about this project at Wikimania. The recording can be found on Youtube starting at 1:24:30. We received a question about having a central place for Warning messages that can be used across apps. We are planning to work with the Growth team to explore the possibility of centralized warning messages.
  • We became aware of the deprecation of ORES and migration to Liftwing which will host the revertrisk model. Due to our team using the MediaWiki API to show the recent edits feed, we will continue to show the Goodfaith and Damaging Models at least in V1. Using the Revert Risk model would require two API calls, which could slow down the feature, this also doesn’t address filtering. If you have feedback about this technical decision, feel free to comment on our discussion page.
  • We created designs for Onboarding to the feature T343242.
  •  
    Explaining the purpose of the feature
  •  
    List of the main actions
  •  
    Data collection
  •  
    Explaining the 'Quality' score
  •  
    Explaining the 'Intent' score
  •  
    Indicating where one can submit feedback about the feature

2023年7月：設計の調査と聞き取りの結果

使い心地のテスト

A moderated remote usability test was conducted involving six experienced contributors who use Android devices and volunteer their time for monitoring activities on Spanish, French, English, and Chinese language wikis. We selected participants from these wikis due to their high number of editors with sysop/rollback rights, which allowed us to gather usability feedback effectively.

巡回者から、〈編集の巡回〉の簡略な設計と使い心地について貴重な洞察が寄せられました。

テスト期間に編集者にはツールの限定的な試作版を使ってもらい、短い課題の完成に取り組んでもらいました。あわせてそのタスクの処理中に、体験をリアルタイムに言語化して伝えるよう依頼しました。

  •  
    Find the tool in the app
  •  
    Choose an edit from the list of recent edits
  •  
    Review the diff and additional information on the page
  •  
    Decide whether the edit should be reverted
  •  
    Send a warning message to the editor
  •  
    Add the page to their Watchlist

For additional details about the usability test, please refer to the following link.

成果

テスト参加者からフィードバックと貴重な提案を受け取り、1段階目の設計の実現に活用しました。以下の改善点を盛り込んでいます。

The feedback and valuable suggestions from participants have contributed to enhancing the initial designs. The following improvements were implemented:

  • テスト参加者は「最近の編集」ページに載せた編集単位でタグの表示を希望しており、そうすれば特定の編集者が1ページに複数回の編集を行ったかどうか識別できると良いと指摘しました。
  •  
    <gallery>
  •  
    それまでの編集一覧
  •  
    新規編集の一覧
  •  
    ドロップダウンメニューの項目を追加、特定のページで同一の編集者が実施した編集かどうか判定

File:Edit patrol - list of edits.png|New list of edits File:Edit patrol - list of edits - dropdown.png|Dropdown to see the edits made by the same editor on the same page </gallery>

  • Our goal was to incorporate filters on the page where recent edits are showcased. To determine the filters for the initial version of the 'Edit patrol' tool, we consulted with the patrollers. They suggested including the following filters: languages, user registration and experience, automated contributions, contribution quality predictions, user intent predictions, latest revision, and significance. Subsequently, we designed a page that features these filters, which is presented below.
  •  
    Filters
  • The diff view page was simplified
  •  
    Previous diff view page
  •  
    Previous diff view page with full bottom sheet
  •  
    New diff view page
  • Minor design enhancements have been applied to the process of creating, saving, publishing, and editing warning messages for patrollers. The revised designs are displayed below for your reference.
  •  
    Composing a new warning message
  •  
    Choosing a saved warning message
  • Participants provided further feedback regarding patrolling, which we aim to convey within the tool via onboarding, tooltips, or reminders. The points include:The significance of communicating with empathy to fellow contributors, and the emphasis on prioritising the quality of patrolled edits over their quantity.

The 'Edit patrol' tool is currently in the development phase, and you can track its progress through Phabricator T343224.

February 2023: Initial Design Concepts

The team developed initial design concepts by incorporating insights from existing tools and feedback we gathered on their limitations. We shared these concepts with the Moderator Tools Team and Research Team for their input and feedback.

  •  
    Concept for filtering contribution date, ORES Scores or Language
  •  
    Design concept for expanding the bottom sheet
  •  
    Design concepts for the overflow menu
  •  
    Design concept for using templates when leaving a warning

We will use the feedback received to improve the design concepts and establish a protocol for both moderated and asynchronous feedback from app users. Currently, we are initiating direct outreach to schedule moderated feedback sessions and ensure translation mechanisms are in place for those sessions. Once the protocol is available on MediaWiki, we will send out a broader communication to start gathering asynchronous feedback on our talk page.

The design concepts provided are meant to offer examples of how we aim to address the issues faced by other tools. However, please note that we are in the early stages of this process, and the designs will evolve and expand based on the feedback we receive in the coming weeks.

January 2023: Comparative Review and Existing User Research

The team conducted a comparative review to analyze the existing patrolling tools. During the review, we assessed the similarities and differences between the tools and identified which ones were effective on mobile devices.

The complete list of tools we reviewed include:

  • Huggle
  • Twinkle
  • TwinkleMobile
  • STiki
  • SWViewer
  • WikiLoop Double Check
  • CheckWiki
  • ReWarn
  • CheckWiki

Through our evaluation, we found that the majority of tools were optimised for Desktop use, with only three exceptions. Additionally, only two of the above solutions performed well across various language wikis.

Common elements found in these tools were:

  • Icons to represent the main action (e.g. revert, speedy deletion, welcome user, etc.)
  • Access to templates
  • Filters
  • List of contributions
  • Diffs
  • Information about the editor

In addition to the comparative review of existing tools, we also collected recurring requests from patrollers regarding those tools. The common requests include:

  • Filter contributions by topics of interest and specific time period
  • An easy way to exclude contributions that have already been patrolled
  • No requirement for special privileges like rollback rights to access the tools
  • An option between "skip" and "revert" for more nuanced actions
  • Tools that promote collaboration among patrollers

By taking these requests into account, we aim to develop a comprehensive patrolling solution that addresses the needs and preferences of patrollers across various languages and user experience levels.

December 2022: Proof of Concept Prototype

The team developed a proof of concept prototype to explore the capabilities and possibilities using existing APIs. The prototype allowed us to test and experiment with different functionalities and assess the feasibility of our ideas. This early exploration helps us understand the potential of the tools and refine our approach before moving forward with further development.

  •  
    Anti-Vandalism Patrolling task proof of concept
  •  
    Anti-vandalism patrolling proof of concept
  • Screen recording of prototype

During our development process, we successfully exposed the ORES score, which provides users with an indication of the likelihood that something may be vandalism. This feature allows users to get insights into the quality of edits and helps them make informed decisions during the patrolling process. We would ideally send feedback to the research team's API for improvements, which will be valuable in improving the accuracy and effectiveness of the ORES score.

2022年11月：当初の慣習を定義する

当チームは調査を実施して管理者や巻き戻し権限を預かる人のウィキペディア・アプリ利用者数の概数を調べました（T322065）。分析の結果、Android アプリ利用者372名、iOS 利用者176名が権限を預かっていて、このデータに基づき、ロールアウトの優先順位は Android アプリを先にして、iOS 版の開発チームにはウォッチリスト機能の構築に回ってもらいました。

The team conducted research (T322065) to assess the number of Wikipedia app users with sysops and rollback rights. The findings revealed that 372 Android app users and 176 iOS users had these rights. Based on this data, we decided to prioritise the rollout of the feature in the Android app, allowing the iOS development team to focus on building the Watchlist feature.

さらに、さまざまな言語版にわたるシスオペおよび巻き戻し権限を預かる利用者の分布を理解しようとも努めました。この情報は初期調査段階で直接の支援を行うにあたり、対象第一弾の言語コミュニティを決める際に重要な役割を果たしました。こうして仲裁と巡回の経験者から貴重な洞察とフィードバックを得たいと考え、以下に示す特定の言語コミュニティに焦点を当て、より広く使える機能開発へ踏み出すのに役立ちます。

  • 英語版
  • フランス語版
  • 中国語版
  • スペイン語版
  • 英語版ウィキペディア
  • フランス語版ウィキペディア
  • 中国語版ウィキペディア
  • スペイン語版ウィキペディア

当チームでは特定の言語版ウィキならびに OS 利用者に注目して直接、働きかけていますが、 フィードバックとアイデアの投稿はアプリの種別にとらわれず、巡回の経験者ならどなたからも歓迎している点を強調させてください。ウェブ利用者の皆さんの投稿も大切であり、フィードバック募集の優先順位は、やはりアプリ版利用者とさせていただきます。

プラットフォームが変わっても統一した経験を作成することが不可欠であり、その達成に向け、私たちはウェブ利用者向けの体験を構築する仲裁用のモデレータ・ ツール・チームと緊密に意思疎通しています。先方の調査結果から学んで連携を緊密にすることで、アプリとウェブ・プラットフォームの双方で継続性と一貫性を確保しようと目指しています。すべてのユーザ向けの巡回機能の開発と改善を形作る上で、皆さんからのフィードバックとアイデアは重要な役割を果たします。

Retrieved from "https://www.mediawiki.org/w/index.php?title=Wikimedia_Apps/Team/Android/Anti_Vandalism/ja&oldid=6797503"