ウィキ・ハイライト
ウィキ・ハイライトという試験用のプロジェクトでは、ビジュアルな記事を自動生成して人間が評価し、サードパーティにいる世界の若者に提供した場合、その人たちは常駐するプラットフォームで得られる読書体験をこちらに切り替えるだろう、そしてウィキメディア・プロジェクトに関する認識が高まり読者および寄稿者として関与を増やすだろうという仮説を立て、これが妥当か妥当ではないか検証します。
ウィキ・ハイライト
当プロジェクトではウィキペディアの概要版を設けて、そのコンテンツを読みに来るグローバルな若者層観衆との相互作業の評価に取り組みます。その人たちが常駐するサードパーティのプラットフォームで、当該ウィキペディアのコンテンツとどのように関与するでしょうか。
|
背景
ウィキメディア財団の製品技術部門は2023-24年次の作業領域として、「将来の観衆」※を識別し主な成果目標を立てました("※"=Future Audiences)。 将来の視聴者バケットでは、インターネット上のどこにいても誰もが知識を利用できる方策を探し、ウィキメディア運動が無償の知識のエコシステムに不可欠なインフラとなるよう模索します。
上に述べた取り組みは、情報消費を他のプラットフォームで行うグローバルな若者層に当財団のコンテンツを届けて、こちらのプロジェクトへの意識を高めて関与を増やそうと目指しています(将来の観衆KR 2.1)。
この将来の観衆 KR 2.1 とウィキ・ハイライト・プロジェクトは整合性があり、Inuka チームの2023-24年次計画で立てた仮説の一つが妥当かそうではないか検証します。
- If we offered automated, human-reviewed, visual article summaries generated from Wikipedia articles as an alternative reading experience for younger readers and have them indexed by viable search platforms (Google, Tiktok), we would be able to test whether their interactions with our content & projects would increase engagement. Human-reviewed summaries would be generated for multiple articles and surfaced to a group of readers on mobile, to evaluate their preference for different content formats and topics. We would evaluate if global youth readers show significant interest towards Wikipedia article summaries and measure engagement based on time spent, number of summaries consumed, summary completion rate and topics with the highest readership.
ウィキハイライトが有効な理由
若手世代の観衆には、情報消費のパターンがグローバルに目まぐるしく変化の傾向が見られ、私たちのプロジェクト群に集まるトラフィック全体に波及しています。この傾向を3種類に分類しました。
- コンテンツ形式:文字の文章、画像、動画、音声(静的・双方向含む)
- コンテンツの長さ:短い
- コンテンツの展開先:検索、ソーシャル・メディアおよびその他の第三者のプラットフォーム。
若手観衆の洞察による指摘 (年齢層は18-24歳):
- ウィキペディアは学校の宿題や事実調べ、新しい情報探しによく使われている。
- ウィキペディアは友だち関連や他人と出会うためにはほとんど使われない。
- ウィキペディアを使っているかどうか尋ねると、他のブランドの平均50%以下。
- 記事が長すぎるから、というのがウィキペディアを使わない理由の第3位。
- 画像を増やしたり記事を短縮すると、利用率を改善できると思われる。
ウィキペディアには長大で文字だらけの記事がたくさんあります。こういう種類のコンテンツは若手の観衆の間でどんどん人気がなくなっていて、需要の面で一方ではコンテンツに関するこの面の改革が求められる点と、他方で重要な情報を保ち続けることを両立させて、他のプラットフォームに滞留する観衆に訴求することが必要です。
目標
当プロジェクトの主な趣旨は、ウィキペディアの要約を自動生成し人間の目で校正してビジュアル要素を加えたなら(これをウィキハイライトと呼びます※)、サードパーティのプラットフォームに滞留する世界の若手層にとって、そういうものも読書体験となりうるのではないかと考えます(※=Wiki Highlights)。
実験の取り組み方
The team will use microsites to conduct A/B testing between different content formats (standard Wikipedia articles and Wiki-Highlights format) to evaluate the engagement and the experience of the targeted audience on mobile devices:
- For this 1st iteration of this experiment, we will not attempt to test this portion of the hypothesis "....have them indexed by viable search platforms (Google, Tiktok)..." as we'd like to validate the viability of this approach before engaging with a partner.
- The Wiki-Highlights content will be generated by extracting a concise overview of facts (2-4 sentences, or 300 characters or less) from the sections of lengthy Wikipedia articles, combined with relevant images sourced from Commons. This first experiment will focus on using English articles selected from the following categories and topics.
Categories | Topics | Quality of articles |
---|---|---|
History | Art, monuments, sites and artifacts | Articles must be rated "Featured" or "Good" |
Life style | Food, fashion, language, travel, media | |
Places | Countries, cities, islands | |
Personalities | Biographies, personalities | |
Sports | Sport, games and recreation | |
Topical | Climate, sustainability, equity, health, social justice | |
Nature | Plants, animal, water and land bodies |
計測の指標
関与の指標計画 | |||
---|---|---|---|
評価指標 | 追跡の対象 | ||
一次指標: | |||
利用者は記事要約欄を使いたいと思うか(Wikihighlights) | Total time spent per session | ||
Secondary metrics: | |||
Users' willingness to complete a summary | Number of summaries consumed per session | ||
Users' willingness to view subsequent summaries. | Number of summaries per articles viewed and number of sessions | ||
Topics with majority of reads. | Number of read summaries per article based on topics. |
製品設計
The team considered different layouts, interaction styles and experience design prototypes and subjected them to rounds of user testing exercises on Userlytics, Instagram, and Tiktok to come up with the strongest preference of design for the microsites as shown in this link.
-
This is a homepage for Wiki-Highlights microsite.
-
This is a homepage for Wiki-Highlights microsite.
-
This is how readers can consume different highlights related to an article.
-
Discover page gives option to readers to select the next Wiki-Highlight or go back to main page
-
Each Wiki-Highlights start page has “swipe up for more” to guide users that they can start scrolling to see more content
日程
2022/23年度 | 2023年 | 2024 | |||||||
---|---|---|---|---|---|---|---|---|---|
第4四半期 | 第1四半期 | 第2四半期 | Q3 | ||||||
6月 | 7月 | 8月 | 9月 | 10月 | 11月 | 12月 | JAN | FEB | MAR |
Ideating and scoping | |||||||||
設計:試作版と使い心地テストの提案を繰り返して出す | |||||||||
小規模サイトの開発、試験、実装 | |||||||||
焦点グループを対象に実験 | |||||||||
評価と報告 |
Experiment results
The WMF developed 2 prototype test sites that were shown to 2400 participants aged between 18-24 year olds across 6 countries (US, Brazil, Germany, India, Indonesia, Nigeria). Half the participants were shown the article microsite, the other half shown the highlights microsite.
The testing happened in 2 ways;
1. Survey testing:
We ran a series of survey questions before and after participants interacted with the 2 sites. Half the participants were shown the article view, and the other half shown the wiki-highlights view. The survey result document has the outcome in detail, and below are summarized insights from the survey:
Learnings | Suggestions/ Recommendations | |
Overall | Wiki-Highlights had more appeal, and is seen as more unique, though only marginally. | Although differences in appeal & uniqueness are statistically significant, they are slight differences.
|
Audience related feedback | Wiki-Highlights appeals more to 23-24 year-olds & among more frequent Wikipedia users. | With technology changing so fast, 23-24 year-olds may have a different relationship with it versus 18-19 year-olds.
|
The Wiki-Highlights appealed more to users in Nigeria and Indonesia and much less for users in Germany & US. | We see Wikipedia overall less appealing in more developed markets - perhaps due to more established digital environments & brands.
| |
Feature related feedback | Wiki-Highlights outperformed controlled site on perceptions of 'fun' & 'easy' | 23-24 yr olds find Wiki- Highlights more fun, perhaps due to less use of apps like TikTok.
Also, heavier Wikipedia users are more likely to find Wiki- Highlights easy.
|
Users liked the topics & imagery of both sites; Wiki- Highlights stood out on simplicity of language & content length. | The current lack of perceived difference of imagery on Wiki- Highlights versus Control suggests this could be an area of further development.
| |
Users would improve the navigation and they desired more topics. | Review the back button (this may have been impacted by the survey buttons, but we don't believe so given the verbatim).
|
2. A/B testing:
During the survey testing, we instrumented the microsites to capture specific metrics for the 2 groups to gauge the depth of engagement further.
- The Experiment Group participants were shown the wiki-highlights/ summarised content as seen here.
- The Control Group participants were shown long form Wikipedia Article-type of content as seen here.
Overall metrics and observations are documented in the table below; available with a full report .
Overall Metrics | Observations |
Time on site (session length) | Time on Homepage + Content page: Overall, the experiment group seemed to stay longer than the control group. |
Time on Homepage: Experiment group spent the same amount of time as the control group on the homepages. | |
Time on Content Page: Experiment group stayed longer than the control group on the content pages. | |
Completion rate (willingness to complete the content) | Experiment group: 1,658 highlights opened with a 72.2% completion rate.
Control group: had 1,112 articles opened, with a 78.1% completion rate. |
Number of content consumed per session (willingness to view subsequent content) | Experiment group: 95% of sessions consumed 0 to 4 highlights per session.
Control group: 95% of sessions consumed 0 to 3 articles per session. *** 0 means users only viewed the homepage in certain sessions. |
Majority reads by content type | Experiment group top 3 visited pages:
Control group top 3 visited pages:
|
Majority reads by category type | Experiment group top 3 visited categories:
Control group top 3 visited categories:
|
We also evaluated the metrics across the 6 countries we ran the experiment;
Country Metrics | Observations |
Time on site (session length) | Experiment Group: Brazil, India, United States and Nigeria, users spent more time on homepages and content pages.
Control group: Indonesia and Germany, users spent more time on homepages and content pages. |
Experiment Group: Brazil, Nigeria, and India, users spent more time on homepages.
Control group: Indonesia and the United States, users spent more time on homepages. ***In Germany, users spent similar time on homepages for both the experiment and control groups. | |
Experiment Group: Brazil, India, Indonesia, Nigeria, and US had more users spending more time on content pages.
Control group: Germany, in 90% of the sessions, the control group had more users spend more time on content pages. ***Nigeria had significantly much longer time spent on content pages compared to other countries. | |
Completion rate (willingness to complete the content) | Experiment Group:Content completion rate in experiment groups is lower in every country except India and Indonesia. |
Number of content consumed per session (willingness to view subsequent content) | Experiment Group: Brazil, India, Nigeria, and the United States, users viewed more content in the experiment group.
Control group: Germany, users viewed fewer content in the experiment group than the control group per session. ***In Indonesia, users viewed a similar amount of content per session in both groups. |
Majority reads by content type |
|
Majority reads by category type |
Experiment Learnings:
- Learnings from the Survey testing:
- Wiki-Highlights had more appeal, and is seen as more unique, though only marginally. Although differences in appeal & uniqueness are statistically significant, they are slight differences.
- This suggests that more could be done to differentiate Wiki-Highlights from the current Wikipedia reading experience.
- Wiki-Highlights had more appeal, and is seen as more unique, though only marginally. Although differences in appeal & uniqueness are statistically significant, they are slight differences.
- Learnings from the A/B Testing:
- Users spent more time and consumed more Wiki-Highlights content than articles. However, more articles were completed over Wiki-Highlights which could be attributed to the fact that users did not expand each article section to and this may have affected the reading time of the control group.
- Brazil, India, and Nigeria users favored Wiki-Highlights;
- Germany users favored articles where they spent more time, consumed more and completed more content.
- Users spent more time and consumed more Wiki-Highlights content than articles. However, more articles were completed over Wiki-Highlights which could be attributed to the fact that users did not expand each article section to and this may have affected the reading time of the control group.
更新情報
Q3 January to March 2024
2024年3月
- Published experiment results on MediaWiki. Experiment concluded.
- T355224: Analysis for Wiki Highlights experiment
2024年2月
- Released Wiki-Highlights Survey Results.
2024年1月
- Microsite development tasks
Q2 October to December 2023
2023年12月
Microsite development tasks
2023年11月
Microsite development tasks
- T349863: Misalignment in "Discover more" view
- T342542 Microsite Design explorations
- T349134: Test and fix browser support
- T345013: Implement site chrome, navigation elements, and footer for WikiHighlights and Long-form articles microsites
- T346200: Create instrumentation spec for Wiki Highlights experiment
2023年10月
Microsite development tasks:
- T345014: Implement 'End of Highlight' screen for WikiHighlights
- T345546: Highlights interaction & flow
- T345553: Long-text Article view
- T349243: Blank page for long text view
- T345554: Homepage Design for Longtext Microsite
- T345551: Homepage Design for Highlights Microsite
- T342543 Wiki-Highlights article shortlisting & summaries
- T346239: Setup CI for Wiki-highlights
- T344256: [SPIKE] Investigate microsite set up