ウィキ・ハイライト

グループ:	Inuka team
開始:	2023-07
チームのメンバー:	Runa Bhattacharjee; Stephane Bisson; Mary Munyoki; Eduardo Medina; Huei Tan; Sudhanshu Gautam; Uzoma Ozurumba; Eli Asikin-Garmager; Connie Chen;
バックログ:	#Wiki-Highlights
リーダー:	Purity Waigi （製品オーナー）

This page is a translated version of the page Wiki-Highlights and the translation is 55% complete.

ウィキ・ハイライトという試験用のプロジェクトでは、ビジュアルな記事を自動生成して人間が評価し、サードパーティにいる世界の若者に提供した場合、その人たちは常駐するプラットフォームで得られる読書体験をこちらに切り替えるだろう、そしてウィキメディア・プロジェクトに関する認識が高まり読者および寄稿者として関与を増やすだろうという仮説を立て、これが妥当か妥当ではないか検証します。

背景

ウィキメディア財団の製品技術部門は2023-24年次の作業領域として、「将来の観衆」^※を識別し主な成果目標を立てました（"※"＝Future Audiences）。将来の視聴者バケットでは、インターネット上のどこにいても誰もが知識を利用できる方策を探し、ウィキメディア運動が無償の知識のエコシステムに不可欠なインフラとなるよう模索します。

上に述べた取り組みは、情報消費を他のプラットフォームで行うグローバルな若者層に当財団のコンテンツを届けて、こちらのプロジェクトへの意識を高めて関与を増やそうと目指しています（将来の観衆KR 2.1）。

この将来の観衆 KR 2.1 とウィキ・ハイライト・プロジェクトは整合性があり、Inuka チームの2023-24年次計画で立てた仮説の一つが妥当かそうではないか検証します。

If we offered automated, human-reviewed, visual article summaries generated from Wikipedia articles as an alternative reading experience for younger readers and have them indexed by viable search platforms (Google, Tiktok), we would be able to test whether their interactions with our content & projects would increase engagement. Human-reviewed summaries would be generated for multiple articles and surfaced to a group of readers on mobile, to evaluate their preference for different content formats and topics. We would evaluate if global youth readers show significant interest towards Wikipedia article summaries and measure engagement based on time spent, number of summaries consumed, summary completion rate and topics with the highest readership.

ウィキハイライトが有効な理由

若手世代の観衆には、情報消費のパターンがグローバルに目まぐるしく変化の傾向が見られ、私たちのプロジェクト群に集まるトラフィック全体に波及しています。この傾向を3種類に分類しました。

コンテンツ形式：文字の文章、画像、動画、音声（静的・双方向含む）
コンテンツの長さ：短い
コンテンツの展開先：検索、ソーシャル・メディアおよびその他の第三者のプラットフォーム。

若手観衆の洞察による指摘 (年齢層は18-24歳):

ウィキペディアは学校の宿題や事実調べ、新しい情報探しによく使われている。
ウィキペディアは友だち関連や他人と出会うためにはほとんど使われない。
ウィキペディアを使っているかどうか尋ねると、他のブランドの平均50%以下。
記事が長すぎるから、というのがウィキペディアを使わない理由の第3位。
画像を増やしたり記事を短縮すると、利用率を改善できると思われる。

ウィキペディアには長大で文字だらけの記事がたくさんあります。こういう種類のコンテンツは若手の観衆の間でどんどん人気がなくなっていて、需要の面で一方ではコンテンツに関するこの面の改革が求められる点と、他方で重要な情報を保ち続けることを両立させて、他のプラットフォームに滞留する観衆に訴求することが必要です。

目標

当プロジェクトの主な趣旨は、ウィキペディアの要約を自動生成し人間の目で校正してビジュアル要素を加えたなら（これをウィキハイライトと呼びます^※）、サードパーティのプラットフォームに滞留する世界の若手層にとって、そういうものも読書体験となりうるのではないかと考えます（※＝Wiki Highlights）。

実験の取り組み方

The team will use microsites to conduct A/B testing between different content formats (standard Wikipedia articles and Wiki-Highlights format) to evaluate the engagement and the experience of the targeted audience on mobile devices:

For this 1st iteration of this experiment, we will not attempt to test this portion of the hypothesis "....have them indexed by viable search platforms (Google, Tiktok)..." as we'd like to validate the viability of this approach before engaging with a partner.
The Wiki-Highlights content will be generated by extracting a concise overview of facts (2-4 sentences, or 300 characters or less) from the sections of lengthy Wikipedia articles, combined with relevant images sourced from Commons. This first experiment will focus on using English articles selected from the following categories and topics.

Categories	Topics	Quality of articles
History	Art, monuments, sites and artifacts	Articles must be rated "Featured" or "Good"
Life style	Food, fashion, language, travel, media
Places	Countries, cities, islands
Personalities	Biographies, personalities
Sports	Sport, games and recreation
Topical	Climate, sustainability, equity, health, social justice
Nature	Plants, animal, water and land bodies

計測の指標


関与の指標計画
評価指標	追跡の対象
一次指標：
利用者は記事要約欄を使いたいと思うか（Wikihighlights）	Total time spent per session
Secondary metrics:
Users' willingness to complete a summary	Number of summaries consumed per session
Users' willingness to view subsequent summaries.	Number of summaries per articles viewed and number of sessions
Topics with majority of reads.	Number of read summaries per article based on topics.

製品設計

The team considered different layouts, interaction styles and experience design prototypes and subjected them to rounds of user testing exercises on Userlytics, Instagram, and Tiktok to come up with the strongest preference of design for the microsites as shown in this link.

<span lang=
This is a homepage for Wiki-Highlights microsite.
This is a homepage for Wiki-Highlights microsite.
This is how readers can consume different highlights related to an article.
Discover page gives option to readers to select the next Wiki-Highlight or go back to main page
Each Wiki-Highlights start page has “swipe up for more” to guide users that they can start scrolling to see more content

日程

2022/23年度	2023年						2024
第4四半期	第1四半期			第2四半期			Q3
6月	7月	8月	9月	10月	11月	12月	JAN	FEB	MAR
Ideating and scoping
設計：試作版と使い心地テストの提案を繰り返して出す
		小規模サイトの開発、試験、実装
		焦点グループを対象に実験
	評価と報告

Experiment results

The WMF developed 2 prototype test sites that were shown to 2400 participants aged between 18-24 year olds across 6 countries (US, Brazil, Germany, India, Indonesia, Nigeria). Half the participants were shown the article microsite, the other half shown the highlights microsite.

The testing happened in 2 ways;

1. Survey testing:

We ran a series of survey questions before and after participants interacted with the 2 sites. Half the participants were shown the article view, and the other half shown the wiki-highlights view. The survey result document has the outcome in detail, and below are summarized insights from the survey:

	Learnings	Suggestions/ Recommendations
Overall	Wiki-Highlights had more appeal, and is seen as more unique, though only marginally.	Although differences in appeal & uniqueness are statistically significant, they are slight differences. This suggests that more could be done to differentiate Wiki-Highlights from the current Wikipedia reading experience.
Audience related feedback	Wiki-Highlights appeals more to 23-24 year-olds & among more frequent Wikipedia users.	With technology changing so fast, 23-24 year-olds may have a different relationship with it versus 18-19 year-olds. The difference suggests further development should be targeted at the needs of the younger group to truly future-proof Wiki-Highlights.
Audience related feedback	The Wiki-Highlights appealed more to users in Nigeria and Indonesia and much less for users in Germany & US.	We see Wikipedia overall less appealing in more developed markets - perhaps due to more established digital environments & brands. It may be that in these markets there would need to be more difference between Wiki-Highlights and current Wikipedia experience to have an impact.
Feature related feedback	Wiki-Highlights outperformed controlled site on perceptions of 'fun' & 'easy'	23-24 yr olds find Wiki- Highlights more fun, perhaps due to less use of apps like TikTok. The above suggests a bigger incremental change is needed for younger users. Also, heavier Wikipedia users are more likely to find Wiki- Highlights easy. There may be a need to compare it versus other sites to explore how to improve Wiki-Highlights to bring in lighter users.
	Users liked the topics & imagery of both sites; Wiki- Highlights stood out on simplicity of language & content length.	The current lack of perceived difference of imagery on Wiki- Highlights versus Control suggests this could be an area of further development. The shorter content length works well, and should be retained.
	Users would improve the navigation and they desired more topics.	Review the back button (this may have been impacted by the survey buttons, but we don't believe so given the verbatim). We could explore additional topics relevant to this audience - e.g. movies, music, tech.

2. A/B testing:

During the survey testing, we instrumented the microsites to capture specific metrics for the 2 groups to gauge the depth of engagement further.

The Experiment Group participants were shown the wiki-highlights/ summarised content as seen here.
The Control Group participants were shown long form Wikipedia Article-type of content as seen here.

Overall metrics and observations are documented in the table below; available with a full report .

Overall Metrics	Observations
Time on site (session length)	Time on Homepage + Content page: Overall, the experiment group seemed to stay longer than the control group.
	Time on Homepage: Experiment group spent the same amount of time as the control group on the homepages.
	Time on Content Page: Experiment group stayed longer than the control group on the content pages.
Completion rate (willingness to complete the content)	Experiment group: 1,658 highlights opened with a 72.2% completion rate. had more highlights read but a lower completion rate. Control group: had 1,112 articles opened, with a 78.1% completion rate.
Number of content consumed per session (willingness to view subsequent content)	Experiment group: 95% of sessions consumed 0 to 4 highlights per session. The users in the experiment group consumed more content than the users in the control group. Control group: 95% of sessions consumed 0 to 3 articles per session. *** 0 means users only viewed the homepage in certain sessions.
Majority reads by content type	Experiment group top 3 visited pages: Lionel Messi, Climate Change Elephant. Control group top 3 visited pages: Lionel Messi Friends Japan.
Majority reads by category type	Experiment group top 3 visited categories: Topical Personalities Nature Control group top 3 visited categories: Lifestyle Personalities Nature

We also evaluated the metrics across the 6 countries we ran the experiment;

Country Metrics	Observations
Time on site (session length)	Experiment Group: Brazil, India, United States and Nigeria, users spent more time on homepages and content pages. Control group: Indonesia and Germany, users spent more time on homepages and content pages.
	Experiment Group: Brazil, Nigeria, and India, users spent more time on homepages. Control group: Indonesia and the United States, users spent more time on homepages. ***In Germany, users spent similar time on homepages for both the experiment and control groups.
	Experiment Group: Brazil, India, Indonesia, Nigeria, and US had more users spending more time on content pages. Control group: Germany, in 90% of the sessions, the control group had more users spend more time on content pages. ***Nigeria had significantly much longer time spent on content pages compared to other countries.
Completion rate (willingness to complete the content)	Experiment Group:Content completion rate in experiment groups is lower in every country except India and Indonesia.
Number of content consumed per session (willingness to view subsequent content)	Experiment Group: Brazil, India, Nigeria, and the United States, users viewed more content in the experiment group. Control group: Germany, users viewed fewer content in the experiment group than the control group per session. ***In Indonesia, users viewed a similar amount of content per session in both groups.
Majority reads by content type	Content, such as Lionel Messi and Climate change, appears in the top-viewed lists of most countries. The rest of the top-viewed content differs from country to country.
Majority reads by category type

Experiment Learnings:

Learnings from the Survey testing:
- Wiki-Highlights had more appeal, and is seen as more unique, though only marginally. Although differences in appeal & uniqueness are statistically significant, they are slight differences.
  - This suggests that more could be done to differentiate Wiki-Highlights from the current Wikipedia reading experience.
Learnings from the A/B Testing:
- Users spent more time and consumed more Wiki-Highlights content than articles. However, more articles were completed over Wiki-Highlights which could be attributed to the fact that users did not expand each article section to and this may have affected the reading time of the control group.
  - Brazil, India, and Nigeria users favored Wiki-Highlights;
  - Germany users favored articles where they spent more time, consumed more and completed more content.