Documentation/Maturity model for MediaWiki technical documentation

Overview

This page contains an outline, research and citations for a maturity model for the MediaWiki / Wikimedia technical documentation organization and content. The maturity model is intended to guide assessments about the quality of the existing organizational structure, processes for producing technical documentation and assessing the quality of technical documentation content on MediaWiki and across Wikimedia projects, and help set standards for planning future technical documentation on MediaWiki / Wikimedia projects.

Benefits of maturity models

According to the Software Engineering Institute at Carnegie Mellon:

"Using a maturity model as the foundation for improving processes, practices, and performance provides organizations and communities the ability to:

Benchmark internal performance. Using a standard measurement approach based on the model content, organizations can determine where they are in their improvement journey and set targets for future investments in performance improvement. Different operating units in the same organization can also use the benchmark to compare performance, particularly when similar functions are performed in the operating units.

Catalyze performance improvement. By taking measurements against the model over a period of time, organizations can use the model as the basis for continuous performance improvement. And, because the model reflects the best practices of the domain or discipline, it can be used as the basis for developing action plans to close performance gaps and improve maturity.

Catalyze improvements in community performance.  Because the model and measurement approach tie the community together, organizations can not only compare their performance against peer organizations but also determine a “community” performance profile. Creating such a profile may spur additional community investment in common and shared challenge problems.

Create and evolve a common language. Maturity models often create a consistent way of thinking and communicating about a domain that is embodied in model language or taxonomy. Consistent language and communication helps domains of knowledge evolve into disciplines where a common language can translate into repeatable, consistent, and predictable performance over time"

History: capability maturity model levels

Capability maturity models became the basis for many other types of maturity models including those for information management.

There are five levels defined along the continuum of the model and, according to the SEI: "Predictability, effectiveness, and control of an organization's software processes are believed to improve as the organization moves up these five levels. While not rigorous, the empirical evidence to date supports this belief." The progression of levels outlined in the capability maturity model provide a framework for creating maturity models in other disciplines.

1. Initial (chaotic, ad hoc, individual heroics) - the starting point for use of a new or undocumented repeat process.
2. Repeatable - the process is at least documented sufficiently such that repeating the same steps may be attempted.
3. Defined - the process is defined/confirmed as a standard business process.
4. Capable - the process is quantitatively managed in accordance with agreed-upon metrics.
5. Efficient - process management includes deliberate process optimization/improvement.

Learn more about the capability maturity model.

Information process maturity model levels

The information process maturity model speaks to how information is produced within the organization. This can be applied to the technical documentation / communication organization within the context of the larger organization.

The following descriptions are specific to Mediawiki / Wikimedia organizational structure and resources, specific to technical documentation and communication:

1. Ad Hoc: (Needs Improvement): Characterized by lack of uniform practices. Lone technical writers/communicators managed by non-technical writers/communicators; volunteer technical writers/communicators with varying skill sets. Processes are unique to individual technical writers/communicators, rather than standard, agreed upon across the organization or community. Lack of formal method for prioritizing projects. No clear understanding of scope and scale of existing technical documentation, future needs for technical documentation, or overall quality of content collections. No clear understanding of users or mechanisms for user feedback.
2. Rudimentary: (Needs improvement): Rudimentary standards and processes are being implemented under the guidance of a technical writing/communication professional. Style guides and standards are introduced. Formal processes for managing technical documentation are discussed with attempts at implementation. Early formal processes may include, quality assurance, peer review, copy and developmental editing. However, implementation of processes is not prioritized. Support or resources (time, money, staff) may not be committed. Processes still lack consistency in implementation.
3. Organized and repeatable: (Baseline) Under strong leadership, staff and community agree on uniform processes and are committed to following them. Uniform tools, standards and templates are used. Audience / user needs take priority and inform technical documentation content and organization. Documentation projects are identified by priority and scoped; planning and quality assurance are recognized as key elements to their success. Qualified technical writers/communicators guide this process. At this stage, staff and volunteers may recognize technical content collections in need of redesign and revision. User research is identified as essential to successful quality technical documentation. Technical writers/communicators are working across teams in the organization to surface shared needs, establish benchmarks and identify projects.
4. Managed and sustainable: (Superior) This organization is well-managed and has made a strong commitment to follow through on practices and practices outlined in level three. These process and practices (and with them the quality of technical documentation) will not suffer from changes to management or overturning staff or volunteers. Planning and training would continue. More specialized staff members are hired with a focus on specific aspects of technical writing / communication: document managers, writers, designers, user researchers, etc. An increased focus on user experience takes place at this level. The organization is committed to understanding its users needs and implementing strategies for improvements to technical documentation that best suit these needs. The organization focuses on ways to widen the scope of its technical documentation and training and engage new audiences.
5. Optimizing: (Superior) A self-actualized organization with the maturity to evaluate its own established practices with quality and innovation in mind, an agile organization that can respond to the needs of users and technical collaborators swiftly, as they arise. An organization that seeks improvement and alignment with the strategic direction of the larger organization. Staff members are experienced and knowledgeable; they collaborate effectively with the community and seek to understand the needs of users. They encourage and guide volunteer contributions and work to build sustainable relationships and partnerships with community members who have technical writing / communication skill sets or who wish to grow their skill sets in the context of the movement. This organization looks to similar organizations to evaluate its practices and standards. This organization seeks to be a standard barer and example of excellence within the context of similar organizations.

Descriptions here are specific to technical documentation content collections and articles:

1. Ad Hoc: (Needs Improvement): Everyone agrees that somebody should do something about documentation, but no one is responsible for establishing or guiding a process. No one is responsible for clarifying or answering questions users may have about documentation. Documentation is created on the spot or as an afterthought, rather than as a priority or as part of the development cycle. Wikipages are created as containers for information that needs to be somewhere, but organization and layout of information are not a consideration. Information may be duplicative across collections or projects. There are no agreed upon style standards; documents are formatted in whichever way the page creator and subsequent editors decide.There are no standard templates for creating different genres (walk-throughs, recipes, FAQs, reference guides, etc) and the documents may be mixed in genre. The purpose any given technical document is often unclear. It may cover many topics or just one topic minutely. Information maybe outdated or missing. Similar pages or complementary resources are not linked. There is no clear process for requesting new documentation or requesting updates to existing documentation or collections. There is no process for formally peer editing or testing the technical documentation for completeness. Audiences are not considered. Community conversations about technical documentation may not be healthy or productive.
2. Rudimentary: (Needs improvement): Technical documentation content collections exist but may have limited and inconsistent coverage of topics. Some topics maybe thoroughly covered; others may not be. It may appear there is plenty of technical documentation even when there is a paucity of quality technical documentation. There is no method for prioritizing more relevant or needed information within a content collection. Little attention is paid to the structure of content collections, and it is difficult to know where information is or where it should be once it is created. Pages and articles do not follow a standard naming conventions. The genre of a document may be more sophisticated and recognizable than at level one; however it does not follow a standard template. Though technical documentation is being created, the audiences for it are largely undefined, unknown, or misunderstood, and coverage of concepts may be incomplete or irrelevant for their needs. Developer documentation is basic and may exist as lore (information held by individual technical contributors but not written down). For software documentation, only basic installation, configuration or running details are documented. These may be incomplete or untested. Technical documents are often abandoned after they are created with no system for few periodic updates to ensure the accuracy or completeness of information. There may be a style guide available, but it is not widely used. Technical documentation and communication is created based on industry best practices and assumptions about the audiences who are using it. The community of editors working on these documents may be dedicated but isolated, with few resources or collaboration. Community conversations about technical documentation may not be healthy or productive.
3. Organized and repeatable: (Baseline) Simple processes and tools have been defined for surfacing technical documentation needs and asking for creation or assistance for creation with documentation. Processes and tools are well documented, and individual contributors can follow these processes and use these tools with minimal on-boarding. When and where technical documentation can be automated, it is. There is a style-guide, which defines agreed upon standards for quality, and it is used to guide the creation of new documentation. There is a system in place to ensure documentation is periodically reviewed for accuracy. There is a process for keeping the documentation up-to-date and accurate. As existing technical documentation is reviewed and updated, it is edited to fit the agreed upon style guidelines. Documentation content collections are well-defined and easy to recognize -- though they may not be consistently organized, topics are thoroughly covered. Technical documents have a clear purpose and audience. Technical documents follow a defined template, so there is consistency about types of documentation across collections. There is a method for identifying which content collections need attention and resources, and this work is planned and professionally managed. There is a good faith and supported effort to engage and collaborate with members of the community to produce quality documentation content. Audience research becomes more important at this stage. While the efforts may not be sophisticated, efforts are made to better understand who is reading and using the documentation.
4. Managed and sustainable: (Superior) There is a comprehensive strategy for improving quality and growing sustainable technical documentation content across MediaWiki/Wikimedia projects. There are specialized subject matter experts working together to implement the aspects of this strategy. There is an effort to begin bench marking quality of technical documentation and user experience against other similar organizations. The processes for surfacing technical documentation needs and calls for improvement are well-established and a part of the culture of technical community. Content collections are well-established and organized. It is easy to find information across and within collections. Subject matter experts, members of the technical community and other audiences collaborate and contribute to the design of content collections to ensure they are organized to best fit the needs of users. Support and resources are dedicated to engaging the community and training volunteer technical writers and documentation project managers. Community collaboration and understanding of audiences are critical at this stage. User-research (both for content and design of collection and pages) take on added emphasis. Outreach to new potential technical collaborators is taking place. Technical documentation and communication are focused around user needs. Technical documentation content is high quality. The purpose of each document is clear. The information is concise and up-to-date. The documentation has been peer edited and reviewed for accuracy.
5. Optimizing: (Superior) (Alignment with movement strategy)

Technical documentation maturity model questions

At the organizational level

• Does the organization support its staff and the community, within the reporting structure and through concentrated efforts to promote efforts to produce and maintain high quality, current technical documentation to technical collaborators? Are there adequate resources (people and funding)? Is there support from the leadership team?
• Is there an established planning process to for MediaWiki technical documentation, to ensure proper coverage of timely, quality technical documentation, which meets the needs of the intended audience?
• Is there a system for peer reviewing and editing technical documentation to ensure quality and accuracy?
• Are the tools and methods used for technical documentation and content management agreed upon and consistently employed by staff and community members across the organization?
• Does the organization have the ability to estimate the time or cost (in staff and volunteer hours) of specific technical documentation projects? Is the organization able to effectively judge whether there is value in concentrating efforts on specific technical content or if resources would be more effectively directed elsewhere?
• Is there appropriate staffing to undertake technical documentation projects that produce quality content? Are there sufficient resources and support for volunteers in the community who wish to work on these projects?
• Does the organization actively promote activities (training, edit-a-thons/hack-a-thons, conference attendance, community outreach) to demonstrate interest and support for technical documentation efforts?
• Does the organization actively support efforts to understand audience/user needs in order to ensure technical documentation is appropriate and usable?
• Are technical communicators and writers collaborating to improve quality and consistency of technical documentation across the organizations projects?
• Is there a solid, supported strategy for working with the community on technical documentation projects?
• Are the technical documentation/communication organizations content and community strategies aligned fully with the movement strategy?
• Is improved technical communication a priority for and measurably supported by management and senior leadership, at the organization?

At the technical documentation content collection level

• Are technical content collections organized consistently across projects?
• It is easy for visitors to find and discover the information they need?
• It is clear where new technical documentation goes and where it fits into the collection?
• Is content being created and managed using the same tools?

At the technical documentation article level

• What are the top success measures for this documentation as defined by its audience? Does this documentation meet those success measures?
• Were usability requirements defined for this document prior to creating it? Were they met?
• Is the documentation accurate and up-to-date?
• Is the purpose of the documentation clear?
• Is the audience for the documentation well-defined?
• Is the genre of the document appropriate for its content? Was the appropriate template used to create the document?
• Does the structure and layout of the document facilitate easy information flow and access?
• Does the documentation meet the standards outlined in the MediaWiki Style Guide? Is the documentation error-free?
• Has the documentation been reviewed and tested for usability?
• Is the community conversation / discussion about the document healthy and productive?

What does mature technical documentation look like?

What does mature technical documentation look like?

Increasing maturity -- strategic directions

Technical documentation organizational strategy

• Better discoverability of documents and document collections
• Better getting started flows for writers on MediaWiki.org
• Clearer more navigable landing page for documentation
• More mechanisms and guidance for performing content audits
• Develop a culture of user research and create resources for these
• Socialize the implementation of review processes for documentation

Technical documents content collections and articles strategy

• Make standard templates available and encourage their use
• Make content collections more discoverable
• Use content types to make documents more consistent across collections
• Make it easier for people to identify what is "good content." This can be tied back to feedback flows
• Improve style guide to include more frequently asked questions and more inclusive language
• Continue to create resources to support well-written and well-constructed documents

Community engagement and collaboration strategy

Analyzing and measuring improvements to technical documentation

• Identify the main business goals for improving technical documentation content. Where is the organization meeting these? Where is the organization falling short?
• What are the main community / user identified success measures? Where is the organization meeting these? Where is the organization falling short?
• Is it possible to measure increased effectiveness/productivity/satisfaction for users who find information and are able to implement it with more ease?
• What does user feedback look like over time?
• Are you documenting process improvements?
• Perform periodic content and community audits

Citations, resources, readings

• J. Hackos, Managing Your Documentation Projects. New York: John Wiley & Sons, 1993
• S. Huang, S. Tilley, Towards a documentation maturity model, SIGDOC '03 Proceedings of the 21st annual international conference on Documentation, 2003
• A. Gentle, Conversation and Community: The Social Web for Documentation, 2nd ed, California: XML Press, 2012
• w:Maturity model
• w:Capability Maturity Model
• Maturity Models 101 - Carnegie Mellon SEI:
• Measurement and Analysis in Capability Maturity Model Integration Models and Software Process Improvement
• Key Practices of the Capability Maturity Model Version 1.1
• A maturity model for READMEs | Opensource.com
• Understanding the Content Maturity Model