User:Mariapacana/OPW Progress Report

Progress Report

December 2013

Week 1: December 10 to December 16

1. Familiarized myself with developer tools like Bugzilla, Mediawiki Gadgets, Gerrit, etc.
2. Submitted patch to streamline the setting of debugging flags.
3. Researched different ways of dynamically debugging Node applications (e.g., node-debugger, node-inspector).
4. Began working on disabling generic tracing (e.g., --trace without any tracing flags).

Week 2: December 17 to December 23

1. Continued to work on disabling generic tracing.
2. Learned about how the parser pipeline works and how different transformations relate to one another.

Week 3: December 24 to December 30

1. Cleaned up some indentation errors and typos (patch 1, patch 2).
2. Disabled generic tracing (e.g., --trace without any tracing flags; patch).
3. Began working to create a generic error logging function in Util.js. (in-progress patch)

January 2014

Week 4: December 31 to January 6

1. Transferred logger function from Util module to its own class, on mentor's suggestion (patchset 4).
2. Added wrapper functions such as trace() and debug()
3. Visited User:Ssastry in Minnesota! Learned how...
   • Synchronous & synchronous token transform managers transform Wikitext to HTML
   • Parsoid evolved and how it relates to the original PHP parser
   • Parsoid tests work; how/why some tests are blacklisted

Week 5: January 7 to January 13

1. Moved logging function from own class into MWParserEnvironment object ("env"), after getting feedback from the team (patchset 9).
   • The many-logger approach was considered overly complex; abandoned in favor of a single log function that dispatches to different backends.
   • Consequently, we put the logger into the env object, which is widely available throughout Parsoid codebase and has access to tracing / debugging flags.
2. Removed wrapper functions (e.g., trace(), debug()) from logging, consolidating everything into a single all-purpose function.

Week 6: January 14 to January 20

1. Rewrote logger function (patchset 12)
2. Began rewriting error and warning messages to use logging function (patchsets).
3. Wrote blog post about meeting my mentor in person.

Week 7: January 21 to January 27

1. Continued to replace error and warning messages throughout the codebase. (patch 1, patch 2)
2. After User:cscott made enhancements to the logging function so that it could accept arbitrarily many parameters, checked all function calls to make sure they still worked.

Week 8: January 28 to February 3

1. Updated patch for replacing error / warning messages. (patch 6,patch 7, patch 8)
   • Based on team feedback, rewrote logger function to accommodate logging to different backends (e.g., files, HTTP response objects).
   • Again moved the logger function into its own Logger class, although it will still continue to be accessed from the MWParserEnvironment object.
2. Began replacing tracing and dumping calls with logger function. (patch 1)
3. Wrote blog post about the Parsoid team's consensus-based code review process.

February 2014

Month of the Never-Ending Error Logging Patch

Week 9: February 4 to February 10

1. patchset 9
   • Set up default backend to log errors to and made it possible to register new backends (generally error callback functions).
   • Make logger exit process when logType is fatal.
2. patchset 17
   • Changed emitMessage to emit an object to a logging backend, rather than a message string.
   • Made it possible to register backends using regular expressions in addition to strings.

Week 10: February 11 to February 17

1. patchset 18
   • When the logType is fatal, wait until all backends are logged to before ending the process.
2. Began to address recursive error logging issue.
   • Parsoid service on main branch crashed the servers because of a recursive error logging problem in Connect (middleware handler for Node). Researched the cause of the error logging problem in the main branch, as well as ways to prevent infinite recursion in my logger branch.

Week 11: February 18 to February 24

1. patchset 19
   • Changed Logger to prevent recursive logging. If the logger is currently processing a log request but gets called again, it ignores the subsequent call unless the logType is fatal. If the logType is fatal, the process must be ended.
   • Used Node.js's async.parallel to log to a default backend after all other backends have been logged to.

Week 12: February 25 to March 3

1. Submitted patchset 22
   • Restructured Logger class to remove nested functions
   • Changed log levels; now "fatal/request" ends an HTTP response *and* terminates a process, while "fatal" ends an HTTP response. Additionally, "error" is for errors that must be known about at runtime, while "warning" is for errors that do not need to be known at runtime.
   • In Parsoid web server, added helper method / flags to prevent headers from being set twice (triggering infinite recursion issue in Connect.)
2. Blogged about my never-ending patch, and some ideas about how to avoid doing that.

March 2014

Week 13: March 4 to March 10

1. Never-ending logger patch got merged at last!
2. Submitted first versions of patch to support structured logging (with arbitray input data) and to allow logging backends to customize their messages.
3. Made the --quiet option on parserTests truly quiet (patch).
4. Converted some tracing code to env.log (tokenstreampatcher patch, peg-tokens patch)
5. Blogged about a favorite Javascript trick I picked up while working on Parsoid (first post in a series of 5)

Post-Internship: March 10 - April 16

1. Structured logging patch is merged.
   • Created logData module that can accept an arbitrarily long array of arguments from Logger and convert it into different types of messages that can be accessed by backends. Arguments can consist of any type of object.
   • Logger backends can now customize their messages by accessing the relevant properties from a logData module.
2. Patch to make logger support multiple backends for the same logType is merged.
3. Patch to improve tracing and debugging in the logger is merged. It supports handler-specific debugging flags and creates a tracing / debugging backend in the Logger, with formatting specific to tracing / debugging events.

Project Status

My main accomplishment during this project was creating a Logger class that could be used throughout the Parsoid codebase in lieu of console.log / console.warn. The logger can send logging data to a variety of backends (console, file, HTTP response object), and will end a process if the logType is "fatal". It relies on a logData class to convert log information (which can consist of an arbitrary number of objects, of any type) into message properties that are accessible to backends. The logger is now being used to log the vast majority of errors and warnings throughout the codebase, and is starting to be used in tracing and debugging.