Extension:ContactManager

MediaWiki extensions manual

ContactManager Release status: stable
Implementation	Hook , Special page
Description	A complete email client, contact manager and bulk email app for MediaWiki based on VisualData
Author(s)	thomas-topway-it (thomas-topway-ittalk)
Latest version	1.2.3 (20-10-2024)
MediaWiki	1.35.3+
License	GNU General Public License 2.0 or later
Download	Download extension Git [?]: Download Git master browse repository (Phabricator · GitHub) commit history repository contributors (GitHub) code review
Parameters $wgContactManagerSchemasMailbox $wgContactManangerDisableSidebarLink $wgContactManagerSchemasMailer $wgContactManagerMailboxFoldersArticle $wgContactManagerSchemasMessageFilter $wgContactManagerSchemasComposeEmail $wgContactManagerSchemasRetrieveMessages $wgContactManagerAttachmentsFolder $wgContactManagerMailboxInfoArticle $wgContactManagerSchemasMailboxFolders $wgContactManagerSchemasMailboxInfo $wgContactManagerSchemasContact $wgContactManagerSchemasMessageHeader $wgContactManagerContactsArticle $wgContactManagerSchemasDefineJob $wgContactManagerSchemasIncomingMail
Added rights contactmanager-can-manage-mailboxes
Hooks used BeforeCreateEchoEvent BeforeInitialize BeforePageDisplay LoadExtensionSchemaUpdates ParserFirstCallInit SkinBuildSidebar VisualData::OnFormSubmit
Quarterly downloads	0
Translate the ContactManager extension if it is available at translatewiki.net
Issues	Open tasks · Report a bug

ContactManager is a complete email client based on MediaWiki and VisualData . Thanks to the flexibility of MediaWiki, the power of json-schema and the modularity of VisualData, it also aims to function as Contact manager (hence the name), and email marketing software, comparable to Mailchimp or MailUP.

ContactManager is also the first wiki-application based on VisualData, and as such represents both a test and a demo of its advanced features, like forms with preload-data, navigation using the parser function queryLink, TinyMCE integration, queries in form inputs (case insensitive), printouts with base64 modifier, form fields interactively shown based on conditions, form buttons, and much more. Of course it also competes with efforts in creating web-applications based on SemanticMediaWiki and PageForms, like SemanticActions or Eikolo and aims to introduce some advancement or contributions for this challenge.

Key-features:

• multiple mailboxes (IMAP and SMTP)
• all IMAP search criteria supported
• email filters (all email metadata supported)
• multiple mailers (all mailers supported by Symfony are supported: smtp, sendmail, native, Amazon SES, Mandrill, Mailgun, Mailjet, OhMySMTP, Postmark, SendGrid, Sendinblue)
• automatically retrieves contacts from sent/received email
• autocomplete to, cc and bcc using standard VisualData form queries
• fully configurable using VisualData's schemas, templates and Mediawiki articles
• TinyMCE integration

UI elements

After the installation the extension creates the following links in the sidepanel:

Installation

• install PHP-Imap on your server. This on ubuntu is done with the command

sudo apt install php7.1-imap

(replace "7.1" with your php version)

• Download and place the file(s) in a directory called ContactManager in your extensions/ folder.
• Run composer update --no-dev in the extension's folder, to install the required PHP libraries
• Add the following code at the bottom of your LocalSettings.php

wfLoadExtension( 'ContactManager' );

• Run php maintenance/update.php (this will install the required libraries, and will import on the wiki the articles in the ContactMananger namespace, templates and schemas required by the extension)
•  Done – Navigate to Special:Version on your wiki to verify that the extension is successfully installed.

Install the following extensions:

• Extension:UrlGetParameters
• Extension:Tabs or Extension:TabberNeue

Enable StringFunctions:

$wgPFEnableStringFunctions = true;

The basic configuration should appear as follows:

wfLoadExtension( 'VisualData' );
// wfLoadExtension( 'TabberNeue' );
// $wgTabberNeueEnableAnimation = false;
wfLoadExtension( 'Tabs' );
wfLoadExtension( 'UrlGetParameters' );
wfLoadExtension( 'ContactManager' );
$wgPFEnableStringFunctions = true;

Now create two entries in the crontab file as follows:

* * * * * php www-data [absolute path to your mediawiki install]/maintenance/run.php "[absolute path to your mediawiki install]/extensions/ContactManager/maintenance/CheckMessages.php" > /dev/null 2>&1
* * * * * php www-data [absolute path to your mediawiki install]/maintenance/runJobs.php --type ContactManagerJob > /dev/null 2>&1

(this assumes that the username of the webserver is "www-data" and it can execute the related files)
The first script creates the jobs to check for new email messages if are due, and the second execute the queued jobs each minute. A future version of the extension will be completed with Extension:Echo notification with logs for administrators.
Here is a tutorial to manage crontabs.

In order to create a crontab for a specific user (i.e. "www-data") use the following command:

sudo crontab -e -u www-data

Then the command above should be set as follows:

* * * * * php [absolute path to your mediawiki install]/maintenance/run.php "/var/www/html/mediawiki-1.42.1/extensions/ContactManager/maintenance/CheckMessages.php" > /dev/null 2>&1
* * * * * php [absolute path to your mediawiki install]/maintenance/runJobs.php --type ContactManagerJob > /dev/null 2>&1

Workflow

The extension creates 4 pages in the ContactManager namespace by which to create/edit mailboxes, add mailers, create/edit contacts, compose, read, reply email and navigate all the articles, schemas and templates created by the extension.

Here is all you need to have the "wiki-app" up & running in few minutes.

1) Create a mailbox in the page "ContactManager:Mailboxes"

Just insert the desired name for the mailbox and a list of legitimate email addresses in the form "name <email>" (or just email), as well as the reply-to field.

2) After creating a mailbox the app redirects you to the following page:

This is the main control-panel for each mailbox, from which you can see the mailbox info, the mailbox folders, create one or more filters, navigate/access messages, and compose email.

Each primary button is associated to an editable schema pre-filled with the data to run the related job, however before to go with that, it is necessary to set-up the mailbox credentials, in the following way.

3) Create a key in the global parameter $wgContactManagerIMAP (used to receive email) and $wgContactManagerSMTP (used to send email) in Manual:LocalSettings.php, with the name of the created mailbox, as follows:

$wgContactManagerIMAP = [
	'mailbox#1' => [
		'server' => '',
		'username' => '',
		'password' => '',
		'port' => 993,
	],
	// ...
	
];

$wgContactManagerSMTP = [
	'mailbox#1' => [
		'server' => '',
		'username' => '',
		'password' => '',
		'port' => 465
	],
    // ...
];

("mailbox#1" should be replaced with the mailbox name set above)

Use PHP's getenv in order to store the password as environment variables for security reasons.

4) Create more mailboxes as needed. Here is how the same page appears after creating a few mailboxes.

5) Once that the credentials have been stored in the server and associated to the mailboxes created through the UI, start pressing "Get Mailbox info" and "Get Folders". Then run the following command from the console from the MediaWiki installation path.

php maintenance/runJobs.php

6) Now that the list of folders have been retrieved from the mailbox (if all went well) and stored in the json-data associated to the article, press the edit button besides the primary button "Get messages", in order to set-up the related schema.

The form allows to select the folders to be retrieved, and to set a pagename formula for each of them.

Additionally, it allows to set-up search criteria. All the IMAP search criteria are supported and accessible/editable through the form.

• ALL
• ANSWERED
• BCC
• BEFORE
• BODY
• CC
• DELETED
• FLAGGED
• FROM
• KEYWORD
• NEW
• OLD
• ON
• RECENT
• SEEN
• SINCE
• SUBJECT
• TEXT
• TO
• UNANSWERED
• UNDELETED
• UNFLAGGED
• UNKEYWORD
• UNSEEN

For testing purpose, it is suggested to set a low limit (the last field of the form) and to remove the limit only when you are happy with the data structure.

When you are done, press the primary button "Get Messages" and run again

php maintenance/runJobs.php

(a next version of the extension will run jobs automatically with some interval)

7) Optionally create one or more filters and assign them to the schema associated to the "Get messages" button in order to download only the messages fitting some criteria.

All the following are supported:

• id
• imapPath
• mailboxFolder
• isSeen
• isAnswered
• isRecent
• isFlagged
• isDeleted
• isDraft
• date
• headersRaw
• headers/date
• headers/Date
• headers/subject
• headers/Subject
• headers/message_id
• headers/toaddress
• headers/fromaddress
• headers/ccaddress
• headers/reply_toaddress
• headers/senderaddress
• mimeVersion
• xVirusScanned
• organization
• contentType
• xMailer
• contentLanguage
• xSenderIp
• priority
• importance
• sensitivity
• autoSubmitted
• precedence
• failedRecipients
• subject
• fromHost
• fromName
• fromAddress
• senderHost
• senderName
• senderAddress
• xOriginalTo
• toString
• ccString
• messageId
• textPlain
• textHtml
• visible_text
• attachments/id
• attachments/contentId
• attachments/type
• attachments/encoding
• attachments/subtype
• attachments/description
• attachments/name
• attachments/sizeInBytes
• attachments/disposition
• attachments/charset
• attachments/emlOrigin
• attachments/fileInfoRaw
• attachments/fileInfo
• attachments/mime
• attachments/mimeEncoding
• attachments/fileExtension
• attachments/mimeType

The string fields can be matched depending on whether they contain, do not contain, on based on a regex, a given value, and to trigger an action between "skip", set category, or set an alternate pagename formula.

The fields of type integer can be matched based on an interval of values, and the boolean fields based on a toggle.

8) Press one of the folders that you have included in the schema associated to the "Get messages" button.

It contains a list of retrieved email. Note the url with query parameters (created using VisualData's QueryLink parser function) the "open" and "source" links in the first field and the "reply" button in the action field.

The link "open" will redirect to the "Read email" page where some of the data related to the email are shown in an accessible way, and where you can answer the message as you would do with a standard email client. In order to display properly the text, html and headers content ContactManager makes use of shadow DOM and of the Tabs extension. The form "Compose message" supports both plain text and html thanks to the integration with TinyMCE (available in VisualData since version 1.0.8)

The original message is quoted both in the text and html format using a Extension:Scribunto/en module and then passing the result in base64 encoding to the form through the VisualData's parameter "preload-data".

At this point it is possible, or you have learned, to do the following:

• create one or more mailboxes
• associate mailboxes with their credentials (although possible, those shouldn't be stored on the wiki for security reasons)
• retrieve mailbox info, folders list
• create filters used to retrieve messages
• retrieve messages from the mailbox using any criteria and the created filters
• navigate messages of any folder, access their content and reply
• compose a new message

All the retrieved data are stored in the wiki itself as json data, either within an additional slot (for instance in case of the mailbox page) or in the main slot (as for contacts and email messages themselves). When the data are stored in the mail slot, they can be accessed directly, and can be programmatically retrieved using a VisualData's query or print parser function, also taking advantage of the QueryLink parser function to pass required parameters and then retrieving/using it through the UrlGetParameters extension. No additional tables are required or used by ContactManager, nor additional functionalities for the UI/UX side that aren't already provided by VisualData. ContactMananager just implements the required background operations for sending/receiving email and provides a set of schemas, articles and templates to build the wiki-app in a way that is fully configurable and editable by users.

9) Create additional mailers. A "mailer" is a program that sends email messages, that can either reside on the server (like sendmail) or be provided as a SaaS, like SendGrid, mailgun and more. Thanks to the integration with symfony mailer, ContactManager supports all the providers supported by Symfony. They can be used to send bulk email messages and usually they support templates and substitutions. Ça va sans dire that MediaWiki is an optimal platform to implement them, since they are already hallmarks and core principles of its architecture.

In order to create an additional mailer (either sendmail or one of the third party providers) go the page "ContactManager:Mailers", choose the provider, and assign to it a name using the following form.

Once created the page shows an editable table:

10) Set-up the credentials for the created mailer. Each mailer can use SMPT, HTTP and/or API transport, and requires its specific set of credentials, according to the following table.

Provider	SMTP	HTTP	API
Amazon SES	username/password	access_key/secret_key	access_key/secret_ke
Gmail	username/app-password	n/a	n/a
Mandrill	username/password	KEY	KEY
Mailgun	username/password	KEY/DOMAIN	KEY/DOMAIN
Mailjet	access_key/secret_key	n/a	access_key/secret_key
Postmark	ID	n/a	KEY
Sendgrid	KEY	n/a	KEY
Sendinblue	username/password	n/a	KEY
OhMySMTP	API_TOKEN	n/a	API_TOKEN

(source: https://symfony.com/doc/5.x/mailer.html)

Correspondingly, ContactManager will search an object in LocalSettings.php with the following structure.

$wgContactManagerAMAZON = [
	'mailer#1' => [
		'smtp' => [
			'username' => '',
			'password' => ''
		],
		'http' => [
			'access_key' => '',
			'secret_key' => ''
		],
		'api' => [
			'access_key' => '',
			'secret_key' => ''
		]
	],
    // ...
];

// ...
$wgContactManagerSENDGRID = [
	'mailer#2' => [
		'smtp' => [
			'username' => '',
			'password' => '',
		],
		'api' => [
			'KEY' => '',
		]
	],
    // ...
];

where $wgContactManagerAMAZON is the name of the global parameter composed with uppercase name of the provider as suffix, "mailer#n" is the chosen provider name, and the required keys reflect the table above for each provider and transport.

Therefore the approach is to use an object for each provider, each of them containing one or more accounts, for better readability compared to using a single object for all of them.

11) When sending an email from a specific mailbox, select the created mailer from the form. When selecting the mailer, the form shows the "from" list associated to that mailer.

(note that this form, as all the forms used by ContactManager, is a standard VisualData form created through the Schema Builder)

12) Manage contacts. Retrieved contacts are created under the ContactManager:Contacts article (in the form "ContactManager:Contacts/{first name} {last name}" and they can also be edit/created manually through a similar interface of mailboxes and mailers.

The contact name is parsed from the email thanks to the library The Iconic name-parser and each contact is saved with all the parsed parts supported by the name-parser, plus additional properties:

• first name
• last name
• salutation
• middle name
• nick name
• initials
• suffix
• email addresses (array)
• phone numbers (array)
• links (array)
• picture

As mentioned, registered email addresses are queried in the "to", "cc" and "bcc" fields, while the phone numbers may be used by a Twilio integration in future versions of the extension.

Of course such data structure allows to easily implement replacements, for instance:

dear {{first_name}}, 
    ....

which, together with sending email to categories of contacts, is expected to be implemented soon by the next release of the extension.

Known issues

• attachments are only partially implemented
• jobs must be executed manually

Road map

• display the reply using Lua to quote the message and base64-encode the result in conjunction with the printout modifier +base64
• show html in shadow root
• assign categories to the replies of sent messages
• send to categories of contacts
• implement/complete implementation of attachments
• add Extension:Echo notifications
• implement email templates
• showcase the use of filters when used in conjunction to bulk sending (for instance to redirect specific campaigns to different areas of the wiki)
• TinyMCE integration (implemented within VisualData)
• email campaigns
• Twilio integration

Support & bugs

For professional support please write at the email address posted here

See also

• Extension:VisualData
• Extension:ContactManager/Legacy
• php-imap
• PHP imap-search
• symfony mailer
• name-parser
• EmailReplyParser
• html2text
• Extension:SubpageNavigation