API:Holidays viewer

Overview

This tutorial covers how to create a demo app that fetches holidays and observances for a given date from Wikipedia, with an option to log in to add new holidays.

Download the code from GitHub Browse the app on Toolforge

The tools and technologies used to create the demo app are:

Python 3 and Flask, a Python framework.
jQuery and Bootstrap. They are loaded from Wikimedia Toolforge for privacy reasons.
MediaWiki Action API modules: API:Parse , API:Login and API:Edit .

A step-by-step process for building this application

Step 1: Set up Python and Flask development environment

Python comes pre-installed on most Linux distributions. For other operating systems, see the Python beginner's guide for installation instructions.

Install Flask by running pip install flask. If you don't have Pip, get it from the official Pip website

Step 2: Create a simple Flask application

In your home directory, create a folder named holidays-viewer which will contain all the app's files. Inside the folder, create a file named app.py and place the following code in it:

#!/usr/bin/python3

from flask import Flask

APP = Flask(__name__)

@APP.route("/")
def list_holidays():
  return "Holidays and observances"

if __name__ == "__main__":
  APP.run()

Run the app using the command python app.py and open http://127.0.0.1:5000/ on your browser. You should see "Holidays and observances" displayed.

Step 3: Create the base layout

The app will have four pages: the homepage, a search page, a login page and an add page. Each page will have some common elements, so we need to create a base layout file called layout.html to contain these elements.

Note that we are using Bootstrap classes to apply a specific CSS style to an element, Materialize icons for the add, search and arrow-back icons, and Jinja to extend the base layout to other pages and to pass variables from Python to HTML.

$HOME/holidays-viewer/templates/layout.html

<title>Holidays</title>

<link rel="stylesheet" href="//tools-static.wmflabs.org/cdnjs/ajax/libs/twitter-bootstrap/4.1.3/css/bootstrap.min.css">
<link rel="stylesheet" href="//tools-static.wmflabs.org/fontcdn/css?family=Material+Icons">

<div class="content bg-secondary rounded m-auto">
  <div class="title-bar bg-primary-dark text-white pl-2">
    <small>Holidays and observances</small>
  </div>
  <div class="header-bar bg-primary text-white shadow p-2">
    {% if request.path != url_for('list_holidays') %}
    <a class=" btn text-white" href="{{ url_for('list_holidays') }}">
      <i class="material-icons">arrow_back</i>
    </a>
    {% endif %}
    <h5>{{header}}</h5>
    <div class="filler"></div>
    <a class="btn text-white" href="{{ url_for('add') }}">
      <i class="material-icons">add</i>
    </a>
    <a class="btn text-white" href="{{ url_for('search') }}">
      <i class="material-icons">search</i>
    </a>
  </div>
  {% with messages = get_flashed_messages() %}
    {% if messages %}
    <div class="alert alert-primary mb-0" role="alert">
      {% for message in messages %}
        {{ message }}
      {% endfor %}
      <button type="button" class="close" data-dismiss="alert" aria-label="Close">
        <span aria-hidden="true">×</span>
      </button>
    </div>
    {% endif %}
  {% endwith %}
  {% block content %}{% endblock %}
</div>
<script src="//tools-static.wmflabs.org/cdnjs/ajax/libs/twitter-bootstrap/4.0.0/js/bootstrap.min.js"></script>

Other pages will extend layout.html using the code below:

{% extends "layout.html" %}
{% block content %}
  <!--content for other pages-->
{% endblock %}

Step 4: List holidays

The root url of the app will trigger the list_holidays(...) function, which lists holidays for a certain date.

In the function and throughout the app, holidays_date refers to the date of the holidays to be listed, header refers to the title of the page, and holidays_html refers to the html which contains the holidays to be listed. We'll also be using the render_template(...) function which renders a specific html file from the templates directory. Other arguments added to the function are variables which are being passed to the html file.

In app.py, update list_holidays() with the code below:

@APP.route('/', methods=['GET', 'POST'])
@APP.route('/<holidays_date>', methods=['GET', 'POST'])
def list_holidays(holidays_date=None):
    holidays_html = ""

    return render_template("index.html", header=holidays_date.replace('_', ' '),
                           holidays_html=holidays_html)

$HOME/holidays-viewer/templates/index.html
{% extends "layout.html" %} {% block content %} <div class="holidays-html"> {{holidays_html\|safe}} </div> {% endblock %}

Get today's date

If no date is specified, we'll list holidays for today's date. To use Python's datetime module to get today's date, import the module with from datetime import datetime then create the following function:

def get_todays_date():
    current_month = datetime.now().strftime('%B')
    current_day = datetime.now().strftime('%d')
    if current_day.startswith('0'):
        current_day = current_day.replace('0', '')

    return current_month + "_" + current_day

Call the function in list_holidays(...):

if holidays_date is None:
        holidays_date = get_todays_date()

Get the holidays to be listed

Once we have the date, we get the holidays for that date. Wikipedia has a page for each date and the holidays are under a section titled "Holidays and observances". To get the holidays, we need to get its section number and the content in that section number.

Create a function to get the section number using API:Parse :

def get_holidays_section(url, page, date_to_get):
    params = {
        "format":"json",
        "action":"parse",
        "prop":"sections",
        "page":page
    }

    response = S.get(url=url, params=params)
    data = response.json()
    sections = data['parse']['sections']
    section_number = "0"

    for index, value in enumerate(sections):
        if value['anchor'] == "Holidays_and_observances":
            section_number = index + 1

        if url == TEST_URL:
            if value['anchor'] == date_to_get:
                section_number = index + 1

    return section_number

Create a function called get_holidays(...) to get the holidays in that section using API:Parse as well, then call the functions in list_holidays(...):

section_number = get_holidays_section(URL, holidays_date, None)
holidays = get_holidays(URL, holidays_date, section_number)
holidays_html = holidays

Update holiday links

The HTML of the holidays returned contains internal links that point to those holidays, e.g "/wiki/New_Years_Day". We need to prepend "//en.wikipedia.org" to these links using jQuery to make them external links in our app, and make them open in a new tab. To do that, add the following code to $HOME/holidays-viewer/static/update-links.js:

$( document ).ready( function() {
    $( ".holidays-html a" ).attr( "target", "_blank" );

    $( ".holidays-html a" ).attr( "href", function( i, href ) {
      return "//en.wikipedia.org" + href;
    });
});

Then add jQuery to layout.html using:

<script src="//tools-static.wmflabs.org/cdnjs/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
<script src="static/update-links.js"></script>

Step 5: Search for holidays of other dates

To get holidays for other dates, create a search route to display a form that collects the month and day to search for:

@APP.route("/search")
def search():

    return render_template("search.html", header="Search date")

$HOME/holidays-viewer/templates/search.html

{% extends "layout.html" %}
{% block content %}
  <div class="instructions m-3">
    Search for holidays by date
  </div>
  <div class="base rounded shadow bg-white m-3">
    <form class="m-auto" action="/" method="POST">
      <fieldset>
        <div class="label-field">Select Month</div>
        <select class="bg-secondary mb-5 border-0" name="monthList">
          <option value="January">January
          <option value="February">February
          <option value="March">March
          <option value="April">April
          <option value="May">May
          <option value="June">June
          <option value="July">July
          <option value="August">August
          <option value="September">September
          <option value="October">October
          <option value="November">November
          <option value="December">December
        </select>
      </fieldset>
      <fieldset>
        <div class="label-field">Select Day</div>
        <select class="bg-secondary mb-5 border-0" name="dayList">
          <option value="1">1
          <option value="2">2
          <option value="3">3
          <option value="4">4
          <option value="5">5
          <option value="6">6
          <option value="7">7
          <option value="8">8
          <option value="9">9
          <option value="10">10
          <option value="11">11
          <option value="12">12
          <option value="13">13
          <option value="14">14
          <option value="15">15
          <option value="16">16
          <option value="17">17
          <option value="18">18
          <option value="19">19
          <option value="20">20
          <option value="21">21
          <option value="22">22
          <option value="23">23
          <option value="24">24
          <option value="25">25
          <option value="26">26
          <option value="27">27
          <option value="28">28
          <option value="29">29
          <option value="30">30
          <option value="31">31
        </select>
      </fieldset>
      <button type="submit" name="search" class="bg-primary btn btn-submit text-white">Submit</button>
    </form>
  </div>
{% endblock %}

Once the search form has been submitted, update holidays_date to be the date that has been entered. To do that, add the following code to list_holidays(...):

if request.method == 'POST' and 'search' in request.form:
        search_month = str(request.form.get('monthList'))
        search_day = str(request.form.get('dayList'))
        holidays_date = search_month +"_"+search_day

Step 6: Add a holiday

The page to which we'll be adding a new holiday is protected from edits by anonymous users, so we need to log in using API:Login#clientlogin first.

To add a holiday, send a request to API:Edit with the date and description of the holiday. The edit adds new holidays to this page on Test Wikipedia: Sandbox/Holidays_and_observances. This is to prevent adding test holidays to English Wikipedia.

After the holiday is added, redirect to the homepage where the holidays added will also be shown, and formatted in bold to differentiate them from the real holidays. To fetch the test holidays alongside the real holidays, update list_holidays(...):

test_section_number = get_holidays_section(TEST_URL, TEST_PAGE, holidays_date)
test_holidays = get_holidays(TEST_URL, TEST_PAGE, test_section_number)

holidays_html = test_holidays + holidays
flash("Holidays added through this app are in bold")

$HOME/holidays-viewer/templates/login.html

{% extends "layout.html" %}
{% block content %}
  <div class="instructions m-3">
    <p>You need to login to Wikipedia in order to add a new holiday
  </div>
  <div class="base rounded shadow bg-white m-3">
    <form class="m-auto" action="/login" method="POST">
      <div class="form-group">
        <div class="form-field">
          <div class="label-field">Username</div>
          <input class="bg-secondary mb-5 border-0" name="username">
        </div>
        <div class="form-field">
          <div class="label-field">Password</div>
          <input class="bg-secondary mb-5 border-0" type="password" name="password">
        </div>
      </div>
      <button type="submit" name="login" class="bg-primary btn btn-submit text-white">Login</button>
    </form>
  </div>
{% endblock %}

$HOME/holidays-viewer/templates/add.html

{% extends "layout.html" %}
{% block content %}
  <div class="instructions m-3">
    <p>Add a new test holiday
  </div>
  <div class="base rounded shadow bg-white m-3">
    <form class="m-auto" action="" method="POST">
      <div class="form-group">
        <div class="form-field">
          <div class="label-field">Date [MMMM dd]</div>
          <input class="bg-secondary border-0 mb-5" name="date" placeholder="e.g April 1">
        </div>
        <div class="form-field">
          <div class="label-field">Description</div>
          <input class="bg-secondary border-0 mb-5" name="description" placeholder="e.g April fools' day">
        </div>
      </div>
      <button type="submit" name="add" class="bg-primary btn btn-submit text-white">Add</button>
    </form>
  </div>
{% endblock %}

Step 7: Styling the app

To add more style to our app, create a stylesheet named style.css and link to it from layout.html by adding <link rel="stylesheet" href="static/style.css">.

$HOME/holidays-viewer/static/style.css

.content {
    width: 420px;
    min-height: 100vh;
}

.holidays-html{
    overflow-y: auto;
    overflow-x: hidden;
    max-height: 88vh;
    scrollbar-width: thin;
}

.base {
    height: 400px;
    display: flex;
}

input, select {
    width: 300px;
    height: 40px;
}

.btn-submit {
    width: 300px;
}

.btn {
    cursor: pointer;
    align-content: center;
    background-color: transparent;
}

.bg-primary {
    background-color: #36c !important;
}

.bg-primary-dark {
    background-color: #2a4b8d !important;
}

.bg-secondary {
    background-color: #eaecf0 !important;
}

.header-bar {
    height: 48px;
    display: flex;
    flex: 1;
    align-items: center;
}

.filler {
    flex-grow: 1;
    text-align: center
}

h2 {
    display: none;
}

ul {
    margin: 5px;
    padding: 0;
}

li  {
    list-style-type: none;
    margin-bottom: 4px;
    background-color: white;
    padding: 8px;
    border-radius: 5px;
}

ul li li {
    box-shadow: 0 .5rem 1rem rgba(0,0,0,.15);
}

Application layout

At this point, the structure of your app should be:

$HOME/holidays-viewer
├── templates/
│   └── add.html
    └── index.html
    └── layout.html
    └── login.html
    └── search.html
├── static/
│   └── style.css
    └── update-links.js
├── app.py

With app.py and layout.html being:

$HOME/holidays-viewer/app.py

#!/usr/bin/python3

"""
    app.py

    MediaWiki API Demos

    Holidays viewer: A demo app that fetches the day's holidays from Wikipedia with options to search for holidays of other dates, and login to add new holidays.

    MIT license
"""

from datetime import datetime
from flask import Flask, render_template, flash, request, url_for, redirect
import requests


APP = Flask(__name__)
APP.secret_key = 'your_secret_key'

URL = "https://en.wikipedia.org/w/api.php"
TEST_URL = "https://test.wikipedia.org/w/api.php"
TEST_PAGE = "Sandbox/Holidays_and_observances"
S = requests.Session()
IS_LOGGED_IN = False

@APP.route('/', methods=['GET', 'POST'])
@APP.route('/<holidays_date>', methods=['GET', 'POST'])
def list_holidays(holidays_date=None):
    """ Lists holidays for the current date or a custom date
    """

    if holidays_date is None:
        holidays_date = get_todays_date()

    # Update date to a custom date
    if request.method == 'POST' and 'search' in request.form:
        search_month = str(request.form.get('monthList'))
        search_day = str(request.form.get('dayList'))
        holidays_date = search_month +"_"+search_day

    # Get the section numbers for the holidays on Wikipedia and for those on the test page
    section_number = get_holidays_section(URL, holidays_date, None)
    test_section_number = get_holidays_section(TEST_URL, TEST_PAGE, holidays_date)

    holidays = get_holidays(URL, holidays_date, section_number)
    test_holidays = get_holidays(TEST_URL, TEST_PAGE, test_section_number)

    holidays_html = test_holidays + holidays
    flash('Holidays added through this app are in bold')

    return render_template("index.html", header=holidays_date.replace('_', ' '),
                           holidays_html=holidays_html)


def get_todays_date():
    """ Get the current month as text and the current day as a number
    """

    current_month = datetime.now().strftime('%B')
    current_day = datetime.now().strftime('%d')
    if current_day.startswith('0'):
        current_day = current_day.replace('0', '')

    return current_month + "_" + current_day

def get_holidays_section(url, page, date_to_get):
    """ Get the section number for holidays on Wikipedia and holidays on the test page
    """

    params = {
        "format":"json",
        "action":"parse",
        "prop":"sections",
        "page":page
    }

    response = S.get(url=url, params=params)
    data = response.json()
    sections = data['parse']['sections']
    section_number = "0"

    for index, value in enumerate(sections):
        if value['anchor'] == "Holidays_and_observances":
            section_number = index + 1

        if url == TEST_URL:
            if value['anchor'] == date_to_get:
                section_number = index + 1

    return section_number

def get_holidays(url, page, section_number):
    """ Get the html which contains holidays
    """

    params = {
        "format":"json",
        "action":"parse",
        "prop":"text",
        "page": page,
        "section": section_number,
        "disableeditsection":1
    }

    response = S.get(url=url, params=params)
    data = response.json()
    text = data['parse']['text']['*']

    return text

@APP.route("/search")
def search():
    """ Search for holidays of custom dates
    """

    return render_template("search.html", header="Search date")

@APP.route("/login", methods=['GET', 'POST'])
def login():
    """ Login to Wikipedia
    """

    if request.method == 'POST' and 'login' in request.form:
        params_0 = {
            "action": "query",
            "meta": "tokens",
            "type": "login",
            "format": "json"
        }

        response = S.get(url=URL, params=params_0)
        data = response.json()

        login_token = data['query']['tokens']['logintoken']

        params_1 = {
            "action": "clientlogin",
            "username": str(request.form.get('username')),
            "password": str(request.form.get('password')),
            "loginreturnurl": "http://127.0.0.1:5000/login",
            "logintoken": login_token,
            "format": "json"
        }

        response = S.post(url=URL, data=params_1)
        data = response.json()

        if data['clientlogin']['status'] != 'PASS':
            flash('Oops! Something went wrong -- ' + data['clientlogin']['messagecode'])
        else:
            global IS_LOGGED_IN
            IS_LOGGED_IN = True
            flash('Login success! Welcome, ' + data['clientlogin']['username'] + '!')
            return redirect(url_for('add'))

    return render_template("login.html", header="Login")

@APP.route("/add", methods=['GET', 'POST'])
def add():
    """ Add a new holiday to a test page and redirect to that date's holidays to show the added holidays
    """

    if not IS_LOGGED_IN:
        return redirect(url_for('login'))

    if request.method == 'POST' and 'add' in request.form:

        # Wiki markup to format the added holiday's text as a list item and in bold
        holiday_text = "* '''" + str(request.form.get('description')) + "'''"
        date = str(request.form.get('date'))

        params_2 = {
            "action": "query",
            "meta": "tokens",
            "format": "json"
        }

        response = S.get(url=TEST_URL, params=params_2)
        data = response.json()

        csrf_token = data['query']['tokens']['csrftoken']

        params_4 = {
            "action": "edit",
            "title": TEST_PAGE,
            "token": csrf_token,
            "format": "json",
            "section": "new",
            "sectiontitle": date,
            "text": holiday_text,
        }

        response = S.post(url=TEST_URL, data=params_4)
        data = response.json()

        if data['edit']['result'] != 'Success':
            flash('Oops! Something went wrong -- ' + data['clientlogin']['messagecode'])
        else:
            flash('New holiday added successfully!')
            return redirect(url_for('list_holidays', holidays_date=date.replace(' ', '_')))

    return render_template("add.html", header="Add holiday")

if __name__ == "__main__":
    APP.run()