Toolserver:OpenStreetMap

This page was moved from the Toolserver wiki.
Toolserver has been replaced by Toolforge. As such, the instructions here may no longer work, but may still be of historical interest.
Please help by updating examples, links, template links, etc. If a page is still relevant, move it to a normal title and leave a redirect.

projects

OpenStreetMap in Wikipedia

The multilingual maps are an interesting aspect of OSM.
We render different styles on toolserver.
query-to-map extract some datas from OSM.

OSM-Geohack seems useful to come with 2 clicks from Wikipedia to each OSM-map. Example (Potsdam,Germany)
OSM-Wikipedia place name tool adds semi-automatically new name:XX values to Openstreetmap using wikipedia links from OSM and interlanguage links from the Wikipedias.
WIWOSM

first steps

With a toolserver account you can access the database with:

psql -h sql-mapnik osm_mapnik

from the normal login server

first steps in PHP

Here a little PHP-Example to get a list of streets in a bounding box:

<?php

// set the charset to utf-8 -- all data in OSM ist stored as utf-8
header('Content-Type: text/html; charset=utf-8');

// bbox of mainz
$bbox = array(8.11, 50.07, 8.46, 49.91);

#use pg_escape_string() against sql-injection if you have input

// open psql connection
$pg = pg_connect('host=sql-mapnik dbname=osm_mapnik');

// check for connection error
if($e = pg_last_error()) die($e);

// select all streets with a name in bbox
$sql = 'SELECT osm_id, name FROM planet_osm_line WHERE 
	highway IS NOT NULL AND 
	name IS NOT NULL AND
	way &&
		ST_Transform(ST_SetSRID(ST_MakeBox2D(
			ST_Point('.floatval($bbox[0]).','.floatval($bbox[1]).'), 
			ST_Point('.floatval($bbox[2]).','.floatval($bbox[3]).')), 
		4326),900913)
	
	ORDER BY osm_id
	LIMIT 200';

// query the database
$res = pg_query($sql);

// check for query error
if($e = pg_last_error()) die($e);

// produce some output
echo "<ul>\n";
while($row = pg_fetch_assoc($res))
{
	echo '<li><a href="http://www.openstreetmap.org/browse/way/'.intval($row['osm_id']).'">';
	echo htmlspecialchars($row['name']);
	echo "</a></li>\n";
}
echo "</ul>";

?>

How to get your own style

thanks to river we have a quite nice rendeing stack on willow. You can use the command line tool /home/project/o/s/m/osm/bin/render to test your own styles and if they're ready you can mail them to this list so we can configure renderd to render tiles for them.

To show the capabilities of the new hstore scheme in conjunction with the rendering system, I created a style that renders surveillance cctvs on a transparent overlay. It's a quite simple style but it shows how to do things: http://svn.toolserver.org/svnroot/p_osm/styles/surveillance/surveillance.xml

to render a map of germany in this style, just log on to willow and call

render --bbox 5.56,55.04,15.35,47.15 --style /home/mazder/styles/surveillance/surveillance.xml --size 2000x2000

it will create a map.png if the current directory. Call /home/project/o/s/m/osm/bin/render --help to learn about the possible parameters.

You'll want to add /home/project/o/s/m/osm/bin/ to your PATH in your .bash_profile so you can just all render without a path.

I also added the style to the renderd config and you can take a look at it under http://toolserver.org/~osm/styles/?zoom=12&lat=52.50546&lon=13.42461&layers=00TB0 The style is not perfect, the icons are too big but hey - its just a sample.

test the style on your own machine

Toolserver uses osm2pgsql with default.style and hstore to import planet.osm into the database. So it's easiest to use the same setting on your local machine.

adapt the style

Change the location of entities to
<!ENTITY % entities SYSTEM "/home/project/o/s/m/osm/data/mapnik/inc/entities.xml.inc">
Change the prefix of the databases from planet_osm (e.g. planet_osm_line) to &planet; (e.g. &planet;_line)

test your style on toolserver

Use the render-command (as explained above) to test your style on toolserver.

announce on the list

Write a mail to the list maps-l@lists.wikimedia.org. Describe your style and why it should be rendered.

deploy on the server

this is only possible for members of the OSM-MMP, all others should send a mail to maps-l@lists.wikimedia.org

gaining priveleges

To deploy a style, you need to log on to ptolemy:

yourpc:~$ ssh username@login.toolserver.org
username@nightshade:~$ ssh ptolemy
username@ptolemy:~$ become osm
osm@ptolemy:~$

now you're on ptolemy, the render-toolserver, with the privileges of the OSM MMP. Everything of interest is now in your homedir:

~/bboxes - various bounding boxes, commonly used with the tirex-batch or the render-tool
~/bin - common tools (included in $PATH)

~/tirex - tirex installation
~/tirex/etc/tirex - tirex configuration
~/tirex/var/log - tirex log files
~/tirex/bin - tirex tools (included in $PATH)

~/data - when downloading osm-related data (planet files, extracts, etc.), place them here
~/data/planet - the planet files are updated once a week, they can be accessed from everywhere on the ts-cluster as /home/project/o/s/m/osm/data/planet/planet-latest.osm.bz2

~/p_osm - a checkout of https://svn.toolserver.org/svnroot/p_osm/
~/public_html - what's currently online on http://toolserver.org/~osm/styles/
~/src - when you're about to compile osm releated tools, place the sourcecode here

~/renderd - old renderd installation, not longer used
~/tools - various tools; some old, some updated; some useful, some not...
~/trash - a trash can. instead of rm -rf, move things here

testing the style

All user-styles are placed in ~/p_osm/styles, one per directory.

New styles are usually located in another users home directory, so first copy them to ~/p_osm/styles. Next, test the style:

the directory should contain an CONTACT and an LICENSE file like this: https://svn.toolserver.org/svnroot/p_osm/styles/hikebike/
the xml should be valid (test with xmllint -noout path/to.xml)
make some test-renderings
- render --bbox $BBOX_WORLD --file world --style path/to.xml
- render --bbox $BBOX_EUROPE --file europe --style path/to.xml
- render --bbox $BBOX_GERMANY --file germany --style path/to.xml
- render --bbox $BBOX_BERLIN --file berlin --style path/to.xml
- render --bbox $BBOX_MAINZ --file mainz --style path/to.xml

If all tests succeed in reasonable time and with good looking results, process to the next step. If not, ask the author to fix the issue.

install the style

Add the style using svn add and commit the change to the svn repository, placing a link to the discussion thread about the style on http://lists.wikimedia.org/pipermail/maps-l/ into the commit comment:

svn add ~/p_osm/styles/stylename
svn commit ~/p_osm/styles/stylename -m "new style as discussed in http://lists.wikimedia.org/pipermail/maps-l/...

Now create the tirex configuration file and edit it:

cp ~/tirex/etc/tirex/renderer/mapnik/template ~/tirex/etc/tirex/renderer/mapnik/stylename.conf

Don't forget to create the directory where the tiles will be placed:

mkdir /osm/tiles/stylename

Next restart the tirex system and take a look at the job-log:

~/bin/stop-tirex
~/bin/check-tirex
tail -f ~/tirex/var/log/jobs.log

Now, get a coffee and wait ... 15 minutes or more. The problem is, that the tirex-backend-manager needs around 5 minutes to load all mapnik xmls. The tirex-master is not willing to wait that long and sends render requests while the backend is still loading. The best thing is to wait until the render-requests time out and get retried.

If you are impatient, you may want to restart the master (and only the master!) when you're sure, the backend is up:

~/bin/stop-tirex-master
~/bin/check-tirex-master

General information about the tirex status can be obtained by running

~/tirex/bin/tirex-status --short

Category:Documentation