Talk:Map improvements 2018/Flow

Leave your thoughts and ideas here about the "Map improvements 2018" project.

People watching this page may want to look at m:Community Wishlist Survey 2019/Maps/Maps Improvements: Vector Structure, Disputed Borders, Cleaner Style and related wishlist items.

The wishlist uses approval voting (vote to support any/all wishes that you like), so "oppose votes" are pointless, but useful information, examples, or explanations of problems would be very much appreciated.

Most of the other map-related wishes can be found at m:Community Wishlist Survey 2019/Maps, although at least one (the m:Community Wishlist Survey 2019/Multimedia and Commons/Easy, low-maintenance possibility to include color coded maps) seems to be in a separate group. There are about four days left to vote for the things you would like to see.

Following some discussions here a few months ago, we've now agreed in placing the new map onto cywiki. Can someone let me know which templates need copying over and any other instructions, please? It will be placed within our new Wikidata infobox for all villages, towns, cities...

Wikidata infobox - https://cy.wikipedia.org/wiki/Nodyn:Gwybodlen_llefydd

You may want to see the implementation in https://commons.wikimedia.org/wiki/Template:Wikidata_Infobox - which uses https://commons.wikimedia.org/wiki/Template:Mapframe/Wikidata and https://commons.wikimedia.org/wiki/Template:AutoMapZoom in particular.

Hi Mike. I had a good look at WD Infobox. Our WD infobox (link above) will be enabled, however, as it contains items related to cywiki eg includes names of MPs, AMs. If you can enable the Wikimedia Map onto this infobox, I would be most grateful. I think I've copied most templates, but I'm hitting my head against that wiki-wall! Thanks and best regards.... Llywelyn2000 (talk) 17:49, 13 October 2018 (UTC)

All sorted! Llywelyn2000 (talk) 11:30, 15 October 2018 (UTC)

How frequently are the osm-intl tiles rerendered to reflect changes to OpenStreetMap? Map improvements 2018#April 18, 2018, Special Update on Map Internationalization says it currently takes up to 48 hours, which I take to mean the time it takes for the extract or planet diff to be pulled in, as opposed to the tile rendering schedule. A change I made 15 days ago, correcting the name of a major city in Vietnam, appears at z10 and above but not at z9 or below. Is there a way to force-update a particular tile, similar to appending /dirty to URLs on the OpenStreetMap tile server?

Sorry for the late answer (vacation). The process is as follow:

1) upgrade our local copy of OSM (postgresql database) every day (we are planning to increase this frequency)

2) regenerate vector tiles for any tile that has changed, triggered by the above update, but can take a few hours to complete 3) raster tiles are then generated on request, from the vector tiles and cached for up to 24h

There is no public ways to trigger steps 1) and 2). You can always bypass the cache by adding a dummy parameter to the URL to test if the issue is in 3).

All that being said, an update not showing after 15 days definitely sounds like a bug on our side! Could you please open a ticket on phabricator and tag it with "Maps"? Thanks!

We follow the standard practice of only regenerating high-zoom tiles based on updates. I've created task T194787 about regenerating low-zoom tiles.

I also ran a manual run to regenerate z0-z9 in both data centers, which should result in those tiles being fixed.

Even with a dummy parameter, I'm still not seeing the update as a result of the manual run.

Interesting question @Mxn. I'm pinging @Sbisson (WMF), who may be able to help you with an answer.

It's been over a month since the tiles were manually regenerated, but the edit still doesn't show up at z9. Would you mind taking another look at this, Pnorman? Thanks!

This is mostly fixed now, but this tile stubbornly refuses to update.

As my contract with WMF has finished, I'm not responsible for this anymore, and without the new style being deployed, the updated scripts aren't being redone. I suggest you open a phab ticket so the issue doesn't get dropped.

Hello @Mxn, we have worked on this issue for a while, as you can see here:

- https://phabricator.wikimedia.org/T194787

We are also working on some related tasks considering caching improvements, which I believe is the issue now:

- https://phabricator.wikimedia.org/T159977

I would reinforce what Pnorman said and ask you to create a phab ticket with the issue and detailed descriptions so we can track it.

Thank you!

Thanks, I opened phab:T202201 to track this specific case.

I tried the new features in testwiki and they are really cool! While trying and after reading the comments below, a simple question arises:

Is there any chance for, default or per wiki, arbitrary language fallbacks? As an example: english are not related with greek language. But greek readers when greek labels are missing on a map of China, would prefer to read english labels rather than chinese labels.

lang="en" would not be a solution as all labels would be in english.

To my understanding the feature exists, so it is only a matter of choice (or policy?).

Thanks for the question @Geraki. Would you want Greek maps in general to fall back on English? That could be done pretty easily, though it might require some kind of community consensus.

Know, though, that the fallback list for maps is separate from the general language fallback list for the wikis, which governs things like what languages interface messages get shown in. At the moment, the lists have the same fallback values. But my point is you could change the map-specific list without changing the main language fallback list. So changing the map list is probably not too hard.

Yes, if the technical team is ok with that, the community consensus will be certain. I will raise the matter for discussion on elwiki.

When you have a consensus, we will be happy to make the change.

At that time, if you know how, please fill out a Phabricator ticket and tag it to "Collaboration-Feature-Rollouts (Collaboration-Maps)". Or you can ping me.

The question of how to change fallbacks is bound to come up from other volunteers and languages. Perhaps @CKoerner (WMF) can comment on what process he'd recommend for such a change request (knowing, as I say above, that this is a map-specific change and would NOT change the general language fallbacks)?

The issue could be a bit more complex. For example, a Luxembourger will want to see name:fr over name:de in France and Luxembourg, but not in Germany, and not everywhere in Belgium.

Hi @Stereo. Yes, these issues can get tricky. There is good info and strategies on this in my posts about Should your wiki change its map fallback language? and also OSM name data quirks and the uses of lang="local".

The Map Improvements 2018 was always meant to have a hard stop at the end of June, 2018.  Now, having accomplished the project’s main goals, the Collaboration team has moved on to other work (and taken a new name, Growth). This project has concluded and this page will no longer be updated or monitored.

Members of the Reading Engineering team will handle map bugs and maintenance going forward. To read about or discuss their activities, please see the Wikimedia Maps project page.

Hi. I've put T198235 on Phabricator, but I'm posting this here as well because I think it wouldn't be very difficult to fix. The issue is that route shields containing numbers only appear to be distorted, probably because someone changed the font size and character spacing. These displayed fine all through last year. (Images are on the Phabricator task.) Could someone check if this is intentional? Thanks.

Hi @Jc86035. This project is concluded (I'll post a note to that effect today). The main Maps project team will look at your ticket. To new location for all maps discussion is the Wikimedia Maps page.

Is there a plan to include the zoom and pan capabilities within mapframe? At present they only appear in preview mode, which seems a bit pointless. Similarly the really helpful scale bar also vanishes when I publish changes. Are they likely to make it into the final vesion?

Hi Robin. Best would be if you could include a URL, so we can have a look at your specific issue.

But in general, I think what you're describing is this: most Wikipedias are configured so that the mapframe map embedded on the page is not dynamic. It's a graphic, which means you can't pan or zoom it, as you say. I think this is done to reduce page load times (or server load?). To get pan and zoom, you click on the map, which pops up the dynamic version. On Wikivoyages, by contrast, the dynamic version is embedded in the page. In theory we could reconfigure your wiki to show dynamic maps, but I don't know what the impact would be, especially if it's one of the larger wikis. Perhaps @Catrope might know?

The missing map scale gauge is something I hadn't noticed before. Thanks for pointing it out. I can clearly see what you mean on these test pages: testwiki is configured for static maps (and shows no scale markers) while test2 wiki is configured for dynamic mapframes (and has the scale). I can't think of a reason why we wouldn't want to display this useful informational item, so I've created a ticket for this bug.

I was about to ask whether zoom and latitude/longitude values could be accessed while in editing preview in en:mapframe, and discovered that it already works on a right click. Great work ... now the interactive mode all makes sense to me. Really useful feature for setting up a map, which probably needs talking up in the documentation. Thanks.

>...discovered that it already works on a right click.

I'd love to document the feature you're referring to but I'm not following. Can you take me through step-by-step? Are you talking about using Visual Editor with maps, perhaps?

No, it is within the normal traditional editing preview. If I have understood phabricator correctly, it is the feature described at https://phabricator.wikimedia.org/T129875 to 'Show current coordinates & zoom in edit preview & VE mode'

The value lies in the fact that you might add a mapframe knowing just the coordinates of a feature. However, that simply plonks the map centred on that feature. You can then preview the map, and then adjust the zoom and position to show the segment of map that best displays what is wanted, and then right click in the centre of the frame and copy the coordinate details, ready to paste back into the template.

Yes, the ability to refine map placement interactively is a really nice feature of the Visual Editor maps interface. Thanks for mentioning that. I'm pinging @CKoerner (WMF) to bring this idea to his attention, since he is writing the VE section of the maps help.

There are complaints on bgwiki that these static maps (i.e. you have to click to be able to zoom in/out) are inconvenient. We previously had a system with static switchable maps of different scale—settlement, region, country—that was apparently intuitive enough as the maps were arranged as clickable tabs (example). So there are now questions whether we should go back to that system. Indeed, it isn't too obvious that you can actually click on the new maps. Considering that bgwiki is not too large, is it possible to have the dynamic maps embedded as on Wikivoyages? @JMatazzoni (WMF), @Roan Kattouw (WMF)?

Sure, that'd be fine. We only really need static maps on large wikis, but as a first step we just enabled them on all Wikipedias. I'll make the bgwiki maps dynamic, and then I'll see if I can figure out a way to find out which wikis are large and small for this purpose that's more scientific than looking at who complains on this talk page :)

Wow, that's great, thank you so much, Roan! My colleagues will surely be very happy :)

This is done now. However existing maps won't update immediately, they'll only change from static to dynamic when the page is edited or purged. For testing, I purged the Кожани article, so you can see what the new dynamic maps look like there.

Perfect! The maps are much more convenient and useful now. Many thanks for the quick reaction again, Roan!

Great to see the publicity around this project.

OSM has a long-standing community consensus that transliteration does not belong in the OSM database: https://wiki.openstreetmap.org/wiki/Names#Avoid_transliteration . There are concerns that this project could encourage people to add transliterated names.

Could you clarify your instructional/publicity materials to emphasise this?

Alternatives are: adding a Wikidata id to the OSM object, so it can be crossreferenced to a set of transliterated names in Wikidata; or machine-generated on-the-fly transliteration (this has been successfully carried out by the German OSM community for many years now).

Hey IP, could you help me understand the concerns a little better?

We've been telling folks that if there isn't a label in their local language on Wikimedia Maps to go to OSM to add a label in their local language (like "name:fr=Foo"). No encouragement in either direction regarding bulk automation. Which, if I'm reading the linked article correctly is the largest concern? How might we better update our documentation here on MW.org to make it more apparent that transliteration is a nuanced thing in the OSM community? It's a wiki, feel free to edit directly or make a suggestion here. :)

The Wikimedia Foundation is not doing any active promotion of maps as the project is winding down and entering ongoing maintenance (no new features for the time being). In fact, in short time this page will be marked as historical and ongoing discussions will encouraged to take place on Wikimedia Maps' talk page. I say this to alleviate any immediate feelings of concern and suggest that we perhaps work on adding documentation to that page going forward.

Edit: I just now saw the comments on the blog post. That provides more context, but I am still happy to have more feedback.

The comments on the post put it well. Thanks for your understanding.

My personal opinion is that data should be useful. OpenStreetMap is open source and I am totally in favour to have all the names available as they are known in any language. This allows us to provide maps we need in all our Wikipedias. When Wikidata and OpenStreetMap are linked on the object level, then it is irrelevant where useful labels are kept. You can avoid transliteration as much as you like and we still have the functionality that is required. That is the beauty of collaboration and that is the beauty of open source. Thanks

Hi Gerard, that's fine, but your personal opinion doesn't trump OSM best practice when it comes to entering data to OSM. It's very likely that OSM contributors will revert widespread additions of transliterated names. Encouraging people to add data which will only be reverted doesn't help anyone.

OSM wouldn't be so irresponsible as to ask thousands of its contributors to break Wikipedia rules. Please don't encourage thousands of your contributors to break OSM rules. Thank you.

Hoi, the point of this best practice is that it served you well. You want at least a situation that is as good as you have it now. What you have to consider is what use Wikimedia will make of your data. It is likely to become the single biggest usage of OSM. What I want is to consider options, see how we can make things work. On your servers on ours, the point should be that we play nicely together. I am not interested in a public conversation with you, I am interested in talking about the needs that I perceive. When you care to hear from me as much as I like to hear from you, I am sure you will contact me.

> It is likely to become the single biggest usage of OSM

Given that Facebook, Apple, Snapchat, and many others use OSM, I doubt it. Please have a little more humility and respect for the settled opinion of the OSM community rather than trying to bully us with your self-perceived importance.

Gerard, did you read the link that the IP user included in their first post? Here it is again: osmwiki:names#Avoid transliteration. The most relevant part is:

As well, many transliterated names frequently have been imported as is from Wikipedia (or Wikidata) for naming their articles, but the names may have been chosen quite arbitrarily on these wikis. It's not needed to import these transliterated names in OSM: we can just link to a single Wikidata entry or a single Wikipedia article in a single language, preferably the main language used locally, in order to find the other articles.

So data *is* "being useful": the purely geographic and *on-the-ground verifiable* data is being useful in the OSM database, and the encyclopedic and otherwise cultural relations are being useful in Wikidata.

Arlo Barnes, you are not saying anything that was not said before. You do not add a thing and thereby make things worse for your point. Talk to me privately, not publicly is what I said. I will not engage in any argument in this way as it is not productive. I only get a repetition that does not improve the argument.

Hi. Just a quick note to mention that the amount of maps being shown on Commons (as opposed to being linked to) is increasing at the moment as I roll out commons:Template:Wikidata Infobox, for example see: https://commons.wikimedia.org/wiki/Category:Five_hundred_meter_Aperture_Spherical_Telescope I'm hoping that doesn't cause any performance issues. It might be an interesting test case for multilingual maps (do they auto-change based on the language setting on commons?), and if by any chance you fancy working on improving the integration with Wikidata then that would be appreciated (at the moment we're using a lua/templateparser hack, and it is lacking features like auto-determining the zoom level to use)!

Hi @Mike Peel. I'm glad you asked about using mapframe in templates. I think this will be a really powerful use case for internationalized map. The answer to your question is that the maps will display in the content language of whatever wiki they are on. (Or they will attempt to, subject to availability of multilingual name data in OpenStreetMap for the language and location.)

You ask about the 'language setting on commons"; that would be the interface language preference, I assume. Since maps are content, they conform to the wiki's content setting, not the interface language preference. I hope that answers your question.

Thanks! By the "language setting on commons", I meant the language selector that's at the top of the page (here also), to the left of the username. If you change that on Commons, then the infobox changes to show info in the chosen language - will the map do that as well?

> If you change that on Commons, then the infobox changes to show info in the chosen language

Ahhh. I didn't know about that Commons-specific feature. I think the answer is that the map won't change on Commons in this way. Although the language selector changes the display in the way you have pointed out, the underlying content language of Commons remains English, and I'm told the map will respect that. (Also, I seem to have misunderstood the purpose of your infobox: I thought it was being stored on commons for use on articles (about telescopes) on different wikis. But I see now that this info box is meant to live on Commons.)

OK. Out of curiosity, what would happen if we fed the lang parameter with the reader's language choice? That's possible with Lua. It might confuse the system/caching though?

Yes, the infobox is commons-specific, it's there to provide basic background info on all topics in all languages. It exclusively uses info from Wikidata. It would work easily enough on other wikis if anyone wants to do that, though.

I suspect there would be caching issues with that, but if you're interested you could try it out on testwiki and see what happens (or test it on Commons once internationalized maps have been rolled out there).

On the client side caching shouldn't be an issue, because of the different query parameter values. Query parameters can even be used perfectly fine for cache busting...

On the backend I assume Cassandra cached tiles are separated as different Kartotherian sources, Vanish does caching based on the entire URL just as web browsers.

Yes, all those things will be fine. The caching issues I'm worried about are in MediaWiki's parser cache. That might not respond well to a mapframe tag with a dynamic lang attribute, and might try to reuse cached HTML that was generated for a different language.

I hadn't spotted that the multi-language support had been switched on until now ... I've added language-switching into the sandbox, which is currently demo'd at https://commons.wikimedia.org/wiki/Category:Lovell_Telescope (try switching to Russian!).

I can roll that out to the main version - as long as it won't break Commons?

@Mike Peel ooh, this is cool, and it works with ULS switching (as well as ?uselang=xx parameter) as well. Super cool!

So, we actively chose against making map language follow user (interface) language, for various technical and product reasons, but -- and @JMatazzoni (WMF) and @Roan Kattouw (WMF) can correct me if I'm forgetting something -- that is more relevant for language-specific wikis. It might actually not be as relevant set of issues for a multilingual wiki like Commons.

I think that if that usage stays on Commons only, it should be safe. Just to verify (I remember talking through some of the technical limitations with the parser) @Roan Kattouw (WMF) am I missing something? The fact that switching interface language seems to display the map with the interface language seems to suggest that technical limitation is not as limiting on Commons.

@Mike Peel that should be fine. The infobox already localizes itself to the user language, so whatever parser cache problems that causes are already there and won't be made worse by this.

OK, thanks for the reassurances, it's now live on around 600,000 commons categories!

Is there a page for requesting new functions to the map functionality. Specifically the one one use on Wikivoyage when you click on a listing number? Or is this the page. What I am looking for is the possibility to see my current position. As this is a travel site, mobile usage is important.

Hi @Traveler100. This project, Map Improvements 2018, is coming to an end, so this page isn't a good place. Here is the new page for discussing Wikimedia maps developments. But in all honesty, I don't think that developers there will be taking requests for a lot of new features. You can also make such requests in Phabricator. Here is the workboard with all tasks tagged to "Maps," which is the tag you'd want to use.