Email or username:

Password:

Forgot your password?
Top-level
Ilya Zverev

I dabbled in creating map styles like ten years ago, with Mapnik and TileMill. For our local cycling community I made a style that looks excellent printed in black&white, on which we could overlay routes. It was called like the community: "Veloroad".

Trained eye could notice multiple things on it that even modern styles don't do: look at road and river names, station symbols, place visibility.

There is a working tile server, but super slow. Probably gonna take it down.

osmz.ru/veloroad.html#7/58.374

Veloroad map style, centered on Viljandi, Estonia, zoom ~12. Visible are black and grey lines for roads, road and lake names, a railway in the style of soviet military maps. Some labels are in Russian.
66 comments
Ilya Zverev replied to Ilya

Obviously running command-line tools to produce a map is not ideal for non-GIS people. So I also made a web UI to prepare route maps.

You just upload a GPX trace, adjust the printing bounds, add a scale bar if needed, and download a PDF. Another online tool would merge several PDFs into a booklet.

It's actually a set of tools underneath, some fix mapnik's SVG issues, some preprocess the values. I know of at least one business spawned from this (not mine obviously).

Screenshot of "Get Veloroad" page. To the right is a map with a blue bounding box. To the left, a set of settings for a printed maps. These include paper format, bounding box adjustment, trace file upload, scale bar, map style, and an output image format.
TheStroyer replied to Ilya

@zverik would you have preferred it would be your own company to make a product out of it? Or do you want to have a different company based on OSM?

Ilya Zverev replied to TheStroyer

@TheStroyer I don't want to run a company based on a single product, because I don't have an attention span and management skills for that :) I'm pretty fine with my tools used for business, and with this one, I helped set them up actually.

TheStroyer replied to Ilya

@zverik ah yes short attention span. That's why you have so many different projects 😂
But seriously, manh of these projects are really great. They just needs l have started just one project in OSM. And it's probably less complex than many of your projects

Ilya Zverev replied to Ilya

Ten years ago everybody made maps with Mapnik. There were several scripts that simplified the task, but all had issues. So I wrote another.

Nik4 makes preparing a map image easy. No calculations needed, just choose an area, put physical or pixel dimensions, and you got the exact map you expected.

github.com/Zverik/Nik4

There were many little features inside that removed hassles. Like partial rendering, scale factors, ppi and route fitting. And fixing cairo's SVG output.

Ilya Zverev replied to Ilya

Of course another question that was heard from every business, "how do we get the tiles". Mapnik had a sample python script packaged, but it was slow and hard to use. I'm pretty good at APIs, so I made my own.

Polytiles can render regular raster tiles for a given bbox in multiple threads. Obviously. It also can use a polygon file or a relation from OSM, it can render retina tiles, generate an mbtiles file, or just produce multiple lists for other tools.

github.com/Zverik/polytiles

Ilya Zverev replied to Ilya

Obviously setting up Mapnik locally is hard, and people wanted to download OSM tiles with a proper UI.

In 2011 I took over the development of JTileDownloader, which does exactly that. Bulk downloading was frowned upon even then. I tried to ease the load: imposed a 5 tiles/sec limit and altered downloading order to meta-tiles (8x8 tile packs).

wiki.openstreetmap.org/wiki/JT

Still, @osm_tech don't care, JTD was banned by user-agent, and people returned to anonymous bulk downloaders with no safeguards.

Obviously setting up Mapnik locally is hard, and people wanted to download OSM tiles with a proper UI.

In 2011 I took over the development of JTileDownloader, which does exactly that. Bulk downloading was frowned upon even then. I tried to ease the load: imposed a 5 tiles/sec limit and altered downloading order to meta-tiles (8x8 tile packs).

JTileDownloader java app screenshot with two panels. To the left, a pane with dozens of options, including bounding boxes, GPX file, zoom levels, tile server, and a button to download tiles. To the right, a slippy map with a box over Lithuania selected.
Ilya Zverev replied to Ilya

And back to my projects from days back, that still operate. In the same trend, seeing something and improving on it, I took Frederik's BigMap and added an online tile-stitching service.

bigmap.osmz.ru/

The UI is the same, still bad, but it's the easiest way to get a raster map for an area. And also georeferencing files for OziExplorer or QGIS. Many people use it daily.

I see how it could be made better, faster and easier, but as always, not enough time. And no alternatives haha.

Screenshot of a map, on which is overlaid a block with many controls. It says "Map is 4x3 tiles (1024x768 px) at zoom 10, aspect 1.33:1" on top. There are three 4-directional buttons: Expand, Shift, and Shrink. And four options to zoom in or out. Below there are links to download georeferencing files, perl or python scripts, or put the job into a queue.
Rihards Olups replied to Ilya

@zverik Have used this one (grabbing the generated Perl script) - thank you for this tool :)

How is georeferencing done?

Ilya Zverev replied to Rihards

@richlv Thanks for using it! Georeferencing is simple — it's tiles, we know exact locations of all corners :) Just download a WLD file and put it near the image.

Rihards Olups replied to Ilya

@zverik Ah, is this a usecase like "download tile files locally and use them as a georeferenced layer"?

I'm only doing OSM in an amateurish way, thus these GIS things are another world for me =)

Ilya Zverev replied to Rihards

@richlv Yes, exactly. A good alternative to adding a live OSM layer and re-downloading tiles every time something moves.

Rihards Olups replied to Ilya

@zverik Neat, thank you for the explanation. Maybe one day I'll find time to dig more into these topics :)

Ilya Zverev replied to Ilya

When belgian OSM guys made the Welcome Tool in 2016, I set up an instance for Russia and started welcoming people.

I sent a few hundred messages, and after promoting the tool, some other mappers did too.

Alas the response rate was like one percent.

And when I got replies, sometimes I wished I hadn't.

But without welcoming messages, people in OSM feel themselves in a vacuum. There are still no links to communities anywhere on osm.org.

Here's the global WT instance: welcome.osm.be/

Screenshot of a "New OSM Contributors in Russia" web page. There's a short instruction on logging in, and a big button "Log in with OSM". The footer says, "An OpenStreetMap Russia project. Source code available. Powered by neis-one.org."
Eugene Alvin Villar 🇵🇭 replied to Ilya

@zverik maybe you have some suggestions on how to improve openstreetmap.org/communities ? One idea being floated around is to embed the OSM Community Index map somehow in that page. (The list of local chapters is already dynamically based on the index.)

#OpenStreetMap

Ilya Zverev replied to Eugene Alvin Villar 🇵🇭

@seav Start with a big bold link to the forum on the "Communities" page!

Generally I'd take the hint from the iD editor. It does a pretty good job of listing links after you upload a changeset.

Luis Villa replied to Ilya

@zverik ooof I feel that one. Have started making my first contributions in years (because I realized I can add bike racks) but it might as well be posting into the void.

Ilya Zverev replied to Ilya

Today, brownouts start for OAuth1a endpoints. In a month, only OAuth2 can be used.

One of my services using OAuth1 is OSM Authentication Proxy. Since storing passwords is bad, and OAuth sometimes is too hard to implement (you need a browser!), I made this proxy.

Essentially it's a website that says "yes, the person with the token is indeed an OSM user". It doesn't upload anything to OSM API.

I thought of using it for command-line tools and plugins, but ended up not.

auth.osmz.ru/

A screenshot of a logged-in website "OpenStreetMap Authentication Proxy" for user Zverik. It shows a lengthy description of the service. The master token is hidden, while three temporary tokens are listed.
Ilya Zverev replied to Ilya

Command-line applications have a hard time with OAuth, because it requires a browser. The auth flow has so many steps, it was much easier to just give up and use Basic Auth with a password.

Taking a hint from a Google's library, I made cli-oauth2: a Python library that makes authenticating a one-line task.

github.com/Zverik/cli-oauth2

Fun fact: the idea came when I was rewriting Level0 for command-line. Didn't get it to work (yet haha), but there's one less problem to solve.

Ilya Zverev replied to Ilya

This one came up yesterday, while preparing for @organicmaps GSoC student work.

When developing an editor, you need to work with OSM Sandbox, a dev copy of API with no data. Sometimes it's frustrating because complex data is where an editor breaks. Usually I downloaded an OSM extract in JOSM, fixed ids, and uploaded it to the Sandbox, but on the 100th time I gave up.

github.com/Zverik/osm_to_sandb

This script helps you transfer a bbox worth of data in two commands, one of which is pip install.

This one came up yesterday, while preparing for @organicmaps GSoC student work.

When developing an editor, you need to work with OSM Sandbox, a dev copy of API with no data. Sometimes it's frustrating because complex data is where an editor breaks. Usually I downloaded an OSM extract in JOSM, fixed ids, and uploaded it to the Sandbox, but on the 100th time I gave up.

Organic Maps replied to Ilya

@zverik Does the script also change IDs? What does "fix ids" mean? For readers' info, Organic Maps Editor doesn't store IDs and doesn't rely on them.

InsertUser replied to Organic

@organicmaps
I'd guess it flips them to be negative and resets any version numbers so that the dev server recognises them as new and assigns IDs to them rather than rejecting them as unknown.
@zverik

Ilya Zverev replied to Organic

@organicmaps To upload objects to the sandbox, you need to give them negative ids and keep the references. This is not related to OM, just a low-level API stuff.

Ilya Zverev replied to Ilya

Okay I skipped three days, so here are three tweets on a big thing I did at Maps.Me. It's on the deep end of the iceberg.

In short, I was tasked with bringing subway networks to the app. Which was based on OSM data, so naturally I thought how to fix the data model: every city had its own tagging schema.

To monitor the process, I wrote the validator, which now powers @organicmaps :

cdn.organicmaps.app/subway/ind

Alexey later improved it, a lot. Now there are over 300 cities, all uniformly mapped!

Okay I skipped three days, so here are three tweets on a big thing I did at Maps.Me. It's on the deep end of the iceberg.

In short, I was tasked with bringing subway networks to the app. Which was based on OSM data, so naturally I thought how to fix the data model: every city had its own tagging schema.

"Subway Validation Results. 305 of 308 networks validated without errors. To make a network validate successfully please follow the (link) metro mapping instructions. Commit your changes to the OSM and then check back to the updated validation results after the next validation cycle, please. See (link) the validation instance description for the schedule and capabilities". There is also a map link, and a long list of countries, mostly in green, each having 1 to 15 cities with subways.
Ilya Zverev replied to Ilya

It was made possible with planning, iterating, and preparing a detailed tagging schema that incorporated most tagging practices:

wiki.openstreetmap.org/wiki/Me

It started as a proposal, which failed the voting. Then I made another proposal, but didn't have the nerve to go through voting the 2nd time.

But also I noticed how most objections come from just a few people in EU, so I just started tidying up networks in many cities, and when we passed 80 (with help of other OSMers), it became the standard.

It was made possible with planning, iterating, and preparing a detailed tagging schema that incorporated most tagging practices:

wiki.openstreetmap.org/wiki/Me

It started as a proposal, which failed the voting. Then I made another proposal, but didn't have the nerve to go through voting the 2nd time.

Hand-drawn schema of a complex subway station. There are two platforms with two intersecting lines. Each platform has overground exits, and some escalators.
Ilya Zverev replied to Ilya

At that point I was ready to tackle the general public transport schema. After all, PTv2 is impossible to route over and a pain to use.

So, with experience in processing PT data for routing and validation, I generalized it for all kinds of transport, and presented at @sotm in Milan:

wiki.openstreetmap.org/wiki/Pr

Technically it's flawless. It has full backward compatibility, easy to map, tagging is pretty and conscise.

But at that point I got burnt out at my job and couldn't make the final step.

At that point I was ready to tackle the general public transport schema. After all, PTv2 is impossible to route over and a pain to use.

So, with experience in processing PT data for routing and validation, I generalized it for all kinds of transport, and presented at @sotm in Milan:

wiki.openstreetmap.org/wiki/Pr

"Stop Areas and Groups": three pictures denoting correct and incorrect ways of grouping stop elements into stop_area relations.
Ilya Zverev replied to Ilya

(should mention that there are TONS of nuances that don't fit into tweets.)

Two main issues with the proposal that don't make it a perfect one:

- It's too wordy, nobody would read it, despite the core idea is very simple (stop nodes).

- It challenges the current tram stop mapping (and nothing else).

The new PT schema should've been perfect in 2018 to be approved, and now it should be double-perfect, otherwise it won't pass any discussion. Everybody in OSM has their own opinion on PT mapping.

Ilya Zverev replied to Ilya

Keeping an up-to-date rendering database means relying on worldwide replication streams from osm.org or per-country streams from @geofabrik . But what if I want my own extract area?

In 2014 I dug into osm2pgsql's "slim" tables, and wrote a script to trim the data outside a given polygon:

github.com/Zverik/regional

I use it for my "veloroad" tile server, but not sure if it works: the database grows pretty fast. Maybe it's just because people are mapping more than ten years ago.

Ilya Zverev replied to Ilya

This one is new. Couple days ago I released @everydoor 5.0, and for it I made a new thing.

GeoScribble takes inspiration from OpenStreetBugs, which was where we put our OSM notes before 2013. Just a simple server with no auth and a few fast endpoints.

wiki.openstreetmap.org/wiki/Ge

The idea is, you draw lines and leave short notes on the map. Just like with walking papers.

I see it already being used in multiple countries. The layer can be added to @josmeditor and Rapid, or browsed online.

This one is new. Couple days ago I released @everydoor 5.0, and for it I made a new thing.

GeoScribble takes inspiration from OpenStreetBugs, which was where we put our OSM notes before 2013. Just a simple server with no auth and a few fast endpoints.

wiki.openstreetmap.org/wiki/Ge

The idea is, you draw lines and leave short notes on the map. Just like with walking papers.

Map of an italian castle grounds. There are few red solid and dashed lines marking missing footways.
TheStroyer replied to Ilya

@zverik @everydoor @josmeditor wow, that looks like it can improve the editing workflow a lot

Ilya Zverev replied to TheStroyer

@TheStroyer @everydoor @josmeditor

Indeed! Just last Sunday I was walking with my friends in a forest, and surveyed a few streams and boardwalks this way. They would be lost if not for the drawing feature.

Editing ways properly (e.g. in @vespucci_editor ) needs too much involvement, compared to just a single finger movement in ED.

vespucci replied to Ilya

@zverik @TheStroyer @everydoor @josmeditor why draw? just record a GPX track ...

PS: in a forest you probably want to use a DTM anyway for tracing.

PPS: not to mention that you are massively overstating the effort to simply directly add things.

Ilya Zverev replied to Ilya

Since we're mentioning @josmeditor , here's one thing I made a long time ago:

josm.ru/

Yeah, that's a proper landing page (in Russian), with a few buttons, shortcuts to most important pages like a forum thread or a tag list, and a tutorial video.

I even browsed the changelog and added interesting changes to the feed, but stopped in 2020. I still use the page to download updates though.

"Start with JOSM! Download stable version 19067. Installers for Windows, Mac OS X, WebStart, jar file. Development version 19075." Also a few wiki buttons below, and a youtube video.
Ilya Zverev replied to Ilya

I'm a bit out of breath, but def not out of projects :) Gonna keep doing this until I get an answer from NLNet.

In 2010 Microsoft granted to OSM permission to trace Bing imagery. It was the only detailed imagery layer we had in Russia, so that enabled making a validator that found illegal imagery usage (google / yandex):

vakhtyor.osmz.ru/

Basically it watched where people uploaded buildings and checked for available Bing imagery.

In 2014 we got Mapbox layer, and I turned this off.

"OpenStreetMap Watchdog. This table lists mapped buildings in places where there's no detailed Bing imagery. <...> Many traced buildings might mean an import, drawing from memory, or — what we're trying to catch — tracing illegal imagery or proprietary maps."
Ilya Zverev replied to Ilya

There are different kinds of imports. Safest are those done by hand, but also the dirtiest, in my opinion: it's easy to miss or to get lazy and just press upload.

For towns and villages, the best source we had was GNS, an american database of toponyms. I converted it to OSM format and wrote a short manual on copying it onto the map:

gns.osmz.ru/

Interesting that while box=yes parameter doesn't work on osm.org nowadays, parameters minlat, maxlat etc still do.

GNS data converted to OSM format. Open in JOSM as a new layer, copy tags with Ctrl+C, then Ctrl+Shift+V onto a node placed on IRS imagery. Don't forget to fix the place type. <..> Made on 9.12.2015 with gns2osmr.pl script. Everything is in public domain.
SK53 replied to Ilya

@zverik Still loads of redundant GNS tags on place nodes in Ireland (probably imported by Dermot - mackerski). Until the townlands project got going they were still the major source of places in rural Ireland. Still a lot of dedup & other cleanup needed.

I also did a bit with them for Pakistan, but never could trust the data in a country where the population had increased so much : sk53-osm.blogspot.com/2011/03/ (probably should correct GNIS->GNS

Ilya Zverev replied to SK53

@SK53 yeah, what felt a good idea in 2011, not that much now :)

Ilya Zverev replied to Ilya

So, imports. When I was working at Maps.Me, many companies asked if we can put them on the map. So I made another thing.

OSM Conflator takes a data source and carefully merges it with OSM data, with a way to update it later.

wiki.openstreetmap.org/wiki/OS

While it can take a plain JSON with tags, it is most powerful with profiles: Python scripts that control the behaviour.

It can export a preview GeoJSON and has other ways of adhering to the guidelines. Alas, few people had enough skill to use it.

So, imports. When I was working at Maps.Me, many companies asked if we can put them on the map. So I made another thing.

OSM Conflator takes a data source and carefully merges it with OSM data, with a way to update it later.

wiki.openstreetmap.org/wiki/OS

While it can take a plain JSON with tags, it is most powerful with profiles: Python scripts that control the behaviour.

Ilya Zverev replied to Ilya

Having a GeoJSON to preview an import is good, but how to suggest amendments?

That's why I made CF Audit: an interactive website to go through changes one by one, cross-reference them, and fix anything wrong.

Those fixes are kept for consequent imports with OSM Conflator, so less work next time.

audit.osmz.ru/

I know RocketData iterated on it for their operations, and there's a new IFS fork by @sfkeller team: cfauditgui.geoh.infs.ch/

CF Audit screenshot. There are two maps to the right: OSM and Satellite, with markers for a location of a fuel station being imported. The text on the left says, "Update tags on a point. You can move the marker to a better location. Imagery may be misaligned.". Some tags are available for modification (e.g. brand and opening_hours), and there are three buttons: "Good", "Don't Change", and "Skip".
Ilya Zverev replied to Ilya

So I got the conflator, and I got the audit tool, and the entire pipeline as shown on the picture. Did I use it?

Oh did I even. 5k Walmart supermarkets, tens of thousands Shell fuel stations, ~10k addresses in Moscow, various restaurant and shop chains... It all worked perfectly.

Of course it was just a few imports. I envisioned an automated system for high volume of data, still controllable by OSMers, but nobody asked for that, and I didn't have time.

Still, mentioned it at SotM EU 2023.

A chart. External Dataset → Profile → OSM Conflator → Changet GeoJSON → cf_audit browse → cf_audit validate → Audit JSON → OSM Conflator (+Overpass API) → JOSM XML → JOSM → OSM.
Ilya Zverev replied to Ilya

OSM Cards is a postcrossing thing for OpenStreetMap members. You get a person's postal address, write them a few encouraging words on a postcard and add a 5-letter code. When received, they type those letters in a form, and the connection is set up.

I announced this website at SotM 2020 and got a few beautiful cards. I lived in Minsk back then and printed like 50 postcards with the city map to send out — and now idk what to do with them :)

Send me one if you like: osmcards.org/profile/bqh2on123

OSM Cards is a postcrossing thing for OpenStreetMap members. You get a person's postal address, write them a few encouraging words on a postcard and add a 5-letter code. When received, they type those letters in a form, and the connection is set up.

I announced this website at SotM 2020 and got a few beautiful cards. I lived in Minsk back then and printed like 50 postcards with the city map to send out — and now idk what to do with them :)

Amateur radio card with id "uz9osm".
Ilya Zverev replied to Ilya

You know OpenWhateverMap.

openwhatevermap.xyz/

Grant made the original version, then we lost the domain, and then I updated it with modern tiles and some interactivity: clicking on a tile reveals its name and URL.

Map of Lodz, Poland (where SotM EU 2024 is held). Every rectangular tile is in different style.
rugk replied to Ilya

@zverik
Hmm giving out tokens on a HTTP site does not sound secure, does it?

Ilya Zverev replied to rugk

@rugk yeah I decided not to set up certificates given nobody uses the service :)

Simon Poole replied to Ilya

"There are still no links to communities anywhere on osm.org." @zverik every time you use iD and then there's a whole very visible tab dedicated to the subject openstreetmap.org/communities

Ilya Zverev replied to Simon

@simon *every time you upload a change in iD — which is a pretty long way to a link.

On the tab there's a list of Local Chapters, essentially legal entities. Works for a business, too much for a person. I just want a link to a forum at least, or that localized panel from iD ideally, visible at all times on the main page.

Andy Allan replied to Ilya

@zverik @simon I intend to make much more use of the OCI integration into openstreetmap-website, but I'm juggling a lot of different things.

I would like it so that when you set your home location, you see the list of local communities (taken from OCI) on your dashboard.

OpenStreetMap Ops Team replied to Ilya

@zverik it is a tough one, yes there are some legitimate uses of bulk downloading, but most of the tiles are never viewed. If we accept bulk downloading it will have an negative impact on others seeing fresh tiles, which is a particular concern during vandalism. We should make a clearer metric for the served fresh:stale ratio. EU week daytime hours have a very poor ratio. With some development effort the tile service could have a way to be told the request is bulk, changing render/caching logic.

Go Up