Email or username:

Password:

Forgot your password?
Ilya Zverev

I've submitted another application for @NGIZero grant for @everydoor. I'm at a point when working a regular job feels like a loss to myself, OSM and OSS world. Looking for funding to focus on my multiple geo-related projects, but that's near impossible. As Guillaume told me at the last SotM, hiring me for what I do is impossible to justify. Mostly because it's not a single super-useful project. Like xz. So... you know, I keep writing good stuff on 4 hours a week I got.

139 comments
NLnet Labs

@zverik @everydoor We are NLnet Labs, who develop #OpenSource software for core Internet infrastructure, such as #DNS and #BGP. The NLnet Foundation, who fund projects, uses the @NGIZero account.

Ilya Zverev

@nlnetlabs Gah! Thank you, fixed the mention :) Also thanks for your work, I hear about you in the infrastructure context now and then, albeit not enough imo!

NLnet Labs

@zverik Most people know us from Unbound #DNS resolver, but we manage about 7 major #OpenSource projects with 16 full-time developers. github.com/NLnetLabs

Ilya Zverev

It's a bit hard to track indeed — like when I started listing all my conference talks, I got over 100 entries, all with different topics. When I tried listing my software for an about page, I barely fit the most prominent things in a screen and had to remove much.

So in this thread, I'll try mentioning geo-adjacent things I did in the past 13 years. One thing daily. As a self-recognition practice. That's gotta get me through a couple months at least.

Ilya Zverev

And yes, I'll start with the thing most of you know me for: @everydoor editor. Used by a thousand mappers monthly, started as a way to quickly collect POI, but on the way also became an ultimate micromapping and address collection tool. And soon to be a next-gen walking papers replacement.

every-door.app/

I should mention I also collected several thousands (!) of shops and amenities with it.

Thank you to everybody who's making edits with this app. So happy to see it used for good.

Screenshot of Every Door app. There's a map with colored dots, and a legend that lists "street lamp", "swing", "trash can", "seesaw", "playground" and other things.
tinderness

@zverik @everydoor Eine sehr schöne App, die ich schon mehrmals im Feld benutzt habe!

NiceMicro

@zverik @everydoor Ohh no, are you suggesting that there is a better way than taking pictures and editing the map in JOSM later?! 😂

Ilya Zverev

My previous map editor was... a messenger bot! I made it for finding local shops and businesses, because we had like half a thousand. And OSM didn't have that much, neither it had photos, so I made my own database and my own data collection pipeline — right inside the Telegram messenger! No interactive maps or geolocation, but surveying was fast and enjoyable.

I left Belarus many years ago, so alas it's not updated. But without it, there would be no Every Door.

github.com/Zverik/bot_na_rayon

My previous map editor was... a messenger bot! I made it for finding local shops and businesses, because we had like half a thousand. And OSM didn't have that much, neither it had photos, so I made my own database and my own data collection pipeline — right inside the Telegram messenger! No interactive maps or geolocation, but surveying was fast and enjoyable.

Three screenshots of the telegram bot. First shows an address and suggests sending an apt number or listing shops in the buildind. Second shows a map with cafes and a numbered list. Third one shows information for Jazz Coffee with buttons for editing.
Ilya Zverev

My focus on ground surveying started basically when I joined OSM. Among my first wiki pages was "Solo Mapping Party": thoroughly documented experience of drawing and filling in map sheets.

wiki.openstreetmap.org/wiki/RU

Seven years later I made a QGIS plugin for automatically forming walking papers of any area, with special care for mapping parties and making the map as big as possible.

openstreetmap.org/user/Zverik/

It resulted in my first pull-request to QGIS 3 — alas still not supported by the plugin.

My focus on ground surveying started basically when I joined OSM. Among my first wiki pages was "Solo Mapping Party": thoroughly documented experience of drawing and filling in map sheets.

wiki.openstreetmap.org/wiki/RU

Screenshot of QGIS 2.18 with an OSM map. A button for the walking papers plugin is pressed, and four menu items are visible: "Download OSM Data", "Open OSM Data", "Create Pie Layers", and "Prepare Atlas".
Ilya Zverev

Surveying from a car passenger seat is something people rarely do. They prefer to, idk, make photos for Mapillary and call this mapping.

Precise surveying from a car is virtually impossible: when moving at 90 kph, each second you move by 25 m, which is like a bridge length.

So in 2012 I developed a tool that saved the exact time you pressed a shortcut. I also made a JOSM plugin to precisely align the measurements with the imagery.

wiki.openstreetmap.org/wiki/Na

Found ten logs, but no OSM uploads.

Surveying from a car passenger seat is something people rarely do. They prefer to, idk, make photos for Mapillary and call this mapping.

Precise surveying from a car is virtually impossible: when moving at 90 kph, each second you move by 25 m, which is like a bridge length.

So in 2012 I developed a tool that saved the exact time you pressed a shortcut. I also made a JOSM plugin to precisely align the measurements with the imagery.

A screenshot of "NanoLog" program. There is a list of events starting with "let's assume we're driving and there's a milestone". And a list of keyboard shortcuts, like "F1: edge of a bridge; F2: town border; F4: Speed limit 40 kph". All timestamps have 1/100 of a second precision.
Luc

@zverik the biggest problem with highway mapping (since you mention 90 km/h) is interpolation for me: the GPS just doesn't update frequently enough to place the exact point where this emergency phone or camera exists

Does the JOSM plugin do that when you import the gpx track and timestamp?

Ilya Zverev

@luc You noticed the problem exactly: GPS often logs points just once a second. That's why NanoLog makes a separate list of timestamps. When paired with a GPS track, you use that only as a line with a few fixed points.

But NanoLog points, you can move back and forth with a centimeter precision, using visible bridges and road markings for georeferencing.

I frankly don't remember the state of the plugin. I think I got it to move the points, liked what I saw, and jumped to another project :)

Ilya Zverev

Speaking of JOSM plugins, I've made a few.

This is Relation Toolbox. It reverses how we work with relations: instead of editing relations themselves, we edit an object's membership.

With it, you choose a relation, and then click on nodes and ways (and other relations) to add or remove them from the chosen one.

It also introduced automatic multipolygon construction and de-construction.

wiki.openstreetmap.org/wiki/JO

Fun fact: my first ever SotM talk (in 2011) was on this plugin.

Speaking of JOSM plugins, I've made a few.

This is Relation Toolbox. It reverses how we work with relations: instead of editing relations themselves, we edit an object's membership.

With it, you choose a relation, and then click on nodes and ways (and other relations) to add or remove them from the chosen one.

Screenshot of the "Relation Toolbox" panel from JOSM. It shows six relations (restriction, route, disused:route). One is highlighted in yellow. There are "new", "multi", and "find" buttons.
Ilya Zverev

You know how most imagery is offset, and you need to align it with GPS traces and other stuff before mapping? No? Haha.

My another project from that time was the Imagery Offset Database. A huge list of reference points than helps align imagery automatically. Complete with a JOSM plugin and tons of documentation. Turned eleven last month, still very active.

offsets.textual.ru/

iD editor was written by americans with perfect imagery, so it still doesn't support it:

github.com/openstreetmap/iD/is

You know how most imagery is offset, and you need to align it with GPS traces and other stuff before mapping? No? Haha.

My another project from that time was the Imagery Offset Database. A huge list of reference points than helps align imagery automatically. Complete with a JOSM plugin and tons of documentation. Turned eleven last month, still very active.

A screenshot of Imagery Offset plugin that shows a single stored offset of 18 meters. It was created by me in 2013, and calculated 1.1 km away from the screen center. There are also checkboxes for "Calibration geometries" and "Deprecated offsets".
Ilya Zverev

The key to IODb's usefulness is collaboration: by using the stored offsets, you indirectly interact with other mappers to share the work.

Direct interaction was made possible with the GeoChat JOSM plugin. You see somebody having the same area open in JOSM — you say hi. Also helps avoid conflicts.

The protocol and the server are very simple (and hosted on OSM dev instance), I'm suprised it still works. Maybe somebody should remake it for modern age and mobile editors...

wiki.openstreetmap.org/wiki/JO

The key to IODb's usefulness is collaboration: by using the stored offsets, you indirectly interact with other mappers to share the work.

Direct interaction was made possible with the GeoChat JOSM plugin. You see somebody having the same area open in JOSM — you say hi. Also helps avoid conflicts.

The protocol and the server are very simple (and hosted on OSM dev instance), I'm suprised it still works. Maybe somebody should remake it for modern age and mobile editors...

GeoChat plugin panel in JOSM. There are two tabs: "Public" and "Debug_Man". A short interaction is visible:

Users mapping nearby: Alexandr_Zeinalov
[10:34] Zverik: This is a test
User Debug_Man is mapping nearby
[10:35] Debug_Man: Zverik: hi!
[10:35] Debug_Man: how're you?
Ilya Zverev

How much do you like mapping? Do you map every day? Do you WANT to map every day?

Inspired by other streak-tracking tools, from Pascal's HDYC to github, I have made a simple game: submit a changeset every day, and gain points. A bonus point if you do the task.

streak.osmz.ru/

The healthy thing would be to reach several hundred points and drop this. Please make breaks in your mapping! Tomorrow I'll show you how I did it.

Screenshot of OSM Streak. User Zverik is logged in and has 504 points (four stars). His task is to map a farmland. He has one changeset made today, he's updated a supermarket. He has 16 hours to submit a changeset.
Ilya Zverev

After two years of mapping, I noticed how OSM took over the major part of my life. So I've turned to programming, because technical solutions are the best, I mean, easiest.

"No more mapping" JOSM plugin, once installed, prevents JOSM from launching. Given at the moment the only alternative was Potlatch 2, which was sub-par for JOSM fans, it essentially blocked all mapping.

A few days later other developers deleted the plugin and its source code from the JOSM repository. Still remember that.

JOSM plugins list with a single plugin filtered: no_more_mapping. Description reads, "Disallow using JOSM forever (WARNING: this plugin prevents JOSM from loading and is hard to rid of)".
DELETED replied to Ilya
Ilya Zverev replied to DELETED

@1363487 hahaha right :) Alas JOSM development model requires core devs to update all the plugins after API changes, and this plugin was excluded from the updates, so now it just throws exceptions instead of shutting down JOSM.

Ilya Zverev replied to Ilya

So, back to editors, I guess. I wrote quite a few. Before Every Door, the most known was the low-level one. I didn't think of a name for too long.

Level0's main invention was not a user interface, but a user-editable representation of OSM XML. With it, it's pretty easy to edit tags, move or add POI, or even edit line geometries. All while feeling like a hacker.

wiki.openstreetmap.org/wiki/Le

It was obviously inspired by the french RawEditor, for which I even made a wrapper first.

Level0 screenshot from OSM Wiki. Some objects from a latvian city are downloaded, and there is a ton of buttons.
Amᵃᵖanda | OSM Witch 🧙🏻‍♀️ replied to Ilya

@zverik Level0 was a great, easy way for sites (like townlands.ie) to do simple “imports”/“mechanical edits”. I had the tags (in level0 format) which people could add to objects & a link to level0 edit URL. Mappers could open in new tab and paste the tags. Done. Edits in a few clicks.

Certain versions of this could be done with @MapRoulette now, I suppose. But level0 formatted tags & a URL is much faster to get working.

mtmail replied to Ilya

@zverik I still use it regularly. ♥️ addr:postcode and addr:city mixed up on 50 buildings? I query overpass for a list of ids, query level0 with those, search&replace, then upload again. Or easy to remove a member from a relation.

Ilya Zverev replied to Ilya

Copy-pasting an URL into Level0 just to edit tags was too much work, so recently I've made it into a browser plugin.

wiki.openstreetmap.org/wiki/Op

Really simple and obvious stuff: see a tag error? Click "Edit tags" and fix it. No need to go anywhere or deal with the javascript-heavy iD.

Inside it's a bunch of weird hacks. I'm still surprised that in two years nobody has tried to integrate it into the website itself. Like, whom are we making that website for?

github.com/openstreetmap/opens

Copy-pasting an URL into Level0 just to edit tags was too much work, so recently I've made it into a browser plugin.

wiki.openstreetmap.org/wiki/Op

Really simple and obvious stuff: see a tag error? Click "Edit tags" and fix it. No need to go anywhere or deal with the javascript-heavy iD.

Screenshot of osm.org website with a tags editing panel. Editing "Way: Veskitammi Selver ABC", there is a long list of tags in the form "key = value", and save/cancel buttons. On the map to the right the selected building is visible.
TheStroyer replied to Ilya

@zverik it's unfortunate that there is so little improvement of the interface of the official OSM website. It seems to be stagnant. Is there no one who want to work on it?

Caleb Maclennan replied to Ilya

@zverik A tag editor should absolutely just come baked in to the #OSM site out of the box.

Ilya Zverev replied to Ilya

So this reminds me, right around the Level0 I also finished the project I spent three months on, day to day. It introduced maps to multiple forum engines.

MapBBCode is primarily a way to mark a map data in bbcode, which is text. The hard thing was to make a series of libraries and plugins and modules to integrate it and Leaflet into engines like phpBB and FudForum.

mapbbcode.org/

And then it turned out, finding people to share supporting would be much harder.

An interactive map with a route from Saint-Petersburg to Tallinn via Border control rendered from the bbcode displayed below. It reads [map]coordinate(S); other coordinates and annotations[/map]. There are two buttons: "Update" and "Edit".
Simon Poole

@zverik @NGIZero

Getting employed to "do what just happens to cross my mind" is quite rare, typically you need to have a Noble prize or similar associated with your career to get that kind of position.

Just because there is some marginal usefulness of what you just thought up for society, doesn't mean that society is obliged to pay you for it.

If your product has actual value, then there is always the time honoured avenue of selling it, just like every baker, farmer, artist etc. has to do .

Ilya Zverev

@simon Sure, I understand that. This is a product talk. If you have a product, sell it.

What I do, is not products. It's R&D. Taking two or more things, mushing them together and seeing if it blows up or cures white spots on the map.

From some things I do, products grow. Like Every Door and other editors, Easy Transit, Nik4. Or other people take my R&D and form it into products they sell, like RocketData and InkAtlas did.

Cashable "value" comes from general public. My audience is mappers.

Ilya Zverev

@simon ...and that's the problem with most open-source: they target other developers, just like I target other mappers, and that's not a very visible or paying audience. You do pip install, not buy a library. You download every door, not pay for a privilege of improving OSM.

When OSMF sees something the general public can benefit from, they pay. E.g. Jochen's work on the data model. I suggested different API and data model improvements that benefit _mappers_ in 2019, but that's long forgotten.

Ilya Zverev

@simon And if you think that's a popularity or visibility problem, ask Tobias how much he earns from StreetComplete. Which is a product seven times more popular than ED and is the second most popular editor, between iD and JOSM.

I'm willing as much to advocate for him to get on a payroll for his work as for myself. He absolutely deserves it.

Simon Poole

@zverik maps.me/OM is just as popular. But that is not the point, the question is if spending the money on something provides a reasonable return/value for the money spent. And if you look it that way, spending money on anything except iD maintenance is literally pointless.

And given that there is a fixed pot of money, spending it on anything outside of what gives the best bang for the buck, actively damages the project.

Ilya Zverev

@simon regarding the "fixed pot", that's an attitude I cannot support. If we're talking about OSMF, it's far from fixed, though OSMF makes it look so. In development in general, "a return for money spent" is not how you get new things. Like, no company besides Cloudmade and MapZen innovated in OSM on their own money. Because R&D doesn't bring money. Is spending on it pointless? Well, if your goal is to make more money, then yes. Is the goal of OSMF to earn money?

Simon Poole

@zverik the goal is to get more people mapping. So lets do the numbers, for simplicity lets assume the OSMF can spend $1/year on development per new mapper, so $152k for 2023 (this is substantially more than what the OSMF actually spends) and split that up

New mappers 2023 per editor
Simon Poole

@zverik so now you will likely argue that a new maps.me/OM mapper is not worth the same as a new everydoor mapper because the retention/conversion rate is much lower. So lets factor that in

Cost for new larger mappers per editor 2023.
Simon Poole

@zverik and wrt the argument that you don't get new stuff without spending money.

Literally all editors in that list were initially developed by private initiative, and those private individuals,/organisations/corporations carried the risk that nothing would come of it (and the list of the projects with that outcome is long). Why would the OSMF subsume your risk over that of others?

Ilya Zverev

@simon Why me and not others? Good point — I don't know! In this R&D thread I'd argue that SC and ED are the only two editors actively innovating OSM editing today. We see the outcome: two products that slowly crawl to replace iD and JOSM as primary editors for beginner and dedicated mappers. OSM got those for free. But just as with Merkaartor, OWL, 360°, and - almost - iD, it can lose it for free as well.

Good thing all nine editors in your lists are made by people from first-world countries!

Simon Poole replied to Ilya

@zverik "2 two products that slowly crawl to replace iD and JOSM as primary editors for beginner and dedicated mappers." the numbers clearly do not support that conjecture.

Ilya Zverev replied to Simon

@simon OSM is a long game; let's get back to this in five years :)

Simon Poole replied to Ilya

@zverik 5 x 0 = 0

Less snarky: there is hardly any replacement of conventional editing now, so why should that be different in 5 years, more supplemental use, yes I can see that (that is assuming current product profiles, obviously a name is just a name, so can be used for something completely different).

Ilya Zverev

@simon Haha no, I'd argue the opposite: drive-by mappers probably make edits that are out of reach to mappers who stay. I made this point at SotM in 2015 and stand by it today. OM is important to us because it reaches audiences SC/iD cannot ever.

Rihards Olups

@simon @zverik
That sort of makes OSM[F] reactionary, stripping it from proactive projects.
Which sort of vibes with "supports not controls" approach, at least partially.

But OSMF potentially could explore more active stance (taking care of becoming dependent of sponsors).

Oh, BTW, perhaps the great OSM community developers can be polite one to another - even negative views can be expressed in ways that don't leave the other person much worse off ;)

Simon Poole

@richlv @zverik the thing is, funds are in reality not unlimited, so there has to be choices.

Sure the OSMF could move to a centralized development model, but if wikipedia is anything to go by then we would then just have one and a half editors.

As is we have a competitive market place in which everything goes. That has the advantage that there is more choice and the disadvantage that the majority of devs are working on their own dime (or whatever they are able to scavenge from 3rd parties).

Rihards Olups

@simon @zverik
Yeah, and OSMF supporting some editors (or projects in other categories) would likely kill others in that space.

And I say this still being sad that OWL disappeared, had wished to see it on osm.org.

Perhaps OSMF could act more as a facilitator, helping to connect developers with potential grant sources, having some templates for applications and the process?
It would reduce the risk of favoritism, help the devs and keep OSMF at "supports not controls".

Simon Poole replied to Rihards

@richlv @zverik OWL never really worked, and the last dev that worked on it tried to use it as leverage to get employed by the OSMF ... which as we know, failed.

A full blown replacement for it is probably out of the financial reach of the OSMF and it isn't even clear if that approach still makes any sense.

Ilya Zverev replied to Simon

@simon @richlv Regarding OWL, yeah, I made WhoDidIt which basically replaced it, and then Wille made OSMCha that got supported by Mapbox and basically fully covers the standard low-level changes monitoring.

I wouldn't say they used it as a leverage... Or I don't remember it all too well. It definitely was too early for that.

Ilya Zverev

@simon @richlv Currently I see the choice is between all or nothing: if we can't fund everything, we'll allow developers go at their own pace with their own money.

I heard this Wikimedia thing repeated so often, I started doubting that it's either-or proposition.

And pretty sure OSM is far from a competitive market. If we count every single attempt at making an editor, we will hardly get to a hundred. There are 27 editors with 30+ users _in_a_year_, OSMF could start supporting them all easily.

Ilya Zverev replied to Ilya

@simon @richlv (not that I'm suggesting that making editors is more important than other, non-IT things)

Simon Poole replied to Ilya

@zverik @richlv it isn't even the only IT thing and I would argue that money spent improving the website would have a lot more impact (not to mention that a substantial part of what is actually spent is ops).

Ilya Zverev replied to Simon

@simon @richlv Well, this now goes into "just do anything" narrative I've been pushing for OSMF for years :) Yes, website update is long overdue. It still broadcasts a wrong message about OpenStreetMap, and omits many things people are searching for.

Go Up