|
Top-level
 Of course another question that was heard from every business, "how do we get the tiles". Mapnik had a sample python script packaged, but it was slow and hard to use. I'm pretty good at APIs, so I made my own. Polytiles can render regular raster tiles for a given bbox in multiple threads. Obviously. It also can use a polygon file or a relation from OSM, it can render retina tiles, generate an mbtiles file, or just produce multiple lists for other tools. 60 comments
And back to my projects from days back, that still operate. In the same trend, seeing something and improving on it, I took Frederik's BigMap and added an online tile-stitching service. The UI is the same, still bad, but it's the easiest way to get a raster map for an area. And also georeferencing files for OziExplorer or QGIS. Many people use it daily. I see how it could be made better, faster and easier, but as always, not enough time. And no alternatives haha.  @zverik Have used this one (grabbing the generated Perl script) - thank you for this tool :) How is georeferencing done? @richlv Thanks for using it! Georeferencing is simple — it's tiles, we know exact locations of all corners :) Just download a WLD file and put it near the image. @zverik Ah, is this a usecase like "download tile files locally and use them as a georeferenced layer"? I'm only doing OSM in an amateurish way, thus these GIS things are another world for me =) @richlv Yes, exactly. A good alternative to adding a live OSM layer and re-downloading tiles every time something moves. @zverik Neat, thank you for the explanation. Maybe one day I'll find time to dig more into these topics :) When belgian OSM guys made the Welcome Tool in 2016, I set up an instance for Russia and started welcoming people. I sent a few hundred messages, and after promoting the tool, some other mappers did too. Alas the response rate was like one percent. And when I got replies, sometimes I wished I hadn't. But without welcoming messages, people in OSM feel themselves in a vacuum. There are still no links to communities anywhere on osm.org. Here's the global WT instance: https://welcome.osm.be/  @zverik maybe you have some suggestions on how to improve https://openstreetmap.org/communities ? One idea being floated around is to embed the OSM Community Index map somehow in that page. (The list of local chapters is already dynamically based on the index.) @seav Start with a big bold link to the forum on the "Communities" page! Generally I'd take the hint from the iD editor. It does a pretty good job of listing links after you upload a changeset.  @zverik ooof I feel that one. Have started making my first contributions in years (because I realized I can add bike racks) but it might as well be posting into the void. Today, brownouts start for OAuth1a endpoints. In a month, only OAuth2 can be used. One of my services using OAuth1 is OSM Authentication Proxy. Since storing passwords is bad, and OAuth sometimes is too hard to implement (you need a browser!), I made this proxy. Essentially it's a website that says "yes, the person with the token is indeed an OSM user". It doesn't upload anything to OSM API. I thought of using it for command-line tools and plugins, but ended up not. Command-line applications have a hard time with OAuth, because it requires a browser. The auth flow has so many steps, it was much easier to just give up and use Basic Auth with a password. Taking a hint from a Google's library, I made cli-oauth2: a Python library that makes authenticating a one-line task. https://github.com/Zverik/cli-oauth2 Fun fact: the idea came when I was rewriting Level0 for command-line. Didn't get it to work (yet haha), but there's one less problem to solve.  @zverik Does the script also change IDs? What does "fix ids" mean? For readers' info, Organic Maps Editor doesn't store IDs and doesn't rely on them.  @organicmaps @organicmaps To upload objects to the sandbox, you need to give them negative ids and keep the references. This is not related to OM, just a low-level API stuff. (should mention that there are TONS of nuances that don't fit into tweets.) Two main issues with the proposal that don't make it a perfect one: - It's too wordy, nobody would read it, despite the core idea is very simple (stop nodes). - It challenges the current tram stop mapping (and nothing else). The new PT schema should've been perfect in 2018 to be approved, and now it should be double-perfect, otherwise it won't pass any discussion. Everybody in OSM has their own opinion on PT mapping. Keeping an up-to-date rendering database means relying on worldwide replication streams from osm.org or per-country streams from @geofabrik . But what if I want my own extract area? In 2014 I dug into osm2pgsql's "slim" tables, and wrote a script to trim the data outside a given polygon: https://github.com/Zverik/regional I use it for my "veloroad" tile server, but not sure if it works: the database grows pretty fast. Maybe it's just because people are mapping more than ten years ago. @zverik @everydoor @josmeditor wow, that looks like it can improve the editing workflow a lot @TheStroyer @everydoor @josmeditor Indeed! Just last Sunday I was walking with my friends in a forest, and surveyed a few streams and boardwalks this way. They would be lost if not for the drawing feature. Editing ways properly (e.g. in @vespucci_editor ) needs too much involvement, compared to just a single finger movement in ED. @zverik @everydoor @josmeditor @vespucci_editor can you choose a color? That would be useful @TheStroyer @everydoor @josmeditor @vespucci_editor kinda see my sunday notes for an example https://geoscribble.osmz.ru/#17/59.24465/25.68334 @zverik @TheStroyer @everydoor @josmeditor why draw? just record a GPX track ... PS: in a forest you probably want to use a DTM anyway for tracing. PPS: not to mention that you are massively overstating the effort to simply directly add things. Since we're mentioning @josmeditor , here's one thing I made a long time ago: Yeah, that's a proper landing page (in Russian), with a few buttons, shortcuts to most important pages like a forum thread or a tag list, and a tutorial video. I even browsed the changelog and added interesting changes to the feed, but stopped in 2020. I still use the page to download updates though. I'm a bit out of breath, but def not out of projects :) Gonna keep doing this until I get an answer from NLNet. In 2010 Microsoft granted to OSM permission to trace Bing imagery. It was the only detailed imagery layer we had in Russia, so that enabled making a validator that found illegal imagery usage (google / yandex): Basically it watched where people uploaded buildings and checked for available Bing imagery. In 2014 we got Mapbox layer, and I turned this off. There are different kinds of imports. Safest are those done by hand, but also the dirtiest, in my opinion: it's easy to miss or to get lazy and just press upload. For towns and villages, the best source we had was GNS, an american database of toponyms. I converted it to OSM format and wrote a short manual on copying it onto the map: Interesting that while box=yes parameter doesn't work on osm.org nowadays, parameters minlat, maxlat etc still do.  @zverik Still loads of redundant GNS tags on place nodes in Ireland (probably imported by Dermot - mackerski). Until the townlands project got going they were still the major source of places in rural Ireland. Still a lot of dedup & other cleanup needed. I also did a bit with them for Pakistan, but never could trust the data in a country where the population had increased so much : http://sk53-osm.blogspot.com/2011/03/garmin-overlays-of-gnis-names.html (probably should correct GNIS->GNS Having a GeoJSON to preview an import is good, but how to suggest amendments? That's why I made CF Audit: an interactive website to go through changes one by one, cross-reference them, and fix anything wrong. Those fixes are kept for consequent imports with OSM Conflator, so less work next time. I know RocketData iterated on it for their operations, and there's a new IFS fork by @sfkeller team: https://cfauditgui.geoh.infs.ch/ So I got the conflator, and I got the audit tool, and the entire pipeline as shown on the picture. Did I use it? Oh did I even. 5k Walmart supermarkets, tens of thousands Shell fuel stations, ~10k addresses in Moscow, various restaurant and shop chains... It all worked perfectly. Of course it was just a few imports. I envisioned an automated system for high volume of data, still controllable by OSMers, but nobody asked for that, and I didn't have time. Still, mentioned it at SotM EU 2023. You know OpenWhateverMap. Grant made the original version, then we lost the domain, and then I updated it with modern tiles and some interactivity: clicking on a tile reveals its name and URL. For the @sotm in Firenze I went all-out with the remote participation part. One thing I did was a Telegram bot, where people from anywhere could introduce themselves. The idea was to put a face to hundreds of names on online channels. Because once you have a face and a voice attaches to words, it's impossible not to feel a proper conversation going when you type. Not many people contributed, but still, was fun. The bot was disabled after the conference ended of course. For a very long time the leading repository on my github was "leaflet-grayscale" with a hundred stars. I love @leaflet and made too many plugins for it. This one was quite simple — but I guess useful enough. It just makes the map grayscale. There are too few layers fit for a visualization background. Sometimes removing colors (and labels if possible) is good enough. @rugk yeah I decided not to set up certificates given nobody uses the service :)  "There are still no links to communities anywhere on osm.org." @zverik every time you use iD and then there's a whole very visible tab dedicated to the subject https://www.openstreetmap.org/communities @simon *every time you upload a change in iD — which is a pretty long way to a link. On the tab there's a list of Local Chapters, essentially legal entities. Works for a business, too much for a person. I just want a link to a forum at least, or that localized panel from iD ideally, visible at all times on the main page.  @zverik it is a tough one, yes there are some legitimate uses of bulk downloading, but most of the tiles are never viewed. If we accept bulk downloading it will have an negative impact on others seeing fresh tiles, which is a particular concern during vandalism. We should make a clearer metric for the served fresh:stale ratio. EU week daytime hours have a very poor ratio. With some development effort the tile service could have a way to be told the request is bulk, changing render/caching logic. |
Gregory's Server
Obviously setting up Mapnik locally is hard, and people wanted to download OSM tiles with a proper UI.
In 2011 I took over the development of JTileDownloader, which does exactly that. Bulk downloading was frowned upon even then. I tried to ease the load: imposed a 5 tiles/sec limit and altered downloading order to meta-tiles (8x8 tile packs).
https://wiki.openstreetmap.org/wiki/JTileDownloader
Still, @osm_tech don't care, JTD was banned by user-agent, and people returned to anonymous bulk downloaders with no safeguards.
Obviously setting up Mapnik locally is hard, and people wanted to download OSM tiles with a proper UI.
In 2011 I took over the development of JTileDownloader, which does exactly that. Bulk downloading was frowned upon even then. I tried to ease the load: imposed a 5 tiles/sec limit and altered downloading order to meta-tiles (8x8 tile packs).