maps

Anything about maps, cartography

Sedona Hike

XYZ Tiles and WMS Layers in QGIS 3

I did a lot of hiking around Sedona, Arizona a few weeks ago, and wanted to map my GPS way points and tracks in QGIS over some WMS (web mapping service) base map layers. I recently switched to QGIS 3 since I need to use that in my book (by the time it comes out 2.18 will be old news), and had to spend time starting from scratch since the plugin I always used was no longer available (ahhh the pitfalls of relying on 3rd party plugins – see my last post on SQLite). I thought I’d share what I learned here.

I was using the OpenLayers plugin in QGIS 2.x as an easy resource to add base maps to my projects. You could pull in layers from OSM, Google, Bing, and others. It turns out that plugin is no longer available for QGIS 3.x. So I searched around and found some suggestions for a different plugin called QuickMapServices which was a great replacement. But alas, that worked in QGIS 3.0 but is not compatible (as of now) for QGIS 3.2.

So I’m back to adding WMS layers manually. There is a new feature in QGIS for adding XYZ Tiles; this is a little better than WMS because the base map can be rendered a bit quicker. I found a tip in the Stack Exchange that you can add an OSM tiles layer with this url:

http://tile.openstreetmap.org/{z}/{x}/{y}.png 

Select XYZ Tiles in the Browser, right click, New connection, give it a name, add the URL. You can modify the X Y Z coordinates where the map centers and zooms by default. Once you’ve created the connection, you can simply drag the OSM layer into the map window to render it.

Adding the OSM XYZ Tiles in QGIS

One problem that always creeps up: when you add other layers and adjust the zoom, sometimes the rendering of the base map looks poor, i.e. the features and labels look blurry or blocky. When you’re pulling data from a web map layer, as you zoom in it swaps out the tiles for more detailed ones appropriate for that scale. But when you’re zooming in QGIS things can get out of synch, as your map window zoom may not be enough to trigger the switch in the map tiles, or those map tiles are just not meant to be rendered at that scale. If you right click on a blank area of the toolbar, you can activate the Tile Scale panel and can use the slider to adjust the window zoom in synch with the tiles, so you can operate at the scales that are appropriate for the tiles. The way points and track for our hike alongside Schnebly Hill Road are shown below, and the labels for the points represent our elevation in feet.

OSM Tile Layer with Tile Scale Panel

If the slider is grayed out, select the OSM layer in the Layers menu, right click, and select Set CRS  – Set Project CRS From Layer. Web mapping services typically use EPSG 3857 Pseudo Mercator as the coordinate reference system / map projection by default. If your other vectors layers aren’t in that system, you can have the base map draw to their system or vice versa by selecting the layer, right clicking, and choosing Set CRS. But for the tile scale to work properly EPSG 3857 must be the project CRS.

Lastly, I’ve always liked the USGS WMS layers, which are never included in the plugins that I’ve seen. The USGS provides layers for: imagery, imagery with topographic features, shaded relief, and the USGS topographic map layer:

https://basemap.nationalmap.gov/arcgis/rest/services

USGS Link for WMS Layer for Topographic Maps

You can click on one of the services, and at the top (in small print) are urls for their services in WMS and WMTS. The last one is a web mapping tile service, which is a bit faster than WMS. Click on the WMTS link, and copy the url from the address in the browser. Then in QGIS select WMS / WMTS layers, right click, add a new connection, give it a name and paste the url. This is url for the topographic map:

https://basemap.nationalmap.gov/arcgis/rest/services/USGSTopo/MapServer/WMTS/1.0.0/WMTSCapabilities.xml

Once again, you can drag the layer into the map window to render it, and you can use the Tile Scale panel to adjust the zoom. Here’s our hike with the topo map as the base:

USGS Topographic Map in QGIS

The Map Reliability Calculator for Classifying ACS Data

The staff at the Population Division at NYC City Planning take the limitations of the American Community Survey (ACS) data seriously. Census estimates for tract-level data tend to be unreliable; to counter this, they aggregate tracts into larger Neighborhood Tabulation Areas (NTAs) to produce estimates that have better precision. In their Census Factfinder tool, they display but grey-out variables where the margin of error (MOE) is unacceptably large. If users want to aggregate geographies, the Factfinder does the work of re-computing the margins of error.

Now they’ve released a new tool for census mappers. The Map Reliability Calculator is an Excel spreadsheet for measuring the reliability of classification schemes for making choropleth maps. Because each ACS estimate is published with a MOE, it’s possible that certain estimates may fall outside their designated classification range.

For example, we’re 90% confident that 60.5% plus or minus 1.5% of resident workers 16 years and older in Forest Hills, Queens took public transit to work during 2011-2015. The actual value could be as low as 59% or as high as 62%. Now let’s say we have a classification scheme that has a class with a range from 60% to 80%. Forest Hills would be placed in this class since its estimate is 60.5%, but it’s possible that it could fall into the class below it given the range of the margin of error (as the value could be as low as 59%).

The tool determines how good your classification scheme is by calculating the percent of estimates that could fall outside their assigned class, based on each MOE and the break point of the class. On the left of the sheet you paste your estimates and MOEs, and then type the number of classes you want. On the right, the reliability of classifying that data is calculated for equal intervals (equal range of values in each class) and quantiles (equal number of data points in each class). You can see the reliability of each class and the overall reliability of the scheme. The scheme is classified as reliable if: no individual class has more than 20% of its values identified as possibly falling outside the class, and less than 10% of all the scheme’s values possibly fall outside their classes.

I pasted some 5-year ACS data for NYC PUMAs below (the percentage of workers 16 years and older who take public transit to work in 2011-2015) under STEP 1. In STEP 2 I entered 5 for the number of classes. In the classification schemes on the right, equal intervals is reliable; only 6.6% of the values may fall outside their class. Quantiles was not reliable; 11.9% fell outside. If I reduce the number of classes to 4, reliability improves and both schemes fall under 10%; although unreliability for one of the classes for quantiles is high at 18%, but still below the 20% threshold. Equal intervals should usually perform better than quantiles, as the latter scheme can make rather arbitrary breaks that result in small differences in value ranges between classes (in order to insure that each class has the same number of data points).

Map reliability calculator with 5 classes

Map reliability calculator with 4 classes

You can also enter custom-defined schemes. For example let’s say you use natural breaks (classes determined by gaps in value ranges). There’s a 2-step process here; first you classify the data in GIS and determine what the breaks are, and then you enter them in the spreadsheet. If you’re using QGIS there’s a snag in doing this; QGIS doesn’t show you the “true” breaks of your data based on the actual values, and when you classify data it displays clean breaks that overlap. For example, natural breaks of this data with 5 classes appears like this:

24.4 – 29.0
29.0 – 45.9
45.9 – 55.8
55.8 – 65.1
65.1 – 73.3

So, does the value for 29.0 fall in the first class or the second? The answer is, the first (test it by selecting that record in the attribute table and see where it is on the map, and what color it is). So you need to adjust the values appropriately, paying attention to the precision and scale of your numbers. In this case I bump the first value of each class up by .1, except for the bottom class which you leave alone:

24.4 – 29.0
29.1 – 45.9
46.0 – 55.8
55.9 – 65.1
65.2 – 73.3

In the calculator you have to enter the top class value first, and just the first value in the range:

65.2
55.9
46.0
29.1
29.4

Map reliability calculator with user defined classes

In this case only 7.1% of the total values may fall outside their class so things look good – but my bottom class barely makes the minimum class threshold at 19.4%. I can try dropping the classes down to 4 or I can manually adjust this class to see if I can improve reliability.

If you’re unsure if you made the right adjustments to the classes in translating them from QGIS to the calculator, in QGIS turn on the Show Feature Count option for the layer to see how many data points are in each class, and compare that to the class counts in the calculator. If they don’t match, you need to re-adjust.

QGIS natural breaks and feature count

This is a great tool for census mappers who want or need to account for issues with ACS reliability. It’s an Excel spreadsheet but I used it in LibreOffice Calc with no problem. In addition to the calculator sheet there’s a second sheet with instructions and background info. Download the Map Reliability Calculator here. You can try it out with this test data,  workers who commute with mass transit, 2011-2015 ACS for NYC PUMAs.