maps

Anything about maps, cartography

Factorio forest landscape

Exploring New Worlds in Factorio

The first draft is finished and I sent my book off for review earlier this month, and I’ve been back to work full-time for two months now. It’s been a difficult transition, so I thought I’d write a more lighthearted post this month about imaginary geographic worlds (as luck would have it, the Geo NYC Meetup group is discussing fictional mapping next week).

I’ve always enjoyed top-down simulation games; I still have my original copy of SimCity from 1989, in the box with the diskettes. More recently, I started playing a top-down, world-exploration, operations management, logistical simulator game called Factorio. The premise is you are the sole survivor of a team of scientists and engineers who have crash landed on an unexplored world. Using the scrap metal of your ship, a few simple tools, and the abundant resources on the planet, your goal is to build a rocket to launch a satellite into space to alert the crew of a successive spaceship of your presence. Scattered across the planet are concentrations of resources: water, trees (for wood), stone, iron ore, copper ore, oil, coal, and uranium. With an ax and a few scavenged plates from the ship, you begin by building a stone furnace for smelting metals. You use your ax to mine some stone to build the furnace, some iron for smelting, and some coal for fuel. Once you’ve smelted some metal you can construct a drill to mine the materials and insert them into the smelter automatically.

Smelting the ore converts it into refined material: stone to bricks, iron ore to iron plates, and copper ore to copper plates. Initially you can take these materials and manually craft them to make products: iron plates become iron gear wheels, copper plates become copper wire, which in turn can be crafted to create higher order parts like electronic circuits and finished machine products. Ultimately you’ll construct assembly plants that take the necessary materials and build the products for you, and the outputs can be used as inputs for other products.

Factorio mining drills

Mining ore

Factorio smelters

Smelting ore to plates

Factorio assemblers

Assembling products

 

 

 

 

 

 

The game becomes a logistical puzzle, where you mine ores from various deposits and move them to be smelted, and then move the refined materials to different assembly plants to create higher-order products. You transport everything using conveyor belts and inserters, which grab materials from belts and insert them into the smelters, assemblers, and other structures. You construct pumps, boilers, and steam engines powered by coal or wood to generate electricity to power the entire factory, and in order to keep developing higher-order goods you combine certain materials to produce “science”; little colorful beakers of liquid that you move on belts to laboratories to keep research humming.

As the game and your research progresses you develop technology that allows you to better explore the world and access additional resources, as you’ll eventually deplete the original deposits near the factory. You can develop solar and nuclear power as cleaner electricity alternatives, drill and refine oil to create fuels and plastics, build cars to explore the landscape, and construct railroads to transport more distant materials to your base. As your factory expands you have to grapple with the logistical hurdles of moving products created at disparate ends of the plant together in order to create new products, forcing you to either plan ahead or reconfigure your layout as time passes (or build some drones to fly the materials around). The clock is always ticking as the game is played in real time (it’s not turn-based).

Factorio factory

Main Factorio screen showing portion of a factory and map layout

At some point you face a new problem: you are not alone on this planet. There are some large, scary-looking insect creatures living there that don’t like all the pollution that’s coming from your factory, and they don’t particularly like you. Once they become irritated enough they attack and chew up your factory, and you along with it. Sadly there is no negotiating with them (they’re not sentient), so some of your attention and resources must be spent on weapons. You can take a purely defensive approach, building walls and gun turrets to protect your base as well as armor and shields to protect yourself. Or you can barrel out in a tank or use artillery to destroy them as they encroach on your operations. You can also develop cleaner, less polluting technologies to irritate them less.

An additional challenge is that the game keeps changing. Even though it’s been out for several years Factorio is still in a beta phase, but given it’s maturity and update cycle it’s super stable. The developers are part of a small company in the Czech Republic who focus primarily on this game. Factorio is available for purchase via their website and via Steam for all operating systems, and has been downloaded over a millions of times. The fanatical fan base appreciates the ability to mod practically every aspect of the game, and they form a community that’s crucial to the game’s development through testing and feedback. Factorio is definitely a member of a new generation of games where part of the challenge is learning how to play it. I’ve crawled through the extensive wiki, scoured Reddit for advice, and watched several YouTube series to figure out how it works.

Regardless of how many times you’ve played it, there is always something new to tinker with. Many players enjoy the engineering and mathematical side of the game. Their goal is to build the most efficient system, perfectly balance inputs and outputs, and create the best ratios for production. Others go for scale, building the largest possible factory with the most throughput. There are railroaders who enjoy building the trains, and warriors who focus on combat with the voracious bugs. Beyond building the rocket, the game has a number of challenges that players attempt to master, and it can be played solo or multiplayer for gamers who want to work together or simply explore each other’s layouts and solutions.

As a geographer, I enjoy the actual worlds themselves and the unique challenge each environment presents. While you can create blueprints and use the same design for a railroad station or solar power generator over and over again, you’re forced to change your overall factory layout based on the location of resources and configuration of the terrain. Prior to launching a new game, you specify the general size, frequency, and richness of resources, trees, water, and enemy bugs, and you can keep generating maps until you find one you like. While many of the efficiency aficionados want flat playing surfaces, I enjoy the complexity of fitting your factory in around the oceans and forests, and the challenge of exploring and shipping in materials from far flung places.

The world itself is quite beautiful. The developers provide extensive details about the development and inner-workings of the game, including the processes for generating logical and realistic looking landscapes. There are lush deciduous forests in vibrant autumn colors, desert wastelands strewn with rocks, and clusters of baobab-like trees on the dry plains. Even though they’re just bits and bytes I limit what I harvest, because I hate chopping them down. Unlike the real world, mining ore is much less destructive and the material is simply scraped off the surface, leaving unblemished soil behind. A finite portion of the world is visible on the map when you begin the game, and the surrounding area is cloaked in darkness. You can reveal more of the terrain by building radar stations at your base, and can explore on foot or go further afield once you’ve constructed vehicles. The world has no end, and stretches into infinity.

Factorio forest landscape

Factorio has sparked my curiosity in unexpected ways. As I’m mining ores and moving them into smelters to produce metals, I started to wonder: what is smelting anyway? How do you actually extract metals from rocks? My exposure to chemistry was limited to my junior year of high school where I struggled with balancing formulas and memorizing the periodic table. Fortunately I discovered some fascinating books and videos that made the subject engaging. Material scientist Mark Miodownik’s Stuff Matters: Exploring the Marvelous Materials That Shape Our Man-Made World, is an accessible, informative, and often hilarious exploration of the materials we use everyday. You’ll learn the basic chemistry behind paper, iron, ceramics, even chocolate! Periodic Tales: A Cultural History of the Elements, from Arsenic to Zinc by Hugh Aldersey-Williams is perfect if you want to learn some basics about chemistry and material science from a historical science perspective. NOVA aired a solid three-part series a few years back called Treasures of the Earth that revealed the secrets behind gems, metals, and power sources.

I resisted the temptation to play for the year I was on sabbatical, as it’s too easy to get sucked into it. A few hours here and there throughout a month, and by the time I launch that rocket into space 30 hours have gone by! Initially I feel a bit guilty, sinking so much time into a game. But when you consider how much time the average person spends watching TV or looking at stupid stuff on their phone (4 hours and 2.5 hours respectively, EVERY DAY!), enjoying the occasional game that challenges your mind and sparks your imagination is a good alternative. Similar to the Minecraft phenomena, I think it has great potential as an educational tool for learning about logistics, planning, geology and materials science, and engineering. And for the geographers out there, there are infinite worlds to explore.

Factorio desert landscape

Lying with Maps and Census Data

I was recently working on some examples for my book where I discuss how census geography and maps can be used to intentionally skew research findings. I suddenly remembered Mark Monmonier’s classic How To Lie with Maps. I have the 2nd edition from 1996, and as I was adding it to my bibliography I wondered if there was a revised edition.

To my surprise, a 3rd edition was just published in 2018! This is an excellent book: it’s a fun and easy read that provides excellent insight into cartography and the representation of data with maps. There are concise and understandable explanations of classification, generalization, map projections and more with lots of great examples intended for map readers and creators alike. If you’ve never read it, I’d highly recommend it.

If you have read the previous edition and are thinking about getting the new one… I think the back cover’s tagline about being “fully updated for the digital age” is a little embellished. I found another reviewer who concurs that much of the content is similar to the previous edition. The last three chapters (about thirty pages) are new. One is devoted to web mapping and there is a nice explanation of tiling and the impact of scale and paid results on Google Maps. While the subject matter is pretty timeless, some more updated examples would have been welcome.

There are many to choose from. One of the examples I’m using in my book comes from a story the Washington Post uncovered in June 2017. Jared Kushner’s real estate company was proposing a new luxury tower development in downtown Jersey City, NJ, across the Hudson River from Manhattan. They applied for a program where they could obtain low interest federal financing if they built their development in an area were unemployment was higher than the national average. NJ State officials assisted them with creating a map of the development area, using American Community Survey (ACS) unemployment data at the census tract level to prove that the development qualified for the program.

The creation of this development area defies all logical and reasonable criteria. This affluent part of the city consists of high-rise office buildings, residential towers, and historic brownstones that have been refurbished. The census tract where the development is located is not combined with adjacent tracts to form a compact and contiguous area that functions as a unit, nor does it include surrounding tracts that have similar socio-economic characteristics. The development area does not conform to any local conventions as to what the neighborhoods in Jersey City are based on architecture, land use, demographics, or physical boundaries like major roadways and green space.

Jersey City Real Estate Gerrymandering Map

Census tracts that represent the “area” around a proposed real estate development were selected to concentrate the unemployed population, so the project could qualify for low interest federal loans.

Instead, the area was drawn with the specific purpose of concentrating the city’s unemployed population in order to qualify for the financing. The tract where the development is located has low unemployment, just like the tracts around it (that are excluded). It is connected to areas of high unemployment not by a boundary, but by a single point where it touches another tract diagonally across a busy intersection. The rest of the tracts included in this area have the highest concentration of unemployment and poverty in the city, and consists primarily of low-rise residential buildings, many of which are in poor condition. This area stretches over four miles away from the development site and cuts across several hard physical boundaries, such as an interstate highway that effectively separates neighborhoods from each other.

The differences between this development area and the actual area adjacent but excluded from the project couldn’t be more stark. Gerrymandering usually refers to the manipulation of political and voting district boundaries, but can also be used in other contexts. This is a perfect example of non-political gerrymandering, where areas are created based on limited criteria in order to satisfy a predefined outcome. These areas have no real meaning beyond this purpose, as they don’t function as real places that have shared characteristics, compact and contiguous boundaries, or a social structure that would bind them together.

The maps in the Post article high-lighted the tracts that defined the proposal area and displayed their unemployment rate. In my example I illustrate the rate for all the tracts in the city so you can clearly see the contrast between the areas that are included and excluded. What goes unmentioned here is that these census ACS estimates have moderate to high margins of error that muddy the picture even further. Indeed, there are countless ways to lie with maps!

Sedona Hike

XYZ Tiles and WMS Layers in QGIS 3

I did a lot of hiking around Sedona, Arizona a few weeks ago, and wanted to map my GPS way points and tracks in QGIS over some WMS (web mapping service) base map layers. I recently switched to QGIS 3 since I need to use that in my book (by the time it comes out 2.18 will be old news), and had to spend time starting from scratch since the plugin I always used was no longer available (ahhh the pitfalls of relying on 3rd party plugins – see my last post on SQLite). I thought I’d share what I learned here.

I was using the OpenLayers plugin in QGIS 2.x as an easy resource to add base maps to my projects. You could pull in layers from OSM, Google, Bing, and others. It turns out that plugin is no longer available for QGIS 3.x. So I searched around and found some suggestions for a different plugin called QuickMapServices which was a great replacement. But alas, that worked in QGIS 3.0 but is not compatible (as of now) for QGIS 3.2.

So I’m back to adding WMS layers manually. There is a new feature in QGIS for adding XYZ Tiles; this is a little better than WMS because the base map can be rendered a bit quicker. I found a tip in the Stack Exchange that you can add an OSM tiles layer with this url:

http://tile.openstreetmap.org/{z}/{x}/{y}.png 

Select XYZ Tiles in the Browser, right click, New connection, give it a name, add the URL. You can modify the X Y Z coordinates where the map centers and zooms by default. Once you’ve created the connection, you can simply drag the OSM layer into the map window to render it.

Adding the OSM XYZ Tiles in QGIS

One problem that always creeps up: when you add other layers and adjust the zoom, sometimes the rendering of the base map looks poor, i.e. the features and labels look blurry or blocky. When you’re pulling data from a web map layer, as you zoom in it swaps out the tiles for more detailed ones appropriate for that scale. But when you’re zooming in QGIS things can get out of synch, as your map window zoom may not be enough to trigger the switch in the map tiles, or those map tiles are just not meant to be rendered at that scale. If you right click on a blank area of the toolbar, you can activate the Tile Scale panel and can use the slider to adjust the window zoom in synch with the tiles, so you can operate at the scales that are appropriate for the tiles. The way points and track for our hike alongside Schnebly Hill Road are shown below, and the labels for the points represent our elevation in feet.

OSM Tile Layer with Tile Scale Panel

If the slider is grayed out, select the OSM layer in the Layers menu, right click, and select Set CRS  – Set Project CRS From Layer. Web mapping services typically use EPSG 3857 Pseudo Mercator as the coordinate reference system / map projection by default. If your other vectors layers aren’t in that system, you can have the base map draw to their system or vice versa by selecting the layer, right clicking, and choosing Set CRS. But for the tile scale to work properly EPSG 3857 must be the project CRS.

Lastly, I’ve always liked the USGS WMS layers, which are never included in the plugins that I’ve seen. The USGS provides layers for: imagery, imagery with topographic features, shaded relief, and the USGS topographic map layer:

https://basemap.nationalmap.gov/arcgis/rest/services

USGS Link for WMS Layer for Topographic Maps

You can click on one of the services, and at the top (in small print) are urls for their services in WMS and WMTS. The last one is a web mapping tile service, which is a bit faster than WMS. Click on the WMTS link, and copy the url from the address in the browser. Then in QGIS select WMS / WMTS layers, right click, add a new connection, give it a name and paste the url. This is url for the topographic map:

https://basemap.nationalmap.gov/arcgis/rest/services/USGSTopo/MapServer/WMTS/1.0.0/WMTSCapabilities.xml

Once again, you can drag the layer into the map window to render it, and you can use the Tile Scale panel to adjust the zoom. Here’s our hike with the topo map as the base:

USGS Topographic Map in QGIS

The Map Reliability Calculator for Classifying ACS Data

The staff at the Population Division at NYC City Planning take the limitations of the American Community Survey (ACS) data seriously. Census estimates for tract-level data tend to be unreliable; to counter this, they aggregate tracts into larger Neighborhood Tabulation Areas (NTAs) to produce estimates that have better precision. In their Census Factfinder tool, they display but grey-out variables where the margin of error (MOE) is unacceptably large. If users want to aggregate geographies, the Factfinder does the work of re-computing the margins of error.

Now they’ve released a new tool for census mappers. The Map Reliability Calculator is an Excel spreadsheet for measuring the reliability of classification schemes for making choropleth maps. Because each ACS estimate is published with a MOE, it’s possible that certain estimates may fall outside their designated classification range.

For example, we’re 90% confident that 60.5% plus or minus 1.5% of resident workers 16 years and older in Forest Hills, Queens took public transit to work during 2011-2015. The actual value could be as low as 59% or as high as 62%. Now let’s say we have a classification scheme that has a class with a range from 60% to 80%. Forest Hills would be placed in this class since its estimate is 60.5%, but it’s possible that it could fall into the class below it given the range of the margin of error (as the value could be as low as 59%).

The tool determines how good your classification scheme is by calculating the percent of estimates that could fall outside their assigned class, based on each MOE and the break point of the class. On the left of the sheet you paste your estimates and MOEs, and then type the number of classes you want. On the right, the reliability of classifying that data is calculated for equal intervals (equal range of values in each class) and quantiles (equal number of data points in each class). You can see the reliability of each class and the overall reliability of the scheme. The scheme is classified as reliable if: no individual class has more than 20% of its values identified as possibly falling outside the class, and less than 10% of all the scheme’s values possibly fall outside their classes.

I pasted some 5-year ACS data for NYC PUMAs below (the percentage of workers 16 years and older who take public transit to work in 2011-2015) under STEP 1. In STEP 2 I entered 5 for the number of classes. In the classification schemes on the right, equal intervals is reliable; only 6.6% of the values may fall outside their class. Quantiles was not reliable; 11.9% fell outside. If I reduce the number of classes to 4, reliability improves and both schemes fall under 10%; although unreliability for one of the classes for quantiles is high at 18%, but still below the 20% threshold. Equal intervals should usually perform better than quantiles, as the latter scheme can make rather arbitrary breaks that result in small differences in value ranges between classes (in order to insure that each class has the same number of data points).

Map reliability calculator with 5 classes

Map reliability calculator with 4 classes

You can also enter custom-defined schemes. For example let’s say you use natural breaks (classes determined by gaps in value ranges). There’s a 2-step process here; first you classify the data in GIS and determine what the breaks are, and then you enter them in the spreadsheet. If you’re using QGIS there’s a snag in doing this; QGIS doesn’t show you the “true” breaks of your data based on the actual values, and when you classify data it displays clean breaks that overlap. For example, natural breaks of this data with 5 classes appears like this:

24.4 – 29.0
29.0 – 45.9
45.9 – 55.8
55.8 – 65.1
65.1 – 73.3

So, does the value for 29.0 fall in the first class or the second? The answer is, the first (test it by selecting that record in the attribute table and see where it is on the map, and what color it is). So you need to adjust the values appropriately, paying attention to the precision and scale of your numbers. In this case I bump the first value of each class up by .1, except for the bottom class which you leave alone:

24.4 – 29.0
29.1 – 45.9
46.0 – 55.8
55.9 – 65.1
65.2 – 73.3

In the calculator you have to enter the top class value first, and just the first value in the range:

65.2
55.9
46.0
29.1
29.4

Map reliability calculator with user defined classes

In this case only 7.1% of the total values may fall outside their class so things look good – but my bottom class barely makes the minimum class threshold at 19.4%. I can try dropping the classes down to 4 or I can manually adjust this class to see if I can improve reliability.

If you’re unsure if you made the right adjustments to the classes in translating them from QGIS to the calculator, in QGIS turn on the Show Feature Count option for the layer to see how many data points are in each class, and compare that to the class counts in the calculator. If they don’t match, you need to re-adjust.

QGIS natural breaks and feature count

This is a great tool for census mappers who want or need to account for issues with ACS reliability. It’s an Excel spreadsheet but I used it in LibreOffice Calc with no problem. In addition to the calculator sheet there’s a second sheet with instructions and background info. Download the Map Reliability Calculator here. You can try it out with this test data,  workers who commute with mass transit, 2011-2015 ACS for NYC PUMAs.