Lies, Damn Lies, and Whatever This Is
There are many ways that statistics, data visualization, and language can be used propagandistically. One of the pernicious behaviors of our current wave of Abundance™️ YIMBY-brained technocrats is to engage in this in a completely obfuscatory way. This is a group of people who love to rationalize their market-driven, pro-capitalist policy preferences as objectively correct by browbeating their audience with math and language that they themselves frequently misunderstand.
A prime example that prompted this post is one that falls into my decade-long frustration with the intellectual laziness of the astrorturfed YIMBY movement. For a few years now, a repeated talking point amongst this crew is the repeated insistence that Austin, TX somehow represent a success of deregulatory YIMBY policies backed by a mostly supply-centric view of ₛᵤₚₚₗy ₐₙd dₑₘₐₙd dynamics.
The recent report from Pew is a prime example of the typical sort of intellectually-browbeating pseudoscience trying to justify this mythology, which is ultimately only supported only by facile chart worship and bad statistics. In what is dishonest and extremely on-brand, the title of this article is “Austin’s Surge of New Construction Drove Down Rents”. For a normal reader, the implication is:
- There was a “surge of new construction” in Austin over some time period.
- “Rents”, in terms of the particular way that they decide to measure them, are now lower.
- The cause of that decrease in “rents” is the “surge of new construction”.
They want to use this to broadly advocate for a deregulatory approach to “unlock housing supply”, implying that the primary impediment to affordability is a lack of supply and allowing for the price-reducing mechanisms of supply and demand to work their magic. They say as much explicitly, which more-or-less represents the Abundance™️ YIMBY party line:
Austin’s success serves as an important example of how regulatory barriers to building more housing are often varied and interconnected. No single solution can solve a housing shortage, but Austin has taken multiple steps that have helped to unlock large amounts of housing supply in its market and reverse rent growth, including rent for tenants of lower-cost, older apartments. The city continues to take forward-looking steps—among them reforming building codes, streamlining permitting, and facilitating the construction of small apartment buildings—to reduce housing underproduction and improve affordability for existing and future residents.
In Obamanian terms, let me be clear: this is entirely unsupported by any evidence presented in this article. There’s a lot to say, but given e predilection of the crew in question to do chart worship, let me just give a few such examples that indicate some of the broader problems.1
A Very Bad Chart
The title of this chart “Housing Affordability in Austin Has Improved as More Homes Have Been Built” is a masterful example of how to lie using statistical implication. There are a number of reasons why this is dishonest:
What’s in a Name?
“Housing Affordability” here is equated with “relatively lower increase in rents”. Of course, if rents were already unaffordable, then slowing their increase does not “improve housing affordability”. The scale of the supposed rent decline is related to the so-called “ratchet effect”, where housing prices are sticky downward (a fact that everyone who has looked for apartments knows).
Indeed, in Austin, the prior decade showed massive growth in Austin’s rents. The figures in the article itself mention this - Austin’s 2026 median rent was substantially above its pre-boom level. Things were already unaffordable for many for a while.
It’s also worth noting that the scale of this rental increase also corresponds to a marginal decline that likely doesn’t change the landscape of affordability - a rent drop from $1,546 to $1,296, say, does not make housing particularly accessible to households that were already priced out at the higher rate. They already left or have to live in existing subsidized housing, wherein this rent reduction amounts to a discount for people that could already afford the $1,546 rent.
The claimed figure in the Pew article makes this even clearer: they claim that affordability improved from 95% AMI to 84% AMI for a one-bedroom. This means that rent remained unaffordable to anyone below 84% AMI, which doesn’t sound like a Triumph Of Affordability to me.
Grouping
The other aggregates presented are arbitrary and completely decontextualized. The presentation of similar statistics in the broader U.S. and in the state of TX is meant to imply that Austin is meaningfully doing better by some universally coherent metric (“Housing Affordability”).
There are about a million contextual geographic and temporal factors that need to be taken into account to consider The City of Austin as a comparable geographic unit to The Entire United States. These are obviously not the same type of thing.
This is what we might call a category error: cities, states, and countries are simply not the same type of thing! There’s no reason to expect, therefore, that even sensibly-defined statistics associated to them might be comparable without some sort of significant adjustments that reflect these differences.
As a matter of statistics, the choice of how to group things together can overwhelmingly change the results of calculations. See the examples associated to Simpson’s Paradox, wherein the choice of aggregate can completely flip the relationship between two things.
Time After Time
The time period is both truncated and aggregated. We don’t get any indication of what happened before 2021 (were there existing trends in Austin that should be taken into account?!), what happened since then, or even how things were happening per-year during those years.
This has some very clear problems. If Austin rents peaked near the beginning of 2021, then it’s necessarily the case that rents declined since then, and what happened before 2021 matters quite a bit. Indeed, in Austin, rents in Austin nearly doubled from 2010-2021, and so there might be a partial explanation of the downward trend as a “regression to the mean”.
This sort of aggregation also ignores the nature of building construction, which happens on a lag associated to things like permitting, physical construction, etc. As presented, the timing of construction completion and rent movements is obscured, which makes it difficult to look at things at the resolution needed to check for any causal relationship.
Garbage In, Garbage Out
The data source for rent data is biased and based on an aggregate that collapses too much to be useful. This comes from Apartment List, which is a for-profit company that has their own estimation methodology for a “Rental Estimate” price index associated to cities.
One could write pages about the flaws in the methodology associated to these estimates, which I might leave for another post, but for now I’ll just try and summarize some of them. This does necessitate a bit of technical discussion, but that might be unavoidable in expressing the problems with the use of these statistics.
First, let me just say what the “Rent Estimate” actually is: it attempts to estimate what a new tenant would likely pay to rent an apartment in a given city. It does this by taking a Census-derived baseline rent and projecting it forward using a growth rate calculated from repeated observations of the same units being leased multiple times on their platform. Let us be clear that this is not a measurement - the actual transaction prices are never observed in their data.
The most obvious problem comes from how they source the underlying data around prices.
Their rental prices and availability data come from “verified partners” and “a number of listing feeds”. Given how different housing markets are across cities, states, and the whole country, why in the world would we expect “estimated median prices of whatever subgroup of housing units Apartment List decides to track” to be a useful cross-geographic, cross-market comparison? There are a lot of apartment rentals in Austin, sure, but what about markets that have more Single-Family Housing?
The Apartment List crew claim to adjust for a specific “luxury bias” using incomplete Census data, but that cannot address the broader issues coming from flattening segmented markets that are segmented in different ways across geographies. After all, in SF, no one sensible would think to consider SRO rentals as easily comparable to new luxury condo rentals.
This makes their estimates of the absolute figures for their estimates questionable - the bias can shift the actual numbers that we are presented with.
Besides that, there’s the typical “mathwashing” of these sorts of statistics - the use of arbitrary, complicated, and limited-scope mathematical models to produce deceptively-named statistics like “Rent Estimate”.
If you dig into the methodology of Apartment List’s “Rent Estimate” calculation, you’re presented with this:
I have a PhD in mathematics and quite a bit of professional/academic experience in doing statistics (applied to house pricing, even!), and it took even me a little time to unpack the technical aspects of this model and see where it could be problematic. Without going into too much detail, the model makes a number of assumptions that, if violated, might make the entire thing useless:
- The last observed price of a unit is approximately equal to the transaction price.
- Removal of units from listings implies that there was a new lease. This might just come from gaps in the data.
- Parallel growth trends across unit types (e.g. luxury units had similar trends to other units). This impacts the growth rate, which is primarily what the authors are interested in.
- Census figures provide correct baselines.
- 3 month moving averages correctly reflect the price dynamics. This is an arbitrary time frame, and for a market which experiences shocks and seasonality, might not make sense at all.
- Units that repeatedly show up are reflective of the overall market. There might be selection bias associated to units that get relisted.
- There are no omitted variables associated to things like renovation, amenities changes,etc. This is almost certainly not the case, many units are relisted after changes to the unit happen in order to drive up the price.
- The adjustment for recent movers is correctly calculated.
- The standard mathematical assumptions for OLS regression are satisfied in order to make this mathematical model sensible. This is dubious.
The results of violating these assumptions may cut both ways: some would tend to produce overestimates and some would produce underestimates. The overall result is that the conceptual uncertainty around these assumptions produces completely unquantifiable uncertainty in the estimate. For something that’s meant to be a consistent index, this is potentially a fatal flaw. The article presents this unknowably-uncertain estimate as an objective measurement, which is incredibly dishonest.
I want to stress that flaws in this particular statistic invalidate huge swaths of this analysis - this non-measurement of price underlies many of their claims that they use to support their base argument that Austin’s rental prices dropped at all.
Causality
Even if you accept the premises of this data and chart being somehow valid, the causal implication that they want you to take from this is not remotely justified. The implication of the title is that because more homes have been built, housing affordability has improved. This is logically flawed.
First, there are obvious and unaddressed phenomena lurking in the background: a shift in the tech-centered migration patterns after 2022 to/from Austin brought population growth down steeply and shifted the income demographics of people moving to the city.
Second, there’s the lack of obvious comparisons. A causal claim requires a counterfactual; in this case, the best we might hope for is an extremely similar city that was more or less identical to Austin besides the supposed choice to Build More. However, Austin was already an extreme outlier in the years building up to 2021, making any sort of national trends a poor choice of comparison. The best we might hope for are similar high-growth Sun Belt cities that didn’t build as much, but even there, the uniqueness of Austin’s shift in demand makes comparisons difficult.
Besides all that, there’s the basic issue of using this sort of correlational measurement at all. See Post Schlock Ergo I Post Drop.md for more details about this specific type of error in the context of causal inference, but at the end of the day, the only actual way to determine a causal relationship is via some form of experiment that involves randomness or quasi-experimental data that has been treated appropriately. The data being used here simply cannot provide this sort of analysis.
An Even Worse Chart? Oh Damn
A big part of the rhetoric and ideology associated to Abundance™️ is an insistence that their deregulatory, market-based solutions are actually beneficial to those with lower incomes, the working class, etc. The Pew article attempts to make this point with the following (very silly) chart.
Once again, we see problems from the previous chart:
- Truncation of the time period to 2017.
- The title implies that whatever building Austin did was the cause of this affordability increase. The logic of this is “because number go down after thing happens, thing did number go down”.
The point-in-time figures that are given here are even more egregious, given the time period that they wrap around. 2017-2024 includes the entire global COVID-19 pandemic. For those who forgot (!?), a bunch of things happened during that time:
- Millions of people died or were made ill by an infectious disease.
- Sectors of the global economy shut down or was stifled in a number of ways.
It doesn’t take much to recognize that Some Other Stuff Happened during this time period which may have changed rental prices in Austin.
It’s difficult for me to say that it’s anything but ignorant or dishonest to see this before-and-after number and claim that it’s somehow easily-attributable to some uniquely-Austin building boom.
The Upshot
There are some points here that I want to be absolutely, 100% clear on:
- The YIMBY argument is oversimplified nonsense.
- Charts like the ones presented above provide absolutely no evidence of anything.
- These criticisms also need to apply to analyses that we like the conclusions of.
I also gently suggest that many people who read these articles would do well to have a bit of epistemic deference: if you cannot understand the technical aspects of this calculation, you are not qualified to judge its trustworthiness. As a result, you simply should not take these numbers at face value, and you would be better off reading more into the more obvious political economy associated to the motivations of those who are presenting these numbers to you.
Footnotes
Footnotes
-
Some of the reasons here will be rehashing of some of the important qualitative points in Harlo Pippinger’s broader article, which you should read. There are important local factors associated to Austin’s development that absolutely need to be considered, and he gives an excellent overview of several. ↩