Now for something a bit off the beaten path for this blog. A couple of years ago, during my enforced period of inactivity, I began a personal due diligence project that I mentioned in passing: Building a database of information on potential retirement spots. With California continuing its descent into a nanny state tax hell for both business and individuals, that information may get used sooner than later, so I've taken advantage of the current recuperation (which is going well) to revisit and update that database. I'm sure I'm not the only one engaged in such a thought process, so for those who are starting or in the middle of the job and find they way here through Google, here is an annotated list of some of the more generally applicable data sources that I've used, roughly sorted by category. General caveat: I am a spreadsheet-wielding, database wonk, computer scientist and systems analyst trained, libertarian venture capitalist and manager. The types of data and their application are biased accordingly. Season to taste.
General Sources
The well-known Places Rated Almanac is my starting point. It covers essentially all of the US metropolitan statistical areas. Unlike some ratings books, it provides most of the raw data that went into its category ratings (though not the formulas). Like any printed source, it has problems with aging data. Particularly the employment and real estate cost information have been turned into garbage by the Great Recession, though relative rankings might still be of some use. But the amount of sheer grunt work involved in compiling (for instance) the variety of entertainment and cultural resources documented make it worth the price. If you are constructing your own evaluation model, you can often find a piece of proxy data here that can be used until you find a better source. The biggest drawback is that large metro areas are lumped into one entry - if you want to dissect an area such as Portland Metro down to the county or town level, you'll have to go back to primary data. For simplicity, I abbreviate this source as 'PR' below.
Retirement Places Rated, from the same group, is the little brother of PR. Its strength is a concentration on locations where retiree demographics are growing. There is substantial overlap with PR, but enough different, smaller locations to serve as a source of ideas and data. The weakness is that fewer data points are given than in PR, and you may need to go to primary data to fill in the gaps, depending on your interests. I abbreviate this source as 'RPR' below.
There are two prominent online competitors for general location information, City-Data and Sperling's Best Places. Both provide data in depth on a large variety of places, even small locales. Some of it is obviously scraped from the US Census, FBI and other government sources. Other sourcing is often obscure or undocumented. Of the two sites, I prefer City-Data. Its data page formatting is old school and rather clunky, but easy to use once you master the layout (and likely would be easy to scrape and parse, should it come to that.) However, City-Data's biggest advantage is its large and diverse forums. Most locales of 50,000+, and some smaller, have their followers including boosters and nay-sayers. Since, as I've observed elsewhere in a different context, the best way to learn something is often to watch experts argue, reading several months of relevant posts is a good way to get some of the feeling of a place. I prefer this general bulletin board to those specifically oriented towards retirement, as it is more likely to bring out the individualistic, quirky aspects of a location that are hard to capture in a database.
I've sorted my specific source data into eight model categories, which don't necessarily match those in PR or RPR.
Climate
PR and RPR are both quite good on climate data, giving a numerical score for both summer and winter conditions, and a score for 'seasonal affect', roughly equating to how much sunshine you get. Annual trends in temperature and precipitation are also provided. There's also a risk score that apparently sweeps in everything from blizzards to earthquakes. The one thing I found wanting was humidity information, since I tend to wilt in muggy climates. Dew point maps available from NOAA's climate mapping service are a useful supplement in that regard.
I've tried to find systematic ratings of airborne allergens and biting insects by location, but to no avail.
Crime
Both PR and RPR provide FBI crime rate statistics. Probably good enough for most purposes, though a few years old. You can get updates, or information for other locales from the FBI itself.
Cost
The real estate data from PR and RPR has been turned to rubbish by the crash. The City Data pages provide historical sales information for some communities, but the quality is very inconsistent. Some locales have up-to-date time series of median sales prices, while some are missing recent years or have sales counts so low that the data may be worthless. A median price also tells you little about the spread of prices and where your choice of housing falls within it.
I started with PR and RPR data to get relative rankings, and then got more recent numbers from the big three real estate sites: Zillow, Trulia and Hotpads. Be sure to look at closed sales; many 'current' listings at all three are quite stale. Zillow's "zestimates" are worthless, and should be ignored. Also be sure to look at foreclosures and short sale concentrations to get an idea of probable stability of a market. (If you'd be renting, start with Hotpads, since it specializes in the rentals.)
A source for general tax information is America's Best Low-Tax Retirement Towns. It provides state level guides for sales and income tax, and drill-downs to some, but not all, metro areas to show real estate and additional sales and income taxes. This book does have one serious flaw: It uses the same income and housing cost brackets for all locales. This is obviously the wrong thing: $500,000 will buy you very little in the Bay Area real estate market, or a mansion and a spread in the Midwest. Likewise with general costs of living with respect to local income levels. What you have to do is back out a tax rate from this source and/or PR and RPR's Housing information, and apply it to the likely property value you determined above. Once you've narrowed to a few places, you can rough out a budget and figure out the income and taxation implications for each.
PR and RPR have estimates of utility costs, which you can inflate or deflate based on your guess of relative square footage of what you'd buy or rent compared to the average home.
For those who are self-employed or retiring early, you can pick up average health insurance cost by state at the American Health Insurance Plans site, and scale it to fit your circumstances (age and number of persons).
Healthcare
PR provides physician counts by specialty, and counts of hospital beds. RPR provides only a total of physicians and the number of hospitals. You can look up more detail on hospital's bed counts and services at the American Hospital Directory.
It's also worth knowing the rate of Medicaid usage in a locale. Medicaid generally reimburses providers at less than their costs. The resulting deficits are typically dumped onto patients with private insurance, driving up its costs and presenting an ongoing burden on the local medical establishment. You can get Medicaid data at the State Health Facts site.
Leisure
What goes here will depend on what you think is fun. PR and RPR provide a variety of raw data under Recreation and Ambiance categories. I found available acreage of public lands and waters, and performing arts bookings to be useful. If you care about learning (or teaching) in a university setting, both provide university names and student counts. If you envision a lot of travel, consider airport and airline availability.
If you have hobbies that are important to your happiness, you can often find an online data source to compare availability. For instance, I checked up on the number of geocaches and the availability of shooting ranges in each locale.
Community
Another category that will depend on what you care about. If that includes ethnicity, PR has raw data in its Appendix B, or you can get more detail at City-Data. All are based on census information. You can get a relative score for educational level and affluence from the same appendix in PR, or primary data on educational attainment and income distributions at City-Data, and decide how to score it yourself. I also used a score for traffic congestion from PR's Transportation section; even if you're not commuting, packed roads affect the availability and value of all community activities.
Community also includes what's available in the local markets, so I threw in the 'Good Eats' ratings from PR. RPR unfortunately uses a different scoring system for restaurant quality and availability, so you have to use locales covered in both volumes to figure out a normalization between the two. Having turned into a wine snob during our time in California, I was pleased to find that most every state now has a winery association website that gives some idea of numbers of establishments and often some basis on which to guesstimate quality.
Many 'leisure' activities are really excuses for socializing, and will make for an easier transplant if they are available in a new locale. Online data sources will be idiosyncratic to the activity, but MeetUp is a good way to check if there are already groups of interest or with similar tastes in the area. If you're of a religious bent, don't forget to check on relevant congregations - most every faith and sect has an online presence these days.
Economy
The recession has turned the job growth estimates from PR into scrap. They shouldn't even be trusted for relative rankings, since the loss of jobs has not been uniform geographically or by sector. Fortunately, the New Geography group has compiled a set of jobs ratings for metro areas since the downturn began, and unlike PR, provides a description of the methodology. Hopefully they will keep this updated. Unfortunately, it doesn't cover all the locations in PR and RPR, but you may be able to do some geographic interpolation.
Both PR and RPR provide 'job quality' scores. These may still be useful since the type of work available in an area typically doesn't change quickly.
If you're going to be working after relocation, or anticipate part-time or seasonal jobs in semi-retirement, you'll want to look more closely in your field. The usual job posting sites are probably the best source. Some data points from PR or RPR could be relevant depending on your situation; since my wife sometimes works part-time in schools, I used a combination of the public and private school support ratings from PR's Education section.
If you're the entrepreneurial sort, or contemplate self-employment, you'll be interested in the State Business Tax Index compiled by the Tax Foundation. Look at their site for the latest version.
Another concern for would-be businessmen and general economic health is the level of state unemployment debt to the Federal government. Many states have exhausted their unemployment benefit funds, in some cases due to Federal or state extensions of benefit periods, and are in debt to the Feds for billions in loans that have been extended to cover the shortage. This might eventually be written off at the Federal level, in which case the resulting increase in the public debt is an equal problem no matter where you are located. If the debts are eventually paid as general obligations by the states, then they will be a budget burden for all residents, creating upward pressure on taxes. If, as typical in the past, they result in an increase in unemployment insurance rates to surviving and new businesses in the debtor states, then this will increase the fixed costs of hiring and create downward pressure on employment and economic activity for years to come. These numbers are changing rapidly, so seek out a fresh set. Update: The source data is at ProPublica, and is updated weekly.
Government
Being motivated by a dislike of taxes, regulation and nannyism, I'm concerned to avoid jumping from the frying pan into the fire, in a location that later turns into another California-style debacle. Other than a single liberal/conservative rating, PR and RPR are silent on political climate, as are the general information websites. If you're trying to avoid statism (or seek it, if you like that kind of community) you'll need to do your own digging. I've found a number of useful surrogates for current and potential future political and government finance risks.
A starting place for general political environment is the Cook PVI (partisan voting index) scores for Republican vs. Democrat leanings. These attempt to back out the benefits of incumbency and other candidate-level factors to yield an underlying affinity at the level of congressional district and state. They are updated every two years. Congressional district borders seldom align neatly with census statistical areas, so you will have to use some judgement in interpretation. YMMV as to how to evaluate this; I prefer 'purple' states where close election contests keep both sides relatively honest and away from indulging the desires of their extreme fringes.
A highly useful source for state-level political outcomes is the Freedom in the 50 States report from the Mercatus Institute at George Mason University. This provides an overall score as well as specific score for Fiscal Policy, Regulatory Policy, Economic Freedom and Personal Freedom. Each of these is supported by a detailed factor model that is fully described in supplementary documentation, with separate tables of the factor 'loadings' for each state. If you care specially about particular issues, for instance, policy towards gun ownership, you may be able to find a specific statistical rating for them.
A more narrow study is the Pacific Research Institute's US Economic Freedom Index, again at the state level. 2008 is the most recent version, which is available as a free PDF download.
The government is good for some things - US Census data can give you a current picture of governmental burden, since it compiles state and local government statistics. You can easily find total state and local employee counts, and total expenditure levels. Get the estimated state population corresponding to the most recent year and you can compute per capita government spending, and fraction of inhabitants that are on the public payroll. Remember that 'local' in this case means that all local jurisdictions in the state are summed up. This is still likely useful for comparison purposes, but will bias urbanized states upward compared to those with fewer cities. If you want to drill down to specific metro areas, you'll need to use state government only figures from the Census, and then augment with employee and spending figures from the locale's public disclosures.
Another useful government burden indicator is welfare caseload per capita. The raw caseload data is available here. Illegal immigrants also create a government burden by using more public services (e.g, education, healthcare) than they provide in taxes. DHS sporadically updates an estimate of illegals by state, for those with the highest concentration. The latest survey dates from 2006 (PDF).
Accumulated government debt is a recipe for higher taxes for fewer services in the future. State general obligation bonds are rated by three agencies: Fitch, Moody's and S&P (Standard and Poors). There don't seem to be compiled tables of ratings by state, at least in a stable location. However, all of the agencies issue press releases when they take a rating action, so just pick one of them (their rating systems differ) and Google that name along with the state of interest. There's an explanation of the rating systems at Wikipedia. The agencies' notoriously over-optimistic ratings of mortgage backed securities were one cause of the current recession, so take their state ratings with a grain of salt, but they should be useful for relative ranking.
Agency ratings encapsulate some amount of ability-to-pay information, but if you want to get more explicit, the same Census state and local government statistics pages include data on state and totaled local debt burdens, readily convertible to per capita figures. Municipal debt rating and amounts will require separate research; if you do business with a brokerage it may be able to help with muni bond ratings.
Update: Many states not only suffered a sharp fall in revenues in 2009, but delayed the day of reckoning for budget cuts and public employee layoffs by using Federal stimulus money to fund their budget. With revenue drops continuing, and another dollop of government cheese unlikely, there are now gigantic gaps between income and proposed outgo in many states. You can find out how bad it is at the Pew Center's 'Beyond California' study. Take a look at the 'budget gap' column. You can ignore Pew's 'score' as it includes whether or not it requires a supermajority to raise taxes in the state. That's important to bond holders, and is presumably embedded in the ratings above, but presumes that the way out of the mess is to shake down the taxpayers, rather than lay off bureaucrats. (These numbers are also volatile, and many states won't receive the revenue they are forecasting in 2010.)
Underfunded pension funds for public employees have become a notorious problem, through a combination of political give-aways to unions and shorted contributions, compounded by the crash in the funds' values. Note that in most cases this actuarial shortfall is not included on governmental balance sheets, and you will have to dig deeper to find it. The National Association of State Retirement Administrators compiles statistics voluntarily submitted by retirement funds. Their latest survey is from 2008 (PDF), though much of the data is earlier. It contains most state general, educational and safety employee retirement funds and a few municipal funds. Given the market downturn since the reporting dates, the actual funding shortfalls will have only gotten worse, but the relative levels should still be useful. Both percentage coverage and actual dollar values are provided, in case you want to compute a per capita liability for retired bureaucrat pensions. Note that this burden will be relatively less important in a locale where population and the economy are growing, and more so where there is shrinkage of population and economy over time, e.g., Michigan. Update: The Pew Center has compiled a more up to date assessment of unfunded liabilities in state pension and retirement benefit plans. A little math is required to combine the two into an overall funding ratio, whereupon you'll find that states like Kentucky, New Hampshire, and West Virginia have funded half their commitments - severe budget pain and/or repudiation of promises make to retired public employees lies in their future.
"Nannyism" is hard to score numerically. Some of the factors in the Mercatus survey may be useful, depending on what types of government intrusions annoy you. If restrictions on gun owners and alcoholic beverages are on your list, PR has a compact table of laws by state, and Wikipedia has a comprehensive table of state liquor laws. If you care about land use restrictions such as urban growth boundaries and zoning policies, it's back to digging into public records and Google on a city-by-city basis.
Politics and Migration
While not sources of primary data, there are two books of analysis of population and economic and political patterns that are worthwhile reading for someone contemplating a move. Those are Andrew Gelman's "Red State, Blue State, Rich State, Poor State" and Bill Bishop's "The Big Sort". The former is a compilation of statistics with analysis, the latter is more a think-piece about political polarization and migration. I find them useful, not because you need to become a political scientist to consider retirement or relocation, but because they raise pointed, sometimes uncomfortable issues about one's own desires and comfort zones. Worth cranking into the thought process while you consider what data is worth assembling.
Being There
None of this is a substitute for seeing the locations and people yourself. Maps are not the territory, and abstracted and aggregated figures are even less so. My wife and I have started visiting some of the places that are top rated in our own model, and find that in over 50% of the cases, we decide within a day or two that it's not for us, for a variety of reasons. The good news is that, as with house-hunting, you do get more efficient with practice. And the worst thing that will happen is taking a series of trips to interesting spots you might never have visited otherwise.