taw's blog

The Curious Case of Senate Filibuster

taw — Wed, 23 Dec 2009 09:39:00 -0800

American Constitution states as clearly as it ever states anything, that to pass a law majorities of House and Senate are required, and doesn't know anything about filibuster process - this silly Senate regulation in which 60 votes are required to even bring anything to final voting.

And it would be so easy to get rid of such rule - mere 50+VP majority in Senate can change Senate procedures. Even more curiously - it would clearly be in interest of the majority party - be them Republicans or Democrats - to get rid of such silly rule. The argument that they might want to keep it for the future, as they might end up in minority later is obviously wrong, as your Democratic/Republican Senate keeping filibuster doesn't do anything to stop the next Republican/Democratic Senate from abolishing it.

Persistence of such major non-Constitutional feature which clearly favors the minority party is extremely curious.

Voting Power

The mystery will be resolved soon. For now let's step aside for a moment and imagine a Parliament of a fictional European country with four parties - Armadillo Party, Bear Party, Cat Party, and Dog Party, with 45%, 45%, 6%, and 4% representation in their Parliament. As it's a fictional country, let's assume perfect party discipline, and simple majority required to pass anything. How important are the parties?

Naively looking at their statistics, it would seem that Armadillo and Bear parties should be vastly more important than Cat Party, which should be just slightly more important than the Dog Party. This is entirely wrong. To pass anything you need one of the following sets on board:

Armadillo Party and Bear Party
Armadillo Party and Cat Party
Bear Party and Cat Party

The Cat Party with its 6% of vote has as much power as Armadillo and Bear parties with 45% each! And Dog Party with its 4% has no power whatsoever - its support makes no difference for whether a bill will pass or not!

This highly counterintuitive result is mathematized as two "power indexes" - Banzhaf power index and Shapley-Shubik power index. They differ in minor details, but basically they tell you how important are different players in a complex voting system.

Now there's important caveat with voting power indexes - they assume interests of parties are entirely independent - Armadillo Party is as likely to agree with Bear Party as with Cat Party as with Dog Party - something which is not true in the real world at all. No see why it is important imagine a Bear-Cat coalition - having 51% of votes together they have 100% of power, while Armadillo and Dog Parties become entirely irrelevant. Why wouldn't Bear and Cat Parties want to start one? Mostly because coalition victories won't do you any good if you don't agree with other coalition members. Bears might push for taxpayer subsidies for honey production, something that Cats would much rather not have - and could shut down the idea together with Armadillos if they weren't in this silly Bear-Cat coalition. Increased coalition's voting power is usually countered by having to vote for things coalition wants that you're not terribly enthusiastic about.

Anyway, let's look at American system, both as its specified in the Constitution, and as it works with the filibuster. USA is no Europe, so we cannot assume any Party discipline - let's go further and assume every Representative's and every Senator's votes are entirely uncorrelated. This isn't true for many issues which follow left-right party lines, but is very much so for the highly essential pork barrel spending and other things they do for lobbyists in exchange for campaign contributions. The only exception to the independence rule I'm going to make is assuming that Vice-President always does what the President tells him to, and is no independent actor. If there were any votes in history in which they disagreed, I'd like to know about it.

Assuming I coded it right, Shapley-Shubik voting powers of the President, individual Senators, and individual Representatives in Constitutional system are 16.66%, 0.391%, and 0.102% respectively - president is worth almost 43 Senators, and a Senator is worth 3.8 Representatives. By the way that's why elections for President - supposedly "executive" function - focus primarily much on legislative issues. American President has large levels of control over legislative and judiciary branches of government, in addition to his executive responsibilities - as extremely unhealthy concentration of power.

Now with filibuster, the situation changes drastically - President's power drops to 9.115%, Senators' powers increase to 0.716%, and Representatives' plummet all the way to 0.044%. Now President is worth only 12.7 Senators as opposed to previous 43 - and more what's even important - each Senator is worth over 16 Representatives!

Numbers are considerably different with Banzhaf index - it's 3.996%, 0.295%, 0.153% Constitutionally and 2.468%, 0.564%, 0.094% with filibuster - but the story is the same - filibuster greatly increases Senator's powers. Knowing that it's not terribly surprising that Senators are not too eager to abolish filibuster - they would be giving away a lot of influence with it!

This result might be surprising at first, but there's straightforward intuition behind it - if there are multiple requirements for a law to pass - the one which is most difficult determines the process - Constitutionally passage in House and in Senate are equally difficult, giving each of them equal powers - but if Senate decides to make passage harder - it will be very unlikely for any bill to have 60% support in Senate without also accidentally having at least 50% in the House.

State Size

But wait - there is more! Small states are massively over-represented in Senate, while having only fair representation is House of Representatives. By unduly strengthening the Senate, filibuster increases power of small states. By how much? As situation becomes a bit too difficult for exact methods, I ran a quick Monte-Carlo simulation of Shapley-Shubik voting powers, assuming all states vote as blocks - so we have 50 states voting, each with 2 Senate votes and some number of House votes; plus independent President.

Constitutionally President has 14.88% power in such case, the most populous state California 8.08%, while the least populous states with one Representative like Wyoming has 0.79%. That's a huge unfair and undemocratic overrepresentation already - California has 69x the population of Wyoming, but merely 10x as much power. With filibuster it gets far far worse - President's power drops to 9.49% - not surprisingly since more of it goes to Senate, but now the difference between California and Wyoming is merely 5.10% to 1.34% - 3.8x. Californians are underrepresented relative to Wyomingites by a factor of 18x. Not terribly democratic, is it? Actually I'd love to discuss the point that less democratic systems like American are not in any obvious way less effective at solving real world problems than more democratic systems like let's say German - but let's leave that for another post.

Even if you're from a small state, you shouldn't be getting too happy - small state Senators are the most corrupt members of Congress, and the baseline isn't terribly good either. There's the usual disclaimer that power indexes refer to idealized situation in which everyone votes independently from everyone else - which describes pork barrel spending and miscellaneous lobbying much better than it describes issues of economy or abortion. Still, let's not be as naive as Al Gore about the Senate - if this lobbyist-infested place couldn't even get popular, life-saving, deficit-reduring, and sensible from every other possible point of view public option, what are the chances of it passing expensive and unpopular climate bill which will only benefit some Bangladeshis decades from now? Nobel Prize offers no protection from wishful thinking.

Misunderstanding statistical significance

taw — Wed, 16 Dec 2009 11:40:00 -0800

People are natural cargo cultists - they (I'm saying "they" as I'm secretly a cat - just don't tell anyone) have natural tendency to focus too much on superficial characteristics - if something is good, everything vaguely associated with it must be good too! Or if something is bad, everything vaguely associated with it must be bad. So as people got into their heads once that Soviet Union and its "socialism" were "bad", everything "socialist" must be "bad" - like single payer healthcare, financial system regulations, and minimum wages. Never mind that even the basic premise is wrong, as performance of Communist economies was fairly decent, and as research says, Soviet growth performance was actually slightly above the global average in the period 1960-89, at 2.4 percent per year. Eastern Europe's growth was somewhat lower than USA's, Western Europe's, or Japan's, but it managed to beat by huge margins growth in capitalist countries of Latin America, South Asian, or even UK for that matter. Not to mention China vs India, or spectacularly good performance of the "socialist" Scandinavia. Or how pre-Reagan socialist USA was much better off economically than after Reaganite revolution.

Anyway, science. Two practices of science get fetishized to a nauseating degree. They are peer review and statistical significance. I cringe every time some (let's generously say "misinformed") person says that peer review is somehow central to science, and something being "peer reviewed" is somehow evidence of quality. The only thing peer review does is enforcing standards of whichever field the research is in. For example while Dialogue: A Journal of Mormon Thought peer-reviews its articles, I doubt they're worth that much more than Youtube comments. And creationists have their own peer-reviewed journals too! And in spite of widespread use of peer review, there's very little evidence that its variations affect quality in any way.

Most disciplines are not as bad as Mormon theology or young-Earth creationism, but standards of the field have nothing to do with solid science. First, there's a problem of entirely arbitrary standards which simply waste everybody's time like this and various incompatible citation rules:

I remember when I first started writing on economics, I was scolded for formatting my papers in a two-column, single-spaced format. While that format was common in computer science, to be taken seriously in economics a paper must be formatted as single-column, double-spaced.

But all academics eventually learn those. A much greater problem than these time-wasters are standards which are actively harmful, like...

Statistical significance

Probably the single biggest problem of science is appalling quality of statistical analysis. As a funny anecdote, I remember back at the University when I had to go through a bunch of database optimization papers - which implemented various performance hacks, and measured how well they do - and more often than not they did it on completely bogus data generated just for this case. If you spend even five seconds thinking about it, it's obviously wrong - all non-trivial optimizations highly depend on characteristics of data access patterns, and differences in results are many orders of magnitude. But for one reason or another using realistic data never got to be part of "good" database optimization research.

For one historical reason or another, almost all science got affected by obsession with "statistical significance". Typical research goes as follows:

Gather data
Throw data at some statistical package
Keep tweaking "hypothesis" until you get "statistically significant" answer, and send for publication
If no "statistically significant" answers are obtained, usually forget about the entire thing, or maybe send for publications anyway and hope it gets accepted
Watch people misinterpret "statistically significant" as "almost certainly true" and "not statistically significant" as "almost certainly false"

But of course, "statistically significant" results are false very often. Assuming perfectly run research, which is never true, 95% statistical significance only guarantees at most 1 false positive per 20 tested hypotheses (well, independent hypotheses, but let's skip complications here). So if you tweaked your hypothesis 50 times in your statistical package, your "significant" results are very likely to be wrong. Then according to established scientific practice you can publish a paper claiming that "broccoli reduces risk of cancer in Asian women aged 40-49 (p<0.05)", "global warming causes malaria (p<how dare you question Al Gore)", or somesuch.

Still, that's nothing compared with uselessness of something being "not statistically significant". Even otherwise smart people routinely misinterpret it as "with high probability not true". If so, let me start a study in which I will take wallets of randomly selected half of 10 such people, and we'll measure if there's any statistically significant reduction of wealth from me taking their wallets. Double blinded and all! And as science will prove no statistically significant reduction relative to control group, which court would dare to question science and convict me for doing so?

While false positives - wrongly rejecting null hypothesis - can come from either bad luck, or bad study; false negatives - not rejecting wrong null hypothesis - can come from either of those, or from insufficient sample size relative to effect strength. How strong would the effect need to be?

Let's get back to the smoking trial:

A randomised controlled trial of anti-smoking advice in 1445 male smokers, aged 40-59, at high risk of cardiorespiratory disease. After one year reported cigarette consumption in the intervention group (714 men) was one-quarter that of the “normal care” group (731 men); over 10 years the net reported reduction averaged 53%. The intervention group experienced less nasal obstruction, cough, dyspnoea, and loss of ventilatory function.

During the next 20 years there were 620 deaths (231 from coronary heart disease), 96 cases of lung cancer, and 159 other cancers. Comparing the intervention with the normal care group, total mortality was 7% lower, fatal coronary heart disease was 13% lower, and lung cancer (deaths+registrations) was 11% lower.

Which is not statistically significant. And some libertarian will surely use it to "prove" that smoking is perfectly fine, and it's all government conspiracy to mess in people's lives.

But what were the changes of getting statistically significant results? For clarity let's skip all statistical complications and do it the most brutal possible way, and even make both groups the same size. Control group size 720, intervention group size 720, true chance of death in control group 45%, true chance in intervention group 42% (7% reduction in mortality) - we just don't know it yet. Statistical significance levels 95%. So control group had 95% chance of getting between 298 and 343 deaths (I'll skip the issue of one-sided and two-sided tests of significance as the entire idea is highly offensive to Bayesians). Chance of intervention group having fewer deaths than 298 - merely 38%. So assuming entirely implausibly that this 1440-person-strong 20-year study was perfectly run, there's be 62% chance that the results will be worthless because 1440 people is very far from enough. Except as maybe fodder for a meta-analysis.

By the way reduction is merely 7% as what was studied was "trying to convince people to stop smoking". Most people wouldn't be convinced or would relapse; and many in control group would stop smoking on their own.

Anyway, how many people would we need for study with 45% and 42% death rates groups to have less than 5% chance of both false positive (conditional on null hypothesis being true) and false negative (conditional on null hypothesis being false)? 3350 in each group, or 7100 altogether. And that was smoking - the biggest common health risk we know. How many people would we need if we studied normal-risk people, let's say 10% death rates during study time, and levels of relative risks typical for diet/lifestyle intervention, let's say 1%? Two times 1.2M, or 2.4M people. More than the entire population of Latvia. And that's before you consider how highly non-random are dropouts from such studies, and how they will swamp out any results we would get. And any double-blindness here?

In all likelihood, we will never get any data about effects of this kind of interventions. Science will have no idea if fish oil cures cancer, vegetables lower risk of obesity, or if organic is better for you than non-organic.

The only thing we will have will be lower quality indirect studies with highly ambiguous interpretations, which we might actually have to accept. That, and misunderstandings of statistical significance.

Libertarianism is externality denialism

taw — Thu, 10 Dec 2009 19:07:00 -0800

Yesterday I wrote about libertarian blogosphere, skimming 5 posts each from what is supposedly top 20 libertarian blogs.

There seems to have been quite a few angry reactions, which I will partially address in this post.

Methodology

All ways of discussing politics are flawed, and so was the method I used. But it was hopefully flawed in a different way. It was a methodological experiment.

As my attitude towards libertarianism is not difficult to figure out, I tried to limit my bias by relying on someone else's list of top libertarian blogs, and use transparent selection procedure. At first I wanted to just take the most recent 5 posts from each blog, but there were too many link-only posts, personal stuff, meta-information, and such, which I believed would only introduce noise to the results. I didn't establish clean-cut criteria for selecting which posts qualify for inclusion, so a few decisions might have been somewhat dubious. It's inevitable when something new is tried, and I don't think it affects the results significantly. From one of the comments:

I can only assume that a certain amount of skimming was involved in garnering your links as it shouldn't come as much of a surprise that smoking would feature high. Freedom2Choose is a grass roots pro-choice organisation, set up with the express purpose of opposing bans on lifestyle choices (and before you follow the usual line - no, no funding except from working class members who are angry about the increasing restrictions and bans). The subject would therefore be naturally prominent. The Frank Davis blog was set up expressly in anger at the smoking ban (the clue is the fact that his sub-title is "Banging on about the smoking ban"), and seeing as libertarians oppose authoritarianism, and that the UK ban is as authoritarian a policy as we have seen recently, of course it's going to figure.

I assume the top 20 list is representative of libertarian blogosphere. Pro-smoking articles appeared on 4 different blogs on this list, so it's not accidental inclusion of just one or two smoking-centric blogs. And unlike with climate change which is in news right now, and therefore overrepresented, nothing new is happening that's related to smoking, so I doubt this is an artifact - smoking seems to be quite solidly one of the big libertarian subjects.

Externality denialism

One thing which I observed in both original posts, and in feedback comments I got was persistent denial of existence of externalities.

This is economics 101 diagram of any economic activity - a firm produces a product, which it gives to consumer for their money. Both production and consumption have some external costs and benefits, which they impose on others regardless of their wishes.

Libertarians are really uncomfortable with economics - at least with anything resembling modern science - they sure have their own folk version of it. They pretend that red part of the diagram doesn't exist. The thinking seems to be something like:

I feel that governments are always and without exceptions evil
If externalities existed, governments would actually be useful
So I won't believe in existence of externalities

It starts with values, and ends with statement about reality, as if the nature cared about your values. It is of course logically invalid, but humans are extremely prone to this kind of rationalizations.

Disregarding artificial controversy influenced to large extent by industrial interests (tobacco industry was the leader of this, and this is extremely well documented), science is quite sure about existence of health risks of second hand smoking, anthropogenic climate change, and such. And equally real is plain irritation of non-smokers due to people smoking in their faces. But these facts are very uncomfortable to libertarian position, so you can find arguments like these:

You will also not find one of those posts disputing the link between smoking and cancer. Passive smoking and cancer, yes, simply for the reason that there are hugely more studies showing it to be a construct of the hysterical anti-smoking lobby shilling for the superrich pharmaceutical companies (thought you were against such things?). That debate is nowhere near over as many would have you believe.

Not one post I have read on any of the aforementioned blogs talks of non-smokers being denied the right 'not to have smoke blown in their faces'.

The fact that you use the phrase "climate change denial" shows that you believe the AGW hypothesis. Fair enough. However, if you do not believe the hypothesis (or think that it is pretty darn suspect) then cap-and-trade—for all its internalising of externalities—is a total nonsense because CO2 does not do damage to the environment (in fact, it's actively beneficial because it helps trees and plants grow). Do you see?

They not only disagree with particular solutions, such as smoking bans, or cap&trade - but deny existence of externalities in the first place.

Externalities and the Pigou Club

The mainstream economic position is that externalities are not only real but very important, and the best way to deal with them is internalizing them, usually by appropriate taxes or subsidies, or mostly equivalent ways such as cap&trade, which strangely some consider Pigovian, and some don't:

While Pigovian taxes are economically-preferred way to deal with externalities, they're not the only way and that's due to the second big concept which libertarians pretend doesn't exist - transaction costs.

Transaction costs

Now I'll get quite far away from the main subject. Smoking is harmful to others, but amount of harm depends on circumstances - smoking in your own home without anyone around is less harmful than smoking inside a train filled with people. Standard Pigovian solution would be taxing different contexts of smoking at different levels, but such scheme would be highly impractical.

Driving is also harmful to others - not only due to greenhouse gases emissions and straightforward pollution, but also because of traffic congestion it generates. Taxing driving on a rural road differently from driving in rush hours in Central London would be a good idea, and it would also be highly impractical. This is actually attempted in spite of impracticality, and very high costs of running the system with the scheme barely breaking even if capital costs are included.

So every now and then economic analysis would advise doing something more brutal than just taxation - like an outright smoking ban in public places, ban on leaded gasoline, or my favourite ban on leaded solders. Sometimes no realistic method of Pigovian taxation can be implemented, and so direct regulation is necessary.

Standard anti-Pigovian arguments

Mankiw described 4 basic reasons why someone could be against Pigovian Taxation in a logical flowchart.

1. You deny the existence of these externalities as a type of market failure. Perhaps you think you live in a Coasian fantasy world where people bargain without transaction costs to reach efficient allocations. (Note: I am not suggesting that Coase himself thought we lived in such a world—he considered it only a useful thought experiment.)

2. You recognize the externalities but you don’t think the government should try to respond to them. You are such a believer in small government that you are willing to live with inferior economic outcomes, such as pollution and congestion.

3. You recognize the externalities, think the government should try to correct them, but think the current low taxes we put on gasoline are sufficient. In this case, you have weighed and rejected the evidence, such as that of Parry and Small, that higher Pigovian would be optimal. (Parry and Small calculate an optimal tax of $1.01 for the United States in today's dollars. After my proposed phase-in of a $1 hike, the U.S. tax would be $1.40. Assuming 10 years of 3 percent inflation, the tax in real terms would approach almost exactly what Parry and Small recommend. By the way, the published version of Parry and Small was in the American Economic Review, September 2005.)

4. You recognize the externalities but think the government should try to correct the market failure through regulations (such as CAFE standards) or through market-based solutions that do not raise government revenue (such as cap-and-trade systems). Perhaps you are concerned that government would waste the extra revenue on useless government programs.

It's definitely possible to make an intelligent case against Pigovian taxes, based on #2, #3, #4, or some more creative argument - here's one by Bryan Caplan, here's another by Arnold Kling. My problem is not libertarian conclusions (that internalizing externalities is bad), but that they take the #1 route of living in fantasy land in which externalities don't exist.

Back on-topic

I strayed quite far off-topic, but off-topic asides are as central to my blogging as cats. This extended writing on externalities is due to libertarian blogosphere's obsession with externality denial - climate and smoking posts together constitute 41% of all posts.

The second most popular subject is taxation - quite related to the first one, as the point of government is largely taxing and subsidizing in a way that corrects market failures, and running parts of the economy which market is particularly bad at running - such as police, healthcare, pensions, social security and such. And at least in Britain the train system, which seems private companies seems to be completely incapable of running compared to Deutsche Bahn or even TFL.

Now it's perfectly fine position to support lower or higher levels of taxation, deficits, or spending. But what libertarians care most about?

Lowering taxes on rich bankers, and other super-rich, even though their very jobs exist only thanks to taxpayers' bailouts. This increases deficits, which leads to...
lowering deficits by layoffs and pay cuts in public sector, reducing welfare payments and such.

And we end up with massive transfer of wealth from everyone else to the super-rich. If that's not shilling, what is it?

By the way I'm surprised that my post was misunderstood - I wasn't trying to find out if libertarian blogosphere is any good, but mainly what it writes about. My finding that I find it boring was coincidental.

Timing of peak CO2 emissions

taw — Thu, 10 Dec 2009 02:36:00 -0800

I previously argued (1, 2, 3) that we can expect peak CO2 emissions around 2050. Copenhagen summit draft calls for peaking before 2020. Now it's not entirely impossible, but I would say it's quite unlikely.

Here's a quick list of my assumptions, and how timing of the peak, size of warming, and likelihood of geoengineering would change if they were wrong:

Economic development of poor countries - faster development means more emissions. As this is the main driver of energy demand for the next few decades in my analysis, much slower development of poor countries will indeed move the peak a lot earlier. The consequence of slower growth is economic genocide, and it's far worse than anything climate change could do, with or without geoengineering. It's not a theoretical possibility, economic genocide is happening right now.
Correctness of climate science - putting Climategate fraud accusations entirely aside, IPCC has ridiculously wide confidence intervals. Their A1 scenario which is closest to what I describe here expects likely temperature growth to be between 1.4C and 6.4C by 2010, and their likely means merely a 66% confidence interval. If you know anything about science, "95% confidence" turns out to be wrong more often than not, and these estimates are merely scenarios conditional on countless assumptions, not actual forecasts. So what IPCC is saying is essentially "it will get hotter, we have no idea whatsoever by how much". If warming turns out to be on the low end of IPCC estimates, we'll likely see slower adoption of technologies to reduce CO2 emissions, and no geoengineering. If warming turns out to be on the high end of IPCC estimates, we'll likely see faster adoption of emission-reducing technologies, and massive scale geoengineering will be as inevitable as I predict. This is not as relevant as it seems as timing of CO2 peak depends on decisions we'll make in the next few decades, long before we find out 2100's warming.
Technological progress - faster technological progress can result in lower emissions. If we get breakthroughs in cheap solar energy, battery technology, algae biofuels and such, it might result in much faster peak, maybe even before 2020. But I wouldn't rely on this, a lot of money was spent on these technologies and progress has been slow - what we need is not just some incremental progress which we'll undoubtedly see, but huge breakthroughs and really fast.
Multicentric world - if we had a world government, we could just take a vote, set up whichever emission caps or taxes we wished, problem solved. If we had a small number of important powers like we do now, we could get them to agree on some flawed solution. But the world seems to be evolving towards multicentricity - right now only US, EU, and China matter, plus maybe Japan, India, and Russia a bit - but in the future other countries will probably count a lot too - and the more players you have, the harder it will be to reach meaningful agreement.
Economic development of rich countries - CO2 emissions per capita don't increase significantly past certain level of income, so it won't matter much either way.
Demographic growth - faster growth means more emissions, but uncertainty about 2050 population isn't that high, so it can only have modest effect.
Attitude towards nuclear energy - if people become more supportive of nuclear energy, it will reduce CO2 emissions somewhat.
Policy of US, China, India, and the entire developing world - matters a lot, as they are current and future main emitters, they have a lot of demographic and/or economic growth ahead, and it's highly uncertain what their policies will be.
Policy of EU - matters very little, as EU is already highly efficient for GDP it has, has little economic or demographic growth, and it can be relied on to do further reductions no matter what others will do.

Politics

And regardless of those, the most important factor is politics - global warming is a very unusual situation in which not only everybody's emissions affect everybody equally (straightforward externality / tragedy of commons), but also everybody's current emissions only affect people living in far future. We're asking for 2009's Americans to make significant sacrifices for the sake of 2100's Bangladeshis. 3000's Bangladeshis even, as IPCC says that "anthropogenic carbon dioxide emissions will continue to contribute to warming and sea level rise for more than a millennium".

Let's take an outside view, and make a reference class forecast - in situations in which current sacrifices of people living in one country could make a difference to people living in entirely unrelated country a century ahead - how often were the sacrifices made? To the best of my knowledge the answer is a reliable never. It just doesn't happen. If one country like EU now (and to not that significant extent) were to sacrifice money to limit its emissions, others would just take advantage of cheaper fossil fuels due to reduced demand and burn more of it.

In theory a global carbon emission market could be established, but good luck getting everyone to agree on allocation of emission rights, or income from their sales. The more severe the cuts, the more expensive will emission rights get, and the worse will the fight be. If we allocate most rights to rich countries, poor countries will correctly complain that it hinders their economic development. If we allocate most rights to poor countries, it will amount to massive transfer of wealth from the rich to the poor, and good luck getting voters to agree to that.

We're talking about tens of trillions of dollars. It's not a metaphor, just the proposed American system is estimated to be valued $80bln / year - $80bln times 40 years and 5x (world emissions / American emissions), it will be worth $16 trillion, and growing as supply gets restricted and demand grows due to economic development. How will these tens of trillions of dollars in either emission rights or auction profit receipts be allocated? Yes, good luck with that.

The alternative is not caring about greenhouse gases emissions, and geo-engineering the temperature growth away if need be.

What is libertarianism - blog edition

taw — Thu, 10 Dec 2009 01:16:00 -0800

Usually when people talk about politics, they use language of good and evil. So if you ask an American "liberal", "conservatives" are stupid, bigoted shills for the super-rich willing to destroy the planet and kill people home and abroad in name of some short term profits.

And if you ask an American "conservative", "liberals" are baby-murderers who destroy economy by excessive taxation, overregulation, useless spending, and huge budget deficits.

Some of these accusations are valid, some are nonsense, in many cases politicians from both sides are equally guilty. And it's not a uniquely American phenomenon - in most countries people's levels of rationality reach their lows when discussion turn to politics. This isn't really intellectually satisfying.

Another way to talk politics is what Tyler Cowen's has done in his series of posts about progressivism, conservatism, transhumanism, and then some others like Leigh Caldwell's about libertarianism etc. The idea is to describe a political view as a list of beliefs, about which all sides can agree that Xers believe in them and non-Xers don't, without taking sides on their truth-value.

One problem with this approach is that it has no way of distinguishing core values of an ideology from insignificant baggage that grew around it due to historical processes. Is opposition to gay marriage central to conservatism or fairly irrelevant? How important is public option to liberals? These are interesting questions.

Blog edition

So I wanted to try something else - figuring out what some political ideology is based on what its adherents blog about. It is in a way a fairly objective standard - assuming we can figure out a list of top blogs from some perspective, and that we can categorize posts' subjects in a unbiased way, every analysis should reach similar conclusions regardless of analyst's views.

I took the following list of "Top 20 Libertarian blogs", with top 5 posts ignoring extremely short ones which just link somewhere else, and other funny/unrelated posts. List is very UK-centric, but it doesn't matter.

And the winner is...

I'm actually surprised by the most popular subject - climate change denial. 20% of posts (++++++++++++++++++++) deal either deny climate change or are skeptical about it, which in most cases just denial without balls. Further 12% (++++++++++++) are about Copenhagen summit or environment without being obviously denialist. It's really surprising, as global cap-and-trade system would be the best test case of how well libertarian / free market solutions can deal with problems of externalities, so naively libertarians should be extremely enthusiastic about it - and in any case there's no obvious reason why political ideology should have so much influence over question which is essentially apolitical - temperature changes.

Shilling for the superrich

The second most popular subject doesn't surprise - it's good old shilling for the rich. 10% of posts (++++++++++) were against taxing the rich, or regulating salaries and bonuses in taxpayer-bailed-out banks. Because owners' right to regulate management salaries ends the moment the owner is government. Related 4% of posts (++++) are generic anti-tax posts - but it's quite clear which taxes anger libertarians the most.

A surprise third most popular subject...

I sort of expected climate change to be popular, as Climategate and Copenhagen summit are in the news right now, but this one baffled me. It turns out the third most popular subject with 9% of posts (+++++++++) is smoking - with posts either opposing public places smoking bans (what about "rights" of non-smokers not to have smoke blown in their faces?), or even expressing skepticism about links between smoking and cancer.

The long tail

Further categories don't suprise. 5% (+++++) are offensive posts without much content - something you can expect in political blogs. 4% (++++) are anti-EU rants, 3% are (+++) culture wars / political correctness rants, further 3% (+++) are about budget deficits; copyright (++), religion (++), new media (++), and UK politics (++) get 2% each. There was a single post about each of: discrimination in Iraq (+), gold standard (+), Twitter (+), modern art (+), quality of mainstream media (+), regulation of banking (+), NHS (+), science (+), UK voting by mail (+), forced labour (+), Gordon Brown's typos (+), privacy (+), family law (+) natural rights (+), anarchy (+) Kosovo (+), and public funding of festivals (+) - all expressing the positions you'd expect. There was one pro-Iraq war post (+), one against ideology in politics (+) - notice the irony, and one posts each about Tiger Woods (+), Sarah Palin (+), and Ludwig von Mises (+). The libertarians' favourity shitty book writer Ayn Rand is strangely missing from the list.

Summary

I read a lot of blogs with which I disagree, but they tend to be high quality and quite insightful. So my contempt for libertarianism doesn't automatically mean I wouldn't find at least some good libertarian posts - Tyler Cowen and Robin Hanson are fairly libertarian, and they're both in my RSS. But the posts I've seen were just so bad that of 100 I've checked I cannot point a single one that had any new insights or was interesting in any way. Few even pass basic sanity tests - not just by being contrarian - contrarian posts are much more interesting to read than ones that repeat the conventional wisdom - but by simply not having any idea what they write about.

Why CO2 emissions will not fall before 2050

taw — Wed, 25 Nov 2009 20:33:00 -0800

In my previous posts I explained why emissions are unlikely to fall below current levels before 2050, and why due to peak heating occurring long after peak emissions we do have a very long term problem. Conclusion being - geoengineering is pretty much inevitable.

In this post I'd like to talk about two things - what the world will look like in 2050, and the Climategate.

First, the Climategate. As shown by the leak, scientists from East Anglia University engaged in some highly unethical practice, such as knowingly fudging data to support their thesis, violating FOIA and hiding data to the point of deleting it to avoid release, and organized bullying campaigns against scientists with the opposite views. This makes them bad persons, and if God existed, they'd suffer in Purgatory for it / or got shitty reincarnation in the next life / or something like that. Fortunately for them there is no God, so they will get off scot-free with it.

But this behaviour, while highly unethical, doesn't really seem to change the main conclusion of IPCC reports that anthropogenic releases of greenhouse gases have caused significant warming, and are likely to cause a lot more in the future.

The only consequences for climate predictions I can think of is that it might damage people's opinion of credibility of climate change science disproportionally to the breach of proper scientific practice (which did occur, but were fairly minor), thus reducing political will to act on climate change, and increasing the problem. But then, I doubt the damage will be that big.

Now to the main subject - world in 2050. There's no question that we'll stop emitting ever increasing amounts of greenhouse gases at some point. We already have at least four ways to entirely solve the problem even without any significant technological breakthroughs: nuclear, solar thermal, solar photovoltaic, and wind. But the question of when is a big one, and I doubt it will happen soon.

As I wrote in previous blog posts, in 2050 population will be around 9-10 billion, per capita GDP will be around current EU levels, and so will per capita energy use. But energy use and carbon emissions are not that closely connected - there are gains to be made in efficiency, both nuclear and renewables emit little in terms of greenhouse gases, so that doesn't directly imply much about per capita CO2 emissions (nor emissions of other greenhouse gases).

Let's make a simple table - what would have to change for 2050's CO2 emissions to be equal to 2009's CO2 emissions. For simplicity let's assume every country's per capita emissions will be the same for now. If some countries don't take their share of the burden, others will have to.

Country	Population (mln)			CO2 emissions (mln ton)			CO2 emissions per capita (ton/person)
Country	2009	2050	Change	2009	2050	Change	2009	2050	Change
China	1334 [19.6%]	1462 [15.7%]	+9.6%	6103 [21.5%]	4458 [15.7%]	-27.0%	4.57	3.05	-33.3%
USA	308 [4.5%]	397 [4.3%]	+28.9%	5752 [20.2%]	1210 [4.3%]	-79.0%	18.68	3.05	-83.7%
EU-27	499 [7.3%]	424 [4.5%]	-15.0%	3914 [13.8%]	1293 [4.5%]	-67.0%	7.84	3.05	-61.1%
Russia	141 [2.1%]	104 [1.1%]	-26.2%	1564 [5.5%]	317 [1.1%]	-79.7%	11.09	3.05	-72.5%
India	1173 [17.3%]	1572 [16.9%]	+34.0%	1510 [5.3%]	4794 [16.9%]	+217.5%	1.29	3.05	+136.9%
Japan	127 [1.9%]	109 [1.2%]	-14.2%	1293 [4.5%]	332 [1.2%]	-74.3%	10.18	3.05	-70.1%
Other countries	3214 [47.3%]	5253 [56.4%]	+63.4%	8293 [29.2%]	16021 [56.4%]	+93.2%	2.58	3.05	+18.2%
World	6799 [100.0%]	9322 [100.0%]	+37.1%	28431 [100.0%]	28431 [100.0%]	+0.0%	4.18	3.05	-27.1%

Data sources: emissions, population projections. EU-27 is used as European Union will most likely expand before 2050, and I don't want considerations like that to complicate matters.

Right now just six big entities - China, India, USA, EU-27, Russia, and Japan - emit over 70% of global CO2. But poor countries will develop, and this means more emissions. India's emissions are ridiculously low now - due to poverty - but once Indians get richer they will want cars. They will want pretty things. This will use energy, and this will emit CO2. The same story will take place in the "rest of the world" category (which unfortunately mixes small rich countries with big poor countries, obscuring the change).

What kind of emission cuts will the big emitters have to do just to counteract population growth of the world and economic growth of poor countries? Assuming the burden will be equally shared, USA will have to cut by 83.7%, or to 1/6th of current per capita levels. EU, Japan, and Russia will have to cut by about 2/3 per capita, and even China will have to cut by 1/3 per capita, in spite of so much economic development still being ahead of them! Other countries will have a mix of increases for the poor, and very significant decreases for the rich.

Now how likely is this scenario? I quite doubt we'll actually achieve equal sharing of burden. Cuts for developed countries people talk about tend to be around 15% levels by 2025, so presumably around 40% by 2050, and even these are best case scenarios.

Intrade thinks that odds of any cap and trade system being established in USA by 2011 is about 50:50, and that says nothing about levels of caps, and severity of loopholes passed due to industrial lobbyists' money. After that it's Palin's administration as Krugman says, so don't hope for any meaningful limits. In general, people vote for current party when economy is good, and vote against current party when economy is bad, no matter who's guilty. Current predictions of American unemployment for 2012 are so high, that Democrats are very likely to be voted out of power unless they get their act together.

Anyway, 40% by 2050 is nowhere near enough to even keep 2050's emissions equal to 2009's! So rich countries, especially USA, are not providing the leadership here. What are the chances that poor countries will voluntarily take most of the burden of reducing carbon emissions all on their own? I'd say extremely slim.

And, most importantly... even if reductions from the table did happen, and 2050's emissions were at 2009 levels and falling, this still leaves us with massive global warming problem as peak warming happens long after peak concentrations, and peak concentrations happen long after peak warming.

What brings us to where we started - there's simply no realistic scenario under which geoengineering can be avoided.

Medieval 2 Total War Concentrated Vanilla 0.08

taw — Thu, 19 Nov 2009 00:08:00 -0800

Time for another version of my Medieval 2 Total War minimod. It doesn't contain that many changes over 0.07.

The biggest change is fixing the "everyone hates you" bug I wrote about a few days ago. That doesn't mean everyone will love you now - but now you have the option of acting chivalrous, getting good reputation, and if you do your alliances will be more likely to survive, your enemies will be more likely to sign peace with you instead of insisting on silly wars until you destroy them and so on. The game still makes everyone hate you if you're too strong, or if you're playing on very hard difficulty, so don't worry about it being too easy.

Rubber swords bug

The second bug fixed is the extremely well known "rubber swords" bug. In Medieval 2 Total War units have between 1 and 3 weapons, usually 2. It's absolutely necessary for missile units, which need some sort of missile weapon, and also some secondary melee weapon in case they have to fight man to man. But many units have two melee weapons - most knights have a lance for charging and sword for close combat.

Like many things this all makes sense in theory. The problem is with pikemen. They have long pikes for fight in formation, and swords for close combat. Unfortunately their logic for weapon choice is pretty much the same as with knights and their lances - instead of trying to use pikes as long as they can, they switch to swords almost immediately after first charge. And with very weak swords, no shields, and weak armor pikemen make for really horrible infantry.

If it was just a matter of them being weak, it wouldn't be that big of a deal - there's plenty of weak units in game. The real problem is that they switch to rubber swords too early, and you don't see any pike action at all! Removing their "rubber swords" completely makes them much stronger, and behave properly.

Now there's a valid counterargument, that is raised on Total War forums every time the rubber sword bug is mentioned - the game was balanced with rubber sword behaviour, Pikemen are extremely cheap, and so shouldn't be expected to be much good.

It has some validity - Pike Militia costs only 150 florins, half as much as Town Militia at 290, so making them into some sort of super-unit would unbalance the game. Recruitment costs are only meaningful in battle mode (that is mostly multiplayer). In campaign mode, cost of an unit is cost of its recruitment, upkeep over multiple turns, and of buildings necessary to recruit the unit.

Compare scenarios of building required buildings (other than basic things like walls and barracks up to city watch and you need anyway), recruiting 10 (for vanilla, or 15 for mod as buildings are 50% more expensive) units from it, keeping them for 5 turns in garrison, and 5 turns in field. Costs per unit would be:

Town Militia: 290 recruitment + 0 garrisoned + 625 in field + 0 only basic buildings = 915
Genoese Crossbow Militia: 570 recruitment + 0 garrisoned + 500 in field + 0 only basic buildings = 1070
Peasants: 110 + 450 garrisoned + 450 in field + 60 mustering hall = 1070
Italian Spear Militia: 460 recruitment + 0 garrisoned + 625 in field + 0 only basic buildings = 1085
Peasant Archers: 220 recruitment + 500 garrisoned + 500 in field + 120 bowyer = 1340
Peasant Crossbowmen: 220 recruitment + 500 garrisoned + 500 in field + 360 practice range = 1580
Pike Militia: 150 recruitment + 0 garrisoned + 625 in field + 960 militia barracks = 1735
Mailed Knights: 680 recruitment + 1250 garrisoned + 1250 in field + 0 only basic buildings = 3180
Lancers: 930 recruitment + 1600 garrisoned + 1600 in field + 2760 all stables = 6890

Recruitment costs are simply a tiny fraction of total costs! Now this depends on which building you'd build anyway (you always need walls; city watches increase public order, shipyards increase trade etc. so you might buy them anyway), how many units you're going to recruit from each such building, how long you're going to keep them garrisoned, and how long in field and so on. These are not definite numbers - but they should give you some idea. It's quite easy to spot the most overpowered and most underpowered unit on the list here - Genoese Crossbow Militia is as cheap as Peasants, but they're ridiculously stronger - it's one of the best missile units in game versus absolutely the worst unit in game!

So anyway, I did some experimenting with pikemen. I've heard about two solutions on Total War forums - either remove their secondary weapons, or make their formations tighter. I made two AIs fight each other - 3 Pike Militia versus 3 Italian Speak Militia. Results were:

Normal Pike Militia - lost, 86% dead vs 37% dead
Tight formation - lost, 83% dead vs 58% dead
No secondary weapons - won, 66% dead vs 80% dead
Tight formation + no secondary weapons - lost, 88% dead vs 88% dead

The only reason tighter formations work is by making it harder for the enemy to get real close, and making pikemen use their pikes longer. Now AI was clearly unbelievably dumb here - happily charging on the pike wall. Manual results were:

Me normal Pike Militia, AI Italian Speak Militia - I won, 65% dead vs 89% dead
Me Pike Militia without secondary weapons, AI Italian Spear Militia - I won, 37% dead vs 85% dead
Me Italian Spear Militia, AI Normal Pike Militia - I won, 1% dead vs 20% dead
Me Italian Spear Militia, AI Pike Militia without secondary weapons - I won, 4% dead vs 13% dead

So properly used, Italian Spear Militia is far stronger than Pike Militia - properly used means flanking, and not blindly charging onto the pike wall. Nothing too complicated. Claims of Pikemen being overpowered without secondary weapons seem greatly overstated.

Scaling down previous changes

I decided to scale down a few changes. Cavalry no longer costs 50% extra - nerfing them was fun, but now that most infantry gets free upkeep in castles, and pikemen actually work, cavalry is decently nerfed anyway.

I reduced extra ammo for missile infantry from +100% to +50%. It doesn't really make that much difference, and I'd rather stay closer to vanilla values unless I have a good reason not to.

Full list of changes

Here's the full list for reference

Fixed standing bug
Fixed rubber swords bug
Castle garrisons free for basic infantry
City garrisons twice as big
King's purse x2
Rebel spawn rates 10x lower (supposedly, I have my doubts if it works)
Missile infantry ammo +50%
Bodyguard unit sizes halved
Building construction times all 1 turn
Mining income x2
Building costs other than mines +50%, mines x3
When defending a settlement, towers are activated from much greater distance
Wall and gate strength x5, but tower strength unchanged
Tower fire rate x2, but only for non-flaming ammo
Spy costs x2
Campaign movement speed +75%

Here's the download.

Dietary consequences

taw — Tue, 17 Nov 2009 08:33:00 -0800

This post is a follow-up to The myth of 5 a day - people never ate that much fruit and veg.

"Because" is not a logical connective of classical logic, it's not possible to construct a truth table for "A because of B" statement. Even when A and B are true, it's really quite difficult to prove that causality flows from B to A - correlations, no matter how strong, don't prove causality.

On the other hand, while proving such statement true is hard, proving it false can be very simple - if either A or B is false, "A because of B" must necessarily be so too. So "crime is on the rise because of violent video games" can easily proven to be false, because the consequence part ("crime is on the rise") is false - therefore collapsing the whole argument before we get anywhere near analyzing causality.

"Guns prevent crime" / "death penalty prevents crime" arguments collapse equally fast - countries with more guns and more executions don't have less crime - so we don't even need to think much about causality here. That doesn't mean that reverse statement "gun control prevents crime" is necessarily true, causality might be missing even if correlation exists. By the way there is actually some pretty good research indicating that violent movies deters violent crime, at least in short term - apparently because violence-loving young males go to theaters to watch violent movies instead of getting into real crimes. It's not too unlikely that it might be the same with violent video games.

Now that we're past the obligatory cat and the obligatory detour, let's get back to diets. There's a widely repeated claim, which basically goes "people are fatter than in the past because their diets have much more fat, especially animal/saturated fat than in the past, and nowhere near as much fruit and vegetables as in the past". The consequence is undoubtedly true - obesity is on the rise pretty much everywhere.

On the other hand, the part about too much (saturated/animal) fat, and not enough fruit and veg is entirely wrong. We actually eat a lot less fat; the fat we eat is increasingly unsaturated vegetable fat; and we eat a lot more fruit and vegetables than in the past.

Take a guess - what are the biggest sources of calories in American diet? Here's the list for 1961 and 2003, in kcal/day. I don't believe 1961's diets were perfect, and I'd love to have data earlier than 1961, but that's as far back as FAO statistics go.

Vegetal products consumption increased a lot - 1871 (65%) to 2708 (72%) kcal/day, while animal products consumption stayed constant in absolute terms, and decreased a lot proportionally - 1010 (35%) to 1045 (28%).

Top vegetal products:

cereals without beer - 627 to 832 (of which wheat - 504 to 603; rice 26 to 94)
sugar and sweeteners - 515 to 657 (of which sugar 453 to 320; HFCS 56 to 331)
vegetable oil - 276 to 606 (of which soybean oil 157 to 492)
alcoholic beverages - 108 to 150 (of which beer 68 to 100)
"fruits" without wine - 79 to 117 (of which oranges 14 to 32)
potatoes - 77 to 100
maize (without HFCS, maize oil etc.) - 59 to 98
"vegetables" - 63 to 77
beans - 32 to 30

It's quite surprising list - foods which can be considered traditional like wheat, potatoes, and beans, are a really small portion of the whole. Modern vegetal products like soybean oil (unsaturated fat), sugar+HFCS (sugar, obviously not fat) dominate the list.

And for animal products:

meat - 335 to 451 (of which poultry 64 to 197; pork 127 to 132; beef 125 to 115)
whole milk - 304 to 199
animal fat products (butter etc.) - 199 to 116
cheese - 48 to 149
eggs - 67 to 55
butter - 65 to 40
fish and seafood - 19 to 28

Total fat and protein consumption in g/day:

Animal fat - 69g to 71g (63% to 46%)
Vegetable fat - 40g to 83g (37% to 63%)
Total fat - 110g to 155g

What increased here was poultry - the approved lean meat; and cheese - the most vegetarian of animal products; what decreased most was whole milk, butter, eggs, and animal fats as whole category.

This is as far away from the conventional story as it gets. Proportionally to their diets, people eat less fat, a lot less of the supposedly "unhealthy" saturated animal fat, less milk and butter (saturated fat), vastly more "healthy" vegetable fat, more lean meat, more vegetables, more fruit. Sugar+HFCS consumption increased a lot in absolute terms, but stayed fairly constant proportionally, even decreasing slightly from 17.8% to 17.5% of all calories.

So it's not that people lack personal responsibility - people adjusted their diets to follow every single point of conventional dieting advice. To keep blaming people's eating habits for obesity epidemic is to blame the victim. You cannot claim that people are fat because they don't follow dietary advice, as they actually do!

The only alternative is that the conventional advice doesn't work. Eating more fruit and veg, switching from animal to vegetable fats, reducing proportion of fats in diet and so on - does not make you slimmer. I won't answer the question if the advice is merely useless or actually harmful in this post (causality in reverse direction) - that would require a lot of research. But we sure know it doesn't work.

I doubt that another round of minor adjustments to the advice (look at all reversals on butter vs margarine), let's say by insisting on more fish oil this time, are likely to magically make it work now, in spite of consistent track record of failure.

I don't have enough data to give you reliable advice, but the most obvious alternative would be eating reasonably-sized portions of the most delicious food you can think of. This will usually be something fairly traditional (even not necessarily from your culture), often but not always with plenty of animal products in it, and extremely rarely with much sugar or vegetable oil. If type of food doesn't matter, and only portion size does, then you can as well enjoy it, instead of torturing yourself with diets.

For the record, author of this post has BMI 23. Never trust overweight people to give you diet advice.

Why everyone hates you in Medieval 2 Total War

taw — Sun, 15 Nov 2009 21:34:00 -0800

Short answer - because of a really stupid bug in data files. If you want it fixed, download corrected file and put it in your C:\Program Files\SEGA\Medieval II Total War\data\ directory or similar. For the long version, read on.

Reputation and standing

First, technicalities. The game keeps track of two important numbers - your reputation (also called global standing), which is global, and standing with a particular faction. Both are numbers between -1 (the worst possible) and +1 (the best possible).

Good behaviour increases your reputation, and your standing; while bad behaviour decreases it - as expected. There are two kinds of deeds - some actions are unconditionally good or bad - they simply add or subtract from your reputation/standing. If you do 5 good deeds and 4 bad deeds of this kind, it's pretty much the same as just doing 1 good deed.

The other kind use what the game calls "normalization". This "normalization" operation has two parameters - target value, and "divisor". It takes your current standing or reputation, and moves it by 1/divisor towards the target.

For example if your current standing is 0.6, then "normalizing towards +1 with divisor 10" will result in standing of 0.6 + (+1 - 0.6)/10, that is 0.64. If your initial standing was -0.8 instead, the result would be -0.8 + (+1 + 0.8)/10, or -0.62. In other words - if your reputation/standing is bad, good deeds improve it a lot, while bad deeds damage it little. And if your reputation/standing is good, good deeds improve it little, but bad deeds damage it a lot. In our example, the good deed had value of +0.04 for the well-reputed faction, but +0.18 for the ill-reputed one.

Normalization to some value with divisor 1 is equivalent to completely changing your reputation/standing - by our equation initial + (target - initial) / 1 = target. Doing both good and bad things of this kind will move your reputation towards neutral values around 0.0, even if one kind of deeds is more frequent than the other. It's all quite sensible in practice.

Obviously we want our relations with different factions to be high - factions that hate us will be more likely to start wars against us, disrespect treaties and so on. Even with enemies we would prefer higher standing, in case we want to sign peace treaty someday.

What changes reputation

Most reputation changes are absolute.

First, your reputation normalizes to 0 with divisor of 200 anyway, which is extremely slow - it takes 140 turns to halve your good or bad reputation. Having an ally normalizes your reputation to +1 with divisor of 400, and being at war (except with rebels obviously) normalizes your reputation to -1 with divisor of 800. These are extremely small divisors, but if you have multiple allies, or multiple enemies they add up.

On top of that, some generic good and bad deeds:

"Taking back" settlement, which you gave by diplomacy (but then do you ever give settlements by diplomacy?): -1.0
Attacking an ally: -0.5
Canceling alliance (also when forced by one of your allies attacking another - tough luck): -0.1
Bribing settlement (it's virtually impossible in Medieval 2 Total War anyway, unlike in Rome Total War where you would be doing it all the time): -0.15
Breaking treaty terms (I'm not sure what counts here): -0.15
Military assistance to your ally in battle: +0.1

Treatment of prisoners is probably the most important source of reputation. If you put your prisoners up for ransom, it doesn't affect your reputation either way - this is the standard expected behaviour in Medieval warfare. What changes it is either releasing or executing prisoners.

Releasing prisoners: +0.01; +0.025 for groups of over 80 soldiers; +0.025 if any family member was released.
Executing prisoners: -0.01; -0.025 for groups of over 80 soldiers; -0.025 if any family member was executed.

I'll get back to this later in more detail.

And here we arrive at the bug. Treatment of civilians is supposed to affect your reputation:

Occupying a settlement peacefully: +0.05
Sacking a settlement: -0.02
Exterminating population: -0.05

But due to a bug the first never happens. Here's the data file:

;------------------------------------------
Trigger 0103_occupy_settlement_increase_global
WhenToTest OccupySettlement

FactionStanding global 0.05
FactionStanding target_faction normalise 1.0 20
FactionStanding target_allies normalise 1.0 40
; FactionStanding target_enemies normalise -1.0 40

;------------------------------------------
;Trigger 0102_city_razed_decrease_global
; WhenToTest CitySacked

FactionStanding global -0.05
FactionStanding target_faction normalise -1.0 10
FactionStanding target_allies normalise -1.0 20
; FactionStanding target_enemies normalise 1.0 20

Semicolon means code is commented out and inactive. The first event describes what happens when settlement is peacefully occupied. The second is old sacking code, probably from Rome Total War times. It's supposed to be commented out, but only trigger was commented out, the effects were not. So the results are equivalent to:

;------------------------------------------
Trigger 0103_occupy_settlement_increase_global
WhenToTest OccupySettlement

FactionStanding target_faction normalise 1.0 20
FactionStanding target_faction normalise -1.0 10
FactionStanding target_allies normalise 1.0 40
FactionStanding target_allies normalise -1.0 20

So not only you don't get the promised reputation bonus, your standing with the target and their allies gets lower, instead of higher - they're actually happier when you sack their settlements than when you occupy them peacefully, what is nonsense! By the way, all websites about Medieval 2 Total War I've seen seem unaware of the bug, even though it's trivial to test.

Just see how different the game is with and without bug. If you take 6 neutral settlements, your reputation should rise to 0.3 (Reliable); with the bug it will stay at 0.0 (Mixed). This has huge effect on how others will treat you!

Standing

Now that we know about your reputation, how about standing? There's extremely long list of rules related to the Pope, crusades and such, which I'll cover later. For normal factions it's much simpler. I'll spare you the numbers, but basically:

Factions with high reputation get liked more, factions with low reputation get liked less. Thresholds are -0.4, -0.1, +0.1, +0.4.
Depending on difficulty, your standing tends to drift towards +1.0 (very easy), 0.0 (medium), -0.5 (hard), and -1.0 fast (very hard).
Factions which are strong get liked less, factions which are weak get liked more.
There's small advantage to having the same religion.
Your allies grow to like you more, and your enemies to like you less. Allies/enemies of your allies/enemies change their standing accordingly, by smaller factors. If this works correctly, you can see two big coalitions developing, and it often happens.

There are miscellaneous actions that increase or decrease your standing. In addition to the obvious acts of war, bribing, sabotage, and spying decreases your standing - spying much more than I thought - to -1.0 with divisor 20, and with divisor 80 for all allies. I'm not sure if it counts only when spy is caught, but it's far higher than I expected. Only 13 acts of spying will move perfect standing all the way to neutral.

What doesn't affect your standing is merchants buying each other out, your princesses marrying (and stealing) their generals, and your priests preaching on their territory. Marrying your princess to their faction heir is worth a lot - +0.2 instantly, and normalization to +1 with divisor 20 every turn.

Adding it all together - the way it's supposed to work is that increased standing due to high reputation should counteract decreased standing due to your size and difficulty level (at least on hard, maybe not so much on very hard). But because you don't get reputation points you deserve for peacefully occupying settlements, there's no balance, and everyone gets to hate you.

The Pope

It's only vaguely related to the bug this post is about, but as I started talking about Medieval 2 Total War standing system, I can as well describe how it works with the pope.

First you get points for building religious building - from 0.02 for small churches and chapels, to 0.2 for huge cathedrals, and additional 0.2 for your first huge cathedral, and additional 0.2 if it's the first huge cathedral in the world - so you can get +0.6 with the pope for just one building, after first getting +0.52 for first cathedral in the world.

On the other hand destroying religious building for money makes the pope hate you, about 5x as much as building them. But as you never really destroy buildings anyway, you probably don't need to bother yourself with it too much.

Crusade get you huge bonuses with the Pope - but only arriving at target region, and taking the target settlement, not just joining. You get bonuses for every general, and every unit, but a lot more if you send your heir (+0.2) or your leader (+0.6) to the crusade.

Something I didn't know about before reading the code is that attacking Orthodox factions while on crusade (usually taking Constantinople on the way to Jerusalem or Antioch) gets pretty high penalties with the Pope. You'll get your papal standing back if you actually conquer your target, but if the crusade is just an excuse for sacking the Byzantine Empire the Pope will not be happy. As you probably already know, attacking non-excommunicated Catholic factions while on Crusade, or Islamic factions while on Jihad is prohibited by the rules.

Recruiting priests and doing papal missions get you points. Getting excommunicated loses you -1.0 points, obviously. Getting your cardinals promoted get you points (0.1 per cardinal), and there are even more points in elections - Pope from your country means +0.8, from allied country means +0.4, enemy country's Pope means -0.6, and voting for the winner pope is +0.2. Surprisingly I don't see any negative points for voting for the wrong candidate, even though in-game text insist on wrong vote being a major transgression.

You also lose points with the Pope for being at war with other Catholics. And look after heretics in your lands - if the Pope has to appoint an Inquisitor, it costs you immediate -0.2.

Now it's going to more or less even out - you will build churches and recruit priests anyway, especially if you're close to Islamic or Orthodox regions - what will score you major Pope points. And you'll lose points for inevitable wars with other Catholics. You're supposed to get extra points for good reputation, but as I said, you won't due to the bug.

What to do with prisoners

There's one more thing I wanted to write about - treatment of prisoners. You naturally know that you shouldn't normally sack or massacre settlements, as these will be your subjects, but why do anything except execution with the prisoners? Let's do some game theory:

Value of reputation gained or lost: A (by marginal view it's supposed to be the same either way, it's not exactly true, but let's keep it this way for simplicity)
Value of money gained from ransom: B
Cost of enemy getting units back: C

So payoffs are:

Released: A-C
Ransom accepted: B-C
Ransom rejected: 0
Executed: -A

It helps to know if the enemy is likely to accept the ransom or not. Usually the answer is clear - due to war they don't have any money, so they cannot pay you. In this case if A>C, you release prisoners, otherwise you ask for ransom and don't get it. A>C is the usual situation, you've just defeated their armies, so they probably weren't that good anyway. The biggest exception I can think of is when you're on your way to take a weakly defended settlement, and these armies would join the defenders and make your work much more difficult when released. Or when it's really late in game, you're at war with everyone, and you don't care about reputation anyway.

How about when you think the ransom will be accepted? First, decision between release and ransom. If A-C>B-C, release straight away. Unless you captured their leader (10000) or heir (5000), reputation will be almost always more valuable than money - mostly because for reputation purposes they're treated just like any other general, but for ransom purposes they're a lot more valuable. Except your enemies are quite unlikely to have that much money, so it's not a common occurrence.

How about executing them? It only makes sense when C>2A, and C>A+B. Reputation cost of execution is twice as high, as now you have to actually execute the prisoners, instead of just request ransom. Your loss due to release of troops is very rarely that high. Even tiny group of soldiers released or executed will make a difference of 0.02 reputation points, about as much as you need to counteract being at war with someone for 16 turns. If you act chivalrously wars improve your reputation instead of damaging them. Paradoxically the worst kind of war for your reputation is a prolonged war in which nothing happens.

This is all a lot less true once the bug is fixed. Due to the bug releasing prisoners is the only way to significantly improve your reputation, and you need that just to stay positive, as you'll lose it due to broken alliances, executions of prisoners, and sackings of settlements that will inevitably happen sometime during the game. Having allies and assisting them militarily just won't be enough - once big major start, getting allies will get really tough, especially if your reputation isn't very high (chicken and egg...), and assisting stupid bots requires way too much of effort.

But without the bug, you should have decent reputation from just straightforward conquest, unless you kill and sack on your way - and value of A gets a lot lower.

To fix or not to fix

With minor bugs the question is clear - definitely fix them. But this bug is so big, and changes the game so much... Not only AI will like you more, AI will like each other more too. And acting chivalrous will be much more important - with buggy M2TW you really had to go out of your way to get decent reputation, but with the fix if you behave you'll reap diplomatic rewards.

Most likely the fix will make the game significantly easier - as long as you act chivalrous it will delay your wars and make your enemies much more likely to agree to cease fire. On the other hand it will make AI factions hate each other less too - so who knows, maybe they'll unite against you? At some point you'll be big enough to overwhelm your reputation-based bonus, so maybe it doesn't matter that much...

I'll try my next game with the fix, and tell you how it went. It's a really long post for a bug that's basically 3 missing semicolons.

Things I read

taw — Thu, 12 Nov 2009 02:34:00 -0800

As some people expressed interest in RSS feeds I follow, here they are.

For every feed I want to include some recent good item (if there's none, then obviously I should just unsubscribe). Standard fair use disclaimer here (some comics are explicitly CC-*, but most aren't; it's all legit fair use, no matter what Rupert Murdoch thinks).

Webcomics

First, I love webcomics, and there's plenty of those in my feed. Most of them are "one self-contained funny story per item" type, which work very well with RSS readers.

Abstruse Goose is sort of xkcd-ish, crudely drawn smart jokes about science and pop culture, less obsessed with girls than xkcd, and without mouse-over tooltips.
oglaf.com calls itself "Comics. Often dirty.", and is exactly that. Most of it is NSFW, so if you need something to read while sipping a morning coffee in the office, and waking up, it's probably not it. This sample is from quite far back, not because more recent ones weren't good, but because it was hard to find one that was SFW. It's also very nicely drawn, unlike most comics here.
Cyanide & Happiness is an extremely well known comics, strangely with multiple people drawing it, all in more or less the same style. I'll use this opportunity to rant about one infuriating thing about their RSS feed is that very often comics get into RSS feed before they get live on their website (it happened like 4 times just last month).
Order of the Stick is something completely different - instead of being "one funny a day", it actually has a multi-year story of a team of adventurers playing Dungeons&Dragons-inspired campaign and making fun of the rules as they go. Well, at first it was "make fun of D&D vaguely connected strips" format, I think it got significantly worse as it tried to get serious, and most of the humour is about OotS characters, not D&D. Here's one really old strip that you'll get even if you don't follow OotS.
Saturday Morning Breakfast Cereal is another one of those good funny comics - it delivers daily, and has fairly mainstream appeal.
xkcd probably got more cult following than any other webcomics ever. In addition to fans, there are blogs of xkcd hate, and even RSS-able Web 2.0-style website Is xkcd shitty today - their answer always being YES, most often unfairly. It can get really cutesey and sad, or mindlessly evil, or philosophical, or anything else. There are many meciocre xkcds, but it reaches pure comics awesomeness level more often than any other comics I know. And don't forget the mouseover tooltips.
Zero Punctuation isn't really a webcomics - it's a weekly sarcastic video game review. It's universally loved as it tends to find faults in games and make fun of them, while most of the so called "reviewers" just rehash press releases. Here's one old review of one particularly good game:

Reading

All right, I sometimes read and not just look at pretty pictures.

538 is probably the only blog about political polling in existence. It has some highlight, such as predicting results of 2008 American elections much better than anyone else, and finding out that one polling firm was a total fraud, and all their results were fabricated.

It contains a mix of detailed analyses of particular elections (not terribly interesting, unless you happen to live there), criticism of bad polling and bad analysis, and probably most interesting of all - big picture view of shifts in public opinion. Unfortunately it's mostly about USA, and rarely writes about Europe and other places.

Less Wrong is a spin-off of Overcoming Bias, or what Overcoming Bias used to be before becoming Robin Hanson's personal blog. It's filled with insightful posts about human rationality, and biases in thinking, and philosophy of thought. It also has scary amount of discussion that's only understandable to insiders. Some examples:

There's plenty more, and I even sometimes post there.

Marginal Revolution is one of the Freakonomics-style "economics made fun, now with more anecdotes" blogs. It publishes insanely often, five posts a day pace is entirely typical. Unlike this blog, which gets more like five posts a month. A lot of it is reposts of interesting things they found somewhere else, what saves you from following too many feeds. Like repost of this bathroom scales that tweets your weight for all world to see as motivating factor, and experiments with placebo coffee.

Actually 90% of Marginal Revolution is reposts with tl;dr summaries, comments, and counter-arguments. It's pretty good this way. Sometimes they get into crazy libertarian mode, especially when discussing health care, but it doesn't happen that often.

Paul Krugman's blog, and New York Times column (which is like blog, just with longer posts). Krugman can get annoyingly dogmatic about whatever the left wing of Democratic Party believes at time, and often gets horribly "wonkish" about tiny details of macroeconomic analysis. Some good posts:

Krugman can be as bad as dogmatic libertarians, but at least it's bad in the opposite direction. It would be highly amusing if Paul Krugman and Tyler Cowen were locked into one room for a week, and forced to Aumann.

Overcoming Bias used to be Eliezer's blog about rationality, then Eliezer moved to Less Wrong, and Overcoming Bias is all Robin Hanson's. Robin has three big ideas: everything people do is about signaling status, modern medicine doesn't work, and prediction markets are universal solution to every problem. Robin has way too many crazy libertarian moment, so if your libertarian-phobia is stronger than mine do not read. Some good recent posts:

Overview of evidence for panspermia
Estimated 41% of "elections" worldwide are fraudulent (mostly poor countries obviously)
Evidence for flu vaccines is dubious

Peak Oil Debunked used to fight peakoiler doomsdayers during their high days of doom. The doom didn't come, peakoiler movement sort of died, so there are few posts these days, but it was really good back then. Most posts are just facts and figures, something peakoilers hate with passion. (my recent post about geoengineering and its followup were in similar style)

Some nice posts:

Schneier on Security is probably the least controversial of blogs on my reading lists - most of them do politics from time to time, but it's hard to be angry at Schneier about anything. Some good posts:

And the last feed I follow - the War Nerd. Unfortunately it hasn't been posting much since The Decider left. He's way better at predicting course of events on the Middle East than the mainstream media - mostly because mainstream media is just rehashing White House press releases instead of doing any independent thinking. Some good pieces, most pretty old:

Plus countless pieces about Iraq and Afghanistan of course.

Enjoy your reading.

Inevitability of geoengineering, part 2

taw — Tue, 10 Nov 2009 02:18:00 -0800

In my previous post, I explained why geoengineering to counteract global warming seems inevitable. To summarize, data suggests that between 2009 and 2050:

World population will increase by about 45%
Poor people are getting a lot richer, average wealth will increase by 200%
This means they will use a lot more energy than now
Neither renewables nor nuclear power seem likely to be able to replace them by 2050, at realistic growth rates
Therefore only plausible source of that energy are fossil fuels, and we already have plenty of examples of countries drastically increasing their fossil fuel use and CO2 emissions extremely quickly.
Reduction in energy use by rich countries is nowhere near big enough to counteract this

What changes around 2050?

World population is expected to more or less flatten out around 2050. We're already adding fewer people each year than during peak growth, so growth is sublinear, let alone expecting it to be exponential. Long term projections need to be taken with a pinch of salt, but it's as safe as it gets to expect much slower growth in second half of 21st century than now.
While world economy will keep growing, average GDP per capita will be more or less where EU levels are now, and energy use past this level doesn't increase that much.
In all likelihood, sources of energy other than fossil fuels will be far better developed than now, so responding to any new energy needs by more renewable energy instead of by more coal power plants, will be much more likely.

Now I don't know exactly when we will have peak CO2 emissions, and at what level - somewhere vaguely around 2050, and at levels twice current seems likely, as decades before it will have much higher pressures to increase fossil fuel use than decades after.

The first problem, is that peak CO2 concentrations will happen long after peak CO2 emissions. Here's a random graph with emissions on the top, and concentrations on the bottom, color-indexed. In all scenarios, including almost immediate halving of emissions, peak is somewhere in 21st century, but CO2 concentrations keep growing long past 2300, peaking hundreds of years from now!!!
Now I don't actually believe CO2 emissions will be falling that slowly - once we figure out how to use renewable energy cheaply, we'll be able to switch almost all our use to it, so peak concentrations will be somewhere during 22nd century.

It gets worse - many effects of CO2 concentrations, like temperature change and ocean acidification, lag it significantly. So peak warming and peak acidification will occur much later than peak concentrations. With 2050 emission peak, and 2150 concentration peak, we might have 2250 climate change effects peak. (last figure completely made up, as it depends on too many details, different effects have different lags, and nobody makes serious estimates that go that far into the future). So even with a lot of optimism about alternatives to fossil fuels, we will have to deal with massive effects of global warming.

This gets us back to geo-engineering. No matter what we do now, short of a genocide and collapse of global economy (world economy grew during the recession, and fell only a few percents in the worst hit countries; this isn't the collapse I'm talking about), our choices are geo-engineering and adapting to higher temperatures.

Disclaimer time

Now disclaimer time. I'm making the common mistake that anthropogenic CO2 emissions are responsible for all global warming. This is not true - added CO2 is only 2/3 of it, the rest being methane, nitrous oxide, CFCs. But there are good news here - their lifetimes in atmosphere are much shorter - while we're talking tens of thousands of years for CO2, it's only 12 years for methane, and about 100 for nitrous oxide and CFCs, so peak concentrations will occur relatively quickly after peak emissions. And in fact, peak concentrations of methane and CO2 seem to have already happened or be happening about now. In longer perspective global warming problem is CO2 problem, with only small methane and nitrous oxide contribution.
The second disclaimer is that there are two kinds of geoengineering - solar radiation management (stratospheric sulphides, cloud seeding, albedo changes and such), and carbon capture and storage.

With solar radiation management we let warming happen, and create equal negative effect.

With CCS we could pull more CO2 from atmosphere than we're emitting. If we went this way peak concentrations might happen much faster after peak emissions than otherwise, and due to rapid fall in concentrations peak warming will be earlier and smaller.

I'm mostly talking about solar radiation management, as we already know how to cool down the planet with current technology and for ridiculously small money; while CCS solutions don't seem to be anywhere near scalable enough soon enough.

Any alternatives like fussion power, nanotech, and new kinds of renewables that people love to talk about won't seriously affect emissions until long after 2050 even in the most optimistic scenarios.

The myth of 5 a day - people never ate that much fruit and veg

taw — Mon, 09 Nov 2009 23:12:00 -0800

NHS (to American readers - it's the British socialized medicine destroying UK, and coming to destroy your country too now) strongly promotes the idea of eating "5 servings of fruit and vegetables" a day, with which I disagree on so many levels.

Here's a quick summary of the alleged benefits, straight from their website:

They're packed with vitamins and minerals.
They can help you to maintain a healthy weight.
They're an excellent source of fibre and antioxidants.
They help reduce the risk of heart disease, stroke and some cancers.
They taste delicious and there's so much variety to choose from.

That's all nonsense.

You are extremely unlikely to have vitamin and mineral deficiency, and if you suspect some (or want to get pregnant, have symptoms of anemia etc.), get real multivitamin pills, or supplements of the right kind. Most fruit and vegetables are fairly unimpressive as for their vitamin contents anyway - for all practical purposes most fruits are just water and sugar, and have been selected this way by farmers, as larger and sweeter fruit sells better, and vitamins don't have taste.
There's absolutely no evidence linking fruit and vegetable consumption to weight - you get stable weight when number of calories taken balances number of calories expended, what's possible on any kind of diet, including a really delicious one based on loads of meat and cheese and chocolate that NHS would not approve of. I'll get back to this point later.
Fibre doesn't do anything - it just passes through your digestive system and goes right into the toilet. "Antioxidants" are one of those terms that make me rage, as they've been stripped from their original scientific meaning, and turned into pure value judgment, and not even of the right value. Your cells know how to deal with oxidative stress already. That's what they're evolved to do over the last 1.7 billion years. They have plenty of ways to do so. And increasing amount of antioxidants will not help your cells. Meta-analysis of all available data shows clearly that eating more of them doesn't do anything for you, and can even be harmful.
There's been no proper long term trials of that kind. There's very few proper long term trials of anything related to diet, and those that are performed invariably show that simple diet advice (eat this, don't eat this) doesn't work - and unless your diet is spectacularly wrong you'll be fine as long as you have adequate exercise, decent portion control, and don't smoke.
As any kid will tell you, fruit and vegetables taste awful - if they were delicious people wouldn't need NHS's bothering about them. Do chocolate and ice cream need NHS campaigns? Obviously not. That's because chocolate and ice cream are tasty, while fruit and vegetables are not, at least not in those ridiculously large amounts.

So it's wrong on all counts. Anyway, that's not what I wanted to post about - I just get distracted easily. Oh, something shiny. Oh, funny cat. I go for all distractions. Now back on track.

What I wanted to talk about was the myth that people used to eat enormous amounts of fruit and vegetables - 5 servings a day or more as NHS advices, and that's it's something modern that they do not do so now. Let's look at FAO data, annual consumption per capita of fruit and vegetables, as far back as it goes (that's unfortunately not that far back). I included some particularly fat countries (USA, Mexico), a particularly healthy country (Japan), random countries from Europe (France, Germany, Poland, UK), and totals for Europe and World. For interest of science I'll say I selected countries and criteria before I had a first look at the data, so it's not a biased selection, but feel free to look at others.

Vegetable consumption, 1961 to 2003, increased in all selected countries:

France - 150kg to 142kg
Germany - 49kg to 90kg
Japan - 95kg to 104kg
Mexico - 25kg to 58kg
Poland - 91kg to 100kg
UK - 60kg to 91kg
USA - 93kg to 123kg
Europe - 90kg to 117kg
World - 63kg to 117kg

Fruit consumption (excluding wine), 1961 to 2003, also increased in all selected countries:

France - 53kg to 95kg
Germany - 79kg to 113kg
Japan - 29kg to 54kg
Mexico - 57kg to 120kg
Poland - 18kg to 47kg
UK - 55kg to 115kg
USA - 76kg to 113kg
Europe - 65kg to 88kg
World - 38kg to 62kg

Fruit and vegetables together, 1961 to 2003:

France - 203kg to 237kg (+17%)
Germany - 128kg to 203kg (+59%)
Japan - 124kg to 158kg (+27%)
Mexico - 82kg to 178kg (+117%)
Poland - 109kg to 147kg (+35%)
UK - 115kg to 205kg (+79%)
USA - 169kg to 236kg (+40%)
Europe - 155kg to 205kg (+32%)
World - 101kg to 179kg (+77%)

So consumption of fruit and veg in the last four decades drastically increased - and so did obesity, heart disease and so on. Increase in UK and Mexico were particularly large - and these are some of the fattest countries in the world, something that was not true in 1961. In increased least in Japan - by far the healthiest of developed countries. NHS was really successful at making Brits eat 5 servings a day - making 63.8% of them overweight in process. So much for fruit and veg consumption helping fight obesity? Here's the list of selected countries ordered by fruit and veg consumption and percentages of overweight people in them:

France - 237kg - 40.1%
USA - 236kg - 74.1%
UK - 205kg - 63.8%
Germany - 203kg - 60.1%
Mexico - 178kg - 68.1%
Japan - 158kg - 22.6%
Poland - 147kg - 47.5%

Is it just me, or do countries that eat more fruit and veg tend to be fatter, not thinner? The answer is probably a lot more like this instead:
And it's not like 1961's fruit and vegetable consumptions were unusually low - they were in fact unusually high by historical standards - normally it was impossible to eat any other than those that grew in your region, in harvest time, what all amounted to very very little compared to what people eat now with globalized industrial agriculture.

The take home message: Ignore NHS; eat chocolate, ice cream, beef, cheese, or whatever else you find delicious. In reasonable amounts.

Inevitability of geoengineering

taw — Wed, 04 Nov 2009 19:49:00 -0800

I want to write a quick post showing some arguments about inevitability of large scale geoengineering which I find extremely convincing. I'm sure I won't convince that many people, most have made up their minds already.

First, look at list of countries by CO2 emissions. In 2006 global emissions were 28.43 billion tons per 6.55 billion people, or 4.34 ton/person. World population is still growing and is estimated to stabilize at levels around 9.5 billion for 2050-2100, which is the time in question.

Now let's see what would happen if everyone got to current European standards of living, and European CO2 emissions, which are 7.84 tons/person. This would result in CO2 emissions of 74.5 billion tons, or 2.6x times current levels. This number is already quite optimistic - it makes a huge assumption that all countries with higher CO2 emissions like US, Japan, Canada, Australia, Russia etc. scale them down quite considerably.

Now a brief discussion is in order on why numbers like that are relevant. First, I assume that there won't be a massive genocide, or nuclear warfare, or pandemic, or anything else that could drastically reduce population levels. The last time there was a significant reduction in population levels was 1340s. None of the world wars, or any other events even stopped population growth - in fact people tend to reproduce a lot more during times of crisis, than during times of peace and prosperity, so it would need to be the biggest war, pandemic, or other kind of failure in history to significantly reduce population levels. This assumption is probably the safest.

The second assumption is that the rest of the world will grow quite quickly. World GDP/capita is currently $9,774 PPP. EU GDP per capita is $30,513 PPP. If world economy grows at per capita rate of 2.8%, then by 2050 world average GDP will reach current EU levels. This coincidentally is almost exactly annual per capita growth rate of the world economy of the last 30 years. Unlike with population, where predictions are quite solid, here there are more reasons why growth could be significantly slower, or significantly faster than predicted. We just had a completely unexpected global recession for a fairly obvious example. And not so long ago we had massive completely unexpected boom in China and then India, so it can go both unexpectedly badly and unexpectedly well. Still, 2.8% is a pretty decent first estimate.

I want to write a bit more on the subject of economy. Many people are sceptical about the idea that the "poor" countries will ever attain levels of economic development seen in rich countries. But the rich-poor divide is illusory. As you can see yourself with Gapminder, the divide is long gone and most countries are in the middle. World average life expectancy today is higher than in most richest countries in 1950s years ago. World average life expectancy in all but one countries is higher than world average 100 years ago - all that in spite of wars, malnutrition, malaria, HIV, and lack of clean water in many of them! Life in countries considered by many to be "permanently poor" is vastly better than it was in Europe of your grandfathers, and rapidly improving. To think it will somehow suddenly stop is to ignore history.

Development can happen extremely rapidly - world's largest CO2 emitter has been China for a few years now, per capita emissions of which increased by factor of x2.2 between 1990-2006. Likewise for India, growth was x1.6 during that time. Even for big countries not typically associated with rapid growth you have high growth - x4 for Vietnam, x2.4 for Thailand, x2.3 for Malaysia, x1.9 for Indonesia, x1.5 for Pakistan, x1.4 for Brazil and Nigeria and so on (and these are all per capita rates, their total emissions increased far more). Developing countries are catching up fast.

Let's look at past performance of world CO2 emissions. Between 1992-2007, world's emissions increased by 38% - mostly driven by rapidly developing countries like China and India. Even developed countries weren't terribly successful - US and Japan had significant increases in emissions, and EU's stayed pretty much even, and nominally decreased mostly due to early-90s' collapse of heavy industry in former communist member countries. Even if developed countries got their act together and implemented reductions that were agreed on in Kyoto Protocol, they would still be swamped by just Chinese increases, resulting in net emission increases.

And the outlook for the next couple of decades doesn't look terribly promising. Intrade markets, thin as they are (and forgive me lack of links, but their website is virtually unlinkable), think it's extremely unlikely that China and India will agree to any CO2 limits anytime soon, and have pretty low opinion on what reductions developed countries will agree to - chance of agreeing to 10% reduction by 2025 seems to be about even, and there's quite a big difference between agreement and implementation (especially due to possibility of accounting shenanigans of land use changes and such), and such reductions would be swamped by developing countries increase. And in any case the 74.5 billion tons number already assumes that developed countries average will get reduced to current EU levels.

If you hope on peak oil to make 74.5 billion tons/year kind of number impossible, not only peak oil doesn't seem to be happening, coal production grows insanely fast, and estimates of "peak coal" talk about mid 22nd century.

Alternatives to 74.5 billion tons a year

So with business-as-usual, we'll be emitting enough CO2 to get the entire planet stir-fried. What would be alternatives?

First, either outright genocide or indirect genocide by limiting economic growth of developing nations - remember that poverty kills hundreds of millions of innocent people and causes vast suffering. Not only is it unlikely to happen, it would be probably the worst scenario of all.

Second, learn to live in a warmer world. This isn't necessarily going to be as bad. It's hard to imagine anyone who would like to have their children die of malaria and diarrhea just to satisfy some rich smug liberals. People who measure costs of limiting CO2 emissions usually focus on limits in States, or EU - but this is nonsense! Big increases don't come from SUVs, but from bringing poor people to more civilized standards of living, and limiting that would be disastrous. Costs of just taking global warming would be much less than the genocide that would ensue to keep poor people in their place. I think this scenario isn't entirely implausible - there were quite a few cases in history where international cooperation completely collapsed, and in externalities cases like CO2 emissions, this is a possible result.

The third solution, and we're getting somewhere now, would be to sever the link between GDP and CO2 emissions. This is surprisingly hard. Yes, it's trivial to move from Hummer to Prius, but the problem is people who cannot afford either yet, but will want some means of transportation eventually. And they will want electricity - coming from coal most likely. The alternatives look pretty bleak - nuclear power, and large-scale hydroelectricity, both provide only 3% of total primary energy use now (or about 15% of electricity each). All fancy types of renewable energy together don't even add up to 1%. A surprisingly large amount of energy is "traditional biomass", or wood and agricultural waste used for heating and cooking (but then, estimates of it seem to vary wildly). Other than that, right now we're entirely dependent on fossil fuels.

A popular idea of high standards of living without corresponding increase in energy use seems pretty unlikely to me. Yes, we might be able to break the correspondence at some point, but energy use per capita in poor countries is so low, it's simply bound to drastically increase.

Much more promising path is energy generation without fossil fuels. Our best chance so far was with nuclear power. Unfortunately we suddenly stopped what looked like extremely rapid exponential growth in late 1980s after Chernobyl disaster - estimated 4000 people will die due to increased radiation exposure (much higher estimates based on linear no-threshold model can be safely ignored, as LNT model is known to be empirically completely wrong).
By ridiculously naive estimations, if this growth continued exponentially as it seemed to have up to Chernobyl, nuclear energy would provide around 20% of total energy production today, or half of all oil could be eliminated. Now this kind of extrapolation is ridiculously naive, but you should give it some thought.

So could we switch to nuclear at ridiculously rapid pace? We would need over 50 times as much nuclear capacity to generate enough energy to cover increased energy use by 2050, what seems ridiculously optimistic by any standards, not to mention nuclear proliferation issues. It looks somewhat less ridiculous when expressed as 10% a year growth (or doubling every 7 years like pre-Chernobyl), but it's just not going to happen.
There aren't that many renewable options. Current global energy consumption is 15 TW, and it will be about 39TW by 2050 by extrapolation. The only large scale renewable energy source we use, hydroelectricity, is nowhere close to our needs, and neither is geothermal really, if we look at what small fraction of it we can practically extract thanks to high local concentrations. We're left with just two then - wind, and solar. We would need to use 5% of all wind energy of Earth to fill the energy gap.

In 2008 wind power had 121GW of installed peak "capacity", and produced 260 TWh of energy (what corresponds to 30GW of actual production). It would require order of 1000x increase in wind power, or 19% a year growth, to make it fill the energy gap. And the actual growth is actually fast enough at 29% in 2008. But there are clear diminishing returns here - best locations will be taken, and massive subsidies cannot really scale to TW levels.

The other alternative is solar power. Just Earth-based solar is so plentiful that using even 0.02% of solar power now, or 0.045% of it in 2050, we can replace all of world's energy use. And it doesn't stop there, with space-based solar we can get numbers many many orders of magnitude larger. It would sort of need a space elevator, but we might get there by 2200.

Anyway, while solar is undoubtedly our long term future, it doesn't look that bright in the short term. There's only 15GW of solar thermal installed, and mere 0.6GW of solar thermal. And these are capacity numbers. We're talking x10,000 increase to cover the energy gap, or 25% a year increase, or x200,000, 35% per year for solar thermal. Now these numbers are actually what we see now, but as I said for wind, these are all based on massive subsidies, and there's no evidence that they're scalable to TW range at all.

By the way all this is based on assumptions that all energy is basically equal, and all greenhouse gas emissions is energy production. Neither of these are true - electricity is relatively easy to replace with renewables, at least if we ignore their intermittent character, but transportation much less so. And even with 100% solar electricity-based economy, there would still be some significant sources of greenhouse gases like methane from agriculture - so whatever we estimate here is bound to be overly optimistic.

With all this said, thanks to better energy efficiency, and widespread use of renewables, it's quite likely that humanity energy use by 2050 will be much lower than baseline number of 74.5 billion tons a year. On the other hand expecting it to be vastly lower than current emissions is just extremely unlikely.

And here comes the final solution - without genocides, without unexpected revolution in either nuclear power or renewables, we will have much more CO2 in atmosphere in 2050 than now. And due to massive lag it has, it will keep warming Earth for a very long time after we finally get our emissions to decrease. So if we cannot agree to live in warmer world, we have to geo-engineer to survive the next century without massive and irreversible environmental change.

Nature did some real life large scale geoengineering testing for us, and it works without major side effects (you probably haven't even noticed the experiment in 1991). Economic estimations put it in ridiculously cheap range - some schemes are estimated to cost just 375 million dollar a year, or just 17,000 Toyota Priuses (seems like a highly relevant unit of measurement) - for comparison replacing every American car with a Prius would cost 5 trillion USD, and it would still do very little towards limiting global CO2 emissions. And stratospheric sulphur isn't the only solution, just the one we have best data on (by the way it's not about polluting more, only about moving less than 1% of existing pollution to higher layers of atmosphere).

So the alternatives are:

Outright genocide
Indirect genocide by forced poverty
Accepting the warming
Increase in nuclear or renewable energy far beyond anything that can be realistically expected in this timeframe
Geoengineering

Guess which one is the least bad?

Now I just need to blog on abortion, and I win blogging controversy bingo.

Why Malthusianism refuses to die

taw — Sun, 01 Nov 2009 07:22:00 -0800

It's astonishing how popular Malthusian catastrophism is, in spite of such spectacular evidence to the contrary. As disaster keeps failing to come, its details are changed, but the core prediction that we're all doomed doesn't budge. In this post I'd like to explore some evidence against Malthusian ideology - as with so much evidence to the contrary, it like creationism long ceased to be anything else than ideological view.

Classical Malthusianism

But first, let's say what I'm criticizing. Malthus' idea was that population growth is exponential, while food production growth is arithmetic, therefore we're all going to fall back into extreme poverty due to overpopulation, until we're poor enough to be dying in sufficient numbers to counteract that.

The second part is, and I'm not making this up - "therefore, we should starve and abuse people who lost their jobs in a recession to lower taxes on the rich". That's a forerunner of the Real Business Cycle theory if there ever was one. The predictable result of being tough on the poor was that the working poor were starved to the point of eating pig fertilizer, beaten, and sexually abused.

But that's not the worst of Malthus-inspired politicians - they decided to starve one million poor people in Ireland. During the alleged "famine", Ireland was a major exporter of food, with its food production even at lowest point estimated to be enough to sustain population twice as high as it had. And there was well known solution to such famines already tried in 1782–83 of temporarily stopping food exports, but those good Malthusians would rather have a bit of a genocide than to stand in ways of profits.

Anyway, I'm not here to criticize consequences of Malthus' ideology, just its factual basis. It's been 211 years since Malthus' infamous essay, how far behind are we in food production?

Oh wait, actually food production looks pretty exponential on this graph, while population growth looks rather linear. It's not even that - according to virtually all projections population growth is slower than linear, and world populations are due to stabilize fairly soon, here's Wikipedia:
Unfortunately I haven't found any useful food production statistics going all the way back to 18th century, but by now it should be very clear that these relationships do not hold.

Yet, Malthusianism refuses to die. The most popular mid 20th century's neo-Malthusian Paul Ehrlich famously wrote in 1968:

The battle to feed all of humanity is over. In the 1970s and 1980s hundreds of millions of people will starve to death in spite of any crash programs embarked upon now. At this late date nothing can prevent a substantial increase in the world death rate...

Or more specifically - "I have yet to meet anyone familiar with the situation who thinks India will be self-sufficient in food by 1971" and "India couldn't possibly feed two hundred million more people by 1980."

That was in the middle of yield increases so spectacular they caused major difficulties to even harvest it all (High yields led to a shortage of various utilities — labor to harvest the crops, bullock carts to haul it to the threshing floor, jute bags, trucks, rail cars, and grain storage facilities. Some local governments were forced to close school buildings temporarily to use them for grain storage. as Wikipedia says). India became net exporter of cereals by 1978.
You'd think that after two centuries of predictive failures people would have given up, but no. The latest neo-Malthusian fad just a few years ago was "peak oil". It sort of fizzled since, but in case you forgot, here's an awesome YouTube video describing peakoilnik ideology while dancing:

"There won't be much food anyway, as oil is used to make fertilizers and pesticides. Up to two thirds of the world population may not make it. Four billion people may not survive."

Because as they say, we're basically "eating oil" as the meme says. It doesn't bother them much that oil is used at no step of fertilizer production - it's all natural gas, and entire 3-5% of world natural gas (1-2% of world energy use) is spent on fertilizer production (not even as energy source, more as convenient hydrogen source easier than electrolysis of water), but alternatives based on coal, and hydroelectricity are well known and used depending on local availability. China already produces most of its fertilizer with coal. The first commercial hydroelectricity-based fertilizer plant was opened in 1905, over a century ago, and in case you're wondering use of electric plowing equipment dates to late 19th century.

Pesticide production is so ridiculously low, by some back of an envelope estimates takes about 0.06% of world's oil, and is trivially replaceable by coal if need be (I know of no serious estimates, as it's too small to bother anyone with a clue).

What a Malthusian world would look like

So why so many people believe in Malthusian-inspired catastrophism? I have a two-fold hypothesis. First, some people just enjoy beliefs that we're all going to die. It seems that virtually every disaster scenario, no matter how plausible or not - attracts countless believers.

But there are people smarter than that who still believe in Malthusianism. Here's the second part of my hypothesis. When people look at history, to see if it matches some theory, they just look at things that would be observable if theory was true, and see if there are present. They don't do proper Bayesian updating.

Here's a simple example - let's say you have a theory which says that clash of civilization causes warfare. Then the naive thing to do would be to look at history, note a few wars between civilizations, and take it as evidence for the theory. It would also be wrong, as far more numerous wars were fought within civilizations, and proper Bayesian thinking must look at both.

So let's take a look at evidence for Malthusian worldview against the null hypothesis that food production is just like any other commodity, and not any more connected to population than production of clothes, bricks, or paper.

If Malthusianism was true, food production per capita would be pretty much constant - because any increase in food production would result in increase in population, and decrease in food production would cause mass starvation, and adjust population accordingly. But if Malthusianism was false, food production per capita would also be pretty much constant, because what would be the point of growing more food if there was nobody to eat it? So there's no evidence either way.

If Malthusianism was true, everyone would be poor. Not only is this ridiculously false, now, historical estimates of life expectancies vary wildly from 18 to 60 for the same time period.

Malthusianism predicts almost all food production would be in whatever generates highest number of calories per hectare. Non-Malthusianism predicts food production would be much more varied - and it is so. Just the fact that we're keeping farm animals, which by their nature only destroy and not produce nutrients, proves Malthusianism wrong. There might be exceptions, but most grazing lands would produce more calories when cultivated, not to mention Malthusian ridiculousness of feeding animals human food as we do.

If Malthusianism was true, any sudden decrease in population, due to wars, famines and such, would result in rapid rebound, as agricultural capacity existed to support much larger populations, proven by its previous existence. If Malthusianism was false, population would have no particular reason to rebound. Here the evidence is very clear - there are numerous example of major population loses which were not recovered for centuries. Two best examples affect entire Europe, which used to have high populations at height of Roman empire, then fell to very low levels and didn't recover until 12-13th centuries, almost a millennium later. Second fall was due to Black Death, recovery from which wasn't complete even two centuries later.

Malthusianism predicts that it would be extremely unlikely for vast amount of spare arable land to exist - yet this was exactly the situation for centuries in early Medieval Europe, with vast tracks of unused lands.

Malthusianism predicts that famines would happen during times of higher population a lot, while almost never happening during time of low population. But even a brief look at Wikipedia's List of famines shows plenty of major famines both during population lows of early middle ages (5th century famines in Western Europe, 750's famine in Spain, 809 famine in Frankish Empire, 963-964 in Ireland, 1005 in England, 1016 all Europe, 1030-1032 France, 1066 England again, 1097 France again). There were supposedly 95 famines on British Isles alone during Middle Ages. In spite of very low population levels, we can see plenty of them, and we'd probably know about more if it wasn't for horrible Medieval record keeping. Notice that during all that time there was plenty of undeveloped arable land!

During second period of low populations after Black Death, there are major famines 1390 in England, 1481-1483 in France, 1504 in Spain, 1518 in Venice and so on. I'm not even going to mention that modern periods of European population explosion had fewer famines than any other time in history.

Non-Malthusian view correctly predicts that famines and population levels are completely uncorrelated. Famines are caused not by the mythical "overpopulation", but commonly by wars, government action (like in Ireland, and 1930s' Ukraine, notice how democratic governments have virtually spotless track record of no famines, in spite of plenty of agricultural disasters), and most of all - simply by major delay between increase in agricultural inputs and outputs. It doesn't matter that there's plenty of unused land around, developing it would take years, and plenty of labour, capital, seed, and farm animal input, before any food was grown on it - all of which being in short supply during famine anyway. By the time food was growing the famine would be long over. To give you some indications of time scales involved, settlers in new developments were freed from any taxation for up to 24 years. During that time net effect on food production would be negative.

Most of all, current overwhelming explosion of wealth is completely and utterly incompatible with Malthusianism. Even the second worst country in the world - Angola - has life expectancies far greater than world had 100 years ago (one tiny country which doesn't has 26.1% HIV infection rates, and is too poor to afford HIV drugs - hardly a Malthusian reason). In spite, or maybe even due to, population explonion, world's average life expectancy is 66.57 now. This is level not reached by even the richest countries before 1930, and by most developed countries well into 1940s an 1950s.

However hard I try, I cannot find any evidence for even very broadly constructed Malthusianism that cannot be explained as well or better by treating food as a normal commodity. So worry not - we're not doomed.

Widespread capital punishment never existed

taw — Sun, 01 Nov 2009 00:13:00 -0700

There's this fancy Western European idea of abolishing death penalty. Wikipedia even has a nice map, with blue countries having abolished death penalty entirely, red still carrying it out, and green/orange having it on the books, but not in practice.
But reality is different than that. Most of the red countries almost never carry out the death penalty, reserving it for extremely unusual cases - in 2008 estimated 5727 people were executed, over 80% of them in mainland China, other than that only Iran and Saudi Arabia doing three-digit numbers. Other big red countries like India, USA, Indonesia, Pakistan, Japan and so on very rarely execute people.

As 59 million people die every year, your chance of getting executed is like 1:10,000, or more like 1:200,000 if you don't like in relatively execution-happy China, Iran, or Saudi Arabia.

Numbers like this are ridiculously uncommon - about 100 people die every year from peanut allergy in just United States. And "accidental suffocation and strangulation in bed", which sounds like a codeword for "BDSM gone awry" causes 327 deaths annually in just USA.

It's not just due to lack of crime - annually, half million homicides take place, and many countries have or had death penalty for lesser things. So if you're a killer, the chance of getting a lethal injection for it are 80:1 (assuming they get the right person), or more like 1500:1 unless you live in one of the three aforementioned countries.

Add to that the usual overconfidence criminals have, and it's clear that fear of death penalty is the least of criminals' concerns. Especially since even in United States you're over three times as likely to die as a result of a shootout with the police than due to legal execution, so we're talking at most 30% extra deaths factor here. The idea that people will stop carrying out crime due to fear of capital punishment is just highly dubious.

History lesson

So is this it? Did the abolitionists win? Not necessarily. Googling indicates that capital punishment for crime was never particularly widespread. Yes, governments killed loads of people - massacres during wars and occupations, killing political opponents, summary executions to quell social unrest, and some such. But legal capital punishment of citizen committing ordinary crimes after proper judicial process? Very unlikely.

Here's some statistics. Stalin killed about 60 million people, and Hitler about 20 million (estimates vary considerably, but let's take those numbers). How many actually received death penalties? Only 40 thousand from Nazis (1 in 500), and 750 thousands from Soviets (1 in 80). And these were mostly political opponents, war and civil war cases and such - not proper criminals.

How about other countries? Czarist Russia averaged less than 60 a year, and remember that many of them were political cases. Between 1770 and 1830 England had allegedly bloodiest criminal code on Earth, with death punishment for "writing a threatening letter", hanging out with gypsies, or "strong evidence of malice" and over 200 other crimes, while still averaging barely above 110 executions annually. Even the infamous Spanish Inquisition barely managed 20 burnings a year (but then, these were mostly religious and political opponents not criminals, so maybe they shouldn't be included here).

It's just really difficult to find any government, that routinely carried out large number of legal executions of criminals during time of peace.

When governments kill

That's not to say governments don't kill. But these killings rarely affect common criminals. Nazis killed people for being of wrong ethnic group. Soviets killed people for being of wrong social class, or just because. The same 18th century United Kingdom which was so cautious with criminals of right ethnicity, killed million people for being poor and Irish (and no, it wasn't any Malthusian bullshit, Ireland was a major food exporter even during this politically caused famine).

Even countries which don't execute criminals at all like Denmark with no executions since 1892, change their mind during and briefly after war time, when 54 people were executed (it's not really a legal capital punishment, as it's all ex post facto). Likewise in Finland, after not executing anyone since 1825, about 550 people were killed during war time (unlike Denmark where it was mostly proper war criminals and people supporting Nazis, here you could get a firing squad for conscientious objection, and without even a real trial).

What does it mean for capital punishment debate?

Other than perhaps in mainland China, Iran, and Saudi Arabia, the debate is purely speculative now. There is virtually no capital punishment anywhere. And other than during wars, occupations, major civil unrests and alikes, it was extremely rare for many centuries. Before that, executions might or might not have been more frequent, but it's hard to talk about proper legal proceedings, so they don't count.

People who support death penalty have a big problem, as there won't be any statistics showing serious effects of widespread application of it during time of peace, as it never happened. There are some bullshit statistics, like those in the original Freakonomics book (very entertaining book, by the way), but due to vast non-random annual variations in number of homicides and executions, we're talking about effects of a rounding error, and social science just cannot do anything useful with them.

So it's mostly about what makes you feel better. If you're a real Christian, then you should follow Jesus and be against death penalty. If you're a fake Christian like Southern Baptist, you should support death penalty because you hate black people more than you love Jesus. And if you're an atheist, do whatever you want, it's not like I can convince you.

Here, I solved it for you.

Medieval 2 Total War Concentrated Vanilla 0.07

taw — Tue, 27 Oct 2009 20:42:00 -0700

Here's another release of my Medieval 2 Total War Concentrated Vanilla mod.

Old releases suffered from the too-much-money problem, just like the pre-crash Goldman Sachs, so I decided to scale it down a bit. Instead I introduced massive free garrisons.

Here's the full list of changes relative to vanilla:

For more dynamic game, all units move 75% faster on strategic map.
One big problem with vanilla was that money didn't matter. No matter if you had a lot or not, there wasn't all that much to spend it on. So now all buildings take 1 turn to build, but are 50% more expensive. This drastically increases importance of money.
To add some strategic richness, mines give 2x the profits, and cost 3x as much (x2, +50% like all other buildings). Some settlements with rich mines like Zagreb or Vienna are worth a lot, maybe even worth starting a war over.
There used to be 50% increase in settlement and merchant trade, and mine profits used to be 3x, but this was a serious overkill. If you manage well, settlements and merchants will give you more money than in vanilla as you can build relevant buildings faster.
There also no longer double population growth penalty/bonus for tax rates - it was an overkill.
To help small nations, King's Purse increased 2x (that is - free money a nation gets unrelated to its settlements - from 1500-2500/turn to 3000-5000/turn).
To reduce annoyance with rebels, rebel and pirate spawn rates reduced 10x. (I'm not actually sure if it reduced them as much as the file says it would, but it feels more or less right now).
Cavalry, especially heavy cavalry was seriously overpowered. Small groups of knights could easily crush infantry armies ten times their budget, so infantry was useless other than free militias to patrol cities, and cheap grunts to operate siege equipment. Because with cavalry-centric warfare missile units are useless, this limited army composition even further - and destroyed rich tactical variety that M2TW could possibly have. So all cavalry is now 50% more expensive to recruit and upkeep. Cavalry is still extremely powerful when used right, except now other kinds of armies are useful too.
Bodyguards were particularly overpowered - in early periods they simply destroyed everything, were numerous, self-healing, essentially free (you get new candidates for adoption if you lose old generals all the time), and as every faction has identical bodyguards they severely reduced tactical richness of the game. So bodyguards units are the half size now (upkeep cost is proportional to unit size, and they're cavalry, so mod's bodyguard unit costs 75% of vanilla's). Old versions of the mod reduced their hitpoints from 2 to 1, but it no longer happens, as this made bodyguards overnerfed in late periods. Unfortunately there no way to get the balance right - as bodyguards stay the same, but other units massively improve, either bodyguards will be too strong early on, or too weak late on. I moved the balance point somewhat towards the weakening.
Vanilla sieges are ridiculously easy, with a single unit of ballistae being able to take down large stone walls. This made sieges was too easy, and as a consequence turned the game into a siegefest. So now gates and walls (but not towers) are 5x stronger. Serious artillery will be needed to take anything but the weakest walls.
Towers now have 2x fire rate for normal ammo, and 1x fire rate for flaming ammo. Increasing flaming ammo rate too much as old versions did made siege equipment like siege towers almost useless. But high normal fire rate should help defenders a lot, even for undermanned settlements that AI leaves. This wouldn't be necessary if AI was competent, but we know well it's not, and leaves valuable settlements defended by one unit of Peasants far too often. This balances AI stupidity considerably, and also frees the player to play more strategically by making defense easier (this is not as essential as in old versions of the mod due to increased free garrisons).
Tower activation range is 8x larger (a bit of reversal to Rome Total War). So you cannot shut down towers just by scaring off defenders with some arrows like in vanilla. Strong walls, and massive towers' firepower together mean sieges will be very difficult. Prepare a lot of artillery, or spies, or massively overwhelming force that can take severe loses before giving up. I'm yet to take a well-manned Fortress in this mod (but then AI often mans its settlements insufficiently, and there are other options like waiting their food supplies out, or provoking a sally).
For consistency with making sieges harder, spies cost 2x to recruit and upkeep. And AI is quite competent at counter-espionage operations, so spy flood doesn't work all that often.
To help defenders in sieges, missile infantry ammo increased 2x. It very rarely affects field battles, as in those they rarely get to expend even half of their ammo.
A big change is doubling free garrison in cities. Village 0, Town 4, Large Town 6, Small City 8, Large City 10, Huge City 12.
The biggest change is introducing free garrison in castles. Motte&Bailey 1, Wooden Castle 2, Castle 4, Fortress 7, Citadel 12. In vanilla castles are either left barely defended, or are massive bleeding money hole - failure both ways.
Just like with cities, castles' basic infantry units can be free garrison, but not cavalry or expensive infantry. Basic infantry is one with morale up to 5, and cost up to 155. So Peasants up to Armored Sergeants, and Peasant Archers up to Genoese Crossbowmen, Venetian Archers, and Hand Gunners can all be free when garrisoned. On the other hand Dismounted Knights of all kinds, and most faction-specific elite units cannot. I guess it might introduce some faction unfairness, but I can live with it.

Overall, what you get is much tactically and strategically richer game, with a lot larger differences between factions, but one that still feels a lot like vanilla Medieval 2 Total War. In the mod defense is easy, so you don't have to mindlessly attack everyone.

The main problem left is the spectacularly stupid AI, about which I cannot really do that much.

I also did some other experiments, that didn't get to the mod:

Making ocean-traveling ships available from the beginning (well, as soon as you can build sufficient port infrastructure) enables fights with natives. But Americas are disconnected from the mainland Europe, and it doesn't really feel like it's part of the same game. Try it once to see what fighting Aztecs is like, then forget they're even part of the game.
Making Mongols (or maybe even Timurids) come earlier. It's a lot more fun to fight Mongols, but then if they come very early, they will totally crush AI on auto-resolve. Unfortunately auto-resolve favours stronger armies too much, so they won't even suffer too much loses, and you'll have to deal with them yourself. One full stack of Pavise Crossbowmen and Feudal Knights led by semi-decent General (or equivalent) can defeat one full Mongol stack without that much trouble in field, and well defended settlement with ballista towers completely destroys them. It's another thing to try once. I haven't fought Timurids yet.
Making Gunpowder available immediately unfortunately counters the effects of very strong walls, and yet good gunpowder infantry and cavalry won't be available until much later. I'm not even sure if gunpowder units are better than plain Pavise Crossbowmen, they don't seem to be from the stats, and I never had an opportunity to play much with them.

Anyway, download the mod here.

Blog posts about previous releases: 0.02, 0.04, 0.06.

Folk Complexity Theory

taw — Wed, 14 Oct 2009 19:03:00 -0700

There's one kind of posts on this blog that get the most hostile reactions. It's not politics (nobody reads those), flaming programming languages, or computer games - the most controversial subject is Big O analysis! Here are relevant posts:

Basically I claim that Big O is useless in most practical situations. And I get flamed a lot. At first I tried to explain to the flamers how they are mistaken, then I basically ignored them after explanations proven to be pointless, but now I see that it's all just big misunderstanding. What I call Big O analysis is a fairly precise construct of computer science. But that's not how most people use the Big O notation - they follow Folk Complexity Theory, which I want to describe here.

The Official Complexity Theory

First, here's a brief repetition of Algorithms and Data Structures 201, or whatever your course explaining this was.

An algorithm is O(f(n)) if in a particular model of computations, if there exist some constants n₀, and c, such that for all input lengths greater than n, computation time is less than c × f(n). A problem is O(f(n)) if in particular model of computations there exists a O(f(n)) algorithm that solves it.

Models of computations are things like different varieties of Random access machine, and different models usually only differ by some logarithmic factors. Sometimes you want to assume that adding two numbers takes constant time, sometimes that it takes time proportional to logarithm of their sum, and details like that. If you disregard logarithmic factors, you don't have to be all that precise about which variety of RAM you're using. That's about it.

In particular, if the problem is only defined for constant-sized input, then by definition it has complexity O(1). There exists some c, such that for all possible sizes, running time of the algorithm is less that c × 1.

So breaking AES-128 is O(1). As is breaking RSA-4096. Breaking RSA-n for arbitrary n isn't, but that's entirely unrelated problem. Breaking AES-n is meaningless concept, as AES isn't defined for arbitrary n.

In particular classes like P, and NP, don't depend much on what reasonable computation model you're using. They're defined on Deterministic Turing Machines for reference (which is polynomial factor slower than RAM), but if something is P on DTM, it's P on RAM and vice versa.

How cryptography handles it

As I got a lot of flames about my claim that breaking RSA-4096 is O(1), let me explain how cryptography handles it. So cryptography doesn't use Big O notation much.

Security is measured in bits - a cryptosystem has k bits of security if the attacker must perform 2^k times as many operations as legitimate user to defeat security of the cipher. For some types of security key size and security are related - for block ciphers security = key size, for hash collisions security = key size/2. For public key cryptography these bit measures can be only approximately specified in terms of key size, and whatever formula we have we're completely uninterested in its asymptotic form, as all feasible key sizes are small, and constant factors are of great interest to us.

The Folk Complexity Theory

And now the main subject of the post - The Folk Complexity Theory. When people say some algorithm is O(f(n)), they actually mean that for "normal" sizes of inputs, on actual computers, it's expected running time is between two "small" constants times f(n) times reasonable number of log(n) factors as long as they don't push the constant too high.

O is used sort of like Θ of the official theory. Folk O(n) algorithm is not considered to be O(n²).
So bubble sort is O(n²), as number of operation tends to be something like 10n². Logarithmic factor for addressing and arithmetic is not considered at all.
Merge sort is O(n×log(n)), as number of operation tends to be something like 10n×log(n). Notice how log factor for number of operations is included here as it's significantly higher than allowed small constants; but log factor for cost of addressing and arithmetic is not, as on normal computers it's constant up to 2³², which is very big n. And only doubles for 2⁶⁴, which is too ridiculously high n to bother. Exactly the same reasoning can be used for both log factors, so this is highly peculiar.
Quick sort is O(n×log(n)), the Folk Complexity Theory always deals with typical case, not worst case.
Breaking AES-128 is O(2ⁿ) because the constant is too big to be allowed. It doesn't matter that there's no n here, and AES is not defined for other key sizes. The problem must be reformulated to have n, no matter if it makes any sense or not. Number of operation like 2¹⁴⁰ is simply "not allowed", while 2¹²×2ⁿ is.
Breaking RSA-4096 is likewise O(2ⁿ), even though there's no n here at all.
Forging signature of n-byte message signed with RSA-4096 is, uhm... O(2ⁿ). Wait, n is already something else... O(2^k) then maybe? The problem is reformulated as RSA-k because large constants are not allowed, and usually there's no point saying O(2^k+n), as n is too small to matter for all practical n.
As an added bonus "NP" usually means Θ(2ⁿ), for real or imagined n. This is spectacularly incorrect (most problems are not decision problems; there can be no Θ bounds even for NP-complete problems until NP != P is proven, and every P problem is NP), but doesn't cause much confusion in practice.

This is the source of all confusion - I was talking about the Official Complexity Theory, while most Redditors interpreted it as Folk Complexity Theory statements, in which they were obviously untrue.

Is Folk Complexity Theory useful?

It's quite amusing that so many people flame me for saying the Official Complexity Theory is overrated, and don't even know it. But I'm not going to blame them - who needs it once they're past exams anyway? In practice Folk Complexity Theory is actually better heuristic than the Official kind. We program real computers. On real input sizes. ceil(log₂(n)) factors shouldn't be ignored, but ceil(log_4294967296(n)) factors which represent cost of arithmetic and addressing on most 32-bit computers hardware totally can and should. And so should all log(log(n)), which nobody ever bother with past exams.

The Folk Complexity Theory has weird status - it's neither mathematics like the Official theory, nor practice like actual benchmarking. It's just a bunch of heuristics dressed as something much more formal than they are. Often even fairly useful heuristics. Just nothing more than that, and not worth the flames.

The Web 2.0 of Metaphors

taw — Wed, 14 Oct 2009 18:30:00 -0700

After hearing that something is Wikipedia of whatever it is about, and getting slightly annoyed by it, I decided, in interest of science, to compile popularity ranking of "the X of Y" metaphors, by Google counting.

I found quite a few false positives, like "the livejournal of Nikki", and "the Linux of your choice", but upholding the glorious scientific tradition of ignoring every methodological issue that cannot be fixed, I completely disregarded them.

"the windows of" - 15,100,000
"the internet of" - 7,790,000
"the apple of" - 6,470,000
"the google of" - 3,160,000
"the youtube of" - 766,000
"the microsoft of" - 397,000
"the imdb of" - 307,000
"the craigslist of" - 303,000
"the livejournal of" - 290,000
"the ebay of" - 277,000
"the amazon of" - 262,000
"the facebook of" - 237,000
"the myspace of" - 235,000
"the linux of" - 197,000
"the wikipedia of" - 180,000
"the yahoo of" - 142,000
"the flickr of" - 124,000
"the ibm of" - 109,000
"the blogspot of" - 108,000
"the web 2.0 of" - 92,900
"the bing of" - 90,900
"the firefox of" - 63,200
"the aol of" - 63,100
"the napster of" - 61,200
"the msn of" - 52,100
"the digg of" - 23,600

It seems the most popular activity is explaining one web service as metaphor of another:

Is Craigslist the "Napster" of the Sex Industry
YouTube Redesign: Becoming the Google of Video?
Is Facebook: the Google of Social Networks
Google Earth Is the AOL of the Geoweb
MySpace is the AOL of Social Media/Web 2.0
YouTube.com — the Flickr of Video

There are also some unusual finds:

The Wikipedia of Slutty Dressing
The Craigslist of Antibiotic Resistance
Is Python the Apple of Programming Languages?

Another interesting find is that while I've been expecting a lot of "the X of sex", only Wikipedia (36,200) and Facebook (29,500) have decent number of matches.

"The X of porn" is a lot more popular, here Google (378,000), YouTube (354,000), and surprisingly Napster (22,200) are taking the lead.

The other most popular Internet activity, spam, got almost no hits.

Efficient go hypothesis

taw — Wed, 14 Oct 2009 10:00:00 -0700

Efficient market hypothesis states that you cannot really beat the market by much, all the prices are about right, and everybody behaves it more or less rational way. Yes, treating it too seriously leads to ketchup economics thinking, but it's often a good starting point - if you postulate that some prices are seriously wrong, you better come up with a story why they're wrong.

But let's forget about economics for a moment. Something very similar to efficient-market hypothesis applies to the game of go. Unlike most other games like chess and football, go is extremely balanced.

Black player, who moves first, pays 6.5 points for the privilege, what makes both players' chances of winning as close to 50:50 as it gets.
If one player is stronger, the weaker player gets as many free stones as necessary to get the chance of winning very close to 50:50.

What's interesting is that during the game, it's impossible to make a move that highly increases your advantage, or "beats the market" - if you could get significantly ahead by playing a stone, then the position was already unbalanced before you've done so due to this possibility. Values of positions consist of complex balance of territory, influence, flexibility, good and bad aji, and many other things - if you ask a computer program like gnugo to estimate balance of the board, you will get widely varying numbers with each moves, each totally ridiculous.

Just like on the market we cannot give correct formula for a price, but we know it's right because people trade at it; in go we don't have a formula for value of a position, but we know it's even if good moves by two players led there. If you postulate imbalance, you better come up with a story telling which move was bad and why.

Go players often perform some complex trades of moves called joseki - standard sequences that lead to more or less even results. Again, why even? Because neither you nor your opponent would purposefully play bad moves to fall behind, and if there are no good moves available it means one of you was already losing earlier.

Joseki example - every single move here and the sequence are about even

This requires significant shift in thinking compared to games where you act to get as far ahead of your opponent. You can play go like an index fund - just try to stay even with each move, and let your opponent make some mistakes and fall behind you.

That doesn't mean there's no room for creativity - sometimes there's only one good move to make, most of the time you have multiple good options with subtly different balances. These tiny variations are enough to win - there is no possibility of draw in go (not counting super-ko and such exceptional situations), so even if you get just 1 point ahead through the entire game, 0.01 point advantage per move, that's just enough. It's kind of like high frequency trades, which earn tiny profits, but with sufficient volume it's worth it.

Most professional games end with just a few points advantage for one side or the other. Or one of the players falls behind just a bit, and then risks it all on a great all-board fight to avoid the few point loss.

So as you can see, go is just like market. Except it never leads to crises costing trillions of dollars.

Practical O(n) problem impossible to solve even for n=1

taw — Sun, 04 Oct 2009 06:24:00 -0700

In computational complexity a fairly popular claim seen in most textbooks equates class P with practically feasible algorithms, and everything not known to be P (but usually within NP, "not known to be P" is an awkward thing we need to say until P!=NP is finally proven, even though we all know it to be true already) with not practically feasible algorithms. While it's often noted that it might be untrue for some cases, with some P algorithms being impractical, and some not known to be P algorithms to be practical, such disclaimers usually consider these cases to be contrived and not worth any further consideration.

Here's a really simple example of a genuinely useful problem in P (linear even) that's not practical at all.

Utterly impractical O(n) problem

Imagine problem of falsifying digital signature for a message in some reasonable digital signature scheme. Inputs are public key of constant length, and message of length n. The algorithm is as follows:

Extract private key from the public key by brute force, or some other algorithm (constant time)
Sign the message (linear time in n required just to read the message, and no more than linear time necessary)

So we have an O(n) algorithm, and because we need to read entire input to sign the message, no faster algorithm is possible. Therefore the problem is Θ(n).

Yet, unless someone breaks the public key system and provides vastly faster method of extracting private key based on the public key, we won't be able to solve it even for n=1.

This problem is of great practical importance - the entire online banking system would fail, and billions of dollars of fraud would happen if someone could practically solve it. So it's not fair to call utterly unfeasible P problems contrieved.

Google Everything and some alternatives

taw — Thu, 24 Sep 2009 06:04:00 -0700

Do you feel a bit uncomfortable using countless web services all operated by a single company? I can feel your pain. Here's a list of things I use (or tried but didn't like), and some alternatives if I know any.

Google Search

It used to be the case that Google Search was vastly better than anything else. Not longer so.
Don't believe me? Try the Blind Search - it searches the same term in Google, Yahoo, and Bing, then you can vote for the winner. Most of the time the results are extremely similar, and the scores are Google - 41%, Bing - 31%, Yahoo - 28%. Google still leads but not overwhelmingly.

While with text searches there's no reason to switch away from Google, Bing Images and especially Bing Video is quite significantly better than Google equivalents. Bing is simply a better search engine for porn than Google, they know it, and even advertise this fact:

I recommend we start using the word "to bing" as "to search Internet for video porn", just like "to google" means "to search Internet for textual websites".

Google Mail

I tried plenty of other webmail providers, and plenty of offline email programs. Even some Intranet email solutions like Zimbra and Exchange. Google Mail is so overwhelmingly better than anything else it's not even funny.

Hotmail, and Yahoo Mail, which I tried as throwaway spam accounts, are just too painful to use. There are no alternatives, it's Google or pain.

Google Maps

Google Maps was quite revolutionary when it first appeared. Unlike with Google Mail, here the competition did catch up quite successfully. Yahoo Maps, Bing Maps, MultiMap, StreetMap (well, that one's UK-only), and probably some others, are all quite usable for directions, and all suck for searching local businesses.

I've heard some drivers having preferences for one or the other, but as a public transport user/occasional cyclist, I'm still waiting for the first to offer decent TFL integration.

YouTube

YouTube annoys me considerably - they don't want to host anything even remotely risky. They limit access to videos by geography, remove those that include some copyrighted music in the background to unrelated funny thing.

On the other hand they have no problem hosting this video of a brutal rape (NSFW).

There are some alternatives, none anywhere near the size of YouTube. Not counting the porn tube of course. I have limited experience with them, so no in-depth comparison here.

Google Chrome

It's nowhere near being the most popular browser. I'm also not sure what's its main selling point, other than tab sandboxing, which Firefox badly needs.

Other than that, Firefox is vastly better. It has countless useful plugins. It has Greasemonkey, and Greasefire. It has Firebug. It has AdBlockPlus. Chrome isn't even close.

Not like I'm complaining, if it can get even one user away from IE, it's a win.

Blogger

Let's face it - Blogger is a pretty crappy blogging platform. But then I haven't heard users of any other being happy about theirs, so perhaps all are equally bad. And effort of moving the blog somewhere else (plus losing the URL) seems too big for me to bother.

In particular the way they handle spam comments is simply atrociously bad. I get a notification of a comment, and then it's about ten clicks to remove it if it's spam, instead of just one or two.

Google Reader

It's basically an online RSS reader. What I hate about most desktop RSS readers is that they tend to push notifications onto you, and don't live inside the most natural place possible - the browser.

With Google Reader Watcher, Firefox tells me how many unread items there are, and which website they're for. Which is exactly as much notification as I need.

I think the only thing left to want is "open every new item in a fresh tab" function, but there's probably some Greasemonkey script for that, or if not it would be pretty simple to write it.

I don't see much need for alternatives.

Google Talk

The main problem with IM programs is that everyone is on a different network, so you need way too many programs to talk with everyone. Adium has most of them. Except Skype. And the most basic functionality only - not webcams, no microphones, not even all the icons.

I don't have particular problems with Google Talk, but I'm going to use this unique opportunity to rant about Skype a bit. OSX version of Skype doesn't seem to have any way of disabling text notifications of contacts becoming available or unavailable. So it constantly spams me with irrelevant garbage. It's not possible to use Skype network from other programs even for IM chats. Many networks try this restriction - and abuse their control over access to other users, but Skype seems to be the only one which did it successfully. Would you be happy if you needed an iPhone to call a person using iPhone, a Nokia phone to call a person using a Nokia phone, and basically 20 different phones to speak with everyone you want to? Me neither, so why are we allowing exactly the same thing in the IM world?

Google News

I don't see Google News as solving any real problem. My main problem with news is that they focus on events of the day, while the important things are usually long ongoing processes, with little need for daily updates. And if you're away for a week or two, there's no easy way to look back at this time and see if anything important happened.

As far as I can tell, it's equally true for all alternatives. If you know a good way to follow important news, tell me about it in comments.

Google Docs

Google Docs is a pretty cool webapp for editing documents together. I have irrational hatred towards WYSIWYGs, especially web WYSIWYGs, so don't ask me about fancy advanced features, but it solves the basic need well enough, and I don't see what could be the big selling point for its alternatives.

Google Code

Google Code is a nice hosting service for Open Source projects. It supports Subversion (obviously), and Mercurial, but not git.

Some of the alternatives are SourceForge (both Subversion, and git, also some other VCSs I don't care about), github (only git, duh). I'm not sure how bad github's limitation of 300MB per open source repository is - my personal code repository's .git is easily bigger than that, but then I freely store binaries and stuff in it (like font files for jrpg).

Overall I don't really see any major advantage for Google Code over it's countless competitors, and it's not even particularly big.

Google Analytics

I'm going to be pretty clueless about it, as Google Analytics is the only one I tried.

Good things - it gives you pretty decent overview of traffic, and is really easy to install.

Bad things - no decent API (screen scrapping possible but quite painful), no access to raw data, I'm not a big fan of their interface, which seems to be focused on all the wrong things.

If I started a website these days, I would definitely try something else, if only to see if it's any better. Switching an existing site would probably not be worth the effort, as all historical data would get lost.

Summary

These are just some of Google services and programs I tried. It's scary in two ways. First, a single company has way too much control over the Internet, and over people's often private data. Google has been relatively well-behaved as far as huge soulless corporations go.

Second, and perhaps more important problem is - what the fuck is the entire competition doing? The only reason people use Google Absolutely Everything is because most of everything else sucks so much. Google didn't succeed just once or twice, it had long stream of spectacular successes where everyone else failed (search, maps), and often even failed to catch up years later (webmail, text ads). Seriously, how difficult is it to innovate a bit online? Difficult enough that a single company does most of it? Hardly anything good on the web ever came from Yahoo, Microsoft, Apple, Sun, and countless other huge Internet companies in ages. The only other major company that seems to be doing anything interesting is Amazon. A damn bookstore. A damn bookstore is doing more web innovation than all the software and web giants put together.

Yes, there are still startups doing the cool stuff, but most of them stop as soon as they get bought. Like del.icio.us, which was revolutionary, but then got bought by Yahoo, and is stagnant ever since.

In search for Just Wars

taw — Thu, 24 Sep 2009 03:51:00 -0700

I know people don't really read my political posts. If I wanted to be read, I'd be writing angry rants about video games, programming, and Harry Potter fan-fiction.

Here's a quick post about politics, namely about just wars, and other violence. Right wing nuts are generally pro-war, no matter who's fighting whom. What's surprising is that even left wing nuts are pro-war. Here's some Chomsky, it doesn't get much left-wing-nuttier than that:

ROBERT B. SILVERS: ... Under what conditions, if any, can violent action be said to be "legitimate"? ...

NOAM CHOMSKY: My general feeling is that this kind of question can't be answered in a meaningful way when it's abstracted from the context of particular historical concrete circumstances. Any rational person would agree that violence is not legitimate unless the consequences of such action are to eliminate a still greater evil. Now there are people of course who go much further and say that one must oppose violence in general, quite apart from any possible consequences. I think that such a person is asserting one of two things. Either he's saying that the resort to violence is illegitimate even if the consequences are to eliminate a greater evil; or he's saying that under no conceivable circumstances will the consequences ever be such as to eliminate a greater evil. The second of these is a factual assumption and it's almost certainly false. One can easily imagine and find circumstances in which violence does eliminate a greater evil. As to the first, it's a kind of irreducible moral judgment that one should not resort to violence even if it would eliminate a greater evil. And these judgments are very hard to argue. I can only say that to me it seems like an immoral judgment.

Oh, really? So, could you please point me to some examples of wars that successfully eliminated greater evils? The criteria are, you can agree that they are sane:

Without the war, very large number of people would have died or otherwise suffered.
There was no other plausible way of saving them, and all alternatives have been tried.
Thanks to the war, most of those people were saved.
Few people died or otherwise suffered in the war, especially civilians, but also participants. Some suffering is allowed, but only if it's vastly overwhelmed by suffering prevented.
It must have been possible to predict it advance that this success was very likely. Lucky wins don't count. And of course, it must have worked. Failed good intentions don't count.

I have really difficult time finding out wars that fit these criteria, even with very liberal grading.

The latest war in Iraq

The latest war in Iraq is easily not anywhere close to a just war. Saddam wasn't a nice guy, but amount of suffering he was causing recently was quite insignificant as for third world dictator standards, so it fails on count one, and automatically two and three.

According to Lancet estimates there were 1.3 million excess deaths due to it. Many other Iraqis suffered in other ways. So fail on count four.

Only fools would have expected an easy win, and being greeted with flowers, it's the Middle East after all. All claims about WMD, Saddam's connections to Al-Qaeda and so on don't count, as they were known lies, and even if Bush believed them, they would fail by count five.

Score: 0/5

The latest war in Afghanistan

This war is slightly better, as number of casualties on all sides is closer to 60 thousand, not counting another 10 thousands or so in the spill-off to Pakistan, what's most likely an underestimate in a poorly researched place like Afghanistan, but still much lower than Iraq's. And Bin Laden was in Afghanistan.

Still, it fails on all counts. There were people who died in Al-Qaeda terrorism, but other than a single well-known case, repetition of which was fairly easy to prevent, their numbers were really low. Even with 9/11 much lower than number of people who could be expected to die in the war, so count one fails.

There were plenty of plausible alternatives for getting or killing Bin Laden, and dealing with terrorism, so fail at criterion two. See IRA and ETA for good examples. Or even Clinton-style targeted killings.

Terrorism increased greatly after the war, so it fails criterion three. Count four doesn't fail that badly, assuming figures are not overly underestimated (they usually are, counting only direct deaths), but it's still pretty large number just to catch one guy. Let's be generous and count it as a point.

Anyone who expected an easy win in Afghanistan must have slept through history classes. And Afghanistan is still in chaos, terrorists still terrorize, and Bin Laden was never caught or killed, so criterion five spectacularly fails.

Score: 1/5

The Second World War

Let's move back in time to World War II, widely proclaimed to be the most just war in history, without any basis in fact.

So first, suffering. This is very likely - if everyone just gave Hitler what he wanted (mostly Central Europe to share with Stalin), millions of people would be reduced to status of second class citizens, and depending on how it would go, hundreds of thousands or even millions would die. It's not certain if large scale death camps would be implemented without the war - it seems Nazis were fine with Jews and everyone else they didn't like just getting out of their country (500,000 or so emigrated from Germany before the war, also see the famous Madagascar Plan), but let's assume they really wanted to perform massive genocide. In any case, the first criterion is quite plausible, so full point.

The second criterion fails. The obvious plausible alternative of letting people persecuted by Nazis emigrate to other countries, most were not terribly happy about letting them in, even through Nazis were quite glad about just kicking them out (first confiscating most of their property). Forced resettlement would suck, but it still beats getting gassed.

Now, the third one fails miserably - Nazis killed pretty much everyone they wanted to kill. Most people who would have suffered living under Nazi dictatorship, were instead forced to live under Soviet dictatorship. Approximately 70 million people have died, most of them outside territories that Nazis would have if the world decided against going to war with them. So it's spectacular failure on the counts three and four.

There could be no expectation of an easy victory - spectacular fall of France might have been unexpected, but World War I style trench warfare with millions of casualties was the primary alternative. It's not even obvious if Allies really cared about defeating Nazis - if they were serious about winning, going after them in September 1939 while they were busy in Poland seemed like the best moment, but they didn't do anything until they were attacked.

Score: 1/5

American Civil War

Here's another "glorious" war - American Civil War. It's really historically inaccurate, but let's grant the common myth that the cause of war was abolition of slavery. There were four million slaves in USA in 1860, to be freed. So a somewhat mythical point, but granted.

As for alternatives being untried, most of the world abolished already or soon after 1860, without getting into any major wars. Even countries highly dependent on slave labour, which quite smoothly transitioned into poor waged labour, with or without compensations for former slave owners. There's no reason why it wouldn't work in United States, sooner or later. No points here.

As for point three, yes, all slaves were freed, so entire point here. Sure, there was a century of discrimination ahead, but let's not get too nitpicking.

Point four utterly fails. 620 thousand soldiers were killed, plus some unspecified number of civilians. It was quite clean as far as wars go, so I wouldn't expect a huge value for civilian excess deaths. Still, that's only 6 slaves freed for one death. I'm sure most abolitionist activists had far better than this number - by collecting money, and buying off slaves' freedom, or by nagging slave-owners' conscience until they agreed to free slaves one person could surely free far more than 6 slaves with lifetime's effort.

As for point five, it was successful, and due to North massively overwhelming South in industrial capacity, military strength, and population, it could be expected to be an easy win. It wasn't all that easy, but point granted.

Score: 3/5

American War of Independence

American Revolutionary War is another big one, but I really don't see why.

First, suffering - there were some extra taxes white protestant elites weren't terribly fond of. That's zero points, or else Washington, D.C. has as much right to rebel right now.

Second, alternatives. It's automatic fail due to lack of significant suffering, but other countries like Canada successfully got as much autonomy as they wanted without armed rebellion.

Third, prevention of suffering. Also automatic fail, but notice how United States kept slavery long after the rest of British Empire abolished it. It's not at all obvious that it was more representative, or more democratic than other colonies.

Four, casualties. There were about 52 thousand soldiers killed, plus unknown number of civilians (but not too many, as it was one of the rare clean wars). Vast majority of those were from disease, most commonly scurvy, a disease that was already known to be easily preventable. Battle deaths were barely over 10 thousand. That's small by modern standards, but not trivial relative to two million people living in rebelling States back then - just about 2%. I'll grant a point, but it's just being generous.

As for point five, expecting an easy victory against British Empire at peak of its power was foolish. It was a lucky win. No points.

Score: 1/5

Cambodian–Vietnamese War

That covers the usual suspects, which I all mentioned mostly to point out how they fail at being just wars. Now let's look at some obscure wars that might actually succeed.

Cambodian–Vietnamese War is my best candidate. Khmer Rouge killed approximately 1.5 million people out of population of 7.1 million, and weren't stopping. They didn't even limit themselves to their borders, but also raided some towns in Vietnam and massacred civilians there. Overall, spectacular victory at point one.

It's hard to tell what alternatives could be tried, but none are obvious, and due to massive urgent suffering we cannot blame the Vietnamese for not looking too hard. A second point.

Pol Pot was successfully overthrown. Guerrilla lasted for another decade, but it was on much smaller scale. Full point.

Point four is not entirely successful, as China attacked Vietnam to stop their Cambodian operation, with wide variety of estimated, but something of order of 100,000 soldiers and civilians killed. Still, that's far less than number of Khmer Rouge casualties, so point granted.

War could be expected to be easy - huge, well equipped, and experienced Vietnamese army could and did easily overwhelm Khmer army, and it did so successfully. Full point.

Score: 5/5

Most other wars of independence

Cambodian-Vietnamese War proves a just war is possible, even if really rare. Let's go back to other wars for a moment. One very popular kind are all independence/liberation wars, and uprisings against dictatorships.

They usually fail on suffering front - most colonial powers don't engage in massive killing, so the suffering is mostly widespread discrimination, and lack of representative democracy.

They fail on the second front, as countless countries succeeded in getting significant autonomy, or independence, by political action. It worked even against the Soviets, so it doesn't depend on the colonial power being "nice".

As for the third point, most governments that get into power after such wars are dominated by military, or otherwise very far from democratic ideal, and hardly much better than previous colonial powers. You might be lucky sometimes, but I wouldn't count on it.

Fourth - there's usually plenty of civilian casualties. Independence wars, unless the colonial power simply decides to give up early, tend to be really dirty. No points.

Fifth - by balance of power, it's really naive to expect a victory here. Yes, sometimes it happens, not that rarely even, but lucky wins don't count. And there's even lower chance of the new government being much better than the one being replaced. No points.

Related reading: From Dictatorship To Democracy.

Score: usually 0/5, occasionally higher score.

Terrorism

If we go even further from guerrilla uprisings to terrorism, it gets even worse.

Now occasionally like in Palestine the suffering is real, so point granted. In many places like Basque Country it's fairly insignificant.

As for alternatives, political methods not involving violence have been quite successful in countless cases. So zero points.

One interesting thing about terrorism, is that is pretty much always fails. Of very long list of organizations employing terrorism, the only totally successful case is Hezbollah, which kicked Israelis out of Lebanon. Tamil Tigers - who had semi-independent state in North Sri Lanka for quite a long time before being defeated recently, and Hamas and other Palestinian groups, which fought out some very limited autonomy for Palestinians, can be counted as partially successful. Everything else was a spectacular failure. Notice how the successful cases are really guerrillas employing some limited terrorism against mostly military targets as a small side activity, so they shouldn't even count as primarily terrorist organizations. Zero points here.

As for random unrelated casualties, they're present by design. Usually they're not terribly numerous compared to a proper war, so actually we could grant half a point here, but that would be stretching it. And that would only count if it worked, what not a single case of terrorism so far did.

As for chances of success - well, so far they're 0 successes in far too many tries, so no points.

Score: 0/5

Bombing of Bosnia and Herzegovina

Let's look at operations like bombing of Bosnia and Herzegovina.

First, suffering. Over 100,000 people died in the war, about half of them civilians, so it was a pretty good cause. Plus millions displaced, usual looting, rape, and what you usually see in dirty wars. Full point.

Second, alternatives. Some effort was taken, and out of three fighting parties, UN managed to get two to agree to stop. There were also attempts to limit civilian casualties by creating safe zones (openly abused for military advantage, but that's a completely different issues). You could argue about lack of fairness by UN, and if really everything that was possible was tried, but I'll grant the full point. Also military action was quite limited, aimed at getting quick political solution, not full annihilation of Serbian military capacity.

Third, prevention of further suffering - that's an easy point, operation was fully successful.

Fourth, no extra casualties - bombing is estimated to kill over 100 Serbian soldiers, a trivial number, and many more of just Serbian soldiers would have died if the fighting continued. Full point.

As for estimating chances of victory... well, wars are rarely won by just dropping a few bombs, but NATO was overwhelmingly stronger than any of the fighting parties in Bosnia, and it was successful, so full point.

Score: 5/5

Kosovo and other operations

While bombing of Bosnia was really successful, other peace enforcement campaigns like one in Kosovo much less so.

There were around 2000 casualties of the war between Serbian military and police and KLA, but no massive genocide ongoing. The best example of war crimes found was Račak massacre of about 40-45 people, and even that isn't really terribly solid. There was killing due to fighting, and large scale displacement, but that's rather underwhelming, so just half point this time.

Alternatives were under-explored. There was some attempt at negotiating, but it's not really obvious if NATO seriously wanted a solution that would be acceptable to both parties, as opposed to just getting some cover. No pressure was put on KLA, even though they would be fairly easy to press. And during the negotiations all sides issued a joint statement agreeing to "substantial autonomy for Kosovo", and "invited international civilian and military presence in Kosovo", so it's not really obvious what the subsequent fighting was about. No points.

Prevention of suffering was also quite unsuccessful, the war erupted even more strongly after bombing started, with some proper massacres this time. 80% of people killed, and 90% of people displaced happened after the bombing started. 250 thousand Serbian refugees never returned to Kosovo. No points.

Bombing as such didn't kill terribly many people, around 500 civilians, but due to massive increase in fighting on the ground huge number of people died. There was also a later spill-over of fighting into Macedonia, but it wasn't that severe. No points again.

I'm undecided if the last point should be granted or not - previous successes like Bosnia suggested that it would be an easy victory, but it worked only so-so, and large scale ground fighting caused a lot more suffering that was prevented. Half point.

Score: 1/5

Summary

According to criteria of successfully and reliably preventing significant suffering, the only good wars I could find where overwhelming force is used to stop already happening war or other large scale atrocities, and they limit themselves to minimalist goals instead of trying major political restructuring.

All wars where one side is not overwhelming, like WW2, and American Civil War, tend to cause massive suffering, comparable or worse than whatever can be prevented, even if one of the sides is clearly doing a lot of evil.

"Wars to End All Wars" are as old as known history, and include not only WW1, and WW2, but events as ancient as civil wars of late Roman Empire, and probably many before it. Success rate - 0%. Using this argument to support any war is just ridiculous.

All fighting from position of weakness - anywhere from terrorism to full scare insurgency - tend to cause massive suffering, and have even few successes at preventing major suffering.

If you know of any other wars that score very high according to my criteria, or have problems with my rating system (ratings are quite subjective, but it's hard to come up with anything much better), post it in comments.

Chomsky is a total idiot

Which all means that Chomsky is a total idiot, as he is a big fan of the kind of unsuccessful wars from position of weakness like Tamils in Sri Lanka (80 thousand or more people killed with nothing to show for it), and a big critic of successful wars like Bosnia (apparently Srebrenica was "not really a massacre" with only 8,000 or so people killed).

Yes, American government engages in many stupid wars, and causes a lot of suffering. But that doesn't mean that everyone who happens to criticize American government is automatically right and entitled to the high moral ground. Reversed Stupidity Is Not Intelligence, as my favourite philosopher Eliezer Yudkowsky says.

Lasik retrospective

taw — Thu, 27 Aug 2009 12:14:00 -0700

It's been over six month since I had my eye operation, so it's time for a quick retrospective.

I was a fairly atypical patient. Before operation I had mild short-sightedness, about -1.5 in each eye. It wasn't annoying enough for me to ever bother with glasses or contact lenses. I might had some use for them if I ever bothered driving or something like that, but driving in London is a ridiculous idea. And perhaps not even that, I cycled without problems for years.

If you think about it, there are surprisingly few common activities that require highly precise long distance vision. Most of our lives are spent in small rooms, interacting with people and objects no further than a few meters away from us, and the only benefit from long distance vision is knowing that the bus is coming towards you and you should stay off road, not its registration number. Long-sightedness seems like a much worse problem in modern civilization - what use is precise distant vision if you cannot even read what's on the screen, or see faces of people standing next to you clearly?

Anyway, perhaps more due to my transhumanist drive towards self-improvement than an actual pressing need, I decided to go for surgery. To get laser eye correction you need your vision to be stable - unlike with glasses and lenses you cannot change the lasering every couple of years. And you need money, as unfortunately NHS doesn't yet fund corrective eye surgery. What good is socialized medicine, if the only time I needed it I had to pay with my own money? Oh well, that's a rant for another day. Here's the clinic I used.

What operation looks like

Before the operation I underwent a series of eye tests with a lot of fancy machinery. There was obviously the "focus on the picture" machine, looking at letters through a series of lenses, and also machines I haven't seen before - one that puffed into my eyes and observed effects of puffing (I was supposed not to blink), and another that projected rotating light line onto my eye, also no blinking allowed. And there were a few more tests I don't remember too well.

On the day of operation, it went really quickly. First I got anesthetizing eye drops and had to wait there for half an hour or so. Then I was laid on a table, and the operation begun. To prevent blinking some plastic contraption was put around my eye, keeping the eyelid open. Water was poured onto it to clean it up and lubricate, and then some big machine pressed it so it stayed in place. Normally eyes wander around all the time, it feels really weird to have an eye in place and pressed upon. Really weird. It was time for the first laser - it cut a flap in my eye. Then the machine was removed and the flap was moved aside by the surgeon. Of course with a flap removed from my eye, I could barely see anything, it was extremely blurry. I was told to look at a green dot in front of my, while then the second laser reshaped my eye. After a few seconds the lasering was over, and the surgeon put the flap back on its place. I was given a minute of rest, and the same procedure was applied to the second eye. The whole thing lasted perhaps 10 minutes.

After that, I was left in a darkened room to rest for 20 minutes. A good thing I remembered to take my MP3 player with me, otherwise I would be bored to death (of course no reading allowed). Then a quick vision check, I got a bunch of eye drops and was sent home.

First weeks after operation

There was no waiting for the results - I instantly had superclear vision. But obviously that's not the end of story. Eyes after operation are extremely sensitive and fragile - it takes some time for them to heal. I was issued with a long list of activities that I wasn't allowed to do - no Tube on the first day (due to some air pressure issues). Not letting water or sweat into the eyes for the first week. Going outside only with sunglasses. No exercise, no alcohol, no cycling, no make up, no flying, no swimming, no gym, I think the list pretty much implied no sex too, even though it wasn't spelled explicitly. And a complicated schedule of eye drop - lubricating ones to take every 1-2 hours for a week, anesthetic every two hours on treatment day, antibiotic four times a day for 6 days, steroids every 2 hours for 3 days, then every 4 hours for another three days. And sleeping at night only with very uncomfortable eye shields, to make sure they don't touch the pillow.

Fortunately most of the limitations were lifted within a week - the ones that were left for longer were swimming, long list of sports I don't care about like football and skiing, and definitely under no circumstances rubbing, touching, or putting pressure on my eyes. Then a few appointments to check progress (1 day, 1 week, 2 months, 6 months) and that's pretty much it.

There was no pain involved. My eyes were extremely ridiculously oversensitive to sunlight at first - hence the sunglasses - but that passed quickly. They were also very dry, and I had to use lubricating eye drops for a few months. At first it was every 1-2 hours, just as the leaflet said, then it was maybe twice a day, and now I perhaps use them once a week if my eyes feel particularly dry, but usually I don't bother.

After six months

Apparently my vision is "better than 20/20", which is better than human average, but not by a huge margin. No problems left, other than occasional dryness. And I cannot cry, but that's more likely a personality issue than anything related to my eyes.

So it's top human values, not superhuman values yet. But we might get there. Right now, if your vision is bad and unchanging, you should get lasered. But I can imagine that with time we might get to the point when eye lasering with simply be a smart thing to do for pretty much everyone. We are superhumans already. We are resistant to diseases that could kill wild people with vaccination, we have access to ridiculously good memory with Google search, now we can even get superior orientation in unknown territory with Google Maps and a phone. We can communicate with people who are on the other side of the world. We got so far away from our wild state, it would just be another step on the way to get better vision than they had.

What makes a good sniper game

taw — Wed, 26 Aug 2009 19:54:00 -0700

As a big fan of FPS, especially stealthy/sniperish kind, I'm highly opinionated about what makes a good sniper game. Actually, ignore that part. I'm highly opinionated about pretty much everything. Yes, that's more like it. So here are my thoughts on what constitutes a good sniper game.

Realism

First - the game needs to completely and utterly ignore realism.

You know what real war is like? It's basically walking around in large groups, and not killing anyone for years and years. Most deaths are civilians - by way of intentional killing, not caring much if they live or not, and hunger and diseases that tend to accompany war. Just look at one of the cleanest wars ever - American Civil War. There were 3.1M soldiers, and 625k deaths - that's a fairly respectable 20% death rate (lets skip civilians among them), but only 213k combat deaths - barely 6.7% death rate. Yes - even in a clean war like syphilis is far more dangerous than enemy bullets. In other world, you can expect to kill 0.067 enemies in all four years of that war. Wouldn't that make an amazing game? Play 60 years 24/7 to get one kill. How interesting would that be?

This number is of course nonsense - even most combat deaths are artillery, air bombardment, tanks, machine guns, and other big military hardware. An infantryman's chance of killing anyone is just ridiculously low. It's basically years and years of walking around.

Do you want to know what are the chances of a bullet fired killing someone? The world produces something between 10-14 billion rounds of military small arms ammunition a year. Well, we don't have any war now, but using reasonable (within order of magnitude) assumptions that World War II era's ammo production was more or less the same as today's, and that 20% of military deaths were from small arms fire (as opposed to non-combat and big guns), that gives 1 kill per 17,500 rounds shot. And now imagine a typical game where you're given 100 rounds and sent against a hundred or so of enemies. Ridiculous much?

[Probably the best sniper in history]
OK, you might say, but we're talking elite snipers here, not masses of lowly grunts. Then yes, it gets much more interesting indeed. Sniper kill rates are one kill per five round or better. The best sniper in history of the world - Simo Häyhä killed between 500-800 Russians with his sniper rifle, plus over 200 with submachine gun. And he used iron sights, not any fancy hardware.

Still this completely ridiculous number of 700-1000 kills took place over 105 days - that's less that one kill per hour, even assuming he didn't overwork himself and took plenty of rest. And that's so far beyond what a typical sniper can except to get, unless of course it's a video game sniper - those get 5 kills a minute, if it goes well.

[I'm not making up rocket pistols]
Does it mean realism is completely unnecessary, and the games should be using plasma rifles, and rocket pistols instead? Well, that might be cool as well, but reasonably accurate reproduction of battlefield mechanics as an emergent property can also result in reasonably accurate balance of different kinds of weapons, and different strategies, and reproduce richness that would be difficult to program manually.

Hollywood realism

The second thing we should abandon is Hollywood realism. There are so many ridiculous tropes in movies and video games that are repeated over and over again - people shooting without aiming, or shooting machine guns while holding them in their hands, grenades having ridiculously small casualty range (in most games you're safe if 2-3 meters away, and you need to be pretty much hit in your head to die from a grenade). Fragments from real granades can hit you even more than 200 meters away. And of course in movies wounded people either die instantly, or keep fighting. Nobody is ever wounded in need of hospitalization.

I'm not a fanatic of realism. Real war sucks. If you want some, volunteer to Afghanistan. You don't even need to hurry, that war isn't going anywhere. But often just going the easy way and making it realistic does improve gameplay, so why not do it?

Elements of good sniper games

Pretty much the best sniper game ever is Crysis. Now I'll wait for half of the readers to close this tab. Done? All right. So what makes Crysis such an amazing sniper game?

First, it's a PC game. Sniper games pretty much need mouse and keyboard, no controller gets even close to that. Unfortunately more and more crappy FPS games move to consoles-only, or are console games with crappy PC ports like FarCry 2, what's not much better.

Next, you do have peripheral vision. Humans have plenty of senses they can use to know what's happening in their vicinity. In addition to narrow field of view in front of them, our eyes can see large space around us, though quite poorly. The most practical way to represent peripheral vision in game is with some sort of radar view, just like Crysis does it.

And you can stay camouflaged, while moving at reasonable pace - in so many other games you can spend five minutes quietly crawling through grass, and then being instantly killed by an enemy sniper. In Crysis as long as you're in invisibility mode (which lasts only short time between rechargings), behind cover, hidden in vegetation, or sufficiently far away, you'll be safe.

Enemies behave reasonably sanely, as far as games of this kind go, even though I'd hope for a lot more in the future. They try to get cover, and shoot where they think you might be, and if you're not careful you will die. Crysis did not have to resort to giving them ridiculous aiming powers, or other dirty tricks like that as many other games do.

What's wrong with Crysis

Hey, I like a computer game? That doesn't sound much like me, does it? So let's start talking about what's wrong with Crysis.

First, weapons balance. You're most likely to use the assault rifle for everything. It beats everything else at every distance. It has much higher chance of killing a nearby enemy without aiming than either shotgun or SMG; with single fire mode, decent scope, and Maximum Strength mode it's extremely competitive with sniper rifles on long distances; and it's far better for short to medium distance kills from concealment (where silencer is a must) than anything else. For medium distance burst fire shootouts there's no alternative. And it has far more ammo than sniper rifle and the briefly featured gauss rifle. As a bonus you get a good grenade launcher to instantly get rid of enemy vehicles. There's simply no situation where anything except assault rifle is significantly better than it.

I have some more complaints about weapons. First, ammo is ridiculously sparse. You have to salvage ammo from your kills, and each Korean has just one magazine on him, and no grenades. What of course doesn't stop him from firing infinitely many rounds, and throwing as many grenades as he feels like throwing. It's especially bad with sniper rifles - forcing you to use assault rifles on long range - or move to midrange and use assault rifle from cover, what's actually pretty cool.

Another problem I have is that cover is too strong. Yes, you can blow most kinds of cover like metal sheets, but as long as they stand they protect whoever is behind them. Games like Call of Duty 4 get it right, thin cover offers protection, but not terribly much. I blame Hollywood realism. There's also the usual problem with grenades and RPGs being ridiculously underpowered, even if not as much as in many other games.

There are also some Crysis-specific issues. Invisibility mode, while really cool, can be abused - all energy is lost if you shoot while invisible, forcing you to become visible just like with D&D Invisibility spell. But - if you turn it off, headshot, and turn it on back, you can kill someone in a crowded room without losing any energy! It's an useful combo to learn.

Also rather annoyingly many seemingly cool features are pretty much useless. Tranquilizer darts would be useful for quietly taking down enemies, except they get up in 5 minutes, and set up the alarm - making them pretty much useless. Strength mode for melee attack is quite pointless, as you can as well be invisible, and shoot your target. Speed mode can only be used while high on energy - but if you're high on energy you can get invisible for safety, and if you're low on energy and have to escape, well too bad, speed mode won't work either. Vehicles are pretty much suicidal to use - their weapons have ridiculously low accuracy compared to your small arms, low firepower, and while inside you're a sitting duck waiting to be killed.

That's all still pretty cool for a sniper game.

Games that get it wrong

First, FarCry, no matter how much I liked it when it came out. You have sniper rifle which is good for what it says on the tin, except for its severely limited range. So you had a choice of either assault rifle - which instantly killed the enemy on headshot, but you needed to be close (for headshot), and it was loud enough to alert everyone. And MP5 SMG, which is silenced, but it doesn't even kill on headshot. So in either way, you get detected straight away. But the binoculars/radar mechanism is brilliant (and reused in Crysis).

Call of Duty 4 - there are sniper rifles, but because enemies respawn almost instantly, sniping is completely and utterly useless. You cannot clear the building and then move, or even clear the building, and order your team to move. Just forget about sniping and get the assault rifle back.

Then there are sniper games like Sniper: Art of Victory, which gets everything wrong.
Concealment being hit and miss for no reason, retarded AI, no peripheral vision, amazingly crappy grenades (less than 1 meter casualty range), no penetration of even glass windows, missing point blank shots with SMG, ridiculously low ammo, especially SMG ammo (like 10 bullets), and a very long list of bugs. And these are just some of the gameplay mechanics issues. Boring missions and crappy graphics can be ignored if sniping in fun, but the game is atrocious in every possible way imaginable.

And of course every single console FPS gets sniping wrong by using a controller.

That's it. Now you can go and design a good sniping game. Good luck!

Get better at go with gnugo

taw — Sat, 22 Aug 2009 20:10:00 -0700

Usually I write about things that work quite well - who wants to learn about stuff that doesn't? Or about spectacular failures. Near misses - things that almost work but not really - are a rare subject. This post is about one of such near misses.

I'm sometimes casually playing go at KGS. My account is at 10k now, not too impressive, but then I don't play that much. That's just what my personality is like - I don't do that many things with any regularity, I start something, get really into it, and then a few weeks later my enthusiasm wanes. Not always, but it's quite typical.

So of course I would like to get better at go, without too much effort if possible. I know what's your first thought, but it's not about not losing. You know what's the biggest problem with multiplayer games? For every winner there's a loser, and 50% of people must lose, and the winners tend to be asshats without life who spend all night playing the game, making it impossible for casual players to get anywhere - sure, a casual player like me could slowly learn how to play in multiplayer mode, but losing to asshats a few times too many tends to be too discouraging.

So go is not like that. Everyone has rank assigned, and when people with different ranks are playing the weaker player gets big head start. If you win too much, your rank increases. If you lose too much, your rank decreases. So typically everyone wins half of the time, and loses half of the time - unusually convenient situation for casual players. My rate of wins and loses is pretty much independent of how good I am.

If that's the case, why am I interested in getting better? Mostly because I really hate losing due to spectacularly bad moves. Sure, there are legitimate ways of losing the game - playing too slowly and not getting enough territory, losing a big fight, the normal cases. What is not cool is losing the game by moral equivalent of putting yourself in atari, like this 9dan pro did:

How can one get better at go? First one can play a lot. That definitely helps, but there are quite a few permanoobs - players with way too many games that never progress much. And just by playing and not doing anything else it would take ages to get better.

The second way is doing go problems. They let you get better at fighting and endgame, maybe teach you some common joseki - all very useful things, but they won't teach you any big concepts, or improve your grand strategy. I've seen quite a few players who could win every fight with me, and then lose 20 points by playing some completely useless move, many times during one games.

The third thing is serious studying, like watching games of pros and good amateurs, reading Sensei's Library and books etc. I tried that, but I don't really see what's going on in pro games, and Sensei's Library is mostly targeted at dan players, so it's of limited use to me. And of course it takes a lot of time.

The fourth way, and one that I really like, is reviewing your games. You can review alone or with your fellow player - especially useful in handicap games where White can teach Black something, no matter who won. Or you can send your game to The Go Teaching Ladder, to get a review by a stronger player. Unfortunately that takes ages, and you cannot review every single game this way.

So I thought - bots aren't that horrible at go, some are near shodan level, why not use bots to do a quick review? Now certainly, it won't be as good as a human review, but it's really quick, and hopefully would be of some value.

Gnugo already has some analysis mode - it can take sgf and annotate each step with its idea of what possible moves for Black and White there are and how good they are.

Because its interface is quite inconvenient, I wrote this tiny Ruby script, so I can say annotate_sgf foo.sgf to get annotated-foo.sgf:

level = 15
ARGV.each{|sgf|
 annotated_sgf = File.dirname(sgf) + "/" + "annotated-" + File.basename(sgf)
 system 'gnugo', '--level', "#{level}", '--output-flags', 'dv',
 '--replay', 'both', '-l', sgf, '-o', annotated_sgf
}

And here the problems start. First, while top bots are around shodan level, gnugo is not. It's more like 8k, not horrible (I wouldn't mind a review by an 8k player), but not impressive either. Especially for weak double kyu players it might be pretty attractive.

What gnugo is pretty decent about is noticing groups that were unsettled but both players forgot about them.

Here both players ignore T16, which is obviously vital point for White group if you look at it. I haven't bothered thinking if S6 is another such point, but if gnugo think so, that's at least reason enough to check.

Here's another example - players didn't notice that Black had a ko to mess with White. Gnugo is pretty decent at endgames, and it's surprising how often double digit kyu, and weaker single digit kyu players make such big mistakes.

Gnugo can also point at players making moves that are too small. N1 is a valid move in endgame, where nothing else interesting is going on. At this point it's just small gote reduction. This is extremely common among weak players - territory is real, and influence is vague, so they're willing to fight for scraps of territory instead of grabbing major influence over the board.

Gnugo can also help you stop making unusual fuseki and joseki moves. Unfortunately it's mostly just applying its joseki dictionary without thinking much, so you must think some more if the move is really bad, or gnugo just doesn't like it. Unlike with endgame and semeai, it's hard for you to verify if gnugo is right.

So why don't I like gnugo in spite of all this? Well, I don't really think I'm learning much from it. It doesn't tell me why a move is good or bad, it doesn't create variations showing what are the consequences of different moves (even though it obviously tries them internally). It might tell you which move lost you the fight if you're really curious, but it's difficult to generalize this to new games.

Basically, I don't think I got much better thanks to gnugo. But try it if you want.