BrainWorks

Strange and Wonderful

2010-11-15T06:00:00.000-08:00

It's been a while since I've actively updated this blog, as I've had many other obligations and duties in life this past year. Having the opportunity to express thoughts in a public forum is, quite frankly, overrated in today's world. Between email, FaceBook, YouTube, blogging, and your cell phone, there's plentiful opportunities to talk.

No, what I really appreciate is that so many people wanted to listen. There's such an information overload these days that it's actually a challenge to decide what's worth your time. This past year I confess I've been doing a lot of listening myself and not much talking.

As with all blogs, the BrainWorks blog has undergone blog drift, the phenomenon whereby a blog gravitates away from the original subject and towards whatever is on the author's mind. So I thought it would be nice to do a retrospective of this blog, categorizing all previous posts according to topic. Not everything I write is Kant, but I hope it's not all Nietzsche either.

AI - General Discussion:

AI - Routing and Aiming:

AI - Objective selections:

Programming:

Philosophy:

Education:
Mental Filters
Learning to Learn
Follow Your Dream
Self Education
Religion:

In the Cleft of the Rock
The Man Behind the Curtain
The Emperor's New God
God of Stone
Love Thy Neighbor

Science:

Parallel Evolution
Armchair Neuroscience
Anatomy Of The Brain
Society:
Social Mentality
Boot Camp
I Like This Duke
Living In The Right
The Meaning of Life
Bring The Future To Pass
Optional Law
Being Mr. Right
A Pearl of Great Price
Badass

Games:

But if I were to summarize all of it, it is this:

Life is strange and wonderful!

Badass

2009-11-02T06:00:00.000-08:00

I love the internet! It's such an amazing storehouse of interesting information, much of which is even true. A while ago I stumbled on the website www.badassoftheweek.com. Every week the site features the linguistically embellished biography of some person who lived a legendary life. By and large they are factually correct (in the case of real badasses) or canonically correct (in the case of fictitious ones).

A smattering of the many biographies:

For some reason this site really tickles my fancy. It's the intersection of two things I both love and appreciate: learning and kicking ass. Take this excerpt from the article on Leonardo da Vinci:

I can't overemphasize how goddamned ridiculous it is that Da Vinci conceptualized the freaking helicopter at a time when most people were riding around on donkeys and using a sundial to approximate the time of day. Seriously, the freaking printing press was considered cutting-edge technology in these days, and Da Vinci was one step away from dusting Versailles in a goddamned Apache Gunship.

I suppose I could make some point about how "even you can be a badass," but lets face it. That's probably not true. The whole reason stories about these people are worth telling is that the average person simply cannot do what they did. That's what makes the story legendary, after all. You are not Johann Sebastian Bach. You aren't Prince. You aren't even Hannah Montana, and you never will be. Neither will I.

But that doesn't bother me. Humanity has done some truly impressive things, including walking on the surface of the moon and returning to talk about it. It's worth taking the time to be legitimately impressed, to really understand what makes these feats of strength so difficult and how certain humans triumphed over them anyway.

We Can Worry About That Later

2009-10-19T06:00:00.000-07:00

As I've been in the middle of purchasing a house, my life has been extremely hectic. The past several weeks have been filled with housing inspections, mortgage application, reviewing and signing legal documents. And naturally every detail has someone who wants to renegotiate it. After closing, we'll still have painting, moving, decoration, and furniture purchases to handle. My life is really stressful.

A few days ago my wife and I were reviewing the upcoming task list and for one item I mentioned, "we can worry about that later". I then realized I use that phrase an awful lot, and to me it means "we can handle that later". In other words, to me worrying is synonymous with work. Put negatively, I don't stop worrying until the work has been completed.

It was an excellent moment of self reflection. I simultaneously realized why I'm so driven, intense, productive, and stressed. This life attitude has its benefits, but it's certainly not healthy in the long term. A few weeks ago I wrote about how the secret of a successful marriage is reducing stress. Perhaps it's better to say:

The secret of happiness is reducing stress.

After all, what else is stress but discontent about the possible future? It's a tricky balance though. If you live life in the future, you'll solve all these potential problems but always be tense as a result, never enjoying the present. If you live in the present you'll enjoy it only until something happens that you should have dealt with. How do you focus on the present while building your future? Personally, I feel like I don't balance these constraints very well.

So I've been thinking about different ways to manage stress. Some common things people try include:

Eat and/or drink
Have Sex
Exercise or spend time outside
Sleep or practice deep breathing
Read a book, watch a movie, or play a game
Daydream or imagine good things
Procrastinate by doing less important work
Remove the source of stress

Personally I spend a bit too much time on the last two items. Classifying these options, they seem to fall into one of three categories:

Solve the issue
Ignore the issue
Accept the issue

Unfortunately if the only mechanism for relieving stress is to solve the problem, you're in for a rough life-- there's always something else you can worry about, and many things you can't fix Ignoring issues seems fine for small problems. And acceptance is the only option available for problems too large to be solved or ignored.

I believe the real secret to happiness is properly identifying which problems should be accepted and which should be solved. And then realizing that most problems are of the former type. It's easy to get caught up in trying to fix everything, especially as a perfectionist. But the more you genuinely accept misfortunes as Not A Big Deal, the more you can enjoy the truly good things in your life.

If that's true, then the real secret to happiness is forgiveness.

Squeezing The Margins

2009-10-05T06:00:00.000-07:00

My father has been a business consultant for decades, my mother is a certified financial planner, and my brother has run several small companies. So as you'd expect, my family talks a lot about money-- both now and when I grew up. Much of the discussion is about how to make the most money.

For example, if you have a savings account that earns 3% a year and have a credit card balance that costs you 10%, you have no reason to keep anything in savings until that credit card is paid off. Not paying off the credit card costs you 10%, bringing the net return on savings to 3% - 10% = -7%, or a 7% loss. The 3% return isn't the full story on investment. You also lose 10% in "opportunity cost", since saving the money means you forgo the opportunity of paying down debt.

On the other hand, if you have some subsidized government school loan with a 4% interest rate and you are earning 5% on average from savings, you will make more money by paying the loan off as slowly as possible, putting all additional money into in savings. When you put $100 into savings rather than paying this loan, you owe 4% more in loan interest ($4) but earn 5% more in savings interest ($5). This is a net gain of 1% ($1).

I've heard all kinds of other, more complicated ideas to leverage money for potentially greater returns. All the ideas come down to this basic concept though:

Maximize the spread between your expected investment interest rate and your debt interest rate.

If you could get a loan with a 7% rate that funds a business with an expected return of 8%, then that's 1% better than not doing anything at all. In theory.

The problem is that more complicated investment strategies have a greater chance of going wrong. There's simply more potential source of failure. In the real world there is no such thing as a guaranteed 8% return on investment. More typically what happens is that there's a pretty high chance that there will be between say, 6% and 10%. And there's a very low chance you'll get a 0% return or even less. So maybe you'll get lucky and get a free 3%. But another possibility is that the investment will give 6%. The business would consider this a success, but it cost you 7% to make this investment, so you're down 1% for your effort. And there's always the chance of a catastrophic failure, meaning you lose most or all of your investment.

Lately I've been rethinking this strategy of squeezing the margins. The increased risk is certainly part of it. Mathematically you will maximize your money if you go from a guaranteed $1,000 to anywhere between $900 and $1,400 most of the time and occasionally $0. But maybe maximizing your money isn't the end goal of life. The potential life impact of losing between $100 and $1000 is probably a whole lot worse than the best case scenario of gaining $400.

Additionally though, more complicated investments just take more time to manage and stay on top of. The purpose of money, at least in my life, is to give me more time and reduce my stress. Working hard for an extra 1% isn't worth it if I can't turn that money back into the time I spent to get it. Squeezing the margins isn't the key to financial security, though it does maximize your odds of getting lucky and striking it rich. Here's the strategy I use:

Reduce, Reuse, Recycle

This is the recycling motto. Most people don't know this, but these steps are ordered by importance. In other words, the most important thing you can do is reduce the amount of things you obtain that need to be recycled. The second most important thing is to reuse the things you have. And if can neither do without them nor reuse them, then as the last option you should recycle them. Sadly most people these days focus on "recycle" and ignore the other two.

This mindset applies just as well to finances though, and with the same priority scheme:

Reduce: Don't buy things you don't need
Reuse: Don't have money in so many places you can't keep track of all of it
Recycle: Put your money in the place it gives the highest return

That third rule is the same one as "squeezing the margins". Squeezing gets you into trouble when it violates the second rule-- once it takes a lot of effort to manage your investments, you are setting yourself up for a fall. But most importantly, the secret of wealth is not buying things you don't need.

A Pearl of Great Price

2009-09-21T06:00:00.000-07:00

My wife and I have been house shopping for most of this year. It's an interesting experience. How many things in your life do you buy that cost over $100,000 and take over a decade to pay for? The process is exactly what you'd expect of a six digit purchase-- exciting, intense, stressful, and a bit scary. The whole thing is making me rethink my basic economic mindset.

If you go to a store to buy something you think will cost $100 and it actually costs $110, more likely than not you'll buy it anyway. Presumably you went there because you needed it. Driving around hoping to save $10 probably isn't worth the time. And even if an economist's equations state the value of time is worth it, most people just won't bother.

But think in terms of a house. If you plan on spending $200,000 and the house you want actually costs $220,000, what do you do? $20,000 is a lot of money, and even if you're willing to pay it, you might not qualify for the higher mortgage.

There was an interesting study done on how price changes affects willingness to buy things. In an experiment, they presented some people with this situation:

You have decided to buy a full package vacation to Hawaii. The price is $2000. But when you sign up to pay, the price has dropped to $1600. Do you still buy it?

Obviously everyone still does. Keep in mind that $1600 is a pretty good deal for a full package vacation. Then they gave people a different situation, with only the prices changed:

You have decided to buy a full package vacation to Hawaii. The price is $900. But when you sign up to pay, the price has increased to $1200. Do you still buy it?

Overwhelmingly people in this situation will not buy the vacation, despite $1200 still being an amazingly cheap price. For some reason, we as humans get fixated on the relative differences between prices. We approach the situation as, "If I buy the $1200 vacation, I will have to give up $300 in other things I wanted to get." It doesn't matter that we can't think of what those things are, nor that $1200 is still a great deal. It's just that by comparison, the option that is no longer available is superior. People would rather choose nothing than accept the feeling of a loss.

Note that this is distinctly different from the traditional economic view of humans as rational consumers. Economics is on the assumption that humans rationally choose things that maximize utility, the benefit the consumer gets from consumption.

Suppose someone values a vacation to Hawaii as being worth up to $2000. That is, if given the choice between gaining $2000 and gaining a vacation to Hawaii, they would have no preference between them. For any value above $2000, they would prefer the money, and for values below $2000, they would prefer the vacation. According to economic theory, this person would have bought both the $1200 and the $1600 vacation in the experiments above. But the experiments prove that this is not how humans operate. At best, humans act semi-rationally. And the extent of "semi" is up for debate.

For me, this is the hardest thing about house shopping. I realize it's in my best interest to act like the rational economic consumer. I want to act that way, but my brain isn't wired like that. There is this concept of "fair market value" for a house, which essentially means, "what everyone else thinks someone else will pay". But as what prices people will pay depends on the relative context of other prices, the fair market value is at best a hazy estimate.

What I'm learning is that while fair market value might affect whether a bank will give you a mortgage, it shouldn't affect your buying decisions one bit. Yet it's human nature to think it matters. Buying a house for $200,000 with a fair market value of $220,000 doesn't make sense if the house doesn't provide $200,000 worth of value to you. If it isn't the right size, in the right neighborhood, and so on, don't buy it even if it's theoretically a good deal.

Fair market value doesn't tell you whether you'll enjoy your purchase. At best it tells you what kind of bargaining the seller might accept. When you compare prices to fair market value, you are making the same mistake that vacationers face with the $1200 Hawaii vacation. You are comparing an option available to you ($1200 or the seller's asking price) to an unavailable option ($900 or the "fair market value"). That's not a good basis for making financial decisions.

The rational choice is to compare the seller's asking price to prices from other sellers. Suppose you like a place but the seller wants $240,000 for it. Compare it to other available homes with similar features. If the next best option costs $260k, then $240k is good deal, fair market price be damned. And that's doubly true if you haven't found any suitable alternatives.

It's easy to fall into the trap of thinking of financial success means maximizing your money. Money is only as useful as the things it can buy. It's better to make decisions that maximize the number of things you want to have. It's true that saving money is the best financial decision the vast majority of the time. But when you find what you're looking for, be willing to pay whatever you can afford to get it.

Being Mr. Right

2009-09-07T06:00:00.000-07:00

I've been happily married for the past nine years, and we dated over four years before that. Like everyone, we've had ups and downs, but the experience has been overwhelming positive. We have very much grown together. We complement each other well. Not everyone has the same experience, however. I'm conscious that our happy marriage is in large part because of the effort we've put into it, not because "we're so totally in love" or "we're perfect for each other". Those things are true, and they're required for a successful marriage. But neither being in love nor being compatible are sufficient. Contrary to popular opinion, love isn't all you need. If you want your marriage to be a success, or any relationship for that matter, it helps to understand the purpose of that relationship.

When we were teenagers, my brother said something really insightful to me:

Americans don't get divorced when they fall out of love. They get divorced when it's less trouble than staying married.

His point was that many people stay in unhappy marriages because to them, it's better than being alone. But ultimately he's hinting at a rather pragmatic view of marriage. There's so much description of marriage in terms of love and everlasting commitment, but I think that glosses over this simple fact:

People get and stay married because it improves their life.

Simply stated, people think they are happier married than single. The list of common causes for divorce looks strikingly like common causes for depression:

Financial trouble
Child raising issues
Sexual incompatibilities
Infidelity
Lack of Communication
Physical or Mental Abuse
Addictions
Lack of Compatibility

Really, all these issues boil down to one root cause:

Stress

Either the couple disagrees about how to handle an issue and knows it (a disagreement) or they don't know they've disagreed (a miscommunication). Suppose one person wants to spend money on food, housing, and a fancy car. And the other person wants to spend it on food, housing, and travel. If they buy all four things and get into debt, they'll end up in a situation where they can buy neither fancy cars nor nice vacations. Financial stress will tear a marriage apart.

But so can any stress. When parents disagree about how children should be raised, they are likely to blame problems the children create on the other spouse's decisions. If one spouse wants sex once a day and the other wants it once a month, then at least one person will be unhappy, but probably both.

The secret to a successful relationship is using it to reduce life stress rather than create it.

That's why love is necessary but insufficient. Love is merely the motivation that makes you decide the other person is worth the effort. The secret ingredients are solid communication skills and a willingness to compromise.

Good communication prevents small problems from becoming large problems. For example, suppose one partner says, "Lets talk about that later" whenever she's feeling a bit overwhelmed with an issue and needs some time to process alone. But her husband interprets that as, "I don't want to talk about this at all." He might end up feeling emotionally shut out by her. And she might feel neglected because he never initiates the conversation at a later date. This problem is completely avoidable as long as both people take the time to express how they interpret what the other person says and does. The misinterpretation would be immediately clear. Without taking time for that though, these irritations build up into years of needless emotional pain.

Of course, sometimes both people understand each other perfectly well but disagree about what they want to do. In the case of a husband who wants to buy a fancy car and a wife who wants to travel abroad, trying to do both will put them in financial trouble, so that's not an option. Sometimes they just have to compromise on what they want for the sake of the other person. Maybe that means buying a Nissan instead of a BMW, and that they travel to Miami instead of Paris. Or maybe it means they travel to Paris this year, but they buy a BMW in two years. Both people need to realize that their needs can't be the first priority 100% of the time, nor can the same be true of the other person. The overall happiness of the couple needs to be more important than any one particular desire.

There are also situations where couples understand their differences of opinion and are unwilling to compromise. Abusive relationships fall under this category. When the husband believes it's okay to beat up his wife and the wife disagrees, they shouldn't compromise by saying it's only okay to beat her on certain days of the week. If you can't agree to disagree, that's a sign the relationship needs to end. No amount of love or compromise will stop an abusive spouse or convince someone to quit their drugs. They've declared what they want out of life, and it's simply not compatible with what you want.

This is really the category of "irreconcilable differences", and applies to innocuous things as well. For example, desired frequency of sex. If one person is unwilling to have sex more than once a month and the other person wants sex daily, there aren't a lot of options. One person could start having an affair, they can agree to an open relationship (essentially a sanctioned affair), they can "compromise" by only having sex monthly and building long term resentment, or they can break up. Breaking up seems like it causes the least emotional pain in the long run in most cases, which means it's often the best choice. There's no shame in breaking up when you realize things won't work out. You just weren't the right people for each other.

Self Education

2009-08-24T06:00:00.000-07:00

We can say that Muad'Dib learned rapidly because his first training was in how to learn. And the first lesson of all was the basic trust that he could learn. It is shocking to find how many people do not believe they can learn, and how many more believe learning to be difficult.

I strongly believe in the value of higher education, but it's just a guidance and support network. The real motivation and effort must come from the learner. As such, I also support autodidacticism, the practice of self-directed learning. I don't just link Wikipedia pages as an academically offensive form of citation. They contain information that's genuinely useful and accurate more often than not for people who want to learn more.

There are famous autodidacts in virtually every field of mathematics, science, music, and the arts. The list includes Albert Einstein and Frank Zappa, for example. That suggests there are no subjects a self-learner couldn't learn to master. The barriers are just natural aptitude and motivation, not field of study. Of course not everyone can understand quantum mechanics. But I suspect more often than not, people are held back by a lack of motivation, not a lack of aptitude.

That's not to suggest raw talent isn't important. Famous self-learners are famous explicitly because they happened to be so excellent in their field. Of all the people in the world, on those who both try to learn physics and have the talent will become the next Einsteins. Isaac Newton is famous for his work on Mathematics, Gravity, and Optics. But he also spent years working for the Royal Mint. We remember him most for the areas he excelled at.

I don't think it much matters if you try something new and aren't amazing at it. As long as you enjoy it, it's worthwhile. Over the past few months I've been teaching myself how to play classical piano. I'll never be Bach or even just a concern pianist. But I will be good enough that I can play some of Bach's works and that's all that matters to me. More than latent ability, I believe the most important factor is the audacious belief that you can teach yourself sometime new. BrainWorks itself is the result of that belief. I started with no relevant formal training in artificial intelligence and produced some pretty groundbreaking game AI.

You don't have to pick classical piano or AI research if you don't think you're up to the task. But surely there is something you don't know but could. Can you solve a Rubik's Cube? What meal do you enjoy the most that you can't yet cook? Do you know how a television works? What about how the Enigma code was broken in World War II? Can you sail a sailboat? Do you recognize constellations in the night sky? Have you ever made a piece of pottery? Repaired an automobile? Written a short story?

Humans are far wealthier now than at any previous time in history, and the mere fact that you can read this on a computer proves it. The latest Wall Street fallout is a drop in the bucket compared to the technological gains of the past century. More wealth converts itself to more leisure time if you want it to. That makes this coming century the setting of the greatest potential renaissance ever. Don't waste that time on television or Facebook. Find something you don't know you love and master it!

Follow Your Dream

2009-08-10T06:00:00.000-07:00

I've been watching some presentation videos on TED.com from their yearly conferences and as I scientist, I've found them very inspiring. There was a demonstration of a cheap water filter that could be deployed across the entire globe and seriously reduce infectious diseases that billions of humans are currently in danger of. Deaf percussionist Evelyn Glennie talks about music and listening. I watched a video of Elaine Morgan talking about the major differences between humans and other great apes. She makes a compelling argument for humans evolving from a species of apes that lived in water.

But most of all I appreciated this talk about living a passionate life. The speaker tells a story of a Yugoslavian Jew who escaped Nazi Germany and at every stage in his life found the resources to make a profound impact on the region he was living in. After watching these videos, I found myself very motivated to make an impact on the world at large. I believe that in several decades, I too could present something of value, and if I don't do that, then perhaps I'm not living up to my potential. But then I wondered if my motivation was misplaced. Michael Pritchard worked on a water filter because he was dismayed by the aftermath of Hurricane Katrina, not because he wanted to "make an impact on the world".

I rethought all the presentations, trying to find the common thread. What did these these people have in common that helped them accomplish their dreams? They all have some raw talent, but primarily I got the sense they were strongly motivated, and their motivation was directly for the the task they wanted to accomplish. Evelyn Glennie wanted to be a musician even though she is deaf. Through this she has become inspiring, but her motivation wasn't to be an inspiration.

Rather, I see these characteristics making up their stories:

Be motivated by the dream itself, not the result of the dream
Make plans on the scale of decades but don't follow them exactly
Don't ever give up when facing opposition

All of these people are somewhat talented and have a clear wealth of motivation. There weren't any presenters who were totally brilliant but only somewhat motivated. Success is about persistence. Each person had enough motivation to focus on something that would take years or decades accomplish. They all got discouraged and did not quit. When something stood in their way, they changed their plans to get around that obstacle. This leave us with two questions to answer:

What things do I care about?

How can I increase my motivation?

No two people have the same answers for these questions, and I believe you can only find these answers through introspection. In general though, people are motivated by positive feedback, concrete objectives, and small, discrete tasks. Things get hard when there aren't any positive results, the objective isn't clear, and the tasks can't be easily divided into portions.

This suggests that successful people are intrinsically optimists with a healthy dose of realism. They need to believe things will work out even when there's no feedback suggesting that while remaining grounded enough to change plans to get around adversity. Too much optimism and you'll stubbornly try the same thing and fail. Too little optimism and you'll just quit.

The good news is that this provides a metric for determining what you need to change to achieve your dreams. If you try things and then quit, then you need to become more optimistic. If you keep trying the same thing and it doesn't work, you need a healthy dose of reality to see why your original plan didn't work and how to revise it. If you try different things and they don't work, expand the realm of your search. But don't ever give up on the things that truly matter to you.

Only When It's Funny

2009-07-27T06:00:00.000-07:00

As I believe the purpose of life is to be enjoyed, making people laugh to be one of the most important things I do. That's ironic, given that this blog is particularly unhumorous. Don't worry thought; that's not a mistake I intend to correct today. Rather, I want to share a bit more about humor and what makes things funny.

When I was a kid, I saw the movie Who Framed Roger Rabbit. While it's full of cliche and loaded with cheap slapstick gags, it also includes a surprising amount of sophisticated humor and wit. I'll always remember one line from the movie though. Roger Rabbit is wanted for murder and trying to avoid the police. He inadvertently handcuffs himself to detective Eddie Valiant. The two finally make their way to a safe location where Eddie gets his hands on a hacksaw to remove the cuffs. If you want to watch how the scene unfolds, here it is.

Roger just slips his hand out of the cuffs and asks if that helps. Eddie is clearly pissed and asks if Roger could have done that at any time. Roger responds:

No, not at any time. Only when it was funny!

I love the implication that the toons don't have complete and total power over reality. They can only break the laws of physics when its funny to do so. For example, Wile E. Coyote is only allowed to run off a cliff and walk on air when he doesn't notice he's not standing on solid ground anymore. Once he realizes that he's supposed to fall, he falls.

Furthermore, Roger's response speaks to the value of comedic timing. Things are funny in large part because of their context. If you watch any professional comedian, you'll see a clear difference between waiting one and two seconds before delivering the next line. Some responses would simply be less funny if the response was one second sooner. John Stewart of Daily Show fame is a particularly good example of this. Comedians with perfect timing learn just how long they need to wait before their audiance comes to a certain mental conclusion. The humor derives from constrasting the comic's next statement with your current thought, and if the line is delivered too soon (or too late), it doesn't provide the right contrast.

Of course, knowing that timing matters isn't the same as knowing what to say or when to say it. No two people laugh at the same things either, so humor is a very personal thing. What makes something funny anyway?

I believe all humor reduces to cognetive dissonance between expectations and reality. When you expect life to be one way and it's different, your mind has a few possible responses. You either get offended or you laugh about it. Which response you have depends on how much you care about the topic. For example, suppose you mention that you've been feeling a bit sick for the past few days and your friend deadpans, "It's probably swine flu." Whether you chuckle depends on whether you actually think you have swine flu.

Not to be pedantic, but it's worth analysing this joke in detail. The crux is that people overestimate the probability and danger of catching swine flu. Your friend is pretending to be one of these people who overreacts, pointing out the reality of these people. Rationally we know that the odds of actually catching it are extremely low-- more people die of the regular flu than swine flu. So in an ideal world, people wouldn't be worried about it, and that's the source of cognitive dissonence. The joke boils down to, "people are worried about swine flu but shouldn't be."

If you agree with that statement, you'll laugh with your friend. But if you think people aren't overestimating the deadliness of an epidemic which has claimed far fewer lives than car accidents have in the past six months, you'll be offended. Your friend's joke became a criticism of your own perspective.

And that's the beauty of comedy. Laughter is a reflection of what we consider unimportant, and the vast majority of our lives really don't matter. A 14 year old girl might be mortified for farting in class, but she would be a lot happier if she could laugh like the rest of her classmates. After all, no one cares as much as she thinks they do. If you want to be happy in life, you have to be humble enough to accept that you just aren't that important. Only then can you laugh at just how crazy and awesome this world really is.

Imitating Life

2009-07-13T06:00:00.000-07:00

As a hobbyist game designer, I quickly learned a simple truth about game players:

Players always know whether they are having fun but rarely know why.

If you don't believe this, read the fan forums for literally any game you can think of. You'll see all kinds of suggestions, 80% of which would clearly make the game less fun. But when someone says, "I don't like it when this happens," then they're stating a personal opinion. It's not a debatable point-- they are certainly not having fun. This is a bit ironic. How can they know what they like but not why they like it?

The answer is that while humans are capable of self reflection, they aren't actually that good at it, nor do they usually enjoy it. Humans excel at pattern matching and general recognition problems. Not coincidentally, the vast majority of fun in games comes from what could be abstractly labeled "pattern matching".

Pattern matching is more than just seeing the Virgin Mary in a grilled cheese sandwich. In an abstract strategy game like Chess, it involves recognizing a board position as similar to one you've played before, so you have an idea for how to approach it. In Poker, you learn to recognize how a player's bids correspond with their potential hands. Basketball and football players need to recognize openings in the opposing team's defense. Party games like Charades and Apples to Apples are nothing but pattern matching: can you connect the dots between someone's gestures and a previously known phrase?

So I was intrigued after reading an article by Mark Rosewater on resonance in Magic: The Gathering. A game mechanic has "resonance" if it reminds the players of already known concepts. If you see that a dragon has the special power "flying" or giant has "trample", you don't even need to know how the game works to appreciate the design. We know that dragons fly, giants are large, zombies are slow, merfolk swim, and so on. It's just fun when a game does a good job of embodying its theme! And game mechanics with resonance are also easier to learn, since you can apply your standard intuition to situations they apply in. You would assume that a creature with flying is harder to attack and has an easier time attacking other people. A trampling giant probably causes more damage on average than another creature would. Zombies are probably weak overall, but easier to use as minions than something like a Dragon.

I confess this lesson took me a long time to learn as a game designer. The first games presented players with interesting decisions but totally lacked theme and resonance. This made them as entertaining as sudoku, which puts them the same realm as, "things I'm willing to do while in a train." Not exactly a rousing endorsement. But just because of how our brains are wired, humans just like seeing connections between abstract actions and known concepts. When kids play cops and robbers, "house", or pretend a doll is a real baby, they are taking enjoyment from resonance. Since even the very young participate in imaginary play, it suggests this enjoyment is somehow wired into our brains. Some people think of games as escape from real life, but it's specifically when games are connected to life that they are most enjoyed. A game doesn't need dragons, ninjas, aliens, or Roman emperors to be fun, but these things help a lot. Especially the ninjas.

Optional Law

2009-06-29T06:00:00.000-07:00

Imagine there was a law that required everyone to pay a $100 tax each year. However, there is no way to find out who paid their tax and who didn't. All the government knows is the total amount of money that comes in and therefore the percent of people paying the tax. Is avoiding this tax ethically justifiable?

I don't mean to suggest that actions are only unethical if you get caught; I don't believe that is the case. When 10 people commit similar crimes and only 9 get caught, that doesn't make it right for the guy who got away. Rather I'm asking whether it's ethical to break a completely unenforceable law. I'm not sure, and perhaps it depends on how heinous the forbidden action is. But our perceptions of right and wrong change over time.

As a concrete example, consider the anti-sodomy laws that many American states had before they were struck down as unconstitutional. These laws restricted the sexual conduct of consenting adults and were virtually impossible to enforce. In the centuries since these laws were first passed, the majority public opinion no longer regards oral and anal sex as immoral.

The general argument against unenforceable laws is that they only punish people honest enough to follow them. In the tax example, both honest and dishonest people would get the benefit of however the nation spent the tax money, but only the honest people paid for it. This is not fair, though even unfairness does not necessarily make ignoring the law ethical. The argument also gets murkier when the law is more than 0% unenforceable, and when the prohibited action has a negative impact on society as a whole. Murder is wrong even if it could never be enforced, for example.

A more recent example is the set of laws against unauthorized music downloads. Music companies don't want people using the things they produce without paying for them, but modern technology has made it very easy for people to do this anyway. The basic structure of the problem looks like this:

Party A (the music company) owns the rights to some information.
They allow party B (the purchaser) to use this information, but not to distribute it.
However, they cannot prevent party B from giving it to party C (the downloader).

At first, music companies tried to stop downloaders, but it's really too difficult to figure out who they are. So the latest tactic has been going after purchasers who distribute the music online. Even that has been very difficult, often requiring subpoenas to ISPs, and these tactics have not substantially increased music company profits or decreased illegal downloading.

The question is whether it's ethical to download music you haven't purchased. There is not much benefit to society from unauthorized downloading, but there is also not much cost. It's true that record companies lose some money from people who would otherwise have paid, but not every downloader was a potential customer. Plus the increased exposure from free downloads has in some cases turned into more paying customers, though I suspect the net effect is still a loss most of the time.

At any rate though, the real problem is that music companies are distributors who can no longer control their product distribution. No amount of legislation can make this business model profitable again; they need to find a new way to add value to customers before customers will give them money again. I'm undecided about whether illegal downloads are ethical. That said, music companies need to suck up the fact that it will happen anyway and stop trying to fix unsolvable problems.

Of course, it's easy for consumers to tell that to music companies, but harder to take that advice yourself. Here's the grand irony: The same people who support legalization of free online music are generally opposed to unauthorized government surveillance. When it's someone else taking their own words without permission, they strangely don't see freedom of information in the same light.

But the government's wiretapping program has clear parallels with music downloads:

Party A (music companies or a citizen on a phone) is the owner of information.
Party B (music purchasers or the telephone company) was given this information with the understanding it would not be given to people that A did not authorize.
Party C (music downloaders or the government) was given the information by party B anyway.

Similarly, it's not clear what the costs and benefits to society are. The government might be making the country more secure, or maybe the information doesn't help. No one wants to have their privacy invaded, so clearly there is a cost, but I'm not convinced the cost is freedom. Using the information as a way to ferret out "unpatriotic citizens" isn't that practical given the volumes of data that need to be analyzed by hand. Government agencies only have so much money to spend, and the biggest threats to national security are not unpatriotic ideas-- they are crazy people with bombs.

The biggest societal danger is of high level politicians trying to access information from specific domestic adversaries and using it for personal gain, but you don't need a massive government program to cull information on just a few thousand civilians. While this danger is real, I suspect it existed well before the Bush administration set up this widespread surveillance program.

So just as I haven't made up my mind about music downloads, I also can't decide on warrantless surveillance of citizens. They seem similar enough that they should both be moral or both be immoral. It's possible to argue for one and not the other, but that's a difficult argument to make; I'm not sure what that argument would be.

But there's one thing I am certain of:

Like the music companies, we should worry about solvable problems, not unsolvable ones.

No amount of legal pressure will have any impact on warrantless wiretapping. If you really care that the government might find out that you're meeting friends for drinks, then don't tell people that over the phone. Certainly you should stop posting it on Twitter. Or alternatively, learn to care just a little less about privacy. In the grand scheme of things, none of us are really that important, so you might as well be happy instead.

Learning to Learn

2009-06-15T06:00:00.000-07:00

As a child I didn't care for most kids my age, generally finding adults to be more interesting. As a teenager I realized that not all adults were interesting either. Some adults always had new stories and ideas to share and liked listening as well. Others adults were set in their ways, only repeating the same concepts over and over. In retrospect, I was measuring their open-mindedness. On average it seemed like older adults were more likely to be close-minded, but that wasn't always the case. I knew interesting people in their 50s and boring people in their 20s.

I eventually concluded that the capacity for learning was what made people interesting to me. Adults stopped being interesting when they stopped learning. Most high school kids hadn't learned how to learn. And many adults stopped learning after they left college. This also explained why I got along best with college students and professors, even as a 13 year old. College is the environment that most demands an open mind. As a teenager, I resolved that I would never stop learning, lest I become that same kind of close-minded adult.

Lately I've been reevaluating my for learning. I've been at my current job for close to a year, doing research into helping computers understand natural language. The median degree among my coworkers is a Masters of Science, and there are a number of PhDs working for the company as well. Having an advanced degree generally means you understand your subject matter well, but it has no correlation with your ability to teach the subject to others. Some PhDs are great teachers and some aren't.

In this work situation, there's a lot of opportunities for teaching and learning, and generally it's more challenging than learning in school. Even in college I typically understood things without much effort. I got excellent grades in hard courses while putting in half the study time or less of others. But at work, learning new concepts often takes much more effort than I'm used to. There are a few possible explanations:

It's me: My capacity for learning has decreased
It's them: Some people aren't very good instructors
It's the subject: Research is harder to understand

I spent a lot of time thinking about this and I think the issue is the subject. Learning in school is very different from a work environment. In school, the information is well understood and presented in pre-digested manner. The instructors are paid professionals. Schools are supposed to be ideal learning environments, and while I have issues with the modern schooling system, it generally meets the objective of disseminating information.

Contrast this with learning in research environment. Innovation skill matter more than communication for researchers. And because the very topic is being researched, even the person explaining can have a weaker grasp on the overall concepts being explained. So learning in this situation can involve more than just figuring out what someone is saying. You must also determine what they tried to say and sometimes even what they should have said.

I'm convinced that I need to better learn how to learn. School is good for spoon-feeding information to masses of children, but it won't teach you the art of learning. A good university gives you the opportunity but it's still no guarantee. Learning is more than just obtaining information. It requires synthesizing new concepts from data and extracting data from concepts. A good curriculum draws clear lines between concepts and data, but most people do not. And that's why learning is difficult.

Bring The Future To Pass

2009-06-01T06:00:00.000-07:00

I last wrote about the evolution of civilization and wrapped up by concluding that almost all world-changing inventions involve making it easier to move materials and information. The past century in particular has brought astounding technological marvels, far more impressive than the last several thousand years combined. It seems reasonable to conclude that this next century will put the past to shame, as will the one after that.

But what kinds of things can we expect of future civilization? Predictions of future technology are remarkably hit-and-miss. In the 1950s, people were predicting flying cars by the year 2000. But people in the early 1900s also predicted things like email and television. I think the lesson is that it much easier to move information than matter.

So at the risk of being wrong, let me make my own technology predictions for the future:

Abundant Electricity:

There isn't enough oil to provide for more than a few more decades of consumption at current technology levels. And while our coal and uranium supplies can last for a few centuries, they won't last for a few millenia. But since the use of electricity, every major invention has depended on it. The earth can support life for millions of years, but without a more efficient means of generating electricity, humanity won't be able to survive in its current state for long. Better use of nuclear and solar power will help this. But long term, hopefully humanity will develop some kind of new technology like cold fusion, if its even possible.

Artificial Intelligence:

I believe true AI, one that can pass a full turing test, is the ultimate goal of civilization. An artificial intelligence is essentially a thought generation machines. This is an incredible boon for any area of scientific research, as it enables the easy recruitment of able-minded researchers. Estimates on when this can occur range from decades to centuries, but most scientists believe it is possible. I'm in agreement that this is both technically possible and extremely difficult. There is also the distinct possibility that even after the first AI is created, it takes centuries before more than a handful can be created. The energy requirements for this kind of technology could be unimaginably large in today's terms. But if and when it happens, civilization will change forever. Hopefully for the best.

Matter Synthesis:

A large portion of new discoveries depends on very rare or hard to obtain materials. For example, most superconductive materials require extremely strange composites. Or consider the extraction of aluminum. It requires a crystal that can only be found in large quantities at a specific mine in Greenland. The mine is now empty, but a synthetic version of the crystal is an acceptable substitute. And these days, scientists can even turn lead into gold-- they just don't do it because the energy cost is higher than the value of the gold. A low energy method of creating arbitrary elements and basic crystalline structures will go a long way towards ensuring the technologies we develop can be deployed for lower production costs.

Space Elevator:

The earth can sustain life for millions of years, but eventually this planet will become inhospitable. If humans cannot learn to survive in space or on other planets, we will eventually go extinct. And while a few of the richer nations have space exploration programs, we are nowhere near having a real colony in space, much less the moon, mars, or another solar system. The issue is that the cost of even getting things off of Earth is very high. Each pound of cargo costs on the order of $10,000. There is research into cheaper means of propulsion, but the best hope seems to be a space elevator. Conceptually, a space elevator is a structure connected to earth that is high enough that the top is in geosynchronous orbit with the planet. Then goods could simply by hoisted up for a fraction of the current cost. Think of it as the world's largest dumbwaiter. There are some clear technical challenges to a construction like this, but it would mark the beginning of the industrialization of space.

Disposable Electronics:

For decades, people have talked about how extremely portable computing devices could change our lives. Many people have suggested the logical conclusion is the human/computer hybrid known as a cyborg. In other words, humans will be embedded with microprocessors. I actually think this concept is a bit farfetched. This is primarily because the mere suggestion of this technology offends the moral sensibilities of many people, so it has major political obstacles. But there's clear utility to having processing power everywhere, as shown by the popularity of Apple's iPhone. It's more likely that electronic devices will instead get so cheap and thin that they will be everywhere-- built into the side of a can of soda or on the cover of a book, for example. And all of it will have wireless network connections. This sidesteps the discussion of embedding technology in human flesh while meeting the objective of ubiqutious computer interfaces.

All that said, I'm sure there's another dozen things I haven't even considered. This is such a great time to be alive! Celebrate the extreme unlikelihood of having life at such a wonderful time in history and cherish it. When you think about it, each person's life is much better than they give it credit for, simply because it is life.

The Meaning of Life

2009-05-18T06:00:00.000-07:00

When I was a Christian, one of the biggest comforts was having a well defined purpose to life. The greatest thing was to glorify God. That's easy to do on Sunday, but I figured that if God was as incredible as Christianity claimed, it was worth honoring God every day of my life. After thinking about this for a while, I decided God must care an awful lot about humanity, and seeking the advancement and peace of human civilization was the primary method of bringing God glory on a daily basis. This kind of thinking is supported by Bible texts such as "If you love me, feed my lambs", the parable of the talents, and the parable of the sheep and goats.

Psychologically, losing meaning in life was one of the three hardest parts about converting from Christianity to Atheism. However, I soon realized that removing God from the equation didn't have to change anything. I could still find meaning through the advancement of human civilization.

As a result, I've become interested in the origin and destiny of civilization. I consider a group of people to be a "civilization" when there exists a caste of the population that spends a large portion of time in activities other than the acquisition of food and reproduction. So a group of hunters living in caves wouldn't be a civilization until there is some kind of a chieftain or shaman who is supported through food from the other villagers. Civilization can't advance until people have leisure time to spend on art and science, so the beginning of civilization is when some people first have this free time.

Humans have made three major discoveries that were necessary for civilization to begin. They are language, fire, and agriculture. Language is required to share the complex, learned concepts that are the basis of civilization. Fire is required for cooking, which increases the nutritional value of food, thereby increasing the lifespan and health of the population. And farming is the most crucial of all. Through farming, a group of people can produce more food than they can consume. This both provides a buffer against years of famine and allows a segment of the population to spend time on things other than food production.

Once those three things were in place, the stage was set for the rapid advancement of technology, resulting in a far higher standard of living and population than humans had ever seen. The past 10,000 years of life on Earth have put the previous millions to shame. Of the many things humans have invented and discovered, however, I'd like to list what I consider the most important discoveries. All of these you use on a daily basis, probably without even thinking about it. They are:

Writing:

Creating a writing system is not as easy as most people would think. There's an excellent discussion of why this is hard in the book Guns, Gems and Steel. The basic issue is that pictorial systems (like the Chinese alphabet) require memorizing a different symbol for each word, so learning the system difficult. And phonetic systems require a stronger understanding of phonetic units than most people have, to the extent that most people learn English spelling without even understanding the phonetic backing.

The Wheel and Roads:

These two inventions together are necessary to move large quantities of goods over long distances, and is a crucial part of trade with other cultures. Trade is the mechanism by which different cultures share their discoveries.

Currency:

In trade and barter systems, it's easy for a baker who wants meat and a butcher who wants bread to work out a deal. But what happens when the butcher instead wants a knife, the blacksmith wants a table, and its the carpenter who wants bread? Now you have to orchestrate a four way trade. When you expand this problem even to a town of just 1000 people, its complexity as an NP complete problem becomes apparent. Money provides a linear time solution. Every trades what they have for money, then trades the money for what they want.

Mathematics:

Most of the time, having directions or instructions is meaningless unless they are precise. Understanding the relationships between numbers is required for measurements, trades, and general mechanics.

Plumbing:

Once you have even 10,000 people living in the same general area, getting fresh water and removing sewage becomes a genuine problem. Without a good system of moving water, the population is much more likely to contract and spread diseases. Sewers make cities less susceptible to plagues.

The Printing Press:

Even if you have a written language, the production of written documents is time consuming without a printing press. The press allows for mass production of books, which in turn puts written materials in the hands of the common person. That makes it the precursor of widespread education. Historically, the printing press was also one predecessor of the Protestant Reformation, as it enabled the Bible to be translated out of Latin and into a language the average person understood (at least if it were read to them).

Electricity:

Fire is a powerful source of energy, but it's difficult to use. Contrast this with electricity, a power source that can be transmitted over large distances and even stored for later use. Among its many uses, electricity provided more efficient means of lighting buildings. This in turn meant humans could be productive for more hours in the day. It also created more efficient climate control systems for buildings, which allowed humans to live in harsher climates. And it enabled the creation of more efficient factories using the assembly line for production.

Radio Communication:

Three thousand years ago, kings were happy that their exact instructions could be written down and sent to another place in the kingdom within a month or less. But due to the discovery of radio waves, encoded messages can be broadcast across thousands of miles in mere seconds. With radio communication, decisions require crucial information can be made in minutes rather than weeks or even months.

The Internet:

I am convinced that the internet is by far the greatest invention of our generation. While radio and television allowed rich corporations and governments to communicate quickly and easily, the internet has put that power in the hands of the everyone. Massive amounts of data can be send anywhere on the entire planet in a matter of seconds due to the power of the internet. It has brought about an era in which anyone with access to a computer can express their thoughts. And if Twitter is any indication, almost none of it is worth reading. But I hope that this blog can be one corner of the internet where the information is worthwhile.

When I see how far humans have come, I'm excited for the future. Some other week I'll write about some options for what The Next Big Thing(tm) might be. But given the increasingly rapid pace of technology development, it could be something we see in our lifetime. You might notice that everything on this list has to do with more efficient movement of ideas, matter, or energy, so that's a clue for what kinds of technology this next century could bring.

Living In The Right

2009-05-04T06:00:00.000-07:00

I recently had the chance to watch Alexandra Pelosi's documentary Right America: Feeling Wronged. (I'm assuming the video is legally viewable on the internet; if it's not, this review contains a reasonable synopsis.) The film follows the McCain/Palin 2008 presidential campaign and interviews the supporters on their beliefs. It's roughly 45 minutes long, but I found it well worth my time.

First, a few disclaimers. Yes, director Alexandra Pelosi is the daughter of current House Speaker Nancy Pelosi. And like every other documentary, I'm sure there's an agenda behind it. But this isn't a Michael Moore documentary, where the whole film is a setup to make a targeted person or group feel uncomfortable and mocked for their beliefs. That's not what this film is about. Alexandra seems to understand that her political connections automatically make her motives suspicious, and she generally works hard to ask unbiased questions, without making the subjects feel like the purpose is to laugh at them. For example, after seeing an ornament at a trailer park labeled "Redneck Wind Chimes", she asked the residents this series of questions:

Are you rednecks?
What is a redneck?
Do you use redneck as a derogatory term?

Whether or not you actually laugh at their responses is your choice. I found most of the people surprisingly articulate, and more than anything, I felt pity for them. At any rate, the movie gave me a lot to think about, but I took two very important things out of it. Both of them changed my perspective on the conservative / liberal divide in America right now.

A) Most conservatives viewed the election as a choice between right and wrong.

Overwhelmingly the people interviewed has the viewpoint that McCain was clearly the right choice for president, and Obama was the wrong choice. In contrast, I feel like most liberal voters considered Obama the better choice and McCain the worse one. In other words, conservatives tended to evaluate candidates from the standpoint of black-and-white morality. For example, if you believe that abortion is a sin, then voting for a president who supports abortion must be a sin as well. That makes Obama an immoral choice.

I believe most liberals evaluated the candidates pragmatically. A typical liberal voter's top issues aren't morally charged in nature, so their voting selection process is based on which candidate will better address their issues. Voting for "the other guy" is merely a bad decision, not something that moves you one step closer to Hell.

B) Most conservatives have logically consistent opinions derived from a different set of premises.

These people gave Obama a good serving of all kinds of random accusations. Different people accused him of being a Muslim, a terrorist, the next Hitler, and even the Anti-Christ. These viewpoints are totally preposterous, but I want to stress that for the most part, the conservatives are acting rationally given their assumptions. If Obama really is a terrorist, then of course you shouldn't vote for him!

Most of the conservatives interviewed didn't even seem to be aware that other people might not share their premises. I got the distinct impression that many of these people couldn't figure out why liberals would vote for a known Muslim, and the best conclusion they can come to is that "those liberals must hate God".

I really want to stress this point, because it's very important. America's far right wing conservatives are not crazy. They are acting rationally from the basis of their premises, and they are not aware their premises might be incorrect. And for the record, liberals are the same. They are also not aware that their premises might be incorrect.

So if someone claims that Obama is the next Hitler because both were charismatic leaders who came to power at times of national distress, you won't make much headway if you flat out disagree with them. Saying, "Don't be silly; of course Obama isn't going to invade foreign nations, killing enormous numbers of innocent civilians," might be true, but it won't change their mind. It is better to discuss their flawed premise rather than disagree with their conclusion. For example, "Lincoln, Roosevelt, Washington, and Churchill were also charismatic leaders who came to power during national distress, and they were some of the best leaders of all time. Being charismatic doesn't mean he's evil."

This doesn't just apply to political groups either. It's easy to assume that any time someone disagrees that they are simply stupid or crazy. But more often than not, they are simply mistaken. That's why I have pity for both rednecks and Christians rather than disdain. And as a Christian, I pitied athiests. The vast majority of people are simply misinformed, and if you take the time to have an honest conversation with them, both of you will be better off for it.

Mental Filters

2009-04-20T06:00:00.000-07:00

I write about a pretty wide variety of topics. Some common ones are:

Algorithm Design
Ethics and Morality
Philosophy
Politics
Learning

While these topics seem somewhat distinct, they all intersect on one question:

What things are good and how can I best do them?

The site's tag line is "Genuine and Artificial Intelligence", but the purpose of intelligence is to do good things, rather than bad things, inefficient things, or evil things. I think of each post as an opportunity to learn how to learn.

So it shouldn't surprise you that I love well reasoned and articulated explanations, especially things explaining how we think. For example, this essay on Lies We Tell Children was worth a blog post. I found one last week on the subject of open-mindedness. It's well worth the 10 minutes, so I highly recommend watching it:

There's a lot of good stuff in there, but I particularly liked the point about mental filters, starting at roughly 7:23. A lot of people think that being open-minded means you are willing to consider any idea, which essentially means your brain lacks a mental filter. They define being open-minded as accepting any idea you are told, no how far-fetched the circumstantial evidence is. What this really means is, "you should agree with me no matter what." And ironically, that's the exact definition of being close-minded, as it means they aren't open to other ideas.

The main point is that being open-minded means considering the evidence and logic for different explanations, then believing the explanations that best fit the evidence and throwing out the ones that don't. In other words, being open-minded is predicated on having an evidence filter.

Everyone is confronted with both true and false concepts all the time, and it's impossible to have a world view that only believes the "correct" things. Statistically speaking, you are virtually guaranteed to believe some ideas that are flat out wrong. But the objective is to minimize the percent of incorrect ideas you believe. That means the key word is "filter". If you have a brick wall for your mind, you are unable to accept any new ideas, so the incorrect premises you have will only turn into more incorrect conclusions. And if you have no mental protection at all, you'll believe all kinds of crazy things.

I believe many people who are truly close minded mistake the mental filters other people have for brick walls. They falsely conclude that because someone rejected their idea, that person must not be open to any opposing ideas, rather than simply having a good reason for rejecting theirs.

All of this is well and good for understanding "the other guy". But how do you make sure that you aren't "the other guy"? Remember, the most close-minded people think they are open-minded, in the same way that arrogant people think they are humble. There's no firm method to conclude whether you are actually open-minded, but keep this in mind: Truly open-minded people occasionally change their beliefs. Ask yourself this question:

When was the last time I philosophically disagreed with someone, and after we expressed our beliefs, they changed my mind?

If you think you are open-minded but can't remember the last time someone else changed your opinion on something substantive, you are probably more close-minded than you think. Recognizing when you are wrong and having the humility to admit it is the true mark of an adult mind.

Ballistic Programming

2009-04-06T06:00:00.000-07:00

Last Wednesday I attended a talk at MIT from Larry Wall, the designer of the Perl programming language. The talk was entitled, "Ballistic Programming", but really it was excuse to talk about Life, the Universe, and Everything. And by everything I mean programming languages, their design, shortcomings and benefits, natural languages, grammar, and how his perspectives on all of these things impact the design of Perl.

Perhaps it's better to say that the talk was about "everything".

At any rate, the study of ballistics is literally about the art of projectiles. It's the source of the the term ballista, and it's based on the Greek verb for "to throw". Pretty much every sport involving a ball is a ballistic sport, as well as a few others, like the javelin toss and gun target practice. Ballistics is the art of throwing an object from one point to another, and the process works like this:

Select intended destination
Estimate force required to move object
Make attempt
Evaluate effectiveness
Correct next attempt using these results

Of course, this procedure applies just as well to programming as it does golf or basketball. Larry had an excellent set of diagrams explaining the two major theories of software project design. First, he showed Waterfall, a complete failure of a process in which each stage seamlessly and perfectly flows from one step the next with no backtracking. It's been known for decades that it doesn't work and causes projects to be over budget, but here's are the stages:

Specification (select destination)
Design (estimate force)
Implement (make attempt)
Test (evaluate)
Maintenance (correct)

The problem with waterfall is that it has no flexibility anywhere. Sometimes something will come up in implementation that demonstrates an issue with specifications or design, and those portions need to be corrected. But waterfall states that all the specifications have to be done first, then all the design, and so on. Either the whole project gets held up as flaws in the process are found, or the project is completed with flaws, producing something that no one needs and is impossible to maintain.

The other common methodology is known by the laden buzzphrase "Extreme Programming". The two main tenants of extreme programming are that you want lots of feedback loops so problems can be corrected as soon as possible, and that people should program in pairs. Since pair programming isn't relevant to the discussion at hand, I'll just give my two cents and let the issue rest. It's very useful for educating half of the pair and otherwise it's a waste of time, in that it takes more than twice as many worker-hours to do.

Larry Wall joked that in Extreme Programming, you start with the test which shows you how to implement the program. After you've implemented it, you can write your design document, and finally at the end of the project you can write the specifications. In other words, it's the reverse of Waterfall:

Test
Implement
Design
Specs

Naturally it's easy to run into problems early if you don't know what your design and specs will actually end up as. And just because you are constantly refactoring when you encounter a problem doesn't mean you refactored in the right way. Knowing something needs fixing is different from knowing how to fix it. Sometimes your next attempt isn't even an improvement on the last issue.

That said, it's is a clear improvement over Waterfall. It's easy for Extreme Programming to beat Waterfall in the same way that it's easy to beat a blind man in boxing. Just because you won doesn't mean you're any good.

Waterfall is always bad, but there's "good" Extreme Programming and "bad" Extreme Programming. Whether or not Extreme Programming works depends heavily on how you skip between the different ballistic steps. Sometimes you start in specs and move directly to implementation, then before you even get to test, you detect a problem in design. You fix the design problem, quickly make an implementation fix to reflect it, and then test it. In you can skip from any step to any other step if you are truly following the fast turn-around tenet to its optimum. In other words, Extreme Programming is just another name for:

Cowboy Coding

Losely put, Cowboy Coding is a system in which "every man does as he sees fit." No system can ever tell you the optimal way to construct the program. There will be speccing, design, implementation, and test, but who knows in what quantities and chronology. Good programmers just know from experience and intuition when to move to the next step. Now cowboy coding is used pejoratively by some programming methodology evangelists, especially those in the Agile community (a subset of extreme programming). When bad programmers try to use Cowboy Coding, the result is always a train wreck. But ironically it's the only good part of Extreme Programming, even if its proponents don't realize it. Think about it like this:

Software design metholodigies instruct programmers when to switch from one ballistic step to the next. Sometimes the methodology's suggestion is optimal and sometimes it is not. Good programmers will generally outperform the methodology's suggestions and almost never do worse. Bad programmers don't know why the next step is the correct one, so even if the methodology suggests the right step, their chances of doing the next step well are low.

You can lead a man to reason, but you can't make him think.

No amount of restriction on the software design process or programming language can turn bad programmers into good ones. They tried this with languages like Java and Python, and while they are fine languages, bad programmers still write equally horrific code in them. So you are better off optimizing your design process for the good programmers and letting natural selection weed out the bad ones.

Best of Both Worlds

2009-03-23T06:00:00.000-07:00

A while ago I spoke with the head of a computer game development company. He explained to me that from an economics perspective, it's really important to keep the perspective on the player as a client of your company:

Imagine a graph of the player's enjoyment over time. The ideal arcade game starts the player with an amazing level of enjoyment, like eating chocolate cake with naked supermodels. Then after a minute or so, the enjoyment drops to zero so they put in another quarter.

Retail games are totally different. You only get their $50 once, meaning you want them playing the game as long as possible. This is so they'll recommend the game to friends. Having an absolutely amazing first minute isn't as important as long term playability and the general amount of content.

His comments stuck with me. A game designer needs to understand why players are supposed to enjoy the game, and whether it's more important to spend resources in one area or another. An arcade style game should focus on eye-candy and general visual "bling", including modern pay-as-you-go online games. The latest flavor of PC and console games only need enough eye-candy to initially attract people, and then get more mileage from more content. No one is happy with a pretty game that's finished in 3 hours.

He implicitly made one other point. Games are a tool for invoking emotional responses. He measured enjoyment, but games have the power to make us laugh and cry. They can frustrate us and make us rejoice. They can make us feel fearful and triumphant. It takes a truly brilliant game to do all of these things.

For me, that game is Thief: The Dark Project, from Looking Glass Studios. And by Thief, I also include the amazing sequel, Thief 2: The Metal Age. If you are unfamiliar with the game, I recommend watching Yahtzee's review at Zero Punctuation. It's delivered in his usual style of sarcasm and obscenities, but it's a very good review.

To summarize, Thief is a game that's the categorical opposite of a standard shooter. Rather than being armed with a dozen different firearms and enough ammunition to kill an entire brigade of bad guys, you play the role of Garrett, the master thief. Garrett is so weak that he's likely to die if he's involved in a fair fight with even one guard. Your whole job is to make sure that you never have to fight fair, ideally by avoiding fights altogether. The missions generally involve breaking into and out of heavily armed facilities, with every level providing a multitude of ways to approach it.

For example, to break into a manor, you might climb up onto the second floor and break a window to get in, sneak around to the back door and pick the lock, or knock out man guarding the front door. Each option has different challenges and benefits. Breaking a window is noisy and sure to attract guards to investigate, but being on the second floor means you're much closer to wherever the lord is keeping his valuables.

As a character, Garrett is a typical anti-hero, something I found refreshingly more believable than the standard do-gooder hero of today's games. Garrett is self-centered, distrusting, arrogant, and apathetic. He just wants to steal enough valuables to retire in style. But he's also extremely clever, and as a character, he is manipulated into both putting the world in danger and saving it through his natural reactions. Garrett only wants to save the world because it implicitly saves his own skin, and he'd rather not put himself in that much danger. But he doesn't have a choice, and the puppeteers in the game know this and take advantage of him because of it. To me, that's a much more believable character than an obscenely powerful special ops soldier who fights evil terrorists Just Because He's That Nice Of A Guy! I won't spoil the story because even after 10 years, it's still excellent.

I found the gameplay intense because it was the first game where I felt genuine fear. I'm not talking about the kind of frightening situations they'd put in a game like Resident Evil, where a zombie pops up out of nowhere and charges at you. That contains all the subtlety of a carnival funhouse. There's a difference between frightening and feeling fear. True fear comes from an impending sense of dread and worry, and that's something a zombie surprise cannot deliver. In Thief, however, you spend the entire game as a weakling. You know that if you make a single mistake, a whole slew of guards can appear and bring a world of hurt. So the entire game is spent wondering if you are hiding in a dark enough shadow, or if you can find a nearby window to jump through if things go downhill.

I'm so impressed that a game could do such a great job of bringing out a variety of emotions. In the space of one minute, you can go from suspense to fear to terror, and then feel extremely pleased with yourself by barely escaping death and finding a well hidden piece of treasure. And never have I played a game where the main character was so despicable, and yet I found myself liking him anyway.

I know that games should either be a lot of fun for a little bit of time, or a little fun for a long time. But somehow Thief manages to be the best of both worlds. The gameplay is always intense, and the story is so good that it hasn't gotten old in the half dozen times I've replayed the game. Somehow, this game embodies the best of both worlds. And while Looking Glass Studios is no more, I hope that someday someone makes another game as good as Thief.

Love Thy Neighbor

2009-03-09T06:00:00.000-07:00

Earlier this week, a reader made an interesting comment on my God of Stone post. Bryan analyzed the logic in the post and pointed out that it seemed to argue both in favor of and against revisionism in religion. My argument is actually progressive in nature. I favor revision religion when it's an improvement for humanity and I'm against revisions that are not. For example, removing the moral restrictions on eating pork is a good revision. Praying to a statue of the infant Jesus is not.

As a reminder to newer readers, I am not a Christian anymore, although I was a highly devout one for thirty years. For an explanation of why I left my religion behind, you can read the posts In The Cleft of the Rock, The Man Behind The Curtain, and The Emperor's New God. I am now a humanitarian agnostic. In other words, I'm not sure if God exists, but I believe morality and ethics have meaning even outside the context of God.

While I don't agree with everything the Bible says, I certainly understand the perspective of those that do. Having studied the Bible for decades, I'm convinced that the biblical authors had a similar perspective-- that the true measure of morality and immorality is whether an action helps or hurts humanity. Furthermore, religious authorities in the Bible had no problems revising laws when the new law provided a greater benefit to humanity.

The clearest statement of this philosophy comes from Jeremiah 7. Jeremiah has received a message from God for the people is Israel and soundly chastised them for their idolatry and other immoralities. Then in verse 19, it says:

"But am I the one they are provoking?" declares the Lord. "Are they not rather harming themselves, to their own shame?"

In other words, no amount of immorality can possibly harm God. The whole reason things like idolatry are sinful is that they harm the sinner, by pushing them away from a God that loves them and will bless them with his presence.

This is a crucial argument. Given the premises of the Christian religion, it is absolutely impossible to harm God. Because God loves humanity, he wants them to prosper. Therefore, the only things that God considers sin are those things that work against the humans he loves. So if something doesn't harm any portion of humanity, it cannot be a sin.

Things are only sinful if they harm humanity.

I encourage Christians to stop thinking about God as a random set of likes and dislikes. "God liked Jews and hated pigs. Then later he decided pigs were alright, but homosexuality was still bad." That's just irrational. If you believe in intelligent design as most Christians do, then you need to accept that your God is rational and work from there. God likes stuff that helps humanity and hates stuff that hurts humanity. He might have a better understanding of what "help" and "hurt" mean, but that's as simple as it gets.

When you view the evolution of religion as a gradual improvement on rules that benefit humanity, many stories in the Bible make more sense. For example, in Genesis 9:3-4, Noah has just survived the Ark, and God changes the law of what food humans are allowed to eat. Before the flood, humanity was supposed to be strictly vegetarian, but now God says it's okay to eat animals. Why did God revise his law? Animals were no different than they were before the flood, and now there are fewer of them. Either you think this story is factually true, in which case God must have changed his mind. Or you think this is just a story, so at least the Bible's author has revised the religious law. In either case though, the reason for the revision is clear-- the new law benefits humanity more than the old law did.

God isn't the only important religious figure who revised the Judaeo-Christian religion. Jesus did too, in his Sermon on the Mount. Starting at Matthew 5:17, Jesus says:

Do not think that I have come to abolish the Law or the Prophets; I have not come to abolish them but to fulfill them.

He then proceeds to give nearly a dozen different examples of things people should do different from what the Torah (the law from Moses) instructs or permits. Here are some of these:

Insulting people is sinful
Looking at a woman lustfully is as bad as adultery
Divorcing a woman who has not been unfaithful is sinful
You should not seek retribution on people who have offended you
Love your enemies

The entire Sermon on the Mount was a revision of the Torah, or at least a reinterpretation. If Jesus said he didn't come to abolish the law and then proceeded to contradict the Torah, then the Torah must not be the law he was talking about. The simplest explanation is that the Torah is merely one interpretation of the laws "Love your God" and "Love your neighbor as yourself". This interpretation can be improved, and that's exactly what Jesus set out to do in his Sermon on the Mount.

If God was willing to revise his own law and Jesus could revise the law God gave to Moses, then everyone who believes the stories in the Bible must logically conclude that revising religious laws can be a good thing, as long as the new law does a better job of loving your God and neighbor. Sometimes the Bible got it wrong and needs to be improved with a good dose of common sense. If Jesus did it and Christians are supposed to act like Jesus, they should not be afraid to apply common sense to their religious text either.

And while I'm not a Christian anymore, this is the reason I believe that every Christian should be pro-gay. There is nothing about homosexuality that is inherently harmful to any human. It makes the couple happy, so logically there's no reason it should offend God. If there are rules against it in the Bible, then maybe those authors just didn't hear from God correctly and people should reconsider whether a ban on homosexuality actually embodies "love your neighbor". It's not at all loving to telling two consenting adults that they cannot marry each other just because they are of the same gender.

Nowhere To Run

2009-02-23T06:00:00.000-08:00

Recently I was contacted by the project leader for Q3/PMX. As far as the AI for PMX goes, the plan is similar to BrainWorks: they want a solid AI groundwork for Quake 3 that other mods can easily build off of. As it's been nine years since the initial release of Quake 3, computer processing power and memory storage has greatly increased, so Q3/PMX has some pretty cutting edge objectives for new AI technology. In fact, computers these days are so fast that they can run Quake 3 in a browser. As I understand it, one major portion of PMX will be reworking how Quake 3 AI handles its pathing and routing, which has been a major thorn in my side. If that problem were solved, in my mind there would be one hard problem remaining for artificial intelligence in Quake 3:

Where do you go when there are no items on the level?

This might not seem like a big deal at first glance. But remember that bots use items as the primary motivating factors for deciding where to go. The pathing and routing code will tell you how to get from point A to point B, but all of that is meaningless if you don't know what point B is. The typical strategy is for the bot to pick up the items that help it the most and require the least amount of movement. If the bot can't find any enemies, it will head to the nearest generally useful item and hope a target wanders by. Item placement is the core component of goal selection in BrainWorks.

If you take away the items, everything falls apart.

The bots will simply run after enemies and hope to maintain a reasonable distance. Even a marginally competent player could exploit the bot's predictable behavior.

When two human players are in a level that contains no items, however, they don't get confused at all. Instead they strategically run and hide around the map, using cover to create good shots on the opponent while dodging their return fire. What BrainWorks is missing (and all first person shooter AI bots for that matter) is this dynamic tactical movement. You just don't notice it's missing from BrainWorks because of the item pickup code.

It's very difficult to design AI that can recognize and avoid dangerous areas of terrain while simultaneously taking advantage of opponents in those spots. But it's not impossible. And thanks to nine years of increased processing power and memory, I think it could be a reality in today's games.

Here's the basic algorithm I have in mind. The first objective is to create a "danger map" for the level. The danger map estimates how tactically risky it is to be in each area of the level. Note that "area" needs to be relatively small-- roughly one to two square meters on the ground. Too much larger and the algorithm will get muddied, mistaking good regions for bad. Too much larger and the computation will become prohibitive.

It's far too hard to analyze the geometry of a level to create a danger map, although some simple techniques could be used for a first pass analysis. Instead, the AI could analyze statistical data of players on the level, recording where players are when they got shot and where the attacker was when they made the shot. When this data is combined with statistical tracking of how often players are in a given area, you can create an estimate of how likely you are to get shot in a given region, and how likely you are to shoot someone else. The danger map would be generated using actual statistics from hundreds or thousands of hours of play time on the map.

The likeliness of getting shot is your "danger estimate", and the chance of shooting someone else is the complementary "tactical estimate". Using these values, the AI can evaluate which areas it should definitely avoid (high danger) and which areas it should head towards (high tactics). Note that some areas might be both dangerous and valuable. Standing on top of a pillar gives great vantage for shooting nearby targets, but offers very little cover and escape routes. Bots that were high on health might consider maximizing their tactical position more important than minimizing danger, while a wounded bot would do the reverse.

This all sounds well and good on paper, but if it were actually that easy, it would have been done already. You might be wondering what the catch is.

This algorithm requires an awful lot of training data. If you divide the level into one cubic meter boxes and track danger estimates for all of them, it will take a very long time before you have reasonable estimates for the entire map. There are some tricks that can improve the estimates when less data is available, however.

The basic idea is that if you don't have enough information for the current location, you average together the data for nearby locations. You might not know exactly what position (107,523) looks like, but it's probably similar to the data you have for (110,520). If you have enough other sources of nearby information, you can come up with a reasonably good estimate. The less data you have, however, the further away you have to look for estimates, and the less likely that data is relevant.

Once you have a solid danger map, you just have to teach the bots to avoid dangerous spots and seek out high-value spots. The algorithm could even be expanded to learn which tactically advantageous spots are the best at shooting at targets in a particularly dangerous location. In other words, if the bot realized its target was in a pit, it would look for high ground that's specifically good for shooting into that pit. This might sound more sophisticated than any first person shooter AI currently in existance, and it probably is. But I believe that even simple algorithms can give results far better than the current crop of AI code. This is something that could work on today's computers.

God of Stone

2009-02-09T06:00:00.000-08:00

A few weeks ago, I found a Catholic religious pamphlet in a subway car extolling The Infant Jesus of Prague. According to the pamphlet, praying devotions to the statue will cause God to intervene on your behalf and bless you with prosperity. Most non-religious people would regard such claims with mild amusement and derision, but I found myself deeply offended. My issue was not with the mystical claims, but the encouragement to pray to a statue. It goes directly against one of the Ten Commandments, something all Christian denominations (including Catholics) uphold to this day.

The second command says, in abridged form, "do not make or worship any idol". This command was ostensibly from God himself. It doesn't say, "do not worship idols unless they are of me or my messiah". The text is very clear. So when the Catholic church simultaneously says "don't worship any statues" and "give reverence to the Infant Jesus of Prague", they are being logically inconsistent. More than anything, I am offended at the irrationality of the claims. The Catholic church cannot have it both ways. I'm sure they would claim that this is not really idolatry because the statue is of Jesus, the true God. But the actual text in the ten commandments is clear that no physical object should be worshiped, even ones that represent God.

When I went to church as a child, I was taught that the Bible's warnings against idolatry were really a warning against making other things more important than God. In other words, if you value your friends or your grades higher than God, then you've made those things into an idol. While this explanation makes logical sense given the premise that nothing is more important than God, it is fundamentally reinterpretive. When the authors of the Bible warned against idolatry, they really were warning against worshiping stone statues. Now some of the post-exilic writings contain threads reminiscent of the modern reinterpretation, such as the Book of Haggai.

But I wonder how much reinterpretation is too much. Is it really fair to say that the Torah really meant to say, "idolatry is making anything more important than God?" If that statement is true, why does it matter whether the Torah said it or not? The source of a statement doesn't matter as much as the content. It is better to let the ancient writings say what they say, and be honest about our revision. "We allow people to eat pork because there's no good reason to disallow it, even though the Torah says otherwise."

Ancient religions depend on modern reinterpretation to remain relevant today. Or put another way, religions that fail to remain relevant lose followers. I believe this is why there are so many different religions and sects. The religions that have survived for millenniums are those that are most tolerant of revision. Christians and even many liberal Jews have no problems eating pork or letting women wear pants, something that would have shocked Moses. Even one hundred years ago, interracial marriage was outlawed nearly every Christian church in America. I suspect that in another hundred years, the majority of Christian churches will marry gay couples. When a religious law no longer has practical benefits, the religion always revises itself, even if it's a century too late.

This makes me wonder, however, why religions are so willing to revise their laws but so hesitant to revise their gods. The status of legendary religious figures does change slowly over time though. For example, in pre-exilic writings, Satan is portrayed as a henchman of God whose job is to test humanity. After the Babyblonian exile, Satan becomes the adversary of God whose job is to destroy humanity. The clear concept of the messiah doesn't appear in Jewish writings until this time either, although post-exilic rabbis began to interpret the Torah as if it subtly implied a messiah. And the Christian church didn't official decide Jesus was actually God (not just the son of God) until a few centuries after his crucifixion. But in the grand scheme of things, the identity of a religious figure changes much slower than the interpretation of a religious law.

Perhaps the application of ancient laws to modern life produces a stronger discorde than imagining diefic powers operating in Manhattan. The revision of religious figures only seems to occur after the religion's followers as a whole face extreme persecution-- the Babylonian captivity, or Christian oppression under Roman emperors.

From my perspective though, I don't see much of the use of these religious figures. There's a lot of good philosophy in Christianity, and that is worth keeping. But the concepts of God, Jesus, and Satan don't have any real impact on my life. I used to imagine they did in the same way early Jews worshiped idols of Baal. God was my idol of stone, and I realized the good Christian philosophy didn't have anything to do with praying to Jesus. Revising my concept of God might shock modern Christians, but to me it's no different from revising prohibitions on pork. In the end, all religious philosophies evolve to become a set of rules that benefit society as a whole. Dieties aren't required.

I Like This Duke

2009-01-26T06:00:00.000-08:00

As I've mentioned before, the book Dune shaped much of my adult thinking. In Dune, Liet Kynes meets the Duke Leto Atredies to give him a tour of Arakis, the planet the emperor has given to Duke Leto. Kynes is a servant of the emperor as well as imperial planetologist-- think head of the EPA on a galactic level. Kynes is also a member of the Arakeen native population, and Duke Leto has come to the planet to mine it for spice, an obscenely valuable commodity. So Kynes has good reason to hate Duke Leto. Leto is consuming resources from Kynes' home planet, undermining Kyne's task of protecting and understanded the planet. Leto is an external force that could disrupt the planet's entire ecosystem and destroy Liet Kynes' people.

Yet the second time Kynes meets the Duke Leto Atredies, Kynes changes his opinion on the duke. The two go out to survey a spice mining operation. Something goes wrong with the harvester which has almost a full load of spice. The load was worth rough one million times a worker's lifetime wage. Yet rather than worry about the spice, Leto goes through extraordinary means to save every last person working in the harvester. He shrugged off the value of the spice without a second thought, arguing that they could always get more later. In the words of Liet Kynes,

"This Duke is more concerned over his men than the spice! I must admit, against all better judgement, I like this Duke."

My interest in politics stems from the fascinating drama of the story. And politically, I'm very jaded. I believe that almost invariably, politicians act in their own best interests. They only helping the populous out of self-motivation. For example, senators get additional funds for their state so that they will be re-elected, and they try to get elected so they can acquire "campaign contributions" (essentially bribes) in return for passing laws that favor wealthy organizations. Politicians do both good and bad things with their power, but nothing is motivated out of selflessness. Even their good deeds have selfish motivations. So when politicians promise change and a renewed pledge to defeat corruption, I don't think anything of it.

I find myself in a similar position to Liet Kynes as I watch President Barack Obama. I'm not expecting a miracle from the man, and his promises of hope and change strike me as typical campaign messages from younger politicians. As a politician, Barack Obama is untrustworthy until proven otherwise, and even then he's still suspect. But against all my better judgment, I like this President. The feeling is extremely disturbing.

As I have very different expectations for President Barack Obama as other people have, I suspect I like him for very different reasons. I think a lot of people like Obama simply because he's not George Bush. Certainly many people like Obama because he's not white. Obama is empirical proof that minorities can earn just as much success as white people. He is also a symbol of a new generation, being the youngest president America has had in decades.

Symbols inspire, and living symbols have high expectations. But I'm unmoved by Barack Obama, the symbol. Rather, I am inspired by Barack Obama, the man. In many of the decisions he's made so far, I feel like he is paying well more than lip service to his promises. For example, his secretary of energy, Steven Chu, is a professor with a Nobel Prize in Physics. Who was the last secretary of energy that even had a PhD? I know the standard decision is to choose a board member of an oil company like Exxon and have them recommend policies that involve deregulating polution control and not investing in alternative energy. But actually chosing someone who is respected by the entire science community as a top member of his field? That's totally unheard of, and well beyond my low expectations of who would be selected for a political position.

Similarly, Obama's adamant stance on closing the Guantanamo Bay detention center is beyond the standard political posturing statements. Normally politicians will decry how human rights are being violated, but they won't actually do anything about it, or they risk losing votes from people who think that they can secure safety for the nation by torturing potential enemies. Obama seems very intent on doing something about the situation even if people disagree with him.

I believe George Bush initiated the two wars with Afghanistan-based terrorists and the country of Iraq out of vengeance. As a man, Bush views loyalty as something to be rewarded and dissent as something to be punished, and this is the extent of his motivations. Even though Bush claims he always "does what he feels is right", his definition of whether someone is right or wrong seems eerily correlated with whether or not they agree with his opinions. It's a bit of circular logic which always concludes, "I'm going to do what I'm want no matter what." The true test of whether you do what is right is how often you do things you don't want to do.

In contrast, Obama is very pragmatic man. He wants peace in Iraq because destabilizing the middle east makes life worse for America, not better. He wants to continue the fight in Afghanistan because the terrorists intend to strike America again. And he wants the US government to stop torturing prisoners for two main reasons. First, it makes many other nations hate America. And second, torture simply does not work. Studies have shown time and again that information extracted from torture is highly unreliable. Torture has no use as a tool of interrogation. It is only a tool of revenge, which is why it was used in the Bush administration and why Obama wants nothing to do with it.

I still think Obama is a politician, and his objective is to get reelected. But his apparent plan for reelection is to do as much good for America as possible, picking the most practical and pragmatic solutions rather than the solutions with the best image. If he wants to make America a better place for selfish reasons, that's fine by me. I hate to say it, but I like this president.

Ride With Hitler

2009-01-12T06:00:00.000-08:00

"Hitler ruined that mustache for the rest of us."

It's true too. Culturally, western society so strongly associates Hitler's mustache with a genocidal dictator that even seeing it conjures up an emotional connection. I find it fascinating that a man has been so thoroughly associated with his mustache that more than 60 years after his death, people would face social stigma if they wore it. This is a testament to the effectiveness of the the World War 2 propaganda. Hitler's mustache and right-to-left hair part became icons that were easy to caricature:

That meant American propaganda wasn't America versus Germany; it was America versus Hitler. Mentally it's much easier to view a person as evil than an entire nation, so Hitler became the face of evil. Here's one of my favorite propaganda posters:

Yes that's right. When you ride alone, you ride with Hitler! The poster's message was that you waste gasoline when you don't car pool, and this in turn hurts the war effort. Technically stated that wasn't true. Commodities like gasoline, bread, and meat were strictly rationed on a nationwide level, so that the war effort would always have enough of these things. From an economic perspective, if the general public consumed more gasoline, then the price of gasoline that the public had access to would increase, but it would have no effect on the quantity of gasoline the soldiers could use. The rationing merely makes publicly accessible gasoline more price sensitive. Put another way, you don't even have access to gasoline until the army says it has enough, so buying more gasoline only affects what you and your neighbors will pay.

Of course, the poster's message is still in the viewer's best interest even if the logic is incorrect. Somehow "When you ride alone, you pay more for gasoline" just isn't as compelling as "When you ride alone, you ride with Hitler." Propaganda works because most people don't think rationally. The emotional connection of "Hitler is a bad guy so I won't help him" is easy to reinforce through caricatures. But the logic of "Because gas prices are high, I should save myself money by using less of it" is too subtle to convey through a simple poster with a one line slogan.

The propaganda is a lie that encourages people to do things in their own self interests. Do the ends justify the means in this situation? I don't think so, but then again I'm a purist. I can certainly understand why people would disagree with me.

The Year In Review

2008-12-29T06:00:00.000-08:00

Today marks one year since I publicly released the source code for BrainWorks, the artificial intelligence rewrite for Quake 3. I've written slightly more than once per week, giving roughly equal time to the technology behind BrainWorks and general philosophical topics related to AI. While I have more things to write about, I feel like I've covered the basics of BrainWorks in sufficient detail. If people want to hear more about a specific algorithmic part of the code, please let me know and I'll write about it. But for this next year I'm going to focus on more abstract topics likes philosophy, ethics, religion, and the meaning of intelligence. And of course there will be computer science related topics like programming style, structure, game design and so on.

I confess I'm been a bit disdainful of blogs. This is primarily because they are so many of them, and they always focus on the same thing: whatever the author finds interesting. The problem is that so few people have interesting thoughts. And now internet fads like Twitter have made it even easier to share information no one cares about. That's the reason I rarely link to other blogs: It's only when they have something profound to say. If I can only regurgitate someone else's ideas rather than writing new thoughts myself, those ideas had better be very meaningful. The only things worth writing are those things worth reading, and 99% of blogs break this rule.

With that in mind, I plan on doing posts once every two weeks rather than once a week. Quality is far more important than quantity, and good posts can take several days (or weeks) of thinking to best articulate. Here's a rundown of some topics I'd like to cover:

Reading between the lines (extracting truth from lies)
Living as a non-Christian who still has Christian ethics
What does it mean to have feelings?
Designing games that invoke emotional responses
Intellectual consistency (why I'm not offended when people pray for me)
Learning how to learn

If you want to hear my opinions on other topics, now is your chance to make your voice heard.

The Meaning of Truth

2008-12-22T06:00:00.000-08:00

I recently had a conversation with my mother where she was bemoaning the current bias in media. She is very conservative, so hearing about the liberal media bias is something I've heard about from her for over a decade. But this time her point was different. For the first time in my memory, she called the conservative media sources untrustworthy as well. And if you can't trust Fox News, how do you find out what's really true? Everyone's message is tampered with their own personal biases.

I explained to her how facts ("truth") can and must be extracted from the biased information we receive from others. Nothing can be accepted at face value, even one's own first impressions. In other words, I believe there's no such thing as unbiased information. In contrast, my mother's contention is that people can and do recognize absolute truth, but immoral people twist it to suit their own purposes. I spent a good amount of time thinking about this dichotomy, and I believe what side you fall on depends on how you answer the following two questions:

1) Can absolute truth be perfectly understood?
2) Can absolute truth be communicated without distorting it?

My mother believes that the answer to both of these questions is yes for the vast majority of truths. My personal belief is that absolute truth cannot be perfectly understood, but you can get a really close approximation most of the time. And it's possible but extremely hard to communicate absolute truth without distortion. That's why learning to extract truth from imperfect information is such a crucial skill for a free thinker.

The problem is that statements must fit within the confines of language to be expressed to others, but not everyone has the same underlying definitions. Much of language depends on context. For example, consider this claim by Alcoholics Anonymous:

Alcoholism is a disease.

The majority of lay people believe this is true, while most medical professionals disagree. Whether you agree with this statement depends on how you define a disease. And that in turn depends on what conclusions you want to apply. The impression I get from Alcoholics Anonymous is that it's important to classify alcoholics as diseased so they can receive sympathy for their troubles in life. If alcoholism were caused by personal choices, then friends and family would be more inclined to chastise the alcoholic than rally around and support them in their addiction.

Meanwhile, medical professionals think about diseases differently. When someone has a disease, that means specific kinds of medication and treatment may be effective. From a medical perspective, diseases tend not to require psychotherapy to treat. So calling alcoholism a disease is misleading for the purposes of treatment. But it's a reasonable statement if you define a disease as, "a malady that requires outside assistance to treat."

The question is which definition of disease are people thinking of when you use the word, and I tend to think the medical professional definition is the more common usage. After all, if all ailments that required psychiatric help were considered diseases, that would include depression and schizophrenia, and no one is claiming that is the case. That's a sign the statement "alcoholism is a disease" lacks internal logical consistency. Nevertheless, that statement is still true within the context of a rather contrived set of definitions.

Perhaps its best to think about statements as belonging to one of two categories: scientific or sociological. A scientific statement is any statement that uses precise terms that are universally understood and agreed on. For example, "the sum of 0 and 1 is 1." Scientific statements, being the basis of science, must be both well defined and measurable.

A sociological statement is anything that's not scientific and therefore not well defined. For example, "Pablo Picasso was a great artist." Not all sociological statements are obviously opinions, however. The statement, "Rappers are musicians" depends on your definition of musician. I personally think of rap stars as poetry artists and not musicians, because their work lacks both melody and harmony.

Even statements that are thought of as facts can be disputed. Consider the following statement:

Senator Obama won a decisive victory over McCain in the past presidential election.

Most but not all people believe this statement is true. The disagreement centers around the definition of decisive. It's true that Obama had over double the electoral votes of McCain, but Obama only had 13% more votes than McCain had. Some people claim that a victory is not decisive when 46% of the country voted for the loser. (I disagree for a variety of reasons, but that's not really the matter at hand.)

The further statements get from scientific statements things like "0 + 1 = 1" and the closer they get to sociological statements like "Rappers are musicians", the harder they are to agree upon and therefore test. And as a result, their absolute truth becomes impossible to verify and bias works its way in. Unfortunately, it is sociological statements that interest most of humanity, not scientific ones. Almost every statement you've read in this post has been a sociological one-- its truth can only be approximated, not scientifically tested and verified. The vast majority of "facts" you interact with on a daily basis are untestable. That is why its so important to analyze whatever you hear rather than accepting it as fact. It's also why artificial intelligence is the most difficult area of computer science. AI is the area that deals with answering sociological questions like "what is best" or "what do you want".

First Causes

2008-12-15T06:00:00.001-08:00

In recent years, I've come to realize the most formative event in my life occurred when I was three years old. My parents took us kids out on a walk, and for reasons that aren't clear to me, we didn't take the dog alone. My parents put him out on the deck so he could get fresh air, but kept him on a leash tied to the deck so he wouldn't run off and get lost. When we got back an hour later, I ran ahead to the house and discovered the body of our dog hanging from the deck. He had jumped over the railing to follow us and accidentally hung himself on his leash.

This would have been a traumatic experience for any child. I'm certain that if my parents had found the body first, they would have tidied things up a bit and told me a good lie to lessen the blow of what actually happened. But I found him first, and the memory is firmly etched in my mind. As my three year old mind tried to make sense of the situation, I was confronted by an overwhelming sense of loss and waste. I wasn't just sad that my dog had died. I was particularly hurt by how needless the death had been. If my parents had taken my dog along, left him in the house, or tied him to a low post on the deck staircase, none of this would have happened. I didn't blame my parents for not thinking of this. It was just an unlucky situation that could easily have been avoided.

I didn't recognize it until decades later, but that experience framed the rest of my life. It wasn't framed with an objective or a rationale, but with an emotion:

I despise waste.

I've lived my entire life in a constant struggle against inefficiencies and minimizing potential risks. I work hard to prevent problems before they occur. For example, when I get out of any car and shut the door, I always try the handle to make sure the door is locked. I always check that I bring my wallet with me when I take my keys and vice versa. If I have to be somewhere in an hour and it will take forty minutes to get there (including the margin of error), I find something to do for exactly twenty minutes.

I think this is why I was attracted to computer programming as a profession, and why I'm so good at it. Good programming means you can teach a computer to do menial tasks that would otherwise cost a human lots of time. I'm anal about error checking and commenting in my code because I absolutely do not want things to go wrong. Carefulness has become a way of life for me, so "clean" programming is second nature. Many programmers complain that writing good code takes a lot more time than sloppy code, but I don't agree. I've been writing good code so long that it's actually faster than writing "bad code".

As the story left off last week, I had finished writing the most impressive homing missiles ever. And for game balance reasons, I couldn't use the work the way I wanted to. Obviously this pushed my buttons about wasting time and effort, so I designed a Quake 3 game modification (aka "mod") using the homing missiles. Thus Seeker Quake was born. As this was released over eight years ago, it's a bit hard to find sites where you can download it, but I believe it's still available from this site.

The PlanetQuake article does a good job of summarizing the game, but here's the basic gist. Each player starts with a rocket launcher and gets one seeker shot active at any point in time. When you shoot, it selects a random person with higher score than you and mercilessly tracks them down. They can try to outrun it, but they absolutely cannot hide from the seeker. While your seeker is active, all other rocket shots act like normal rockets. And of course, you can use any other weapon available on the level too.

It's a pretty simple twist but it has a really interesting effect on game balance. I tried games with four to eight players on it of varying skills. On a typical level with a point limit of 20, the scores would probably range anywhere from -2 to 20. But in Seeker Quake, final scores were generally in the 13 to 20 range. The best player still pretty much always won and the worst player always lost, but the range was much tighter. Seekers act as handicapping mechanic, as the best player got targeted with a lot more seekers. He has to work harder to maintain his edge. Meanwhile, the worst players on the level have almost no seekers targeting them, so they have more time to catch up.

Here's the bottom line: When a bunch of players with widely varying skill play Seeker Quake, everyone has a good time and everyone is challenged relative to skill. The game is a total blast.

My life hasn't been all cupcakes and roses. But I've worked hard to turn bad situations into good ones. I'm glad I've taken the time to do so.

Building a Better Wizard

2008-12-08T06:00:00.000-08:00

I write a lot about the artificial intelligence in BrainWorks, but that's not everything I program. Seven years ago I created a Quake 3 mod called Art of War. It's a class-based teamplay game, not so different from Team Fortress in that respect. Each class belongs to a faction with its own play style-- there was a faction with strong assault classes and weaker defenders, another with stronger defense and weaker assault classes, and so on. But even the heavily defensive faction needed one competitive assault unit. It just needed abilities that were "in theme" for the flavor of the faction. The resulting class, the Wizard, became a stealth attacker. When upgraded, it could blink through walls and turn invisible, making it ideal for hit-and-run guerilla warfare tactics but terrible for an all-out overrun of the enemy base.

Since each class had three upgrade levels, I was looking for a third ability for the Wizard that was in theme for the stealth assault play style. I decided to try homing missiles. The idea was that the wizard could shoot fireballs that would navigate through corridors and eventually home in on enemy structures. The wizard could assault the enemy base without even being inside it, forcing the defenders to track down the wizard and kill them. That's a very different gameplay situation from defending a swarm of attackers, and variety makes for good gameplay.

Given a choice between doing something the standard way and the excessive way, I usually choose excessive, and it was no different for these homing missiles. First let me explain the standard algorithm for homing projectiles.

Repeat every 50 milliseconds:
Check for possible targets in the missile's cone of view.
If there are no targets, keep heading forward.
Otherwise, turn a bit towards the target closest to the missile's forward direction.

That's about it. You can write that in 15 lines of code, and they are reasonably effective. The big problem with these missiles is their lack of memory. They can only latch onto a target that fits in their field of view. If the target can duck around a corner, the missile will forget about them and keep moving forward into a wall. There's no way you could lose track of a real opponent by stepping out of sight for an instant, but the homing missiles just don't know any better.

Not being content with the standard solution, I fortified homing missiles with some industrial strength awesome! These seeker missiles locked onto a target when it was fired and remembered that target's location (even if it went out of view). Obviously the seeker does a standard turn towards the target when it can get line of sight. When the seeker can't view the target, however, it employed the equivalent of sonar to get a navigational reading on its surroundings. The seeker would look in front of itself for walls, pillars, and other obstacles, trying to find large open hallways and rooms. It considered every direction that had sufficiently large space. From that list of possibilities, the seeker picked the direction that was most in line with the target's current position.

Now it's still possible for the seeker to get caught in a corner or alcove. Abstractly this is a local minima: something that appears to be an immediate improvement but is not useful for reaching the optimal goal. To combat this problem, the seeker kept track of how much free space it wanted. Remember, it only considers directions that have enough open space. By modifying thd definition of "enough", the seeker could control the tradeoff between getting to the target and getting to a wide-open area with lots of navigation options.

Whenever the seeker spent time turning, it increased the amount of space it wanted, making it favor open areas (and deprioritize finding the target). And whenever it went straight, it decreased how much space it needed (prioritizing finding the target). The result was that when a missile got stuck in a corner, it would start desparately searching for any hallway it could find, just to get somewhere else. When it found one, it would "relax" a bit and be a bit pickier in the next area. This is a form of simulated annealing.

This entire bit of code took about one week to write and was around 500 lines of code. I tested it for about an hour, tweaked some numbers, and it just worked. Watching it was incredible, as you could see the seekers do amazing things. They would seamlessly turn down narrow hallways with sharp turns, open doors, fly up and down ledges. Think "precise robotics control on a remote control missile" and you'll have the basic idea.

I was really excited to try this out in the next build of Art of War, because I wanted to see how this would affect the play style of the Wizard. To my dismay, it was a total disaster. The homing fireballs ended up being far too good. A wizard could shoot a fireball anywhere on the level and they would automatically find and kill enemy structures. In fact, a team of wizards could launch an assault by standing in their own home base, and there was just no way to defend against that. To balance the class, I had to remove all the cool homing missile code.

Of course, I wasn't willing to let good code go to waste...

A Change of Perspective

2008-12-01T06:00:00.000-08:00

This past Thursday was the American holiday of Thanksgiving. President Abraham Lincoln instituted the holiday as an annual occurrence, although the story hearkens to a tale in 1621 of how the native American ("Indians") welcomed the British colonists ("Pilgrims") to American, and how the Pilgrims gave thanks to God for the safety of the Atlantic voyage. Over the centuries, the religious factor was de-emphasized and the holiday was rewritten as a story of the Indians greeting the Pilgrims with a feast, and the Pilgrims thanking the Indians for their hospitality.

It's a fabricated holiday in that the actual feast probably didn't occur, but instead captures the spirit of thankfulness the Pilgrims had. These days, Thanksgiving is a holiday to get together with family and (hopefully) be thankful for the good things in your life. So for most people that means travel, a big meal, and the stress of being with people you might not get along with. But you still have the opportunity for thankfulness if you want to take it.

There's a lot of value in thinking positively. I'm not talking about pretending bad things are good, or completely ignoring things that are obviously issues. Rather I'm referring to appreciating the good things, and not letting problems get in the way of that appreciation. When nine things go well and one thing goes wrong, it's easy to focus on the negative-- it's the part that needs your attention. Everyone sees their current set of problems as the biggest mountain in the world, even if it's really just a molehill in the grand scheme of things.

I've written a lot about the importance accepting that things can't be perfect, even though it's good to strive for it. But that's different from being happy. When you work hard on something and some parts work out while others don't, you can accept this fact with a depressed attitude or with a joyful one. The attitude you pick won't change the world at all, so you might as well pick what makes you happiest. Focusing on the positive things can be hard, but it's a simple thing that makes your entire life better.

So with that in mind, I'm taking this opportunity to remember the successful portions of BrainWorks:

Highly realistic aiming
Intelligent item pickup selection
Context dependent weapon selection
Awareness and scanning that lets players try to outsmart bots
Dynamic feedback systems that let bots learn as they play

If you've read this blog for a while, you'll know there's a few things I'm not happy with. But to me at least, the project was overall a big success. I met or exceeded the objectives I set and I'm very happy with the results of the AI.

Last, working on this project provided an enormous amount of personal growth which I am also thankful for:

Increased self-confidence
More self-responsibility
Freedom from the burdens of Christianity
Less perfectionistic
More optimistic
Greater joy in life

Working on BrainWorks forced me to take an enormous amount of responsibility. I thought my God would come through for me, but things only came together when I took responsibility for myself. I'm sure the Christian readers will say that this was just God's way of building me up in strength. My perspective is that I finally realized there was no Christian God. But if what really happened was God gave me strength by teaching me not to believe or trust in him anymore, then praise God for that, and I'll continue following his instructions of relying on my own strength.

In short, I feel like I've won at life. This isn't the hardest project I'll ever work on, and not everything in my life is perfect. But working on BrainWorks gave me so much joy and freedom that I know I can handle whatever else life has in store for me. I'm still relatively young (barely into my thirtys) and I figured out the purpose of my life. I love making awesome things, and I plan on doing that as long as I'm alive.

Boot Camp

2008-11-24T06:00:00.000-08:00

An old friend of mine was a former army drill sergeant. I was surprised to learn this, as he didn't fit the stereotypical personality at all. Thoughtful and friendly are not the first words that come to mind when you hear the phrase, "Drill Instructor". This former job came up when someone mentioned they were leaving for basic training (aka. "Boot Camp") in six weeks, and my friend immediately told the guy, "Start doing push-ups now." Apparently you do so many push-ups in boot camp that it's never to early to get in shape, and the better physical shape you are in, the easier it goes. Not that boot camp is ever easy.

Intrigued by learning that my friend had been a drill sergeant, I asked him about basic training from the sergeant's perspective. According to him, the most important thing you can do to survive boot camp is to be practically invisible. You need to blend into the crowd and never even be noticed by the drill instructors. The instructor's job is to spot recruits that don't fix the mix and make them fit in.

"The instructors don't hate you," he said. "It's just that nothing you've learned in civilian life is of any use in the army, and they need to beat that crap out of you."

The typical civilian doesn't take order very well. Generally orders are considered "loose guidelines". They're likely to ignore orders outright, only do the parts they want, or do things differently based on their preferences. That kind of attitude is acceptable (although not optimal) in real world business settings. In the army though, the lives of thousands if not millions of people are at stake, and defying orders can have large consequences that the enlisted units can't possibly be aware of.

If your army company controls a bridge, but your officer tells you to march one mile down a river and swim across at midnight, then that's exactly what you do. The option of taking the bridge across and walking down the other shore shouldn't even enter your mind, even though your officer never told you why he gave you your orders. Soldiers who disobey orders either get killed or get other people killed. That's why the number one objective of a drill instructor is to teach privates to follow orders.

There's a similar issue for AI in team play video games. If your team contains some bots or possibly "side kicks", the game designers generally give you some way to give them orders. No game designer considers making your minions sometimes ignore orders, of course. Rather, the problem is this: Just as most people aren't used to following order precisely, they also aren't used to giving them precisely. There's a whole set of problems determining what the player even intends when they say something as simple as, "go back through the door". Which door? How far back? So much of language depends on context and it's just hard to determining meaning from a small snippet of words.

The second issue is that no matter what order a player gives, they almost never want the bot to perform that task indefinitely and precisely. When you say "go guard the red armor", it's just until they spot an enemy. When the bot spots an enemy, you don't want it to let them get away if that enemy leaves the red armor spot. The bot should stop guarding and start attacking in that case. Similarly, if no one goes by the red armor for a while, the bot should stop guarding and find something useful to do. When the player says, "Guard the red armor", the bot needs to understand that as "Guard the red armor until you see an enemy or until you've been there for three minutes".

This is the basic strategy BrainWorks uses for its message processing, which is largely unchanged from the original Quake 3 message code. It does the best job to determine what the player means and treats that instruction as a relatively important high level task. But it's still just a guideline. Stuff like combat takes precedence, and the bot will forget about the order after a few minutes. Ironically the bots need to function like civilians rather than soldiers, or the person playing the game wouldn't be happy with how the bots would follow orders.

Ignorance Is Bliss

2008-11-17T06:00:00.000-08:00

Having written twice about the dangers of believing everything you are told, I'd like to give some face time to the opposing argument:

Ignorance is bliss.

It's all well and good to say, "You should understand things, not just follow blindly." But to be pragmatic, there is only one time this is a serious improvement: when the blind belief is wrong. What about the times that blind belief is right? Humans survived for centuries without understanding how gravity truly worked and that didn't stop them from creating some awesome things. They were even able to use the fact that "things fall towards the ground" to great effect, such as building water clocks, without ever learning Kepler's laws of motion.

Even if someone did want to totally understand the world today, it wouldn't be possible. There is such a vast corpus of information that learning even 1% of it is literally impossible in the span of a human life. Mathematicians who are experts in their field rarely have the chance to keep up on other branches of mathematics, to say nothing of physics, chemistry, or medicine.

The fact of the matter is that most of the information we've been told is correct. If I get a bus that says "Downtown" as its destination, it really is going there. The driver could take it anywhere but I'm very certain that the destination is downtown. When I order a meal at a restaurant, I take it for granted that the cook can actually make the things on the menu and that I'll be served food that's reasonable close to the description provided. The waitress serves me food assuming that I will pay the price listed. I suspect that less than 1 in 10,000 people really understands what a computer does when you turn it on, but hundreds of millions of people use a computer every day. Understanding is a luxury, not a necessity.

Like it or not, our lives as humans are anchored in faith, not reason and understanding, and this is the cornerstone of the religion well all share: Causality. If we do something ten times in a row and got the same result, we expect that the eleventh time will produce the same result. And it does, even though we rarely know why. Understanding everything is impossible, and the whole purpose of culture is to provide structure so that everyone can use the discoveries other people made.

If that's the case, why bother thinking about anything at all? Why not let someone else do your thinking for you? The primary purpose of education isn't to give people information, but to teach them how to think. And most importantly, to teach them when to think. Thinking is really important in uncharted waters. In any situation that doesn't match your typical life experience, thinking will give you a much better idea of what to do than trying something at random.

Unsurprisingly, the same problem comes up in artificial intelligence. There's only so much processor time to go around. So if seven bots all need the same piece of information and they will all come to roughly the same conclusion, it's much faster to do a rough computation once and let them all use those results. This leads to an information dichotomy, where there's general "AI Information" and "Bot specific information". Each bot has its own specific information, but shares from the same pool of general information. In BrainWorks, all bots share things like physics predictions and the basic estimation of an item's value. If a player has jumped off a ledge, there's no reason for every bot to figure out when that player will hit the bottom. It's much faster to work it out once (the first time a bot needs to know) and if another bot needs to know, it can share the result.

These are the kinds of optimizations that can create massive speed improvements, making an otherwise difficult computation fast. If you think about it, it's not that different from one person researching the effects of some new medicine and then sharing the results with the entire world.

Social Mentality

2008-11-10T06:00:00.000-08:00

I cannot help but comment on the results of the recent American presidential election, in which Barack Obama became the first non-white to be elected president. As a jaded American, I recognize that America has its share of both wonders and problems. But I have never been more proud of America than I was on this past election night. To me, the election of Barack Obama is symbolically a major victory in America's war against racism. And were Obama the Republican candidate and McCain the Democratic one, I would be no less overjoyed. Some things in life are more important than what percent of a candidate's positions we agree with.

I want to stress that this election is a victory over racism, not slavery. Most cultures in pre-modern times practiced slavery, but it was enslavement of people from the same ethnic background. Slavery in America and the Caribbean isles differed from other forms in slavery in that the belief was also coupled with a sentiment of racial superiority. So even after the abolition of slavery in 1863, the underlying tone of racism permeated much of American life. In contrast, the Britain empire outlawed slavery in 1833, but their enslavement wasn't particularly racially biased, so their past two centuries haven't been filled with racial tension. Abraham Lincoln won the war on slavery in 1865, but completely decimating the southern American states couldn't change the racist opinions that much of the country still retained. The election of Barack Obama to the office of President is proof that a large percent of America is now racially blind-- the ethnic background of a candidate is not a reason to select against them.

Of course, not all of America feels that way. That's why this election is a sign of major progress on the issue of racial discrimination, but it doesn't represent the end of it. Looking at the final electoral vote distribution, this election was heavily slanted towards Barack Obama but unanimous. For example, Alabama and Mississippi again voted for the conservative candidate as they've done for decades. But Virginia and North Carolina both voted for Barack Obama, two states that split away from America during the American Civil War because they wanted the right to keep slavery legal (among other things).

This might seem like a trite point, but each major population center in America has its own local way of thinking. People in Los Angeles are more liberal than people in Salt Lake City, for example. The southern states tend to be more conservative than New England states. And the way they vote is a reflection of their local societal beliefs. If this were not the case, every city and state would vote exactly the same way. There's nothing magical about the geography that makes people think in certain ways. The social mentality is a purely contained in the minds of all people in that local society, and if you think about it, that is an incredible thing.

For North Carolina to vote for a black man 147 years after it tried to secede from the United States, the entire cultural mentality had to change. It does not change quickly either. It wasn't until all the adults of that generation died, along their children and grandchildren, that the majority of the state decided that maybe blacks and whites were equals. That should be a sign of how easy it is to believe the first thing you're told and how hard it is to consider outside opinions.

More often than not, people belong to the political party and religion of their parents, quite apart from the actual merits of those positions. Over 90% of America is Christian and well under 10% of India is Christian, despite extensive missionary work. Even if there were strong, logical reasons to believe in the Christian religion, it's clear that those reasons are not why America is a Christian nation. If those reasons were so logically compelling, then India would be a Christian nation too.

Like it or not, if your political and religious views closely match your family's and city's views, the odds are high that you haven't thought very hard about them. That doesn't mean your beliefs are wrong, but if they are right, you probably don't know why they are. It means you accept the basic beliefs of society's "hive mind", and you've ceded some of your thinking.

While this can be dangerous, accepting society's beliefs without too many questions can be really helpful. I'm not advocating, "question everything", but, "question everything important". You might be wondering why that is, and what this has to do with Artificial Intelligence. Next week I'll explain what I mean.

Do Not Want!

2008-11-03T06:00:00.000-08:00

I promised an in-depth explanation of how bots in BrainWorks perform dodging. But sometimes a picture provides the perfect overview:

That's the core of the algorithm. For whatever reason, this dog has decided he absolutely does not want to catch the ball someone threw at him, and the only thing his mind is thinking about is how he can safely get away.

When a bot needs to dodge a missile, the bot considers the eight possible directions it can move in (forward, backward, left, right, and each diagonal), plus the "dodge" of standing still. Any movement direction that collides with an incoming missile is considered completely off limits. Yes, it's bad to get "herded" where the enemy wants you to go, but that's a situation the bot can deal with later. There's no sense in eating a rocket now so that the bot might not take other damage later. Priority number one is always avoiding incoming damage.

So for each of the nine total dodge options, the bot computes what would happen if it moved in that direction for around a second. If a missile would hit the bot, it flat out ignores the option. It also has to check for some other problems. Obviously "dodging" into a wall isn't much of a dodge at all, so bots only dodge in directions with enough open space to move. And if the dodge is off a ledge, into a pit, then the bot also ignores that option. Those are the three reasons a bot will automatically disqualify a potential dodge direction:

Would hit a missile
Would hit a wall
Would fall down a pit

The bot might then dodge in any of the remaining dodge directions. There are two obvious and wrong ways the bot can choose which direction to move in.

Option 1: Move in the direction that gets the bot closest to its intended destination

The problem here is that the bot is very predictable. If you know where the bot wants to go and what route a missile shot cuts off, you know exactly where it will head next That makes it trivial to hit the bot in your next shot, and being predictable is not a luxury the bot has.

Option 2: Choose one of the remaining directions at random

This solves the predictability problem. But when you pick the remaining directions at random with equal probability, the bot's natural movement will be away from the missiles, without any regard for where the bot wants to end up. This random selection lets the attacker control the general direction the bot will move on, possibly keeping the bot from its destination indefinitely. In this situation, it's easy for the attacker to route the bot into a corner and finish it off.

The solution is to combine these two strategies:

Solution: Select at random, preferring directions that move towards the intended destination

Rather than assigned an equal chance of dodging in any direction, the bot analyzes all options and rates how well they help the bot reach its goal. The more helpful an option is, the higher the probability it will take it. However, any option could be taken. Over time, there is a high probability the bot will reach its goal, but its actual dodge direction is not predictable.

For example, suppose there is an incoming missile on the bot's east side, so it's unsafe to dodge east, northeast, or southeast. And suppose the bot would prefer to move northeast were the missile not in the way. The bot's dodge probability table might look like this:

North: 40%
Northeast: 0% (Ideal)
East: 0%
Southeast: 0%
South: 15%
Southwest: 5%
West: 15%
Northwest: 20%
Stationary: 5%

Once the bot knows where it wants to move, it keeps dodging in that direction for anywhere between three quarters of a second and a second and a half, at which point it chooses another direction (or possibly keeps going the same way). The result? A simple weighted probability function prevents the bot from running into missiles, being predictable, and being routed away from their goals.

Dodging The Question

2008-10-27T06:00:00.000-07:00

There's a class of questions that have no good answers to them. For example:

Have you stopped beating your wife?

The question presupposes something that is hopefully false, and giving either a yes or no answer implies the question is legitimate. The solution is to provide an answer that challenges the premise. In other words:

I never started.

First person shooter games like Quake 3 are really games of questions and answers. Every shot you take at an opponent asks the question, "How will you survive this hit?" And your opponent would prefer to respond, "You aren't going to hit me". Of course, he doesn't have much say for an instant hit bullet weapon like a machine gun or shotgun. Either you hit him or you don't. But these weapons are balanced to do less damage. For projectile weapons like a rocket launcher, it takes the missile some time to reach them, and they might move out of the way by the time it gets there. In return, they deal a lot of damage if the opponent fails at dodging. Whenever you shoot a rocket at an opponent, you are giving them the opportunity to make a mistake by not moving out of the way.

Tactically it seems like a poor choice to give your opponent the opportunity to not to take damage. All other things being equal, you'd rather they didn't have a say in the matter. Why ask a question ("Do you want to take 100 damage?") when you can make a statement ("Here, take 50 damage!")? But missile weapons provide a tactical advantage. They make large sections of the map dangerous, denying the opponent safe access to whatever items and safe cover locations are in that place. As a result, missile weapons give you control over where your opponent might go next.

I believe that no good player should take damage from the grenade launcher or plasma gun. These weapons exist solely for the purpose of controlling where your opponent can run. In particular, the plasma gun stream is so fast, constant, and deadly that it can literally herd the opponent into a position where another weapon can finish them off. If they don't want to be herded there, they are free to take 200 damage per second from the plasma shots. Missile weapons are more about controlling position than actually damaging the opponent.

Of course, none of that matters if the target can't dodge the incoming projectiles in the first place. The original Quake 3 bots were notoriously bad at this, obliviously walking directly into a path of fire. So I made sure to give BrainWorks bots a very effective missile dodging system. Just watching a BrainWorks bot play, you can shoot a straight stream of missiles at them and watch them sidestep out of the way. It's such a simple, obvious thing for a human that it might not even seem that special.

For the bot, it's not that simple. The bot needs to analyze the trajectory of all incoming missiles for possible impact and the bot needs to compute a safe direction to dodge from that. Sometimes there will be no safe areas, but some areas will be safer than others. And the final dodge direction needs to take into account the bot's intended final destination. If the bot only moves to the safest possible location, the attacker will simply herd the bot to a place that's even more dangerous. The bots selection of where to dodge also needs to eventually get them towards their goal location, albiet at a slower, safer pace.

Those are the requirements for the dodging system, and they are all challenging. Next post I'll explain in detail how BrainWorks solves this problem.

Groupthink

2008-10-20T06:00:00.000-07:00

There is a psychological concept known as groupthink, where members of a group discussing a topic gravitate towards a consensus based on a desire to minimize conflict rather than rationally discussing the subject. The play and film 12 Angry Men is a famous view of groupthink in action. A jury must decide whether a man is guilty or innocent, and the general group consensus is that he is guilty. But one juror refuses to ignore his intuition, even though it differs from the other jurors. The drama of the film is primarily based on how the other 11 jurors come to change their minds and eventually find the man innocent. Groupthink is a dangerous phenomenon because it can cause rational people to make irrational choices purely because it is the most common choice made by other group members. Just because a decision is right for everyone else in the group doesn't mean it's right for you.

Surprisingly, groupthink can be a problem in video game AI as well. This seems counterintuitive, as a bot must be programmed to follow logical rules. How can following logical rules produce illogical results? The issue is one of scope. In a teamplay game variant like capture the flag, each bot might be doing the best thing without knowing what the other players are doing, but not coordinating as a team. Suppose one bot does some logical deductions and concludes the optimal strategy is for him to guard the flag. Since all other bots have roughly the same understanding of the game state, some poorly designed AI code could cause all of them to defend the flag. It's optimal for more than zero players to defend, but having no players attack is clearly the wrong choice.

So bots that don't take into account the overall strategy of the team are likely to fall into a groupthink scenario. This results in a poor strategic choice even if each individual choice is "optimal" when consider without knowledge of the other choices. The solution is for bots to make teamplay decisions as, well, a team. Each bot's decision needs to take into account the likely decisions of other players, bots and humans alike.

The good news is that if bots are good at general combat tactics, the amount of additional logic to handle a teamplay scenario is minimal. It's not like players have better or worse aim when they're fighting in a team, or suddenly some weapons are sigificantly worse when you're playing capture the flag. None of that has to be rewritten to handle play in a group. Teamplay logic is just a question of priorities: when an enemy player has your flag, it's a lot more important to kill that player than any other opponent, and guarding your own base is a lot less important than attacking theirs. And in almost all situations, the choice is between defending and attacking. How the bot attacks and defends doesn't really change.

So BrainWorks can get away with some extremely simple logic to handle team games and still produces reasonable results. One bot becomes the defacto leader for choosing the strategy the bots will take-- exactly how offensive or defensive it is. Some number of players are slotted for defense and the rest for offense, but never 0% in either category. Then the bots are sorted based on how close they are to the two bases. If there are 5 bots on a team and the leader decides on a 2-3 split of defense to offense, then the two bots closest to the home base are assigned with "defense" and the three bots furthest from home are the "offense". Any human players in the game have no assignments, obviously.

It's worth noting that "offense" and "defense" depend on the context of the game. In capture the flag, for example, the defenders guard your team's flag when it's in the base, but they also track down the enemy flag carrier when it's been taken. Attackers try to take the opposing flag, but once someone has the other flag, the attackers need to escort their flag carrier. Once the concept of whether to defend is separated from how to defend, the actual algorithm to defeat groupthink is simple.

The Theory Of Fun

2008-10-13T06:00:00.000-07:00

Last week I ended with a simple question: Does making a game more balanced make it more fun? And like so many other things in life, the answer is both simple and complicated:

It depends.

To put it briefly, it depends on the context of the game and the reasons people are playing it. While on the surface, the original Doom appears similar to Quake 3, they are polar opposite games from a game design perspective. Doom is all about mowing downs armies of generally stupid demons with your increasingly diverse stockpile of weapons. Quake 3 is about fighting a skilled opponent with access to the same weapons and powerups you can use. Doom is a visceral game while Quake 3 is tactical. Rather than saying Doom is fun, it's more accurate to say that visceral experiences are fun, and Doom happens to fill that role. But a game like Serious Sam is just as fun, and in the same way.

If games are candy for the mind, then whether or not you like a particular flavor of game is a question of personal preference. What does your mind enjoy? Each kind of thinking corresponds to a different style of game.

Analysis
Action
Discovery
Creativity
Socialization
Identity

As best I can tell, these are the six basic thought elements that make games fun. Allow me to explain them in more detail:

Analysis

Examples: Chess, Go
Requires: Discrete set of Options

People who like analytical games approach them as a puzzle to be solved. The game needs to provide decisions for the player to make, any of which could be helpful in the right situation. The fun is in selecting your decision and finding out if it was a good choice. Winning the game usually means you were right, or at least more right than your opponents. Analytical games can also be single player games, such as a Rubix Cube. They need not include perfect information to all players like Chess and Go do, of course. A game like Poker contains a lot of analysis. But the analysis is in large part about probabilities, risks, and rewards. You can win a game of poker by making the wrong play, or lose making the right play, but people who deduce the right plays more often will win more often, and that's what makes the game analytical.

Action

Examples: Golf, Football
Requires: Physical Interaction

An action game is anything that involves a physical component. If there is ever a game situation where you know what you need to do but your muscles might not be able to do it, it's an action game. The joy in an action game is seeing how well you can do what you tried to do. It's really satisfying to shoot a three point shot in basketball, score a goal in football, or hit a home run in baseball. Every physical sport is an action game by this definition. But other games have action components as well. All first person shooter games are "action games" because they require you to aim at your target and shoot them. You know what you want to do, but your muscles might fail at moving the mouse to the right place. Even a game like Jenga is an action game, because it depends on precise muscle control.

Discovery

Examples: Final Fantasy VII, World of Warcraft
Requires: Artistry

Discovery games are fun because the player experience new things. Their joy is the joy of appreciation. A common theme in these games is a story line. As a result, discovery games generally have lower replayability. They are highly engaging the first time through, however. What the player experiences doesn't have to be the story, of course. It could be enjoying the look and feel of a new World of Warcraft zone. What all discovery games have in common is artistically designed content that the player enjoys experiencing.

Creativity

Examples: Charades, Magic: The Gathering
Requires: Very large set of Options

Some people really enjoy thinking outside the box. To become a medium for expressing creativity, the rules of the game must be extremely flexible, allowing for a wide variety of possible options. The number of options is far larger than that of an analytical game: it needs to be so large that players cannot exhaustively try all of the alternatives. This gives creative people the option to express themselves by making choices other people might not even have considered. Creative games often have a verbal component to them, but this is not necessary. For example, deck design in Magic: The Gathering is a very creative endeavor. There are more legal Magic decks than atoms in the Universe! And a number of Magic players enjoy making decks more than playing with them.

Socialization

Examples: Mafia, Shadows over Camelot
Requires: Cooperation

While it's true that games are only as fun as the people you play them with, some people explicitly enjoy games because of the social interactions. In their opinion, a game's purpose is to generate social interactions, and whether you win or lose isn't as important as how you got there. These games categorically encourage players to work as a team towards a common goal. Victories and losses are shared. The fun is about being part of a group and, for some games, deducing who isn't really a member of the team from their actions.

Identity

Examples: Counterstrike, Team Fortress 2
Requires: Winners and Losers

There are some gamers who play games to win. Obviously the typical sore loser falls into this category, but that's not the only type of person who fits this description. This type of gamer considers winning an expression of who they are. The sore loser is someone who considers that expression to be a validation of identity-- they are distressed when they lose because the loss injures their ego. By definition, every game has winners and losers. But in almost all cases, gamers who want to win prefer games against human opponents, as it makes winning more meaningful. Note that these gamers need not be good players. Often bad players who want to win will gravitate towards teamplay games like Counterstrike. No matter how bad you are, you will win roughly half of your games if the teams are large enough. When a gamer wants to express themselves through winning, all they really need is a human opponent and the plausibility to claim they won through skill. Actual skill could have been involved, but it's not required.

Less Is More

2008-10-06T06:00:00.000-07:00

Previously I wrote about a Quake 3 modification I made named Art of War, and how it was the inspiration for BrainWorks. But it wasn't the first Quake 3 mod I made. That honor belongs to an unreleased game variant simply titled Less. The concept of Less is simple: All items are less powerful, less health or ammo per pickup. It's the opposite of Excessive Quake. Excessive is fun because who wouldn't want to be uber powerful. Doesn't that mean Less is less fun to play?

I believe that when done correctly, less is more. The actual numbers on an item don't mean anything; only the relative values matter. For example, you could multiply all health, armor, and damage values in Quake by 1000 and the game play wouldn't change. Players would start with 125,000 health and each rocket would do up to 100,000 damage. Strategically nothing has changed. I call this "Pinball Inflation", as pinball games have been adding extra zeros for decades. Some tables now have scores that reach the trillions.

Online RPGs like World of Warcraft have a similar issue, where you can gain levels and deal extra damage, but you fight monsters that have more health. The difficulty of the fights don't change that much. At best you might gain a totally new kind of ability as you gain a level, which gives you another tool at your disposal, and that's the only way the difficulty increases. So for MMOs, the number inflation game is a method of unlocking more content. Players would be daunted if the first time they picked up a class of character, they had 30+ different abilities to choose from. It's better to start them with 3 to 5 and have them gradually learn more.

All that said, it's crucial to realize that gaining levels doesn't actually make your character more powerful, relative to the content you're doing. Sure you might kill goblins in 2 hits instead of 3, but you'll eventually move onto killing stronger goblins. The primary purpose of gaining levels is letting you experience more content in the game while maintaining the same difficulty.

I started the design of Less from a similar standpoint: I wanted Quake 3 to have more content. Reducing the benefit of each item might seem like a strange way to add content, but listen to the logic. A typical game of Quake 3 involves controlling the two to five best items on the map: red and yellow armor, megahealth, and powerups like quad damage and haste. But on the level there are dozens of items that rarely matter, things like boxes of ammo and weapons. You'd think weapons would matter more, but when they respawn every 5 seconds after pickup, pretty much everyone has any weapon they want. And the ammo a weapon provides makes that weapon's associated ammo box irrelevant. There may be 50 items on a level, but only 3 of them matter for a typical game.

The concept of Less was to make all items have a roughly equal play value on average. Then the best players would be those who knew which item was most important in the current situation, and what their opponent needed the most. You could end up with a game where the winner was the person who best controlled a box of rockets and a box of machinegun bullets. And that adds a deeper level of strategy and tactics, thereby adding more gameplay to the game.

You'd be suprised, but it's actually possible to tweak the numbers for each item to make this possible. The game plays like Bizzaro Quake 3, where you start timing the respawn of ammo boxes and small health balls in addition to armor and quad damage. It's definitely the same game on the surface, but the high level strategy of how you play the map is totally different.

While I don't have the source code in front of me, here are some of the changes made, to give you a sense of how dramatically the gameplay shifts.

Railgun Weapon: Provides 2 slugs (3 seconds) instead of 10 (15 seconds)
Railgun Slug Box: Provides 5 slugs (7.5 seconds) instead of 10 (15 seconds)
Lightning Weapon: Provides 24 ammo (1.2 seconds) instead of 100 (5 seconds)
Lightning Ammo: Provides 40 ammo (2 seconds) instead of 60 (3 seconds)
Quad Damage: Lasts for 8 seconds instead of 30
Red Armor: 50 armor instead of 100
Yellow Armor: 25 armor instead of 50
Orange Health: 25 health instead of 50
Yellow Health: 15 health instead of 25

When playing Less, you have this constant sense of never having enough of anything, and that in turn creates a sense of fear and tension. The big question is, though, "Do these changes make a better game?" I'm curious what other people think, and next week I'll share my own thoughts on the difference (and connection) between making a game balanced and making a game fun.

Anatomy Of The Brain

2008-09-29T06:00:00.000-07:00

Last week I wrote in brief about the differences between conscious and non-conscious thought, and how experiments show humans use both. Even the most primitive animal brain is an extremely complicated organ, and the human brain is obviously the most advanced brain we've encountered. Conceptually there are three main sections of the human brain, corresponding to the evolutionary processes that produced the brain.

Pardon the aside, but I wanted to say a word to the Christian readers who believe in Intelligent Design instead of evolution. Feel free to interpret "evolution" with "how God chose to create life". Your personal belief about how humans came to exist won't change your interpretation of what I have to say, so don't let my choice of language get in the way. Remember, I was a Christian for 30 years. I was taught creationism by my parents. In high school I believed the theory of evolution, and just said "God used evolution to create humans". Now I just think "humans were created through evolution". I still think people who believe in Intelligent Design are wrong given the data, but I see no reason to be condescending or judgmental about it.

As I was saying, there are three main sections of the brain. At the core is the so called "Reptilian Brain", or more formally the brain stem. This section handles basic reflexive responses such as "fight or flight", mating instincts, and the fear of other species. It also handles exactly one emotion: rage. Next is the Mammalian Brain, or Limbic system. This is the area of the brain that handles all other emotions, as well as concepts like family, culture, and attachments. Some aspects of conscious thought and self-identity are handled by the mammalian brain as well. Last is the Neo-cortex, which is responsible for higher level thought such as speech, reasoning, imagination, and speculation.

While there's a clear physical boundary between the reptilian and mammalian brains, the division between mammalian (limbic) and neo-cortex is not as clear, and there seems to be a stronger bleed between the functions. For example, some conscious thoughts are handled by the mammalian brain while others are handled by the neo-cortex.

Humans have all three brain sections, as do higher mammals such as other primates and larger mammals. Small primates such as rodents do not have a neo-cortex, although they do have the limbic system and reptilian brain. And as the name implies, all reptiles have the reptilian brain, but lack the mammalian brain and neo-cortex. So the brain sections correspond to different evolutionary forks. The primary feature that separates mammals from reptiles is not hair or internal gestation, but a more advanced brain.

If that's true, then reptiles don't actually have conscious thought. They simply respond to stimuli in the same fashion every time. But mammals, having a memory, can learn from past experiences and modify their behavior. Perhaps this is how mammals survived the dinosaurs after a natural disaster hit the earth. Dinosaur brains weren't programmed to handle the "meteor crashed into the earth and all your normal food dies" situation, whereas mammals could learn to find new food sources.

In computer science terms, the reptilian brain is analogous to a state machine, or a simple circuit board. It's pure hardware, and if you give it the same set of inputs, it produces the same set of outputs every time. It has no memory. The mammalian brain is more like a simple computer program, in that its responses are based both on sensory input and on past memories and experiences. Running the same computer program multiple times might produce different results. Mammals have the ability to learn throughout their life while reptiles do not. And since the neo-cortex handles imagination, reasoning, and "what if?" scenarios, it's closer in function to an operating system. An OS can run multiple programs in parallel, similar to the neo-cortex's ability to think about multiple thoughts at the same time, even conflicting thoughts.

This biological framework provides an interesting context from which to answer the questions "What are emotions?" and "Can a computer have emotions?" Some day I'll write on that, but there's a lot more to say than should be stuffed at the end of this column. In the meantime you'll just have to speculate.

Armchair Neuroscience

2008-09-22T06:00:00.000-07:00

I'm really grateful for my undergraduate education at the University of Chicago. I believe it's the best liberal arts school in the country, in large part because it lacks the name recognition of Harvard or Yale while still placing academically in the top ten undergraduate institutions in America. No one goes to the University of Chicago for the fame or connections they might get at Harvard, and as a result the school attracts only those students who really want to learn. At the University of Chicago, they teach you how to think and analyze everything.

The joke in college was that a University of Chicago student didn't need to know anything about a subject to have an opinion on it. This is pretty much true. I distinctly remember getting an A on a paper about The Brother's Karamazov when I had only read the first quarter of the book. The trick was to relate the concepts discussed in class with the material I had actually read and with other texts read in that class. While I'm sure my professor would have been mortified to find this out, it speaks a lot about learning to analyze abstract concepts and come to conclusions that happen to be right.

In other words, The University of Chicago taught me to bullshit well.

Most of the time I talk about a topic, I really do understand the subject matter. But I'm not willing to let a lack of expertise or study prevent me from sharing my opinion, so today's topic is neuroscience. As a disclaimer, I've never studied this subject in an official capacity in my life. All the information I have is from personal reading into other studies, most of which are (I think) peer reviewed.

There! That's a more accurate disclaimer than you'll hear from any journalist, and yet they are often no better qualified than I am to write about these topics. In particular, I recently read an article on msnbc.com that hypothesized that your brain could be controlled by an inner zombie.

Yes, that's really their conclusion. And by "conclusion" I mean "hypothesis supported by circumstantial evidence that's phrased as a question so no one can refute your claim because you're not officially making one". Since there's no challenge in ridiculing a claim like this, let me instead give a fair and impartial summary of the article. Then I'll rip it to shreds and propose a hypothesis that better fits Occam's Razor by several orders of magnitude.

I'll admit the term "inner zombie" is just a term chosen to generate interest in the article. And in some sense it worked-- I'm linking to it. All they really mean by inner zombie is a thought process that's not conscious and not (always) connected to the rational decision making. The article cites some studies where people are blinded by deactivating the visual processing section of their brain, and then shown a word. The people have no conscious knowledge of the word, but when they are asked to "guess" what word might have been shown, they properly guess the word with much higher than expected accuracy. There's also a study that shows people who are temporary blinded can still reflexively react to visual information they can't see. And while the article doesn't mention it, there's also a study done on people who are blind because their brain doesn't process images even though their eyes are fine. Compared to other blind people, these people do a much better job than of dodging obstacles while walking down the street, like other people or sign posts.

I'd like to note that these results are all from real, peer reviewed studies. Scientists aren't questioning the validity of these results. But they also aren't jumping to the same conclusion that MSNBC did. Here's the basic argument the article makes:

People show the ability to reflexively process and react to visual information even when they consciously report blindness.
Therefore an unconscious process has control over their conscious minds.
Perhaps all consciousness is controlled by these unconscious processes and consciousness is a myth.

It's step 3 that doesn't make any sense. Sure, humans have unconscious processes that handle the same information conscious sections of the brain handle. But why does that mean the whole brain is controlled by the unconscious part? Given our daily experience, I'd suggest that the reverse is true. Our general actions are primarily controlled by conscious processes, and only when consciousness shuts off do the unconscious sections take control. There's no reason to suggest consciousness is a myth and a "zombie" is "controlling your brain".

But the brain is a really strange organ. The reasons the brain work this way are rooted in the differences between human brains, standard mammal brains, and the reptilian brain. Next week I'll talk more about these things and explain why I think dogs have consciousness but lizards do not.

The Last Place You Look

2008-09-14T06:00:00.000-07:00

When I was growing up, I always found it ironic how lost things were always found in the last place you looked for them. For some reason, whenever I was looking for a lost toy it was never in my bedroom or the play room. I'd find it outside underneath the deck, in the basement, or inside the sofa. It wasn't until I was older that I realized this wasn't coincidence; it was a mathematical truth. Put another way, when you're looking for something and you've found it, you stop looking. Things are guaranteed to be in the last place you look.

That's not the same as saying all places are equally likely to hold the lost object, of course. If I'm looking for the TV remote, it's less probable that the dog took it as a chew toy than that the remote fell between sofa cushions. And it's far less probable that the remote is now part of an exhibit in the Smithsonian Institute.

The general purpose search algorithm that humans use for locating lost objects involves prioritizing possible locations from most to least likely and searching them roughly in that order. The actual order will get rearranged to save travel time. For example, if an object is most likely to be on top of your dresser, second most likely to be on the bathroom counter, and least likely to be under the bed, you'll still probably check your dresser first and under the bed second before heading to the bathroom. But in general the search is from most likely to least likely. That also explains why the place you find an object is always the last place you'd think to look. The optimal searching algorithm stipulates that you search in the least likely places last.

So this brings me back to last week's topic of octrees. I described how to build the structure that BrainWorks uses to determine which item a player is nearest. But I haven't actually explained how BrainWorks uses the octree to do this. Consider this two dimensional tree (a quadtree):

Each item in the tree is one of the colored dots. The player is the black dot with the circles around it. Every rectangle refers to a particular subsection of the tree. In this example, the red dot is the root node of the tree. The lower left section of the tree is divided by the brown dot into four sub-regions, and the brown dot's lower left region is further subdivided because it contains another item (the purple dot). The circles represent the boundary of the potential search space as the algorithm progresses.

When the search algorithm first starts, it checks the distance from the root node to the player. This distance defines a circle around the player. If an item is closer to the player than the root node is, it must be inside that red circle. At this point the algorithm doesn't know that the green or blue dots are inside the circle. It doesn't even know that no items in the lower left quadrant are closer than the red (root) item is. But it can mathematically guarantee that nothing in the upper right quadrant could be closer, so that quadrant is ignored.

Because the player is in the lower left quadrant, the algorithm prioritizes search in that section first. This is the quadrant that contains the greatest area of the red circle. The next most likely place to look is the lower right quadrant, as the player is closer to the lower right quadrant than the upper left. The upper left quadrant is scheduled third. So the areas to check are, in order:

Lower Left
Lower Right
Upper Left

Then the algorithm merely recurses. First it checks the brown item in the lower left. A quick distance check shows the brown item is too far away, but the search continues on the brown region's subquadrants of upper right, then upper left, then lower right. As none of these regions have any items, the search of the brown region terminates quickly. And at this point in time, the red item is still the closest object to the player.

When the algorithm starts checking the lower right quadrant of the red item, it notices the blue item is closer. So from now on, it compares all potential search areas against the blue circle instead of the red one. No further items exist in the lower right quadrant, so the blue item remains the closest item to the player.

Last, the algorithm wants to check the region to the upper left of the red (root) item. However, when compared to the new blue circle, the search algorithm realizes that the region does not intersect the circle. Since the circle is the area of potentially closer items, nothing in the upper left can be closer that the blue item, so the entire upper left region is ignored. And since all regions were prioritized by distance to the player (from closest to farthest), the search process is done. All potential items have been checked, so the blue item is the closest. The algorithm doesn't even need to check how close the green item is to the player because that entire upper left quadrant was skipped.

The octrees in BrainWorks function the same way, only in three dimensions instead of two, and using spheres instead of circles. That answers last week's question of why the algorithm might need to search up to 7 subregions to find the optimal solution. The closest item isn't necessarily in the same section of the octree as the player, but the diametrically opposed region can always be ignored. That leaves 7 of the 8 regions to check. In practice most regions are quickly pruned away when a closer item was found, in the same way the region containing the green item was ignored when the blue item was discovered. Even though the example tree contained seven items, only three of them needed to be checked (Red, Brown, then Blue) before the closest item could be found.

Dude, Where's My Rocket Launcher?

2008-09-08T06:00:00.000-07:00

Back in March I wrote about how the AI conceptualizes item placement. Rather than thinking of position in terms of raw coordinates, bots consider each item (or cluster of nearby items) to be a potential landmark. When statistical tracking is done for areas players are likely to be, for example, they are grouped by which item they are closest to. Statistical tracking is one of the most powerful tools available to AI if you have a sufficiently large and well organized data set. BrainWorks uses this data to analyze which areas of the level a player is most likely to be in, for example. Not only can this information help bots track down enemies; it also helps a hurt bot them avoid enemies. The more data it can track, the better. So by extension, the faster it can translate from raw coordinates to the nearest item, the better. However, this is not a simple request.

If a level has N items in it and a player is located at position (X, Y, Z), what's the fastest way to determine which item is closest to that player?

Well you can just check the distance to every item and use whichever item is closest. If you read last week's column, you'll remember that this search requires linear time. If you double the number of items on the level, it will take twice as long to find the closest. Certainly we can do better than linear time.

There are a number of data structures that can solve this problem. Some of them might require more setup time but give slightly faster search time. The structure I chose is an octree. It's a tree that holds some number of three dimensional locations. If you want to learn more, I recommend the Wikipedia article on quadtrees as well. A quadtree is the two dimensional analog of an octree, so it's a bit easier to visualize.

In the example I gave last week of a filing cabinet holding 100 pieces of paper, 1 cabinet held 10 folders and each folder held 10 papers. This is a total of 111 pieces of data stored (100 papers + 10 folders + 1 cabinet). In computer science terms, each data storage location in a tree is called a "node". The node where searching begins is called the "root node". In the filing cabinet example, the filing cabinet itself is the root node. The cabinet and folder nodes split 10 ways, and the pieces of paper do not contain any sub-nodes. Nodes that do not further branch are called "leaf nodes".

Getting back to the octree example, the BrainWorks AI code needs to store the spacial locations of 50 or even 200 items in a tree. Each node in the octree contains is one of the stored points, and the node branches 8 ways. Visually think of each node as dividing space into 8 octants-- upper north west, upper north east, upper south west, upper south east, and four more counterparts for lower sections. All points that are above, north, and west of the root node are stored underneath the upper north west node, and so on for all other octants.

For example, suppose there are 50 items on a level. One of them is stored in the root-- say it's a rocket launcher. Of the 49 remaining items, 6 of these item locations are above, north, and west of the rocket launcher. The root node will link to a node for one of these six items-- a megahealth. The other five items will be stored undernearth that megahealth.

Again, the octant division property applies to all nodes, not just the root. So the megahealth also divides its space into eight octants, and the five remaining items will be appropriately placed underneath it. Keep in mind that each of these items is guaranteed to be above, north, and west of the rocket launcher. But items that are above, north, and west of the megahealth will be stored in a different octant than those above, north, and east of the health. Given that there are eight possible areas and only five remaining items to store, chances are that these items will all be stored in one of the eight sub-nodes of the megahealth. (If two items happen to both be below, north, and east of the megahealth, then the first will be stored under the megahealth, and the second will be stored under the first.)

Searching a tree is generally proprortional to the depth of the tree-- the greatest distance from the root node to a leaf node. And for a tree of 50 items, the depth is usually 3 or 4. Going up to 100 items might push the depth to 5 for some level layouts. So the octree structure makes answering the question of "which item am I nearest?" much, much faster than checking all 50 or 100 items.

Given that items have fixed locations on the level, the octree needs to be created once when the level starts, but the tree will never modify during the game. (Ignore items that drop from dead players-- those are handled differently and not useful for statistical tracking anyway.) The actual algorithm is pretty simple, a bit slow, and very likely to produce a well balanced tree. Here's how it works.

Given a set of items:

Average together all their positions to find the center
Check each item to find which one is closest to the center
Add that item as the dividing node
Separate all remaining items into one of the eight octants this center divides
Repeat the process on each octant that contains at least one item

By "not terribly fast", this algorithm could take up to half a second on a very, very large map. So still inconsequential, and well worth the one-time hit to produce a better tree.

Of course, there's still the question of how you search the tree to find the nearest item, and that I will be explaining next week. Here's some food for thought though: even if you identify which octant a point would reside in, you still need to search up to seven of the eight octants to guarantee you've found the closest item to your input point. Searching this tree is not done like most tree searches. For those of you who like thinking through these kinds of problems, try to figure out how the search algorithm actually works. I'll give the answer next week.

Given a set of item locations arranged in an octree and one player location, describe an algorithm to find the item nearest the player.

Good luck!

Needle In A Haystack

2008-09-01T06:00:00.000-07:00

For a long time, I kept all my important documents in a single box. That would be stuff like my passport, car title, insurance information, and so on. After my wife and I bought a home, we had enough papers to file that our simple box wasn't sufficient. We bought large filing cabinet and it's served our needs ever since.

It's not that we needed to look up stuff any more frequently than before. But when we went from 100 pieces of paper to 1000, we didn't want it to take 10 times longer to find anything. To find stuff in the box, I riffled through it until I found the paper I needed, sometimes searching the entire thing. More stuff means the search takes longer.

A filing cabinet solves this problem by providing a natural structure for organizing the papers. For example, all the papers on the home can go in one file folder, all the car information in another, and all the identification documents like passports in a third. Then the lookup process becomes simpler, as I can ignore everything that's not part of the folder that keeps whatever I'm looking for.

This is a real savings in time. Imagine you have 100 different pieces of paper. You can either stuff them all in one box or you can organize them into 10 folders, each containing 10 papers. What's the average number of papers you'll have to look at to find any given piece, assuming you always know what folder should contain that piece of paper?

When looking through the box, you could look at anywhere between 1 and 100 pieces with equal probability. So you'll average (1 + 100) / 2 = 50.5 pieces before you find it. When looking through the folders, you have to find the correct folder from 10 options and then the correct paper of the 10 pieces in that folder. It will take between 1 and 10 tries to find the folder, or 5.5 on average. And similarly it takes on average 5.5 tries to find the paper in the folder. That's 11 tries total for the folder compared to 50.5 for the box. The filing system can be searched roughly five times faster!

Or is it really five times faster? What happens if you have 1000 pieces of paper instead of 100? When you have this much information, you could create folders and subfolders to further organize the papers. If you have 10 main folders, each of which contain 10 subfolders, and each of those contain 10 papers, then it will take 5.5 + 5.5 + 5.5 = 16.5 tries to find any one of the 1000 papers. It would take 500.5 tries if they were in a box. So in this case the filing system is about 30 times faster.

The more data there is to store, the more efficient the filing system is compared to the box. That's because the filing system is a whole order of magnitude more efficient. In mathematical terms, searching through the box takes linear time. When you add ten times as much stuff, it takes ten times as long. But searching the filing system takes logarithmic time. Adding ten times as much stuff just means it takes one more search step. Very roughly speaking, searching is proportionate to the number of zeros on the size of the data. Searching 100 things takes twice as long as 10 things, and searching 1000 things takes three times long. Want to search 1,000,000 things? That's six times longer than searching 10 things.

You might be wondering what all of this has to do with artificial intelligence. At the heart of almost all AI is data, and data doesn't mean anything if you can't find it when you need it. That means efficient data storage is the required foundation for artificial intelligence. An algorithm is only as good as the data it has access to, and good AI needs a lot of data. If all the data were just shoved in a box, there's no way the computer could run the AI's thought process fast enough. Indeed in almost all computer science, the way in which data is organized is just as important as the actual data being stored. After all, what good is information if you can't find it? Next week I'll give a concrete example of how BrainWorks puts this concept to use.

Sitting on the Fence

2008-08-25T06:00:00.000-07:00

For roughly the past month, I've been interviewing at a few different places looking for a new job. I've felt overqualified at my current place of employment and really wanted some more challenging work. At the end of the process I had two offers to choose from, both of which were really similar and far superior to my current job. I suppose most people might decide based on some simple, simple heuristic like "highest salary", "most attractive coworkers", or "did I flip heads?"

But I'm a compulsive optimizer. Naturally I agonized for two weeks over which offer I should. I was hoping one offer would be clearly better than the other, but both the corporate cultures and salary packages were similar. I analyzed every angle I could think of, from the commute times to little things my interviewers had said. I called friends I knew and got as much inside information about the places. I made risk estimates for the companies based on their respective industries. Talk about overkill! My opinion changed on a daily basis, but after two weeks I finally made my final decision.

This is, of course, the exact opposite of what good AI should do.

In most systems where a computer must analyze some data and make a decision, there's a problem when two choices are very similar. Sometimes the system can get caught in an oscillation state where working on either choice makes the other one better. For example, a Quake 3 bot might decide to pick up armor, but the act of moving towards the armor makes it decide that picking up some health is a better choice. So the bot instead moves towards the health, and on the way decides the armor is better. The bot can end up in a situation of indecision where it picks up neither of the two items, and clearly this is the worst of the three choices.

Item pickup isn't the only situation where this occurs. Target selection and weapon selection can have the same problems. A bot can try to chase down one target but on the way decide it would rather aim at a different one and waste valuable time aiming at neither. Or in theory a bot can switch to one weapon and change its range based on the new choice, but by the time it gets in position it decides another weapon is useful. If you've never seen a BrainWorks bot do this, that's great! I've worked hard to remove these indecisions from the AI, but it's neigh impossible to make AI that works in all situations. There are two basic tactics you can use to solve indecision oscillations, each with their own costs.

Most of the time I give the scoring estimate a bonus of between 10% and 25% for keeping the same decision. In other words, the AI won't change its mind unless the new option is at least 25% better than the old. The higher this factor is, the fewer oscillations will occur. However, the risk is that an option that's genuinely 8% better will be missed, even if it wouldn't cause an indecision loop. Additionally, it's rare but still possible that even with a 25% factor, the bot can get stuck in indecision. This method is best used for situations where the estimated value of one option over another doesn't change that rapidly. This is how general item pickup oscillations are handled, as well as enemy selection.

When a small buffer isn't sufficient, the other option is to prevent the bot from changing its mind until it completes its selection, or at least for a reasonably long duration (such as half a minute). This is guaranteed to solve the problem. But there are two drawbacks. Not only will the bot be more likely to make suboptimal decisions, but it could get caught in a situation where it's very predictable. As such, it's best to apply this tactic only when absolutely necessary and ideally when the action can be completed quickly.

This method is used in item pickup for items the bot is very nearby. If a bot can pickup an item in less than one second, it immediately overrides all other item pickups and grabs it no matter how useful some other item may be. It turns out that even with a bonus factor for keeping the same item selection, bots could still get stuck in indecision loops when they were very close to an item of low value. That's because the estimated points per second is the ratio of two very small numbers, so the estimation has a large margin of error. No bonus factor could stop the bot from switching between the bad nearby item and the good item that was far away. The code forces the bot to spend half second to grab the nearby item, thereby preventing the loop from occurring.

Regarding my new job, I was leaning towards the place that offered slightly less money but seemed to have higher job satisfaction. In the end they decided to beat the salary offered by the other company, so it was a no-brainer to pick the job that has both higher pay and happier workers.

Scouting for Trouble

2008-08-18T06:00:00.000-07:00

One of the funny things about designing artificial intelligence is that you run into all kinds of problems you never even considered. For example, I spent a lot of time designing algorithms to handle realistic aiming. But what does the bot do when it's not trying to line up a shot? It has to look at something, even if there's no one else in the area. That's because the bot must either hear or see an enemy to become aware of it and start shooting back. Scouting around for likely areas an enemy might be is the best thing the bot can do when, well, it has nothing else to do.

The issue is that a bot doesn't visualize the game the same way a human does. Technically stated, bots don't visualize anything at all. While we as humans see a brightly rendered three dimensional area, that's too much information for the artificial intelligence to process in a short amount of time. A Quake 3 bot actually uses something similar to sonar to "visualize" the world. It sends out a instantaneous pulse in a perfectly straight line and figures out where that pulse hits something. It can even tell what kind of thing the pulse hits, like a wall or some fleshy target. However, there isn't enough time to send pulses in all directions, so the bot needs to carefully spend its allotment. If it spends too much time figuring out what's nearby, it won't have enough time to think about other important things (such as where to run).

Getting back to the problem at hand, think about how a human player would decide where to look for new targets. In a hallway the solution is obvious-- look in both directions the hallway runs, but not towards the walls. In a large room you look towards the other side of the room, in doorways, and archways. As humans we quickly process the two dimensional rendering of the area and mentally create a three dimensional depth map. Then we look into the areas with the furthest line of sight, for the most part. That's because new targets come from far away.

While it's very hard for a bot to identify which areas are doors and hallways using its "sonar", it can at least determine which areas of the room are far away. To do this, the bot just sends out eight pulses around itself and finds the three that have furthest line-of-sight. It does some additional checking to find the ground below that point, so when the bot is on a ramp or staircase, it actually looks down the direction of the stairs. And then the bot randomly picks one of the three furthest directions. It's a pretty simple solution, and it does a great job of making sure the bot scouts in the areas where opponents will appear.

Now the interesting thing to note is that no human opponent ever actually sees the bot doing this for more than a fraction of a second. Once you're in the bot's field of view, it will start aiming at you rather than looking for more targets. But a human would notice the results of this code, in that the bots notice them and start responding in a realistic manner. An algorithm that no human actually sees is obviously less important than one they interact with all the time. But that's not the same thing as unimportant, and the relative simplicity of the scouting code reflects this.

Project Planning

2008-08-11T06:00:00.000-07:00

As BrainWorks is a complete rewrite of the Quake 3 AI code, it was a relatively daunting project that required a lot of effort in many areas. First person shooter AI requires aiming and firing decisions (not the same thing!), item pickup, weapon selection, movement, visual and auditory awareness, and high level strategy to give a non-exhaustive list. Simply put, there's a lot of things to write and they all have to work together. When the project is this large, it's hard to even know where to begin. How do you prioritize such a wide list of features when, quite frankly, everything is required?

For BrainWorks, the secret is that I didn't start with plans of an entire rewrite. Rather my objective was to get the AI in a state where it was usable for another mod I had written, Art of War. (You can visit the Art of War website if you like, but it's horribly out of date.) The mod was a game designed similar to Natural Selection, although it was released years before Natural Selection even started development. Conceptually, think of each player in Art of War as a single unit in a real time strategy game. You can collect gold for your team's bank, then spend the gold to build buildings that unlock new units and powers. You can change to an assault based unit to attack the enemy base, be a defender, or play an assault support class. There are four unique factions in the game, each with their own basic play style and variations. Basically everyone who played it really enjoyed it. Some day I'll go into more detail about the design of it and post the source code for those who are into that kind of thing.

So if Art of War is so great, why have you never heard of it before?

Well if each player is on a team where you can pick one of four kinds of units, the game is well designed if a good strategy involves one player of each unit type and poorly designed if everyone wants to play the same kind of unit. As this was a well designed game, that means you are required to have 4 players per team to even get a feel for what the game play was like. With four players per side, that's a minimum of 8 players to even start a game. You could play with less, but a 2v2 match just wasn't that exciting. The mod required a strong player base to ramp up in popularity and it simply never got it.

The whole reason I started writing AI was to address this problem. If you could get three players on a server and have five bots filling in the gaps, you could get a semblance of a good game together while more people joined. Once enough people had joined the server, the bots would get kicked and people could play a real game. And so begins the tale of BrainWorks...

When I first started BrainWorks, I didn't plan on doing a full rewrite. I just wanted some additional high level tactical decisions for Art of War. After looking at the initial code, however, I realized a better starting point was doing AI for the basic Quake 3 game, where you just need to run around and shoot things. My objective at this point was just better weapon selection.

Of course, having written better weapon selection code, I realized it was all meaningless unless the bots could pick up useful weapons, so I had to rework the item pickup code. And since weapon selection was based on statistical tracking of weapon accuracies, the selection code wasn't very useful until the bots had reasonable aiming, not inhumanly good aim.

Around six months into the project I took a step back and realized that none of the work would be that helpful unless I redid the entire code based. At this point I had a decision. I could have just thrown up my hands and walked away, declaring it too much work. But I decided to press on with a full rewrite.

I wish I could say I had some grand plan of how to prioritize everything in the project, but it started as a small project. So my plan of attack for solving this involved writing everything in chronological order. For example, bots must scan their surroundings for new enemies before deciding who they should shoot at. They need to pick an enemy before aiming their weapon, and they need to aim before firing. I tried to focus on the earlier steps first because that would define the exact information the bot could use to make its next decision. If you write it backwards, you'll often end up assuming you have data that you can't actually get in the format you need. When you have a truly large project, its often best just to dive in and start working on something, however. The sooner you get your feet wet, the sooner you'll have a feel for all the complexities and intricacies that need to be accounted for.

Mind Candy

2008-08-04T06:00:00.000-07:00

If you haven't worked in the game industry, you might think of a game designer job as a dream job. You get to create your own toys and then play with them! Well it certainly has its moments, but it's important to realize that the job is still a job. There will always be boring parts that need to be done. And more often than not, the job involves creating your own toys and then figuring out why the won't work or aren't fun. In fact, it's a lot like every other programming job except you get to make toys rather than spreadsheets, word processors, or websites. While it's fun to design toys for a living, there's a pretty high cost to it. And I don't mean the "there's boring stuff too" cost that comes with every job. Here is the cost:

The more time you spend designing a game, the harder it is to enjoy playing it.

The problem is that over time, you have to think of the game in terms of its individual core mechanics. But the wonder and enjoyment comes from how those mechanics are organized into something greater. No game has infinite replayability. There always comes a point where you think, "That was fun, but I won't ever want to play this game again." A game might have 10 hours of good fun, or even 100 or 1000. But no game is fun after 10,000 hours. Why is that? Why can't games be fun forever?

Games are mind candy.

They are a thought treat. Personally I approach games as puzzles to solve. The very act of thinking is fun, at least for my mind. The challenge is learning how the make the best move in each novel situation the game presents. As you play a game more, you learn it better. And eventually you simply recognize the situation and can play on autopilot. At that point the game ceases to be fun because it is no longer challenging. It can still be entertaining to execute the actions, and in playing I can encounter a few new challenges, but in general the candy of the game no longer tastes sweet to my mind.

Unfortunately, my job as a game designer isn't done until I've solved the game and determined there are no degenerate solutions. The harder it is to determine a game's optimal solution, the more replayability it has. (Note that replayability just defines how long it takes for the game to cease being fun. It has nothing to do with whether or not the game is fun in the first place, or to what extent.) If it's too easy to solve the game due to an obvious dominant strategy, no one will have fun playing the game once they figure it out and all my design work is for nothing.

While I'm not an author, composer, or movie producer, I suspect this problem extends to all jobs designing entertainment. A really satisfying book is one where all the plot pieces fit together nicely and there are no obvious plot holes. The characters need to feel genuine and you need to understand how their development as people natural extends from their personalities and experiences. No author can convincingly do that without really getting into the minds and lives of these fictional people. To build the depth of a good book, the author must understand that depth before the final editing pass is done and the book is sent to print. They will never have the enjoyment and wonder of watching how the lives of their characters unfold. They sacrifice that potential joy so that other people can experience it too. It's just part of the creative process.

For my part, I find I don't enjoy playing first person shooters as much as I did many years ago. Maybe after a few years break I'll enjoy them again, but that's just the personal price I paid so that other people get more enjoyment from the games they play.

Mission Impossible

2008-07-28T06:00:00.000-07:00

One of the basic realities of life is that if something is easy, it's already been done. In the grand scheme of things, all unsolved problems are hard problems. While that might not have been true 300 years ago, the advancements of widespread education and information technology have made it easy to find potentially talented individuals and make sure everyone knows about their work. The result of that for almost every technological problem you can think of, either someone has solved it already or it's very difficult.

So what do you do when you need to do something that hasn't been done before and it turns out the problem is very hard? How do you break an impossible problem into something solvable? While there's no right or wrong way to go about this, there are three basic approaches you can use. Assume that the objective of a perfect solution is either practically or literally impossible, and you just want the best solution that's actually feasible.

#1: Improve on an existing base

The concept here is to focus on the small, incremental progress you can achieve rather than the large, difficult end objective. This is the approach I took when solving the general high level goal selection in BrainWorks. Bots must constantly decide what they are most interested in doing. Should they track down a target or look for more players? In capture the flag, should they defend their flag or get the opponent's? I confess I've added nothing revolutionary to this logic, and the correct choice is something even experienced human players have trouble making. Instead I just added a few pieces of additional logic to help the bots decide. BrainWorks bots will consider tracking down a target it's recently heard but not seen, for example. It's not algorithmically amazing but it's progress and that's what matters.

#2: Break the problem into pieces and only do some of them

I used this approach when designing the item pickup code. Roughly stated, the implemented algorithm is to estimate how many points the bot would get if it were to pick up different items and it selects the item that gives the most points. While the item selection is reasonably good, there's a lot of corners cut for the purpose of reducing complexity. How valuable is a personal teleporter anyway? BrainWorks just values it at 0.6 points and calls it a day, but clearly such an item is far more useful in some situations than others. And the estimations of where enemies are likely to be are based on proximity to items, not on actual geographic locations (like being in a specific room or a hallway). The estimates aren't remotely perfect, but they are still good enough for the purpose they serve: a rough estimate of what item the bot should pick up next. Designing this code wasn't even possible without ignoring all the truly hard parts of item pickup.

#3: Accept defeat and deal with the consequences

This is essentially The Attitude of the Knife. Sometimes things just don't work out and can't be fixed, and all you can do is accept that reality and handle the consequences as best you can. This was the approach I took for the navigation code in BrainWorks. As I've mentioned before, the navigation tables have huge issues on anything other than a flat, two dimensional map. While I'd love to solve this problem and I'm certain I could do it, I just didn't have enough time to tackle it. There were a few clearly bad navigation decisions (such as jumping between ledges) that I wrote code to manually fix, but those were essentially stopgap measures.

While it's not ideal, there's nothing wrong with accepting defeat against hard problems. After all, you aren't the only person in the world who couldn't solve the problem.

Gaming The System

2008-07-21T06:00:00.000-07:00

Typically, a game artificial intelligence developer views their job as creating an opponent that will entertain the player. There's nothing wrong with this view. After all, if the player isn't having fun, it doesn't matter what the AI does well. The most realistic play in the world doesn't mean a thing if the player doesn't enjoy playing against it. It's possible to make AI that plays like that, of course. If a human can play so well it frustrates their opponent, an AI can do that too. But is the problem the AI design or the game design? I'd argue the game design is at fault. In other words,

AI design is a counterpoint to game design.

Game design really has two parts to it: making the game fun and making the game challenging. Alternatively, you can think of this as making the game fun for the winners and making the game fun for the losers. If a game is too easy for you, then chances are your opponents are getting crushed and aren't having a good time. A good multiplayer game is one in which everyone enjoys it regardless of whether they win. And a good single player game is one where it's neither too hard nor to easy to win.

That means AI can be a valuable tool in the design feedback loop for a game. If the AI continually exploits a certain unsatisfying strategy, you can expect players will do the same, and the game should be changed. When the AI uses a variety of interesting strategies, that's a good sign your game design is solid.

In the design of BrainWorks, I came across a few instances of AI behavior that commented on the game design of Quake 3. For example, on any level with a reasonably accessible BFG, the bots will do everything in their power to grab and use the weapon. They might even camp BFG ammo just to make it harder for other people to use the gun. While it makes gameplay boring and predictable, they aren't necessarily doing the wrong thing. The AI determined that the BFG was 10 times better than the next best option and made it choices accordingly.

There's also the issue of the machinegun. You might notice that if there are a lot of players on a level, the bots will sometimes flat out skip weapon upgrades and just use the machinegun. This also isn't a mistake. When there are more players on the level, each player's life expectancy decreases. When the bot won't live that long anyway, there's no point in wasting time for a new weapon. The starting machinegun is only slightly worse damage and comes with a reasonable amount of ammunition.

In the case of the BFG, the weapon is obviously too powerful, but it's pretty clear that was the point. The machinegun is a different case though. Is it really too powerful or do players just start with too much ammo? Or maybe it's fine that this is a strategy when a level becomes overpopulated, and human players that waste too much time picking up weapons are just playing it the wrong way.

This is just opinion, but I believe the problem is the amount of damage the machinegun deals. The reason I think this is that there are actually has two damage values for machinegun bullets. Normally it's 7 damage per hit, but in teamplay mode it's only 5 damage. Apparently iD software determined that in a teamplayer game, the standard 7 damage was too much. And they're right, but the problem wasn't the weapon's use in team games. It's the weapon's initial power in games with lots of people, even large free-for-all games. What would be best is if the weapon just dealt 5 damage always. Then freshly respawned players had incentive to grab a new weapon as soon as possible. In game design, the simpler solution is usually the best choice.

On a related note, game design is also a hobby of mine. My wife and I have two upcoming board games that will hopefully be released end of this year or beginning of next. If people are interested in more columns explaining the thought process that goes into game design, I'm happy to write more on that subject. In my mind, designing a game is very similar to designing artificial intelligence.

The Power of Spite

2008-07-14T06:00:00.000-07:00

In almost any situation where there are limited resources, people are presented with the option of spiteful action. These are actions where someone else is penalized for an action where you received no benefit. Most of the time the spiteful play even had a slight cost, such as additional time. In war, there can be spiteful military decisions, such as General Sherman's sowing of salt during his march to the sea during the American civil war. Sowing salt makes the land completely unfarmable for a period of several years, and Sherman's choice to do this crippled a large part of Georgia's agriculture.

Of course, one of the cardinal rules of war is "waste nothing" since resources are tight. While these spiteful actions cost Sherman's troops a little time and resources, they had a tremendously detrimental effect on the Southerners, both demoralizing them and making it harder for the local citizens to survive. Without commenting on the ethics of this decision, I will merely note that Sherman's spiteful actions gave an advantage to his own side, the North.

So in a first person shooter game like Quake 3, players have countless opportunities to make spiteful plays. Every time they pick up an item they don't need, they are potentially preventing another player from getting that item. A good AI player needs to know when it should pick up an item out of spite. And naturally it shouldn't pick up every item, or it would waste far too much time. There has to be a method for deciding when the spiteful play is worthwhile and when it's not.

In BrainWorks, the additional time cost is automatically factored into the item selection code, as extra time spent grabbing items means a loss of points from delaying the bot's main goal. But the bots still need to know exactly how much an item pickup costs the opponents. Once that value is known, the bot estimates its relative score gain as the spiteful point loss divided by the number of opponents. In other words, keeping an item from 1 of 2 opponents is a lot more valuable than keeping it from 1 of 10. The fewer opponents the bot has, the more they prefer spiteful pickups. So while these spite values don't matter much in large group games, they can have a significant impact on a bot's item pickup selection in 1-on-1 matchups.

This still begs the question of exactly how much a spiteful item pickup costs opponents. Players know that it's great to grab armor even when you're at the maximum and grabbing ammo will rarely have an effect. But BrainWorks bots need to derive this fact. The AI does that by predicting how an individual item pickup will increase a players score for each of three typical players. There's a freshly spawned player, who generally wants armor and weapons. There's a hurt player, who benefits the most from health. And there's a decked out player who probably wants powerups. The AI computes the scores for each player with and without each possible item. The item's spite value is defined as the greatest point gain it gives any type of player. The code uses the maximum estimate rather than the average because players that most need an item are the most likely to pick it up.

That means the spite value is really just an estimate of the item's general usefulness. In fact, the bots use these estimations to isolate the most valuable items on the level and time their respawns, and just as a general area that other players are likely to be in.

A Simple Solution

2008-07-07T06:00:00.000-07:00

As the story ended last week, I had uncovered a very strange bug in BrainWorks. There was up to a 10% accuracy difference between the first bot added to the game and the last bot. No one guessed the correct cause, but this one from cyrri was the closest:

bots occupy client slots in the same order as they are added.
in the very rare cases of two clients killing a third one simultaneously, it is allways the one with the lower slot id that gets the frag, becuase his usercommands are processed first. the other one gets a missed shot.

This actually does happen, but it only accounts for a 1% to 2% accuracy change between the first and last bots. Also, this value increases as bot accuracy increases, since it's more likely that two bots will be lined up for a good shot at the same time.

The real culprit was the mechanism through which the server processes client commands. The server processes each human player's inputs as soon as it receives them (as fast as 3 milliseconds for a human), but the inputs for bots are processed exactly once every 50 milliseconds. In turn, each bot handles its attacks and then moves. Then the next bot is processed, and they do this in the order they were added to the game. See the problem?

Every bot made its decisions based on where the other bots were currently located at the end of the last server frame, but only the first bot will actually attack against targets in those positions. By the time the last bot aims, every target had already spent 50 milliseconds moving. The first bot had 0 ms of latency and the last bot had 50 ms. Now 50 milliseconds isn't too bad, except the last bot was playing as if its latency was 0. That's why it missed around 10% more.

Since one of my original project constraints was that nothing would change in the server code, that meant that at least some bots had to play with 50 milliseconds of latency. There were no changes I could make to reduce this problem. So the solution was to add latency to all the bots. I wrote a feature into the AI core that tracked the positions of all players in the game for the last half second worth of updates or so, for all bots and all humans. Then if a bot needed to know where a target was, it looked up the properly lagged position and did basic extrapolation to guess where the target would be at the exact moment of attack.

Implementing this system gave all the bots very similar accuracies (within 1% due to the issue cyrri pointed out). But now the problem was that all bots the same accuracy as the worst bot, when they should have had the accuracy of the best bot. It turned out this "basic extrapolation" wasn't good enough. The original Quake 3 AI code used linear trajectories to estimate where a target would end up, nothing more sophisticated than that. So if a bot aimed at a target running to the bottom of a staircase, the bot would keep aiming up into the ground for a while, even though a human would know the target would move straight.

I tried some slightly more advanced solutions, doing basic collision checking against walls, but that didn't solve the problem of running down a ledge. I eventually concluded that humans have a learned sense of physics they take into account, and the bots would need that same sense if they were to play like humans. Solving this problem was both straightforward and time consuming.

I made an exact duplicate of the entire physics engine, modified it for prediction, and placed it in the AI code.

Every detail needed to be modeled-- friction, gravity, climbing up and down ledges, movement into and out of water. Even the force of gravity acting on a player on an inclined ledge. Everything had to be duplicated so that the bots could get that extra 10% accuracy in a human-like manner. It was not easy, and I learned far more than I wanted to know about modeling physics in a 3D environment. But in the end, it was worth it to see bots that could aim like humans.

A Peculiar Bug

2008-06-30T06:00:00.000-07:00

In my postmortem of BrainWorks, I mentioned one of the big things I did right was creating a powerful debugging infrastructure directly in the code base itself. In general, it's easiest to test and maintain software when as much of the testing is fully automated as possible. Now that's easy for a program like Excel, where you can create a battery of sheets with strange formulas that might cause errors, and then confirm that the numbers all match up. If the software creates something measurable, it can usually be automatically tested.

Naturally, this means automated testing is far harder for Artificial Intelligence than other areas of computer science. The goal of BrainWorks is "bots that appear to be human". How do you teach a computer to measure whether a player's decisions appear to be human? That's tantamount to writing a program that can judge a Turing test, which is as hard as passing one. Fully automated testing of the AI isn't an option, but there are certainly things that can assist testing. All you have to do is identify an measurable component and check if it meets the human expectations.

For aiming, BrainWorks already measures accuracies with each weapon in each combat situation. The bot knows that it scores perhaps 35% with the shotgun against an enemy a few feet away and 5% against an enemy a hundred feet away. But it doesn't know if those numbers are reasonable until I tell it what to expect. The vast majority of the testing I did with aiming involved forcing a bot to use a particular weapon, making it play for hours, and then checking if the numbers "looked good". Generally they didn't, and I had to figure out why they weren't matching my expectations. In this testing process, I uncovered a variety of interesting bugs. Large sections of the aiming algorithm were rewritten around a dozen times trying to get things right. I found several errors in how accuracy was even being measured. But the strangest error I encountered was something totally unexpected.

I was monitoring Railgun accuracy as way of testing the overall ability to aim. It's an instant hit weapon with no spread and infinite range, so it's a great initial test case. I loaded up eight bots on a wide open level and forced them all to have exactly the same skill, then ran them for several hours. Curiously when I checked the results, their accuracies weren't all the same. The best had an accuracy around 75% and the worst was around 65%. Moreover, their scores reflected this.

I activated some mechanics in the system to modify aiming behavior. First I turned off all errors, so the aiming should be flawless, like watching a human who doesn't make mistakes. Their accuracies were still stratified. Then I completely circumvented the entire aiming code, forcing the bots to aim directly where they wanted to. That gave the bots what most people think of as an aim hacking, so their aim should have been perfect. But even still, testing showed that some bots would get higher accuracies than others. Sure, there was one bot that would score 99.9% accuracy, but another bot would only score 97%. When a bot has perfect information and reflexes, it should not miss one in 30 shots.

Then one day I noticed that all eight bots were sorted in alphabetical order. The bot with the first name had the highest score (and accuracy) down to the bot with the last name having the lowest score. Since the odds of this are 1 in 8! = 40,320, I considered this curious but still possibly a coincidence. So I tested it again, and each time the bots were sorted alphabetically! That was the final clue I needed to isolate this bug.

The script I used to start testing adds the bots in alphabetical order, so I tried swapping the order different bots were added and their accuracies changed as a result. Each time, the most accurate bot was added to the game first and the least accurate bot was added last. For some reason, the internal numbering of bots was affecting their aim.

So why exactly was this the case? I'll let you puzzle over it for the week. Next week I'll explain why this happened and the extreme amount of work that went into solving it.

Abstract Art

2008-06-23T06:00:00.000-07:00

As part of a special program in high school, one of my classmates, Chris, applied to work under an orchestral conductor during his senior year. Chris was a brilliant musician, but the interview process with this particular conductor was a grueling two hour endeavor, much more than the typical person in the program would have gone through. The conductor wanted to find out if Chris could do more than play music well. He wanted to know if Chris understood what music was. Chris related that the experience was more of an intellectual sparring match than an interview, and everything revolved around answering one question:

What is art?

Now there are all kinds of meaningless philosophical answers to this question. "Art is whatever you say it is" or "Art is beauty". If you dig into these answers, you just end up with the tautology "Art is art". That's true, mind you, but not very helpful. However, the music conductor had a very good answer to the question, and he accepted Chris' application when Chris understood the answer.

Art is expression.

When I heard the answer, it really resonated with me. That's a real answer, not a tautology! The purpose of art is for the artist to convey a point to those people experiencing the art. That means art can be measured by two factors:

How important the point is to the audience
How well the audience understands the point

Note that the audience is a crucial part of art. Immediately when I heard this answer, I understood why I'd always hated abstract art. It's bad art because it neglects the audience. Generally abstract art boils down to one of two types.

Type 1: "This black square set against a mess of orange and red circles represents the current triumph of humanity against nature, but the ultimate futility of the struggle." While that might be a good point, it's not particularly novel, and the point isn't conveyed very well. Other artists have done a much better job of creating works that express this point without even intending to. Fundamentally I feel like this art exists so that art snobs can feel elitist, because they "understand" the point and other people don't. But obfuscating your point doesn't make you clever; it makes you a bad artist.

Type 2: "This black square set against a mess of orange and red circles doesn't represent anything because art doesn't need to have a point. It can mean anything and everything!" In that case, the artist isn't expressing anything at all. They are doing an exceptional job of doing nothing. It's not art because it's not expressing a real point.

This doesn't mean art needs to be hyper-realistic. Reality isn't as fuzzy as the impressionistic paintings Monet did of water lilies, yet millions of people from different cultures and backgrounds have all recognized the beauty of that artwork. The true test of art is not whether its point is realistic, but whether the point has been successfully conveyed to the audience.

With that as the framework, I'd like to talk about something different from Artificial Intelligence: the writing of this column. You might notice that I express a lot of stuff in layman's terms, glossing over details, sometimes going back to them and sometimes not. It's my goal that this column will never require more than a first year undergraduate's experience to understand. That's explicitly because it's crucial that people can understand what I'm writing. If I can't explain things simply, then I'm not doing my job as a writer.

Each week I pick a topic that might be interesting. This column is partially just a personal soap box-- I talk about things I find interesting and usually I relate it to AI. But I cannot escape the purpose of art, that art is only meaningful to the extent that your audience cares. So I want to get a better understand about what topics and styles you, the reader, appreciate the most. It's my job to write about things you want to read.

I analyzed the articles I've written and they fall into one of four categories:

Technical discussions of exactly how an algorithm works. Discusses actual code.
Overview discussions of how to approach an algorithm. Discusses situations that would require programming, but not actual code.
Reflections on artificial intelligence and computer science in the abstract. Discussion relates to philosophy.
Philosophical discussions. Usually relates to intelligence, may or may not relate to artificial intelligence.

I'd love to know what kinds of articles people enjoy the most so I can plan a good mix of them. Please, post your preferences in the comments! And if you're interested in hearing a particular topic covered, also mention that and I'll try to cover it at a later date.

Give It A Go

2008-06-16T06:00:00.000-07:00

While my primary area of expertise is First Person Shooter AI, I still get a number of questions about other AI applications. One of the hardest game AI problems to date, if not the hardest, is the game of Go. What's so surprising is that Go has simpler rules than Chess, but Go AI is several orders of magnitude more complicated than good chess AI. Now consider that the best chess AI to date has only beaten a grand master, Kasparov, when it was specifically trained to counteract Kasparov's play style. In the first set of games, which Kasparov won, he commented that he did better when he started making what he considered the second best move instead of the best move. Apparently the first AI was so hyper-tuned against Kasparov's play style that it defeat other similarly good plays that it did not think Kasparov would make.

Yes, that's the best the world has done for Chess AI: A program that can defeat one really good player. But Go? The best AI programs just rank as decent amateurs. And listen to how simple the rules are:

Players take turn placing stones of their color (black or white) on a square board (usually 19x19 but sometimes 9x9).
Stones of the same color that are adjacent vertically or horizontally are part of the same group and live or die together.
When a group is completely surrounded (no plays exist that add a stone to the group), that group is captured. Each lost stone is -1 point at the end of the game.
If a play is made that would cause both the placed stone and an opposing group to be surrounded, the place stone lives and the opposing group is captured.
You may not make a play that results in a previous board position.
The game is over when both players consecutively pass, opting not to play a stone. Each player scores 1 point for each empty space that is enclosed only by their color of stones.

What makes this problem so hard to solve? Well for starters, the 19x19 board is enormous. There are a lot of possible positions. Moreover, even though the plays are just of single stones, they come together to form different shapes like pixels form a picture. Different shapes have different properties, strengths, and weaknesses. A typical alpha-beta search becomes rapidly unusable when the average branching factor is on the order of 300.

Nevertheless, in the past few years there have been a few significant advancements in the field of Go AI. Rather than trying to solve Go in the same way Chess would be solved, some Monte-Carlo algorithms have shown a lot of promise. In particular, in 2006, "Upper Confidence Bounds applied to Trees" (or UCT) was developed. I'm not an expert on this subject, but in brief, the concept works like this.

While it's very hard to determine which space on a 19x19 board is the best, it's very easy to compute the score of a Go board. Contrast this with Chess where the actual pieces in play are secondary to question of whether or not a King has been captured. So the very rough idea is to try most options (pruning the obviously bad plays) and play a semi-random game for each of them (again, pruning obviously bad plays). Far less time is spent at each stage of the tree. If there are 50 possible moves that aren't terrible, one of them is just selected at random. There's no aggressive search to determine which of those 50 is the best. When the test game is done for a given play, that provides an estimate of how good that particular initial placement is. Other placements are tried and the one with the highest estimated score is selected as the computer's official move.

Of course, the actual algorithm is quite a bit more complicated than that, but that's the general idea behind the innovation. The crucial lesson here for AI designers is this:

Focus on your goal.

In the end, Go is just a score maximization problem. It's very easy to get bogged down in the details of whether a move nets 5 points or 7 points, quickly producing a state where far too many calculations need to be done. Often the best solution involves determining how much processing time you can actually spend, and then working backwards to figure out what methods will give you the most information for the amount of time spent. In Go, calculating the board's score is relatively fast, so latest algorithm that exploited this fact had a major advantage over those that did not.

Making Things Right

2008-06-09T06:00:00.000-07:00

You do it thousands of times each day without even thinking about it. It happens when you put on your clothes, brush your teeth, eat your food, and open a door. Every time you walk down a hallway, drive down the street, or type at your computer, your brain is rapidly, subconsciously processing all kinds of data. Most of the information is uninteresting and gets discarded. But when the brain encounters little bits of unexpected data, for the most part it seamlessly reacts, performing minor adjustments to correct whatever mistakes were made in fine motor control. Someone might brush your shoulder as they pass you on the street, and without even thinking about it you step to the side. Or you see the car in front of you start breaking and you slow down as well. You don't swallow your food until the pieces feel sufficiently chewed in your mouth.

Somehow, the human brain performs thousands of minor error corrections while performing basic motor controls. Large motions are fast but they aren't precise. How exactly does this work? If you mean to move your hand 10 centimeters and actually move it 11, your brain compensates for the extra centimeter and has your hand move back. Well maybe it only moves 9 millimeters instead of 1 centimeter, but it gets close enough that it doesn't matter.

When I discussed how BrainWorks emulates human wrist motion, I explained this elegant mathematical model where errors accumulate based on muscle acceleration. But I brushed over the other half of this system, where the bot accounts for its error and readjusts. With error correction that's not good enough, a bot's aiming just looks erratic. And with error correction that's too accurate, the aiming looks robotic. There's a very fine balance to getting error correction that looks human, and to do that you need to understand how the human brain corrects muscle errors. There's just one problem with that...

I have no idea how it happens.

There have been numerous studies on this kind of thing, but I confess I haven't read them. Maybe they would have been useful; maybe they wouldn't. Instead, I tried a number of different methods for error correction (that number is 5), and only one of them produced realistic results. While I can't tell you how the human brain solves this problem, I can explain the one that worked.

From the bot's perspective, here's the problem. It thinks it's aiming at position X, but in reality it's aiming at X+e, where e is the error factor. As the bot corrects its error (ie. understands the mistake it has made), the error factor e approaches zero. An error factor of zero means the bot thinks it's aiming at X and is actually aiming at X+0 = X, meaning it perfectly understands where it is aiming. Most of my failed attempts used some form of exponential or linear decay. Ironically, the algorithm that worked is by far the simplest. It's just a canonical monte carlo algorithm:

Set the error value to a random number between -e and +e
There is no step 2

It's just that simple. You can read the full explanation behind this algorithm in the DataPerceiveCorrect() function in ai_view.c. It's a 100 line comment describing, quite literally, one line of actual code. Here's the body of the function:

float DataPerceiveCorrect(float estimate_offset)
{
 // Pick a new value in (0, +error) or (-error, 0)
 //
 // NOTE: It doesn't matter what the sign of error is; the random
 // function will preserve it.
 return random() * estimate_offset;
}

While I won't reproduce that entire 100 line description here, here's a portion of it that explains in more detail why this works:

Humans seem to use the following algorithm for refining estimates. For example, finding the dictionary page that contains a given word.

1) Make a reasonable estimate given the human's understanding of the situation. For example, even though W is the 23rd letter of the alphabet, humans don't look at the (23/26) * MAX_PAGES page of the dictionary when looking up W words, simply because they know the X-Y-Z sections are so short. This indexing is similar to the Interpolation Search algorithm. This result is compared to the actual value (ie. is the guess too high or too low?) and this value is fixed as either an upper or lower
bound. In other words, you mark this page with your finger.

2) Possibly make one or two more reasonable estimates to determine both an upper and lower bound that the data must reside in. At this point the human knows that lower < value < upper. He or she knows the precise values of "lower" and "upper" but NOT of "value".

3) Pick a random value in the interval (lower, upper) and compare it to "value". This selection replaces either the lower or upper bound as necessary. Repeat until the selection equals "value".

This might seem unintuitive, but humans don't actually use binary search to narrow down their errors when the range gets sufficiently small. Perhaps it takes too much time to roughly estimate the middle. In practice people will flip through maybe 10 pages at a time, or 1 page at a time, or just pick something and see. It will take more iterations to converge than binary search would but-- and this is crucial-- it takes less time overall than computing the midpoint at each iteration.

That is it say, a computer's most natural method of search is not the same as a human's. Computers usually operate best with binary search while humans "just try something". Only when I programmed the AI to do something counter-intuitive (for a computer) did the AI seem human.

It's All In The Wrist

2008-06-02T06:00:00.000-07:00

I consider the aiming algorithm used by the BrainWorks bots to be the biggest success of the project. More than one experienced Quake player has told me how amazed they were to spectate the bots and watch their aiming, since it appears so realistic. I went through close to a dozen different iterations of the basic aiming engine until I got results I was pleased with, and this is the research that took most of the six year development period. I estimate roughly 60% of the work was spent on aiming! (If you're curious, item pickup was about 30% of the time and everything else took 10% total.) To me, the objective was to create a bot whose aiming realistically matched a human player's aiming. There's a secret to how I did this:

It's all in the wrist.

Most games don't let you spectate their AI units because it would be immediately obvious how the AI is cheating. But if you could, you almost always see on of two things. The AI might look almost directly at the player and but moves their camera away from the ideal position by a random amount, so that they miss sometimes at random. Or the AI could turn at a fixed, slowish speed, but always (eventually) aims more or less perfectly, so that they only miss right after the target suddenly changes.

The problem is that these algorithms don't model how a human wrist would interact with a mouse. So the result is that the AI won't miss at sometimes the human would miss, and will hit when the human won't hit. For example, rapidly moving side to side will confuse the AI whose aiming moves at a fixed, slowish rate, making them constantly miss. With these traditional algorithms, no matter what parameters the bot has for speed and error frequency, its aiming will never look like a human. That means its accuracy with weapons will never match human accuracies either.

The solution to modeling the aiming problem is to stop thinking about the monitor (whether the crosshairs are lined up) and start thinking about the mouse (how human motion changes the crosshairs). Human motion of the mouse doesn't move at a fixed speed, and it doesn't "randomly" move to the wrong area. The motion involves acceleration and deceleration. The faster a human moves a muscle, the larger the potential error. Humans can increase the precision of their movements by moving slower, so typical human motion involves a rapid acceleration to a desired speed, some time moving at that speed, and then deceleration to a rest state.

Mathematically this means that if a human attempts to accelerate at rate X m/s^2, the actual acceleration is somewhere in the interval [X*(1-e), X*(1+e)] where e is the maximum allowed error (eg. 10%). So a small error in acceleration is magnified over time each time the acceleration is applied to the velocity, and the velocity error in turn magnifies position error each time the velocity changes the view position. This, combined with an extremely simple, elegant error correction algorithm, provides an excellent simulation of the view motion created by a human hand on a mouse.

That's basically the algorithm BrainWorks implements. The bot splits its view state into two distinct axies: Up and down on the simulated mousepad make the bot pitch its view up or down, and moving left or right on the mousepad make the bot rotate left and right. This reduces the view problem into two identical one-dimensional problems. For each axis, the bot decides how much to accelerate and decelerate the movement of its simulated mouse to get from its current view position to its selected view position as quickly as possible. The small acceleration error factor is all that's needed to generate the smooth overshooting mistakes that humans make when they try to turn quickly with a mouse.

The calculus to compute exactly how much time to spend accelerating and decelerating is a bit complicated: there's a 150 line comment explaining the derivation in ViewAxisModify() function in ai_view.c. But the resulting equations are fast and easy to compute. It's 30 lines of actual code, only one square root, four floating point divides, and a whole bunch of adds and multiplies.

As I said, it's all in the wrist.

Little Lies

2008-05-26T06:00:00.000-07:00

I read an exceptionally good essay by Paul Graham entitled, "Lies We Tell Kids". It talks about white lies, the kind you tell to protect people rather than manipulate them. It's worth reading in full, but let me summarize what I took out of it:

People usually lie because the listener cannot emotionally or mentally handle the truth. While this is often the best thing for the listener at present, it has a cost: They do not understand reality as it truly is. They are unprepared to handle that truth when they encounter it in another fashion.

As I wrote earlier, I see strong parallels between designing AI and being a parent. There's a lot to be said for the "lies" that we as AI designers tell our AI: "No really, you can have perfect information about the entire world. And if you have to spend a long time thinking, that's okay. The entire world will stop and wait for you to decide." Whenever we let our AI cheat at the game, look up information it shouldn't know, or otherwise do things a player can't do, we are in some sense lying to our AI about what reality is like. And the price we pay is this:

We live in fear that our AI will encounter a game situation where the player will know the AI has cheated.

I spend a lot of time talking about good AI and bad AI, and why it's often worth the time to write good AI. But I'm not going to say we shouldn't lie to our bots! I recognize that "bad AI" has a place in AI design. Sometimes a corner must be cut because a problem is simply too difficult. And often these are not isolated exceptions. It's possible to write enjoyable AI that cuts an awful lot of corners as long as you know the right corners to cut.

For example, a bot can get away with almost anything if the player can't see the bot. The amount of cheating a bot can get away with dramatically decreases once the player actually watches the bot, in the same way that a magician's tricks only work when you watch the hand he wands you to watch.

The real key to writing good AI is learning which corners to cut and which problems must be explicitly solved in a human-like manner. In other words, it's about determining when to write bad AI and when to write good AI. In BrainWorks, I certainly erred on the side of too much good AI, but I think this is still a long term benefit. Once someone has solved the problem (and released the source code), everyone can reap the benefits and we can all move onto harder problems. But actually writing artificial intelligence that doesn't cheat about anything is an excruciatingly difficult problem. An AI that does everything like a human does would appear to observers to be a human. It could pass a Turing Test, the holy grail of Artificial Intelligence research.

When I say "Holy Grail", I mean it in more ways than one. Even the most optimistic estimates on when we'll be able to create a truly artificial mind say it won't be done for decades. My personal estimates are that it will be done in 300 to 1000 years. And some people claim that it is literally impossible to create an artificial mind, that it so complex that it can only be evolved or created by God (depending on your philosophical preference).

If you want an example of what roughly 700 years of research will give you, the atomic bomb was created only 500 years after the birth of Leonardo da Vinci. Think about how much science advanced in those 500 years. While that time was centuries long, it was punctuated by thousands of little scientific discoveries about how the universe worked. Einstein couldn't have had his flash of insight about how Newton was slightly wrong if Newton hadn't done work to come up with equations that were 99.9999% right.

I believe the next 700 years of AI Research (give or take a little) will be similarly punctuated. Humanity's journey to the creation of an artificial mind will be punctuated by little advancements. No one person could do all the work necessary to design everything needed for a true artificial mind, but every time an AI designer chooses to do a bit of "good AI" rather than cutting corners with "bad AI", we get one step closer to the final goal, even though the destination is light years away. I will not fault anyone for writing AI that cheats; I've done the same. But writing AI that cheats even a little less than usual is worthy of high praise.

Making Things Worse

2008-05-18T17:25:00.000-07:00

One of the major challenges of game AI is creating AI that's plays at an appropriate skill level for a range a players. Most games need AI in easy, medium, and hard settings so that as players become better, they can face more appropriate challenges. There are three typical methods for solving this problem.

More enemies appear on higher skill levels
Higher skill enemies have more health and deal more damage
Higher skill enemies act more intelligently

The first two are obviously far easier to implement than the last, but more intelligent enemies gives more satisfying results. While additional enemies works fine for a single player first person shooter like Half Life, it's not an option for Quake 3, since the player selects the number of enemies when they pick the level. The second option, adding more health and damage, was exactly what iD software used for the two lowest skill bots in the original AI. While skills 3-5 gained better accuracy, skills 1 and 2 simply dealt less damage and died faster than a skill 3 bot.

Of course it's the last option, coding additional levels of intelligence, that's "real AI". The other methods have their place, generally when deadlines must be met. But the reason good human players beat bad human players at a game isn't that the bad players accidentally had the "deal less damage" option checked. They just get outsmarted. High skill AI needs to outsmart people more often than low skill AI does, and you don't encounter that kind of AI very often in computer gaming.

When designing BrainWorks, I was determined that all skill gradations would be created by additional intelligence, at least where applicable. So, yes, bots will have more accurate aiming as their skill increases. But they also gain abilities. Here are some of the abilities BrainWorks bots gain and their minimum skill levels. (Skill 1 is the lowest, 5 is the highest.)

Strafe jumping: Skill 4
Blast shots: Skill 3
Item Respawn Timing: Skill 3 (1 item), 4 (2 items), and 5 (3 items)
Simultaneous movement and attacking: Skill 2

That's right, a skill 1 bot stops moving whenever it starts shooting. There are also other abilities that gradually increase in effect as skill increases:

Aiming speed
Aiming error correction
Audio detection of enemies
Memory of enemies no longer in view

And more that I'm sure I've missed. To make realistic low skill bots, it's best to start with realistic high skill bots and then take away intelligence until the result seems appropriate for the intended human's skill level. AI designed the other way, by taking bad AI and giving it more health and damage, never quite feels realistic.

The Emperor's New God

2008-05-12T07:00:00.000-07:00

Last week, I ended by explaining how I came to realize I didn't have a relationship with God. In short, I empirically tested whether or not God was speaking his words into my mind and I concluded that at least some of the time, things that "felt" like they were from God were not. Specific things I thought God had promised me didn't happen, so that must not have been from God. Moreover, even simple thoughts that were no more specific than a horoscope were also no more accurate. In short, I realized I had no evidence to conclude I had ever communicated with a higher power. And lacking a relationship with God, I realized I was not a Christian.

That is not the same thing as giving up my faith.

Indeed, for a few months I desperately wanted "back in". There is a categorical difference between not having a relationship with God and being unable to have that relationship. Even to this day, if it turns out God exists, then I really want a relationship with that God. Who wouldn't want to be friends with the creator of the universe?

So having lost the evidence supporting my religious faith, I started from first principles, to see if the tenets of Christianity logically made sense. In other words, I wanted to logically test whether Christianity was truth but modern Christians had pursued the faith in the wrong manner, or if the religion just didn't make sense and I should give up on it altogether.

For those of you who aren't that familiar with Christianity, here is the core doctrine:

God loves people, but God is perfect and righteous, and therefore cannot allow sin to exist in his presence. Because all humans are innately sinful and God loves them in spite of this, God devised a plan that would allow him to have a close relationship with humans on an individual level. God sent Jesus, who is himself fully God and fully human, to live a perfect, sinless life and then become a penitence sacrifice for all of humanity. Jesus suffered the penalty of death instead of me so that I could live and have a relationship with God. And Jesus.

When I broke the dogma down to this description, there were three potential logical issues that I spotted. If any one of these things is false, the religion doesn't make logical sense. Here are the issues:

Why can't God let sin exist in his presence?
Why is the penalty for sin death?
Why is someone else dieing for my sin considered justice?

According to the Bible, Jesus spent a lot of time with prostitutes, drunkards, and other "sinners", expressly because these were the people who needed ministering the most. If Jesus is fully God, why wasn't Jesus bothered being around sin? That sounds like a strong logical contradiction.

I don't have a problem accepting that everyone is a "sinner", meaning that no one lives a perfectly righteous life, never mistreating or hurting other humans, or taking advantage of them. But why is it fair that living unrighteously should be met by death? Jesus himself talks about how some sins are worthy of greater punishments than others, and a spiritual death in hell is clearly the worst punishment someone could earn. By and large, most people never do anything so disruptive to society that death is a just punishment for them. It doesn't make logical sense that all kinds of unrighteousness should be worthy of the same death sentence.

Last, there's the question of why, if I'm guilty and deserving a death, it's fair for someone else to die on my behalf. This kind of thing goes against the basic philosophy of penal systems. If I'm sentenced to serve 10 years in prison, the courts don't consider it fair if someone else chooses to serve 10 years instead, or even 30 years. That option is simply not allowed because it doesn't even relate to the issue at hand. The Bible supports this opinion as well, according to Ezekiel 18:20:

The soul that sins shall die. The son shall not bear the iniquity of the father, neither shall the father bear the iniquity of the son.

Historically there have been some penal systems that allowed, by consent, some people to receive the punishment intended for others. The result has always been that rich people pay poor people to serve sentences for them, and in most cases the systems were changed to disallow this abuse. So if I really am worthy of death for my sins, I don't logically understand why it would be okay for Jesus to die instead of me. That simply doesn't make sense.

I sent this list of questions four Christians whose opinions I respected. All of them were pastors at the time or had been in the past, and two of them had served time as missionaries in the third world as well. Of them, one actually tried to answer the questions. Two said they would think about it and get back to me, which never happened. The last didn't even respond. And all the answers I received boiled down to, "God knows better than you. It makes sense to God and that's what matters."

I'm sorry, but that's not a strong enough argument for me to base my life around a religion. The overall silence spoke louder than words and confirmed what I had suspected all along: the Christian religion is fundamentally flawed. A God might exist, and I certainly hope he does. But he is not the Christian God. A sentient being that could create the beautiful physics behind our universe can create a religion that doesn't have glaring logical holes in the core of its doctrine.

Now all that said, I'm still a big supporter of many Christian ideals as they relate to society. "Love your neighbor as yourself" is a good general rule for life, and creates a society that's better for everyone as a whole. Just because I've given up religion doesn't mean I've given up ethics.

At any rate, thanks for taking the time to hear my story. Next week I promise I'll be back with more AI themed discussion.

The Man Behind the Curtain

2008-05-05T06:00:00.000-07:00

Last week I wrote about the Christian philosophy I had. The basic conclusion was that if someone has sufficiently strong reason to believe in a religion, nothing external from that religion can ever make them lose faith. Rather you will see them modify the fringes of their beliefs to fit whatever counter-evidence they encounter, without changing the core beliefs.

For example, a Christian may start claiming that God created the world in seven days. When presented with scientific data demonstrating the world is older than 6000 years, they might claim Satan just planted the dinosaur bones so deceive people into believing God doesn't exist. (Yes, I've heard someone present argument in all seriousness.) The more rational believers can apply Occam's Razor to this situation and conclude that maybe "day" isn't the right translation, and perhaps end at the conclusion that God used evolution to create humans. But no amount of external (scientific) evidence will convince a believer that their religion is wrong. I was familiar with all of those arguments and they didn't make me lose my faith in God. Only an internal inconsistency could fracture my faith.

I moved to California in the middle of 2000, a year and a half before I started work on BrainWorks. I started the project by programming on evenings and weekends, but after almost a year I realized that writing truly good AI would take more time than that. My wife and I talked about the options, and after a lot of time spent in prayer, we came to a rather dangerous conclusion. I decided to quit my day job and work on BrainWorks full time as a way of building my portfolio, live off savings, and then look for a job in the game industry. Even though this was a crazy idea, we felt assured by God in our prayers of four things:

God would provide for us
God would vindicate this audacious decision to others upon completion
I would truly meet God through the work I did
I would finish the project before we left the state of California

In the years following, we would return in prayer during difficult times, and also ask for the prayers from others, and we were continually encouraged when we heard these promises repeated, both in our minds and from the words of others.

Concurrently, there were other issues we had lifted up in prayer. Most notably I had back problems for over a decade. Having tried multiple chiropractors, doctors, physical therapists, exercises, and received countless prayer on the issue, I was at a loss. Nothing seemed to work. But I believed in a God that could do real miracles, and healing my back was certainly not the hardest of them. After years of prayer, I kept hearing that God really was going to heal my back, at the proper time, and that receiving healing was about more than just freedom from physical pain. It was about encountering God in a deeper way, which is why the timing mattered and why God didn't just heal me immediately. We felt like God spoke to us that he would give me full healing for my back before we left California. In some sense, our sojourn in California was an opportunity to witness the miraculous works of God, in my work and in my life, and to encounter God.

So what happened? In case there is any doubt about the story's outcome, let me set the record straight. While we did have enough to live while I was unemployed, I did not "encounter God" through the work. I received a job offer in Boston in January of 2007, which my wife and I prayed about and decided I should take. I did not complete BrainWorks before I left California, nor were my back problems healed. In our cross country drive from Los Angeles to Boston, the two of us were faced with a rather uncomfortable situation. Several things we thought we heard from God, including the prayers of others, all agreed with each other, but they were all wrong. That's really not supposed to happen. At the end of that drive, we came to one undeniable conclusion:

We are not very good at hearing from God.

Maybe the reason is that there is no God, or maybe just no Christian God. Or maybe the Christian God exists, and the problem is with us. When we arrived in Boston, we set aside one month for God, each day praying, "God, if you're real, undeniably reveal yourself to us." Since we are very bad at hearing from God, if we are to have a relationship with God, the effort must be his. After one last month, we still experienced nothing. And at that point I came across one truth, written in the Bible, that forever shattered my faith.

If you're not familiar with the Old Testament, one of the most common themes is the warning against idolatry, the worship of false gods through graven images rather than the one, true God, the "I am". The Hebrew scriptures are littered with hundreds of these warnings, but the best story is from 1 Kings 18, where Elijah confronts the prophets if Baal. Here is an excerpt:

Elijah went before the people and said, "How long will you waver between two opinions? If the Lord is God, follow him; but if Baal is God, follow him." But the people said nothing. Then Elijah said to them, "I am the only one of the Lord's prophets left, but Baal has four hundred and fifty prophets. Get two bulls for us. Let them choose one for themselves, and let them cut it into pieces and put it on the wood but not set fire to it. I will prepare the other bull and put it on the wood but not set fire to it. Then you call on the name of your god, and I will call on the name of the Lord. The god who answers by fire—he is God."

Naturally the prophets of the false god Baal have no success, but Elijah has success:

At the time of sacrifice, the prophet Elijah stepped forward and prayed: "O Lord, God of Abraham, Isaac and Israel, let it be known today that you are God in Israel and that I am your servant and have done all these things at your command. Answer me, O Lord, answer me, so these people will know that you, O Lord, are God, and that you are turning their hearts back again." Then the fire of the Lord fell and burned up the sacrifice, the wood, the stones and the soil, and also licked up the water in the trench.

The basic argument is always the same. What separates idols from God was the success rate. You should worship God because God is real and will provide for you. A true God produces results and a false God does not. After all, what purpose does God have if he can't make an impact on you? When I remembered this, I could not escape its conclusion in my life:

God is my idol.

I could not deny the promises that hadn't come to pass, and those promises that had were easily explained by things other than God. At the very least, the Bible I believed in condemned my God as an idol, since it also did not produce results. I had peeked behind the curtain, expecting to see the magnificence of God, and instead encountered a mirror. All this time, I was the man behind the curtain. The things I thought I heard from God in prayers were just my own desires and fears. I am not good at being God.

To continue worshiping this God would be worshiping an idol, something the religion condemns as an obviously pointless and worthless activity, but it was the only God I had ever known. From that point on, I knew I could no longer be a Christian. I realized Christianity did not have practical applications in my life. But that's different from saying Christianity is ideologically false. Next week I'll explain how I concluded the core ideology of Christianity was flawed, and how I went from an outcast of Christianity to living in the freedom of heathenism.

In the Cleft of the Rock

2008-04-28T06:00:00.000-07:00

As I mentioned earlier, I was a devout Christian when I started programming BrainWorks, and now that I've finished, I am an agnostic with no particular religious leaning. My work in artificial intelligence was not the only reason I gave up my religious faith, but it is part of the reason. Moreover, it is a story worth telling, and I hope it will help both Christians and non-Christians gain a greater understanding of each other, something that is sorely lacking in this world where rational justification often takes a back seat to dogmatic conclusions.

To understand the process of giving up my faith, however, you must understand the faith I had. Many people who call themselves Christians, perhaps most, don't have much to do with the actual tenets of the religion. To the average Christian, being a Christian means that God and Jesus love you, and Jesus died for your sins so you that when you die, you go to Heaven instead of Hell. In other words, Christianity is a nice feeling in your heart and your insurance policy for when you die. It has little impact on your actual life, except for a general imperative to "do good things", which many Christians ignore, or act as if it only applies to other Christians.

I was never this kind of Christian.

My core belief has always been in causality, and by extension rationality. I have always believed that if something is true, you can trust in its consequences to be true as well. So I believed in the core tenet of Christianity, which is:

Jesus died to pay for my sins and rose from the dead so that I could receive God's spirit and thereby have a loving, personal relationship with the God in this life, and be with God in heaven after I die.

And I believed everything that logically follows if this is true. So yes, I believed that God can and does do miracles, even in this day and age. Real miracles too, not "I saw the face of Jesus in my pop tart". Stuff like "I was born blind and now I can see". I believed that people can and did hear from God, and that if you pray to God, he hears you as well.

Note that this is fundamentally different from saying that the Bible is the 100% true, infallible word of God. There are some clear logical inconsistencies, and I just wrote that off as "I guess they didn't hear correctly from God about that part", or "someone corrupted this text for their own purposes". The most glaring example is the book of Daniel, which purports to be written during the Babylonian captivity. However, the book contains words whose linguistic roots trace back to Persian empire, a culture the Israelites didn't encounter until well after Daniel died.

There were also some religious statements that don't logically make sense. The traditional explanation for these things, such as "don't eat pork", is that they apply to older cultures for some reason but don't apply to the culture of today. However, I took the stance that maybe people just didn't hear correctly from God and the Bible's authors recorded the wrong thing.

For example, even as a Christian I did not think homosexuality was immoral. There is no logical reason that consenting sex between people of the same gender would be wrong but between opposite genders would be okay. "God hates it for some reason" isn't a good argument. According to the book of Jeremiah (Jeremiah 7:19), things that are wrong because they are detrimental to humans. There is certainly nothing humans can do to injure God! Our actions can only harm humans, so if an action does not hurt anyone, it cannot be immoral. Consensual sex falls into this category regardless of gender.

So as a Christian I had no problem admitting that the Bible flawed and even wrong about some things. I disagreed with some very popular Christian opinions, and not just regarding homosexuality. But I still considered myself a Christian, because you can still believe the Bible is wrong about some things (bacon, homosexuality) and right about other things (Jesus died so God can have a relationship with me). I still believed that I could hear from God, talk to God, and witness the supernatural miracles of God.

But around one year ago everything changed, for two very different reasons. One reason convinced the pragmatist in me and the other convinced the idealist, and the two arguments together made me undeniably admit that I had been very, very wrong for the past 30 years of my life.

Please believe me when I say that I was not looking for a reason to leave my religious faith. As a Christian, I felt totally in love with God. To this day, I have never felt more joy in my life than during a Sunday worship service, singing hymns and praises. Being confronted with the irrefutable reality that I was wrong about God was agonizing and heart wrenching, and met with many tears. For months I felt devastated, and I still wonder if I will ever find something that brings me as much joy. But I had no other choice. I do not worship God; I worship truth, and by extension reality. If the truth is that there is no God, or at least no Christian God, then my only option is to act on that truth.

I am confident that most atheists and agnostics do not have the faintest idea how much safety and security a devout Christian gets from their religion. From the agnostic point of view, it's easy to think of Christians as weak minded for taking so much on faith and so little on reason. If a Christian doesn't change their mind when presented with solid reason, the conclusion seems natural. But it's difficult to comprehend the amount of mental and emotional security that even a religion brings, even a false one. The sheer fear of being wrong is enough to make most religious people flat out ignore solid arguments to the contrary.

I'm glad I took the red pill, but the blue pill would undoubtedly have been less painful. It is my hope that godless people would have compassion on those who still have religion, and understand that even though many may fear they are wrong, they lack the emotional strength to take the large steps that seem easy for us.

Parallel Evolution

2008-04-21T06:00:00.000-07:00

There is a biological concept called parallel evolution which refers to the phenomenon by which the same trait is evolves in the same manner in two different species. For example, all eyes came from the same base creature. But eventually marine and vertebrate animals both evolved the same set of eye advancements, despite lacking a common ancestor that had all those advancements. There are also cultural examples of parallel evolution for concepts like writing and the wheel. In each case, these concepts fit three basic criteria:

Very useful when obtained
Not completely obvious
Little to no room for implementation differences

The key point is the last one. There's just one way to do a working wheel or for writing to work, so in that respect it's not surprising that the ancestors of the Mayans and the ancestors of Chinese came up with the similar concept of putting symbols on surfaces to record concepts. (And yes, the wheel is not a completely obvious concept, because it also requires the design of an axle.)

So I often wonder what an alien life form would be like, evolved totally independent of life on Earth. What concepts are there about life that really only have one good solution? For example, the structure of the carbon atom makes it highly likely that all life would be carbon based. It's theoretically possible to have silicon based life, because carbon and silicon have similar structures (both have four possible atomic links), although silicon is much heavier, but carbon is by far the most likely. Also, water is a fascinating molecule in liquid form, used for a variety of purposes in life, so it would be very surprising to see a life form that did not use water.

All of life exists somewhere in the density spectrum between water (liquid) and carbon (solid). Some are closer to the water side, like a jellyfish. And some life forms are closer to solids, like trees. Land based animals ("fleshlings") are in the middle, and seem to offer a good mixture between the mobility of a jellyfish and a protective shell of a tree. So it seems likely that any sentient life form will be made of some kind of flesh.

So while we can guess about the physical form of another life form, deducing the nature of an alien brain is much harder. That's primarily because we don't know enough about the human brain to guess which features only have one simple solution. We've only encountered one kind of sentient life: humans ourselves. Would aliens need to sleep? Our research into sleep implies our brains use sleep to process memories, similar to defragmenting a hard disk. Important memories are stored, unimportant ones are discarded. But is our brain's implementation of memory the only way to solve this problem?

Well, computers solve the storage of data-- thoughts and memories-- through completely different means. Human thoughts are fluid. People can easily forget things that happened five seconds ago, or even remember the details incorrectly. Even photographic memory isn't perfect, and seems to decay with age. Short of hardware errors, however, a computer's memory management is perfect. All the data can be retrieved without error. The human brain seems to sacrifice accuracy for much greater storage capacity, access speed, and perhaps processing speed. It's possible there exists an alien life form that has found a solution to the memory problem that incorporates human memory capacity with computer memory accuracy, but I doubt it.

So this line of thought makes me think two things about the field of AI research. The first is that it's crucial to identify the portions of intelligence that only have one good solution and solve them first. For video game AI, these are problems like navigation and visualization. At this point everyone knows the navigation solution is A*, but there's still the question of how the mind identifies the potential way points A* requires. For general AI, the most fundamental problem is language processing, which is devilishly difficult and could take centuries to solve. But having a library of known solutions to basic AI problems will accelerate the ability to create good AI. This is similar to how an operating system abstracts away simple solution to problems a programmer doesn't want to worry about very much, such as processor scheduling and disk space access.

The second thought I have is this nagging feeling we may be designing AI on a fundamentally flawed hardware platform. The computer is excellent when what you want is ultimate precision. Designing AI on a computer involves writing sophisticated algorithms to artificially create the "randomness" that real life seems to incorporate. Perhaps one reason artificial intelligence is so hard is that sentient life requires a system of memories that trades precision for increased data capacity and faster access time the way a human brain does, and we'll never create satisfying AI until we start programming on that kind of a hardware platform. I don't know if that hardware platform is still based on transistors or if it's something more like DNA, but deep down I feel like designing AI on a computers is pushing a square peg into a round hole.

BrainWorks 1.0.1 Released

2008-04-14T06:00:00.000-07:00

I've just released the latest update to BrainWorks implementing the fixes I talked about a few weeks ago. Basically there's a much more sophisticated algorithm for tracking and estimating how likely a bot is to shoot a weapon in a given situation. You can download them using the links on the right if you're interested in trying it out. Let me know what you think!

Related to this, I've been thinking about the question of when software is done. In some sense I still stand by my answer in The Attitude of the Knife, which is "when you say it is". The flip side is that as long as you still have ideas and commitment, there's always room for continual improvement. For a simple program meant to meet a specific purpose, such as reading your email, there comes a point where there isn't much more room for feature development. Artificial intelligence isn't like that though. To this day, researchers are still trying to figure out how different areas of the human brain work. And contrary to popular opinion, humans are not the end of evolution. The human brain itself continues to refine and advance itself through generations.

I believe the line dividing things that can be finished and things that cannot is the line of self reference. When the problem is best solved by something that can analyze and correct its own mistakes, a whole new field of issues apply. For an in depth explanation of why this is, I highly recommend the second of three books that influenced my mental framework for understanding the world. That book is Godel, Escher, Bach: The Eternal Golden Braid, and it is about the nature of intelligence. Very roughly paraphrased, the book talks about the mathematical theorem known as Godel's Incompleteness Theorem, which says that any system capable of describing itself can describe statements that are true but unprovable. Originally the theorem was discovered in attempts to work out some issues in Principia Mathematica, an attempt to derive all mathematical truths from first principles. However, the incompleteness theorem sheds unanticipated insights on the areas of philosophy and by extension the nature of thought and intelligence.

Viewed in the context of intelligence, you could conclude that there are things which an intelligent person would do, but there is no describable algorithm that could conclude what those things are. Perhaps this represents acts of creativity, intuition and insight. Or perhaps those things are describable, but other things are not.

Applied to Artificial Intelligence, it means that there are some aspects of AI that we cannot solve; we can only approximate. And there's always room for better approximations. This is the real that AI development can last forever. You're never really done. At least not until you say you are.

Getting it "Just Right"

2008-04-07T06:00:00.000-07:00

There's not much difference between driving 55 and 56. Unless of course the speed limit is 55, in which case that small amount of speed could make you get a speeding ticket. Standard outside temperatures average between 0 and 100 degrees Fahrenheit, but people easily notice the difference between 68 and 72 degrees. And yet for other measurements in our lives, small changes are completely unnoticed. A car with a full gas tank drives just as well as a car with a quarter tank left-- you only notice the problem when the last drop of gas is gone.

When you design artificial intelligence, everything needs to be broken down to numbers, since a computer can only understand numbers. And a lot of times you have no idea what numbers best simulate the behavior you want to elicit from the AI, or even if the number encodes the correct concept.

Think of the problem this way. Suppose I have a value that encodes the maximum error a bot can make while attempting to aim. If the AI doesn't behavior properly with the initial value I selected, I'll want to try other values. By trying different values, I'll rapidly find out whether this number is very sensitive to changes (like temperature) or insensitive (like a gas tank). But if the bot doesn't seem to be doing the right thing, there's still no way to know what the right value is.

A sensitive number is extremely hard to tweak, because you won't see the right behavior until you get things "just right". If the value doesn't encode the right concept, then that "just right" state won't exist, but you'll never know that. You'll just see how all kinds of different values don't work in different ways.

And an insensitive number generally just has an impact when it crosses an important boundary (for example, driving 56 instead of 55, or your gas tank being 0% full instead of 1% full). There's often no indication where this interesting numerical boundary might be.

Additionally, values can depend on each other. Perhaps changing this maximum error value makes some other value be sensitive, when before that value was insensitive (and thus not aggressively tweaked).

All this to say that when you write AI, there's an awful lot of number tweaking going on. Either you do it by simulations, by neural network training, genetic algorithms, or by hand. But one way or another, you'll have dozens of numbers that all need to be "just right" to present the illusion of intelligence. The good news is that once you've spend time finding these values, you'll have excellent insight into how humans approach the problem. That's how research works, I guess.

For example, do you know what value is the single biggest factor in determining a player's aiming accuracy? It's how fast the player moves the mouse, not the error in mouse movement. The most important factor is the maximum allowed acceleration of the bot's simulated mouse. I know this because of all the values that go into aiming (and there's close to a dozen of them), the acceleration factor by far has the largest impact on actual bot accuracy. There are some very good reasons why that is the case, which are a bit to complicated to explain right now. But simply change the the maximum acceleration for high skill bots from 1400 to 1600 and you'll see an immediate increase in actual bot performance. This value was one of the last values I decreased before release, actually, so BrainWorks bots didn't have the same ungodly aim that traditional Quake 3 bots have.

At any rate, getting artificial intelligence to feel "just right" really comes down to finding the appropriate things to measure and the values those measurements should have. There's no right way to do this, and certainly no magic solution. BrainWorks uses statistical modeling for some things (weapon accuracy) and fixed numbers derived from simulations for others (aiming). Seeing how much the final result can change from just minor alterations in one number certainly gives an appreciation for the complexity that goes into genuine intelligence. If changing just one of twelve values has a dramatic impact on how skilled a bot appears, try to imagine the complexity of millions of neurons in your brain. And yet they all operate together in a (mostly) cohesive unit. It boggles the mind.

Parenting

2008-03-31T06:00:00.000-07:00

Writing artificial intelligence is a lot like being a parent. It requires an unbelievable amount of work. There are utterly frustrating times where your children (or bots) do completely stupid things and you just can't figure out what they were thinking. And there are other times they act brilliantly, and all the effort feels satisfying and well spent.

Fascinatingly enough, people rarely ask the question of whether being a parent is worth the effort. There's an implicit belief that once you grow up and get married, you should have children. It's like the option of not having children doesn't exist. And of course once you do have children, you must do everything in your power to raise them as well as you can. Questions of whether becoming a parent is good idea, or how much time should be invested in children rather than your spouse are discouraged if they are even asked.

Now I'm not suggesting that parents shouldn't spend a lot of time parenting. Once you've committed the next 18 years of your life by having a child, it seems like you should live up to the responsibility you create for yourself. I'm just wary of people claiming there's only one answer when they aren't even asking the question. Doing the right thing for the wrong reason might cause you to do wrong thing later, as situations change. A good example from parenting would be an overt amount of hand-holding once your child goes off to college. If you always believe you should give 100% to help your kids no matter what, you might end up doing their laundry and cleaning their room when they turn 25. Certainly the purpose of parenting is to turn children into adults, and at some point good parenting involves letting your child being an adult. So the core question of, "Is it worth the trouble to have children?" is a real question, and the answer isn't always yes, although many people assume it is.

What seems strange to me is how people answer the related question, "Is it worth the trouble to program good AI?" And they almost universally say, "No". There are countless games where the AI overtly cheats to win-- pretty much every real time strategy game. Almost every first person shooter to date has had massive problems navigating around a level, Quake 3 included. Most squad-based AI is exploitable by any player who has encountered it a reasonable number of times. And the solution of heavier scripting might make AI seem more realistic when first encountered, but it's far less realistic every other time. Companies resoundingly see little financial benefit in creating genuinely good AI.

I find these answers very much at odds with each other. Writing good AI requires more intelligence than raising a child well, but certainly less time. Moreover, good scientific research results are never lost. They can built upon for centuries. They still teach Newtonian mechanics in colleges, even though we know Newton was technically incorrect (but close enough for most practical purposes). There is a lot of potential long term value in designing good AI, even if the short term profits aren't there. Similarly, the cost of raising a child these days is easily over $100,000. That's a huge short term investment that, quite frankly, never pays off for a lot of children in long term societal benefit.

The real conclusion is that economic and scientific valuation have almost nothing to do with the actual choices people make. When someone says, "Is it worth it to raise children?" or a company asks "Is it worth it to make good AI?" they are using different definitions of worth. The company wants to make money but the potential parent wants to enjoy life.

People don't do what makes them the most money. People do what they love!

Why are there over six billion people in this world? Because people are genetically programmed to love having children. The existence of hormones that encode these feelings of love doesn't make the love any less genuine. Children require effort, but the love for children overrides the dislike of effort (for most people, at least). Without the love of children coded into the DNA of humanity, good parents might become as rare as AI programmers. There's not a lot of people who love designing AI.

Of all the things I have learned designing AI, the most important is this:

Find what you love doing, then do it.

That said, there are surely people who would love life more if they didn't have children. And there are companies that make money by writing good AI. I would love to see more people enjoying life, even if that means going against societal trends and not having children. I would also love to see more companies making money by writing good AI. Not every company can or should do that, but I believe there's room for advancement in both areas.

My Biggest Programming Fear

2008-03-24T06:00:00.000-07:00

I have encountered some very strange bugs in my programming career. And most of them have been in BrainWorks, being by far the most complicated piece of software I've ever written. One of the worst involved a build that only crashed when run on someone else's system. On my system it was fine, but it would crash on my friend's system. It wouldn't crash right away, mind you. Sometimes it would take up to two minutes, so even testing to see if the bug was fixed took time. Oh, and his computer was in London while mine was in Los Angeles. Not exactly the easiest problem to solve.

I solved it by compiling in debug flags that would turn on or off entire sections of code and then had him run it. "Okay, load bots but turn off all movement, scanning, and aiming. Does it crash now? What about when Item Pickup is turned off?" Through the course of four builds that gradually narrowed in on particular blocks of code, we eventually found the offending bug.

(If you're curious what caused it, an uninitialized value was improperly being treated as an address, crashing the program whenever it was accessed. Because his machine's memory layout was different from mine, it would occasionally access invalid memory and the operating system would kill the program. For whatever reason, the uninitialized values that showed up on my system always happened to refer to memory BrainWorks was allowed to access, so it never crashed for me.)

It took about eight hours to do, but once you've solved a bug like that, you feel cabable of tackling any bug. Bugs bother me, but they don't frighten me. Know what my biggest programming fear is?

0 total errors

Any time I've spent over two hours working on a particular feature and attempt to rebuild the code base, I expect to see at least one error. I'm a good programmer, but that doesn't mean I'm perfect. I'm a good programmer because I know I'm not perfect. Good programmers assume they will make mistakes-- that's why they add all those safety error checks. If some other piece of code does the wrong thing, their error checks will contain it.

Last week I spent four hours tracking down the issues with bots overvaluing the shotgun. The problem was that the estimation of fire frequency wasn't precise enough. I spent another three hours analyzing the similarities between estimating the chance of hitting with a weapon (hits divided by attacks) and the chance of firing a weapon (attacks divided by potential attacks) and designing functionality that merged the two concepts. Actually writing the code took another 2 hours. Nine hours total including changes that could totally break the AI's ability to even attack, and here I am looking at the results from my very first recompile:

0 total errors

I haven't run it yet, but I'm terrified. Zero errors on the first try? That never happens. That's not even supposed to happen. Odds are the code I wrote has an error somewhere; I just don't know where.

At any rate, I plan to test it this upcoming week and get a new release out this weekend. Hope you enjoy it.

Release Retrospective

2008-03-17T06:00:00.000-07:00

While people have been overall pleased and impressed with the BrainWorks release, I've received one common bit of negative feedback. Many people have said the bots use the shotgun too often, especially in situations where it's flat out the wrong weapon to use. Now I don't have the luxury other people have of saying, "that's not a bug, that's a feature!" Or more commonly, "that's working as intended." BrainWorks is intended to feel realistic. If many people say a portion of it doesn't feel realistic, they are right. I have no grounds on which to disagree. There really is a bug in the code because you say there is.

Of course, if you've ever been told quite literally, "this thing doesn't feel right, so go fix it," you know how challenging of a request that is. One downside of fundamentally basing the BrainWorks AI on causality is that debugging is extremely difficult. There's no magic number that tells the bot how often to use the shotgun. Instead, the bot analyzes its situation and incorrectly decides the shotgun is the right weapon to use. To solve this problem, I need to walk through the entire analysis and see how it reaches that conclusion. To make matters worse, the bots don't always choose to use the shotgun. And there are surely situations where the bot correctly chooses the shotgun. It's very hard to isolate the situations where the mistake as made and then narrow down why that mistake occurred.

My solution was to write some debug code that outputted all the data the bot used at each step of weapon selection analysis, starting from accuracy and fire rate data and ending with its final valuation of how quickly a given weapon would score a kill. I had it output this data for all weapons available and just stared at the numbers, checking if each data point was an accurate value of the concept representing it. I found three key contributing issues, two of which are solvable.

#1) The estimation of weapon fire rate was incorrectly bounded

Knowing how frequently the bot will fire a weapon is crucial to determining the actual damage rate of the weapon. If a bot spends 3 seconds aiming at a target and only fires for 1 of those seconds, it will do one third the damage as a bot that spends all 3 seconds aiming-- provided it's always lined up for a good shot. The bot tracks how often it actually fires the weapon, but I gave this value a lower bound of 50%. In other words, the bot assumes it will spend at least 50% of its time firing with the weapon, possibly more.

I added this assumption to deal with another problem. Bots would switch to a weapon they had never fired before, not have a perfectly lined up shot, and think... "I've spent 0 seconds firing and 100 milliseconds aiming, so I spend 0% of my time attacking. This weapon is terrible. I'm going to use something else." And they would never try out the weapon to see that they could in fact make shots with it. One problem with relying on historical data is wide variance in the initial data sets. I ran some tests with most of the weapons and found the fire rates were in the 60% to 80% range, so 50% seemed like a good bound.

The problem was that the shotgun only had a fire rate of between 30% and 50%, meaning it's hard to line up good shots with the weapon. There were situations where the bot would think it fired 50% of the time when in reality it had only attacked 30% of the time, which inflated the estimated value of the shotgun by a solid 60%. (160% of 30 is 50.) So it's no wonder they thought the weapon was good. Lowering the minimum bound to 30% had other problems though. When that happened, sometimes bots would stop using genuinely good weapons like the rocket launcher. That's a sign that a lower bound is not the correct way to solve this problem.

Related to this is the second issue:

#2) The estimation of weapon fire rate doesn't take location into account

If you're wondering why the shotgun is generally a bad weapon, it's because it's so situational. At point blank range, it can do more damage than the railgun in two thirds the time. But the spread on the weapon makes its value decrease rapidly at medium and long range. There are situations where the shotgun is good, but not many of them. In contrast, weapons like the railgun are consistently good in a wider variety of situations.

This means that a bot might shoot more often with the shotgun at point blank range than at long range because the shots are easier to line up. If the bot had a 40% fire rate with the shotgun, that might be a 50% rate when close to the enemy but only 30% when far away. Similarly, it's easy to line up shots with the rocket launcher when the target is below you, since you can just shoot at the floor. If the target is above you, you need a direct hit, and that's very hard.

If the bot knew how the firing rate could change depending on the combat situation, it would have a much better understanding of whether a weapon is a good choice against the current target. Unfortunately, the code only tracks a single firing rate for each weapon. The concept of where the enemy was located doesn't factor into the data, and that's a real issue.

The drawback of tracking fire rate data across each of the bot's 12 combat zones is that it will take much longer to get good estimates on actual fire rates. If there were problems before with bots accumulating fire rate data into one data "bucket", now that there are 12 "buckets", it will take 12 times longer before the data stabilizes. In other words, the issue that the 50% minimum fire rate was trying to address has now gotten 12 times worse.

I'm still thinking about the best way to solve this problem, but I'm leaning towards seeding the data with some reasonable estimates of how often the weapons really should be firing. If there's enough seed data, it should encourage the bot to act reasonably until it has enough of it's own data to make conclusions. That just leaves one more issue:

#3) Bots do not take into account how a good choice now could be bad later

The major drawback a situational weapon like the shotgun has is that when you use it, you give your opponent some control. They have the power to make your weapon worse just by backing up. While this is a concern for all weapons, the more situational a weapon is, the worse it is when your opponent exploits your weapon's weakness. In other words, the shotgun isn't just bad because it's only useful at close range. It's bad because your opponent can choose to be at medium or far range after you spend the time to pull out the weapon.

This is unfortunately a much harder problem to solve, since it has to do with the local level geometry. If your opponent has no escape routes, the shotgun is still excellent in close quarters. I haven't thought about this problem very much, but my intuition says that solving it in BrainWorks is well beyond the scope of the project. You might be able to analyze past reactions opponents had to your weapon choices, but it's not clear how good this data would be, how it would affect the bot's choices, and if this problem is even large enough that it needs such a sophisticated solution.

At any rate, I apologize if this post is a bit more technical than normal, but that's the nature of the work. I'm very interested to hear your ideas for how to tackle these problems. I plan on thinking through possible solutions over this upcoming week and writing the fixes next weekend. If you have thoughts on this, I'd love to hear them.

Turn Left at the Quad Damage

2008-03-10T06:00:00.000-07:00

Have you ever gotten directions that included phrases like, "When you see the giant viking statue, go straight and curve left" or "If you pass a third Starbucks, you've gone too far"? People don't always give directions based on street names; sometimes the street names are largely irrelevant. Here in New England, if the street sign is even posted at all, it might not be readable until too late. Directions like "turn left at the Rite-Way Mart" are often easier to follow than "turn left at old route 293".

Believe it or not, bots have the same problem. We might conceptualize a level as rooms, hallways, and open areas, but these places have no names. Humans generally name these areas after a recognizable landmark. In Quake 3, that usually means items. You'd never tell a teammate, "I'm at (540, -973, 108)", which is how the computer records your position. But you would say, "I'm near the rocket launcher".

One requirement of item pickup that I glossed over in a previous post was this: Bots need to know how likely they are to encounter an enemy in a general area. That implies bots need a concept of what an area is. They need to think of enemies being "near the rocket launcher" not "at (540, -973, 108)".

Bots need to think of enemy location in human terms, not computer terms.

The obvious method of approaching this problem is to divide the level into regions, with each region being defined by one item. Technically each region is "the area of space that is closer to this item than any other item". Once you have the level conceptually divided into "near the red armor" and "near the plasma gun", whenever the bot sees an enemy it can say, "Ah, that player is near the red armor, not the plasma gun. I'm more likely to find enemies if I head for the red armor as well." Now whether or not it wants to enemies is a different question, one answered by item pickup. But at least it can track meaningful data about what other players on the level are likely to do.

However, just because it's possible to find out what item a player is nearest doesn't mean it's particularly fast or easy for the computer to determine that. The AI obviously can't spend half a second to calculate data it needs to save 10 times a second. It's okay to spend a bunch of time when the level loads to calculate the best set of regions, but once these calculations are done, bots need to do the location to region calculation very quickly.

I researched a number of different options, but eventually settled on the octree, briefly mentioned in a previous post. Conceptually an octree is a way to layout some number of points in 3-space. In BrainWorks, the item region octree contains the location of all items on the level. Rather than rewrite what has already been written on the subject, I recommend reading the wikipedia article on quadtrees (the two dimensional equivalent of octrees).

For the 99% of readers who don't care how to implement an octree, I'll just say that a well balanced octree will generally let you find which item is closest to any input location very quickly. The run time is O(log₈ N) time, where N is the number of items on the level. That means that finding the nearest item on a level with 64 items will take twice as long as a level with 8 items, and a level with 512 items will take 3 times as long as a level with 8 items. Since even the largest levels rarely have more than 100 items, this calculation is really fast, and that in turn frees up more processor time to think about different AI problems. The bot makes a richer impression to the human playing the game and hopefully the player has more fun too. Sometimes making a game fun to play involves implementing mind-numbingly boring data structures, but the fun just wouldn't be there if they were missing.

Good Enough For Government Work

2008-03-03T06:00:00.000-08:00

This might be a surprise if you've read my post on pragmatism, The Attitude of the Knife, but I'm actually an idealist. I don't want things to be good enough; I want them to be perfect, or at least as perfect as pragmatically possible. I use the attitude of the knife to temper my idealism.

Of course, there is tension between idealism and pragmatism when encountering very difficult problems. I don't just mean, "you'll have to be really smart to solve this problem." I mean, "it's mathematically impossible to solve this problem using the computer you have access to." The idealist wants 100% and the pragmatist wants 99% or even just 90%.

The classic example of a mathematically difficult problem is the Traveling Salesman Problem. If you're not familiar with the problem, it works like this: Suppose you have a salesman who wants to visit every city in a certain area exactly once, and he knows the distance between each pair of cities. What's the minimum distance he'll have to travel to visit all the cities and how do you determine it?

Mathematicians have been studying this problem for almost 200 years now. It turns out that determining the perfect solution is really, really hard, but it's possible. Worst case you can just check every possible ordering of cities. That worst case is unbelievably bad, however. If the salesman had 30 cities to visit and a computer could test one trillion options per second (current no single computer exists that is this fast), testing all these options would take much longer than the current age of the universe. Most of the studying mathematicians have done on this problem revolves around ways to get a "pretty good" answer without spending billions of years to do so.

I say I'm an idealist, which means I want things to be perfect. But I'm also ruthlessly pragmatic. These contrasting ideals intersect on the definition of perfect. If it takes 1 trillion years to find the traveling salesman route that takes 38 days but 1 hour to find the route that takes 42 days, then the 38 day solution isn't perfect. It's really 1 trillion years plus 38 days, which is definitively worse than 42 days, 1 hour. A "perfect" solution must take into account the constraints surrounding the problem and not just the problem itself. The mathematically optimal solution is not the perfect solution expressly because it's not practical enough to use.

What does this have to do with Artificial Intelligence, and specifically item pickup? Deciding what items to pickup is very similar to solving the traveling salesman problem, and that's bad news for me as the AI designer! If there are 20 items on a level, a bot wants to pickup between 0 and 20 different items on the way to its final destination. That's similar to a traveling salesman wanting to visit exactly 20 different cities. And since BrainWorks bots can't spend several trillion years to decide what items they're going to pickup, their item pickup code cuts a lot of corners to get a pretty good solution in a reasonable amount of time. This isn't the mathematically optimal solution, but it is a better choice overall.

Here are some of the tricks BrainWorks uses to reduce the computation time:

Nearby items are grouped into a single cluster. Bots consider picking up the entire cluster at once. This reduces the effective number of things to consider.
Bots do not consider picking up more than four items at once before going to their final goal.
Bots only consider picking up the dozen or so items that are relatively near their current position, or not far off the path to their destination.
When the bot is very near an item (less than a second away), it automatically picks it up rather than doing a full computation.

These changes have a profound effect on the size of the search. Suppose a level with 40 items gets bound into 25 clusters and the bots never consider more than 4 total pickups from the 12 nearest items. Each item pickup decision require testing at most 800 options, rather than more possibilities than there are people on planet Earth.

It's rare that thinking 10 item pickups ahead would really help the bot more than the first two or three choices. And usually the bot doesn't want to travel halfway across the map to pickup some random item. Only the nearby items are worth considering. For the one or two items that really might be that good, the bot specifically notes these items and includes them in the list of possible pickups, no matter how far away they might be. And as it turns out, these results are still "very good". Often the theoretically best solution isn't the best for the problem at hand. If you idealize the results and not the method, being idealistic involves selecting the most pragmatic solution.

At the Heart of it All

2008-02-25T06:00:00.000-08:00

As I outlined last week, the item pickup problem in First Person Shooter games is very difficult to solve, and that goes for humans as well. How do the truly advanced players approach this problem? At the heart of it all is one central question:

How will this benefit me?

If you want to know whether to pickup item A or B, you first must know how much item A benefits you, and how much it costs (in additional time to get the item). BrainWorks solves this problem using a customized resource prediction model, handled by ai_resource.c. Be warned, this code is not for the feign of heart. It's gratuitously well commented and documented, but there are so many bits and pieces it's very hard to get your head around everything at once. Think of it like opening the hood of your car, removing the engine block, and taking everything down to the nuts and bolts. Very interesting, but good luck assembling it into a working vehicle.

First, the AI's resource engine must encode the player's current state. This is handled by the resource_state_t structure (defined in ai_main.h). In brief, that information is:

Health
Armor
Weapons
Ammo per weapon
Powerups (eg. Haste, Quad Damage)
Activatable items (eg. Portable Teleporter)

It's easy to predict the results of picking up a specific item. If you want to imagine what happens when the bot grabs a rocket launcher, you'd just change the player state to have a rocket launcher and some rockets for ammo.

Determining how many points a given resource state is worth is significantly more complicated. Why is that? What other information does the bot need to estimate the points it would gain if it had this set of items? In short, it needs information about possible encounters with enemies:

Chance of encountering an enemy near the bot
How much damage the bot will deal with each weapon
How much damage is required to score a kill
How much damage an attacking enemy will deal to the bot

Once the bot has this information, it can estimate the points gained for any duration of time. The encounter rate can (and does) change based on where the bot is, of course. There will be higher traffic near popular items. As a result, the bot needs to remember when it does and doesn't see other players in different regions of the level. This is actually a complicated enough problem that I'll do a full post explaining how it works another day. For now, assume the bot magically knows how likely it is to find an enemy.

The three damage values should be independent of any changes to the resource states. The amount of damage a weapon deals depends on the bot's accuracy-- getting more ammo will let the bot deal that damage longer, but it won't ever make one weapon deal more damage per second than another. As a result, the bot sorts every possible weapon it could use in a priority list at the very start of prediction. The list might read something like, "Use the plasma gun until out of ammo, then the rocket launcher, then the shotgun, then ..." Now that might not be what the bot actually does when it gets in combat, but it uses this as a rough estimate of what might happen.

Naturally the estimated damage each weapon will deal is computed from the bot's historical data of previous weapon use. If in the past the bot had 30% machinegun accuracy and fired 90% of the time the machinegun was loaded, the machinegun's effective damage rate is 27% of it's maximum possible damage rate.

So when the bot needs to predict 20 seconds of time, it might say, "Well I'll get this many points from firing my rocket launcher for 8 seconds, until it's out of ammo. Then the next weapon is a railgun, which I shoot for the remaining 12 seconds and get this many extra points." There would be 2 distinct calculations in that situation. The code uses a similar system for powerups wearing off. In that example, if the bot had a quad damage that wore off after 15 seconds, it would break the time into three segments (0 to 8, 8 to 15, 15 to 20).

Of course this is all well and good if the bot survives, but what happens if the bot predicts its own death? Surely there must be a penalty if the bot needs health but considers options other than getting it. When the bot thinks its health will reach zero, the resource prediction engine stops giving points, and the bot takes a (1 / opponents) point penalty, since getting killed gives one opponent a point.

All that work just to figure out how many points the bot will gain, not even to select the best item for pickup! Next week I'll explain how exactly the item pickup algorithm uses this resource prediction engine to determine the best item to pickup.

A Journey of One Thousand Miles

2008-02-18T09:00:00.000-08:00

There is a very old Chinese quotation that goes, "A journey of one thousand miles begins with a single step." More literally it reads, "A journey of one thousand miles begins beneath one's feet." Wherever you go, the first stage in any journey is the place you are in now. This might seem trite, but it's crucial, and all kinds of problems arise if you ignore it.

For example, suppose you take a cross-country trip from Chicago to Los Angeles:

View Larger Map

The basic plan is to take Interstate Highways 80, 76, 70, and 15 to cover most of the 2000 miles. But none of these directions are actually useful unless you can get on I-80 in the first place! If you zoom in on Chicago on the map, you'll see the directions start from outside the Chicago city hall. You'd use a totally different route to get on the highway if you started on the south side of Chicago or one of the suburbs. Understanding your current location is crucial to route planning.

Now last week I promised this post was about item pickup, not navigation. So what does route planning have to do with item pickup? Determining what items to pickup is a kind of of route planning. If a bot wants to go to a different area of the level, there might be three or four different routes it could take to get there, each with different items available on the way. There may be a good reason to take a longer route that has better items. Items define way points between where the bot's current location and where it wants to end up.

The item pickup algorithm determines which items a bot should pick up while it goes somewhere else.

That might seem simple, but as best I can tell, it's a fundamental change in how item pickup AI has been done in every other first person shooter to date. Typically the problem is viewed as finding where the bot wants to go, and choices are picking up items, hunting down enemies, or guarding a specific location. I've approached the item pickup as what you do while you are also hunting down enemies or guarding a location.

There is one crucial difference between item pickup and traditional route planning problems, however. If I'm planning a cross country trip, I'm either trying to minimize travel time or travel distance. Item pickup isn't interested in either of these things. Why do players pick up items? To get more points in the game. Bots need to select the items to pickup between points A and B that will maximize their score, not minimize travel time. That's why it's often correct to take a longer route with better items. The better items will increase the player's score more than showing up at the destination completely unarmed. The item pickup algorithm I've implemented is completely focused on one thing: estimating the score gained from taking different items.

And because the journey of one thousand miles begins with the ground beneath your feet, bots need to know how effective they are at earning points in their current state (health, armor, weapons and ammo values). Then they estimate the value of a particular item by imagining how that item would modify their state and how that would change their rate of acquiring points. The overall item pickup algorithm involves estimating the points the bot gets going from point A to item X to point B, where X is any nearby item. But the actual meat of the algorithm is estimating how many points the bot gets given a specific amount of health, armor, weapons, and ammunition.

Next time we'll dive in and talk about exactly how this is done, and what other structures are necessary to make these computations possible.

Putting It All Together

2008-02-12T06:00:00.000-08:00

I've received a number of requests from people who want to see the bots in action, but don't have Quake 3 installed. So I recorded a short demo of the bots in action, spectating from a bot's point of view. All the bots in this game are skill 4 out of 5, so they're intended to give an experienced player a run for their money. While there's a lot to be said about how each individual piece of code works, it's the entire package that matters. Weapon selection is meaningless without good aiming or item pickup, for example. Here's the video:

The biggest thing this demo shows is how realistic their aiming is. The bot's view changes as chaotically and accurately as a real person would, which is to say "generally pretty accurate, but fast enough to be a bit hard to follow". They quickly respond to changes in attention and their aim isn't instantly spot on, but they correct their aiming mistakes relatively rapidly. You can also see some good firing decisions. For example, early on Xaero fights Mynx in a narrow hallway. Xaero correctly chooses the rocket launcher and kills Mynx with two well placed blast hits.

You can also see how the bots do a pretty good job of picking up the crucial items. One of the first things they do when they respawn is grab a weapon, and they make sure to pickup health and armor when needed. It might not seem like a big deal that bots choose to pickup armor when they are near it, but it really is. The original Quake 3 bots were notorious for ignoring perfectly useful items.

This video also highlights areas where the bots could improve. Most notably, the navigation algorithm, which I could not change. Later in the demo, you can see Xaero dodge in place for maybe 10 seconds before Hunter arrives and Xaero kills her. In this situation, Xaero hears Hunter below him and wants to get to her, but he can't figure out how to jump down the ledge. So he dodges in place until she arrives. Once Xaero kills hunter, he goes on his way, picking up the megahealth and some armor.

So BrainWorks represents a lot of progress for First Person Shooter AI, but there's still a lot of room for improvement. As I said before, AI is hard! For those of you interested in more posts describing exactly how BrainWorks, uh, works, fear not! Next post we'll dive into one of the most complicated systems in the code, item pickup, and explain how the BrainWorks bots solve this difficult problem.

The Attitude of the Knife

2008-02-06T06:00:00.000-08:00

When I was 15, I read one of the three books that shaped my early attitudes, perceptions, and thought process. That book was Frank Herbert's Dune. The book is about an extremely gifted 15 year old who rises to political power because of his talent and training. But he has trouble finding people who relate to him as a person because of how special he is. As another gifted 15 year old, the entire story resonated with me, and many sections of the book left their mark on my memory. My favorite quote isn't the popular "Fear is the mind killer..." quote, however. It's about finishing things:

Arrakis teaches the attitude of the knife--chopping what's off what's incomplete and saying: "Now, it's complete because it's ended here."

The attitude of the knife is an attitude of firm boundaries. It is the decision of royal fiat, the parent's "because I said so". The attitude of the knife is the act of will that sacrifices short term opportunities for long term gain by ensuring that things never grow beyond the size of usefulness.

Once you've embraced the attitude of the knife, you have significantly more peace in life. If you stain your favorite shirt, you might be able to wash the stain out, but you'll be stressed until you know one way or the other. If you accidentally rip that shirt in half, there's nothing you can do-- that's the end of the shirt. Certainly you'd rather have the shirt intact (and unstained) than not, but that's no longer an option. The shirt has met its end and your only option is to throw it out. The attitude of the knife reduces many choices to just one, even if it's not the "best" one. It is the recognition of when a problem is not solvable, at least by you, despite being a real problem.

You might be wondering what this has to do with writing Artificial Intelligence. One of the bigger problems in software development is feature creep, the desire to add "just one more feature" or extend the usefulness of an existing feature. But things can easily grow out of control, beyond the ability of the project's developers. Often the features become so numerous that it's difficult for users to do what they want to do. All the features you don't care about get in the way. (Microsoft Office and Window's Vista, I'm looking at you.)

The attitude of the knife puts an end to feature creep. It says, "Sorry, that problem is out of my jurisdiction. I understand that's a real problem and you have to accept that it will continue to be a problem." Because Artificial Intelligence is so complicated, there's an awful lot of problems and it's very easy for a project to grow out of maintainability. You need to the attitude of the knife to write AI, or you'll never finish.

When I first started BrainWorks, I decided on two strict boundaries. First, every piece of source code has to be run on the game server alone, only running when the server requests AI processing. No changes to non-AI parts of the server were allowed. Second, no modifications to the core game engine were allowed either. The core engine that ships with Quake 3 runs the most complicated AI algorithms, such as navigation, motion prediction, and item pickup. If I found issues in the engine, I was allowed to write my own version but I could not modify the engine, even to fix bugs I found.

As fate would have it, all of the core engine algorithms for AI were dysfunctional. I did end up rewriting most of them-- weapon selection, item pickup, and motion prediction were all rewritten from scratch. But the one unsolvable problem was navigation. The core engine doesn't do a good job of navigating anything but a flat, two dimensional map. What's worse is the core engine doesn't even give the AI code enough map information for the AI to try a better job of making navigation decisions. The problem simply cannot be solved given the constraints of the system.

That's a real problem, but thankfully it wasn't my problem. There were enough other issues to tackle in writing BrainWorks. Many times it was a relief to find a bug that could immediately be ignored. Not because it wasn't a problem, but because nothing could be done about it anyway. And to be honest, most of the time the navigation decisions are actually correct. It only messes up in certain areas of certain levels. On the other hand, the core engine's item pickup and weapon selection decisions were universally wrong.

For those of you curious what two other books formed my life attitude, they are Godel, Escher, Bach: An Eternal Golden Braid and the Christian Bible. While I'm no longer a Christian, I've still taken to heart the positive humanitarian parts of the Bible. And I've not forgotten my promise to explain why I'm no longer a Christian and what part Artificial Intelligence played in that story.

Things in the Attic

2008-02-01T06:31:00.000-08:00

Most every child has spent an afternoon rummaging through the attic at their parents or grandparents. Maybe it wasn't an attic for you. Maybe it was the basement or the garage, or just through some boxes shoved in a closet. But the basic experience is the same. You probably found a lot of mundane things and a few things whose existence is a mystery. "Dad, why do you have spare band saw belts in the basement? We don't own a band saw!" And if you were like me, you found one or two things where you couldn't even figure out what it was for. "Grandpa, what's this hunk of metal for?" "Oh, we used that to crank the windmill for water on days when it wasn't windy."

Of course... It seems so obvious now...

Programming can be like that attic too. You never know what things you'll find rummaging through the source code. For BrainWorks, the "attic" is ai_lib.c. Here's a partial list of the stand alone function:

A bunch of functions that compare two values
A function that returns the index of the first bit set of an integer
A function that computer integer exponents of decimal numbers
An assortment of semi-complicated vector math operations
A function that tests ray intersections with a bounding box
Binary search functions with optional insert on search failures

And there are some more complicated structures as well, such as:

A rudimentary memory manager
A hash table implementation (also known as a map)
Timed Value Lists (a kind of priority queue)
Octrees (a tree that organizes points in space for search)
Index Subset Iterators (a highly specialized list iterator)

If you're not a programmer, you're probably thinking, "What on Earth is that stuff?" And if you are a programmer, you're probably wondering, "Why on Earth does he need that stuff?" From time to time, I'll come back to things on this list and explain how they were useful in designing AI.

But for now, I'll take a brief look at the memory manager and the hash table. The hash table is used to track dropped items and the memory manager handles the data in the table. Dropped items are things a player drops on the ground when they die, such as weapons and powerups they had. Without getting into too much detail, the AI code needs to track all dropped items so bots can consider them for pickup, and the code needs to know extremely quickly whether a dropped item is still there. That means the code needed the specialized storage structure of a hash map, since its so fast to read data from it.

One of the constraints of programming for Quake 3 is that the server absolutely cannot allocate any memory at run time, and there are generally no useful library functions. So if you need a widget_extractor() or a cog_spinner(), you have to make it yourself. While other languages have hash maps, Quake does not, so I had to write one. And writing the map required a memory allocator, or at least a good simulation of one.

All of that just so bots can pick up the quad damage someone dropped when they died! Good AI is worth that trouble.

You've Got My Attention

2008-01-28T06:00:00.000-08:00

Everyone has had moments of deep concentration, only to be interrupted by someone's voice. Turning to see who spoke is the natural response. People naturally look at whatever thing has their attention, so a First Person Shooter bot should be no different. And because "looking" (aiming) is literally half of this AI problem, bots need a detailed mechanism for deciding what occupies their attention. Just figuring out what the bot wants to look at is complex enough.

Last post I described how bots become aware of different enemy targets and select one as the target to aim at. But just because the bot has an enemy to aim at doesn't mean there isn't something even more demanding of the bot's attention. And what happens when the bot can't find an enemy to look at? They have to look at something, presumably in the direction an enemy is most likely to appear.

The selection of where to look is handled by the aim engine (ai_aim.c). The selection algorithm uses a simple priority structure to arbitrate between conflicting things. For example, if the bot needs to jump over a pit while there is an enemy nearby, the bot will always look in the direction of the jump so it doesn't mess it up. While aiming at the enemy it might score the bot a kill, doing so makes the bot extremely likely to fall down the pit to its death. The bot can always aim at the enemy after landing the jump. Similarly if the bot hears a noise that might have been made by a player, the bot will only turn to inspect that area if the bot doesn't already have a known target. Aiming at enemies has a lower priority than getting air but a higher priority than finding additional targets.

Here is the list of typical aiming reasons, from most to least important, as taken from BotAimSelect():

Jumping over pits
Aiming at enemies
Shooting destructible objects like mines
Looking forward when swimming
Face a non-hostile target that has the bot's attention
Look at the item the bot is going to pick up
Look in an area enemies are likely to be
Look at the same thing as last time

Interestingly, this prioritization is analogous to Maslow's Need Pyramid. Maslow theorized that humans rank their needs according to priorities, with each need fitting into one of 5 categories:

Physiological Needs (being free from short term danger)
Safety (being free from long term danger)
Love (being accepted)
Esteem (being appreciated)
Self Actualization (just being who you are)

People don't worry about getting love (level 3) when they need a job just to survive (level 2) or need food (level 1).

The base of Maslow's pyramid is comprised of immediate pressing needs, just like the top of the aiming priorities. Bots look forward when jumping because they will die if they don't. Bots don't look at items when there's an enemy nearby who wants to kill them. In this regard, bots are like humans. We are all programmed for survival as the greatest need.

Two Kinds of AI

2008-01-23T06:00:00.000-08:00

There are two kinds of artificial intelligence problems. In both situations, the AI has data describing its world, but the two kinds of problems require vastly different solutions. In a video game, the AI can have perfect information about, well, everything. You can find out the exact location of anything, how fast it's moving, and pinpoint sound effects. The server can even give the AI perfect information about its enemies, like their current health and weapons.

Contrast this with AI in a real world setting, like creating an AI that drives cars:

Now the AI has a whole different set of problems. Your car might have a lot of sensors, but there still can be measurement errors. The data might not be in the ideal format-- in a game, the coordinates of all nearby objects are encoded in three dimensions. But an AI to drive a car would have to extract those coordinates from two dimensional camera images. And if game AI takes too long to think, the worst that happens is a slower server. If your car's AI thinks too long while driving on the freeway, it could get into an accident!

Of course, while writing AI for simulated environments like games is much easier than real world applications, it has a different set of challenges. A bot that always knows where you are and aims perfectly isn't much fun to play against. To make a fun bot, the AI should only use data a real player would have access to.

The challenge of real world AI is computing accurate data.
The challenge of simulated world AI is determining which pieces of data to use.

So when I tackled the problem of selecting which enemy a bot wants to attack, you can imagine the kind of challenge I faced. The simple and obvious solution-- considering each player in the game-- was simply incorrect because it used data the bot absolutely should not have access to. To solve this problem, I wrote an environment scanning engine for BrainWorks (ai_scan.c). Ironically, the purpose of this module is completely different from the scanning a car's AI would do. The BrainWorks scanning engine "fuzzes up" the data so it's less accurate, or simply ignores things the bot shouldn't know about. While it would be nice for bots to perform this task the same way humans do, in this case the bot approaches the problem in the exact opposite manner.

The bot starts by checking everything in the game and testing if it can see it, by testing that no walls are in the way, the bot is facing the thing, and the thing isn't too far away. The bot records all targets it sees in its awareness list (ai_aware.c). If the bot hasn't seen a target yet, there is a short delay before the target is "active" in the list, meaning the bot has officially spotted the target. This is analogous to time it takes humans to react to sudden changes. Furthermore, the bot will remain aware of this target for a few seconds even if they hide behind a corner. Just like a bot can't know about a target behind a corner it hasn't seen, the bot must know about that target if they have seen them. That enables the bot to chase after the enemy, should they run away.

And of course, there are other things the bot scans for. If it sees incoming missiles, it notes them as well so it can dodge out of the way. Sound effects help the bot identify enemies that can be heard but not seen, and so on. Once the bot has a list of enemies, it analyzes them to decide which it should attack. (Generally this is whichever enemy the bot has the easiest time shooting.)

All that work just to figure out who the bot wants to shoot at! The bot might have thought about the problem differently from humans, but it came to the same results. Sometimes good AI means making the AI play worse instead of better.