So what to do

In my previous post I suggested that what has happened to the economy is that we have discovered that we have been victimized by a massive Ponzi scheme. The fact that we victimized ourselves doesn't make it any better. The fact is assets we thought we had don't exist. So now what do we do?

Replace those assets! Let's assume that a total of about $3 trillion has been lost—or never really existed. What if we essentially printed that money and spread it evenly among the citizens of the US.

Here's one way to do that. Three trillion divided among 300 million people is about $10,000 apiece. Let's distribute it over a period of 7 (biblical) years. If I did it right, at 5% interest the present value of $200/month for a year followed by $150/month for 3 more years followed by $100/month for 4 more years (making a total of 7 fat years) is a bit less than $10,000. (This is what I put into my XL spreadsheet (=PV(0.05/12, 12, 50) + PV(0.05/12, 48, 50) + PV(0.05/12, 84, 100)).

Since this makes spending decisions bottom-up it is the purest market based approach to stimulating the economy. Money is not aggregated and then allocated top-down; it is spent bottom-up. Spending decisions are made at the lowest possible level by individual agents.

Each person would make his or her own decision about what to do with the money. People could spend it or save it. They could vote to increase taxes to pay for public works. Or they could let the infrastructure crumble. All the decisions are made by "the crowd."

One must hope that the citizens of the country (who form "the crowd") have learned enough from this experience to make wise decisions. And certainly it would be a good idea for public minded citizens and politicians to take this opportunity to attempt to teach the crowd whatever lessons we think should be learned. But ultimately, we have to trust the citizens to make wise decisions, or the country will always be dependent on rulers rather than its citizens.

Ensuring quality vs. ensuring visibility (or success)

In the CAS group Jochen Fromm, the group founder, raised the issue of how to ensure quality in Web 2.0 systems. I replied as follows.

Your question about how to increase the quality of user-generated content is a very good one. I was one of the original doubters that Wikipedia would be at all useful. I'm very pleased with what it has become. I'm a bit less enthusiastic about it than the general buzz, though. My experience is that many of the technical articles are not useful unless you already understand the subject. I tend to use it for two things: (a) to get a general sense of an area that I'm not sure about and (b) as a user-aided search site. (I never use del.icio.us, but I often use Wikipedia as a search aid.)

In both cases, I feel reasonably confident that the people who work on a page are committed enough to the subject matter to ensure (a) that its not way off the mark and (b) that it has a reasonably good list of current references. Of course that's not guaranteed, but it's generally the case.

This seems to be self-enforcing in that anyone who cares enough about some subject area to work on a Wikipedia page will want the page to be reasonably accurate and up-to-date. This doesn't guarantee that there won't be vandalism. The Wikipedia structure seems to handle that reasonably well. It also doesn't guarantee that the person creating the page is knowledgeable enough to do a good job. But at least it comes close to guaranteeing good intentions, which is a lot. (This is probably the case about most web sites.)

Of course there are always cases in which people want pages to make something look good as in the examples of corporations and politicians spinning Wikipedia pages. But that's a somewhat different kind of quality question. Commercial ads, for example, tend to be high quality in terms of production value as well as a certain kind of information content--even if they can also be misleading in that they may leave out important information. So that's like listening to an advocate rather than an objective observer. We have to learn how to deal with that.

You mentioned mechanisms to ensure quality. Many of them do more to identify quality than to ensure it. Reputation systems, for example, point to people who have produced quality. But they don't ensure that others won't produce low quality work. That seems to be what Google and BBs that rate posters rely on. The people with better reputations become more visible. So this is a mechanism for finding quality, which may be enough. In some sense that's what market mechanisms are supposed to do. Better products become successful and more visible. Of course there are lots of ways of gaming that as we all know. But I think that's the basic idea: build mechanisms so that quality will become more visible.

As you said, this is really quite different from Wikipedia, which wants to ensure that all pages have a certain level of quality and not just that the good ones are more visible. That seems like a much harder job.

What now strikes me as interesting is the difference between mechanisms (like evolution and market mechanisms) that encourage the establishment and visibility of successful variants and mechanisms that attempt to ensure an overall level of quality. The latter seems much harder. Evolution works because elements that do well in an environment are (by definition) more successful at establishing themselves in that environment.

An implication of this is that recommendation systems (and Google) work only because the recommenders actually favor quality. If recommenders favored cranks, that's what Google would serve us. So recommendation-based systems only reflect the tastes of the recommenders. I guess that's obvious. But it's worth noting how dependent we have all become on the fact that apparently the Internet favors quality overall. One can also point to this as a Wisdom of Crowds effect.

As I said above, recommender systems are very different from systems that attempt to ensure overall quality. The latter seems to be much more difficult. It strikes me that the latter in some ways reflects the dreams of centrally controlled economies. The fact that they don't work shows how difficult it is to ensure quality overall. Economies that are more market driven can allow poor quality because better quality products and services (in general) become more successful.

Computer Scientist Fights Threat Of 'Botnets'

I keep hearing about networks of unsuspecting PCs controlled by underground and malicious forces, but I don't know anything more definitive about them. Here's the start of an article from ScienceDaily.

Computer scientist Paul Barford has watched malicious traffic on the Internet evolve from childish pranks to a billion-dollar “shadow industry” in the last decade, and his profession has largely been one step behind the bad guys. Viruses, phishing scams, worms and spyware are only the beginning, he says.

“Some of the most worrisome threats today are things called ‘botnets’ — computers that are taken over by an outside party and are beyond the user’s control,” says Barford of UW–Madison. “They can do all sorts of nasty things: steal passwords, credit card numbers and personal information, and use the infected machine to forward spam and attack other machines.

“Botnets represent a convergence of all of the other threats that have existed for some time,” he adds.

One of the most menacing aspects of botnets is that they can go largely undetected by the owner of a personal computer. That feature has allowed botnets to grow exponentially online, with millions of infected computers bought and traded on an underground market that one security company estimates has surpassed $1 billion in activity, Barford says.

Here are some other recent articles.

• Storm botnet 'services' could be sold
• Cybercrime: How online crooks put us all at risk
• Aaargh! Zombies!
• A Hacker's Holiday Shopping List

There's even a Wikipedia article on the Storm Botnet. It appears that the article is being kept up to date quite actively. That phenomenon itself is worth noting, namely that Wikipedia can serve as an instant forum for discussion and new about items of interest.

The discussion page of the Wikipedia page noted that the Storm Botnet seems to have passed it's peak. A PC World article from Oct 21 says it is only 10% of it's former size. See Storm Worm Now Just a Squall.

The market says the surge is not working

Steven Levitt of Freakonomics points to a paper by an MIT economist that concludes that the surge is not working.

Michael Greenstone, an M.I.T. professor, good friend, and one of the best young economists in the world, has just released an incredibly thorough and thoughtful analysis of the impact of the surge. …

The most interesting part of Greenstone’s paper is his analysis of the pricing of Iraqi government debt. The Iraq government has issued bonds in the past. These entitle the owner of the bond to a stream of payments over a set period of time, but only if the government does not default on the loan. If Iraq completely implodes, it is highly unlikely that these bonds will be paid off. How much someone would pay for the rights to that stream of payments depends on their estimate of the probability that Iraq will implode.

The bond data, unlike the other sources he examines, tell a clear story: the financial markets say the surge is not working. Since the surge started, the market’s estimate of the likelihood of default by the Iraqi government has increased by 40 percent.

Word of the day: crowdsourcing

Apparently this word is about a year old — and I'm amazed that I hadn't heard it before now. I came across it in Tapscott and Williams (2006) Wikinomics, which I mentioned in my list of new cooperation books. This May 27, 2006 article from Jeff Howe's Crowdsourcing.com blog documents the word's origin. It starts out by saying that 9 days earlier Google returned only 3 hits for crowdsourcing. At the time of the writing Google returned 182,000 hits! From 3 to 182,000 in a little more than a week! (This was a year ago, and it was only a year later that I first heard the word.) This follow-up article completes the history of the term.

Wikipedia's Crowdsourcing article (appropriately) captures the meaning.

Crowdsourcing is a neologism for a business model in which a company or institution takes a job traditionally performed by a designated agent (usually an employee) and outsources it to an undefined, generally large group of people in the form of an open call over the Internet. The work is compensated with little or no pay in most cases. However, in a few examples the labor is well-compensated. In almost every case crowdsourcing relies on amateurs or volunteers working in their spare time to create content, solve problems, or even do corporate R&D.

When Debora and I talked about it she raised the issue of prize money offered for accomplishment. Are the various DARPA Grand Challenges crowdsourcing? Was the reward offered for the first one-person non-stop flight over the Atlantic crowdsourcing?

Some of the uses of the term seem to include those cases as when a corporation offers money for the solution of a problem. The more general and intuitive use doesn't. Wikipedia is a crowdsourced encyclopedia produced by an entire crowd and not just by one individual or team from a crowd. Like Wikipedia, prediction markets are a crowd-based way of producing a result — in this case evaluating an issue. The entire crowd participates.

In some work that I'm doing, I claim that distillation mechanisms will become increasingly important. By this I mean mechanisms that filter out the essence or value from crowd generated raw material. Wikipedia has one distillation mechanism. In fact, its distillation review mechanisms is also crowd sourced. So Wikipedia is doubly crowdsourced: both in the generation and distillation of information. The academic peer review process is a less crowd sourced variant of this. The original input is crowdsourced. The distillation process is more formal — although also dependent on a crowd of reviewers.

Prediction markets are another distillation mechanism. The market mechanism itself is the distillation process, which is not crowdsourced. It is a way of extracting a single essence from the opinion of many people. Of course markets have been around for a very long time.

A third form of distillation mechanism is the offering of a reward as in the DARPA grand challenge or when a corporation offers a reward for solving a problem. Many people are encouraged to participate and produce a result that satisfies the stated need. Only one of these gets the prize. The decision about which participant gets the prize is made by the offerer, not by a crowd.

One of the oldest distillation mechanisms is the marketplace, both economic and more generic. Like Wikipedia most marketplaces use a two-level crowdsourcing mechanism. At the generation level, anyone has the ability to create and market a new product or service. At the distillation level, the crowd itself is the arbiter of what succeeds. The same is true for the Internet. Anyone can create a new website; the crowd decides which websites develop a following. What's different about the Internet is that the long tail effect allows sites with relatively minimal followings to prosper.

Another distillation mechanism is used in the ESP Game and PeekaBoom in which games are used to get people to contribute knowldge. These were created by Luis van Ahn.

Yet another distillation mechanism isn't really a distillation mechanism. Linus Torvalds talked about his GIT source control management system. It is a distributed system in that there is no central branch. Everyone has his own branch. The important branches are the ones that people decide are important. The SCM system doesn't tell you. It is agnostic with respect to which branch is the main branch. There is no main branch.

Distillation

I've started to read Tapscott and Williams Wikinomics. It's a somewhat breathlessly enthusiastic account of Web 2.0. Their explanation of the difference between the "old web" and the "new web" is that the old web was like an online newspaper whereas the new web is like an online coffee shop. In other words (and this certainly isn't new with Tapscott and Williams), the new web is about user participation. The primary point of their book seems to be that user participation can [be used to] produce new value.

Perhaps they make this point later in the book, but if that's the real power of the new web, the critical skill will be distillation: how can one distill wisdom and vision from a mass of contributions? Open source software has developed one way. Companies that open their resources to user input are another. Google's page rand is another form of distillation as is eBay's reputation system. Perhaps that will be an important research area.

In many ways this is like the information fusion problem: how do you put a lot of information together to get a broader sense of what's going on? One can probably make use of public participation here also. This is all about applying collective intelligence to the task of sense making.