In other words, there’s funding for those seeking to mine Wikipedia, Twitter, Facebook, blogs, forums, fanzines, news aggregators, and other social networking sites in order to build up information on individuals, organizations, and their relationships.

From one perspective, this would appear to be a competitive virtual arms race between various U.S. governmental agencies and Europe. The main contender appears to be the U.S. National Security Agency, which is operating and developing domestic surveillance programs in conjunction with intelligence contractors.

The U.S. Central Intelligence Agency, primarily via its investment arm, In-Q-Tel, is heavily engaged in funding companies in the digital identity and security arena. This in conjunction with the Office of the Director of National Intelligence (ODNI), which is funding the maintenance of an Open Source Center to sift through Web 2.0 sites.

For the Europeans, who are playing catchup but are not far behind, we have the example of the EU-wide INDECT Consortium, the "Intelligent Information System Supporting Observation, Searching and Detection for Security of Citizens in Urban Environment." This project is a mix of universities, tech companies, and law enforcement agencies. It is funded by the European Union for €15 million (US$22 million).

Also, major European commercial entities, including Nokia Siemens Networks, Ericsson AB (Nasdaq: ERIC), and Verint Systems Inc., have joined together to likewise develop domestic surveillance programs primarily aimed at data mining of cellphone activity and social networking information.

However, it would be wrong to think of surveillance efforts as competitive between the U.S. and Europe; there is collaboration and overlap. In May of this year, for example, the Government Communications Headquarters (GCHQ), the U.K. equivalent and data sharing partner for NSA, awarded Lockheed Martin a $200 million contract for Internet data mining systems. And one of the main contractors for NSA is BAE (British Aerospace Systems).

Many more examples have come to light, but perhaps we should examine the big question: What is this all for?

Let’s take the INDECT project as an example (and most of the projects mentioned are very similar). This project’s proponents state on their Website: “The INDECT project is out to develop new tools and techniques that will help the potential end users in improving their methods for crime detection and prevention thereby offering more security to the citizens of the European Union.”

Ostensibly, this appears a noble goal; but on reviewing current working papers, an alternative, darker interpretation emerges.

There are discussions of a making a complete dump of Wikipedia to extract the identification details of every person, organization, and geo-political entity. Other participants talk of extracting more than 100 million ratings from 480,000 comments from sports forums and fanzines associated with movie rental site Netflix. Plans call for using this input for data mining to associate personal attributes, such as relationship status, birth date, credit rating, cellphone images, videos, and so on.

Conceivably, it is easy see how a Netflix customer who gives good reviews to antiwar movies and also chats in a forum about how a current government administration is garbage, could wind up with a dossier showing them as unpatriotic and a potential threat to the state.

INDECT as a group acknowledges the ethical issues of its plans on its Website: “This [project] will be done by integrating various lawful and pre-existing sources of information, which are already available to public protection agencies throughout the European Union.”

What this essentially means is that any information about you, me, or anyone else, if it’s somewhere on the Web, is fair game to be extracted and interpreted.

Think it can’t happen? This current wave to develop the ultimate surveillance system for the Web is aimed at creating “automatic dossiers” on any domestic, European, or foreign Internet user.

That includes you!