Complexity and Social Networks Blog

17 November 2008

Events/Announcements

Pentland on Honest Signals on November 24

FYI, for those of you in Boston, you should note that Sandy Pentland will be talking about his provocative new book, Honest Signals: How They Shape our World, on November 24 (details below). Note, also, that MIT Press has offered a 20% discount to readers of this blog, through the end of this year. Just go the MIT Press site, and provide the discount code: PENTLAZER.

Honest Signals

November 24

Taubman-275 (Room subject to change)

12-1:30pm

Prof. Alex (Sandy) Pentland
Toshiba Professor of Media Arts and Sciences
Co-Director, Digital Life Consortium
Faculty Sponsor, Next Billion Network and EPROM in Africa
Massachusetts Institute of Technology

How can you know when someone is bluffing? Paying attention? Genuinely interested? The answer is that subtle patterns in how we interact with other people reveal our attitudes toward them. These predictive patterns seem to be biologically based "honest signals," evolved from ancient primate signaling mechanisms, and we find that they are major factors in human decision making in situations ranging from job interviews to first dates.

By analyzing these signals using data from electronic ID badges and specially-programmed smart phones, we can create a "gods eye" view of how the people in organizations interact, and even `see' the rhythms of interaction for everyone in a city.

---

Professor Alex ("Sandy") Pentland is a pioneer in organizational engineering, mobile information systems, and computational social science. Sandy's focus is the development of human-centered technology, and the creation of ventures that take this technology into the real world.

He directs the Digital Life Consortium, a group of more than twenty multinational corporations exploring new ways to innovate, and oversees the Next Billion Network, established to support aspiring entrepreneurs in emerging markets, and the EPROM entrepreneurship program in Africa. He is among the most-cited computer scientists in the world, and in 1997 Newsweek magazine named him one of the 100 Americans likely to shape this century.

By David Lazer | 4:19 PM | Comments (0)

6 November 2008

Citizen Relationship Management

Events/Announcements

Innovation

311: The Next Wave" - Harvard online event 11/13/08

Nine Imperatives for Leadership of 311-Enabled Government
November 13, 2008: 2:30 - 4:30 pm (EST)

~Online event. Registration required, and free of charge.~

Join us for this free, interactive discussion outlining the findings and discoveries of a report about the "next wave of 311," generated by 25 government leaders and technology and service providers convened at Harvard.

The forum will be moderated by former Indianapolis Mayor Stephen Goldsmith, who is the Dan Paul Professor of Government at the Harvard Kennedy School. The panel includes:

* Michael A. Sarasti, Manager, Strategic Customer Research & Development, Government Information Center, Miami-Dade County
* Neil Evans, 311 Project Director, City of Toronto, Ontario
* Joe Morrisroe, Deputy Commissioner & Executive Director: NYC 311 and NYC.gov
* Gerard Gallant - General Manager, Public Service/311, Motorola
* Zachary Tumin - Executive Director, HKS Leadership for a Networked World Program

For more information and to register for this event, please visit our event page at:

http://www.innovations.harvard.edu/spotlight.html?id=1561&preview=0

Also, when you register for the Government Innovators Network, we encourage you to sign up for our biweekly e-newsletter on emerging government innovations, Innovators Insights. Just be sure to check "yes" at the bottom of the registration page to subscribe to the newsletter. If you've already registered, simply log in to our site, and click "my profile" in the upper right-hand corner to update your subscriptions.

Registration to online event.

By Alexander Schellong | 5:03 AM | Comments (0)

5 November 2008

How Obama won: Conversion or mobilization?

Did Obama win by mobilizing groups who have supported Democrats in the past, or converting new people to support him?

Below is a plot of the support various demographic groups provided Obama in 2008 to the support they provided Kerry in 2004. Groups that increased their share of the electoral pie by more than 10% are represented by a larger dot. The question: did Obama's victory come from supportive groups (African Americans, young voters) turning out in greater numbers, or from moving all groups to greater support of him, as compared to Kerry?

The plot suggests that while the answer is some of both, Obama's victory may be more about conversion than mobilization. In particular, note that all groups plotted increased their support of Obama (if you have suggestions of other ways of slicing the data, please suggest and I will try to update the plot). While there was an impressive mobilization of African American voters (who have a strong tendency to vote Democratic), there were also other groups less supportive of Democrats who made up a larger fraction of the electorate in 2008 than in 2004-- most notably, evangelical voters. However, interestingly, even evangelicals were more likely to support Obama than Kerry.

Other interesting nuggets:

Young voters did not make up a notably larger share of the electorate (note, though, that turnout was up about 7% overall, so it took a really impressive jump to increase share of the electorate). They did shift more than most other voters in terms of support for Obama, although not dramatically so (here note that they had supported Kerry at high levels). The big picture here, though, for the Dems is a good one-- the young voters are settling into very Democratic voting patterns (2:1 for Obama), which research suggests is predictive of a lifelong attachment.

Jewish voters were about typical in their shift toward Obama (notable because of the issues raised by the McCain campaign regarding Obama's support of Israel, and also because of Lieberman's very visible support of McCain).

The group that shifted the most, in absolute terms, toward McCain were Hispanic voters. Also a good long run sign for the Democrats.

Frequent church goers shifted a bit more toward Obama than infrequent church goers.

Alas, no data on plumbers.

Of course, all of the above needs to be taken with many grains of salt about methodology-- e.g., about the statistical significance of the differences, the potential variations in exit poll methods iin 2004 and 2008, etc etc. To really get at these issues one would really want panel data, and so on. The above is really meant at provoking some thought as to the underlying dynamics driving the outcome yesterday.

By David Lazer | 3:15 PM | Comments (1)

Fun with numbers: did Obama redraw the political map?

This is, at best, tangentially related to the mission of this blog. The thing is that I am not only a political scientist but I am also a political junkie. And you can always just not read this if you are not interested.

In any case, one interesting question is how much the map shifted from 04 to 08. That is, did we see Obama do a few points better than Kerry in all states, or did he improve more in some places than others? Below is a plot of Obama's vote share against Kerry's (where states that had significant increases in turnout are enlarged points).

As a point of comparison, see a plot of Obama's vote share against Carter's (where Carter had just a slightly lower share of the popular vote overall):

How did the map change from 2004 to 2008? There are a handful of states where Obama did not do better: West Virginia, Oklahoma, Arkansas, and Tennessee. And there are a few states that stand out a bit in terms of larger jumps: Hawaii is a big outlier, but also Montana, Utah, Nebraska, Idaho, Indiana, Nevada, New Mexico, and Vermont. In short, he did worse in some of the border states, and better in the more conservative midwest and western states. It is interesting that none of these states have especially large African American populations.

When we make the comparison to 1976, the basic realignment of Presidential politics is still quite clear (although the fact that Carter was a Southerner muddles things up a bit). That is, we see the Northeast and West Coast and many Southwest states way above the line, and the South and border states below the line. Virginia is the notable exception there, where Obama actually outperformed Carter.

Thanks go to Matt Vandenberg, who pulled together these figures.

I will have one more posting later today on the election coming up, addressing the question of whether Obama's victory reflected mobilization of particular demographic groups, or persuasion of groups across the board. And then I can return to being a productive member of society.

By David Lazer | 10:39 AM | Comments (0)

3 November 2008

manipulation of intrade

Interesting piece in CQ on the manipulation of intrade.

Two excerpts:

An internal investigation by the popular online market Intrade has revealed that an investor's purchases prompted "unusual" price swings that boosted the prediction that Sen. John McCain will become president.

Over the past several weeks, the investor has pushed hundreds of thousands of dollars into one of Intrade's predictive markets for the presidential election, the company said.

...
Pundits and politicians have used Intrade to track the fortunes of the two presidential candidates. Through the site, begun in 1999 and incorporated in Ireland, traders buy and sell "contracts" that function as stocks, allowing investors to gamble on the outcome of political, cultural, or even geological events such as the weather.

The company asserts and experts have found that the Intrade market is generally more accurate in predicting the outcome of major events than other leading indicators, including public opinion polls. But the relatively small scale of the market and its lack of outside regulation could leave the system vulnerable to unscrupulous investors, scholars of predictive markets say.

By David Lazer | 6:31 PM | Comments (0)

Current Trends

Methodology

Social Computing

Large Scale Real-Time Behavioral Feedback with Sensors

I previously blogged about how environmental sensors could transform how we think about architecture and how this data could be combined with wearable sensors. At the Media Lab's Awareness event, held last Thursday, myself and other Media Lab researchers explored this in more detail. At this event, over 150 sponsors, students, and faculty wore Sociometric Badges for the entire day so that we could show how interactions and behavior varied across different areas of the lab during different parts of the day, as well as give people insights into how different companies behaved: do people from Hitachi and Canon have similar behavior patterns? Do they see the same demos? We gave participants real time feedback on displays scattered around the lab. Here's a screenshot, but a video is coming:

This shows the activity in the different areas. Each circle represents a person, and circles grow as people stay in an area for longer. The solid portion of each circle indicates how much people are talking, and the color indicates how engaged that person is in talking (dark green implies not engaged, bright green is very engaged), which we extract in real time from the microphone on the badges. The circles also move around based on accelerometer activity, so the position information is only based on what basestation people were close to, giving us what room they were in.
Our badges interacted with Ubiquitous Sensor Portals created by the Responsive Environments group, which allowed people to browse video feeds of the building in real time. Eventually this system will enable interaction between the virtual world (i.e. Second Life) and the real world by allowing for voice communication through the portals to other portals or virtual partners in Second Life. Here's a picture of the portals:

At the end of the day we also gave participants feedback about their company's activity and companies that were similar to them. By using the badges to figure out who talked to whom, we grouped companies and people together that saw similar demos, met the same people, had similar behavioral patterns, etc. It was amazing that companies often had more in common with companies outside their industry than with those within it. We actually generated a network of interest similarity, which (not surprisingly) showed the Media Lab at the center with many other companies tapping in to a core of sponsors with many diverse interests, which were not always the companies with the most attendants. The personalized feedback was also very interesting, since we showed people who you may be interested in meeting based on the same features we used for companies. This appeared to work extremely well, since two other people in my research group appeared in my feedback even though I never interacted with them that day. While this information is not useful in and of itself, it did convince me of the system's effectiveness, which for users would help them trust other recommendations.

The point of all of this was not to show how this technology can impact a one-day event. After all, there's not much time for real-time reflection in one day. Instead we aimed to spark discussion about how continuous deployments of these systems could fundamentally change businesses and public spaces in general. Imagine continuous feedback on behavior, personally customized by you to help increase your productivity and effectiveness. Imagine spaces where the line between virtual and physical is blurred to the point where you can just as easily have a conversation with the person next to you as with the person next to you in the virtual representation of your building in Second Life. The Sensor Portals will continue to be active at the Media Lab, allowing us to continually tailor this system to be the most beneficial to users and further research.

We are also nearing deployment of our Sensible Organization tools in the laboratory of one of the world's largest pharmaceutical companies as well as in the call center of a major financial firm. We hope to measure through these interventions whether or not we can raise productivity and enhance community within organizations, as well as answer deep theoretical questions on networks and behavior. In social network theory there is the frequent claim that central individuals tend to be more productive because they have access to more diverse information, but the causality issue has not been thoroughly studied. Maybe more productive people simply tend to be more central, so we should instead try to detect behavioral and psychological characteristics to create feedback systems. By creating recommendation systems that actually make individuals more central (see my paper describing this system), enabling us to give empirical support to theoretical arguments.

This technology will fundamentally change organizations and management as a whole, and this deployment is the first step in this direction. Over the next few months through longer term experiments we'll begin to learn exactly what this means, and how management practices can change to take advantage of this data while at the same time using it to empower employees to make the right decisions. Stay tuned.

By Ben Waber | 4:52 PM | Comments (0)

polls and cellphones

Interesting piece on fivethirtyeight comparing polls that included cell phones and those that did not. The takeaway:

"Obama leads by an average of 10.0 points in the cellphone polls, versus 5.1 in the landline-only's."

For a more thorough look, go to pollster.com

By David Lazer | 9:51 AM | Comments (0)

2 November 2008

Societal Networks

Silent doors and the electoral college

Democracy takes place at the doorway. The doorways of our homes represents the literal and metaphorical threshold between our private lives and society. From the perspective of our democracy, the doorway is the place where our private interests and perspectives meet those of others. In particular, my interest today is in the many thousands of people canvassing today for Obama and McCain (and other candidates), engaging their fellow citizens in discussions of our common interests, in an attempt to persuade and to mobilize. As I discussed in an earlier entry, the weekend before the primaries someone from my neighborhood knocked on my door, trying to persuade me to vote for Barack Obama. There is no one knocking on doors today in Massachusetts. To the extent there are people from Massachusetts canvassing today, they are up in New Hampshire

There is, of course, a clear strategic imperative for this, and it is not that there are more votes to be won or lost in New Hampshire. It is that New Hampshire is a swing state, and Massachusetts (decidedly) is not. This is a clear result of the winner take all electoral system of the electoral college in the US (with the partial exceptions of Maine and Nebraska). As has been noted many times, this winner take all system does create the possibility of a mismatch between the winner of the most votes and the election (cf Bush v. Gore 2000). However, I think this misses the biggest perversion of the electoral college. Democracy is not just about some set of rules that roughly translates the popular will into policy (although it is certainly partially that). Democracy is also about exactly those conversations at the doorway. And one purpose of an election is to spur those interactions. Of course, those conversations are taking place, even in Massachusetts. But the machinery that brings the election to your door just simply is not there. And those quiet doors around most of the country, I think, represent an enormous lost opportunity for our democracy.

By David Lazer | 11:38 AM | Comments (1)

30 October 2008

Current Trends

Technology

eGovernment

On data growth and growing concerns

Are you in favour of more efficient and effective government? Of course you are. If one counted the reasons given most often for any type of government reform, these two would score the highest marks.

It is widely recognised that the characteristics of information and communication technologies (ICT) have strong impact on both. Government was thus among the first to utilise ICT. In the early days, punch-card machines were used for the census, and electronic databases replaced large amounts of data stored in non-digital form (for example, in files) throughout government once the technology was available.

Because information drawn from data is at the core of everything government does - analysis, decision-making or verifying eligibility for access to public services, to name just a few - the proliferation of databases, data mining and ICT in general is unsurprising. However, it is this increase in databases, the kind of data being gathered, the way that data is protected (or rather the opposite) and the way it is used internally and externally, that has come under increased scrutiny and been criticised by many civil libertarians.

But the criticisms are not just about civil liberties. When governments implement ICT, outcomes vary. Large-scale projects such as the FBI's Virtual Case File, the UK's C-NOMIS or Germany's FISKUS either failed completely or largely exceeded their estimated budgets, wasting billions of taxpayers' money. There are, of course, also successful projects but, by and large, the expected impact of eGovernment in moving into a brave new world of efficient, effective and citizen-focused government administration has not happened.

The power of ICT
ICT has characteristics that need to be understood before carrying out any impact assessment. These characteristics underline why digital data and databases will continue to grow in the future, and why it is necessary to find balanced governance mechanisms for ICT, for the organisations they are embedded in, and for us - the individuals using them.

ICT allows information processing, coordination and flows to be structured without the common boundaries of roles, organisational relationships and operating procedures found in government. As a consequence, the relationship between information and the physical factors of organisational size, distance, time and costs are altered.

Digital information makes geographical dispersion irrelevant, allowing for new forms of collaboration and networks. Information technologies facilitate the speed of communication and more selectively control access to, and participation in, information exchange.

Interestingly, the standardisation, routinisation and formalisation of information sharing are not only technical requirements for shared databases to be effective; they are also typical traits of bureaucracies.

Organisational memory that was once hidden in non-digital forms or an individual's memory can be stored, managed and analysed in digital form to improve knowledge or facilitate decision-making - helped by the fact that information storage, provision and search costs are virtually zero once information is digitised. Moreover, the human constraints of processing large quantities of information are reduced (for example, through the use of search engines), and software applications make it possible to combine and reconfigure data so as to provide new information.

This has been spurred by the rise of Web 2.0 applications such as social networking sites, mash-ups, tagging, and wikis, with the underlying philosophy that comes with it - i.e. mass collaboration and data sharing - further facilitating the growth of data.

The public has followed this trend on a scale that no one imagined. Younger people in particular store and share data about their activities, location, buying behaviour or personal lives like no other generation before, and periodical incidents of security breaches, identity theft and fraud have not reversed this trend.

Often, this behaviour is based on a conscious decision: millions of users joined corporate loyalty programmes (offered by, for example, airlines, hotels or shops) in return for personalised services, rebates or points that can be used in various ways. People may also just be following an intrinsic desire to share and connect. Wikipedia is one of the prominent examples of the powerful force of collaborative peer production.

Data is also gathered and stored by companies in ways which customers are unaware of, but while the public has less control over the activities of companies, there is generally greater concern when government is engaging in these types of activities.

The rise of government databases
The counter argument is that governments do not gather more data; they are just gathering and combining data in new ways (for example, databases, biometrics, face-recognition software, remotely readable chips (RFID)).

They do so for good reasons: national security, accountability, to provide better public services and to bridge organisational silos. Yet, since 9/11, more data is being sought indiscriminately rather than selectively, meaning that innocent people's data is included through law-enforcement agencies' screening processes.

Indeed, studies have shown that bigger DNA databases produce better results. This may argue in favour of creating a comprehensive DNA database containing information on all citizens and not just those convicted of crimes, as this may actually help to exclude suspects, save investigative resources and have a deterrent effect overall.

The automatic transfer of data about passengers flying from Europe to the US sheds light on another important aspect of the discussion on databases and data sharing. In a globalised world, should countries grant access to their domestic databases and how can they protect personal data beyond their national borders?

Incidents such as the day in November 2007 when the UK government managed to lose two CDs with unencrypted data of more than 25 million citizens underline four key issues relating to government databases.

First, the government has a mandate to protect the public's data. Second, data security is not only about technology. Thirdly, government needs strategies to manage digital trust. Finally, the characteristics that make ICT so valuable (for example, the ease with which it can be transferred) mean there are increasing vulnerabilities and risks: that data will be shared when it should not be, or that it will be lost, stolen or misused. At the same time, there is a risk that data will not be available when it should be.

Paradoxically, calls for new government databases and better interoperability do arise when the system of government fails. Accordingly, new databases to track and monitor individuals and institutions, or links between formerly separate databases, are built.

Moreover, many ideas for creating pro-active, multi-channel, one-stop and joined-up government simply do not work without databases. The volume of data to be collected will grow constantly in the near future as more government transactions are digitised, and cases of data being cross-referenced ('mined') will also grow as the relevant software improves.

Even if a government body decides to discard data, it faces many difficulties.

First, because data storage costs are continuously decreasing, governmental organisations prefer to keep everything, creating 'data cemeteries'. The expansion in the volume and kinds of data maintained by agencies have made it almost impossible to maintain an inventory of resources.

Second, interim systems sometimes bridge the incompatibilities between the old and the new system, thus keeping the legacy system alive and increasing its overall complexity. For example, the US' Internal Revenue Service launched a new software application to support a total quality management initiative, but never shut it down after the initiative ended. The amount of work it would take to resolve issues relating to data exchange with other systems were considered too high.

Moreover, these 'electronic mounds' accumulate massive quantities of rules that conflict with changes to other systems. For example, to control user access, user behavior or make sure different software applications can work together. The possibilities of storing and searching electronic information may also justify the development of large sets of these rules, so ICT does not always cut red tape. This is why some have proposed a combination of laws and technology to require and make it easier for data to be deleted - and thus "revive our society's capacity to forget".

Policy options
The expansion of databases puts greater burdens on the political-administrative-ethical calculus to strike the right balance between innovation and regulatory regimes. The following questions should be considered by policy-makers in the planning stages of initiatives that include setting up databases:

• What type of data should be collected and why?
• Who collects, maintains and owns the data?
• How is the data collected?
• How long should the data be stored?
• Should the data be shared and why?
• Who will be affected by making data more widely available?
• What impact will the data have on different stakeholder groups?
• Is the data aggregated?
• Is there a minimum opt-out provision for those who provide the data directly or indirectly?
• What security measures and policies are in place to protect the data?
• How to can the data be accessed and changed by those who provide it directly or indirectly?
• How can accuracy of the data be ensured?
• How can the data be reviewed and disclosed?
• Do third parties who provide or use the data have the same security standards and privacy policies?

Policy-makers should also consider educating the public better on issues such as privacy and identity self-management - a process which may need to begin as early as in elementary school. They also need to understand how trust and the perception of security in digital government is created.

In any case, there will be many alternatives for government, businesses and the public to choose from when incorporating ICT into their lives. The perception of what is right and wrong will evolve alongside the values they are measured against, and the databases and techniques they are applied to.

(a longer version will appear in the European Policy Centre's "Challenge Europe" publication)

References:
D. Lazer (Ed) (2004) DNA and the Criminal Justice System, Cambridge, MA: MIT Press.

V. Mayer-Schoenberger (2007) 'Useful Void: The Art of Forgetting in the Age of Ubiquitous Computing', RWP07-22, Faculty Research Working Paper Series, Harvard University, Cambridge, MA: John F. Kennedy School of Government.

National Research Council (2008) Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment, Washington, D.C: National Academies Press

A. Schellong (2008) Citizen Relationship Management, Brussels: Peter Lang Publishing.

D. Tappscott; A. D. Williams (2007) Wikinomics, New York: Portfolio Hardcover.

The Economist 'Data Mining, 27 September 2008.

The Economist 'Privacy in Britain, 28 January, 2008.

The Economist Big, bigger, biggest', 28 February 2008

The Economist 'Identity parade, 14 February 2008.

By Alexander Schellong | 6:24 PM | Comments (1)

29 October 2008

Events/Announcements

Internet

MyFairElection

My colleague, Archon Fung, in conjunction with ABC news, has launched a "crowd sourcing" initiative to identify problems at polling places, called MyFairElection. A full description is below. Please do forward on this information, and contribute information to the project. Here are the details:

MyFairElection is an exciting project to help make the vote fair and accessible this November 4th. To participate, simply sign up (instructions below) to rate your experience of voting this year.

Did you encounter specific problems such as long lines, confusing ballots, broken machines, or closed polling places? Or, was the process of voting quite pleasant and easy?

As thousands of citizens rate their voting experiences, MyFairElection will produce a real-time "weather map" of voting conditions across the country. This map will allow viewers to see where it is easy to vote, and where people have encountered obstacles. Journalists, advocates, officials, and citizens can use this map to address obstacles to voting in real time.

So, go to this link and sign up. You will be asked to rate your voting experience on election day. You can comment, describe particular problems, and even submit pictures.

MyFairElection.com is a project of MyFairElection and ABC News. ABC News will feature the heat map as part of their election day coverage. Journalists will use the information that you submit to help identify and report on election day ballot problems.

Here are more detailed instructions.

Register [Do this before election day, November 4!]

In order to submit your polling place observations, you must create an account on MyFairElection.com.

Step 1: Use an Internet browser (such as Explorer, FireFox, or Safari) on your computer to go to this link:

Step 2: In the upper left hand corner, you will see a box to "Sign up Now." Enter your email address, your zip code, then click "submit."

Step 3: You will receive an email message from MyFairElection.com. Keep this message! You will use the link in that message to submit your polling place ratings on election day.

You will also receive an email message from myfairelection.com on election day reminding you to vote and to rate your voting experience.

Rate a polling place on the web:

Step 1: Go to the link provided in your registration email message. It will look something like this:http://myfairelection.com/rate/7ebe4eb. You don't have to type it in; just click on the link from your email program or copy the link and paste it into your browser window.

If you've deleted the email message from MyFairElection.com or can't find it, just register again. You'll receive another message with the link.

Step 2: After you clink on the link, your browser will take you to the ratings page.

Use the ratings page to tell us about your polling experience: one star is terrible and five stars is excellent.

You can also report specific problems, offer general comments, and even upload pictures of your voting experience on this page. After you've filled in the appropriate spaces, click "Submit."

That's it! Use the map of the U.S. that will appear on myfairelection.com on November 4th to view ratings from people all over the country.

Rate a polling place by email:

If you're on the road on election day, you can also submit ratings by email, on a PDA such as a BlackBerry or iPhone. Note that we will accept email ratings only from registered users.

Step 1: Create a new email message addressed to rate@myfairelection.com

Step 2: In the body of that email message, enter
(1) your zip-code
(2) 1 to 5 asterisks ("***") for overall rating
(3) any comments you might have [one line only!]

And send the message! You will receive an email message that verifies receipt of your rating.

Thank you for participating in MyFairElection!

By David Lazer | 10:12 AM | Comments (0)