Consistency of clustering humans

(Greg) I think this project would work well as a crowdsourcing task. Here's what I'm picturing:

• Pick a crowdsourcing task that people can do together. This could be a controlled game (like my word search puzzles) or something a bit more realistic (like workers classifying images or transcribing an audio clip together).

• Hire a bunch of workers and get them to do some kind of solo qualification task. This task could serve as a training session for the next round, but we could also get other kinds of information (personality quizzes?). Tell the workers we'll pay them extra for coming back for the next round.

• Pick some team building algorithm from the multi-agent systems literature. Algorithmically put the workers into teams.

• Hire the workers again. Put them into their teams. Get them to work on the team version of their task. (If some workers don't come back, let their orphaned teammate work alone or with a bot.)

• Compare the teams' performance to randomly-paired teams. Quantitative measurements make sense here (ie: speed/quality of work), but qualitative feedback is good too (did well-paired workers like their teammates more?)

How teams interact or work together

Wikiprojects are teams formed to improve Wikipedia. They tend to focus on specific topics, such as board games that cover a number of different pages. Members can join or leave at any time, and can be a part of many projects. In general, a lot of interesting research can be done (and has been done) with the data from these Wikiprojects.

Specifically, it might be interesting to look interactions between Wikiprojects. Interactions might be defined as a page that is under the scope of multiple Wikiprojects, or might be more related to Wikipedians that are members of multiple projects. Do these interactions lead to greater success? (Previous work has considered success on Wikipedia as a promotion to Featured Article status)

Some questions that might be interesting to answer:

• Are pages that fall into the scope of multiple Wikiprojects more successful?
• Are projects more successful if their members express diverse interests by being in multiple projects? Does it help if the projects are more or less similar?

We are broadly interested in the efficiency of centralized (chosen by a central organizer) vs. decentralized (chosen by group members) group formation when not all relevant information is known to the central organizer.

The following are some papers we looked at so far:

• Forming k coalitions and facilitating relationships in social networks
  Liat Sless, Noam Hazon, Sarit Kraus, Michael Wooldridge
  http://legacydirs.umiacs.umd.edu/%7Esarit/data/articles/aij2018liat.pdf

  This paper looks at dividing agents into (exactly) k coalitions, where the value of the coaltion is based on an underlying social network. They find that it is possible to find a social welfare-maximizing coaltion structure in polynomial time (if k is fixed and there are a small number of negative edges) but that it is intractable to find the minimal number of edges that would need to be added to stabilize a given coaltion structure.

Questions so far:

• What needs to be modelled as relevant to group performance other than existing relationships (skills, etc.)?