A$kAn$: An Online Marketplace for Q & A

N. Sriram, Ph. D.

As we read, we often engage in an implicit conversation with the absent author, recognizing novel
facts, delighting over fresh approaches, asking questions, and becoming aware of the gaps in our
understanding. A similar internal dialogue unfolds when we are absorbed by a compelling
presentation. Metacognition or “thinking about thinking” is a term that encapsulates the diverse
components of this mental activity.

Research indicates that metacognition is a crucial catalyst in learning. Not only do competent
individuals have good domain knowledge and skills; they are also acutely aware of their knowledge
gaps and skill deficiencies. Paradoxically, less competent individuals, suffering from metacognitive
blindness, have overly optimistic and inaccurate beliefs about their skills and knowledge1.

Educators, going back to Socrates2, have advocated questioning the learner as a superior pedagogical
strategy, especially when compared to the alternative of feeding the learner with a litany of
unmotivated facts. It is reasonable to assume that considerable metacognitive activity occurs,
particularly among motivated learners.

However, due to a variety of reasons, this metacognitive content remains implicit, personal and is soon
forgotten. How often have we heard a speaker request “Any questions?” at the end of a presentation?
And the typical response? Deafening silence! Few can summon the chutzpah required to express
imperfect intuitions in public. On occasion interesting questions are articulated but, more often than
not, are drowned out by the torrents of consciousness gushing from the mouths of vocal incompetents.

Student assessment exclusively operates from the vantage point of students answering questions posed
by teachers. Nonetheless, it can be extremely gratifying for the teacher to have the opportunity to
address challenging questions posed by students.

Sometimes novices pose novel questions of fundamental importance. Even if questions are not of
fundamental importance, they inject a new dimension to the discourse surrounding the domain concept
and can deepen the teacher’s appreciation of the subject matter. And if questions arise from
misunderstandings then the learner can be quickly disabused of the misconceptions that were blocking
learning.

Despite the apparent importance of student questioning, few systems allow students to have repeated
opportunities to ask questions and few students take advantage of these opportunities when they are
presented.

This article describes my experience with A$kAn$3, a system that rewards students for asking and
answering questions. The system was conceptualized in late July 2002 and, following the initial
implementation, refined with the participation of a handful of volunteers during August 2002.

In early September, the system was opened to general use by 230 students in an introductory
psychology course that had the usual complement of weekly lectures, group discussions, midterm
quizzes and final exams. Students used the system for about two months and participation in A$kAn$
accounted for eight per cent of the final grade.

Question asking systems in the domain of knowledge management first made their appearance on the
internet in late 1999 and early 20004 and continue to proliferate as the internet evolves and
consolidates5. These systems encouraged experts to answer questions by rewarding them with real
money or virtual ratings that increased their prestige in online communities. Novices are not rewarded
for asking questions—indeed, on some systems, they pay money to obtain an expert answer. Once the
question is answered, the expert answer is evaluated by the novice and the interaction is incorporated
into a searchable database. The concept of rewarding experts for their answers is quintessentially
Skinnerian6 --- behaviour that is reinforced by money and/or ratings is strengthened, leading to a
positive feedback loop.

While such models are suitable for tapping the expertise of individuals on the internet or the intranet of
far flung corporations, they cannot be applied wholesale to the context of tertiary learning. For one
thing, while students make generalized evaluations of instructors at the end of each semester (this
process has been critically evaluated7), they cannot consistently and accurately rate answers to
questions in a domain in which they are novices. In fact, such ratings would carry very little credibility
with their peers. A$kAn$ makes a radical departure in that both questions and answers are
rewarded. Furthermore, this valuation is solely done by the teacher. Students play the dual role of
novices and experts, with the teacher as evaluator and critical commentator.

Students registered with the system using an existing email address and occupied a slot in one of 10
virtual cities. Each city was represented as a 5 x 5 matrix with 25 cells. Each student chose a
nickname and an icon to be associated with the student questions and answers. Students could change
their appearance, nickname or location; most of them quickly settled in one of 10 cities, adopting a
personal nickname and an icon.

Students ask questions by clicking A$k in their highlighted cell. The question is placed in one of 12
chapters of the introductory psychology text. The teacher, using a special interface, values the question
and assigns a maximum dollar value for the answers to the question. The valued question is accessible
to other students who can now answer it. Each answer is invisible to other students till it has been
valued by the teacher. The author of a question or an answer can delete it at any time prior to the
valuation.
I valued questions and answers when I was online, typically during the evenings, and on average, spent
about 30 minutes each day. Sometimes I would comment on the question or answer. Very
occasionally I asked questions; these can be seen by clicking on my cell in Bangkok. The teacher can
close a question, disabling further answers to the question. I closed questions whenever the marginal
value being added by answers became insignificant. The teacher can also make a question “private” so
that students cannot see others answers to the same question until such time the question is made
“public”. This feature was used only very occasionally.

The black $ value on each cell represents the total value earned by the student and the sum of these
values represents the contribution of each city. The white $ value represents the earnable potential of
each cell—this is the sum of the max dollar values for answers to the questions posed by the student.
Students could interact with each other in private and public discussion forums. “Pals” is a custom
matrix consisting of the student’s friends.

Menu items at the top allowed navigation of questions that had not been answered (NewQ), had
answers with high value (HiQ) and by chapter (SeeQA). A keyword search by questions, answers and
nicknames completed the functionality. The system can display tens of thousands of items across
multiple pages in all views.

The rewards for questions and answers can be adjusted by the teacher. I observed that some students
were asking too many questions of the “hit-and-run” quality; this led to my valuing questions more
critically while simultaneously increasing the rewards for answers. Over a period of two months, 2098
questions and 3369 answers were constructed and valued on the system. 1629 questions received at
least one answer and 469 remained unanswered. Questions, on average, were valued at $1.67 and
answers at $2.26. The final market capitalization was $11,084. The number of questions and answers
are an order of magnitude higher than the quantum elicited by general-purpose discussion forums.

A$kAn$ is database driven and opens up the possibility of many kinds of reuse including course
redesign. A$kAn$ is being generalized to multiple courses and multiple teachers within each course
who are assigned to one or more course content domains. Team based activity, chat and other features
are also in development. The goal is to make the system engaging, effective and scalable; both from
the technical8 as well as the cognitive perspective.

A few outstanding students, who consistently produced high quality questions and answers, were given
special recognition during the last lecture session of the term. It is interesting to note that while these
students did well on traditional evaluations, there were several students who excelled on exams but the
quality of their questions and answers were not outstanding. This suggests that the kind of learning
and accomplishment exhibited on the A$kAn$ is distinctive from that measured by traditional
evaluations.

Student feedback was encouraging, detailed and useful as can be seen from a couple of the more than
fifty responses at the end of the semester. Students frequently reported being amazed by the depth of
understanding in some of their peers. The experience of the past semester indicates that the token
economy approach is well suited to awakening metacognitive awareness during learning.
 References

1. Kruger, J. & Dunning, D. (1999). Unskilled and Unaware of It: How Difficulties in Recognizing One's Own
Incompetence Lead to Inflated Self-Assessments, Journal of Personality and Social Psychology, 77, 1121-1134.
http://www.apa.org/journals/psp/psp7761121.html

2. Garrett, E. (1998). The Socratic Method, http://www.law.uchicago.edu/prospective/headnotes/socratic.html

3. http://www.comunaware.com/matrix/index.htm Use the Nickname=cita and Password=nus123 to Browse.

4. http://www.askmecorp.com/ is a Vendor of Question-Asking systems for Knowledge Management.

5. http://answers.google.com/ is Google’s Answering Service.

6. http://www.bfskinner.org/Operant.asp is an Overview of the Skinnerian Perspective on Learning.

7. http://faculty.washington.edu/agg/bytopic.htm#sri Contains Papers on the Validity of Student Ratings.

8. A$kAn$ is hosted on a J2EE application server running on Linux.