Reveal-it applied in an educational context!

As I mentioned in my previous post, we (Erik, Gonzalo and myself) attended the Chi conference. More concretely, we attended one session about "Tensions in Social Media" due to there was a very interesting paper called "Reveal-it!: The Impact of a Social Visualization Projection on Public Awareness and Discourse" and the fact that the authors of the paper were from my former university (UPF) and one of the co-authors was my assessor Andrew Vande Moere increased my interest to know more about it.

What was Reveal-it about?

Source of the image: blog of the primary author on the paper

I would strongly recommend you to read it. I ask in advance apologies to the authors whether I forget some important details about the paper, but summarizing this paper describes a set of experiments where energy expenses are visualized on an ambient display in different public spaces . The visualization is designed in order to increase the user awareness of energy consumption. The experiment also digs into how these public visualizations can trigger social discourse. Users that pass by the visualization can report their expenses and their information is added to the visualization. Such information is highlighted in order to attract the attention of the user and to trigger reflection. The information of the user can be compared with the expenses average of her/his neighborhood.

What did we do?

We wanted to test this application in our context, education. So we started to think about how we could apply a similar concept to our students. It could also be an alternative to our big table overview.

We wanted to test the application with our students who currently use StepUp! (a bit of explanation  here), Navi (a bit more here) and the activity stream (that aggregates all the activity of the course). So this evaluation could help us to understand how Reveal-it could complement our current work.

Ok, so... what do we have? We have students and we track different activity, tweets, blogs, time, badges... they can work in groups or individually. We wanted to test it with our current students and  the courses are ending... so they do not have to report new activity to the system... but still we wanted some interaction with the visualization with a second device such as highlighting the user activity...

Our first approach was to do the analogy between neighborhoods and groups. But we found two main issues:

•  We had to create one visualization per activity. For instance, for our #chikul13 students, we created four different visualizations (per blog posts, blog comments, tweets and earned badges along the course). Reveal-it aggregates gas and electricity expenses because both are paid with the same currency, but the nature of our data is completely different and to find a common way to measure it was difficult.
• Most of our #thesis12 students do not work in groups, so the analogy of neighborhood and groups did not work for this case study.

 We thought that to keep all the data on one unique visualization made more sense. So we forgot about the analogy and we went for a simpler approach. Neighborhoods are different activities such as tweeting, blogging, commenting, spending time and earning badges. And each user should be represented in every "neighborhood". In this way, the students can see in a glance how their efforts are distributed compared with the others and the mean.

What was the result?


So after Sam and I tweaked the code, this was the result for our #thesis12 and #chikul13 students:

And the students could highlight their usernames using a simple mobile web app:

What did we do?

We evaluated the tool during a poster demo session of our #thesis12 students where also our #chikul13 students were invited to participate. In fact, we did two simultaneously evaluations. Sven that focuses on how we can enhance collaborative reflection with multi-touch and big devices evaluated his tabletop app.

We projected the two visualizations, each one in different walls.

What were the first reactions from the students? A couple of #thesis12 students asked me before the start of the poster session if it was a #chikul13 project or something like that. I told them that the visualization was displaying their data, one stayed for the evaluation, the other almost ran away...

After this, I stayed a bit away of the visualization and I didn't see any student who paid a lot of attention to the visualizations. Consequently they didn't even read the text where was explained that they could interact with the visualization. So before losing the opportunity, I started to ask some students if they could evaluate the visualization.

I introduced them the tool explaining that it relies on a concept of public and ambient displays but that it enables interaction through a web mobile app. I let them use the app with my own mobile and they started highlighting their username and afterwards others usernames.

I highlight some findings got from the interviews (10 interviews):

•  Two groups of two people understood faster the visualization and they had some fun (at least they laughed) while they were comparing with each other.
• All the individuals needed a bit of help to understand completely the visualization.
• Three person understood the bars as a chronological representation of their own activity. Each bar was a week instead of a user. It can be a bias because so far StepUp! and Navi they use weeks as a granularity level to represent the data.
• But the most important perception, at least from my point of view, is that most of them expect to interact with the visualization, for instance, in order to highlight the outlier users. However, when I inquiry them about if they think that was a required feature, they replied that maybe to compare themselves with the mean was already ok.
• Also I asked them what would they rather prefer, whether the big table visualization or Reveal-it. The opinion was quite divided because the big table overview gives them more information, however everybody agreed that it was a fancier and nicer visualization that enables you to get a quick status of your activity compared with the others.

Still we have to research a bit more about it. But Reveal-it is considered more or less equally useful than StepUp! and Navi. On the other hand the #thesis12 students would feel less confident with such public visualizations used in public spaces than #chikul13 students. And they do not find that interacting with the visualization with a second device (in this case a web mobile app makes a lot of sense).