CS 330: Intro to HCI

CS 330: Intro to HCI

REFLECTIONS ON TEACHING: WINTER 2020

This quarter, I had a few experimental learning goals for students – some relating to the design and analysis of human-computer systems, and some relating to practical/technical skill development. These two sets of learning goals came out of challenges I encountered while teaching the course previously (Winter, 2019), as elaborated below. I didn’t formally evaluate any of these lessons, but in the sections below, I share some qualitative accounts of my own experiences, and some student reflections (as documented in a post-course questionnaire that I administered).

1. Design Analysis Activities

In terms of analyzing the design of computing technologies, I wanted students to spend a little more time thinking about the social implications of computer-mediated systems, and how a designer’s values and assumptions can have serious impacts on the world.

Goal 1: A recognition that technologies shape individual behavior AND larger social systems

Broad Question: How do you help students (i.e. future technology designers) examine how computing technologies can and do impact on different groups of people – both intentionally and unintentionally? Note: this class has already been doing this (and this goes for the other questions as well), but I wanted to keep this question present by adding some additional activities.

Some approaches I tried:

  1. Assigned 2 articles (Winner, 1980 and Morozov, 2013) on the politics / ideology underpinning various technologies, and discussed them in Lecture 2: Possibilities and Limits.
  2. Created a homework assignment that asked them to analyze the politics of a technology, based on a similar activity assigned by Sepehr Vakil: HW3: Sociopolitical Analysis. I allowed students to resubmit if they really missed the mark on this one, since this was outside of their comfort zones.
  3. Created an Ethics Lecture that considered some of the challenges of computer-mediated systems, and how the very same technologies that bring us so many wonderful things can be just as easily used in antisocial ways.

My Reflections

Student Comments
When I asked students to rate the value of HW3, most students were neutral towards it or saw it as “not valuable” (see students’ questionnaire at the bottom), which was pretty disappointing. Less than half found it “somewhat valuable” or “very valuable.” While of course I do think that the spirit of the assignment is extremely valuable, there is obvious more work to be done in order to make these ideas land with students. Yet, there were some promising quotes from students in the post-course questionnaires:

I enjoyed being introduced to concepts in HCI like accessibility and ethics that I haven’t been exposed to in this context.

In a future HCI class, I’d like to see more in ethics in AI, more social-justice oriented stuff (I’m not sure exactly what’s currently offered).

The sociopolitical analysis, design aspects, accessibility, etc. helped me gain a perspective which I wouldn’t have gained just through web dev.

Goal 2: A recognition of one’s own assumptions and their impact on design

Broad Question: how do you help students identify their assumptions about the world (and the role of computing within it), and translate these into questions that can be explored through user research?

As future technology designers, I also wanted students to really think about how their unexamined beliefs and assumptions might impact their designs. While this is a key tenet of design research, actually being able to pinpoint one’s own assumptions is difficult for everybody.

Some approaches I tried:

  1. Deconstructing some of students’ individual project proposals in lecture, in order to pull out the assumptions about the problem space.
  2. Asking each team, in studio, to complete a user research prep worksheet. Teams were asked to name some of their implicit assumptions and turn each one into hypotheses that they could evaluate via user / internet research.

My Reflections
I observed two sections, and I think the design worksheet worked pretty well: I did see students discussing their assumptions and making some thoughtful plans about what they wanted to learn from user research. I also read some thoughtful reflections on what they learned from contextual inquiries and interviews.

On the other hand, some students did not see the value of doing this at all, and completely skipped doing any kind of interview / observation, opting for a pretty shallow survey instead (which I found baffling). Overall, I wanted students to really spend more time in the problem space, but there wasn’t time, given the scope of the class. If we were to split this class into two, this particular aspect of the class (understanding the problem space and how computing might be brought to bear on it) could be strengthened.

Student Comments
Some relevant quotes from the student questionnaires:

Learning the design process when the only way you can solve your problem is through an app is difficult. At the beginning we were expected to explore our problem space without considering a solution, but since the final project had to be a website or app, it felt like our decisions were limited so it was difficult to avoid having solutions in mind.

Goal 3. Valuing the design process (not just the final-form artifact)

Broad Question: how do you get students to genuinely iterate on their ideas without feeling like they’re getting “behind” or “wasting time” or “impeding progress”?

I wanted students to understand that trying something, reflecting on it, abandoning it, and trying something else was the point of the class – not something to be avoided. This is already supported in studio, where user testing happens, but I wanted them to also reflect on their process a little more and get credit for pursuing different alternatives.

One approach I tried:

My Reflections
I didn’t get to ask students what they thought of this, but I definitely learned a lot from reading their reports. It made their process more visible, and helped me to see their end-to-end process (e.g. design rationale, and the various avenues they pursued, and how they viewed their final artifact). While I believe that this was valuable for the students, most likely they did not value it. As one student wrote…

I wish that we had less writing assignments and more coding. The skills I am looking to develop are web development skills, and writing long, tedious papers for every project deliverable really just impedes my progress and my team’s progress. We could have more time to code if there was less writing.

Overall, it seemed to me that an over-focus on implementation cheapened the design aspects of the course. Once students went into “implementation mode,” any question of whether the technology will actually solve the problem / meet the need went out the window, with no re-entry into the ‘real world’ in order to actually validate the technology. In other words, too much focus on implementation seemed to direct student thinking away from a reflection of the role computing / apps actually plays in the problem space.

While a more technical, narrow focus on Visual Design / UX is of course important (e.g. buttons, filter options, color schemes, swipe effects, etc.), this kind of design is but one small part of the broader UCD process. It’s also time consuming and requires a lot of esoteric HTML/CSS/JavaScript knowledge. These processes need to be separated, IMO, if we are to do justice to each.

Student Comments
Ultimately, most students seemed to view their project as an implementation exercise. Consider the following:

I wish there were more technical content covered during lecture, instead of talking about higher-level concepts.

I think the biggest issue with the course is that it needs to decide what it wants to be. It felt like the class was simultaneously trying to be a web development class and a visual design class. We would be told on our prototypes “just focus on the UX, UI, and front end, that’s all that matters” but then all of the components we’re asked to implement can all take a significant amount of back end to actually work. It felt like a significant portion of my time was spent doing things that weren’t actually going to be graded, but were still needed in order to make the prototype actually work. I think the class would be more valuable if it focused more on the theory of design aspect, like the visual design and UX lectures, and used interface/prototyping platforms that handle the back end for us while we just focus on the front end.

The course seems to try to be many things at once, accomplishing not as much as it could. Like I feel like I learned a bit on design/UI, but not much, and some stuff about HTML, CSS, and JS, but not much. I think that this course should try to find its identity.

But there were a few who liked the design principles…

I thought the course was best when talking about design principles/the theoretical side of HCI, i feel like i learned a lot from that.

2. Web Development Activities

I also wanted to provide students with more support in terms of learning HTML / CSS / JavaScript, and some approaches to using these technologies to accomplish particular goals. Specifically, I added:

Cooper Barth also designed a lecture and a homework assigment on accessibility.

Goal 1: Introduction to some modern CSS practices

Learning Objective: familiarity with selectors, a subset of common CSS properties/values, and some approaches to doing layouts (which, IMO, is one of the most difficult things to do in CSS).

What I did:

  1. Created an intro to web technologies lecture and a studio activity on CSS Grid (in week 2).
  2. Created HW4: HTML/CSS, and asked students to implement aspects of the Spotify web interface, to examine how modern websites might be designed. The intent was to help students to examine what might go into making a modern web UI.

Goal 2: JavaScript (the DOM, Events, Fetch, & REST APIs)

Learning Objectives: familiarity with event-driven programming, and interacting with the UI. Also creating data-driven templates (using data from AJAX queries).

What I did:

  1. Created several JavaScript lectures (or sections lecture) to cover JavaScript topics in weeks 3-4:
  2. Created a JavaScript lab for Studio 4.
  3. Created a HW5 that asked students to interact with Spotify’s REST API using the API Tutor (a web proxy to simplify OAuth).

Goal 3: Inclusive Design

Learning Objectives: An appreciation of accessibility considerations, and some of the ways in which web apps can be made accessible. Cooper gave a wonderful lecture on inclusive web design (hands down one of the best lectures of the quarter), and created HW6 – an activity that asked students to make the Spotify UI accessible.

Cooper’s lecture on accessibility was very interesting to me, so a course focusing on tools to make websites/apps more accessible or a project course with a focus on accessibility would be cool.

General Reflections

Mine

I found the web dev experience to be really frustrating: I went over these concepts in lecture, live coded, and recorded all of my lectures. I also assigned students Lynda.com / Safari Books videos to review that went over all of the relevant ideas. And yet when it came time to do the assignment, students complained and whined that “no one taught me any of that.” I think it’s because no one came to lecture and assumed the assignment would be easy…so students perceived that they “had to teach themselves everything.” In 130, doing these same activities was uneventful, even enjoyable to students. Honestly, this class has a culture of complaining that I didn’t understand, nor have I experienced teaching other classes.

I was also frustrated because I believed that the assignments were both valuable and doable. Perhaps if there had just been more time, and if I had held people more accountable to coming to lecture / doing the videos, things would have gone smoother. On the other hand, it’s worth asking what new tensions would have emerged…

I also MOSSed HW5 (Javascript Assignment), and 20 students (almost 15%) had an unacceptably high level of code similarity (i.e. copied from one another). After speaking with many of these pairs (which took ~two days of my time), they seemed surprised that it wasn’t OK to have nearly identical code. In many of the cases, I couldn’t tell whether that “surprise” was genuine or manipulative, but this is yet another symptom of a problematic class culture, IMO.

Students

Students’ reflections were also mixed. Here is a sample:

Pros

The hands-on coding homeworks helped me learn a lot. I also liked the accessibility unit, it gave me a wholly new perspective. Learning GitHub is also massively helpful.

I thought HW4 and HW5 really taught me HTML/CSS and Javascript comprehensively.

I think the main strength of this course was the studio sessions and the homework assignments related to coding.

Cons

Lectures were not much help in terms of coding assignments.

I wish that there were more creative projects besides the big group project. I think this is the best way to learn and sometimes the homeworks didn’t seem too valuable since we were trying to copy something, not doing our own HCI. I think if this were a technical coding course, this would be really helpful but it is a lot to squeeze into a quarter with HCI. I think that there should be a prereq for HTML/JS/CSS, since there are varying levels coming into the class.

the assignments (especially spotify one) were pretty rough especially since the amount of time we spent on webdev topics felt like not enough to prepare us. i would’ve preferred more html/js activities to help us learn and more instruction regarding those vs just a large assignment we had to do a lot of things for

I wish this course put more of a focus onto teaching HTML, CSS, and Javascript. I understand that it’s hard to dedicate class time to this, however now that the course is over I still feel as if I’ve not be adequately prepared to use these languages. I wish there was either some other CS course as a pre-requisite that taught these languages more thoroughly, or i wish maybe that HCI would spend more time teaching it (perhaps if HCI was expanded to be a sequence of 2 classes, then the first could focus more on those languages and then the second on using them for HCI?).

Also as far as studio goes I wish they explained github more. I still don’t know how to use it. Every time I need to work on my project I just re-download the whole thing into a new folder somewhere in my downloads. My group tried to show me how to use it with terminal but it wasn’t really working, so I’ve just kept on doing it this way, which obviously isn’t ideal.

I came in having no experience with html, css, or JavaScript, and I think many others were in the same boat. I don’t think that lecture gave me a great foundation in learning these and I think a bit more of a focus on those skills upfront may have been nice. Others may not agree, but maybe some small but lore frequent assignments in addition to more cumulative ones to help us build up these skills would be nice. Alternatively, perhaps splitting up the course into a web programming course where we learn those hard skills and then an HCI design course where we learn about the design process and integrate it into a large final project may work.

3. Course Logistics

Accountability

Student accountability to (a) doing the readings, (b) attending the lectures, and (c) being accountable to other members of the group remains a challenge. I made a few decisions in this regard:

  1. As an experiment, I dropped the in-class quizzes (which effectively made attendance seem “optional” to students). Attendance was pretty poor during the second half of the quarter and hovered around 40-60 students. In some regards, this made students less accountable, but it also made it feel like only the students who wanted to attend actually attended, which felt like a relief (in relation to last year, where students communicated to me in many different ways that they didn’t want to be there).
  2. For me, lectures remain a challenge, and I need to dedicate a lot more attention to making them more interactive and engaging.
  3. There were some issues with group accountability (which surfaced in a variety of anonymous Piazza posts), so I implemented a policy where students had to allocate 100 points across their teammates based on their relative contributions. These were anonymous. I think this worked pretty well. I took 10-20 points away from any student who clearly did not contribute, depending on the circumstances. 4 students fell into this category, with a few on the border.
  4. In the future, I will add this to the syllabus with a strong emphasis that free-riding will will not be tolerated.

Some notable quotes:

Also, thank you for making attendance in lecture optional and making the readings optional as well.

The group project was frustrating and did not add any value to the class. Being matched with random people is never fun, especially when they do not take the project seriously.

To be clear: I did NOT make the lectures and readings optional!

Peer Mentors as Studio Facilitators & Graders

The PMs did a truly fantastic job, and every single PM went above and beyond in terms of effort and care. That said, their strengths were more aligned with UX/UI feedback and HTML/CSS/JavaScript implementation.

Regarding facilitating group projects: I believe they were asked to do work that extended beyond what should be asked of them. First, it was awkward for many of them to grade their peers. It was also hard for them, given their current knowledge and experience, to be able to give their peers any substantive feedback on their design concepts, user research, assumptions about the world, practical significance of the idea, etc. As a result, many problematic design concepts went unchallenged, thereby reinforcing some of the worst aspects of current app development trends, IMO.

In short, PMs were great with coding and UX/UI feedback, but were less effective in helping students’ develop, evaluate, and critique their app design concepts.

Final Projects

Many students really wanted to work with each other a priori and had project ideas coming into the class. Given this, I allowed quite a bit of studio-shuffling to accommodate students’ interests. I figured that if they were invested in their ideas, they’d be more likely to work hard and have team harmony. I also opted not to have a course theme – students could pursue any idea.

Components

The components were useful in providing guidance for what constituted a final deliverable, and probably a necessity in a course of this size (~140 -> 32 team projects spanning 8 studios). However several students mentioned that checking the component boxes did not always align with their vision for their app. As such, a lot of energy went to making their app fit with the assessment instrument versus thinking more broadly. As one student expressed:

I found the components part of the Project to be a hindrance: our group had to bend around to meet component criteria instead of developing the web app out to its full creative potential.

4. Post-Course Questionnaire Responses

I gave students a questionnaire on the last day of class. Of ~140 people, 33 filled it out. Moreover, there is an overrepresentation of people who actually attend lecture who have filled out the form.

Strengths

What did you see as the strengths of the class?

Limitations

What do you wish were different about the course?

Future Course Offerings?

What would you like to see in the HCI track that is not currently offered?

Likert ratings of assignments / lecture / studio

I also asked students to rank how valuable the felt the various assignments / course meetings were to their learning.

How valuable did you find… HW3 HW4 HW5 HW6 Lectures Studio Team Project
Not Valuable 7 1 0 3 3 4 1
Neutral 11 0 1 5 14 5 2
Somewhat Valuable 11 13 9 14 12 16 11
Very Valuable 4 19 23 11 3 8 19

5. Takeaways

In an effort to translate this to a list of lessons learned / next steps, I agree with Nell and Mike that the course ought to be split.

Phase 1: Technical Aspects of Front-End Web Design

By this, I mean exploring, in greater depth, things like UX/UI/Standards, as well as HTML/CSS/JavaScript best practices.

From my perspective, this would involve porting around 1/2 of 330, including…

I would also add to this…

Phase 2: An Introductory HCI Design Course (Projects Course)